Client Overview
Client Overview
“BRANCHOUT: COMMUNITY-BASED INTERACTIVE PERFORMANCE” is an upcoming interdisciplinary, collaborative grant funded performance led by Principal Investigator Alison Dobbins (MSU, Dept. of Theatre). According to Dobbins, “The goal of this project is to shift individual engagement with the environment from a singular to a community perspective. An immersive performance will be generated that prompts the audience to explore their assumptions of individuality, communication, and sustainability.” BranchOut received funding from The Michigan Council for Arts and Cultural Affairs (MCACA) in 2021.
Project Overview
Project Overview
One of the main experiences of the Branch Out performance includes an AR experience. Given my experience in the motion capture and augmented reality space, I was asked to be apart of the AR team to design for the live performance.
My Role: AR Experience Designer
Team: Three AR researchers/designers
The Process
The Process
What are we talking about here?:
On a team of three AR designers and researchers, we began to speak with our interdisciplinary team to get a better understanding of the goals we should have for this performance. We focused on two installments for the majority of our work:
For the first installment, we were tasked to bring trees to life in a way that would show the connection between people and their environment through their ability to text trees and then see that transfer of data through the user’s phones.
For the second installment, we were tasked with illustrating the “wood wide web” through an Augmented Reality experience. Through this experience, users would point their phones at a tree and see the flow of nutrients occurring on the inside. Moving their phone to the ground, the user would see a large system of roots connecting to one another.
Early concept work of the “text a tree” element
Early concept work of the “wood wide web”
Exploring the Realm of Technology:
In January 2022, the AR Experience team met to discuss the logistics of our space. We discussed time commitment, budget, previous experience working with AR, and more. We then discussed the needs and limitations of our audience, from device accessibility, mobility, and familiarity with AR tools. From there, we defined the space in which we would conduct our work.
Seeing the Space:
Touring the space with our larger grant team, we were able to see the vast gardens in which they intended to share their works. We found three potential spaces in which to conduct our AR experiences. We approached the entire team with these choices and began to discuss possible methods to enrich the performance with Augmented Reality.
Exploring What’s Out There:
We then began to test particular AR experiences. We explored AR functionality that worked with plant life, from trees to houseplants. Looking into the logistics of these softwares, we found most offered either a single plant type or had a large paywall to have the kind of features we were looking to utilize in our work.
Connecting to the Tree:
After that, we looked to see if there was a way to get a tree to talk in augmented reality, creating an AR display that was triggered by a tree in its natural environment using Apple’s Reality Composer builder. Being a Michigan winter, we found that the weather conditions made the AR of the software almost impossible to use, as it would rarely process the tree correctly.
Identifying a Solution:
From these experiences, we decided we would need an AR solution that was triggered by a QR code. This would ensure that our user group would be familiar using QR codes, making this method one with the lowest entry level. This solution would also lead to a web-based AR viewer, so there would be no need for an app to be purchased or downloaded by users. This unfortunately limited us to running AR experiences that we insert into the environment, limiting our ability to interact with the gardens as we had originally envisioned.
Working the Web Browser:
Knowing we would focus our energy on a QR trigger, we began to explore possible options to open an AR file in a web browser. Apple’s “Quick View” feature seemed the most robust for the .usdz file types we would be using, so we began there.
3-D Modeling:
Playing with the concept of a 3-D model, we downloaded a beta app ran in Xcode called CaptureSimple. This program allows us to take 50-100 photos of an object that can be imported as a 3-D model into photogrammetry, another beta desktop application. Grabbing a desk globe made of triangular pieces of wood, we scanned this object into photogrammetry. Once imported there, we were able to import a .usdz file of our globe into Quick View with an exciting amount of detail. Although this is a robust system, the current viewer only works with Apple products with Lidar capabilities. (Try tapping the image below on your iPhone!)
Expanding Our Audience Outreach:
Seeking a way to reach audiences on androids or order devices, we chose to explore Google’s model viewer. This runs through Chrome on most devices and gives the user options to move the 3-D model without having Lidar technologies using .gltf files. (Try tapping the image below on your device!)
Getting Ready:
After this, we met with the rest of our grant team to discuss the specifics of our QR code-triggered experiences. To maintain the surprise for future participants, we have chosen not to disclose the details of the experiences. The AR team is continuing to develop the experiences until early March, when materials will be tested for the live performance in early May.
Deliverables
Deliverables
Three to five QR code-triggered AR experiences to appear in the upcoming Branch Out performance occurring in early May 2022.
Project Learnings
Project Learnings
What I Know Moving Forward:
- Exploring new spaces is a continuous learning process. The world of augmented reality is in a space where it is constantly expanding as new technologies develop. Being aware and open to new platforms as they are launched in these spaces may lead to the best solution along the way, as we found out with google’s recently released augmented reality kits.
- Creating unique experiences is a battle of trial and error. Starting this project, we had big plans for an immersive self-discovery experience that would span over the majority of the botanical gardens in which we worked. When we began to implement early iterations of our ideas, we figured out what processes would and wouldn’t work very quickly.