Author: rileymeere

After a semester of dedicated work, seeing it all together in the Black Box was thrilling. Up until recently, we’ve only been able to work the installation in bits and pieces, as time in the studio is preciously limited. At last, the world we created was truly immersive and fully responsive, a garden of sound complemented beautifully by lighting and animations, made real by our note-covered sculptures and tools.

Getting to this point was no small feat. What started as an aimless idea based on a creative art project made by one of us over winter break eventually grew into a fully-fleshed concept, making use of a variety of advanced technologies. We felt that the Black Box Studio was the ideal place to house this garden, thanks to its impressive spatial audio, motion capture, and projection capabilities.

Plant & Tool Design

At the beginning of the semester, one team member brought in the remains of an old notebook, the pages covered in watercolor paint and folded in this interesting way to resemble flowers. With our desired focus of mindfulness and meditation in mind, we conceptualized a thriving garden, each plant uniquely incorporating our old class notes.

Our early plant designs were fleshed out via prototypes and, with those, we designed a miniature garden experience that some fellow classmates participated in for some initial UX testing. It was certainly a challenge to translate the interactions we observed from a tiny setup in the corner of a bright classroom to a large dark studio space. Nevertheless, we noticed with our small plant prototypes that our participants were mostly looking down at their feet. We decided it was important to make the actual plants much larger, in order to guide a participant’s gaze up toward the animations that would be projected behind them.

Making the prototypes helped us better understand the materials for our plant sculptures and informed our process for creating the final products. Some of the major takeaways were that watercolor does not apply well after mod-podge, wire is fairly flimsy and needs structural reinforcement as well as a sturdy base, and overall we were going to need a lot of notes in order to make our plants to the desired scale. Thanks to the donations of several friends and peers, we were able to acquire a sufficient collection of handwritten notes covering a variety of school subjects, such as psychology, math, and physics.

Throughout the process, we experimented with different techniques for applying watercolor and different heights for each plant. Because our notes were sourced from a variety of notebooks and paper types, they were not as color-cohesive as we’d hoped. However, we agreed that the cohesion of concept as well as the lighting and environment in the Black Box brought it all together, and created a beautiful and impactful visual experience.

Animation Design

The animations for this installation went through several different iterations. These animations arguably defined the overall aesthetic for the piece, so it was important for us to make them capture the overall mood and tone we were going for. Our prototype animations are evidence that our initial aesthetic concept was dark and moody, with pops of color in the foreground.

Eventually, with some valuable feedback, we decided to focus on the background as a way to set the scene and build a world made of notes, a world in which our paper plants could conceivably belong and thrive.

We tested different sizes of notes/handwriting and different color palettes. Ultimately we decided that the background should be white/paper-colored to better emphasize the concept and not clash with the lighting that we were in the process of implementing.

Our time in the studio helped us recognize when lag would occur with the motion capture system which influenced the length of the animations.

Motion Capture, Lighting & Audio

The rare times we were able to get in the Black Box were essential in helping us put all the pieces of technology together cohesively. We were able to test out music, animations, and lighting in tandem, and made adjustments to each in order for them to fit together nicely. In the studio we also determined that it would be good to have audiovisual feedback not only for interacting with the sculptures, but also with the garden tools. We created sounds for when each tool was picked up, and programmed a spotlight to shine on them only when all three were placed back in the center together.

The music was designed in Ableton and programmed in Max MSP with techniques for spacializing the sound. The Max program routed to the Qualysis motion capture system as well as to Resolume in order to trigger sound, lighting, and animations. It was a valuable learning experience to work with all of these different programs and understand how they can work together. Each program added something special and beautiful to the piece as a whole, just as each member of the team brought a unique perspective and talent that made our work complete.

3. There’s something about handwriting that gives a unique character of life. In an increasingly digital age, we find ourselves looking at computer’s letters far more often than our own. How does that change the way we read? The way we think? The way we learn? Notes on Growing offers space for humanity and slow growth, even as we are surrounded by technology.

4. Zander’s blurb here.

5. Riley: I think what’s most interesting about our work is that we created a whole immersive world which requires a lot of different layers of technology working together, and sometimes that can slow down the software and lead to random glitches with the lighting or sound or animations. So even in this really calming space you might wonder, “did I do that right? Did I miss something? Is this helping?” And I guess that’s pretty similar to the modern experience of taking notes.

6. Brie: Whats beautiful about interacting in a world built out of notes is that notes are designed just to be a tool for oneself to grow your mind and they are hardly ever shared with the world. Bringing what was created to feed someone’s mind, and being able to interact with these tools in a garden-like way where interactions grow and bloom and you are discovering and creating is a very enriching experience. It is a physical meditation on how all the little things you do interact with each other to grow your mind into the beautiful unique garden it is. Nobody in the Notes on Growing garden will experience it in the same exact way due to the interactivity and the layers, and I think thats another beautiful analogy to how everyones growth and learning is different even when rooted in the same things.

Progress

Brie:

• Finished Paper Mache on Bloom shroom head
• Started planning lighting ideas

Riley:

• Finished all solo plant assets
• Finished animations for BloomShroom

Zander:

• Implemented sound and video interactions in the Black Box
• Conducted user testing and received feedback on the sounds and interactions

Goals

Brie:

• Finish Bloom Shroom – Add notes and paint, Make it stand on its own.
• Program Lights for interactions in Blackbox
• Shrubble construction?

Riley:

• Construct Calculily
• Help Brie with constructing Shrubble
• Finish animations for Calculily

Zander:

• Iterate on and finish sounds
• Add a “set position” function in the Max patch that automatically sets regions for the plants based on the location of each marker

Brad mostly gave us positive and encouraging feedback; it was nice to hear his appreciation for our concept of giving new life to old class notes. We realized, however, with this complement, that even though Brad has seen our animations several times, the idea behind our work was not communicated efficiently. While bringing in the physical sculptures may enhance the concept, we are open to the challenge that there could be more ways to really drive our idea home without necessarily shoving it in the participants’ faces. Riley brought up the possibility of vamping up our website to provide some kind of “lore”, or “bonus features” relating to the plants’ care and environment. Hopefully we will have time to pursue that idea further.

Another thing Brad mentioned in his critique is the fact that our enlarged plant sculptures somewhat undermine the authentic gardening experience, which actually demands a hunched-over posture more often than not. So too, does taking notes. We are wrestling between either creating visual impact and a pleasurable experience, or prioritizing authenticity. Nevertheless, considerable time has been put toward creating these large-scale plants and it would be a real shame to toss them aside. One solution may be to include both the miniature and large-scale plants in the final installation. This could mean reworking both our animations and our Max patch, a fairly intense undertaking. When considering this alongside the website suggestion, it is unlikely that we could accomplish both. We believe that more user testing can help shed light on which direction to go, which we plan to do when we get back from break. To be clear, the minimum viable product will be complete and well-documented by April 9th, but some minor improvements may be “ongrowing”, if you will, before our final presentation on May 1st.

One piece of critique that we plan to address immediately is Brad’s concern about the intuitiveness of multi-person interactions. As it stands now, our logic for such interactions is a bit shake-y. The interaction animations are 4 seconds long, and user testing has shown us that when two people are both interacting with the same plant, they expect more immediate feedback. Over the break we are going to experiment with ways to improve this experience. One approach is to overlay the animations, another would be to speed them up. Perhaps a combination of both will be the most effective.

Overall we are very happy with the progress we’ve made with this project and looking forward to seeing it all come together. Brad has been an invaluable mentor and we certainly couldn’t have done it all without his guidance and expertise.

We showed Brad Gallagher the current state of our project, which currently consists of only animations and sound, in the Black Box again on Wednesday, March 15th. Brad was very pleased with the progress we’d made and appreciative of the kinds of interactions our work would encourage. As we tested these out, we realized that if we wanted our project to work the way we initially planned, we would either have to switch our animations over to Unity (a software none of us are familiar with), or otherwise create perhaps hundreds of animations to accommodate every scenario. For example, if the user first decides to make a plant grow and later decides to make it multiply, there would need to be a scene where the featured plant is already enlarged while the other plants appear. If the user then decides to grow the plant some more, there would have to be yet another scene created. In this kind of experience, where the results of each interaction are maintained, there are countless different paths the user could take. Furthermore, because we envision more than one user at a time interacting with the installation, they could attempt to perform multiple interactions at once, making the animation options even more complicated. Brad offered us an elegant solution to this dilemma: reverse the effects of the animation after they are performed. This way, the results of the interactions would not need to be remembered and compounded over time.

As far as displaying multiple interactions at once goes, the projection software Resolume enables overlaying with multiple videos, so we would only need to adjust the opacities of the videos in order to show both interaction animations. With these changes, we were able to bring the number of animations needed down from possibly hundreds to less than 20. One final suggestion that Brad made was to reduce the number of plants from 4 to 3, reason being that the best camera for documentation in the Black Box is located behind one of the projection screens. If we used all four walls, the camera would be blocked. We agreed that documentation in this case was more important than creating a full-circle world, and anyways construction of the physical plants was already behind schedule. This reduced the workload and improved our documentation prospects. Overall, it was a very productive and helpful meeting.

We returned to Andy DiLallo for further guidance on our visual assets. He was impressed with the updated animation style, which involves scenes of cascading hills that are given texture with overlayed photographs of actual class notes. Videos of the animations captured in the Black Box Studio assured him that our “world-building” was effective. We showed Andy an animated scene with a plant growing and clouds traveling across the sky above the hills and he suggested that we remove the clouds from the scene. He explained that the focus in this scene was unclear, as the clouds drew the eyes away from the actual result of the user interaction (the plant growing). Furthermore, because the images are mostly 2-dimensional, empty space above the hills would actually enhance the sense of depth in the scene. Andy also shared with us a useful technique for adjusting the hue of the animation so that it could be multicolored, rather than just varying shades of green. We plan to employ this technique in our final rendering of the animations.

On Tuesday (2/14) we met with pur technical advisor and Black Box mentor, Brad Gallagher, to verify our ability to trigger one of Zander’s magical foley sounds with a well-timed “watering plant” interaction in the Black Box Studio. The program we created was successful and we even got to test the lights to gauge the aesthetic impact of paper notes on the space. Brad reassured us that our interactions were feasible and encouraged us to lean into the feedback we’d received from Kenny and Garret about creating varying heights to encourage more dynamic movement throughout the space.

Andy DiLallo is a local artist with a lot of experience creating immersive environments and interactive art. His most recent work, “Topographies of Artificial Desire,” combines pastel mountainscapes with live-generated AI messages based on data gathered from the participants of the installation. We knew his familiarity with the sort of technology and spaces that we are hoping to make use of would be invaluable. Andy brought up the concept of “world-building” and helped us recognize that, with the kind of space we are working in, we need to take our idea of paper plants and expand the vision into a paper world. We reviewed the animation prototypes Riley had put together and decided that they needed to be less abstract. Andy advised creating a full paper garden scene as opposed to spotlighting each individual plant. This was crucial feedback and has been a huge help in moving forward with animation asset development.