SoulRounded
An interactive and immersive gallery experience that merges the power of interaction design with Renee Cox’s artistic vision at The Mattress Factory in June - December 2022.
Clients
Renee Cox, Jessica Gaynelle Moss, The Mattress Factory
Timeline
Jan - Jun 2022
↳
Team
6 People (Producer, Programmers, Designers, Artists)
My Role
Interaction Designer, Researcher, Co-producer
Overview
SoulRounded is an interdisciplinary ETC project team. Using modern interactive technology, we aim to organically integrate the artist’s work to create an immersive, interactive, site-specific experience that centers on race, gender, and identity while exploring the boundaries of technology and art.
What I did
Communicated the design with programmers. Conducted competitive analysis, stakeholder interviews, field studies, and usability tests. Used Maya to design and prototype location-based interactions. Designed the motion graphic production pipeline with artists.
↳
1st Room
An interaction installation powered by Azure Kinect; experience from the double sides of a semi-transparent film board.
The experience subtly narrates the history of slavery and domination.
2nd Room
An immersive experience with projection mapping - depicted as ‘Utopia’ by Renee Cox.
+
“… I want people to act … different poses … and be observed by others … fully immerse … be overwhelming in my art…
Challenges
Technology & Others
Meanwhile, we need to take location-based constraints into consideration - the space and the duration of the exhibition greatly challenged our hardware and technology.
Working with the topic of race, gender, and identity also requires a keen observation of details.
Design & Art
Considering there has been a large amount of interactive exhibitions in the market, we need to propose innovative interactions according to Renee’s art style.
Without using guidance (which can intervene artist’s aesthetic expression), we also need to design indirect control for guiding the visitors.
Design
Process
01. Discover
02. Ideate and Pitch
03. Develop*
04. Deliver and Install
*Prototype, Test and Iterations
Discover
Understand Clients
We started the project researching the Mattress Factory and artist Renee Cox for a better understanding of the museum’s branding and Cox’s art style.
The Mattress Factory is a contemporary art museum located in Pittsburgh, which has pioneered the development of alternative art forms through site-specific installations, video, and performance art.
Renee Cox’s works capture the identities and beauty within her subjects. She is using her own body, both nude and clothed to celebrate black womanhood and deconstruct stereotypes.
Stakeholder Interview
After the desk research, we interviewed our stakeholders. Since the project involved the collaboration of the artist, curator, and the museum, it is crucial for us to understand their perspectives, expectations, and needs for the final deliverable.
The main participants of the interview were Renee Cox (artist), Jessica Gaynelle Moss (curator), and Danny Bracken (who represented the museum, the Mattress Factory).
Artist // During the conversation, Renee mentioned that she wanted our team to create a virtual reality (VR) experience based on her previous pieces. She was expecting that…
“… people have wired poses … and be observed by others … fully immerse …
Museum // The team from the Mattress Factory has a slightly different perspective. They mentioned that…
“ Covid is still a concern, VR headset can be a problem … the experience would better be able to accommodate more people …
We talked with Renee more about the VR experience and we found that Renee was expecting the experience as fully immersive by virtual reality. Besides, Jessica (curator) mentioned that we need to take the budget into consideration.
Technology-wise, since the exhibition would last for 6~7 months, we needed to make sure it would be reliable for such a long time.
Field Study
Customer Visits & Observation
We visited the Mattress Factory twice for 1.observing visitors and understand the museum, and 2.get more details about the physical space.
The project would take place on the third floor of Annex Building.
Key Observation Notes
About 80% of the visitors come to the museum by group.
At peak hours, 5~8 visitors come to the third floor at the same time.
Secondary Research
After understanding the client's needs and physical space, we referred to previous research and studies on exhibitions in the digital age.
According to the study of interactive exhibits in the museum context, visitors highly rated (4.3/5.0) interacting with and exhibit.
“ Visitors in all age groups highly rated interacting with an exhibit, giving an average rate of 4.3/5.0 …
Synthesize
Project Design Goal
Based on the research, we decided to include interactive into our design goals.
According to our informal conversation with Renee, we knew that she had a strong wish to build the project unique and innovative. So we also include ‘innovative’ as a factor.
As a summary of our discover phase, we conclude our project design goals for further development. And we will use the design goals as Project Objectives to evaluate design proposals.
A multi-user experience which will be:
1. Immersive
2. Interactive
3. Tech-Reliable
4. Innovative
5. Financially Feasible
We are aiming to design …
Ideate and Pitch
Brainstorm & Proposals
Based on the location, space and research findings, we proposed 3 different interactive experience.
‘Interactive Zoetrope’
The first idea borrowed the design from a cartoon zoetrope. Cox’s work has the power of fractal generation and movements. Transforming her 2D images into connected 3D model sequences by 3D animation, modeling, and 3D printing, we can build the installation based on her work.
Proposal I
Challenges or Problems:
Hard to build a structure reliable for 6 months
3D printing is expensive for the scale
‘Hologram Zoetrope’
The idea combined zoetrope with multiple ‘3D Holograms’. Substituting the 3D printing models with pieces of the hologram, we can customize contents on tablets. The movement of 3D imageries can be easily achieved by adjusting the contents.
Proposal II
Challenges or Problems:
Hard to build large scales of holograms
Need substantial budget
‘Interactive Projection Mapping’
By animation and projection mapping, we could transfer Cox’s work into a space. Using some technology such as Kinect, we could build interactions between visitors’ behavior and Cox’s images. The technology is mature enough (low risk).
Proposal III
Challenges or Problems:
Similar design already exists
Artist’s work requires projectors to have high resolution, which is costly
Pitch and Feedbacks
We presented the proposals with demos or concept images to the artist and curator. They provided positive feedback on all the proposals, so we invited faculty members from the Entertainment Technology Center, built the matrix, and evaluated the proposals as a team.
Pitch and Evaluation
Proposal Evaluation
Based on the project design goals, we set a matrix of Project Objectives. We grouped together to predict the performance through the different aspects.
Decision Making
With the five aspects, interactive projection mapping performs better than the other two proposals in Tech-Reliable, Immersive, Interactive and requires less budget. So we decided to move on with interactive projection mapping.
Drawback
The only drawback of projection mapping is that there existed some similar products/exhibitions in the market; we need to emphasize the creativity and uniqueness of Cox’s style within the format.
We are aiming to design …
An interactive & immersive projection mapping experience that represents the artist’ unique style and artistic identity.
Develop
Based on Renee Cox’s work, we played around the structure and storytelling, and decided to use Learning Tree (Renee Cox, 2022), and Pac-Man a Savoir (Renee Cox, 2022) as the contents to create the interaction experience.
Interaction Implementation
Technology-wise, we would use Kinect (a motion sensing input device) to empower the interaction. With Kinect, users/visitors can control and interact with the media by their own body, which echos with Cox’s art style — human body parts as the elements in fractal and geometric aesthetics.
Based on the space and structure of the third floor, we designed the user’s traffic flow within the space, and rendered the concept.
Design & Prototype
Users will interact and control the tree-man and pac-man respectively; after the pac-man eating certain proportion of the human figures, the tree-man would escape from the pac-man, and then guide the audience into the next projection mapping space.
Interaction Prototype
1st Room
The input system of the first space contains two sets of Kinect. Different poses can trigger the animation and interaction between the tree and the pac-man.
The projection mapping will cover all 5 or 6 walls including the floor and ceiling. For the prototype, we animated Cox’s artwork by extracting the geometry and movements. Then we tested the projection on three walls.
Projection-Mapping Prototype
2nd Room
We extracted the movements from Renee Cox’s previous work, analyzed the different formations and dynamic imageries.
Test Findings
Since the gallery rooms were still occupied by the previous exhibitions, we had the artist and other playtesters play with the demo in another space (which has a similar size and lighting conditions to the gallery). After testing the demo within the room on a single wall, we found that:
01. The two sets of Kinect did not perform ideally with detecting and distinguishing multiple users at the same time.
02. Current version of Kinect could not accurately detect users with dark clothes or dark skin, especially in a dark environment. Kinect failed to detect users with afro-textured hair.
Test I and Iteration I
Iteration & Solutions
01. We optimized the room planing and redesigned the traffic flow within the first room. With a semi-transparent screen film built in the middle of the space, users could be recognized by the two sets of Kinect. Meanwhile, users would have a clear mind of which component to control with the interaction.
02. For better performance, we decided to upgrade Kinect II to Azure Kinect (an uncommercialized new version of Kinect). Azure Kinect performs significantly better in detecting users in dark environments and detecting users with different styles.
