Improving shared TV experiences through mixed reality
Timeline
Winter - Spring 2024
My Role
Context
I spent my master's capstone working with Sony and identify and conceptualize opportunities for next-generation TV products.
Project and deliverables
Based on research about the shortcomings of current TV watching experiences, I designed and validated a novel concept for an immersive, holographic TV product with personalized settings to make group watching more accessible and more enjoyable for everyone involved. I communicated the experience through multiple prototypes in 2D, AR, and video.
Impact
My team and I presented this concept to directors, designers, researchers, and product planners at Sony. Our focus on the importance of shared experiences will inform future product development in the entertainment hardware space.
Imagine it's movie night with your family.
One person can't keep up with the dialogue because of their hearing. They want captions. You don't. Either way, someone else is sleeping in the next room and thee volume is too loud for them, but not for you. You're watching a thriller, but your cheery family pictures on the walls take you out of the experience. At the end of the movie, you turn the TV off and the black screen stares back at you. This is supposed to be family time, but nobody is happy.
During and after conducting research with TV watchers, my team created these sketches to reflect some of the problems participants described.
People really value watching TV with others, but the compromises involved take away from the experience.
Through interviews, diary studies, and contextual inquiries with young adults, we found that different needs and preferences lead to compromises that make it harder to enjoy the content. In the living space, the TV is both underwhelming (not as good as the theater) and overwhelming (dominates the room, even when it's off) at the same time.
An example of a participant's TV setup that we saw during our contextual inquiries.
I set out to create a next-generation TV product that could deliver the best group watching experience within the home.
My team and I specifically focused on making the TV work within a living space (whether it’s on or off) and ensuring that people with different needs could get full enjoyment out of the experience. I created the following design principles to guide our process:
First, we wanted to make it personalized so that everyone shares a core experience, but each person has what they need to enjoy it to the fullest.
Second, we wanted to design an experience that was special: that people would be excited about. This is in line with Sony’s mission to inspire curiosity.
Third, we wanted using this product to feel intentional: it should fit seamlessly into people's lives and homes without taking over the space, competing for attention, or preventing social connection (whether on or off).
We tested TV concepts that were more immersive, more individualized, and had more flexible footprints.
Through extensive brainstorming, sketching, and storyboarding, we brought forward our most promising concepts and gauged interest from potential users. By creating and analyzing user journey maps of the group TV watching experience, I was able to explore questions of how and when our system could support viewers, enhancing the experience without getting in the way.
We brainstormed around different problem areas and technology spaces that could contribute to a better TV experience.
In my sketches, I centered humans' emotions and reactions to our product to remind the team that we were ultimately designing for shared experience and connection between people.
During and after conducting research with TV watchers, my team created these sketches to reflect some of the problems participants described.
The product that resulted from this exploration and feedback was the 'screenless' TV.
The 'screenless' TV uses a volumetric display, based on current emerging technologies, to deliver an ideal TV experience with a minimal physical footprint.
Personalized: the system knows where the viewers are in the room, allowing each person to personalize the experience for themselves (think captions, language, and volume) using directional audio and visuals.
Special: the system can generate custom environments around the screen based on what's playing, making for a immersive experience in any space.
Intentional: most importantly, when users are done watching, the screen disappears, opening the space up for everything else in life.
I prototyped this experience in a variety of ways.
To communicate the features of our product and test it with research participants, I worked with my team to create click-through prototypes in Figma, an augmented reality prototype in Adobe Aero, and a video prototype in Adobe After Effects. Since the concept uses futuristic technology, each prototype allowed us to demonstrate different parts of the system for others.
I created a click-through prototype in Figma to demonstrate primary interactions using a remote. They are intentionally almost identical to those of current TVs, with some additional functionality.
My team worked on an augmented reality prototype (visible through a phone camera) so that users could visualize the volumetric display in their own space and execute functions with the remote.
Our final deliverable was a video prototype demonstrating product features in the context of a group watching scenario. Shown here: a user uses their phone to turn on captions, visible to themselves only.
To maximize feels of shared experience, we did not use any wearables in the design.
We considered implementing this experience with smart glasses, which could deliver immersive visuals and highly personalized experiences to multiple viewers. But even with a small form factor, they introduced barriers between viewers that could prevent social connection. In addition, our research participants were wary of introducing additional devices into their routines when they just wanted to unwind.
Given that we were designing for shared experience and social connection, we prioritized limiting barriers (such as smart glasses) that might prevent people from engaging directly with each other, as shown.
I kept familiar interaction patterns to make innovation feel like home.
This concept is a lot of new, so I made sure to maintain continuity with the current TVs people know and (sometimes) love. The remote control is still the fundamental interaction pattern, and once users get used to the volumetric display, using the 'screenless' TV is designed to be much the same as ever.
Testers felt a connection.
In testing our final concept, we were looking for foundational validation —did it make users excited? Could they see it fitting into their lives? We used our combination of prototypes to take participants through the experience. Using open-ended walkthroughs, we were able to gauge reactions and the emotions the system could create, while also noting areas of confusion or friction.
Overall, our participants did connect with our idea; we got several “wows” during our video prototype. They told us about the disagreements that personalized settings would solve. And they were probably most excited about the possibility of having the TV disappear completely from their space. We also got great feedback on the interactions for new features like manipulating the display and controlling personalized settings, specifically with accessibility and discoverability in mind.
During testing, I invited users to demonstrate the hypothetical interactions that they would use to execute new functions on the TV. This helped us evaluate our ideas and better understand users' mental models.
I learned that when designing for the future, storytelling is essential.
It was really fun to design for the future and invite research participants to think about things that aren’t yet real, but might be sooner than we expect. My biggest learning was that in a speculative design space, storytelling is everything, both for research purposes and to create alignment between stakeholders.
If I had more time, I would have liked to figure out ways to test social dynamics when using the concept rather than just the system itself, since social interactions can be really complex and individual and are hard to recreate in lower fidelity. Also, it would be really valuable to collaborate with domain experts and engineers to assess which of these features are most realistic with emerging technologies.
We presented our concept to the public during our master's capstone showcase (shown) before presenting it in more detail directly to Sony's team of directors, researchers, and product planners.