LA-based visual experiences creator uses Sony’s groundbreaking Spatial Reality Display to help design a thrilling AR-assisted live show.
How we helped
The Challenge
- Visual experiences creator AOIN required high-quality virtual production tools to visualize the pre-production design of an AR-enhanced live show
The Solution
- Sony Spatial Reality Display was used to present an accurate 3D visualization of the Hollywood Bowl venue
The Outcome
- All creative stakeholders are able to visualize the venue and creative elements in context
- No need for viewers to wear 3D glasses or headsets for an immersive experience
“Many of the people we work with in the industry and in the virtual production world told us we had to check out Sony’s new display. It was one of the coolest things I’ve seen.”
-Berto Mora, Chief Technology Officer, AOIN
Creating extraordinary digital experiences
All of it Now (AOIN) is a team of technologists and content creators based in Los Angeles that provides immersive visual experiences for brands, agencies, entertainers, and events. With its roots in film/TV production, motion design and IT, AOIN’s work spans design, previsualization (previs), virtual location scouting, Augmented Reality/eXtended Reality (AR/XR) consultation and camera tracking data.
A recent AOIN project was its involvement in the ambitious realization of Disney’s Encanto at the Hollywood Bowl in Los Angeles, delivered with live AR elements visible to the audience, as well as a post-production pipeline for streaming to Disney+ viewers. To help bring an authentic experience of this magnitude to life at the outdoor venue, AOIN relied on Sony’s Spatial Reality Display (SRD), which presents highly realistic, three-dimensional content without the use of special glasses or VR headsets.
A virtual production game-changer
“Many of the people we work with in the industry and in the virtual production world told us we had to check out Sony’s new display,” recalls Berto Mora, AOIN’s Chief Technology Officer, who saw the technology showcased at the NAB Show. “It was one of the coolest things I’ve seen.”
Of particular appeal to Mora and the AOIN team is the display’s seamless integration with real-time 3D creation tool Unreal Engine. Mora also cites SRD’s ability to visualize 3D content without the need for wearing special glasses or headsets, and its suitability for showcasing immersive content in a highly realistic way using a tabletop device.
An innovative approach to 3D visualization
Co-founder Danny Firpo used SRD to unlock new forms of creativity and help streamline AOIN’s pre-production workflow for the Encanto project. “We used the Spatial Reality Display to help visualize the Hollywood Bowl venue. Also, because the content is motivated by projection, many of the effects used in the AR broadcast were particle-based. Sony’s display provided visuals in both a 2D and a 3D space, enabling a better understanding of the paths that the particles were taking. This allowed us to spatialize and contextualize them.”
As well as giving a better understanding of the particles, SRD was beneficial in finding appropriate camera positions for the show. “It was really helpful to see the Bowl in a 3D space, and to use that collaboratively for identifying camera positions and cable paths,” confirms Firpo. “The 3D environment served as a vital pre-production tool, and using this display provided us with some special moments. We coordinated closely with the director of photography and individual camera operators to show them the path in Unreal within the Hollywood Bowl – this was helpful for planning the cameras’ paths with the jib. We were able to see dramatic plans as a result of seeing the camera movements in the 3D space.”
Immersive understanding of complex workflows
AOIN Executive Producer Nicole Plaza expands on the display’s powerful abilities: “Augmented reality is new for a lot of people, so having a tool that gives directors and others an understanding of what we’re doing is important. It allows them to be immersed in that 3D space, and understand our workflows more quickly.”
“We work in a lot of 3D spaces and 3D worlds through the engines, specifically Unreal Engine,” notes Mora. “It’s difficult to convey what you’re looking at through a flat, two-dimensional screen. The beautiful thing about the SRD is that you can now visualize that content in 3D and better understand it, instead of asking someone to imagine. With that comes the ability to play with depth and the perception of 3D with the human eye. And that all makes the content more valuable.”
Navigate intuitively through virtual spaces
Firpo also values the presentation abilities that the SRD offers. “The display allows us to literally be able to look around and navigate the spaces intuitively. It creates a transparent technology layer for those who aren’t used to Unreal environments. In addition, the display is very easy for us and for clients to use. You don’t need any buttons to navigate it. And at the end of the day, everyone appreciates the technology and loves seeing the content.”
AOIN has received positive responses to the SRD from its own creative clients. “Production designers often deal with the 2D space,” says Firpo. “So for them to see their designs in a 3D space before a final version is completed is helpful to provide context and spatial reference. Also seeing those AR effects interacting with production design pieces is a very special thing for a lot of these production teams, and we’ve received some really great feedback. Beyond just seeing the final output or mock-ups, we can provide insight into the whole process.”
“A lot of the artists that work on the content also get to go on-site and see it live,” adds Firpo. “I think it’s really fulfilling, and it creates a positive reinforcement that artists can see their work first in a 2D space, then the 3D space with the Spatial Reality Display during previs and pre-production – and then finally live on camera during the broadcast.”
Lifelike 3D viewing with no glasses needed
The Spatial Reality Display features several innovations, including a real-time rendering algorithm, a micro-optical lens, and a proprietary high-speed vision sensor. Mora notes that one of his favourite aspects of the display is its high-speed eye-sensing technology. “The gaze tracking of your eyes lets you know this isn’t a conventional display. The real-time perspective shift is a huge selling point that makes it much more interesting. This feature complements XR environments nicely.”
The AOIN team also appreciates the ability to view their content unencumbered by 3D glasses or special headsets. “The display looks more realistic, and presents content more accurately when you’re creating it. It provides a different effect compared to wearing glasses,” concurs Mora. “As somebody who already wears glasses every day, there’s a fatigue layer and added friction,” adds Firpo. “The weight of the glasses on your face limits the amount of time you can use them. It can also cause strain on your neck, and glasses can hurt your face and make you sweat. There’s also the potential for dirt and smudges, which can degrade the viewing experience and affect image quality.”
The reassurance of working with an industry leader
Firpo also asserts that familiarity and experience with the Sony name contributed to AOIN’s decision to choose the SRD. “The Sony brand is trusted by us, our clients, producers, and camera operators. Using a Sony display adds a level of confidence, as well as compatibility since we’re often using Sony cameras on-site. To have a Sony pipeline that uses many different products like VENICE and SRD gives us peace of mind that color reproduction is accurate.”
AOIN is excited by the display’s potential not only for preproduction, previs and AR but also for in-camera virtual effects with longer-form television and film projects, as well as virtual production – areas where the company is rapidly expanding. With the promise that digital characters and metahuman interactions hold, the possibilities are endless. Sony’s SRD will continue to help AOIN socialize these concepts to their clients in a highly realistic and impressive fashion as they continue to experiment with technology and push the boundaries of visualization.
