From the Chauvet Cave paintings of 30,000 years ago, to 6K digital cinema today, we’ve always told stories, we just do it differently as media changes. There’s a new leap in storytelling happening now. Virtual Reality (VR) is going to change the way we express ourselves, communicate with each other and experience the world. That may sound like hyperbole. If anything it’s an understatement. There are innumerable ways VR will change filmmaking that we can’t see yet. Here are a few changes that have already arrived:
1. From seeing to experiencing
The leap from film to VR is even bigger than the leap from radio to film was. There was sound, then sight. With VR, an even more immersive sensation is added: presence. People who try it say, “I was at the Golden Gate Bridge,” not, “I saw the Golden Gate Bridge.” They describe it as if they’re there. And with live VR, it’s an almost indescribable sensation of being there. It’s different from VR that’s recorded. People will regard it as an experience they’ve never had before.
Viewers aren’t watching a story, they’re in a story. Instead of crafting a scene to be watched, we’re creating an environment to be explored. Viewers will want to have control, chose where they look and how they interact with the environment. For a concert, viewers could change locations, standing on stage for one song, then back in the crowd. At the end of a scene the viewer could have choices of what they want to do next, that will alter the story. Like everything else on this list, the options are just being explored.
2. Timelines are more flexible
In regular filmmaking you can use non-linear stories, flashbacks and other devices to alter timelines, but every theatrical release is 72 minutes or more, and starts when the lights go down, and ends when the credits roll. VR won’t necessarily be so. It may be something you come back to many times in order to look at different things, or try different paths. You may jump in for 72 seconds or even 72 hours.
3. New standards
In film you have to have at least 24 frames per second (fps) or the illusion of motion breaks down. For VR, the minimum is 60 fps. There are other standards as well, and they’re evolving. With photographically real, fully immersive 3D VR, the brain no longer notices the screen it’s looking at, it thinks it’s in a real environment. 3D is a requirement here. Footage shot in 2D and put into a VR display doesn’t create the same experience. Other requirements for this are: very high-resolution imagery, no seams where footage from multiple cameras has been stitched together and very accurate scale, depth and edges. As people are experimenting with camera systems, editing, broadcasting, etc., not all content being labeled as VR is up to these standards. If it’s not, you don’t feel like you’re really there in the experience. When these thresholds are met, the perception is astonishingly real.
4. Characters/audience interaction
The way characters are presented, and acting itself, may change. There were amazing silent films actors whose voices didn’t work when sound came along. There may be actors that excel in VR for a reason we don’t know yet. If we had VR at the Oscars, you could have had people saying, “I just met Julianne Moore. She was right in front of me, holding her Oscar, talking to me,” instead of, “I watched Julianne Moore win an Oscar.” Actors are going to connect with their audiences in a way they couldn’t before. This opens up a new set of questions and opportunities for how storytellers deal with characters.
5. Camera movements will change
VR is so immersive some camera movements can give viewers motion sickness. The same is true in IMAX, or even regular motion picture cinematography: you can move the camera but there are certain limitations on how you do it. Also, control of camera movements could shift from filmmaker to viewer, or be a combination of both. This will have a huge impact on viewer experience. How far filmmakers can push the envelope for camera movements in VR remains to be seen.
6. It favors creative experimenters
There’s no off-the-shelf VR camera now. The editing systems for VR are few and rudimentary. Same for VR displays. Most people working in VR are hacking their own solutions. People have been shooting with REDs, or GoPros and other small cameras, and putting footage into Unity, which is designed to build games, but there are hacks to make it work with video. We could create something interesting, but it’s difficult, the files are really big, and it’s not a scalable. Good editing tools will be critical going forward. It’s also important for directors to see what they’re filming in full value, while they’re filming it. No one would review or edit a 3D film in 2D. At NextVR, we can see live VR from the camera, as we’re filming, because of our technology. For a director filming an environment, this is hugely important. It’s tough right now, there aren’t really tools. But that means there are opportunities. Those who are good at coming up with creative tools and techniques will excel in VR.
I don’t think anybody knows where VR is going to go. There are so many things in filmmaking that you can’t say will or won’t work until you try it. Same goes for VR, and people are trying new things everyday. Light Field, for example, is a technology we’ve been working on for some time with huge implications allowing more freedom of movement in VR. In the next few years there will be developments, discoveries, challenges and triumphs we can’t fathom right now. I’m excited about the technology being developed and even more excited to see what filmmakers create with it.
A co-founder of NextVR, DJ Roller is an award-winning producer, director and cinematographer for 3D & 2D IMAX films, feature and digital cinema films, television shows, documentaries, commercials and special effects. His accomplishments include shooting the first live 3D sports broadcast (2007 NBA All-Stars game) and the first digital IMAX 3D live action film, featuring U2’s 3D concert in Argentina. DJ also developed the world’s first digital 3D underwater beam splitter camera system, capable of capturing 4K 3D imagery on land, underwater and in macro 3D. In 2000, he collaborated with Vince Pace on design and construction of the first 3D HD video cameras with James Cameron.