One way to quit a virtual reality game is by eating a burrito.
That’s how you do it in Job Simulator 2050, from Owlchemy Labs—a game that envisions banal jobs decades from now. The “exit burrito” came up again and again during a panel conversation about media’s next frontier at Dumbo’s Made in New York Media Center in April, one of several discussions about storytelling in virtual reality that the Observer has attended over the last few months.
The virtual reality community is in a bit of a frenetic state right now. There’s this sense both that it is about to crack the public’s consciousness because its technology is finally good enough to hold people’s attention, and yet no one has made the piece that convinces the slow adopters to find some goggles and put them on. Maybe watching the Olympics with the Samsung Gear will do it?
Meredith Finkelstein said, “There’s no grammar yet,” during the event in Dumbo. She’s the co-founder of a do-it-yourself VR platform called Kokowa. The medium doesn’t have traditions and forms that consumers can read instinctively yet, such as the feature film’s close-up.
When we see a close up in a movie, we know we are meant to interpret the emotion on the actor’s face. Virtual reality makes a lot of new experiences possible but it also makes it harder to, for example, tell participants to look just here, closely, right now. The whole point of VR is that participants can look anywhere they want, any time.
In fact, it’s possible to look in so many directions that a couch is no place for watching a VR film. Get on one of those desk chairs that spin. Last night I was watching Within’s “Walking In New York,” standing in my apartment, holding a Google Cardboard to my face and spinning in blind circles. It’s a great film but it would have been better sitting in an office chair where I wouldn’t have been distracted worrying about tripping.
“With the Cardboard, it’s less immersion than other headsets,” Michael Calvert of Sketchfab said in Dumbo, “but that first step is a pretty big jump for a lot of people.”
Format
“From the time we actually started trying to figure out, collectively, how to tell stories in this medium, we’re in year two,” Chris Milk, founder of immersive film making shop Within said at Techcrunch Disrupt in May. Within has collaborated on projects in VR with many of media’s biggest brands. “Citizen Kane does not come in year two of cinema. It took decades.”
“The problem is in VR that format isn’t fixed like it is in other mediums,” Milk went on. In books, the format is text on a page. In film, it is a long series of rectangles that projectors shoot light through. In VR “the format is a sequence of spheres,” Milk said. “That format is fluid.”
But that can also create powerful experiences. In May, at the New York Game Center in Downtown Brooklyn, Sony engineer Richard Marks talked about Summer Lesson, a VR game from Bandai where an idealized young woman suddenly asks you to teach her some Japanese when she finds you sitting on the porch of this perfect seaside cabin. As you do so, she keeps moving a little closer to you, until you start to feel what it feels like when someone you’re attracted to gets very close.
Like any VR experience, you can look anywhere, but the designers have a good idea of what you will favor.
Data
In fact, where you look is one of the more uncomfortable aspects of VR, because the developers can know exactly what you’re interested in as you do. Lewis Smithingham, chief technical officer at 30Ninjas, a creative shop, said during a May Northside Innovation Meetup in a Manhattan WeWork. In Dumbo, participants explained how developers can overlay the scenes with heatmaps showing where different participants directed their attention at different moments.
It’s a point that Wired Magazine made somewhat differently in a recent story about Magic Leap, a Florida-based mixed reality hardware pioneer, now partnering with Lucasfilm on a Star Wars themed project. Wired points out that VR is a surveillance state by necessity, and no one has put much thought into creating rules about the data you generate in using it.
During Summer Lesson, did participants mostly look at the scenery or did they look at the girl? The developers know. All YouTube knows is where you pause a video or back up. That’s nothing.
In fact, deciding where to put your eyes can be a little bit distracting. In a collaboration with Vice News, Within (then VRSE) put VR cameras inside a New York City Black Lives Matter protest. Vice correspondent Alice Speri leads the crew through the scene. It’s natural to focus on her, as her voice is the main one that centers you as a viewer, but as I did it, occasionally I remembered that I was watching in VR and thought I really ought to look around, just because I could.
It doesn’t mean I necessarily looked at the right place at the right time. VR makers are either going to soon sort out how to queue viewers where to look or let go of it completely. I’d bet on the former, but anything could happen.
Immersion
That’s a big challenge of storytelling in this new format: how does a game designer or filmmaker let viewers be free and loose when it’s useful but guide them in one direction when it’s best for their attention to land where something’s happening that they really need to see in order to understand what’s coming?
The soundscape becomes even more important. It’s not something we heard talked about much in these talks, but as we previously reported after talking to team members at EEVO, VR directors neglect 3D audio at their peril (EEVO helped to make a non-linear film set on New York’s L train). When the sounds in the scene have the directional quality inside VR that they have when we hear in real life, it deepens the immersion that every VR creator wants.
Immersion brought all these conversations together, but creators talked more about not undermining immersion than they did achieving it. “It’s hard to have user experience in VR because you want them to be totally immersed in that experience,” Calvert said. That’s why ideas like the “exit burrito” grab some creators so much. In any kind of video game or interactive experience, you need a way to let users access the system menu. In a traditional video game there’s a menu at the top of the screen you can mouse up to or a button on your controller that makes it overlay on the screen when you want it.
The creators we saw expressed reluctance to use 2D gaming’s shortcuts in virtual reality because they underscore the unrealness of the scene. That’s why the idea of something like using an object in the game to activate system features (like saving and quitting), is appealing because it helps keep the world itself immersive throughout.
Another way to take users out of an experience is to overwhelm them, such as by giving them motion sickness, which happens all the time in VR experiences. “Think about the people who thought about the first idea of having a roller coaster.” Toph Brown, a producer at game shop, Ustwo, said in Dumbo. “The trick is going to be making it so exciting, but then not killing them.”
Sitting here at my desk and playing around with my Google Cardboard, I can never figure out how to quit. I always end up taking my phone out and futzing with it until I can get it to leave Cardboard mode. It’s never easy.
So the mechanics are a problem, but the focus on immersion seems a little fetishized at this point, like Kandinsky’s attempt to paint so viewers can hear music. No one can hear what he heard, but they are still good paintings. Plus, it’s not as if video game creators and moviemakers don’t want viewers to get lost in their work, too. No doubt in the early days of console gaming, designers were loathe to drop menus on top of games, but eventually they relented and did it, because they made it easier. That said, it probably won’t work the same.
I got a chance to try out a new kind of menu at the Games for Change Festival last week at the New School, which I tested out Google’s Tilt Brush with an HTC Vive. The Vive isn’t just goggles. It also has two hand controllers, like shown in the photo above.
Tilt Brush lets users draw inside VR. You can draw in every direction, though. If you want to paint a cube around yourself, you can. Then you can get under it, look up at it, step outside and walk around it.
The Tilt Brush menu works sort of like a painter’s pallette. While one of your hands is the “brush,” your other hand controls all the things your brush can be (from pens to markers to something that makes twinkling stars). With your thumb controls on the pallette hand, you spin the selector around. You make selections with the brush. It’s still computery, but it’s not like any control you’d use on your laptop.
Ben Throop, who spoke at the same talk as Marks, eschewed hand controls to make his game even easier. His soccer game, Headmaster, is played entirely with users’ head movements. Creators probably aren’t going to have to be so sensitive about making every single moment intuitive for people once the medium goes mainstream. What users can intuit will expand as their familiarity grows.
It could turn out that activating system features (such as opening the store or rewinding) with virtual objects will be the way ahead for preserving immersion in games or interactive films. High-fiving to save? Picking a flower to take a screenshot? Eating a burrito to quit? My bet is that users will end up touching buttons that open traditional system menus in the end, though.
Then again, exit burritos and their ilk might be the way forward after all, just because it’s more fun that way. At this stage, only the dumb money claims to know. “A lot of indies are exploring all kinds of things you can do in VR right now,” Marks said. “Every day I see something new that I wasn’t expecting.”
