The New Yorker: Studio 360

Posted April 26, 2016

Share To

 

  TweetEmail

Virtual Reality has quickly become the new hot technology everyone wants to be a part of.  From the New York Times to the Tribeca Film Festival, people and businesses have been experimenting with how to use VR and the kinds of stories that lend themselves to the new technology.

Andrew Marantz from the New Yorker takes a look at how VR is devolping in its early stages, and the companies that are loking to shape the way we think of virtual reality and 360 degree video:

Janicza Bravo makes short films about loneliness. In one, Michael Cera plays an abrasive paraplegic who can’t get lucky. In another, Gaby Hoffmann plays a phone stalker for whom the description “comes on too strong” is not strong enough. Bravo’s shorts employ the visual grammar of art-house cinema: over-the-shoulder shots representing a character’s point of view, handheld tracking shots depicting urgent movement, lingering closeups to heighten intimacy or unease, carefully composed establishing shots with an actor in the center of the frame.

In March, 2015, Bravo went to Venice, on the western edge of Los Angeles, to meet with a production company called Wevr. The name is pronounced “weaver,” but it can also be thought of as a sentence, with “We” as the subject and “V.R.” as the verb. As anyone who has read a tech blog within the past five years, or a sci-fi novel within the past five decades, knows, “V.R.” stands for virtual reality—a loosely defined phrase that is now being applied to several related forms of visual media. You put your smartphone into a portable device like a Google Cardboard or a Samsung Gear—or you use a more powerful computer-based setup, such as the Oculus Rift or the HTC Vive—and the device engulfs your field of vision and tracks your head movement. The filmic world is no longer flat. Wherever you look, there’s something to see.

The producers at Wevr invited Bravo to write and direct a V.R. project. “I said no,” she told me. “It sounded like a technical thing, and I’m not into technical. But then I talked to my husband, and he said, ‘How often do people just hand you money in this business?’ So I changed my mind.” She thought about what kind of story might be told most effectively in the new medium. “The two words I kept hearing about V.R. were ‘empathy’ and ‘immersion,’ and I wasn’t sure that being immersed in one of my dark comedies would be all that useful.”

Instead, she wrote a naturalistic drama about a group of friends who encounter two police officers. Bravo, who is black, tends to write roles for white actors, but for this project she assembled a mostly black cast. In 1999, Bravo’s cousin, who lived in Brooklyn, had a brief confrontation with the N.Y.P.D. that resulted in his death. According to the police, he choked on a bag of drugs. Bravo read a short article about it in the Post. “Name, cause of death—that was it,” she said. “I wanted to bring you inside the world that was left out of that paragraph.” She called her script “Hard World for Small Things,” after a line from the 1955 film “The Night of the Hunter.”

Anthony Batt, one of Wevr’s three founders and its head of content, is a forty-eight-year-old with artfully tousled hair and a bushy, graying beard. Some of Wevr’s projects are computer-animated, some are live action, and some combine both elements. “We start by identifying people with interesting minds, and then we wrap them in a creative bear hug,” Batt said. This can entail weeks of meetings, phone calls, and test shoots designed to help directors unlearn much of what they know about two-dimensional films—or “flatties,” as V.R. triumphalists sometimes call them. Neville Spiteri, Wevr’s C.E.O. and another of its founders, said, “We’ve had traditional scripts that can’t work as V.R. unless they’re totally rewritten.”

For Bravo, the bear hug was relatively painless. “Hard World for Small Things” would be a live-action short, with two scenes filmed on location. The first scene—five minutes of unhurried, semi-improvised dialogue—would place the viewer in a car as it wound through South Central L.A., then idled outside a bodega. The second, much shorter scene would take place inside the store. Bravo would use four wide-angle lenses, pointing in all directions from a single source, positioned so that the viewer felt like one of the friends. Then, in postproduction, Wevr would “stitch” the footage together to make a single spherical image. A three-hundred-and-sixty-degree camera rig picks up everything within view, including boom mikes, external lighting, and lingering crew members. It’s possible to remove such visual detritus in postproduction, but this adds time and expense. The standard practice is to call “Action!” and then run and hide. (The camera rig itself is edited out later.) On traditional film sets, the director and the crew are present for almost every scene; on this shoot the car would hold only the camera rig and the actors, who would be wearing wireless microphones. Bravo told her cast to think of the project not as a film but as an intimate play with an invisible audience.

Luis Blackaller, a producer at Wevr, said, “We all liked the concept. We had only a few choices to make.” Like most V.R. crews, Bravo and her team would shoot with GoPros—cheap, shatterproof cameras that are marketed to extreme athletes, not filmmakers. Matthew Niederhauser, a cinematographer, noted that most V.R. experiences are viewed on phones, and said, “You can shoot with big, expensive lenses, but what’s the point?”

An engineer at Wevr built a camera rig out of aluminum and sandbags, to minimize jostling, and the crew did a test shoot with the rig in the passenger seat. “Watching it, you had to turn around the whole time to make sure you weren’t missing anything in the back of the car, which felt annoying,” Blackaller said. So they decided to film from the back right seat instead. Bravo tweaked her screenplay to remove minor cinematic vestiges—insert shots, subtle blocking details—that would be either irrelevant or impossible in V.R.

“Then we had another big conversation,” Blackaller said. “Do we film a dummy?” In some V.R. experiences, the viewer feels invisible; in others, one can look down to see one’s body represented onscreen. In a clumsily animated V.R. segment produced by another company, I experienced a nightmarish version of the latter: I flew through the air, my legs dangling below me, scrawny and immovable. My arms were those of a white man in his thirties, which happened to match my anatomy but might have been distracting, if not alarming, to most humans. And when I craned my actual neck downward I saw a sharp line where my virtual neck ended, leaving a black void where my head was supposed to be.

Bravo decided to forgo the dummy. The crew filmed for a day, spent three and a half weeks in postproduction, and then submitted the short to the Sundance Film Festival. It was accepted by New Frontier, the festival’s showcase for new media. In “Hard World for Small Things,” you’re sitting next to Sev in the back of a vintage Cadillac convertible. Sev is talking to Dell, who’s driving, about something that sounds interesting—a James Baldwin book, maybe?—but, before you can be sure, Renee, who’s in the front seat, says, “No one ever gives me any books,” and they let the matter drop. It’s a languid, sunny day. A teen-ager on the side of the road throws a football, and it arcs over the top of the car, above your head, and into a yard across the street. Dell parks outside the bodega, where locals are gathered on the sidewalk. Depending on where you’re looking, you might notice one of the women on the curb directing a side-eye glance at Renee, or you might miss it.

Crosscurrents of conversation overlap around you. Sev walks into the store. Dell gets out of the car to help an old lady cross the street. You and Renee stay in the car, and Renee takes a phone call. You can turn your head slightly to listen to her, or you can turn farther to watch Dell and the old lady, or you can keep turning until you see two plainclothes cops lurking half a block away. If you’ve seen “Hard World” before, you will fix your eye on those cops and track them as they approach Dell’s car and start trouble. As the hostility intensifies, you might feel frustrated by your incorporeality—your inability to prevent the conflict from reaching its inevitable conclusion.

Jump cut—you’re inside the store. So are the cops, and Sev, carrying a box of cereal, accidentally bumps into one of them. The officer draws his gun and shoots, and Sev crashes to the floor, face up. You watch the film again, and again, and every time Sev falls you feel numb. You were just getting to know him, and now he’s gone. You could look anywhere, but your eyes linger on his still body.

Bravo recently released a short film starring Alison Pill, and she is working on a TV show and a feature—all flatties. “Even while making the V.R. thing, I felt ambivalent about it as a medium,” she said. “But now I think I would do it again. I have some ideas about directional sound that I want to play around with.”

Anthony Batt told me, “A lot of tech people are talking a big game about V.R. right now. A lot of scholars, people way smarter than I am, are coming up with theories about it. And then a few people, including us, are just diving in and fucking doing it.” Wevr has overseen more than twenty V.R. projects, and six more are in production. “Does that mean our stuff is always perfect?” Batt said. “Fuck no! It means we start with no idea of how we’re gonna make a project work, and we make it work. Or we don’t, and the whole thing turns to jello, and we learn.”

V.R. “experiences,” as they’re often called, can be fictional or journalistic, narrative or open-ended. They can look like small-budget movies, big-budget video games, or experimental art pieces with no obvious precedent. Some are called “cinematic V.R.,” or “V.R. storytelling,” to distinguish them from pieces made for more practical ends, such as architectural modelling or P.T.S.D. therapy.

Robert Stromberg won an Academy Award in 2010 for his art direction on “Avatar,” which was full of lush computer animation displayed in IMAX 3-D. “After that, I just wanted to keep pushing,” he told me. “How much more mind-blowing can it get?” He now works primarily in V.R. “One of the main challenges for storytellers is learning to think in terms of spheres instead of rectangles,” he said.

Cinematic grammar no longer applies. There is no frame in which to compose a shot. An actor who directly addresses the camera isn’t breaking the fourth wall, because the viewer is already in the middle of the action. The viewer can look anywhere, so the director often adds subtle visual or auditory cues to indicate where to look, or to signal that the viewer’s gaze can wander without missing anything important.

Tracking shots must be steady and slow, because too much camera movement can cause discomfort—viewers have reported headaches, vertigo, and nausea. For the same reason, most V.R. experiences last only a few minutes; more sustained stories tend to be divided into episodes. With the current headsets, “virtual-reality sickness” can kick in after about twenty minutes. It seems to affect old people more strongly than young people and women more strongly than men. While researching this piece, I sometimes had trouble sleeping, which is unusual for me. I avoid looking at computers before bed, because they have been linked with disturbed sleep. I eventually realized that I had been spending much of my evening leisure time with a magnified AMOLED screen two inches from my face.

In “passive” V.R. experiences, you simply enjoy the ride; in “interactive” ones, the environment responds to your choices. Some interactions are simple, relying on nothing more than the orientation of the viewer’s head. In an elegant game called Land’s End, you look around a serene, vividly colored landscape until you see a white orb floating at eye level. If you stare at the orb long enough, it pulls you inside it. Then you look for the next orb, which pulls you forward, and so on; without instruction, you intuit how to navigate your way through a V.R. environment. Other interactive experiences use more complex hardware, including hand controllers and body-tracking sensors, to simulate such activities as painting and mini-golf.

The Google Cardboard and the Samsung Gear have been on sale since last year. More sophisticated V.R. headsets have been available to developers for about two years, in prototype form, and are now reaching the market. The Oculus Rift, which produces precise localized audio, sells for six hundred dollars. The HTC Vive, a “room-scale” system that uses laser emitters to track a user’s movement within a fifteen-by-fifteen-foot space, costs eight hundred. (High-powered computers, sold separately, are required for both.) Omer Shapira, an artist and a software engineer, told me, “The tech is advancing astoundingly quickly, but the storytellers are still catching up. Humans are good at picking up language, including visual language, but first it has to be invented.” He mentioned the Kuleshov effect, which was established in the early days of cinema by the Soviet filmmaker Lev Kuleshov. When footage of a man with a neutral expression was intercut with an image of a child in a coffin, the audience thought that the man looked sorrowful; when the same footage was intercut with a shot of a bowl of soup, the man looked hungry. “Over time, that sort of thing becomes intuitive to an audience,” Shapira said.

Television broadcasting began in the nineteen-twenties, but it took decades for TV to become a medium. In the thirties, actors were filmed standing in front of microphones as they read scripts of radio plays. In 1953, WCAU, a station in Philadelphia, launched “Action in the Afternoon,” a half-hour Western that aired live every weekday. It was an ambitious production, but it wasn’t uniquely suited to TV—it was like theatre, only with more technical glitches. In “The Box,” an oral history of television, James Hirschfeld, who worked on “Action,” said, “Sound was the biggest problem. The mikes had to be hidden in the hitching posts along the street. You had to walk over to a hitching post to do a scene.”

Movies also began as filmed theatre, but directors learned to use the camera to heighten emotions. To represent James Stewart’s fear of heights in “Vertigo,” Alfred Hitchcock introduced the “dolly zoom,” in which the cinematographer moves the camera backward while zooming in, or vice versa. The dolly zoom came to signify a moment of great revelation or terror, and it was used at pivotal points in “Raging Bull,” “Pulp Fiction,” and “Poltergeist.” It’s not clear whether zoom lenses can be used in V.R.; as far as I know, no one has tried yet. Nor do V.R. directors use montages, dissolves, or split screens—though these are all technically feasible, they might seem abrupt or confusing to the audience, which is learning to watch V.R. while its makers are learning to make it.

“There’s minimal editing, because we’re still figuring out how to do it,” James Kaelan, a director who has worked in both film and V.R., told me. “Every transition is still ‘Fade to black,’ ‘Fade up from black,’ like a Jean Renoir film.” Kaelan is exaggerating—“Hard World” and other experiences have used jump cuts, some of which feel more jarring than others. Other V.R. directors are experimenting with what might be called a leap cut, in which the viewer is transported, sometimes with an audible whoosh, from one part of the scene to another. As Julia Kaganskiy, who runs an art-and-technology incubator at the New Museum, put it, “We’re watching the semiotics come together in front of our eyes.”

Read the full article here.