If The Music Is Good We Dance

V.R. experiences, as they’re often called, can be fictional or journalistic, narrative or open-ended. They can look like small-budget movies, big-budget video games or experimental art pieces with no obvious precedent. Some are called cinematic V.R. or V.R. storytelling, to distinguish them from pieces made for more practical ends, such as architectural modeling or P.T.S.D. therapy.

One of the main challenges for storytellers is learning to think in terms of spheres instead of rectangles. Cinematic grammar no longer applies. There is no frame in which to compose a shot. An actor who directly addresses the camera isn’t breaking the fourth wall, because the viewer is already in the middle of the action. The viewer can look anywhere, so the director often adds subtle visual or auditory cues to indicate where to look, or to signal that the viewer’s gaze can wander without missing anything important.

Tracking shots must be steady and slow, because too much camera movement can cause discomfort—viewers have reported headaches, vertigo and nausea. For the same reason, most V.R. experiences last only a few minutes; more sustained stories tend to be divided into episodes. In “passive” V.R. experiences, you simply enjoy the ride; in “interactive” ones, the environment responds to your choices. Some interactions are simple, relying on nothing more than the orientation of the viewer’s head. In an elegant game called Land’s End, you look around a serene, vividly colored landscape until you see a white orb floating at eye level. If you stare at the orb long enough, it pulls you inside it. Then you look for the next orb, which pulls you forward, and so on; without instruction, you intuitively navigate your way through a V.R. environment. Other interactive experiences use more complex hardware, including hand controllers and body-tracking sensors, to simulate such activities as painting and mini-golf.

More sophisticated V.R. headsets have been available to developers for about two years, in prototype form and are now reaching the market. The Oculus Rift, which produces precise localized audio, sells for six hundred dollars. The HTC Vive, a “room-scale” system that uses laser emitters to track a user’s movement within a fifteen-by-fifteen-foot space, costs eight hundred as does LeEco's ExploreVR. The Google Cardboard and the Samsung Gear have been on sale since last year. 

The tech is advancing astoundingly quickly. Humans are good at picking up language, including visual language, but first it has to be invented. Television broadcasting began in the nineteen-twenties, but it took decades for TV to become a medium. In the thirties, actors were filmed standing in front of microphones as they read scripts of radio plays. Movies also began as filmed theatre, but directors learned to use the camera to heighten emotions. To represent James Stewart’s fear of heights in “Vertigo,” Alfred Hitchcock introduced the “dolly zoom,” in which the cinematographer moves the camera backward while zooming in or vice versa. The dolly zoom came to signify a moment of great revelation or terror, and it was used at pivotal points in motion pictures “Raging Bull,” “Pulp Fiction,” and “Poltergeist.” It’s not clear whether zoom lenses can be used in V.R.; as far as I know, no one has tried yet. Nor do V.R. directors use montages, dissolves, or split screens—though these are all technically feasible, they might seem abrupt or confusing to the audience, who are learning to watch V.R. while its makers are learning to make it. There’s minimal editing, because we’re still figuring out how to do it. Other V.R. directors are experimenting with what might be called a leap cut, in which the viewer is transported, sometimes with an audible whoosh, from one part of the scene to another. We’re watching the semiotics come together in front of our eyes.

For most people, normal reality is virtual reality. Current reality is the matrix of all possibilities. Eventually, V.R. software could be calibrated to the user’s body: there might be ways to keep track of pulse, or galvanic skin response and deliver different experiences in reaction to that.

Outside of fiction, "virtual reality," like "angel food" or "infinity pool," is an evocative phrase that can be disappointing if taken literally. An Oculus headset provides no taste and no touch and it registers only head and hand movement. You never fully lose yourself in the simulation, if only because you're worried that it's impossible to look respectable wearing a plastic face mask. Primitive head-mounted displays were invented more than half a century ago. The Headlight, built by Philco, in 1961, used magnetic head tracking and separate video projections for each eye. There was a wave of V.R. hype in the eighties and another one in the nineties, but only now has the technology become sophisticated enough for the wave to crest.