The first published use of the term “virtual reality” is found in Antonin Artaud’s The Theater and its Double, in a section in which he asserts that alchemy and theater have in common that they are virtual arts. For Artaud, writing in 1933, there’s nothing technologically novel about virtual reality; it refers rather to a particular potential for metamorphoses in states of matter and being that may be accessed via alchemy and theater. As Samuel Weber points out, both exist as mirages within which transformations in “philosophical states of matter” are produced: “they do not carry their end — or their reality — within themselves.”
In its most common colloquial use today, at least according to the students in Rendering Worlds, a course I co-teach with artist/creative technologist Sarah Rothberg at The New School, “virtual reality” (VR) refers to any online digital platform — Second Life, of course, but also Instagram — in which the user participates in a virtual world, as well as to the new media technology that bears this name.
Here, I am using VR in its narrowest sense as the name of a contemporary configuration of stereopsis, motion tracking, and head-mounted display (HMD) that allows a user to interact with a world generated in the space of a computer. While it had ancestors in Hollywood, VR as we know it emerged from the project of coupling human and machine parts in a manner that would enable humans to think in new ways. VR pioneers like Ivan Sutherland worked in labs where they developed hardware and graphics rendering systems that would link the senses to the mathematics of the computer’s virtual world. (An early history of VR can be found in the work of Howard Rheingold.)
No previous medium has involved such an intricate system of connections between human perception and media machine. Any VR experience depends upon a perceptual infrastructure that draws upon multiple integrated scientific and technical procedures. In addition to the HMD’s stereo display, VR must track the user’s movements “in real life” (IRL) to map them into the VR experience so that the body can move in intelligible ways within it, and so the hardware and the application can render the world of the experience at a high enough speed to respond to the user’s movements in real time. When I say that no previous medium has employed this kind of perceptual infrastructure, one can think of it this way: High latency (a delay in loading or rendering) on a website evokes annoyance, while high latency in VR triggers nausea.
Because in VR one has the ability to create and inhabit worlds different from our everyday ones, and because the protocols of user interactions determine what bodies can do there, its ways of world-making provide experimental material for the speculative design of embodiment and sociality, as well as for the reimagining of these facts of life for users on an everyday basis. It offers a lab for research-creation of a particular kind — one in which the sensing, tracking, and modulation of perception and experience are the substance of the medium.
Below, I offer five theses to help us think the potential impact of making — and remaking — worlds in VR.
1. VR is not an empathy machine.
One of the most common claims made on behalf of VR is that it is an empathy machine, and many experiences that invoke the empathy logic are indeed very moving. Take, for example, the experience Nonny de la Pena and her colleagues created for Planned Parenthood, “Across the Line”: In the first part you accompany a woman who must cross a hostile picket line to terminate a pregnancy. You are not seeing through her eyes, but are beside her, a friend by her side as she prepares for the procedure. In the second part of the experience, which shifts from live action to computer generated images (CGI), you are the “target” of a group of protesters who hurl insults at you. The experience does compel us to feel for anyone who is forced to undergo either ordeal.
I find it difficult to imagine, however, as Planned Parenthood does, that it could transform people’s attitudes toward reproductive freedom, that it could change “consumer behavior” — that is, that red-state voters would change their political alignments after experiencing it.
This scenario, and the conception of VR as empathy machine, is problematic from a number of vantage points:
You are never seeing through someone else’s eyes in a literal way in VR. (And as Kathryn Bigelow’s underrated 1995 film Strange Days demonstrates, even if VR could accomplish this mind-meld, it would not necessarily lead us in the direction of greater connection with others.) You may be aligned with their subjective vision; you may even be inhabiting their body as an avatar, but this alignment-via-virtual camera does not equal seeing through their eyes. Everyone’s body is different, and the ways in which individuals perceive is different. In addition, perception is inextricably bound to forms of embodied experience that are absent and/or untranslatable; this includes smell, taste, touch (though “haptics” can include some of these multisensorial experiences), and individual habits of kinesthesia and proprioception. But perhaps most importantly, this definition disavows individual histories and frames of reference as they co-constitute our perception of a world.
In order to understand the experience we have in VR in terms of empathy, where it is determined by inhabiting someone’s body, we would need to adopt a very incorporative, even cannibalizing, definition of what empathy is. This involves, as VR researcher Lynda Joy Gerry discusses, an appeal to a sameness of perspective where the otherness of the other is excluded. We need rather to affirm a version of empathy that embraces alterity, that is, that includes an awareness of the other’s otherness.
Finally, this definition of VR-produced empathy seems to leave out the agency of the apparatus and the constraints (as well as potentials) of its perceptual infrastructure. These are not overdetermining features, but rather underdetermined ones — that is, the instrumentality of VR creates conditions of possibility open to multiple elaborations, but not any elaboration whatsoever. This is not to say that this instrumentality cannot change, but only that the pressure of VR’s infrastructure, whatever that may become, cannot disappear. Spoiler alert: It won’t make us more human.
2. VR’s affinity for the heightened moment makes it a) singularly bad at capturing context and b) an example of a post-truth logic of political forces.
Just as importantly, you can show someone a coral reef eroded by climate change, or the experience of someone crossing a hostile picket line to terminate a pregnancy, but you can’t provide enough context to make our empathetic VR response deep enough — or, perhaps most important, lasting enough — to bring about meaningful changes in “consumer behavior.”
Jeremy Bailenson, Director of Stanford’s Interactive Virtual Human Lab, does groundbreaking VR work devoted to social justice issues from homelessness to climate change. He very clearly lays out VR’s affinity for the heightened moment (his term), as well as its shortcomings in relation to complex fields of knowledge and context. He is nonetheless committed to a version of empathy-machine logic that posits issue-based VR projects as effective perspective-changers that can shift behavior in meaningful ways.
But what happens when we spool this logic out further? It tips us into what Jaron Lanier calls the “creepy” side of VR, and we are transported to a new kind of Manchurian Candidate-type scenario. If one VR experience can change our “consumer behavior” in relation to homelessness and climate change, what will happen when VR users enter experiences in which they’re aligned with racist, sexist, xenophobic perspectives? If these can be made to trigger the same type of empathy response that his experience “Becoming Homeless” (which asks us to sign a petition as we leave, stating that we would be willing to pay more taxes in order to help the homeless) does, what are the consequences? What would lead us to the assumption that this approach to consumer behavior will produce changes we desire — rather than leaving us open to manipulations of all kinds, including of course those motivated by desires for corporate profit? The connotations of “consumer behavior” as the measure of meaningful impact highlight the confluence of old-school behaviorism and and neoliberal capitalism.
VR’s affordances actually lend it to the promotion of post-truth politics, for which “objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.” VR used to cultivate “empathy” for contemporary political challenges frames those challenges as affective experiences rather than facts.
3. The most important feature of VR’s medium specificity is to be found in its world-making practices and potentials.
On the other hand, with a more complex understanding of the interactions between user subjectivity and the capacities of the medium, contemporary VR’s post-truth provenance might engender politically positive effects, particularly effects that enable us to relate in new ways to otherness and relationality — that is, to producing new social fields, even a new sociality.
What distinguishes VR from previous media apparatuses is its combination of stereoscopy, motion tracking, and a surround-image. It puts you “in” a world and affords you (at least some) freedom to explore it. The ways in which the world of the experience functions — and this includes its physical rules and protocols for interaction — are the content.
Because VR experiences entail building a world (even where these worlds are created with photographic media) and determining what the users can do within it (pick up objects, fly, teleport, etc.), each experience creates a meta-discourse on what constitutes a world and what it means for our minds and bodies to inhabit it. This opens up another way for us to think about VR: VR builds theories of worlds through our experience of them. It enables us to use aspects of our perceptual systems in ways that other media do not to investigate problematics and scenarios. [1]
Thus we can look at how VR crystallizes common cultural problematics that are generally more amorphous and diffuse. For example, in a social VR experience in which people interact with one another through avatars, personal space is a design issue. [2] One would not typically want avatars colliding, or passing through one another, but in a multiplayer game or social VR one might want them to be able to shake hands, or hug, or do a fist bump. If these contact gestures are possible, then so is unwanted touch. Any function that makes a virtual handshake possible may also enable virtual groping. If a participant activates a personal bubble or shield, the groping is disabled, but so is the handshake. Of course, in real life, conditions are somewhat different. Our physical interactions are not determined by an algorithm (or not entirely, in any event), nor can we choose to ignore or change the constraints of Newtonian physics on our physical bodies.
4. VR reveals and extends “homuncular flexibility.”
In that example, we are looking at interactions between relatively standard avatar bodies interacting in a relatively familiar social space. But it gets much weirder. We can learn to control avatars with bodies very different from our own. “Homuncular flexibility” is the terminology that VR researchers, like Jaron Lanier and Jeremy Bailenson, use to describe the neuroplasticity that allows us to control a non-human avatar, or an avatar with a third arm. [3]
The concept derives from Wilder Penfield’s concept of cortical homunculi (1937, 1950): His motor homunculus is a map of the parts of the brain that control motor activity, while his sensory homunculus maps the parts of the brain that are involved in sensory processing.
In the 1980s, Jaron Lanier and his colleagues at VPL were working on new VR technologies. Lanier coined the term “homuncular flexibility” to describe the process of users’ adaptations to unaccustomed avatar body forms:
It turned out that people could quickly learn to inhabit strange and different bodies and still interact with the virtual world. I became curious how weird the body could get before the mind would become disoriented. I played around with elongated limb segments, and strange limb placement. The most curious experiment involved a virtual lobster (which was lovingly modeled by Ann Lasko.) A lobster has a trio of little midriff arms on each side of its body. If physical human bodies sprouted corresponding limbs, we would have measured them with an appropriate body suit and that would have been that.
Since humans don’t have these arms, Lanier and his team needed to develop a protocol to control them:
The answer was to extract a little influence from each of many parts of the physical body and merge these data streams into a single control signal for a given joint in the extra lobster limbs. A touch of human elbow twist, a dash of human knee flex; a dozen such movements might be mixed to control the middle joint of little left limb #3. The result was that the principle elbows and knees could still control their virtual counterparts roughly as before, while still contributing to the control of additional limbs. [4]
What distinguishes VR from other audiovisual media is the very particular sense of being-here it can invoke. When you recall a VR experience, the quality of the memory-image is more akin to a memory of having done or undergone something, than to a memory of a scene in a movie. In the world of VR, this quality is most often referred to as presence. (Immersion, a closely related concept, is often conceived by researchers as pertaining to media more broadly.) VR researcher Mel Slater’s starting point, according to Kent Bye, is that “VR activates our sensorimotor contingencies in a way that fools our brain that we’re transported into another world, and that what is happening is real.” He calls this the presence illusion, and specifies two of its features: place illusion and plausibility illusion.
Another important VR illusion is the body ownership illusion (BOI). What’s referred to here is the sense that one’s avatar body is their body, or at least is responding to and interacting with a computer generated space in a manner at least very similar to how the subject navigates IRL. In relation to “homuncular flexibility,” the BOI engenders a new body, or a body with new capacities for sensation and action. This mobilization of neuroplasticity and the production of novel experiences of embodiment in VR suggest another approach to VR’s potential impacts, one that incorporates rather than disavowing the material forces that shape VR as a perceptual apparatus.
5. The positive possibilities of VR emerge from its potential rethinking/revision of sociality itself.
I have emphasized these “illusions” as they lie at the source of many of VR’s utopian and dystopian formulations. In the empathy machine view, they are what enable enhanced empathy as a seeing through another’s eyes, as well as the potential for VR experiences to exploit (for good or ill) these effects to shift consumer behavior. These same features underlie speculation on the dystopian futures of VR: Will it be more-real-than-real, summoning people to abandon their everyday lives for simulated experiences? Will it be the ultimate tool for affect capitalism, creating a scenario that updates cold war fears of brainwashing?
In my own view, these features might be better referred to as presence effects, worlding effects, and embodiment effects (rather than as various “illusions”). And for me, what’s most interesting here is the potential media-ecological ramifications. That is, for those who use VR regularly (a tiny fraction of any population, at this point), how might the perceptual apparatus of VR enable the emergence of new forms of subjectivity and sociality? If moving in and out of worlds and bodies, already a feature of the contemporary media ecology, is intensified by VR’s enhanced presence effects, might we think of this as enabling increasingly plastic forms of subjectivity grounded in differences and metamorphosis rather than identity? That is, can VR help to shift our powerful cultural attachments to particular notions of individual and social group boundaries through experiences that, while never the experiences of particular others as the empathy machine logic would have it, reveal the flexibility of the human itself in relation to (its) others?
This is a complicated proposition because, as I stressed above, computation — and a global computational ecology — is always the matrix for any VR experience. VR’s perceptual infrastructure is bound on one side by the apparatus of human perception (both in relation to its limits and its plasticity) and on the other by the infrastructures and operations of computation and the geopolitical inequities that support it. Whatever takes place in VR, takes place takes place in this human-in-the-loop system. (While a film may continue to play when a viewer leaves the room, a VR experience needs the presence of the experiencer to function.) While creating a space of emergence, VR never transcends its loop, although it may reconfigure it. That is, though VR is touted for its production of immediate experience, what is in fact important about it, and what engenders its creative possibilities, is the forces of its particular mediations–its essential difference from “real” reality and the means through which this difference is produced. The sensing and tracking of human embodiment and perception, a source of anxiety in relation to contemporary modes of surveillance, is the stuff of VR as a medium and so what enables its positive possibilities.
One of the most interesting affordances of VR is what it can engender through its looping of human experience into nonhuman forms and scales. Whether it is the expanded and vertiginous inner- and outer-spaces of Hyphen-Lab’s “NeuroSpeculative AfroFeminism,” or the experience of tree-being in New Reality Co.’s “ Tree,” or the even more complex weird-loop layerings of the very human experience of a break-up onto an anthropo(s)cenic imagining of environmental forces in Jakob Kudsk Steensen’s “ Aquaphobia,” VR’s capacity for putting us into different worlds and modes of embodiment is a prominent feature in contemporary artists’ VR. All of these experiences work to introduce us to extra-human times and rhythms as a conduit for new forms of sociality.
In conventional terms, VR is not the first medium one thinks of in relation to the social (despite social VR apps). In contrast to the contemporary foreground of hypermediation and distraction (e.g., multiple windows connecting a computer user to various people and sites at once), VR revolves around a now-unusual singularity of world and focus. It captures you in the world inside the head-mounted display. But in this spatio-temporal capture in which you are, in effect, kidnapped by the apparatus, other worlds may be opened, and the experience of these worlds may enact a tutorial in different times and spaces of being and becoming. There is no way around phenomenology in VR, that is, around its starting point in the specificity of human perception that gives rise to its hardware and software. But here, the intersections it produces between perception, computation, and extra-human scales, offer a tutorial in how to live in a multiplicity of worlds. It thus invites us to reimagine the conditions of possibility for new forms of sociality to emerge.
Deborah Levitt is Assistant Professor of Culture and Media Studies at Lang College, The New School. Her new book, The Animatic Apparatus: Animation, Vitality, and the Futures of the Image, will be released by Zero Books on May 25th.
Notes
[1] Stephano Gualeni offers a generative formulation of VR as philosophical practice in “Augmented Ontologies: How to Philosophize with a Digital Hammer,” Philosophy & Technology (2014) 27:177-199.
[2] Cf. Jordan Belamire, “My First Virtual Reality Groping,” Medium 2016, and the response of QuiVR’s developer, Jonathan Schenker on sexual harassment in QuiVR, Upload VR 2016.
[3] Jaron Lanier, Jeremy Bailenson et al. “Homuncular Flexibility: The Human Ability to Inhabit Nonhuman Avatars.” Emerging Trends in the Social and Behavioral Sciences, 2015, pp. 1–16.
[4] While the early experiments at VPL used “primitive” VR technology and were not tested in scientific studies, Lanier teamed with Bailenson and other researchers to develop new experiments at Stanford’s IVHL, where Lanier’s original results were affirmed. Won, Bailenson, Lee, and Lanier, “Homuncular Flexibility in Virtual Reality,”Journal of Computer-Mediated Communication (2015), 4.12107.
