What I learned about our reality in VR – and what journalists and educators should do with VR

Reality is relative. This is what I have learned after several weeks of research in virtual reality and in the real life of my virtual friends. Virtual Reality is not a second class reality – it works like the world we know. And in the time of coronavirus and climate change, this is actually great news. By Eva Wolfangel

I feel the sun on my skin, and it feels warm – even though it is late at night. It cannot be true and yet it is true. I feel it. This was my first impression when I arrived in arother reality for the first time in June 2016. At first glance, everything seems to be familiar – it looks like earth. I am standing under a tree in the evening sun. The shade of the leaves draws a pattern on the ground. I hear the murmuring of many people around me. The sky is incredibly immens.

In reality, my body is standing in my small dark office, and yet my mind is somewhere else. I put a bulky headset over my head and thick headphones, and suddenly I found myself in this other world, another reality. This reality is called “Altspace VR” and it is a social meeting place in Virtual Reality. This is not about games, it’s about meeting people. I am not a gamer, and that is why I came here: I love to be social and I am naturally curious.

I was getting excited the more I looked around. I could hardly believe how real it all seemed. My brain was telling me that this was real. ‘You are really here.’ At this point I did not yet know how much this research would change my life. It would lead me into new realities, into the real lives of other people, and all of this will seem much more “unreal” than the nice evening I was standing in.

Standing under these amazing trees I wondered: Is this just a big deception, like in The Matrix? In the movie, life takes place in a parallel world that doesn’t really exist. It’s the brain’s imagination. The people in The Matrix don’t know this because the parallel world feels so real. But it only exists in their brains. The world I was in at that moment also didn’t not really exist, but I had to tell myself that again and again as I enjoyed the evening atmosphere that day.

Some researchers say: With virtual reality we have the chance to choose between several realities for the first time in the history of mankind – beyond fantasy trips, delusions or drug trips. “Virtual reality is as real as the physical world,” says philosopher David Chalmers in an interview (https://www.edge.org/conversation/david_chalmers-the-mind-bleeds-into-the-world). The philosopher has been working for years on consciousness and the question of how real reality is.

How do we know that the “real” world around us is not just a simulation? Philosophers have been arguing about this not only since René Descartes’ thought experiment in the 17th century, when he asked: How can we know that we are not controlled by a demon who merely makes the world around us seem real? In fact: we can not know.

Virtual reality forces us to think about this question again. The immersion, the feeling of being there, is so strong that doubts arise as to whether there are any relevant differences to the real world. Chalmers clearly says “No”: “Virtual reality is not a second-class reality.”

This can also have negative consequences, as I realized on my first day in the other reality. I felt like I was rooted to the virtual floor as a big red man came towards me, stepping much too close. I couldn’t move backwards, because behind me there was a staircase going down – a virtual one, but in this situation it feels real. The man grabbed my chest and gave a dirty laugh. I froze. I knew it wasn’t my body, it’s “only an avatar”. He didn’t really touch me. But when I looked down at myself in virtual reality, I saw my avatar – and it felt like me – and I saw this man’s hand, touching it. What is real?

This key experience made me unpleasantly aware that virtual reality was much more real than its name suggested. It may not be material, but it is real. Compared to chats, video conferences or online games on a 2-D screen, it is a quantum leap to virtual reality. In VR, I am right in the middle of it, not just viewing it on a two-dimensional screen. The experiences are close, very close. I asked other women I met in VR as I researched more about the topic for an article, I found outed: sexual harassment in virtual reality is a frequent phenomenon.

But there were nice things about having a body in VR as well: When my friend Sana huged me for the first time, I could feel her breath on my neck, I could feel the warmth of her body and I could feel her arms as a fine tingling sensation on my back. That couldn’t be true! Or can it?

To find out, I visited Thomas Metzinger, a philosopher at the University of Mainz in Germany. For many years he has been dealing with the question of how real our reality is. He points to the red sofa in the corner of his office. “Is that real?” he asks. He waits – and shakes his head. “This red of the sofa, the colours, these are characteristics of a model in our brain.” The red is created in the mind of each individual. What we experience is far less real than we think. “The fact that something feels real means that the brain produces a model with a high degree of predictive accuracy,” says Metzinger. We step under the cone of a lamp and now see everything brighter – as expected. We can hear people from far away and the closer they come, the louder their voices become.

I sat down on the red sofa, and it was soft. It felt exactly as I expected it to. So it must be real.

And this is also the case with our sense of self, Metzinger explains: “According to my theory, the feeling of being yourself is a simulation of the brain, an inner model with many layers.” The brain calculates from all the available information the best hypothesis, he explains, the most probable variant of reality. The brain presents this hypothesis to us as reality. “If we do it cleverly, it’s quite possible that you might think you’re in another body.”

As early as 2007, Metzinger and his colleagues transferred test persons into a virtual body during an experiment: Back then, they created a video of the test person from behind. The person saw the video from him or herself in front of his or her own body. Then the researchers stroked the participant’s back. At the same time, the test person saw how the virtual body in front of him or her was also stroked. “Thus the brain begins to believe that the avatar of the own body somehow belongs to the own body.”

Thomas Metzinger speaks of a “myth of authenticity” that is attached to our apparently real world. “Our real world is, of course, not real, but completely distorted.” That does not mean necessarily that there are no physical bodies and no outside world. But it does question the claim that our material reality is the only reality.

And I felt on my own body what Metzinger says: When I put on the VR headset, my body stayed in my little dark study. But my consciousness was beamed into another body in another world – if there is something like consciousness at all (but that is another question). One day, for example, I found myself in Sana’s VR room, she calls it “Time Machine”. She has sad poems on the walls, and in one corner there is an open fireplace. I felt the warmth of the fire, (at least my brain told me I do) and I heard the warmth in Sana’s voice. She sounded a bit sad.

At this fire place we philosophized  night after night, me, the German science journalist and her, the widow from Kuwait, a strictly religious Muslim. In those moments, I was really there with her in a room. I enjoyed those evenings with Sana, I like her, and we learned so much from each other. Would we ever have met in so-called real life? Probably not.

I realized that this parallel world, into which I immersed myself deeper and deeper, was as much my real life as the material version that most of us call reality. It changed my real life. The melancholy after a thoughtful virtual evening with Sana mixed into my real next days. I worried about her when she said goodbye in a particularly sad way, and I felt the warmth of our encounter the very next day.

How is it possible that VR is both “virtual,” but still manages to trigger such very real feelings? I wanted to learn more about this and so I moved further into this reality. Every free minute I put on the headset and the headphones, and my mind left the real world, and moved to the virutal one.

Sometimes this became adventurous. Together with different people I explored the many different rooms in Altspace, we beamed from one place to another, we flew, we wandered through a maze and fought sword fights that really took up the whole body: When I swung a sword in the tavern, I also swung my arm with the controller in my study. If another fighter broke through my cover, I duck away and cowered on the floor of the room – which for me were creaky wooden boards of the tavern. Luckily, I thought, nobody is watching in this real world.

Once I was standing on a high rock. In front of my feet hundreds of meters of nothing. I turned my head carefully: behind me there was a wall, there was no way to escape. I trembled and I could not move my feet. For a moment I thought of my other world, the safe one, where I was standing on a solid floor. But this idea did not help at all. The view from the rock felt more real. My body signalled: Danger.

What triggered my fear of heights is what researchers call “place illusion”. The feeling of being real on the spot is not dependent on perfect graphics. Even if the place does not look like it would look in the real world, my brain completes everything that is missing. The fact that the avatars in Altspace look more like robots is not a problem. On the contrary, it even makes it easier for the brain to concentrate on the essential: The interaction between me and other people and the virtual environment is important for the “feeling of here”.

This is at least what scientists claim who did some initial studies: Keisuke Suzuki from the Sackler Centre for Consciousness Science at the University of Sussex let users try different virtual worlds and found that “as soon as you can interact with people or things, it feels real”. Because our brain can test its hypotheses, such as whether everything looks brighter when I go closer to the lamp or whether the people I see in the distance become taller when I approach them and their voices become louder. “The brain wants a coherent story”, says Suzuki.

The same things that took me to my limits (although I am not particularly frightened of heights in real life), can also help others: I met experts who are working on therapies for fear of heights or claustrophobia in virtual reality. This so called exposure therapy works good in VR – and is often easier to realise there than in the material world.

One of my virtual friends has started her own therapy: Crystal has social phobia in real life, but in virtual reality she is one of the most networked people I have ever met. Everybody knows her, and she organizes the best parties in Altspace – that take 48 hours. Because of the different time zones, she says. And no, she has no problem getting close to people in virtual reality – it is much easier for her than in real life. She even has built a virtual camping site with an amazing starry sky for her guests – because you might want to rest when it is night in your timezone and you are at a 48-hour-party.

The more time I spent in the virtual world, the more I felt doubts – and my friends and family were amplifying them. “You spend so much time in this virtual reality, you don’t even know these people.” “How do you know they’re not lying to you? That they’re not someone else from who they pretend to be?” I grew defiant. Of course I know Sana! I hear her voice, I see her gestures, we have talked for nights on end – I am not naive after all!

I decide to prove it. “Sana, I’ll come and visit you, in real life!” But my friend didn’t seem to be very excited about this idea. She didn’t show up in her virtual room for days. But finally I convinced her and I booked a flight to Kuwait City. A few days later, she hugged me for the first time in the material world. And this hug, in a hotel lobby under heavily rattling air conditioners, felt familiar. Everything was like in virtual reality. Her voice. Her gestures. I felt her warm body. The only thing that was unfamiliar was her headscarf. She does not wear it in VR.

In those three days in Kuwait City I often thought how important these new realities are for people like Sana. Sana told me that she is not allowed to meet men because of her religion. She also has the feeling that she does not belong in this world. Normally, she keeps to herself. I felt this during my visit: she wanted to show me Kuwait City, but we kept getting lost. Normally, without a visitor, she refuses this reality by not leaving her house if it is not absolutely necessary. But now there is this visitor, and she felt obliged to show her hometown. “I don’t belong here,” she says again and again. It is different in the virtual reality: There she feels at home.

Her loneliness became even clearer when we stoof in front of a plaque of honor for the fallen heroes of the Gulf war in 1990/91. Sana points to a young man. It was her husband. He fought against Saddam Hussein and died. “I miss him so much,” says Sana. I don’t know what to say.

At the airport, as I left, I asked, “Sana, why didn’t you want me to come?” “I have no friends in real life, and I was afraid of what would happen if you came into my real life now.” She worried about losing a friend as soon as we transition to the material world. But she did not. We had a great time together in the material world.

Back in my real life and thus within reach of my VR headset, I got to know Cattz. He is a dude, not especially polite, he has always some rough words on his lips. Sana thinks he is a lout. Cattz, who always talks about his three ex-wives, his five children and his fatal illness. But he’s fun to be with. He is extremely helpful and shows me all the tricks in different virtual worlds. Together we steer a spaceship and swim in virtual pools.

Later I celebrated the engagement of Ben and Shoo. They met and fell in love in virtual reality. Ben has built his beloved a whole virtual house, on the ground floor a bar, on the counter a cake, on the first floor a romantic bed with dark red satin bedding and a warm glowing bedside lamp. After this engagement party I decided to meet Ben in the material world.

When I met him in Atlanta, his mood was depressed. He had recently visited his fiancée in London for the first time in real life. No, she was not a stranger to him at all, but during the long hours in the plane above the clouds, he realized that he was having a long-distance relationship. He never felt the distance in the cuddly shared house in virtual reality. They were together in one room. But now he felt the distance. Or in his words: “Reality sucks.”

“I don’t see why you shouldn’t be able to live a life in a virtual environment that is as meaningful, as fulfilling, at least in principle, as life in a non-virtual environment”, says philosopher David Chalmers (https://www.edge.org/conversation/david_chalmers-the-mind-bleeds-into-the-world). Some things are going to be harder: the role of the body, hunger, thirst, sex, birth and death, he says. “Some things are going to take a while to build into virtual reality, but I suspect within a century there will be Matrix-style virtual reality, which is more or less indistinguishable from our kind of reality.” And for some people, this virtual reality could be even better, says his German colleague Thomas Metzinger: “It’s not the case that the reality we have now works particularly well.” People fight each other for example or do have different interpretations of real situations. And sometimes a virus paralyses the whole world.

Shortly after this conversation, I learned that there are in fact already people for whom virtual worlds today are a revolution compared to their material life. Like Cattz.

When I told Cattz that I wanted to meet him in the material world, he suddenly seemed to have disappeared. I searched for him for months in different virtual worlds we had met. When I asked his virtual friends, they just shrugged their shoulders: “He hasn’t shown up for months, maybe he has dies. He had told everybody that he is fatally ill.”

But he was alive, at least kind of. After months of investigation, Cattz finally answered my question on Facebook messenger. “Don’t come, I am ugly.” But I finally could convince him and flew to Spokane, Washington, where I found him in a place that could hardly be more desolate. Cattz was lying in the cellar of a littered house on a stained mattress, eating nothing but toast, Coca Cola and heart pills. When I saw him in this depressing situation, I cursed my research. I felt out of place. Why was I intruding on his personal life? What was I looking for, digging into the real lives of my virtual friends?

Cattz’s real life is so depressing that it seems to me much more unreal than his funky, funny and sometimes rough way of living in VR.

It was hard to convince him to leave the mattress. I painstakingly reconstruct what happened: His house burned down, he had nothing left, no place to stay, no VR headset. He hitchhiked for months across the US in search of his mean in life. But the material reality is not his reality. He doesn’t want to be here.

After weeks of being homeless, he finally rented this basement room. He could not afford anything else with the little money he has from early retirement. Walking the streets of Spokane with Cattz, who is clearly ill, breathes heavily, sweats and trembles quickly, I realize how important virtual reality was to him. There he can walk, fly, run and beam. Here his body is tired and ill. His real life is in virtual reality. But VR locked him out. He lost his connection to his real life, because he lost his headset. He’s gonna have to save a long time before he can afford a new one – and go home.

On the streets of Spokane I realized not only how relative the term reality is, but also the potential of these new realities. Virtual reality tore me out of my filter bubble. I met people I probably would never have met in material reality. And I made a similar journey as my protagonist Ben: I felt close to people, I hung out with them in a room for weeks – and only on my long journey to Kuwait, Israel and the US did I realize how far away these people are “in reality”. But this distance did not hinder us from having real relationships. In VR it even disappears.

Cattz finally came home. He saved money and friends helped him out, so that he could buy a new headset. I have met him and Sana again and again in VR since then – it remains our medium. And as a journalist I can’t help but see great opportunities for our industry in this medium. But  journalists and communicators urgently need to become more creative in the use of VR. After all, VR offers much more than 360 videos in which – let’s face it – usually only one perspective is interesting. And all those passive “experiences” in VR do not use its full potential. Why should our audience watch things silently and mostly passively, when they could be social and interactive?

I am currently working on such a project as a Knight Science Journalism Fellow at MIT. Educational research shows that we understand and learn things better and more sustainably when we explore them together and interactively. There are already educational projects in social virtual reality, but many of them are just replicating classic classrooms: School desks and tables, and the teacher is at the front of the blackboard. But this doesn’t take advantage  of the medium. Education in VR does not have to be frontal teaching.

My project is an interactive journalistic research in a social VR space. I as a journalist would invite an expert for an interview, say a quantum physicist or a brain researcher, together with my audience. We would meet in a social VR space and have three-dimensional models with us: an atom, Schrödinger’s cat or the human brain, for example. These things are not visible in real life.  But in VR we can not only see them, we can interact with them. While I interview the expert as a journalist, we and the audience can travel together into the atom, open the box with Schrödinger’s cat or fly through the brain and explore it. Everybody in the audience can fly like a neuron or separate two entangled quantum particles from each other, bringing them to different planets and see how entanglement works.

And there is more: This technology also makes us open towards new people and new ideas. It makes holes in filter bubbles. I have met many people for example who are quite skeptical about journalists, but who were amazingly open in our conversations. I have answered many questions, and I have asked many questions. I only have anecdotal evidence for this: my experiences and those of my virtual friends. But many have confirmed to me that they have talked to people with whom they would not come into contact in real life or who they are quite sceptical about.

Journalistic research and experience in addition shows that one of the only things that reliably help in the current crisis of confidence in the media is transparency and meeting people in person. Many publishers have created elaborate dialogue processes with readers. This is a side effect of my project: the audience can be part of a journalistic process, they can explore a topic together with a journalist. They can see how we work, we can debate about things like journalistic research and fact checking. We can meet in person – without having to travel. Without flying, without polluting the environment. Even without transmitting viruses.

And after the first experiences in “remote learning” at Harvard and at MIT in the face of the corona crisis, one thing is for sure: everything that helps to give this learning true spatiality and the feeling of really being together makes digital learning and communication over distance more successful.

We should be optimistic about virtual reality.