Field Notes #2

What we describe as ‘real’ is our personal illusion, made up of a continuous stream of sensory input delivered consistently to the brain by our limbs, eyes, nose, skin, etc. It feels real because it happens in a way that is believable, it is predictable because it mostly happens in the same way. Everything fits within the system we’ve experienced since birth. Gravity is consistent, so if you jump, your brain senses your return to Earth, and your impact, in the way it expects to.

One very important part of this conscious sense of world is around tool use, working with things with our hands, in front of us. Sensing our feet interact with the ground, and feeling pressure pushing back. This sort of reality is now slowly possible to replicate, amplify and play with in Mixed Reality experiences. And when done well, it creates a confusing feeling where the brain received the right kind of sensory input, the predictable amount of response from the world. This is something which we might call ‘presence’.

A lot of research and money is going into making that presence feel even more powerful, even more ‘predictable’. But  having a pair of floating hands on a disembodied character is quite a freeing experience. The non-human form makes me think I really could be anything in any form.

Maybe we’re looking at virtual and augmented experiences with some tunnel vision?

I’m interested in is whether we can experience the world as something non-human. What is emerging now is an awareness that the human form we’re limited in the real world is not a limitation at all, in the virtual. We are physically much more fluid in a virtual setting, be that Virtual or Augmented Reality. Just like someone without his or her hands might develop incredible feet dexterity, as a rather simplified analogy, we can quickly adapt to all kinds of unusual and different human shapes. 

And our brains are actually incredibly adaptable to a non-human way to experience, and interact with a virtual setting. It can adjust to a new set of sensory stimuli. 


One project that explores this concept introduces a virtual tail, which test subjects control with their existing human form. This comes out of UCL, a research experiment by William Steptoe, Anthony Steed, and Mel Slater.

Having a new limb, one that humans traditionally don’t have, is an interesting experiment. That many of the test subjects that were a part of this study quickly picked up the ability to control that limb is amazing. And that the brain starts to quickly feel protective over it, trying to avoid perceived risk for a limb that does not exist, is extraordinary.

“Participants experiencing body ownership were more likely to be more anxious and attempt to avoid virtual threats to the tail.”

Read the full paper here.


Another, more playful take on this idea is Objective Realities, a series of VR experiences and installations made by automato.farm in collaboration with author Bruce Sterling. The experience changes the perspective of the spectator from a human point of view to the one of an ordinary household object, such as a fan, or an electricity outlet. 

In Objective Realities, you can see and act in a virtual home as with the capabilities and limitations of each objects while listening to their inner thoughts, wonders and frustrations as they fulfill their daily tasks.

Watch the video here.


The next leap is that we don’t all control one body, independently. Why not combine forces?

Meet the guy with four arms, two of which someone else controls in VR

Saraiji, an assistant professor at Tokyo-based Keio University, started a project called Fusion, a robotic backpack which is designed to explore how people might work together to control (or augment) one person’s body. 

The backpack includes a PC that streams data wirelessly between the robotic arm-wearer and the person controlling the limbs in VR.

“Saraiji says he wanted to see what would happen if someone else could, in a sense, dive into your body and take control.”

Full article here.

Augmented and Virtual Reality technology seems to be very much focused on recreating the human experience artificially, to satisfy the desire for our brain to experience things predictably. 

But the true power is that we can finally transcend our human limit, even if it’s only in a virtual context. One moment we might be chairs, then we flip to being the table. Next we’re the fly sitting on the surface, and after that we’re the trees and clouds, and chewing gum on the pavement looking for a foot to travel with.

In the longer term, like robotics and artificial intelligence (and if you go back in time, horses!) it may allow us to do many things which we’ve never been able to do using just our existing body parts, or to simply to ordinary human things in a radical new way. 

The immediate application that comes to my mind is this combined with creative tools. But I am sure there are many more impactful or exciting ways in which 'being human' will no longer be limited to one single predictable experience.

Join the conversation, I'd love to hear your ideas on how we could use these new skills.

Thank you for reading.


Inspired by notes from a conversation with Yates Buckley
Header image by Thomas Couderc and Clement Vauchez