Field Notes #5

What Is Real

Like many, and likely you, I enjoyed Kevin Kelly’s Wired article about AR recently, exploring how AR will spark the next big platform which he called Mirrorworld. It paints a picture that is both ambitious, largely already achievable at small scale, and optimistic about how this technology will change the way we interact with the world around us. His enthusiasm comes off the page.

"Eventually, everything will have a digital twin. This is happening faster than you may think." - Kevin Kelly

An exact digital replica of the real world is a powerful concept but the impact is hard to predict. And it has been making me think about what really is real.

Like the internet, The Mirrorworld will reach into every part of your life and represents a piece of infrastructure upon which millions will be able to build applications, tools, and entertaining experiences. Value to all our lives, or value for just some.

Anne McKinnon, VR/AR Consultant and Writer at The Boolean says about VR: “…travelling in VR will break down the barriers we currently face by living in our corner of the world”

The Mirrorworld will only further this goal of shrinking Earth, and in many much more practical ways, allowing for anyone anywhere to be present with you wherever you are is only one small and obvious way in which that will happen.

A virtual layer fitting perfectly over the real, with us wherever we go and giving us powers we don’t currently have, at all times if we so wish. And with memory: what you do in this Mirrorworld will be there tomorrow or next week, and can be discovered by someone else.

The real technology underpinning the AR Cloud is a persistent spatial map of the world. There will likely be competing data-sets, some created by private companies like Google, others by all of us collectively, uploading little bits of data as we go (for example using’s AR scanning & depth cloud technology).

Spatial mapping with

Ori Inbar at Superventures called it ‘moving the internet off of screens and into the real world’. But that hints only at information, as powerful as that alone already may be. More specifically for storytellers and lovers of stories, it allows us to create and enjoy new forms of immersive experiences which firstly are “persistent across space and time, and across different devices.” And secondly, are aware of where you are, what is around you, and what you are looking at any given moment.

Rony Abovits and his team at Magic Leap have a different name for the Mirrorworld, the Magicverse. And they are actively exploring it on multiple levels, with practical work and exploratory research. The Magicverse hints at city-wide storyworlds, experiences that stay with you as you leave your home and continue out in the street, and which connect people over space and time.

Their hugely ambitious and impressive Mica demo is a virtual human, who could travel with you and behave like real people do, in this Magicverse. Future scenarios for what happens when virtual AI driven characters walk with us in the streets and can have a persistent impact on the world will be an article for another day.

And it is not just Magic Leap who are exploring this. VC funded companies like Artie and Fable are exploring it, artists like Marpi are pushing it further. Just a few different outcomes to the same essential premise.

Marpi’s Digital Beings

I can imagine the Star Wars cantina scene being real within years, except all the aliens are virtual.

What I have been thinking a lot about lately, and which strikes me as an especially interesting side-effect of the Mirrorworld, which is more broadly the reason this moment in time, now, is a significant moment in human history, is that in essence you and me now no longer live in the same reality.

We might not be looking at the same Star Wars cantina.

We can no longer say that despite our different interpretations and opinions, we know for a fact that when we’re looking at something together, we’re actually seeing the same thing too.

What we see, and what we experience as humans has always been different in so many ways, open to interpretation and bias and the fallibility of memory. You might have seen the Eiffel Tower during a sunset while I saw it in the morning. Someone saw it in love, another after a breakup. But in essence the physical, actual tower we both saw was the same.

With the Mirrorworld, we’ll both possibly see a different Eiffel tower. You could have an information layer turned on which explains its history, while I have creepy creatures crawling all over it.

The difference the Mirrorworld will make to how we see the shared world we all inhabit has many incredible advantages. It allows for creative expression and information to be layered over the world, like a layer of magic.

And that is especially true when wearable AR becomes affordable enough for you and I to perhaps be wearing glasses that reveal that digital layer everywhere we go. Wayfinding, wikipedia, real-time feedback about whats around you, seeing around corners, real-time collaboration through Holograms. The list goes on. So many apps will provide us with genuine powers that enhances us as humans like never before.

“It's not just about augmenting reality, it's about augmenting humanity” says Helen Papagiannis, Ph.D., Augmented Reality Specialist, Author of Augmented Human.

But as part of that, undeniably, our versions of the real world are diverting. Different apps on your device compared to mine.

Like different journeys through Bandersnatch, along with the digital layer over the real world also comes a customised version of that world, powered algorithmically by our experiences, interests and connections - just like the web is already customized and we no longer see the same homepage, newspaper stories, or Facebook ads. Your Spotify experience is different from mine. Your Google search results are different from mine. Your Twitter timeline, Airbnb holiday suggestions, they’re all different from mine. And I am just scratching the surface.

Imagine personalization applied to the whole world.

Today, where others see grass and pigeons, you’re seeing a Sentret, Swinub, Hoothoot. Or if you’re lucky a Phanpy, Skarmory, Qwilfish (all Pokemon Go characters).

Now imagine if the Mirrorworld blends seamlessly with the real - with graphics that don’t stand out but rather blend in.

This does not just affect the visual world, immersive audio is a huge part of the Mirrorworld, and arguably, much easier to make believable, and harder to distinguish.

And this also affects how we see each other. AR filters suddenly become permanent. They travel with us, become our default ‘setting’. Snap Camera already allows them to appear in Twitch and on Skype or Google Hangouts. Up next, they’ll be a permanent part of our persona. One of several. And a person in your family could forever have Elven ears, or digital tattoos, or have the head of a poo emoji.

Travis Chen showing Snap Camera, which integrates with Skype, Twitch and Hangouts

When I meet two friends in 5 years time, and we’re all wearing our AR wearable headsets, we’ll no longer see the same faces. The filters we have applied to each other will all be different. We just won’t know what they are.

No big deal. So you’re Aquaman to me, and I have floppy dog ears to you. Fun! And our mutual friend looks different to both of us.

But it is fascinating to imagine taking that principle, and adding to it the fact we’ve crossed the Uncanny Valley and are now edging into the age of completely believable graphics, real-time motion and face tracking, and dynamic lighting and shadow effects, all happening through a camera phone.

As I said, blend seamlessly with the real.

Creative expression will spawn many new forms of humans in the Mirrorworld. We’re no longer bound to our natural bodies, nor by the laws of physics or nature. The incredibly creative face filters scene that has adopted AR on Snapchat and Instagram is only just a starting point where we explore beautiful, fascinating new ways to express our identity and experiment with what a human looks like, what a human fundamentally is. Recent academic research by William Steptoe, Anthony Steed and Mel Slater into additional body parts such as virtual tails hints at a future human that looks radically different from today. And it can be done without it being either painful or permanent.

I find it fascinating to think about what that means socially, politically, legally. Philosophically! Because to me it seems like a fundamental shift that affects not just what we do, but what we believe reality is.

When none of us know exactly what the other is seeing, what is left of reality? When we look fundamentally different to different people, what is the true you?

These are early days, but change is here.

The idea of echo chambers in this context is interesting because - if an echo chamber is just the news, music or tweets, or connections on Facebook, it already shapes your opinions and significantly affects how you think and feel. We have seen this play out over the past years to dramatic effect. If you extend that echo chamber out to literally everywhere, to contextual information and experiences overlaid onto the world around you, if your wearable AR glasses are giving you feedback about places or people, or guiding you as you travel, the echo chamber has infiltrated your every waking moment. Until you take them off of course.

In a slightly more dramatic hypothetical ala Black Mirror, you could ‘mute’ people and simply not see them. Your wearable headset will render them invisible. Someone you don’t like. Someone who is homeless. And you won’t know if someone else has muted you.

When reality can be fundamentally altered, it can also be questioned in new ways. When our memories of each person or place, diverges fundamentally depending on your ‘wearables settings’ this impacts us not just in terms of abilities but in every other layer of our society as well.

All of this is not to say the Mirrorworld should be cancelled, and sent back for a refund, of course. But I find the innovations the Mirrorworld will spawn to be fascinating to brainstorm about, good or bad or anywhere in between.

I loved Kevin Kelly’s enthusiasm, as I said at the beginning. And I share it.

And Kelly acknowledges the risk too, saying "we can imagine so many ways it might hurt us."

Matt Miesnieks from and others of course have already explored the ethics of the AR cloud in many ways, saying in one Medium post “…ubiquitous AR enables use-cases and end-user value that no one really understands yet.”

And what is fascinating is that the legal world has not yet even caught up with the impact the internet has had, on many issues policy trails reality by decades. It has only just started to catch up with the fact that we all have cameras with us - yet the Mirrorworld is being built today.

In response to Kelly’s concept of the Mirrorworld, Ashley M Richter, who writes about security and spatial computing, said on Twitter: “And whatever countries build it and secure it, get to set its founding rules and ethics of engagement.”

But like all technologies, the broader cultural impact it has is defined by how its early adopters innovate, create, and experiment on top of the technology to push it forward.

Rachel Uwa, founder of the School of Machines, Making & Make-Believe in Berlin sums up perfectly how we can best respond to this. “We all should start to learn the technology so we can have a say in how this tech will be used.”

Now is the time for people from all disciplines, artists, writers, architects, psychologists, neuroscientists and creators from as diverse a range of backgrounds as is possible, to explore, share and discuss, and help lay those foundations for the Mirrorworld.

There is still much research to be done to understand the Mirrorworld. I’ll do a follow up post I think, diving more deeply into where it might take us in ten years time.

I’d love to hear your thoughts on these points, or other examples you’ve come across of how the Mirrorworld might affect us in the future, or already does today.

In the mean time, as always, thanks for reading.