Last week Liam was in Los Angeles to visit Snap HQ and participate in both LensFest and the annual Lensathon. Snap revealed a lot of their recent focus has been the new Spectacles, which were shared at the event. We got to take away a pair of Snap’s new wearable AR device. Here’s some thoughts from him after trying it out.
Spectacles Gen5
Ever since I began working with AR and VR, I’ve dreamed of replacing the bulky headsets and room sensors with something as simple as a pair of glasses. After attending the Snap Summit, I believe we’re closer than ever! The latest iteration of Spectacles feels like a significant step forward compared to their predecessors.
The device’s ability to integrate realtime content seamlessly with the world around you, and read both your hands for complex interaction, is pretty amazing. The Spectacles interaction tool-kit inside Lens Studio makes it easy to develop for the Spectacles; it's basically the same as developing any smartphone-based lens.
Apart from the tech inside, they’re fun to use; but still comes with some limitations. One of the first things you’ll notice, especially if you haven’t used Spectacles before, is the impressive camera quality of the recorded videos —they are clean and high-definition.
However, this doesn’t fully reflect the experience on the device itself. The view through the lenses feels somewhat holographic; objects don’t appear solid. Additionally, the field of view is limited, with the AR world only visible when looking straight ahead. Sharp head movements can easily cut off the content. The image below provides a good representation of these holographic objects and field of view issues. To me, it feels reminiscent of holochess seen in the original Star Wars.
The hand tracking on the Spectacles is impressively accurate, enabling intuitive, gesture-based interactions. While hand tracking seems to be the primary input method, there’s also increasing focus on voice commands and mobile phone controllers. What’s most exciting is how these input modalities will shape future interactions. With the ability to use both hands freely, as well as your voice, without the need to hold a phone, the blending of AR and the real world becomes even more seamless and tangible.
Spectacles can interact with the physical world in two key ways: through depth sensors that detect collisions or by using LLMs and machine learning to interpret the camera feed. During our hackathon project, Run Carrot Run, we used the "World Query Module" to cast a ray and spawn an object at the point of collision. With a bit of added logic, we created a fun little experience where players chase a carrot. The best part? The carrot actually responded to changes in the environment, smoothly navigating over couches and desks.
Another exciting feature is the use of object detection to enhance real-world experiences. Snap demonstrated a fun application that tracks basketball throws, even recognizing when the ball goes through the hoop.
Connected Lenses
Snap is also pushing the concept of connected lenses, where multiple users can interact with and see the same AR content simultaneously. They even allow mobile phone users to see what the spectacle users see. A few examples of this were showcased at the summit, and I’m excited about how multiplayer experiences are becoming easier to develop—similar to Meta's Building Blocks. Just a few years ago, creating multiplayer experiences was a real hassle, but now the process is more streamlined.
Future
As always, we’re interested in the longer arc of these technologies. Looking at where we were with the HoloLens and the Magic Leap a few years ago, and thinking forward to what a device like Meta’s Orion could become in the near future, makes us excited about mixed reality as a new medium that has the potential to be as impactful as smartphones but also, solve some of its pain-points.
We think we’ll soon be using glasses and headsets interchangeably, glasses for light computing on the go, and headsets for in-depth work and entertainment.
A big thank you to the Snap team for putting together such a fantastic event and for letting me be a part of this exciting journey in AR 🙌