Experimenting with VPS on 8th Wall
An Exploration of WebAR Innovation
This month we’ve been exploring 8th Wall’s ‘Lightship Visual Positioning System’ (VPS) for Augmented Reality, sharing an in-depth look at the process behind creating experiences using VPS. We'll outline our development journey of an AR installation inspired by Thailand's Songkran (New Year) festival, made as part of the 8th Wall ‘Time Travel’ Challenge. From grasping the basics of VPS technology to navigating the practical challenges of real-world implementation, we wrote a few notes as we created a WebAR experience from start to finish.
What is a VPS and platforms used
Visual Positioning Service (VPS) enables precise indoor or outdoor navigation using AR technology, leveraging computer vision and machine learning. A VPS employs visual cues to precisely determine a device's location, offering improved accuracy compared to GPS. 8th Wall’s platform supports Niantic’s Wayfarer crowdsourced real-world locations (used in games like Pokémon GO and Ingress).
The 8th Wall Challenge
The challenge revolved around creating an AR experience on the theme of a time capsule, with a sub-theme focusing on cultural holidays. With our Tech-Lead, Liam, being based in Thailand, we decided to centre the experience around the upcoming Thai holiday of Songkran, known for its week-long water related festivities.
The goal was to incorporate water elements into the AR installation, utilizing scans for occlusion. The project was approached with an open mind, without a client brief, allowing for creative exploration within the given theme. First we wanted to understand how the VPS worked, this process involved testing the 8th Wall VPS in various locations.
During the ideation phase, after finding a suitable location, we visited the site to brainstorm ideas, considering the medium's limitations. Sketches and concepts were developed with this in mind.
Positioning and testing involved using primitives and scanned meshes to assess various elements such as drift and occlusion effectiveness. This testing was crucial as virtual testing within 8th Wall did not perfectly mirror real-world conditions.
Finally, the focus shifted to polishing the project, enhancing visuals and effects. Adjustments included replacing a plastic bowl with a more traditional one featuring real-time reflections and adding flowers to enhance the water's appearance.
Tips for VPS Scanning
To effectively scan using Visual Positioning Service (VPS), it's essential to adhere to specific techniques:
Try to limit the scanning area to a 10-meter diameter, focusing on smaller features like statues or murals rather than larger structures like buildings.
Ensure the scanning environment is obstacle-free and plan your route beforehand.
Keep the object in focus, moving slowly and smoothly around the object being scanned to capture a complete 360° coverage.
Vary distances and angles for optimal 3D mapping, maintaining constant motion and avoiding sudden changes in direction, don't stay static.
Aim for a steady "sweeping figure 8" motion with your device to enhance the quality of the scan while avoiding extreme vertical viewing angles and fast camera movements.
Additional scans can be added to improve tracking and mesh quality.
To find the best VPS location, focus on prominent visual elements like landmarks, patterns, or notable objects for consistent tracking. It's important to have good lighting and a steady environment with minimal activity to prevent interruptions. Steer clear of areas prone to wind, vibrations, or moving objects that might disrupt the AR stability. Look for places with low foot traffic, minimal vegetation, and sturdy structures while avoiding reflective or glass surfaces. We found it easiest to test locally so you can quickly test and get a grip on how a VPS works, for instance we used a local café before setting further a field for this project.
VPS Challenges
In some areas like Bangkok, there's a scarcity of waypoints, prompting the need to create one’s own. Conducting on-site tests is preferred as virtual testing may not accurately reflect real-world conditions such as lighting, foot traffic, or furniture/ decoration movement.
Keep in mind factors like scan quality, weather, and device variation may lead to drift or differing experiences across devices. It's important to maintain visibility of the scan area, especially for experiences involving upward views.
For large structures, we believe it is possible with 8th wall as we did have some success with a large building tracking some basic geometry but ultimately found the drift and tracking to be too unreliable. However this site wasn't close and would require a few visits before we could ascertain if it was usable. Consider alternative solutions like Google Geospatial for better accuracy where possible.
Conclusion
Overall when it worked, it worked great. The tracking was on point certainly for a web based solution. However from a developers perspective there are a few drawbacks. The scarcity of scans in Bangkok compared to other cities like London or those in America posed challenges for this project. Bangkok's lack of 3D data from Google Earth further complicated matters, and existing scans were often messy, requiring additional clean-up. Testing the site before developing ideas proved essential, particularly for assessing drift and stability. Initially aiming for larger structures, such as buildings, revealed issues with drift, emphasizing the importance of site-specific considerations.
Developing for 8th Wall as a Unity developer presented some challenges, particularly in implementing interactions and dealing with file type limitations for effects like water simulations. Working with VPS proved time-consuming, because it required outdoor testing in hot conditions, although incorporating accurately scaled humans into a Blender scene helped visualizations done at home. But other scanning tools like Lens Studio and Niantic inside of Unity, offer alternative approaches.
In conclusion AR scanning and VPS, while versatile, present their own set of difficulties, from weather and distance challenges to technical limitations, but the benefit is also compelling: AR experiences that naturally fit onto a real world location, super tight positioning tracking if done right, and cross-platform compatibility without the need for LiDAR.
Overall we thoroughly enjoyed the process, learned a lot, and are eager to continue exploring and working with 8th Wall and VPS in the future.