Whale in Colosseum
VisionOS Simulator Geo-tile Experience
We were excited about the launch of Apple Vision Pro and wanted to create something using the VisionOS simulator that would start a conversation around the possibilities of geo-location based AR/VR experiences. A key feature in the Scenery app.
My role was to conceptualize ideas that would be fascinating especially in a life size monument. This concept stuck and then I created the shader graphs for water, particle effect for water droplets, obtaining a whale model, compiling the scene in Reality Composer Pro and creating the video for social media.
Video had over 115,000 views on X and 75 reposts. 22 reposts on LinkedIn generating significant engagement and discussion.
Tools
Context
Potential use case for Scenery app in the Apple Vision Pro.
This project aimed to showcase Scenery's capabilities in developing cutting-edge AR experiences for the newly announced Apple Vision Pro, while also generating buzz around potential use cases for the device.
Building the experience in Reality Composer Pro.
Approach
Collaboration between developer, design and marketing.
I collaborated with a developer, designer and marketing head to create a suitable concept.
Utilized Google 3D Tile API to extract an accurate Colosseum model
Developed custom water shaders using Reality Composer Pro
Integrated a licensed 3D whale model from SketchFab
Created particle effects for water spray in Reality Composer Pro
Compiled and optimized the scene for visionOS
Tested and refined the experience using the VisionOS simulator
Building the water shader material map in Reality Composer Pro.
Solution
An immersive experience that can be anchored to the specific geo-location.
The final experience features a lifelike 3D Colosseum model with a massive whale emerging from a pool of water within the ancient amphitheater. Key elements include:
Realistic Colosseum model (250x250m bounding box)
Custom water shader with dynamic reflections and movement
Animated 3D whale model
Particle effects simulating water spray from the whale's blowhole
Optimized performance for smooth rendering on Apple Vision Pro
The scene can be geo-anchored to the physical Colosseum location or experienced in any room.
Geospatial API Tile - Colosseum and SketchFab Whale Asset (purchased)
Takeaways
Developing AR experiences with Apple AR Kit.
This project taught me how to develop for a VisionPro Platform.
Shader development in Reality Composer Pro required extensive experimentation due to the newness of the tool
Optimizing 3D scenes for visionOS presents unique challenges and opportunities
Geo-anchoring capabilities open up exciting possibilities for location-based AR experiences
Early adoption of new platforms can generate significant interest and engagement
Overall, it was challenging and opened up a new world to me.
