CONNECTOME: A Game Of Points is an immersive art experience on Meta Quest and Apple Vision Pro that utilizes hand tracking on both platforms and eye tracking on Vision Pro. Developer Grant Hinkson detailed the pain points of getting his application across the finish line on Apple's headset.
Hinkson and I met at AWE in June for a demo of CONNECTOME on Apple Vision Pro. UploadVR met Hinkson last year at AWE for a demo of the same game on Meta Quest shortly after its May 2025 release. At the time, Hinkson expressed his desire to bring the game to Apple Vision Pro.
Don Hopper
Like Don Hopper's experience last year, I found myself mesmerized by the elegant simplicity of the app, bolstered by AVP's eye tracking to feel more connected to what I was doing. Practically, CONNECTOME is a straightforward series of tasks of connecting dots with hand tracking to create different shapes and progress further into the world, in a soothing spatial environment anchored with a zen musical score. It is the type of app where time disappears. It felt like I was only in the app for a few minutes when it turned out to be over twenty.
After my demo, I asked Hinkson a simple question: what were the challenges in building the same application on Apple Vision Pro versus Meta Quest?
Due to the lack of connectivity of some tools and libraries available for Vision Pro in Unity, Hinkson faced a number of obstacles he did not anticipate. I will do my best in this next section to avoid getting too deep into the weeds.
The primary driver of CONNECTOME is a Unity library called Shapes, built by Freya Holmer and this presented Hinkson's first AVP hurdle. In contrast to Meta Quest, Shapes was not compatible with PolySpatial, Unity's default toolkit for developing for visionOS. PolySpatial renders through a kit that does not accept the custom shaders Shapes uses. Hinkson found a workaround for this by switching the Unity app mode to Metal, a different rendering service. This solved one problem and created another.
Metal apps built in Unity do not provide eye-based hover effects, a foundational part of Apple Vision Pro's user experience. In layman's terms, this is what you see in an eye tracked headset when the cursor moves to whatever icon you are looking at. This is no fault of Apple, who introduced an API for Metal developers at its annual WWDC (Worldwide Developers Conference) in 2025. Instead, the gap here lies with Unity, who has not implemented support for it. For its part, Unity has acknowledged this and confirmed it does not plan to add integration.
Faced with this new obstacle, Hinkson built a custom bridge from scratch, one that drilled down to the native level of Unity's compositor, registered the necessary eye tracking information in each frame, and wrote the texture for visionOS to render the expected hover effect. Unfortunately, this workaround would be wiped out every time a new build of the app was generated as Unity regenerates its visionOS on each build. So a post build patcher had to be built that runs after the Unity export completes.
Then, there were the colors. Each room in CONNECTOME has a single hue as its anchor point, one of the best parts of moving room to room. For this, a second bridge had to be built as the system used, SwiftUI, has APIs that are also not exposed by Unity. Another post-build patcher was necessary to ensure the Unity scene maintained the SwiftUI modifications.
Hinkson's experience is illustrative of the challenges of building for a new platform and offers some insight as to why app development for new XR devices takes longer than consumers expect. Hurdles like those detailed here are not known until the developers get knee deep into these projects and run full speed into the proverbial brick walls. Only then, after seeking alternative solutions, including building their own from scratch, do these projects see the light of day.
As previously stated, Apple was at least aware of these obstacles and provided tools to aid developers. Meta has had a long standing partnership with Unity and multiple software development kits available for Unity and Unreal Engine. Snap is doing the same with its upcoming Spectacles, making all efforts to ensure teams have what they need, but this only goes so far. Without fully integrated support from engines like Unity, Unreal, Godot and others, developers will continue to find themselves seeking workarounds due to the lack of said integrations.
As adoption of XR technology becomes more ubiquitous and the demand for more robust applications increases, these game engines will either have their hands forced or developers like Hinkson will make their custom tools and libraries publicly available, sometimes at a price, to bridge the gaps. That said, until adoption gets to that level, developers for new hardware platforms will continue to face these issues and spend significant time and resources reinventing the wheel in order to stay on the cutting edge of new tech.
CONNECTOME is available now on Apple Vision Pro and Meta Quest.
