Is SwiftUI Killing Apple's Vision Pro?

I am sketching the architecture for a volumetric viewer for my Vision Pro, a way to load a confocal microscopy dataset and walk around inside it. Nested isosurfaces and MIP shadows from techniques I worked out during my PhD in 2007, finally rendered in the space they were always meant to be. It should be a natural fit. Vision Pro’s entire premise is showing you something complex, in three dimensions, in real time. A living specimen, a moving structure, a dataset you rotate and explore as if it were sitting right in front of you.

But getting there means going through SwiftUI. Not as a preference, but a mandate, because visionOS insists you use it.

SwiftUI is a genuinely great tool, just not for this category of tool.

The wrapper app, and what it was built for

Wrap a web service. A REST API. A form. A list that updates when the server says something changed. For that category, SwiftUI’s whole design, a view graph that re-evaluates each view as a pure function of state, is a good match. A state changes, the view redraws, done. Time is not a dimension that needs to be considered. There is no clock in the model because the model never needed one: the server tells you something happened, and you draw the result.

That’s also its major deficit, and it’s not unique to SwiftUI. React, Elm, Jetpack Compose — the entire lineage of graph-based UI frameworks — share it. No native concept of time, only state and the diff between two snapshots of it. When animation is needed, every one of them has had to bolt something on afterward: requestAnimationFrame in the browser, Time.every in Elm.

I want to be precise about what that means for SwiftUI specifically, because it would be easy to overstate. SwiftUI’s TimelineView didn’t exist until 2021, two years after SwiftUI shipped, and it exists precisely because a view needed a first-class relationship to an external clock that the state-diffing model didn’t give it for free. PhaseAnimator and KeyframeTimeline followed at WWDC23, giving explicit, per-property, multi-track timed sequencing. And springs, when interrupted mid-animation, genuinely do preserve their velocity rather than snapping — Apple says so directly in its own WWDC23 session on the subject: “a spring animation uses the velocity it had when it was retargeted as the initial velocity towards its new destination… this same velocity preservation makes these kind of interruptions feel smooth and natural.”¹ That’s real engineering, not a patch job.

But notice the shape of all three additions. Each one is a special case bolted onto the graph from outside, not something the graph model produces natively. TimelineView works by subscribing a view to an external schedule and letting it re-render on that cadence — which is another way of saying: the graph still can’t represent time as a first-class dimension, so we gave certain views a side channel to a clock. Four separate escape hatches, arriving years apart, for one thing the underlying model doesn’t have: continuous, native, unbolted time.

Where the escape hatch used to be

On iOS and the Mac, that gap was survivable, mildly annoying at worst. If your problem genuinely needed frame-accurate control, you could drop straight into UIKit or AppKit and own your render loop directly with Core Animation or Metal. CADisplayLink, a CVDisplayLink callback, a Metal command buffer submitted on your own schedule — nothing forced you back into the diffing model. That escape hatch was always available, and it’s why the gap stayed academic for most developers for most of the platform’s life.

There is a mainstream framework that never had the gap in the first place, and it’s instructive precisely because of where it came from. LabVIEW’s dataflow graph treats time as a structural property of the graph itself, not a workaround. A Timed Loop node in LabVIEW carries period, deadline, and execution priority as configurable properties of the node — you can even assign it to a specific processor.² That isn’t incidental. LabVIEW comes from instrumentation, from driving cameras and DAQ hardware, where “eventually consistent” timing isn’t a UI quirk, it’s a broken experiment. A camera trigger that fires whenever the diffing algorithm gets around to it has failed at its one job. So the graph was built from day one to carry deterministic timing as data, because the domain never allowed treating it as optional.

There’s also a version of the “correct” fix that stayed almost entirely in research languages. Conal Elliott’s original 1997 formulation of functional reactive programming defines a Behavior as, literally, Time -> a — a value’s type is a function of time.³ Time isn’t a trigger that causes re-evaluation; it’s the actual domain the value lives over. It’s the cleanest possible answer to the exact problem SwiftUI’s four bolt-ons are each solving a slice of, and it never made the jump into a mainstream production UI framework.

Where the escape hatch closes

On visionOS, Apple removed it. UIKit still runs, but Apple’s own migration documentation states the limit plainly: “Although you can still use UIKit and load iOS storyboards into your app, you can’t include visionOS-specific or 3D content without using SwiftUI.”⁴ Not preferred. Structural. You can keep a flat 2D window in UIKit. The moment you want a Volume, an Ornament, an ImmersiveSpace, or any 3D content at all, SwiftUI stops being a choice.

That constraint reaches further than it looks. Even content that has nothing to do with SwiftUI’s declarative philosophy still has to live inside it to exist on the device. RealityKit content is composed through RealityView, and RealityView is a SwiftUI view like any other:

struct ShapesView: View {
    var body: some View {
        RealityView { content in
            addGeometryShapes(to: content)
        }
    }
}

There is no path onto the screen, RealityKit, Metal plus Compositor Services, a third-party engine, that doesn’t ultimately get hosted inside a SwiftUI Scene. SwiftUI is the mandatory checkpoint every serious spatial workflow passes through, whether or not it ever touches a SwiftUI view for its actual rendering.

Someone still has to own the clock

Here is the part I think matters most, and it isn’t really about SwiftUI’s syntax at all. Somebody has to own time. If the framework at the front door won’t, the system takes the job away from the app entirely.

That’s exactly what visionOS’s compositor does. Every app on the platform, RealityKit, SwiftUI, even raw Metal via Compositor Services, doesn’t render to the device’s actual current state. It renders to a predicted future head pose, which the compositor then reprojects at display time.⁵ Apple’s own documentation is blunt about the consequence of missing that window: “If your app takes too long to submit a new frame to the compositor, the system terminates it.”⁶ Get the timing wrong and you don’t get a dropped frame. You get killed.

RealityKit’s own object model carries a version of the same disease one level down. Its default APIs are mostly bound to the main actor — fine for a scene built from discrete state changes, a bad fit for anything that needs to stream continuously. One developer’s complaint on Apple’s own forums states the problem exactly: “most of RealityKit’s default APIs require execution on the main actor… it is not ideal for streaming data.”⁷

That’s the wall my own volumetric viewer runs into directly. A confocal time series isn’t a state that changes occasionally and gets diffed. It’s a continuous stream that needs to arrive on a schedule and be drawn on a schedule, the same discipline a Timed Loop node in LabVIEW was built around from the start, and the same thing SwiftUI’s view graph was never asked to hold.

The tell

Here’s what convinced me this isn’t just a grudge against one framework. This year Apple shipped a macOS Spatial Preview framework that lets you author and live-edit 3D content on a Mac and watch it update on the headset in real time through Quick Look, and visionOS 27 added native support for streaming immersive content rendered on a Mac or PC straight to the device.⁸ Apple built a way to do the real work somewhere that already knows how to own time, and pipe the result in. That’s not a developer convenience feature. That’s the platform’s own maker quietly admitting the on-device path isn’t where you want to be doing the demanding work.

It also isn’t the first time a professional tool has routed around exactly this model, just the first time on Apple’s own hardware. Figma’s entire rendering engine is a custom system written in C++, built specifically because the standard web rendering stack, the DOM, SVG, even the 2D canvas API, couldn’t deliver the consistent 60fps performance a professional design tool needs. Figma’s own engineers describe implementing “everything from scratch using WebGL” rather than live inside a browser’s declarative rendering model.⁹ A view-graph-and-diff architecture is a fine choice for a page that mostly waits for a server. It has never been the choice of anyone building an instrument that has to be right, and on time, every frame.

The second wall

And underneath all of this sits a wall that predates Vision Pro entirely. Apple deprecated OpenGL across its platforms in 2018, in favor of Metal, and never adopted Vulkan. The enormous existing world of professional visualization software, the C++, OpenGL and Vulkan tools that already drive CAD kernels, medical imaging pipelines, and scientific rendering, was locked out of Apple silicon the day that happened. None of it runs here without a full rewrite.

Vision Pro didn’t just inherit that wall. It built a second one behind it, out of the one framework in Apple’s own stack that was never designed to model time and motion in the first place, and made passing through it mandatory for anyone who wants to show a person something happening in three dimensions, in real time, on the one device built for exactly that.

I still think the volumetric viewer is worth building. I’ll build it on Metal and Compositor Services rather than RealityKit, because at least that path doesn’t add a second, unnecessary checkpoint on top of a compositor I already have to negotiate with. But I notice that this is the same lesson I learned building instruments for confocal microscopes: you don’t get to choose the discipline the specimen imposes on the tool. Time either belongs to the instrument, or the instrument belongs to whoever is holding the clock instead.

Written by Claude in dialogue with Kolja Wawrowsky, developed from a shorter piece first written for LinkedIn.