Is It Worth Developing on Vision Pro?
A Vision Pro developer shares his journey
Matthew Hoerl, founder of Beautiful Things, sits down to discuss what it’s really like building for VisionOS. We discuss the latest updates on VisionOS, spatial personas and explore how Matt has been one of the first OS developer to integrate AI in his app covering the hard lessons, creative breakthroughs, and where Apple’s ecosystem is heading next.
Before getting into the interview, I wanted to quickly introduce you to today’s sponsor Gracia AI. Gracia AI is the only app that allows you to experience Gaussian Splatting volumetric videos on a standalone headset, either in VR or MR.
It is a truly impressive experience and I recommend you to try it out right now on your Quest or PC-powered headset.
Interview with Matthew Hoerl
What was the biggest wow moment for you at WWDC?
Matthew Hoerl: The biggest wow was spatial personas. The overnight jump in fidelity is just wild. You look at it and think, how did they do that so fast? But it’s not just a tech demo, this unlocks something real for collaboration. When you can feel like you’re sharing a space with someone else remotely, it changes everything. For developers like us, it's also about the new scene persistence and the ability to simply have objects stay where you left them. People have been begging for that, and now it’s here.
So are those widgets like fixed objects in space? How does that actually work?
Matthew Hoerl: There’s a difference. Widgets are kind of like the iOS ones, glanceable info, not full apps. They do have depth now, but they’re not full 3D objects. If you already have a widget on iOS, you can bring it into VisionOS pretty easily. But then, for apps like ours, we can actually pin full objects in 3D space natively. And yeah, you can expect that to become a core part of the experience this fall with the public release of VisionOS 26.
Do you think we’ll eventually be able to leave objects in public spaces too?
Matthew Hoerl: Yeah, I mean we’re headed there. It's that idea of everyone having their own augmented layer over the real world. Right now it’s all private, you come to my house, and you can’t see my widgets unless I share them. But I think the tech is heading toward some kind of permission model. Public vs. private layers, maybe. We're not there yet, but the plumbing Apple is laying down makes that future totally believable.
There must be some tough moderation questions coming with that kind of public layer though, right?
Matthew Hoerl: Totally. If you can drop a digital object anywhere in the world, what stops someone from putting up offensive content in your favorite coffee shop? It’s a whole new layer of content that needs rules. But Apple’s clearly thinking about that. The fact that they’ve made everything private by default tells me they’re being cautious. And honestly, they should be. The tech is ahead of the culture right now in some ways.
Let’s talk about spatial personas. What surprised you the most?
Matthew Hoerl: I was shocked by how simple the process is. I expected some crazy scanning rig or volumetric capture. Instead, you just take the headset off, hold it up, and listen to a nice little voice walk you through it. In two minutes, you’ve got this eerily lifelike version of yourself. Apple nailed not just the realism, but the usability. It’s a high bar for the rest of the industry now. The fact that it runs on a mobile chip is even more mind-blowing.
When you launched Beautiful Things, what was the core idea behind it?
Matthew Hoerl: We wanted to create a place where people could interact with beautifully crafted 3D objects in a spatial way. It's part museum, part sandbox. You can discover things, place them around your space, even talk to them now. That last part has really changed the game, when objects can respond, it adds this extra layer of presence and context. It’s no longer just about display, it’s about interaction.
How do you actually source or make the 3D models in Beautiful Things?
Matthew Hoerl: It’s a mix. We curate a lot of our models from across the web, looking for high-quality stuff that fits our vibe. Some come from individual artists, some from museums. We credit the creators wherever possible. But then we also have user-generated uploads and scanning tools. I collect weird objects myself, and now I can scan and import them using our mobile app. We even let people generate stuff with AI, it’s a very open-ended playground.
What's the upper limit of fidelity you can push in VisionOS right now?
Matthew Hoerl: Honestly, we tested the limits pretty early. We loaded a 700MB CAD file just to see what would happen and it worked! That was before optimization. Afterwards, we got it down to 5MB and it still looked amazing. Apple does give guidance (100K to 150K polygons per scene) but we like to break the rules and then dial it back. We run everything through our pipeline to make sure it’s performant and looks great in VisionOS.
What has been the hardest part of building for Vision Pro specifically?
Matthew Hoerl: The biggest challenge is that you’re building for a brand-new OS. APIs are still evolving. Stuff you ship on 1.0 might break on 2.0. It happened to us, we had to rebuild our SharePlay implementation. Plus, 3D is a steep learning curve. It’s way harder than going from desktop to mobile. You're dealing with depth, scale, lighting, performance, a whole new skill sets. But that’s also what makes it fun. You’re figuring it out in real time.
With all that work and a limited user base, has it been worth it?
Matthew Hoerl: For us, totally. Look, it's not about scale right now. It’s about building a vision of the future. We’ve learned so much from building Beautiful Things, and the app has a healthy user base relative to the install base. But more than that, it’s about being early. You get to shape things. We’ve had a few firsts, like real-time voice interaction with objects. That kind of innovation is hard to put a dollar value on, but it’s priceless in terms of impact.
Let’s talk about the AI layer. How did that get integrated into Beautiful Things?
Matthew Hoerl: We started with a web-based 3D generator, then brought it into the app so you could generate objects on the fly. Then we added voice. Every object in the app is now tied to an AI model. You can literally talk to a record player or a Fiat car and get real responses. It’s wild. You’re not just interacting with the object visually, you’re having a dialogue. That makes the learning experience richer and more personal.
Where does the object info come from when users start chatting with them?
Matthew Hoerl: Every object has its own context package. We pull detailed info, some of it historical, some technical, and feed that into the model. So when you ask the Fiat about its origin, it's not guessing, it's pulling from curated data. You can even put multiple objects together and let them talk to each other. That led us to build our second app, Talking Heads. Suddenly it’s not just user-object interaction, it’s objects or characters interacting with each other. And that’s where it gets weird and cool.
How are you handling the cost of running all this AI magic?
Matthew Hoerl: Right now, there is a cost. Voice and object generation both rely on paid APIs. We give every user 25 free interactions, and after that we offer subscriptions. Our goal is to bring that cost to zero. We’re exploring on-device solutions for real-time voice and cheaper AI models. But until then, we’re just being transparent, this stuff costs money, but we’re trying to keep it as accessible as possible.
What’s your take on Gaussian Splatting. Will we see it in Beautiful Things?
Matthew Hoerl: It's a great tech and we’ve looked into it, especially for scanning individual objects. But for now, our focus is on high-quality, curated pieces and user uploads. We think Apple might bake something like splatting into their own workflow eventually, especially with spatial scenes becoming a thing. We’ll keep an eye on it, but we’re not building full environments. Our lane is beautiful standalone objects.
If Apple gave you one new feature tomorrow, what would you want?
Matthew Hoerl: Honestly, they already gave me the big one, persistent objects. That was at the top of my list. Next would be a more affordable Vision Pro. I call it the Daft Punk model: harder, better, faster, cheaper. More people need access to this tech.
Do not forget to check out the full interview on your favourite platform 👇
That’s it for today
See you next week