Read the Room, Apple

Trying to make sense of Apple’s plans for the Vision Pro, while simultaneously trying to talk myself out of wanting one

I was innocently watching YouTube when I happened upon a clickbaity video warning that Apple Vision Pro has a PROBLEM, and I was powerless not to click on it. Inside, a man was furiously screaming that the company had limited the VR experience to ten feet by teen feet and you had to be sitting on an [expletive deleted]1I promised my mom I’d stop swearing so much in public. couch.

My third response (after “why did I click on that?” and “take it down a few notches my dude”) was that he must be mistaken. He must’ve been taken in by a rumor, or maybe misinterpreted the public documentation.

But then I found an article by Samuel Axon on Ars Technica from last June, confirming that the documentation explicitly says that a VR experience (“fully immersive experience” in Apple’s retina-means-high-resolutionspeak) will be interrupted if the user moves more than 1.5 meters away from their starting point. In other words: the Apple Vision Pro won’t support room-scale VR.

Quick aside for anybody who’s unfamiliar with the terminology: “room-scale VR” just means that you can walk around your own space to move around the virtual space. Other types are seated (on your #@$%&! couch or otherwise) or stationary (standing but not moving from your starting position). All of the current major consumer-level VR headsets support room-scale tracking.

It’s entirely likely that I’d already heard this, and either misinterpreted it myself, or understood it and completely forgot about it. I’ve spent the time since then assuming that of course it must support room-scale tracking, since the device seems entirely capable from a technological standpoint. AR tracking on the last few models of iPhone — which aren’t purpose-built AR devices — is excellent, and you can place a virtual object in space, walk around the room a bit, and return to find it still sitting where you left it.

That Ars Technica article says, correctly, that the “limitation” shouldn’t come as any surprise to anyone who’s been paying attention to how the company is trying to position the device2No pun intended. Apple’s been insistent that this isn’t a VR headset; in fact, they refuse to use the industry standard terms like “augmented reality,” “virtual reality,” or even “mixed” or “extended” reality, in favor of their own “spatial computing.”

The general idea is that the Vision Pro is meant to enhance and extend the way you use Macs and iPads already — watching TV and movies, looking through photos and video, browsing the web, telecommunications, and I guess making keynote presentations? They’re emphasizing that this isn’t some entirely unfamiliar type of computer; it’s the same stuff you’re already doing, but bigger and in 3D.

And that part makes sense. A lot of what they’ve been trying to do is essentially counter-programming, undoing people’s preconceived notions of what a head-mounted display is, based on what they’ve seen from Oculus and others. They’re adamant that you’re not putting on blinders; the video passthrough has to be as flawless as possible. They insist that it’s not isolating; you can see everyone and everything around you. They insist that it’s not just a toy or a video game console, but fits squarely in the “lifestyle device” as all of their other products. And they insist that you should look as creepy AF while you’re wearing it.

I’m in general agreement with that Ars Technica article, and only in mild disagreement with its conclusion:

But I do know two things for sure: Vision Pro is not going to offer the same sorts of experiences as a Meta Quest or a Valve Index, and that’s more good than bad for Apple’s prospects, given the hard market limits those devices appear to have run into.

By every account I’ve seen, the Quest 2 has been a huge success for its company, and it’s outsold every other VR headset by far. And it seems reasonable to assume that Apple would need the Vision line to exceed that in order to justify the investment they’ve made in it. You can see why they’d want to target the device at people who have no interest in Beat Saber or Superhot — or really, the people who have no idea what those two titles even are.

And even with Apple’s messaging — showing families interacting with each other, people sitting on a couch watching videos or looking at photos, one image after the other presenting the Vision Pro as social and comfortable — the complaints (apart from the price, obviously) have still been that it will feel weird and isolating. So the company has clearly done their research to predict how people feel about HMDs, and they’ve got their work cut out for them to convince people otherwise.

Where I (mildly) disagree with Axon in his Ars Technica article, though, is in the implication that room-scale VR is such a niche that it’s not in Apple’s best interest to pursue.

My mistaken assumption had been that the Vision Pro would be a superset of existing consumer-level headsets: it’d support all of the stuff that a Quest 2 can do, and also all of the stuff that’s in Apple’s marketing images. And because the AR component is so well done, that it’d be a no-brainer for nerds like me, who wanted the next generation of HMD to be essentially “backwards compatible.”

Knowing that the VR experience is limited by design — and again, I’d be shocked if it’s a technical limitation — dumps a $3500 bucket of cold water on my enthusiasm for it. So far, I’ve been so impressed by what I’ve seen that I’d all but committed to being an early adopter. Yes, it’s absurdly expensive, but I’d been thinking of it as a once-in-a-lifetime chance to get in at the start of the next generation of computing paradigms. (Yes, my thought process was indeed that pretentious).

Apple got so much right with the AR/”spatial computing” part of the experience. The eye tracking is uncanny, as is the gesture recognition. The light estimation — where virtual objects appear to be lit by the same light sources as everything else in your room — was better than anything I’ve seen before. The demos they showed in their initial announcement video were criticized for being obviously fake, but I had a hard time spotting anything that couldn’t be done by even an intermediate developer with the SDK and/or Unity. It seemed genuinely revolutionary. And even the high price seemed roughly in line with the cost of an Apple laptop and a huge 3D monitor.

But I’ve also been having trouble thinking of a must-have use case for it. I have little doubt I’d be impressed with all of the activities shown in their marketing material; I have little doubt I’d find it magical just to browse through apps on the damn thing. And that was even before they showed someone having the Main Street Electrical Parade on their table, or watching The Mandalorian from inside a landspeeder, which left me feeling almost unfairly targeted. But as for long-term usefulness, I’d been assuming that I could do all of the AR stuff on top of having a top-of-the-line VR headset.

One thing that Axon is absolutely, 1000% correct about: it’s never, ever a good idea to bet against Apple. They haven’t always met with guaranteed success, but more often than not, they turn products from seemingly useless novelties into ubiquitous, essential devices. Even as a shameless and unrepentant Apple fan, I’ve been skeptical that the watch, AirPods, or even iPads would become more than niche products, and I’ve been surprised over and over again.

So my hesitancy doesn’t come from believing that Apple will fail with the Vision line (Pro and, presumably, onwards). It’s from believing that the feature set will stabilize and improve over the next few iterations. If the rule is “Apple doesn’t do it first, but they do it right,” you have to add an asterisk with “eventually, after a few iterations.” My first iPhone only supported web apps and had no App Store. My first Apple Watch gave me the never-used ability to send my heartbeat to a loved one. My first five or so iPads didn’t let me use a stylus for input, which is basically all I use it for at this point.

I realize that basically, this is a variation on the ages-old complaint that Apple doesn’t care enough about video games to make the Mac competitive as gaming machine. They seem to be committed to Apple Arcade, at least, but of course, that’s still iOS-centric, and nobody’s unclear on the fact that games on iPhones and iPads are a much bigger market than on Macs. It seems like they’re trying yet again to cater to game developers, but I think the damage is already done, and it’s irreparable: developers don’t make games for the Mac because nobody plays games on the Mac because there are no games being made for the Mac, etc. etc. All the way back to the time when PC gaming started to take off, and the decision-makers looked at the numbers and said, “Nah.”

I suspect that, assuming the Vision Pro is enough of a success to keep the product line afloat, we’ll eventually see a gradual introduction of software and hardware that is more “Quest-like.” Apple might allow for tracked controllers, made by a third party, to augment the system-level hand tracking.3In the only “gaming” demos I’ve seen so far, they’ve just shown a 2D game playing on a big virtual screen, with the player using a standard Bluetooth controller. Apple might eventually allow users to have “safety settings” which allow for movement when in VR.

Meanwhile, the Quest 3 does almost all the stuff I want from the current generation of VR headsets; the passthrough, while nowhere near as good as the Vision Pro’s, looks to be at least finally somewhat usable; and it’s one seventh the price of the first Vision Pro headset. I’m wondering if I should just throw yet more money Mark Zuckerberg’s way and voluntarily sit out the first iteration of Apple’s headset, like a sane person.

(One thing that only occurred to me as I was writing this: I’ve been assuming that detecting your play area when in VR, and preventing you from smacking into stuff, was a “solved problem” only because I’ve seen how the Quest and HTC’s headsets handle it. But it didn’t occur to me that there might be patents involved, which limit how Apple can handle objects or the player moving out of the safe area?)

In any case, if these changes come to the Vision platform in future iterations, I believe it’ll be too late. While the VR market is, admittedly, probably little more than a rounding error when you’re on the scale of Apple, it’s still not zero. But developers won’t make stuff for a platform unless the audience is there, and the audience won’t be there unless you support them from the start.

Again, I could be completely wrong, as I so often am when I try to second-guess a hugely successful global corporation. But it seems to me that Apple’s always had a frustrating blind spot when it comes to anything gaming-related4And mice, too. How come the company that brought the computer mouse into widespread mainstream use can’t seem to design one that isn’t total shit?, and I’m concerned that they’re artificially limiting the potential of an entirely new platform before it’s even had a chance to get started.

  • 1
    I promised my mom I’d stop swearing so much in public.
  • 2
    No pun intended
  • 3
    In the only “gaming” demos I’ve seen so far, they’ve just shown a 2D game playing on a big virtual screen, with the player using a standard Bluetooth controller.
  • 4
    And mice, too. How come the company that brought the computer mouse into widespread mainstream use can’t seem to design one that isn’t total shit?

9 thoughts on “Read the Room, Apple”

  1. “but I’d been thinking of it as a once-in-a-lifetime chance to get in at the start of the next generation of computing paradigms”

    Sometimes your humor is dry enough that it goes right over my head. I’m assuming that’s the case here, ‘cause this is like the fifth or sixth wave of tech revolution in our adult lifetimes that has been potentially “rideable”.

    But that said, I’m getting one for that exact reason. So…?

    1. No, that was me being completely sincere for once. I meant “once in a lifetime” more as specifically in terms for how ubiquitous AR and 3D displays could change things, to the scale of iPads and wireless headphones if not necessarily as huge a shift as the iPhone.

  2. I suspect one big reason why Apple markets the Vision Pro as a static experience is to minimize a situation where a user Mr. Magoo’s their way to injury or death. I’m sure that will happen because folks want to get their YouTube views and subs, but at least this first round will be folks going against the stated use parameters of the device.

    1. I don’t doubt it! But I still think it’s even more telling that Apple’s marketing of the device looks absolutely nothing like the marketing images of past headsets. I think they’ve gone all in on selling this not as “like a Quest, but better,” but “like an iPad, but 3D” and are only barely interested in stationary or standing VR as an afterthought.

  3. It’s the AR and social/connected-focused stuff that I find intriguing about Vision Pro versus competitors. The lack of support for “full room VR” is something I find to be a fascinating vision statement (pun intended), and I kind of want to see how hard Apple pushes this one. Not everyone has empty rooms for full room VR at home and an AR focus is an accessibility focus. To some respects, IF Apple can work to hold onto its principles here (and I don’t know any other vendor that could; Microsoft tried to some extent with HoloLens, but that’s also why Microsoft never had the guts to see HoloLens as a consumer product); IF Apple can stick to this: it can send a message to game developers that they need to pay attention to our cluttered lives and environments, and to the people around us.

    That isn’t to say that I don’t find value in full room VR and clearly there’s a market there between Valve and Occulus sales figures. I just think that there’s a larger audience with cluttered rooms and roommates and pets and it should be a challenge to game developers especially to build immersive experiences without full immersion graphics. Some of my most immersive experiences were text based games, this could be a good challenge!

    Many of us the only way we will likely get to experience full room VR is places where we can rent empty rooms by the hour. That’s fun and there’s a niche for that. That’s not accessible in our homes. (That’s not accessible to a diverse cross-section of humanity when you throw in all the differently abled we are physically, including but not limited to the motion sickness many feel in full room VR.)

    I admire Apple seeming to be trying to make a political statement with this limitation that certainly isn’t a technical limitation. I hope it doesn’t fall on deaf ears. As a developer myself, I understand there’s a lot of cool excitement to build the ultimate immersive laser tag experience, but a lot more people are going to have a chance to play whatever the AR/MR equivalents to Angry Birds will be that makes use of the weird location of your bed in your apartment and the way your sofa blocks half the room and the way your roommate likes to stand by the fridge and mock you when you game. I think Apple sees that and is challenging developers to think about that. That’s something I find exciting about the Vision Pro in ways that I appreciate but am not excited about the latest Quest or Index or whatever else in this space.

    1. But my main point is that the two don’t need to be mutually exclusive. The Quest forces you to either map out a safe area, or confirm that you’re going to be stationary, before you can get started (or at least the Quest 2 does; I haven’t yet tried the 3). The Vision Pro isn’t going to do that, so you should be able to use it without having to clear any space.

      But it’s still a VR headset, though; it just uses your physical environment as its virtual environment. So I can’t think of any technical limitations to making room-scale VR work for the people who want it. That’s no more a violation of “principles” than letting apps on a Mac run full-screen even though windowed apps should be perfectly fine for anyone.

      I’ve seen case after case of companies — Apple and Disney in particular — insisting on their “vision” as the one true way to do something, and it’s always praised for giving a focus that lets them excel, but it never survives long in contact with reality. Just one examples the iPad was never intended to use a stylus as its primary input, which was always touted as Apple staying true to their principles… until they managed to make one that was really good, at which point they pivoted to saying it’s an all-but-necessary add-on.

      1. I agree they aren’t mutually exclusive and I don’t think Apple is doing it for technical reasons. At least I guess they aren’t.

        I think it is a marketing decision. I know a lot of people that see full immersion VR and immediately check out with a “that doesn’t sound like it is for me” but mostly open/social AR has more appeal and I’m guessing Apple has seen similar anecdata and maybe even has some focus testing data.

        Maybe it is an over-correction that will get fixed in later versions. Maybe it is like the stylus thing and some last piece is still in R&D waiting for its “we’ve done it better than anyone before” one last thing moment. Maybe it is because there’s some other sleeved card: I know people hoping that the eventual “Air” version that “Pro” implies is lightweight because it is more “wave guide based” like HoloLens or Magic Leap and full room immersion will be much more difficult; I don’t know if I believe that, but it’s interesting theory. I’ve got my own crazy theory that phones and tablets play a bigger role in their plans than they’ve talked about so far, and the social stuff is a larger play long term than currently obvious.

        Whatever the marketing reasons, I think it is the right choice for Apple to differentiate their product in the market for the current moment and zeitgeist even if a card doesn’t drop from their sleeve and if they do correct back towards full room VR even without some crazy R&D invention.

        It certainly is a good reason to skip the first version to find out if there is something up the sleeve or to see if they correct course here.

        I find it interesting that it makes me more interested in playing with the Vision Pro in its first version. I want to see what devs do when challenged by the constraints of the device, whether they are ultimately technical or marketing/political. Though I say that and I also don’t really expect the first Mac Book I ever purchase to be a head mounted one and “more interested” is still not yet above “that’s an interesting novelty” so far for me.

        1. I feel like a lot of people were surprised and/or disappointed to see the Vision Pro is essentially a VR device doing AR, instead of going the HoloLens and Magic Leap route. But I think it’s undoubtedly the correct approach, not just for the current technology but for any future version. In other words: even if they did have some unprecedented waveguide implementation waiting in the wings, it would be an inherently different product, and not just swapping one display technology for another.

          I think the greatest achievement in what I’ve seen of the Vision Pro is the near-perfect (IMO) passthrough; it all but obviates even bothering with any approach that tries to overlay virtual elements on your actual view. I’ve got limited experience with the Magic Leap and variations on the HoloLens approach, and I have a MUCH harder time imagining that it will be fundamentally improved over the next ten years, than I do with the potential for smaller, thinner, and lighter displays and the tech to drive them.

          So that assumption is the basis for what I imagine as the future of the platform: it can, will, and should drive more interested AR applications, but there’s no hard limitation on full VR apart from Apple’s somewhat arbitrary guidelines.

          I absolutely agree that it’s a primarily marketing-driven decision, trying to encourage both developers and users to think in terms of mixed reality instead of just porting popular Quest games to the platform. But I worry that it artificially limits the platform, the same way I’ve seen Apple do over and over again. I wish they could just acknowledge what the device will look like after a few iterations, and lean into that instead of vainly trying to prevent it.

          And obviously, I don’t want to be spending that much money on a device that will be hamstrung by not having the capabilities that everybody knows are inevitably coming in future versions. Practically, I know that these are targeted at developers and at the super-rich, and not for people looking for long-term use, but that doesn’t make me want one any less.

          1. I never got a chance to play with a Magic Leap, but I was very impressed with the time I spent with the HoloLens. My understanding is the “wave guide” stuff has known ways to improve, but is already lighter than OLEDs and can be thinner in some interesting ways. I heard that today the problems with “wave guide” are cost and especially “processing cost” (they can in theory do crazy things in tight spaces at ridiculously fine grain [“better than retina”] if you’ve got the raytracing computation power, in theory; none of today’s vendors have proven that potential) rather than weight/thinness.

            The HoloLens has a reputation as a heavy device but that is as much that Microsoft went with a heavier Intel PC because the Windows ARM side isn’t yet as strong for the processing needs of HoloLens as they need it. (Not a huge surprise after the death of Windows Phone.) Microsoft also put the battery on your head instead of resorting to the “backpack”/“pocket pack” compromise Apple is using with the Vision Pro, which also a lot of the weight and bulk. I

            That said, I’m still not convinced “wave guide” is Apple’s actual plan here, I find it an interesting theory, but not my favorite theory.

            I think I somewhat agree that it would feel like more than just a change of “display technology” if they did switch to something like that, but maybe for different reasons because I feel like I have more optimism for that kind of technology. Especially from friends that claim the thing holding back Wave Guide is real time processing power more than hardware innovations, there is an interesting optimism there, especially because if there is a hardware company with the processing prowess in cheap, lightweight machines it is Apple. I don’t know if ten years is a realistic timeline or not, though. I don’t track this all that closely, I just find it interesting.

            I think that switching device classes like that but calling it the same device is definitely in the Apple playbook though. The Macintosh has switched processor architectures alone that it is an incredible Ship of Theseus.

            But again, I don’t know if I believe yet that is what they will do.

            I also selfishly wish Apple wasn’t playing its cards seemingly so close to its vest here, though mostly out of curiosity rather than directly informing a buying decision right now. I really want to know what Apple thinks it can do that Microsoft and Magic Leap failed to do on the AR side. I really want more of an idea of why they’ve dismissed VR as a market they aren’t interested in short term and maybe not long term either, especially given Quest sales figures right now. It would be nice to have an official roadmap and not just trying to read tea leaves and between the lines of keynote presentation reveals.

Comments are closed.