If you see a virtual joystick, they blew it

Killer apps, creativity, and how bad things happen to good computer platforms

This is a tangent off of my previous post about Apple’s apparent plans for the Vision Pro, and some of the follow-up comments.

Back when Apple released the iPhone, it was quickly apparent that it had tons more potential than was realized in its first iteration. The company had nailed the design, and now millions of people had portable touch screens packed full of sensors, a camera, and an internet connection. As a game developer, I was excited at the prospect of entirely new types of games that would be made possible by the technology in this device.

And there were a few games that took full advantage of it. Flight Control is still the standout; it felt as if it would only work on a touch screen, and only on a touch screen of that size.1Which I think is still the case. I tried the VR version, which seemed like a no-brainer until I actually played it, and discovered that the magic wasn’t there for me. Before the Match 3 genre got milked dry and became synonymous with exploitative monetization strategies, it was pretty novel: Bejeweled was a lot of fun, and it still works best with touch input. Device 6 doesn’t depend on tech demo-like game mechanics, but it’s a story-driven game that feels as if it can only work on a smartphone.

But it also didn’t take long for developers to fall back to one of my most hated things in mobile games: the virtual joystick. Whenever I see one2Or worse, have had to implement one, it just feels like the devs have shrugged and said, “that’s it, we’re out of ideas.” It’s not just that it throws out everything that makes the platform unique, in favor of a much older and more familiar interface; it’s that it’s a shittier version of that interface as well.3I should mention that the screenshot attached to this post is from an Apple developer presentation explaining how virtual joysticks can be a fallback for players with accessibility issues, or if a physical bluetooth controller isn’t available. So I’m not necessarily complaining about that presentation in particular.

And yet, using the Apple Pencil with an iPad feels so natural and just plain enjoyable that it’s become my preferred way of interacting with it. Even though the company had repeatedly insisted that the device was specifically designed for touch input, which is why iOS and MacOS were kept separate, and why the company had never developed its own stylus among all of the third-party options.

So it would seem that it’s a good thing for a company to insist that its products be used the way they’re designed and intended to be used, where the unique abilities and constraints of the platform encourage new ways to solve problems and sometimes invent entirely new categories. Except for the cases where it’s not a good thing.

Of course, I’m leaving out a lot of the context from the iPad example. When Steve Jobs gave his quote, “if you see a stylus, they blew it,” — which pundits loved calling attention to once Apple announced the Pencil — it was in service of differentiating the iPad from other tablet computers. Microsoft in particular had spent years pushing Windows as a platform that could run on any form factor, as if an OS that was designed for a mouse-cursor precision and right-clicking or double-clicking could seamlessly work with human sausage fingers.

But I feel like it’s a pattern that keeps coming up over and over again: designers insisting the purity of their vision, and then having to watch with horror as their vision is corrupted by human beings actually wanting to use the damn things.

Take Instagram, for instance. When it started, it was packed full of constraints to keep it focused purely on social snapshots: square photos only, taken within the app instead of from your camera roll, no video, no links, no reposts, no way to post text unless it was accompanying a photo. The constraints were severe, but in my opinion thoughtful. They kept it more spontaneous than more photography-oriented sites like Flickr, but more civil and social than text-heavy apps like Twitter and Facebook, since you had to go through the process of taking an original photo before you posted. The constraints didn’t last long, though, as long-term users who’d built a community were crying out for ways to be shown more ads, more short-form videos taken from other sites, and algorithmically-generated spam. Luckily, Facebook was willing to compromise the original vision of the app in order to give the people what they clearly wanted.

It’s not just computers; it comes up a lot with Disney parks as well. Epcot was pitched as being more like a World’s Fair than a theme park, and the company spent quite some time trying their hardest to keep it untainted by existing IP. But that earned it a reputation for being dry edutainment, and guests complained that they’d spent a lot of time and money on a Disney park that didn’t feel like what they’d expected from Disney. Purists still complain that bringing existing licenses into Epcot is blasphemous.

And they went through a similar cycle again with Animal Kingdom, which still retains a lot of its originality, but feels very different from when it opened. When I spend a day in the park now, I feel like I’ve spent a fun day in a phenomenally detailed theme park full of live animals, instead of feeling like I spent a day walking in 100 degree heat, perpetually worried I’m going to spill my lid-less soda everywhere, and getting lectured about how I’m not doing enough to stop poaching.

But back to my point, such as it is: if you’re in the business of selling things and trying to push things forward, it requires a balance. If you try too hard to maintain a stranglehold on how people use your device, you’re damning it to irrelevance. But if you don’t have enough focus and confidence on your vision, then you’re leaving the future of it to the kind of people who’d see a state-of-the-art touchscreen and immediately try to put a joystick on it.

Since the time I started writing this, Apple announced that their Vision Pro headset is going to be available for preorder at the end of the next week, and it will start shipping in early February. Which means the pressure is on for me to figure out whether I want to get in from the start, while the platform is still in its “purity of our design intent” phase, or try to be a grown-up and wait until it’s had some time in the world outside of Apple Park, losing its innocence and becoming jaded and world-weary.

Today on Mastodon I saw a tangentially-related thread from Christina Warren, who insists that the Vision Pro that’s being released in February is essentially a dev kit that Apple is trying to market as if it were a consumer-ready device. “You cannot be both, and I cannot stress this enough.”

I don’t have anything useful to add to the thread, but I do have an opinion, which is that I don’t quite agree. It’s not really just a dev kit, because that implies that it’s missing functionality that would be intended for a consumer version. But by all accounts, the Vision Pro will be able to do everything that’s in their marketing images.4Which is partly why Disney’s part of the presentation was specifically prefaced as an idea of what might be possible on the device, or similar language.

I believe that the problem is that most people, myself included, have a hard time mapping that level of functionality to a device that costs that much. You can kind of equate it to a top-of-the-line iPad combined with an even more top-of-the-line 3D monitor. But in my opinion, Apple has been so committed to positioning it as “familiar, but 3D” that they haven’t suggested what is going to be the “killer app” for it. “Watch Avatar on a really huge 3D screen” isn’t that.

To me, it feels most like the first iPhone release. The initial release was extremely expensive — not $3500 expensive, but way higher than anybody expected from a cell phone at the time — and limited to Apple’s built-in apps. You had to be sold on the potential of the thing, as the start of something genuinely new and novel. But even though third-party apps were still a couple of iterations away, it did everything that Apple claimed it could do, with several features better than other cell phone and even smart phone manufacturers were doing, and it was a solid phone.

The high cost, and presumably limited supply, of the Vision Pro is going to inherently limit it to developers and early adopters. I don’t think the early adopters are going to be left with a useless brick, though, because I don’t think anyone at any level expects the first version of the device to be the best and most useful one. Even if you’re not actively developing for it — say, hypothetically, you’re some chucklehead who’s been paying for an Apple Developer account since around 2010, and you still have yet to release an app — you’re buying into the potential of it. It’s functional now, and I want to see how and if it becomes something amazing.

Speaking of chuckleheads: I’m still going back and forth over whether I want to be one of those early adopters.5And, I hope it goes without saying, extremely grateful that I’m at a point in my life where I’m fortunate enough to even entertain the idea. On the one hand: I’ve bought the first iterations of the iPhone, iPad, Apple Watch, Apple TV, Air Pods, and Mac mini, and every time I’ve had more than a little bit of regret when they released a newer version that seemed to punish me for my impatience. But on the other hand: I’ve got a streak going, so why stop now?

I should probably mention that I probably wouldn’t be as tempted based on Apple’s marketing alone; I’ve been able to use one in a controlled environment, and I was so impressed by it that I genuinely felt like I was seeing the future. It reminded me of the first time I got to try the HTC Vive headset with Valve’s demos, where it wasn’t just the technology itself that was impressive, but that it was being used in exactly the right way. I haven’t seen anything on the Vision Pro that was as creative as that — the “killer app” that everyone is waiting for — but it does have the Apple advantage where the hardware and software are designed together to such a degree as to be inseparable.

Maybe the “dev kit” feeling comes from the fact that there’s still that piece missing, the one demo or example that makes not just the headset but the entire platform feel like a must-have. The Wii Sports of Apple Vision Pro. Whatever form it takes, it needs to be more compelling than flipping through photos — even panoramic ones filling your living room, although I’m sure they’re impressive — or semi-3D videos of your family you recorded while squatting on a beach. I hope it’s something novel and genre-defining, and not just some dev using the excellent hand tracking to turn my thumbs into virtual joysticks.

  • 1
    Which I think is still the case. I tried the VR version, which seemed like a no-brainer until I actually played it, and discovered that the magic wasn’t there for me.
  • 2
    Or worse, have had to implement one
  • 3
    I should mention that the screenshot attached to this post is from an Apple developer presentation explaining how virtual joysticks can be a fallback for players with accessibility issues, or if a physical bluetooth controller isn’t available. So I’m not necessarily complaining about that presentation in particular.
  • 4
    Which is partly why Disney’s part of the presentation was specifically prefaced as an idea of what might be possible on the device, or similar language.
  • 5
    And, I hope it goes without saying, extremely grateful that I’m at a point in my life where I’m fortunate enough to even entertain the idea.