What If… Nothing Was Different?

Thoughts on the new “What If…?” app and other immersive experiences for the Vision Pro, and revisiting some old assumptions about interactive storytelling

Today I went through1Watched? Played? :shudder: Experienced? The lack of useful verbs is still a problem when trying to talk about interactive entertainment the new What If…? app from Marvel and ILM for the Vision Pro. It’s an interesting and extremely well-made mash-up of the animated series, some light minigames, and the “immersive” format that Apple is pushing with the visionOS platform.

I think it’s currently one of the best examples of what the platform is capable of.

People more cynical than me could probably dismiss it as just another VR experience, just like they insisted that the Vision Pro is just a fancy VR headset and Apple doesn’t want you to say that! I still think that the differences are subtle, but significant. You could absolutely bring the What If app to another mixed-reality headset, and you could even bring it to a pure VR headset without losing much. But I believe it would feel like an inferior port.

It’s designed to fit in perfectly with how (I think) Apple is positioning their headset. In particular: it’s a seated, “lean back” experience, feeling more like an animated series with interactive elements than a simplified game with extended cut-scenes. It also uses gesture controls as its only interface, having you grab infinity stones, fling objects around, fire magic bolts, hold shields, and open portals using only your hands. (Tying it into Doctor Strange and having your guide be Sorcerer Supreme Wong was an inspired choice).

The art direction is great, and in my opinion these character designs work a lot better in this type of immersive presentation than they do on the series.2I can’t explain why, but it’s always bugged me that every male character on the show has distractingly prominent five o’clock shadow. There’s a perfectly comic-book-inspired style to everything, where all of the 3D characters have hand-drawn (and often 2D-animated) details and effects that remind you of the source material: How to Make 3D Environments The Marvel Way. All of the animated sequences are presented as if they’re on shards of crystal, with a level of dimensionality that’s perfectly restrained and never distracting. Characters will walk around your environment, and occasionally in fully immersive 3D environments, and it feels natural and never gimmicky.

It also reminded me of what I think is the underrated MVP of the MCU: the animated Marvel Studios intro that starts every installation. I never really considered myself much of a Marvel fan before the MCU, but I can still remember the start of Iron Man and how the rapidly-flashing comic book panels perfectly set the mood. In What If, the logo surrounds you, a little like flying through the old HBO Feature Presentation opening, and you’d have to have a cold, hard soul not to be caught up in the excitement of it.

As for the experience in its entirety, I liked it a lot, more than I’d expected to. It’s a great example of elements being combined to make something better than any of them would be individually. As three vignettes in the animated series, it would be good, if inessential. As a VR video game, it’d be disappointingly simplistic. And as an interactive narrative, it’d be interesting if not groundbreaking. But all of the components work well together and fit perfectly within the theme of the series.

On SixColors, Jason Snell wrote a review of the experience for the Vision Pro. One part of Snell’s review stood out to me in particular:

Consider that this app is from two Disney-owned companies, and consider it sort of like an interactive theme park ride. You can do stuff, but you can’t really change the ride.

That is, except for one point in the story, where you’re offered a choice. It’s a real, legitimate “final choice” that results in different endings depending on what you choose. It’s the lightest dollop of branching on a story that otherwise goes in a straight line—clearly the budget of this project was not high enough to create numerous scenes that will only be seen by the fraction of the viewers who make those specific choices—but it’s a fun moment nonetheless.

There are two big assumptions in there, still common to any review of interactive entertainment:

  1. Interactive narratives require choices resulting in substantially different outcomes
  2. Not including branching narratives is a limitation of budget (or at best complexity), not a conscious design choice

I’ve already given my opinion about branching narratives on here, when talking about Telltale’s The Walking Dead first season. Short version: branching narratives definitely aren’t the end goal for interactive entertainment, and I don’t even think they’re a welcome luxury for games with a big enough budget. I say that they’re actually detrimental to good storytelling, and putting so much emphasis on them is dooming interactive entertainment to be nothing more than a novelty.

The real genius of The Walking Dead games was that the stories were broken down to put all of the weight on the choice moments and the events leading up to those decisions, not to their outcomes. That’s where real player agency comes in — the act of making the choice itself, not watching the developers’ interpretation of that choice. The choice is interactivity; the outcome is essentially just a cut-scene.

And a diluted cut-scene at that. A key part of good authorship is making decisions. I think the real art of storytelling is going from “what could happen next?” to crafting what happens next. Any time a developer presents multiple outcomes, they’re essentially failing to commit to any one of them. I think it’s notable that we got one outstanding and one entertaining multiverse movie in the same year — Everything Everywhere All At Once, and Doctor Strange in the Multiverse of Madness — and the overriding theme of both was characters presented with a multiverse of infinite possibilities and realizing the value of the simple, flawed life that they already had.

The exceptions are when the story itself is built around the possibility (or futility) of choice; or when the story isn’t intended to be the focus, and the choices are just there to give more content, like the movie version of Clue.

What If…? would seem like a perfect example of the former; the entire premise is based on choices having effects that ripple out to remix or re-invent the familiar. But that’s not really the case. The “choice” is always made at the beginning of the story, with the premise of the comic issue or the premise of the episode. Everything that follows is the storyteller’s decisions about what would happen as a result of that one change.

That’s the version that this interactive experience goes with. There’s really only one choice presented in the story, you can’t reliably guess at the implications of the two options, and the characters don’t really give you any more guidance than “whatever you choose, we’re behind you!” Once you make the choice — and that moment itself is perfectly done; I was very impressed — then you’re shown one of two possible finales. Both are pretty well thought-out and entertaining, and it’s pretty straightforward to go back and choose the other one to see both.

If it were completely arbitrary, then I’d be a lot less impressed with What If…? overall. But it does manage to tie its vignettes together with a common theme that’s emphasized by that choice, the idea of the consequences inherent in using power, even in the name of making things right. Which ties it in perfectly as a counterpart to Infinity War and Endgame, and of course is a recurring theme in the classic Marvel comics.

There is a moment earlier that feels like a more significant player choice, but as far as I can tell, you’re not actually given the option to not choose to do “the right thing;” you’re just warned that it’ll have consequences. I interpreted it less as a choice moment than foreshadowing for the finale.

It’s all a lot less navel-gazing than I’m making it sound; What If…? has got the tone of the series just right, and it ties the story together with just enough weight to make everything feel substantial.

I think it’s a really good model for interactive storytelling — at least on a platform like the Vision Pro — because it understands how “immersion” doesn’t come from any one thing, but from careful combination of a bunch of disparate elements.

It’s not essential for the action to start out in your room, but it adds a lot to have The Watcher and Wong leaving the story and entering your space. And it adds even more when you see the title “EARTH PRESENT DAY” superimposed over your actual space. I already said that I wasn’t crazy about the various minigame portions of the experience, but they are important for pacing and keeping the story moving forward. And there’s no real “wow” moment as you’re brought into one of the animated vignettes (or rather, the vignette expands to surround you) — at least, if you’ve played any VR experience before — but it does drive home the feeling that you’ve shifted from being a watcher to a participant.

On the whole, it’s got me excited again about the potential for “immersive” entertainment to be more than just games or 3D movies. If developers aren’t chasing the wrong thing — in my opinion, branching narratives, or the action/cut-scene/action formula of most story-driven AAA games — then it can genuinely add layers to the experience. Instead of just piling a bunch of gimmicky novelties onto traditional movies and television, it can be adding additional channels of communication and presentation.

Speaking of that, I tried a couple of the other “immersive” experiences on the Vision Pro that I’ve got a few brief thoughts about.

One was a new episode in Apple’s Immersive series, the first of which interviewed a woman as she walked a tight rope across two sides of a steep cliff face. The new episode follows three athletes doing parkour across Paris, culminating in a roof jump near the city center.

I admit that I still think parkour is kind of silly, but it greatly benefited from an immersive presentation. What impressed me most of all was how many of the “rules” of VR and immersive presentation it seemed to break. One was that it frequently froze one of the athletes in mid-jump as they said something significant in voice-over, kind of like the record-scratch “you’re probably wondering how I got here” gimmick that was overused in the early 2000s. I thought it worked really well here, largely because it’s still so unusual to see the entire world pause around you. It ends up working like the technique originally did, before it became such a cliche, putting all of the emphasis on the character, their words, and their action, all at the same time.

Another was that it would often cut to a completely different camera set-up, with none of our main “characters” visible on screen. You’d end up having to search the scene for them, before catching them running out of a corner of the scene before jumping off a wall or something. It’s something that would be distractingly disorienting in traditional filmmaking, but here it added a sense of dynamism: it reinforced the idea that these guys are always in motion, making their way across the city and taking us with them. (And hiding the reality that there was likely a lengthy and laborious camera set-up process before each cut).

The other experience I tried was the Gucci app, something I wouldn’t have tried if not for Andru Edwards’s YouTube channel. This is 100% a marketing app, so I don’t want to oversell it as the future of interactive storytelling or anything. But it does contain a documentary about one of Gucci’s designers preparing for a fashion show, and I’m impressed that they took such a big swing with it.

The documentary itself is presented as a video floating in the middle of your room, like most Vision Pro video. But throughout, elements appear surrounding the screen, behind the screen, filling your space to accentuate whatever’s going on. Words pop out of the screen. Train tickets fill the room, and a 3D train car runs behind the screen. Parts of the show, like bags, will pop out and let you examine them more closely. When they talk about the designer’s cherished dog, the dog walks across your floor in front of the screen.

It could seem gratuitous until you realize that I am 1000% not in the target audience for such a documentary, and I was still completely engrossed.

On the whole, it reminds me of the “golden age” of web design, before all the standards of “good” design had calcified, and people were trying a little bit of anything to see what worked. You can’t help but wonder how designers will build and expand on all of this, and really explore what it takes to make traditional entertainment “immersive.” And I can’t help but be excited again, about Apple’s attempts to drag this stuff out of the niche audience of VR gaming fans and into the mainstream.

  • 1
    Watched? Played? :shudder: Experienced? The lack of useful verbs is still a problem when trying to talk about interactive entertainment
  • 2
    I can’t explain why, but it’s always bugged me that every male character on the show has distractingly prominent five o’clock shadow.

Leave a Reply

Your email address will not be published. Required fields are marked *