The Impact of Spatial Computing

The biggest story of the first half of this year has to be Apple's Vision Pro launch. Virtual and augmented reality have been buzzwords for years now, but once this Vision Pro headset actually shipped, it essentially redefined spatial computing from the ground up.

The term "spatial computing" actually predates 1985, though most people cite the definition Simon Greenwold put forward at MIT in 2003. That paper was written before smartphones even existed. I tracked it down and read it — it's genuinely brilliant.

Right up front, he defines it: "I define Spatial Computing as human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces." His core argument was that something like the ancient abacus fused data with physical form, but the electronic age had severed computation from physical space entirely. He believed it was possible to build computing systems that interact with the real physical world. The whole paper is essentially a thought experiment on whether human-machine interaction in space is achievable, and reading it felt like paging through vintage science fiction. Twenty years after that paper proposed spatial computing, our VR headsets still haven't quite lived up to the promise. I've bought a lot of them — as a nerd who's an easy mark for a shiny new concept, I've owned devices from HTC, Meta, and PlayStation. Everyone warned me before each purchase that it would end up gathering dust in a cabinet, and sure enough, every single one has. The main culprits are still software compatibility headaches and a thin library of usable apps — never mind the resolution, which has its own well-worn list of complaints.

VR Hardware Has Peaked — Software Is Still the Bottleneck

But Vision Pro is, by any measure, where VR and spatial computing currently peaks. People who've tried it describe the gap between it and earlier headsets as the difference between a bicycle and an airplane. What Apple does best is this: on major computing-platform products, the company almost never misses.

Look at the tablet, which turned out just fine. Plenty of people were skeptical of it early on too — critics said the iPad was neither fish nor fowl. But tablets have become a genuine, standalone category. They're not laptops, and they're not phones, but the category is real and solidly established; I probably spend nearly as much time on my tablet as I do on my computer.

The complaint I hear most often right now is that the software just isn't there yet. But compatibility and a thin app library aren't really a problem for Apple specifically — Apple only needs you to be compatible with Apple, and developers, smelling opportunity, will fall in line. Apple functions as the world's mass-deployment engine for new technology; all it takes is Apple's willingness to absorb the rollout and education costs of something new.

That's actually why Meta and HTC aren't too pessimistic right now — they can play the same role Android plays against the iPhone. What happened with smartphones looks set to repeat itself with spatial computing devices: whatever Apple can build, a cheaper Android-style equivalent will eventually follow. The only real question is which category this ends up being — a phone, something everyone eventually needs one of, or a tablet, something you buy once you have spare cash.

One branch of spatial computing already in wide use is immersive FPV drones (also called "through-goggles" drones), which have even been deployed in the Russia-Ukraine war. Reportedly they're highly effective — and while none of us wants to see the technology used for war, it shows that even mediocre-resolution immersive tech meaningfully sharpens human spatial perception. Apple's version, which links directly into augmented reality and lets you interact with the physical world, opens up an exponentially larger playground of use cases. You don't even need to get creative — worst case, you could just make Pokémon physically appear in your living room.

The device's raw capability already outstrips anything the current market knows what to do with, which is why plenty of commentators say even Apple doesn't quite know what Vision Pro is for yet. But that's not really a problem for Apple, because its partners will figure it out for them — the same way App Store developers did. When Apple designed the original iPhone, it had no idea the gyroscope would end up being used for games either.

I'm more optimistic about this category than most analysts. My view is that price stops mattering once a device crosses a certain threshold of genuine usefulness — no matter how expensive it is. Before the iPhone, feature phones had already gotten cheap. Then the iPhone arrived, flagship prices settled at a permanently high level, and people kept buying anyway, because phones had simply become too essential to skip. If spatial technology lets us bring back three-dimensional memories of our kids and family, I think plenty of people will pay up for it. Seeing your late grandmother's warm smile again — isn't that worth $3,000?

Mapping the Inside of Your Home Reopens the Privacy Debate

There's also a real privacy question, because this technology needs precise spatial data about your private space — your home. Think about it: your home's physical layout, and you along with it, can be reconstructed as an image. Whatever privacy concerns already exist today could get a hundred times more serious. Apple has built in a lot of safeguards, but the device still has, at minimum, basic spatial-recognition capability. If it detects your TV's screen size, your sofa's dimensions, or the fact that there's a crib in the room — isn't that exactly the kind of precise data advertisers would love to have? So this debate is already underway.

Right now we're at the stage where the hardware is ready but nobody quite knows what to do with it yet, so a lot of people are just speculating. One of the more developed use cases is cooking along with an on-screen recipe, which seems to work reasonably well. The device also supports Microsoft's Office apps, so if what you want is the feeling of your own private office, that's achievable too — though picturing a room full of people gesturing at thin air is admittedly a strange image.

Here's another interesting question: what happens to social etiquette? If everyone starts wearing Vision Pro to family dinners, hangouts with friends, or on the subway, does that count as rude?