On the Microsoft HoloLens

Microsoft’s big surprise

I want to talk about a big tech story from last week, the surprise unveiling of the Microsoft HoloLens Augmented Reality headset. This new product was revealed in the course of the launch of the Windows 10 Technical Preview. Probably, some of those reading this have seen the promo video:


This is pretty amazing, and completely out of nowhere. There was no advance sign that Microsoft was announcing this. The project, from the same inventor as the Kinect sensor, was a complete secret until the big reveal this past week. This pleasant surprise has had the tech world abuzz ever since.

Now, a word of caution. Yes, this video is a Microsoft promo, with an artistic representation of how its supposed to work. This is a product that is still, as far as anyone knows, in pre-production prototype stage. There is no release date; it may not even be out this year.

But still. From what I’ve seen from other sources, this is the most impressive and mindblowing thing to come along in a long time. This is next level iPhone and iPad big, one of the next steps in the evolution of the computer.

Why this is a very big deal

This sounds like hyperbole, and certainly, Microsoft has had products before that looked great in an early demo but it didn’t pan out. But generally I have a pretty good eye for this sort of thing.

In 2008 was about the only person in my office with a smart phone. My first cell phone. It was the reveal of the iPhone 3G that showed me the time had arrived. I didn’t buy an iPhone, but that was what showed that this technology had arrived. I had stayed out of cell phones to that point, because I was waiting for the technology to get to a certain point where it can become an indispensable all-in-one computer tool that fits in your pocket and connects you to the internet and lets you do everything – take notes, read/write email, read web articles, internet chat, play games, do light office tasks, etc.

A few years later, in 2010, Apple released the iPad, the first one. I understood immediately that this was going to be huge. So for the first and only time, I waited in line on launch day at the Apple store. That first iPad had a lot of limitations (no camera, no multi-tasking, no folders, kind of chunky – and you couldn’t even copy-paste!), but I loved it for its obvious potential.

I’ve been on Netflix since the initial beta in Canada. That was a tough sell back then. The selection was pretty lousy in Canada at that point, though you could see it grow month by month. Now something like 1/6 of Canadian residents has a Netflix account. 

This new product from Microsoft feels like the same sort of up and coming thing. This is literally science fiction type stuff that I previously would have considered maybe possible within  the coming decade. But this looks like it will soon be available. The world moves so fast these days.

This product fits in amongst the ongoing paradigm shift toward Natural User Interfaces (NUI) that I’ve spoken about previously.  The world of computing went through Command Line Interfaces (CLI), Graphical User Interfaces (GUI), and now, Natural User Interfaces (NUI). In NUIs, a user interacts with the system more like he interacts with people and objects in the real world. Touch based computing is part of this. Speech recognition and commands are another part. Virtual assistants are another. Gesture based computing like the Kinect is another. HoloLens fits in this same trend.

Think about the media we experience through computers. We, in a 3 dimensional world, look from a distance at some other 3 dimensional space, through this two dimensional plane, the screen. It’s realistic, it’s lifelike, but it is always separate, a wistful yearning gaze at something you can see but never reach. This new technology in a sense collapses the screen. These two 3d worlds, the barrier collapsed, flow in amongst each other, and coexist in the same physical space. The user lives within and interacts with both. The user experiences a digital object or sprite within his own world. And the user can create his own holographic worlds, and share them with others, who also experience it with the same immersion and intimacy.


Not just a slick concept video

The most amazing thing is that the demo ideas shown in the first video I saw are apparently not just artists conceptions. This is what I initially assumed on watching the promo video shown at the beginning of this post. “This is a sense of where we’ll be by launch. ” But actually, no. These are examples of real software that has been developed by third parties. The Holographic Studio tool from the video where the father builds a spaceship and then sort of just Exports to 3D Printer is real. That 3d workbench tool the father was using. That’s not an artist’s representation. That’s currently existing software. Amazing. Can you imagine?


The Holo Studio tool  actually looks like a wonderful potential tool for (holo)graphic artisits for developing future media assets. And that’s just an early design tool. Imagine what a multimedia design powerhouse like Adobe could cook up (if they’re not already busy in the kitchen).

This could be a great tool for developing holographic sprites for games or holographic multimedia for learning and presentation materials. It’s a fantastic kickstart for helping to support a development community. This means that you could develop media for the environment from within the environment. That’s the beauty of this technology that fuses the space of the interface with the space of the user environment.

The other stuff is apparently real too – the Netflix on the wall, the Skype window in mid air, the Minecraft and gaming within the space of your living room is proven. Also, using these holograms to maneuvre a Windows interface is here.

This article also gives some more insider perspective, and corroborates that there is something real there.



Gaming Possibilities

The gaming application examples, remind me of the alien and robot game scene from “Her:” (Please excuse some of the language in the clip)


Imagine playing a game in your living room. The video of the person playing Minecraft in his living room looked great, as did the video of an alien run and jump game.

           A 3D character running around a living room using HoloLens.

Imagine playing a game with others connected to the same LAN. Say in an environment like an urban paintball course. For example, some sort of first person shooter game.

I look forward to what kind of gaming applications Microsoft can get going, maybe by engaging people developing for Xbox. Apparently apps developed as universal Windows apps will run naturally on HoloLens.

Training possibilities

Imagine the training possibilities. The same sort of facility as this urban paintball could function for military battle training in groups. Some walls and corridors would serve as a scaffold for a virtual environment, which the software could fill with holographic enemies.

Imagine the possibilities for simulation and scenario based training. Talk about immersive and high fidelity.

Imagine the possibilities of interactive 3D media on a range of subjects. Science for visualizations. History for being able to be immersed in historical locations. Math to visualize complex structures and graphs. Or computer science, to visualize the flow of an algorithm or the relationships between object classes in a computer program.

The Skype video communications tool on HoloLens could allow live maintenance coaching, as shown in the video. Imagine a maintenance training course where the learners’ organization could pay for an optional post course support. Give the customer organizations some of the devices, and then when there is an issue with the equipment they can’t solve, they can put on the headset and Skype call, and an available instructor can help to walk them through it. The instructor will be on Skype, watching the live stream from the maintainer HoloLens glasses on an iPad or other tablet. Or for distance learning, for tele-instruction. Imagine a maintenance course given completely remotely using the HoloLens. Imagine a teacher teaching a class live, but with students also tuning in via HoloLens.

I talked about these and some other ideas when I spoke before about possibilities in training using augmented reality on the Google Glass. This new product comes in a wave of other Virtual Reality and Augmented Reality. This includes Facebook’s acquisition, Oculus VR, Samsung’s Gear VR goggles, and Google’s Magic Leap project.

Or imagine if, beyond being able to have a Skype conversation with someone on a flat window in space, if other people in remote locations could be brought into this world as realistic, high fidelity holographic representations in real time. Kind of like Princess Leia calling out for Obi Wan on Star Wars episode 4. Maybe that’s beyond where the technology will be soon, but that would be bona fide tele-conferencing and tele-collaboration. 

 Further reading