Hands-on with HoloLens

I’ve written a lot here previously about the potential for Augmented Reality, specifically the HoloLens, introduced in 2015 and released as a developer edition in 2016. However, the HoloLens being expensive ($4000 in Canada) and a little hard to find, I haven’t previously had the chance to use one hands on. Last week, thanks to a vendor demo at work, I was finally lucky enough to get my hands and eyes on this amazing piece of technology. Here I share some of my observations from this brief hands on session.

Note that I only got to spend something like 30 minutes of time directly with the device, so there was not time for a full and comprehensive review, nor for any deep assessment of how well it would fit into an extended use workflow. But I explored around as much as I could in the time I had.

Strengths

First of all, the strengths:

  • The refresh rate and power seem good. I didn’t notice any lag or stutter while moving around a relatively complex 3d image
  • The device features a decent amount of storage memory. The device has a nominal memory is 64GB. You end up with somewhat less with the operating system installed, but you still have a decent amount of memory.
  • The main interface is similar to Windows with the tiled layout. Menu interfaces are similar to Windows, Xbox One, etc. It runs a special edition of Windows 10.

  • The voice interface and voice recognition seemed to work very well. The interface for the OS uses Cortana voice assistant as one of its main interface modalities (along with gestures). You can use voice commands to launch apps and other basic tasks.
  • The operating system comes with a nice tutorial app to help the user quickly acclimate to the user interface. I didn’t get a chance to go all the way through this tutorial, but what I saw was good.
  • The device has connectivity for Bluetooth and Wifi. Can connect a Bluetooth keyboard. Allows you to use typical apps for productivity, serves as a screen. I did not have the chance to try this out.
  • Battery life is not bad. About 3 hours with active use. I see some reviews online reporting up to 5 hours, depending on the type of usage. This is not enough for a full workday, but enough to say, get you to lunch, you plugin to recharge, and then come back after.  Recharging apparently takes a fair amount of time. However, the HoloLens can be used while plugged in if you’re sitting in one place, which could help extend the length of a session.
  • Video capture was smooth and easy, and will capture your first person perspective, including the apps and holograms in your view. Microsoft calls this “mixed reality capture.” The device features a 2.3 megapixel video and still camera.
  • I found the device to be reasonably “glasses friendly.” I wear glasses, and sometimes VR headsets can be a pain to make sure the two layers of lenses between the digital goodness and my eyes cooperate and align, but this worked well.
  • The device was also reasonably comfortable to wear. It sat well on my head, and was not noticeably heavy. Some VR goggles I find a little front heavy, but the HoloLens is lighter and doesn’t project as much outwards in the front.

Weaknesses

The device is not by any means perfect, however. This is a first generation developer edition for developer experimenters and creative professionals. It’s an R0 product. It’s a lot of the way toward where it needs to be for a widely usable consumer device, but some things could benefit from a few more years to tweak and take advantage of the ever falling prices of hardware to improve the specs and performance. A few shortcomings with the current edition:

  • The vertical field of view is a little narrow.  It’s far enough from your eyes that it only takes up part of the field of view and the edge is visible. If you’re too close to a large holographic object, it will just end at the edge, which breaks the illusion. Perhaps later versions will address this with a screen that fills more of the field of view.
  • Resolution is a little low at a max of 720. It would be difficult to do very detailed 3D models. Hopefully future models can bump this up to at least HD.
  • The device seems somewhat limited in terms of the basic built in gestures. Tap to click on something,tap and hold to drag, resize, or rotate an object and bloom (open your fingers with your palm facing you) to launch a menu. I would be interested to see if the development kit allows you to build new gestures. I believe the Kinect dev kit has a tool for this.  The only thing is that if you build new gestures, part of your work would be to seamlessly educate users on the gestures. All in all though, I did find it relatively intuitive to pick up the basic gestures, even without the built in tutorial.

 

  • The RAM is somewhat limited – 2GB for the CPU, and 1GB for the “Holographic Processing Unit,” a specialized GPU of sorts for handling processing of 3D graphics. The CPU specs are at the level of a smartphone or tablet, so despite the fact you can connect a keyboard, you wouldn’t be able to run very heavy apps. That’s something that would be good to work on for later editions, because one of the places I could see HoloLens ultimately being useful would be as a replacement for a desktop computer and screen. More RAM would be needed to make that feasible.

Apps available

There are a number of apps available. Some of these I was able to try out, others I was not. The apps available include:

  • Netflix
  • Edge browser
  • Camera
  • Movies and Tv
  • Skype
  • Music
  • OneNote
  • OneDrive
  • MS Office
  • HoloStudio
  • Minecraft

Basic Functionality

The HoloLens responds to movement laterally and vertically through space, and to 3 axis rotations of your head.

Windows that are launched initially hover a few meters in front of your eyes, and by default move to stay in your view as you move and turn your head.  However, you can also point and click to pin windows and objects in place. There is some smartness to this; the system software identifies flat surfaces, whether horizontal or vertical. If you turn so that an object or window is hovering above a table, and then click to pin, it will lock to the table top. If you hover a window over a wall, and point and click to pin, it will lock to the wall. You can similarly point and click to unpin. Picture something like pinning an app icon to your desktop or the toolbar in Windows or MacOS, but with the desktop now being now any planar surface in your  surroundings. It’s hard to understand how cool this really is until you try it, to spatially arrange your computing within your environment.

Even more interesting is that the system can remember the placement of Windows and holograms between sessions. For example, you could pin a Netflix window to your living room wall. Voila. Virtual TV. Walk away into another room or to another floor and come back, and it’s still there.  Or rather, it remembers that it should be there. And it will still be there the next time you fire up the HoloLens. There are so many possibilities for this.

Cameras within HoloLens map the environment as you walk and gaze around. This creates a 3d model of the surroundings. If you walk around a building or house, it will map all the rooms visited and store a map of the area.

The HoloLens has built in speakers that allow for an immersive spatial sound experience. Sound sources can have directionality, just as in real life, and sounds attached to holographic objects will rise and fall in volume and adjust in perceived position relative to you as you move closer, farther away, and go around the virtual object. This enhances the immersiveness. And while the speakers in the device manage a good volume to let you hear, it doesn’t seem to project significant noise to those nearby.

Unfortunately the HoloStudio tool for crafting 3D objects did not seem to be installed on the unit I got to play  with, which was a minor disappointment. This is an app I was pretty excited about. Nor did a get a chance to check out the experience of browsing and downloading apps from the store.

Nor, sadly, was the amazingly cool looking Minecraft app installed. (First thing my son asked about when I told about my hands-on)

The Path Forward for HoloLens

All in all, I came out of this brief test of the HoloLens extremely impressed. This is an amazing first draft of this technology. Even with some minor shortcomings it nevertheless reinforced  my earlier perception that Augmented Reality is going to be a key part of the future of personal, creative, and work computing.

This first release of the HoloLens is a developer version is not meant for mass adoption. It’s a first step in a planned series of releases. Previously there was an intention to release an incremental v2 in 2018, and a v3 in 2020.

However, recent unconfirmed reports indicate that Microsoft will instead focus effort on speeding up the v3 release by skipping v2 and refocusing efforts on a big jump with v3. V3 is expected in 2019, and, one would presume, this is where the tech would launch in a more powerful and hopefully cheaper consumer oriented model.

I for one am looking forward with anticipation. Until then, I can only hope I will have the chance to spend more time exploring the the possibilities of this new mode of computing.

Another review:

http://www.vrheads.com/after-24-hours-inside-microsoft-hololens-i-dont-want-take-it-again

Previous Augmented/Virtual Reality Articles on this Blog

Virtual and Augmented Reality and first person media

On the Microsoft HoloLens

Augmented Reality and Wearable Computing: Possibilities for Google Glass in Training