On the Microsoft HoloLens

Microsoft’s big surprise

I want to talk about a big tech story from last week, the surprise unveiling of the Microsoft HoloLens Augmented Reality headset. This new product was revealed in the course of the launch of the Windows 10 Technical Preview. Probably, some of those reading this have seen the promo video:

https://www.youtube.com/watch?v=aThCr0PsyuA

This is pretty amazing, and completely out of nowhere. There was no advance sign that Microsoft was announcing this. The project, from the same inventor as the Kinect sensor, was a complete secret until the big reveal this past week. This pleasant surprise has had the tech world abuzz ever since.

Now, a word of caution. Yes, this video is a Microsoft promo, with an artistic representation of how its supposed to work. This is a product that is still, as far as anyone knows, in pre-production prototype stage. There is no release date; it may not even be out this year.

But still. From what I’ve seen from other sources, this is the most impressive and mindblowing thing to come along in a long time. This is next level iPhone and iPad big, one of the next steps in the evolution of the computer.

Why this is a very big deal

This sounds like hyperbole, and certainly, Microsoft has had products before that looked great in an early demo but it didn’t pan out. But generally I have a pretty good eye for this sort of thing.

In 2008 was about the only person in my office with a smart phone. My first cell phone. It was the reveal of the iPhone 3G that showed me the time had arrived. I didn’t buy an iPhone, but that was what showed that this technology had arrived. I had stayed out of cell phones to that point, because I was waiting for the technology to get to a certain point where it can become an indispensable all-in-one computer tool that fits in your pocket and connects you to the internet and lets you do everything – take notes, read/write email, read web articles, internet chat, play games, do light office tasks, etc.

A few years later, in 2010, Apple released the iPad, the first one. I understood immediately that this was going to be huge. So for the first and only time, I waited in line on launch day at the Apple store. That first iPad had a lot of limitations (no camera, no multi-tasking, no folders, kind of chunky – and you couldn’t even copy-paste!), but I loved it for its obvious potential.

I’ve been on Netflix since the initial beta in Canada. That was a tough sell back then. The selection was pretty lousy in Canada at that point, though you could see it grow month by month. Now something like 1/6 of Canadian residents has a Netflix account. 

This new product from Microsoft feels like the same sort of up and coming thing. This is literally science fiction type stuff that I previously would have considered maybe possible within  the coming decade. But this looks like it will soon be available. The world moves so fast these days.

This product fits in amongst the ongoing paradigm shift toward Natural User Interfaces (NUI) that I’ve spoken about previously.  The world of computing went through Command Line Interfaces (CLI), Graphical User Interfaces (GUI), and now, Natural User Interfaces (NUI). In NUIs, a user interacts with the system more like he interacts with people and objects in the real world. Touch based computing is part of this. Speech recognition and commands are another part. Virtual assistants are another. Gesture based computing like the Kinect is another. HoloLens fits in this same trend.

Think about the media we experience through computers. We, in a 3 dimensional world, look from a distance at some other 3 dimensional space, through this two dimensional plane, the screen. It’s realistic, it’s lifelike, but it is always separate, a wistful yearning gaze at something you can see but never reach. This new technology in a sense collapses the screen. These two 3d worlds, the barrier collapsed, flow in amongst each other, and coexist in the same physical space. The user lives within and interacts with both. The user experiences a digital object or sprite within his own world. And the user can create his own holographic worlds, and share them with others, who also experience it with the same immersion and intimacy.

https://www.youtube.com/watch?v=aAKfdeOX3-o

Not just a slick concept video

The most amazing thing is that the demo ideas shown in the first video I saw are apparently not just artists conceptions. This is what I initially assumed on watching the promo video shown at the beginning of this post. “This is a sense of where we’ll be by launch. ” But actually, no. These are examples of real software that has been developed by third parties. The Holographic Studio tool from the video where the father builds a spaceship and then sort of just Exports to 3D Printer is real. That 3d workbench tool the father was using. That’s not an artist’s representation. That’s currently existing software. Amazing. Can you imagine?

https://www.youtube.com/watch?v=kCMxBw-utEY

The Holo Studio tool  actually looks like a wonderful potential tool for (holo)graphic artisits for developing future media assets. And that’s just an early design tool. Imagine what a multimedia design powerhouse like Adobe could cook up (if they’re not already busy in the kitchen).

This could be a great tool for developing holographic sprites for games or holographic multimedia for learning and presentation materials. It’s a fantastic kickstart for helping to support a development community. This means that you could develop media for the environment from within the environment. That’s the beauty of this technology that fuses the space of the interface with the space of the user environment.

The other stuff is apparently real too – the Netflix on the wall, the Skype window in mid air, the Minecraft and gaming within the space of your living room is proven. Also, using these holograms to maneuvre a Windows interface is here.

This article also gives some more insider perspective, and corroborates that there is something real there.

https://www.linkedin.com/pulse/insiders-guide-new-holographic-computing-lewis-shepherd?trk=object-title

 

Gaming Possibilities

The gaming application examples, remind me of the alien and robot game scene from “Her:” (Please excuse some of the language in the clip)

https://www.youtube.com/watch?v=Qhk70sRV35U

Imagine playing a game in your living room. The video of the person playing Minecraft in his living room looked great, as did the video of an alien run and jump game.

           A 3D character running around a living room using HoloLens.

Imagine playing a game with others connected to the same LAN. Say in an environment like an urban paintball course. For example, some sort of first person shooter game.

I look forward to what kind of gaming applications Microsoft can get going, maybe by engaging people developing for Xbox. Apparently apps developed as universal Windows apps will run naturally on HoloLens.

Training possibilities

Imagine the training possibilities. The same sort of facility as this urban paintball could function for military battle training in groups. Some walls and corridors would serve as a scaffold for a virtual environment, which the software could fill with holographic enemies.

Imagine the possibilities for simulation and scenario based training. Talk about immersive and high fidelity.

Imagine the possibilities of interactive 3D media on a range of subjects. Science for visualizations. History for being able to be immersed in historical locations. Math to visualize complex structures and graphs. Or computer science, to visualize the flow of an algorithm or the relationships between object classes in a computer program.

The Skype video communications tool on HoloLens could allow live maintenance coaching, as shown in the video. Imagine a maintenance training course where the learners’ organization could pay for an optional post course support. Give the customer organizations some of the devices, and then when there is an issue with the equipment they can’t solve, they can put on the headset and Skype call, and an available instructor can help to walk them through it. The instructor will be on Skype, watching the live stream from the maintainer HoloLens glasses on an iPad or other tablet. Or for distance learning, for tele-instruction. Imagine a maintenance course given completely remotely using the HoloLens. Imagine a teacher teaching a class live, but with students also tuning in via HoloLens.

I talked about these and some other ideas when I spoke before about possibilities in training using augmented reality on the Google Glass. This new product comes in a wave of other Virtual Reality and Augmented Reality. This includes Facebook’s acquisition, Oculus VR, Samsung’s Gear VR goggles, and Google’s Magic Leap project.

Or imagine if, beyond being able to have a Skype conversation with someone on a flat window in space, if other people in remote locations could be brought into this world as realistic, high fidelity holographic representations in real time. Kind of like Princess Leia calling out for Obi Wan on Star Wars episode 4. Maybe that’s beyond where the technology will be soon, but that would be bona fide tele-conferencing and tele-collaboration. 

 Further reading

http://www.microsoft.com/microsoft-hololens/en-us

http://mashable.com/2015/01/21/hololens-business/

http://www.wired.com/2015/01/microsoft-hands-on/

http://blogs.wsj.com/personal-technology/2015/01/21/microsoft-sees-holograms-in-our-windows-10-future/

http://www.gamespot.com/articles/microsoft-s-hololens-is-something-different-than-o/1100-6424809/

http://www.gamesindustry.biz/articles/2015-01-22-molyneux-warns-microsoft-dont-overpromise-on-hololens

Instructional Design Lessons from the designers of Stephen Hawking’s text input system

A recent article in Wired magazine told the story of the team of Intel engineers who developed ALS-Inflicted physicist Stephen Hawkings current speech input system. I highly recommend. The story at the link below.

http://www.wired.com/2015/01/intel-gave-stephen-hawking-voice/

As an Instructional Designer, I loved this article, because it talks about the iterative design and development process of a sophisticated support tool. It has so many great lessons about engineering and design, about needs analysis, about iterative design, interface design, and about prototyping and testing. These lessons apply to design in general as well to the specific case of Instructional Design.

Let’s take a closer look at a few of these lessons.

The power of good tools

It shows the power of well-crafted support tools as a solution to performance challenges. Educational Technology students will remember that one of the core messages from Human Performance Technology is that our real business is not so much teaching in itself, but in improving human performance, improving the ability of people to reach their goals for various efforts. Sometimes training is the route to this improvement, but sometimes better tools are the answer. A well crafted support tool fits us like a glove. It becomes like an extension of us and extends our abilities.

Change can be jarring

It illustrates the challenge of insituting change in human-machine systems, particularly with older users. On paper a new tool may be much more efficient. And maybe it would be, with a blank slate. But users bring their attachments to long standing learned patterns of doing things. There is an inertia here. Learning a new way of doing something can involve unlearning the old way. It highlights one of the challenging contraints in design, keeping of continuity with what came before to avoid overhwhelming the user. The tool is supposed to make life easier; be cautious of how big an up-front hill you put between the user and the point where the user starts to see the benefit.

Know your users

It illustrates the importance of taking time to understand your user, where their challenges are, and how to solve those problems. A bit earlier I mentioned the power of tools that are crafted to the user. But to design such tools, you really need to take the time to know the needs of the user.

Iterative design and rapid prototyping

It illustrates the benefit of an iterative design process designing. building, and testing multiple prototypes. While in the textbooks, Instructional Design is presented as iterative, in practice, a lot of training organizations see it as linear. You analyze, then you make the whole design, and then you build it, and implement it. But so often you don’t really understand the needs of the user until you get into building something and let the user try it. And often, even though the user signs off  on a design, they don’t really understand what it means either until they see it live. That’s why an iterative design with rapid prototyping can be so helpful. More back and forth in the design and development phases can save a lot of headaches later. Build something preliminary, test it with the user, then make corrections. Figure out something is not working in the prototype stage, not after investing hundreds of hours into development.

Communication

Finally, it highlights the importance of good communication with the client regularly about design vision, plans. Of listening carefully about what worked and what didn’t in the most recent prototype. Of hearing what they are saying and what is unsaid, between the lines.

“Teach a man to fish…”

There is an old expression that goes,

“Give a man a fish, and he eats for a day; Teach a man to fish, and he eats for a lifetime.”

I always loved that saying.

The idea here is that while performing a service for a person is helpful, it is even more helpful to teach a person so that he can perform the same service for himself. The first is a one-off gift, while the second is an ongoing gift, because the recipient’s capabilities are expanded for as long as he uses and maintains the skills. Rather than a one-off benefit, there is a relatively permanent, stable expansion of the person’s potential. There is also a corresponding expansion of the person’s autonomy and self-sufficiency. This is the beauty of teaching and training, the reward of it – to help others grow through learning.

This is why many of us got into the fields of education, teaching, and training in the first place.

I was reflecting on this idea of “teaching people to fish,” when the thought occurred to me: Can this same idea be applied to teaching and training itself? After all, teaching and training are themselves skills, and we ourselves, as learning professionals (or many of us, anyway) went to school to formally study these skills.

As teachers, as training designers and developers, we win our daily bread by helping subject matter experts translate their knowledge and skills into a form through which learners can effectively and efficiently assimilate those same skills and knowledge. We occupy a strange position between expert and learner, not necessarily experts ourselves, but midwives of sorts, helping birth something of that expertise to a new audience.

But what if, instead of helping in a one-off fashion like this, we instead focused more of our efforts on helping to spread our own expertise in needs analysis, instructional design, training material development. Let’s not misunderstand here; we ourselves have a certain legitimate expertise, and it takes time and effort to properly develop. It’s not something that can just be transmitted instantly to others. There’s room in this world for specialization and division of labor; no one has time to do everything. But what I’m saying is that if we believe in the power of our field of expertise, we should want to diffuse that expertise more widely. And we all know that often a few simple modifications to a lesson or style of delivery can make a world of difference.

There has been a big shift in recent years toward more informal modes of learning. Research indicates that people do most of their learning informally. Mostly, people learn, not by signing up for a formal course, but through less formal means: by searching online resources, asking questions on a web forum,  going on an FAQ, as well as asking friends, family, teammates, coworkers, and fellow members of communities of practice. How many interactions do people have in a given week that involve explaining something to someone else, teaching something? Everyone has times when there is a need to put on the hat of teacher / trainer, whether at home, at work, or at play.

More and more of this will happen as network and communication technology makes it easier to directly connect people that have questions with people that have answers. How much more effective and efficient can this be if across this mass of people is diffused even some basics about teaching and training? What if we were able, as a community of learning professionals, to disseminate enough of what we know to raise the effectiveness of the average learning conversation by even 5%? It really doesn’t take much; quality is probably more important than quantity here. Spread a few key basic principles and best practices widely over time, and give people a chance to practice so that they take it to heart. Overall, cumulatively, it adds up to a large return on investment.

So the question becomes, how can we do this?

What sorts of basic principles and practices should we focus on diffusing widely to the population?

What are the key bullet point takeaways?

What sorts of technology/media should we use for this purpose?

How can we evaluate the effectiveness of our efforts and recalibrate efforts on the fly accordingly?

What learning professionals can learn from video games

Introduction

Video games are a huge global market, bigger than movies and music. somewhere north of $60 billlion a year, and growing, between console, PC, and mobile. In comparison, estimated global box office revenue for 2014 was about $38 billion.

Adults, particularly older ones that didn’t grow up playing games, tend to be somewhat dismissive of the merit and value of gaming. It’s a toy for their kids, a distraction rather than something to take too seriously.

But today, electronic games are also a serious pastime for working adults as well, and for men and women. As well, modern gaming systems are basically sophisticated computers for the living room that handle a range of different forms of entertainment – TV, music, movies, gaming, apps, web browsing.

Some interesting stats:

  • The average age is 31
  • The gaming population is roughly split between men and women, 52% vs 48%.
  • The average US household has at least one gaming console

Source: http://venturebeat.com/2014/04/29/gaming-advocacy-group-the-average-gamer-is-31-and-most-play-on-a-console/ 

So the question becomes, what do video games have to offer us, as learning professionals, beyond fun and entertainment?

Video games have lessons to teach us, as learning professionals in a few key ways.

The Lessons of video games

1. First, they show how to make complex, realistic tasks fun. People play games because they’re fun. There’s struggle and effort and challenge and obstacles to overcome, and often a lot of seriousness, but in the end it’s fun.

2. Second, video games are distinctly successful at attracting and holding people’s attention. People engage and stay engaged for sustained periods. Modern games can take upwards of 100 hours to complete. This is comparable, order of magnitude, to the time investment for a professional pilot to earn a type rating to fly a new type of aircraft.

These games will often involve intricate levels of goals and objectives and sub-goals and side goals. But people are drawn to persist and complete them. Games play effectively with different motivational and reward pathways to produce a compelling need to persist at the task despite challenges and difficulties and sometimes because of these challenges and difficulties.

These are some the same sorts of problems we face as designers and developers of learning materials. How to motivate people to engage in our content, and how to keep them engaged through to the end of the course. The success of games in this regard and the ways games achieve this success should be a motivator for us. We should look at the tricks games use to keep us interested and try to apply the lessons to our learning materials.

3. Third, video games are at the forefront of interactivity. If we want to see the state of the art of near / present term feasible interactions, we need to keep up with games. This is where the innovation is happening. Have you seen modern game controllers used with gaming consoles?

It looks like you could use it to put something in space. So many different buttons and directional controllers, and all in one neat, ergonomic package that fits perfectly in your hand.

Off-brand Xbox 360 controllers (similar to the one shown on the left) are used to fly military drones. Playstation controllers (the one shown on the right) are similar in design.

Microsoft’s Kinect sensor, meanwhile, used for the Xbox, is arguably the most innovative consumer market man-machine interface out there, using multiple cameras and microphones to detect body position and orientation and take speech inputs. It basically lets you, given effective software, control a computer using your voice and the movements of your body.

And then of course there are some of the novel forms of interaction used for tablet and phone games. Touch is one example, with controls based on tapping, swiping, and pinching. The other is controls based on the accelerometer and gyroscope, where games are controlled by moving, shaking, turning, or tilting the device. Some good examples are the Sky Map app on Android and Super Monkey Ball 2 and the Labyrinth 2 on iOS.

Finally, up and coming gaming tech such as the Oculus VR look to take gaming to a new level of immersiveness.

The common denominator here is that effective and meaningful use (not just as a gimmick) of sophisticated interactivity helps to pull the user in and increase engagement.

In comparison, the built in types of interactions we see enabled in eLearning authoring tools like Storyline and Captivate tend to be very simplistic – multiple choice, drag and drop, matching. Video games can motivate and inspire us by showing what is really possible today in interactivity.

 Conclusion

Video games, far from being something only for kids, have become a serious entertainment industry enjoyed by people of a wide variety of ages. They are an established part of our modern lives. Learning professionals looking for ideas on new and better ways to engage and motivate learners should take a serious look at the best practices of modern video games.

And hey – you might even have some fun doing it.