Virtual reality (VR) and augmented reality (AR) offer a possibility that hasn’t been so present in media – a truly first person perspective.
First person perspective, where you see things through the eyes of another person or fictional character, has long been a regular form of narrative for novels. It has also been has been used in electronic media in some cases, within the limits of display technologies. In electronic gaming, the first person shooter genre, going back to Doom, displays a flat representation of the perspective of the player character. In film, there are a few interesting examples where large parts of a film are shot to tell the story through the eyes of the main character. Enter the Void and Diving Bell and the Butterfly are good examples of this.
But newer technology of VR and AR take things another level beyond.
Just quickly for those who may be unfamiliar, VR and AR are two related, but subtly distinct forms of immersive media. In VR, through wearable goggles and other peripherals, a person is immersed in a separate, virtual space, some separate, alternate reality. In AR, the user is still within the visual space of normal reality, but augmentations are superimposed in the field of view over the objects of your space. The first is an escape to a different reality, while the second is a blending of this reality with digitally displayed virtual objects. A good example of VR is Oculus VR, while an intriguing example of AR is Microsoft’s HoloLens.
In both, the display is typically wrapped around your field of view. It takes up your whole visual space, or almost. While watching a film or playing a typical video game, you sit in your normal space and see images on a screen within part of your field of view. VR and AR can go beyond this. You’re not watching a presentation or watching a story from some distance. You’re there in the midst of it. You live it, almost. And this makes a big difference in terms of realism and immersion.
It’s no longer so much imagining being in the story, but experiencing it. You become a first person spectator, or participant, in interactive media. You are the character. You fully enter the world of the media.
Imagine the possibilities for:
advertising – This could have a hugeimpact. With these sorts of technology, there is no need to make someone imagine the experience of the product. They can just have the experience first hand.
creative storytelling and gameplay in games. You already have the first person shooter (FPS) genre. I recently started the game Destiny on my Xbox. You’re just immersed in these majestic landscapes. How much more engaging and immersive if the screen is wrapped around your face and you’re in the middle of it?
creative storytelling in interactive media
teaching and training – to put students within a scientific simulation or visualization, to virtually repair a 3d model of a piece of equipment, to virtually visit a historical site or a recreation of a historic event in an immersive manner
Overall, there are possibilities here for a whole new type of media. It will be interesting to see what creative people can do with it.
I want to talk about a big tech story from last week, the surprise unveiling of the Microsoft HoloLens Augmented Reality headset. This new product was revealed in the course of the launch of the Windows 10 Technical Preview. Probably, some of those reading this have seen the promo video:
This is pretty amazing, and completely out of nowhere. There was no advance sign that Microsoft was announcing this. The project, from the same inventor as the Kinect sensor, was a complete secret until the big reveal this past week. This pleasant surprise has had the tech world abuzz ever since.
Now, a word of caution. Yes, this video is a Microsoft promo, with an artistic representation of how its supposed to work. This is a product that is still, as far as anyone knows, in pre-production prototype stage. There is no release date; it may not even be out this year.
But still. From what I’ve seen from other sources, this is the most impressive and mindblowing thing to come along in a long time. This is next level iPhone and iPad big, one of the next steps in the evolution of the computer.
Why this is a very big deal
This sounds like hyperbole, and certainly, Microsoft has had products before that looked great in an early demo but it didn’t pan out. But generally I have a pretty good eye for this sort of thing.
In 2008 was about the only person in my office with a smart phone. My first cell phone. It was the reveal of the iPhone 3G that showed me the time had arrived. I didn’t buy an iPhone, but that was what showed that this technology had arrived. I had stayed out of cell phones to that point, because I was waiting for the technology to get to a certain point where it can become an indispensable all-in-one computer tool that fits in your pocket and connects you to the internet and lets you do everything – take notes, read/write email, read web articles, internet chat, play games, do light office tasks, etc.
A few years later, in 2010, Apple released the iPad, the first one. I understood immediately that this was going to be huge. So for the first and only time, I waited in line on launch day at the Apple store. That first iPad had a lot of limitations (no camera, no multi-tasking, no folders, kind of chunky – and you couldn’t even copy-paste!), but I loved it for its obvious potential.
I’ve been on Netflix since the initial beta in Canada. That was a tough sell back then. The selection was pretty lousy in Canada at that point, though you could see it grow month by month. Now something like 1/6 of Canadian residents has a Netflix account.
This new product from Microsoft feels like the same sort of up and coming thing. This is literally science fiction type stuff that I previously would have considered maybe possible within the coming decade. But this looks like it will soon be available. The world moves so fast these days.
This product fits in amongst the ongoing paradigm shift toward Natural User Interfaces (NUI) that I’ve spoken about previously.The world of computing went through Command Line Interfaces (CLI), Graphical User Interfaces (GUI), and now, Natural User Interfaces (NUI). In NUIs, a user interacts with the system more like he interacts with people and objects in the real world. Touch based computing is part of this. Speech recognition and commands are another part. Virtual assistants are another. Gesture based computing like the Kinect is another. HoloLens fits in this same trend.
Think about the media we experience through computers. We, in a 3 dimensional world, look from a distance at some other 3 dimensional space, through this two dimensional plane, the screen. It’s realistic, it’s lifelike, but it is always separate, a wistful yearning gaze at something you can see but never reach. This new technology in a sense collapses the screen. These two 3d worlds, the barrier collapsed, flow in amongst each other, and coexist in the same physical space. The user lives within and interacts with both. The user experiences a digital object or sprite within his own world. And the user can create his own holographic worlds, and share them with others, who also experience it with the same immersion and intimacy.
Not just a slick concept video
The most amazing thing is that the demo ideas shown in the first video I saw are apparently not just artists conceptions. This is what I initially assumed on watching the promo video shown at the beginning of this post. “This is a sense of where we’ll be by launch. ” But actually, no. These are examples of real software that has been developed by third parties. The Holographic Studio tool from the video where the father builds a spaceship and then sort of just Exports to 3D Printer is real. That 3d workbench tool the father was using. That’s not an artist’s representation. That’s currently existing software. Amazing. Can you imagine?
The Holo Studio tool actually looks like a wonderful potential tool for (holo)graphic artisits for developing future media assets. And that’s just an early design tool. Imagine what a multimedia design powerhouse like Adobe could cook up (if they’re not already busy in the kitchen).
This could be a great tool for developing holographic sprites for games or holographic multimedia for learning and presentation materials. It’s a fantastic kickstart for helping to support a development community. This means that you could develop media for the environment from within the environment. That’s the beauty of this technology that fuses the space of the interface with the space of the user environment.
The other stuff is apparently real too – the Netflix on the wall, the Skype window in mid air, the Minecraft and gaming within the space of your living room is proven. Also, using these holograms to maneuvre a Windows interface is here.
This article also gives some more insider perspective, and corroborates that there is something real there.
The gaming application examples, remind me of the alien and robot game scene from “Her:” (Please excuse some of the language in the clip)
Imagine playing a game in your living room. The video of the person playing Minecraft in his living room looked great, as did the video of an alien run and jump game.
Imagine playing a game with others connected to the same LAN. Say in an environment like an urban paintball course. For example, some sort of first person shooter game.
I look forward to what kind of gaming applications Microsoft can get going, maybe by engaging people developing for Xbox. Apparently apps developed as universal Windows apps will run naturally on HoloLens.
Imagine the training possibilities. The same sort of facility as this urban paintball could function for military battle training in groups. Some walls and corridors would serve as a scaffold for a virtual environment, which the software could fill with holographic enemies.
Imagine the possibilities for simulation and scenario based training. Talk about immersive and high fidelity.
Imagine the possibilities of interactive 3D media on a range of subjects. Science for visualizations. History for being able to be immersed in historical locations. Math to visualize complex structures and graphs. Or computer science, to visualize the flow of an algorithm or the relationships between object classes in a computer program.
The Skype video communications tool on HoloLens could allow live maintenance coaching, as shown in the video. Imagine a maintenance training course where the learners’ organization could pay for an optional post course support. Give the customer organizations some of the devices, and then when there is an issue with the equipment they can’t solve, they can put on the headset and Skype call, and an available instructor can help to walk them through it. The instructor will be on Skype, watching the live stream from the maintainer HoloLens glasses on an iPad or other tablet. Or for distance learning, for tele-instruction. Imagine a maintenance course given completely remotely using the HoloLens. Imagine a teacher teaching a class live, but with students also tuning in via HoloLens.
I talked about these and some other ideas when I spoke before about possibilities in training using augmented reality on the Google Glass. This new product comes in a wave of other Virtual Reality and Augmented Reality. This includes Facebook’s acquisition, Oculus VR, Samsung’s Gear VR goggles, and Google’s Magic Leap project.
Or imagine if, beyond being able to have a Skype conversation with someone on a flat window in space, if other people in remote locations could be brought into this world as realistic, high fidelity holographic representations in real time. Kind of like Princess Leia calling out for Obi Wan on Star Wars episode 4. Maybe that’s beyond where the technology will be soon, but that would be bona fide tele-conferencing and tele-collaboration.
A recent article in Wired magazine told the story of the team of Intel engineers who developed ALS-Inflicted physicist Stephen Hawkings current speech input system. I highly recommend. The story at the link below.
As an Instructional Designer, I loved this article, because it talks about the iterative design and development process of a sophisticated support tool. It has so many great lessons about engineering and design, about needs analysis, about iterative design, interface design, and about prototyping and testing. These lessons apply to design in general as well to the specific case of Instructional Design.
Let’s take a closer look at a few of these lessons.
The power of good tools
It shows the power of well-crafted support tools as a solution to performance challenges. Educational Technology students will remember that one of the core messages from Human Performance Technology is that our real business is not so much teaching in itself, but in improving human performance, improving the ability of people to reach their goals for various efforts. Sometimes training is the route to this improvement, but sometimes better tools are the answer. A well crafted support tool fits us like a glove. It becomes like an extension of us and extends our abilities.
Change can be jarring
It illustrates the challenge of insituting change in human-machine systems, particularly with older users. On paper a new tool may be much more efficient. And maybe it would be, with a blank slate. But users bring their attachments to long standing learned patterns of doing things. There is an inertia here. Learning a new way of doing something can involve unlearning the old way. It highlights one of the challenging contraints in design, keeping of continuity with what came before to avoid overhwhelming the user. The tool is supposed to make life easier; be cautious of how big an up-front hill you put between the user and the point where the user starts to see the benefit.
Know your users
It illustrates the importance of taking time to understand your user, where their challenges are, and how to solve those problems. A bit earlier I mentioned the power of tools that are crafted to the user. But to design such tools, you really need to take the time to know the needs of the user.
Iterative design and rapid prototyping
It illustrates the benefit of an iterative design process designing. building, and testing multiple prototypes. While in the textbooks, Instructional Design is presented as iterative, in practice, a lot of training organizations see it as linear. You analyze, then you make the whole design, and then you build it, and implement it. But so often you don’t really understand the needs of the user until you get into building something and let the user try it. And often, even though the user signs off on a design, they don’t really understand what it means either until they see it live. That’s why an iterative design with rapid prototyping can be so helpful. More back and forth in the design and development phases can save a lot of headaches later. Build something preliminary, test it with the user, then make corrections. Figure out something is not working in the prototype stage, not after investing hundreds of hours into development.
Finally, it highlights the importance of good communication with the client regularly about design vision, plans. Of listening carefully about what worked and what didn’t in the most recent prototype. Of hearing what they are saying and what is unsaid, between the lines.
“Give a man a fish, and he eats for a day; Teach a man to fish, and he eats for a lifetime.”
I always loved that saying.
The idea here is that while performing a service for a person is helpful, it is even more helpful to teach a person so that he can perform the same service for himself. The first is a one-off gift, while the second is an ongoing gift, because the recipient’s capabilities are expanded for as long as he uses and maintains the skills. Rather than a one-off benefit, there is a relatively permanent, stable expansion of the person’s potential. There is also a corresponding expansion of the person’s autonomy and self-sufficiency. This is the beauty of teaching and training, the reward of it – to help others grow through learning.
This is why many of us got into the fields of education, teaching, and training in the first place.
I was reflecting on this idea of “teaching people to fish,” when the thought occurred to me: Can this same idea be applied to teaching and training itself? After all, teaching and training are themselves skills, and we ourselves, as learning professionals (or many of us, anyway) went to school to formally study these skills.
As teachers, as training designers and developers, we win our daily bread by helping subject matter experts translate their knowledge and skills into a form through which learners can effectively and efficiently assimilate those same skills and knowledge. We occupy a strange position between expert and learner, not necessarily experts ourselves, but midwives of sorts, helping birth something of that expertise to a new audience.
But what if, instead of helping in a one-off fashion like this, we instead focused more of our efforts on helping to spread our own expertise in needs analysis, instructional design, training material development. Let’s not misunderstand here; we ourselves have a certain legitimate expertise, and it takes time and effort to properly develop. It’s not something that can just be transmitted instantly to others. There’s room in this world for specialization and division of labor; no one has time to do everything. But what I’m saying is that if we believe in the power of our field of expertise, we should want to diffuse that expertise more widely. And we all know that often a few simple modifications to a lesson or style of delivery can make a world of difference.
There has been a big shift in recent years toward more informal modes of learning. Research indicates that people do most of their learning informally. Mostly, people learn, not by signing up for a formal course, but through less formal means: by searching online resources, asking questions on a web forum, going on an FAQ, as well as asking friends, family, teammates, coworkers, and fellow members of communities of practice. How many interactions do people have in a given week that involve explaining something to someone else, teaching something? Everyone has times when there is a need to put on the hat of teacher / trainer, whether at home, at work, or at play.
More and more of this will happen as network and communication technology makes it easier to directly connect people that have questions with people that have answers. How much more effective and efficient can this be if across this mass of people is diffused even some basics about teaching and training? What if we were able, as a community of learning professionals, to disseminate enough of what we know to raise the effectiveness of the average learning conversation by even 5%? It really doesn’t take much; quality is probably more important than quantity here. Spread a few key basic principles and best practices widely over time, and give people a chance to practice so that they take it to heart. Overall, cumulatively, it adds up to a large return on investment.
So the question becomes, how can we do this?
What sorts of basic principles and practices should we focus on diffusing widely to the population?
What are the key bullet point takeaways?
What sorts of technology/media should we use for this purpose?
How can we evaluate the effectiveness of our efforts and recalibrate efforts on the fly accordingly?
Adults, particularly older ones that didn’t grow up playing games, tend to be somewhat dismissive of the merit and value of gaming. It’s a toy for their kids, a distraction rather than something to take too seriously.
But today, electronic games are also a serious pastime for working adults as well, and for men and women. As well, modern gaming systems are basically sophisticated computers for the living room that handle a range of different forms of entertainment – TV, music, movies, gaming, apps, web browsing.
Some interesting stats:
The average age is 31
The gaming population is roughly split between men and women, 52% vs 48%.
The average US household has at least one gaming console
So the question becomes, what do video games have to offer us, as learning professionals, beyond fun and entertainment?
Video games have lessons to teach us, as learning professionals in a few key ways.
The Lessons of video games
1. First, they show how to make complex, realistic tasks fun. People play games because they’re fun. There’s struggle and effort and challenge and obstacles to overcome, and often a lot of seriousness, but in the end it’s fun.
2. Second, video games are distinctly successful at attracting and holding people’s attention. People engage and stay engaged for sustained periods. Modern games can take upwards of 100 hours to complete. This is comparable, order of magnitude, to the time investment for a professional pilot to earn a type rating to fly a new type of aircraft.
These games will often involve intricate levels of goals and objectives and sub-goals and side goals. But people are drawn to persist and complete them. Games play effectively with different motivational and reward pathways to produce a compelling need to persist at the task despite challenges and difficulties and sometimes because of these challenges and difficulties.
These are some the same sorts of problems we face as designers and developers of learning materials. How to motivate people to engage in our content, and how to keep them engaged through to the end of the course. The success of games in this regard and the ways games achieve this success should be a motivator for us. We should look at the tricks games use to keep us interested and try to apply the lessons to our learning materials.
3. Third, video games are at the forefront of interactivity. If we want to see the state of the art of near / present term feasible interactions, we need to keep up with games. This is where the innovation is happening. Have you seen modern game controllers used with gaming consoles?
It looks like you could use it to put something in space. So many different buttons and directional controllers, and all in one neat, ergonomic package that fits perfectly in your hand.
Off-brand Xbox 360 controllers (similar to the one shown on the left) are used to fly military drones. Playstation controllers (the one shown on the right) are similar in design.
Microsoft’s Kinect sensor, meanwhile, used for the Xbox, is arguably the most innovative consumer market man-machine interface out there, using multiple cameras and microphones to detect body position and orientation and take speech inputs. It basically lets you, given effective software, control a computer using your voice and the movements of your body.
And then of course there are some of the novel forms of interaction used for tablet and phone games. Touch is one example, with controls based on tapping, swiping, and pinching. The other is controls based on the accelerometer and gyroscope, where games are controlled by moving, shaking, turning, or tilting the device. Some good examples are the Sky Map app on Android and Super Monkey Ball 2 and the Labyrinth 2 on iOS.
Finally, up and coming gaming tech such as the Oculus VR look to take gaming to a new level of immersiveness.
The common denominator here is that effective and meaningful use (not just as a gimmick) of sophisticated interactivity helps to pull the user in and increase engagement.
In comparison, the built in types of interactions we see enabled in eLearning authoring tools like Storyline and Captivate tend to be very simplistic – multiple choice, drag and drop, matching. Video games can motivate and inspire us by showing what is really possible today in interactivity.
Video games, far from being something only for kids, have become a serious entertainment industry enjoyed by people of a wide variety of ages. They are an established part of our modern lives. Learning professionals looking for ideas on new and better ways to engage and motivate learners should take a serious look at the best practices of modern video games.
The past decades, and particularly the past decade, have seen a rapid development of science, technology, the economy, and society. This has dramatically changed the way we live our lives and interact with technology, and with each other, and with how pieces of technology interact with each other. These developments have generated a lot of instability, but also provide a lot of potential opportunities for those of us with expertise in structuring content for learning. Whereas personal computers were a rare luxury 40 years ago, and only really started to become a normal household item perhaps 25 years ago, most people today have several computers, and spend almost every waking minute within the orbit of some interactive computer. Every waking moment is potentially an opportunity for learning. It is a golden age of opportunity for learning professionals. The challenge is to recognize and seize it.
The learning community has embraced this to some extent in recognizing the need to build materials for mLearning on phones and tablets. Unfortunately, this is too often done superficially. Rather than designing from the start to take advantage of the unique affordances of mobile devices, it becomes a matter of “export to HTML 5” and “put it on the iPad.” But, looking on the bright side, at least there is that early recognition. Authoring tool developers have embraced the philosophy of responsive web design that adapts layout and presentation to different screen sizes and resolutions. That’s good, but only scratches the surface. We know from mobile games and virtual assistant technology like Siri/Google Now/Cortana that mobile devices go beyond simply having touch input and different screen sizes. There is speech recognition, there are accelerometer and gyroscope inputs, GPS and location awareness APIs. I haven’t seen learning applications begin to tap these capabilities. We really need to look at mobile and console gaming for lessons about the cutting edge of exciting interactivity.
Part of the problem has been the previous restrictiveness of LMSes and SCORM to be able to track learning activity. You had to log into an LMS account and take some rigidly structured eLearning piece through an LMS player in a browser if you wanted to keep track of the learning. The next generation of technology, Tin Can API / eXperience API is supposed to open this up substantially, at least according to the hype, allowing us to track pieces of learning of different types taken through different platforms in different places. This ties in well, potentially, to ideas of informal and just-in-time learning.
Looking forward, five years down the road, ten years down the road, our challenge becomes even more daunting. We need, as learning professionals, to expand our perspectives beyond this to look at a broader set of environments. We need to recognize the full spectrum of how we interact with computers today, and how we will in the years to come. We need to fully embrace our philosophies of learning and training through continuous professional development, learning while doing, informal learning, mobile learning, and continuous learning.
This should be motivated both by our inherent desire to find new ways to teach and train people, but also by motives of professional self-preservation. A move toward more informal modes of learning, ready, ubiquitous access to quality cameras and editing software, and the availability of relatively affordable rapid authoring tools and screen capture tools make it more plausible for subject matter experts to “cut out the middle man” and build “good enough” training materials on their own. In such an environment, in many cases only minimal coaching on eLearning design principles from Instructional Designers would be needed. If we want to maintain a market for our skills, we need to be adaptive to changing realities by casting a broader net. We may also need to recall the lessons of our university studies in Human Performance Technology, and broaden our focus from training and learning to a more general approach of performance improvement and support. Giving people knowledge, skills, and support to live happier, fitter, more productive lives.
A few different areas are potential spaces to target. Some of these areas I’ve talked about before, while others I plan to break out later in separate articles. Note that these technologies are still in early stages, and may not yet be feasible as platforms for learning. But the world of technology moves increasingly quickly, and we need to look ahead and be ready if we are to seize opportunities and stay relevant.
I will speak here about three different directions:
Augmented Reality and Wearables
Smart TVs, gaming consoles, and the connected home
In-Car Information and Entertainment Systems
Augmented Reality and Wearables
One area that looks ready to grow over the next few years is the area of augmented reality and wearable computing.
Augmented reality is a technology in which displays are aware of location, orientation in space, surroundings, and other contextual information, and use this information to display relavant details on top of the field of view. Google Glass is an example of an early effort in this area. The term is derived from the term virtual reality, with the “augmented” part getting at the idea that instead of replacing your view of reality, it adds (hopefully) relevant information on top so that you can navigate the real world but with computer aided supplementation. I talk more about the possibilities of augmented reality elsewhere.
Wearable computers are devices with sensors and input/output devices that people can wear. Two directions manufacturers are going in are watches and wristbands. Some examples are:
The beauty of wearable computers is that they go with the person everywhere, are always accessible at a glance, and are in contact with the person’s body. The devices have screens to display information, run apps, measure heartrate, in some cases take voice inputs. They can also connect to smartphones and other computers to transmit collected data and receive communication like email and other messages and receive notifications.
These could be and are being linked into fitness and health-related training, collecting information on distance travelled, heart rate over time, elevation over time, and other data, and sending that information to a data analysis program on another computer to summarize and organize the data. Notifications would also be useful for health related applications, helping give reminders on therapeutic activities or on when to take medications. Devices with a touchscreen display could also conceivably engage seniors periodically in little mental exercises to test and promote cognitive sharpness.
Conceivably, such devices could also be used for language learning and support. Something on the wrist that could respond to voice queries, perhaps through twinning to smartphone virtual assistant software, could be helpful in “live” real world situations where someone is practicing a second language.
They could also be an effective platform for performance support in a range of situations to give useful prompts. Something that is right there on your wrist is handier than a phone that needs to be taken out of the pocket. Particularly if you’re looking for support while your hands are busy.
Smart TVs, Gaming Consoles, and the Connected Home
The home also becomes a space where tech platforms will potentially be able to support seamless learning and/or performance support.
One place is on the TV, whether via smart TVs or latest generation gaming systems connected to the television. I have talked about this at some length already, particularly in respect to the Xbox One and Kinect. Again, Apple and Google are also trying to get into this space with Apple TV and Android TV.
Another type of platform that could be leveraged for learning and performance support applications is the connected home. This is related to the idea of the Internet of Things, and is about various intelligent appliances and sensors in the home linked together on a local home network, and possibly connected as well (with proper security safeguards) to the larger internet.
This is an area that is still pretty primitive, outside perhaps the sci-fi homes of tech millionaires and billionaires and certain early gimicks like internet connected thermostats. The security and interoperability challenges alone are daunting. But the vision is compelling, and it’s inevitable probably that some systems along these lines will be developed in the coming decade.
In a connected home, devices are spatially and contextually aware, always announcing their presence, always open to communications with people or other devices, and a person or multiple people is almost always relatively near. Smart software could theoretically interface between you and the embedded components, but could also manage the environment for you behind the scenes .
These could potentially be great platforms on which to deliver or facilitate learning and performance support in creative and fun ways.
The idea would be for software running on the system to keep track of you and your family and to find moments in the thick of life for you to learn and better yourself.
For example, to aid in such things as cooking and organizing an interesting meal plan for your home given your inventory, nutritional and diet goals, budget, and time/equipment available. To suggest new ingredients or new recipes using ingredients you already use a lot.
To monitor your patterns of sleeping, bring patterns of concern to your attention, and help to support you in establishing better patterns.
Or to teach people and support them in exercising and fitness. Tracking estimated calories burned, distance run/walked/biked/swum, times, and perhaps linking that into nutritional data collected. Also, providing virtual coaching assistance, suggesting new workouts and providing targetted encouragement based on psychological principles of motivation (both in terms of initiating behavior and promoting persistance).
Or to support learning and playing with the kids. For example, I can envision a future where school boards embrace technology, storing gradebook data online, with results tagged by key competencies. Home-based computer tutoring systems running on a home entertainment console in the living room connect into that via parent login credentials entered by the parent in some authentication process. The system picks up on downward trends in student performance, or troubles on some competency, if the school’s gradebook system does not. The system can then identify the children that are watching entertainment or playing games in the living room or on a computer connected into the same home wifi network and intersperse the entertainment with e-tutoring content targetting areas of current difficulty. Such software could do this for you while the parents are busy getting dinner ready. Simulataneously, the system is walking Dad through that new Thai recipe it suggested via a computer monitor / TV screen in the kitchen.
For supporting families in managing finances and suggesting ways to optimize to save money.
It will be interesting to see to what extent the work companies like Microsoft, Google, and Apple are doing with virtual assistant software (Cortana, Google Now, Siri) could be embedded in these connected home systems as a front-end interface.
Find ways to get your data and learning services connected into that experience. A mix of education and entertainment, but without mixing the two up. Make it fun and exciting, but keep to the realm of the true.
In-Car Information and Entertainment systems
Cars are being put out with increasingly sophisticated information and entertainment systems that serve purposes of safety, navigation, and entertainment. It’s not unusual for people to spend 10 hours or more in their car every week. This is a sizeable amount of time that could be leveraged for learning. People already traditionally engage in light learning activities; they listen to CBC/BBC/NPR/Sirius, they listen to audio books or TED talks, whether on a CD or from a connected smart phone. But more could be done by linking deeply into and running on top of in car systems.
Apple is trying to get into this space with CarPlay, which basically allows certain properly programmed iOS apps to have in car functionality through the in car entertainment system. (Apple is coordinating with car manufacturers on this) Google is also doing work on extending Android to interface with in car systems.
Or, as a funny example, there was an episode of Big Bang Theory where Sheldon reprogrammed Leonard’s GPS to play Sheldon’s personal narrated sightseeing notes at key points along a route. This was a gag about Sheldon’s neuroses, but an imaginative designer could probably think of some legitimate applications along these lines. For example, a rented car in Europe could bring up historical information about landmarks and towns.
More immersive and interactive learning materials could be developed for in car learning for children, as a way to get homework out of the way on the way home from school, or to pass the time on long road trips.
How far you can take this for adults is limited currently by the fact that the driver should not be distracted from the key task of driving. But the possibilities of the car as a moving information and entertainment system will become more plausible as autonomous car technology continues to evolve.
A lot of this technology is either in the early stages or not quite ready for prime time. Nevertheless, looking at the trends, it seems clear that these are some of the major directions things are going in our ever-evolving and ever more intertwined relationship with computers. Further, the best results will come when the solutions in these different directions can interlink with each other, sharing information, so that the wearable systems talk to in-car systems talk to connected home systems.
But one step at a time. There is a lot of work to do.
To the strange and wonderful future that awaits us.
There’s an interesting trend in media consumption that has come about with the advent of Netflix – binge watching. The term brings to mind colorful imagery of drug addiction, and some might argue that, to a certain extent, it’s sort of fitting. It’s firmly rooted in the culture now. Netflix bingeing. Watch an episode of a series, and if it’s crafted well, you’re ready to go on to the next one. The next thing you know, half a day is gone.
Or sometimes more:
When Netflix rolled out House of Cards in an innovative new approach where they dumped the whole season at once for streaming, their internal data indicated that at least some small number of viewers completed the season 13 hours after they started. Meaning they watched the whole thing in one continuous session.
When Breaking Bad finished last summer, Netflix and torrent sites saw huge traffic as people rushed to catch up so that they could share in the social experience. It was actually kind of interesting, sociologically, almost a return to what TV was like in simpler days when I was growing up in the 1980s.
Even my sexagenarian parents are getting into it. A few weeks ago, my folks got a new smart TV and decided to take out a Netflix trial. I suggested Mad Men as an interesting series to check out, since they came of age at about the time the series takes place. A couple of weeks ago I get a Skype message telling me they checked the first episode and liked it. Recently, I came home for a holiday visit, and this evening, they finished the fourth season.
What is it that makes us do this? What is the secret recipe?
And it works in other media too.
I was sitting with my daughter recently while she was reading to me. It was a longish book, it was getting late after a long day of holiday play, and reading in English (she is in French immersion) is still a work in progress, so she wasn’t going to finish the book that day.
She initially flipped forward a bit, and said, “I’ll read until here.” When she got there, though, she was still engrossed in the story, and said, “two more pages.” And then again, and again, a few more times, before the heavy eyelids made continuing too difficult.
I was reflecting about this, when a thought came to me. What if we could craft learning/training materials like that?
Each bite doesn’t have to be be big. In fact, shorter is good. It makes each piece less of an psychological investment. Easy to dive in for one more. But make it engaging, with each piece ending on an intriguing note, leaving you wanting more, like some weekly serial picture from the 1950s, but with that next episode not one week away, but right there in a link. It becomes easy to click “next,” and rationalize, “just one more episode.” “Just one more hit.”
We should look more carefully into how these programs we love to binge are constructed; what makes us just keep going.
And then try to apply that to instructional design.
Food for thought.
If anyone has any related food for thought, another dish to contribute to the potluck, feel free to share in the comments below. As a wise man once said, knowledge is the one thing that when we give it away, we gain it.
People lead increasingly busy lives. With work, family, household maintenance, and a dozen other things, our schedules tend to fill up. In this context, it’s difficult to find time to engage our need for life-long learning. At the end of a day or week of work, it can seem overwhelming to sit for a course. The hours required can be hard to fit into a schedule. But the fact is, there is always time left in the schedule. The challenge is that we need to be creative in getting learning to fit into the schedule.
I’m reminded of an old story that you see every once in a while on Facebook or other social media. I don’t recall the exact context, but it involved a teacher making a demonstration trying to fill a jar. He started with marbles, “filling” the jar to the top until no more marbles would fit. Is the jar “full?” No. There’s plenty of space, just no more marble sized pockets.
So the teacher continues with some small beads. You pour them into the jar, and they settle into much of the spaces between the marbles. The teacher does that until the beads are level with the top of the jar. Is it full yet? No. The teacher proceeds to add sand to the jar. The sand settles into the tinier spaces between the beads.
Finally, the teacher completes the demonstration by pouring water into the jar.
So this is a nice story. What can we take from it?
Well, the fact of the matter is that we probably have hours of time each week that could be used for learning. The problem is the time is split up into little pieces between other things, 5 minutes here, 10 minutes there. As well, there are chunks of time in contexts that would not traditionally have been thought of as opportunities to access training material, for example, sitting on the couch in front of the living room TV, out walking in the street, or driving in the car.
As training designers and developers, we can direct our efforts in two major directions:
Making training in smaller, bite sized chunks that will fit in easily and conveniently in spare cycles between other things
Making training that can be readily accessed in non-traditional learning environments/media, including:
a. Living room via smart TV and/or internet connected entertainment system such as a modern game system like the Xbox One or PS4
b. Through an in-car entertainment system
c. Through wearable computers and augmented reality
This leads to some challenges for us as designers and developers.
The first challenge requires us to think hard about how to break down training into small, focused, self-contained learning pieces that are well-indexed and findable. These will often be accessed individually as informal learning materials. Though it may be possible to prompt learners with suggestions about connected content they might find useful, depending on the Learning Management System through which users browse content.
The second challenge requires us to learn about the tools needed to develop for these other platforms, as well as the unique affordances involved in terms of interaction and navigation of the platforms.
Some of these points I have touched on previously in posts here. The others I hope to speak about in more depth in the weeks to come.
Haven’t published any posts in a while; have a lot of drafts on the go, but nothing quite finished yet. In the meantime, I thought I’d make a quick post. It’s been a pretty good week in the Anderson household. At work, got a nice early Xmas present in the form of an offer of a full time staff position at work. (Accepted!) Which was nice. Contracting is interesting, but there’s something to be said for stability.
At home, I was happy to receive a large UPS parcel from Microsoft – a new Xbox One with Kinect.
I’ve been interested in the Xbox, and the Kinect in particular, for some time. I’ve written a few articles on this blog talking about the potential of Kinect and sensor technologies like it to expand the possibilities of training, and computing in general. The prospect of predictable horizons on the work front and a sweet holiday sale on the Microsoft web store helped to seal my decision to take the plunge.
So far, I’m quite pleased with the Xbox One as a product. It’s my first time buying a gaming system in about 20 years; the state of the art has definitely advanced since the days of the Sega Genesis 😉
The setup is easy and smooth. The regular controller plus the voice and gesture based interface of the Kinect allow multiple ways to get things done in terms of navigation and interaction.
The voice and gesture controls are introduced via simple tutorials. A fine example of quick, simple tutorial materials.
The voice controls work nicely. My son and I have had no trouble being understood, and the system OS includes visual cues when speech input is activated as to what commands can be made on any particular screen. It doesn’t quite understand my daughter, but she’s a little younger and missing two of her front teeth.
The gesture control is cool too, though it will take a bit of effort to get smooth and precise with it.
I also ordered the optional adapter to plug the Kinect sensor into a USB connector of a Windows computer for Kinect for Windows apps. This is a cool recent development; previously, to play with Kinect on a Windows computer, you had to buy a special $200 Kinect sensor specifically for that purpose. Allowing people to just use the sensor from their Xbox One opens things up another notch.
I’m looking forward to playing around on a hobby basis with the free Kinect For Windows SDK. Basically, if you have an Xbox One with Kinect, and you buy the adapter, the license is open for you to make Windows apps, even commercial apps, for no extra charge. Kudos to Microsoft for being smart and removing barriers to innovation and experimentation
The SDK, along with lots of guidance and tutorial material available on their website:
I’m excited to explore what sorts of interactions can be made with this technology, with an eye to training applications. In particular, the ability of the Kinect to recognize body movements, facial expressions / emotional states, and even heart rate could potentially add a lot to monitoring learners level of engagement and their performance on motor tasks.
Also on the menu is a Udemy course on Web Development I signed up for a number of weeks ago when it was on sale. (Which I hope to finally get started with over the holiday period!) Together, these two hobby projects should take up a good bit of my spare time over the coming months.
I look forward to sharing anything helpful I learn along the way.
When we’re driving around, I often like to remind my eldest son that by the time he turns 16, in the summer of 2021, driving will probably be, or be in the process of becoming, an obsolete skill. (In the city and on the major highways, at least) Driverless cars are in the workable roadworthy prototype stage, and an array of performance support tools are being rolled out in newer cars. These tools serve to guide drivers, augment their awareness of their surroundings, and improve safety. One of the posts I currently have in draft is on the parallels between aviation and driving in terms of increasing automation and performance support. Experts tend to predict driverless cars as becoming a common reality in the early 2020s.
In the meantime, people will drive, and young people will need to learn to drive. But is it being done well? Or as well as it could. Safety equipment in cars and in the engineering of roads has reduced overall accidents and fatalities, but driving remains a relatively dangerous mode of transportation.
Young people entering into the world of driving at age 16 (and new drivers in general) need solid training to stay safe. When I turned 16 I went through the Young Drivers of Canada program. The basic program is similar to high school driver’s education, but with more emphasis on practical skills of situational awareness, threat monitoring, and collision avoidance. The training consisted of a classroom component of about one week in length, and a number of in car driving lessons. The classroom time was for teaching of theory, and consisted largely of lecture, some discussion, and watching of instructional videos. The road lessons in the car allowed practice of skills with grading and feedback by a trained instructor.
I went through this training close to 20 years ago now. This intervening period has seen a plethora of new information and sensor technologies arrive on the market. Oddly, the domain of driver education does not seem to have evolved appreciably in terms of approach. It still seems to be classroom plus supervised road lesson.
Looking at this through the eyes of an Instructional Designer, I see some gaps here in this training approach that, if filled, could lead to more effective learning, improved safety in early experiences on the road, and improved transfer and retainment of skills after the initial training is completed.
The classroom training and road practice are a good foundation, especially within a framework of defensive driving that includes threat monitoring, preventive measures, and emergency maneuvers / collision avoidance. These are necessary components of a strong foundation for an effective training system, though I’d probably advocate exploring some form of blended approach for teaching theory materials. Having some material delivered as online video or eLearning/mLearning content would allow for better flexibility of the learners, particularly adult learners. But overall, the theory and hands on supervised practice are good components of a foundation.
That said, I see two places where additional training could fill a gap and potentially improve performance.
These gaps are:
The need for some intermediate training to bridge the space between classroom (or online) theory and practice on actual roads. There is a wide chasm of experience between the cognitive activity of learning theory material in a classroom and the complex psychomotor skill of driving. Something in between would help to bridge this chasm and soften the transition.
After the course is over, there is a need for some level of electronic performance support while the young driver is out on his own. This is needed to scaffold the learner in applying the skills safely while they are still new, until such point that the skills become internalized, and automatic.
In regards to the first gap, let’s take the example of aviation. When an airline pilot is doing type training on a new aircraft, the pilot does not go directly from the classroom to flying the actual plane. First, the pilot spends time training scenarios on a simulator device. Simulation based training offers a lot of benefits. It is completely safe, but feels psychologically real when the different types of fidelity (the sense of realism of the simulation) are high. There are different dimensions of fidelity – (1) sensory perception/look and feel, (2) the process of operation of the simulated equipment versus real, and (3) the dynamics of the simulation, the relation between action and results. It can be less expensive to operate a simulator than the actual equipment, at least once the initial investment is made to acquire the simulator. You can also control the scenario conditions to have focused training and avoid the problem practical training experiences being ad hoc, depending on the random conditions of the day.
This first gap then could be addressed through some sort of electronic driving simulator, similar to how pilots train on flight simulators. I understand that some driver training programs alresady use these, but it needs to be more universal. At one end of it could be some piece of fixed equipment similar to the flight simulator or fixed based trainer, with a mockup of a real driver’s seat, dashboard, displays/gauges, and steering wheel combined with screens to simulate out the window views, and speakers for sound effects. At the other end of it might be something like a realistic game-like driving simulation on an Xbox with a Kinect sensor and the learner’s hands as controller.
Performance support on the other hand could take the form of some sort of apparatus with cameras/sensors and a built in computer set up inside an actual car. It could collect data about speed, traffic conditions, weather, local speed limit, braking and swerving, and signalling. It could track eye movements to look out the windshield, both toward near objects and far objects, toward rear and side mirrors, and toward the blind spot. It could also monitor hand positions on the wheel (or stick, as relevant). The computer could collect data for later analysis (Syncing to a mobile app, for example).
Ideally, it would also give spoken word support cues / prompts / feedback in response to conditions and what the novice driver is doing:
“Remember to check your mirrors” “Look ahead to spot upcoming problems” “Remember to check your blind spot” “Brake!” “Try not to ride the clutch” “A little more gas” “Slow down a bit for weather/traffic conditions” “Accident ahead. Caution”)
Ideally, this form of performance support could be built into future cars as a “training / support” mode. This mode could be engaged at the push of a button on the dashboard as part of the support/automation systems that are increasingly built into modern cars. For now, though, it would have to be developed as a third party device.