Tag Archives: learning

The Potential for Stereoscopic 3D in Education

Introduction

One relatively old technology idea that has enjoyed a resurgence in the last couple of years is stereoscopic 3D. 3D in the cinema is a technology that has waxed and waned in a faddish sort of way over the previous decades. James Cameron’s Avatar (2009), a high budget, highly successful blockbuster film by a top notch action director making artful use of 3D led the most recent charge in bringing back 3D as a trend.

While most examples of 3D in film since then have used the technology in much less skillful ways, the tech is still going relatively strong in theatres, its staying power buoyed by higher revenue from 3D films and aided the appearance of 3D HDTVs and 3D Blu-ray players on the home theatre front.

So 3D has made a comeback over the past four years in entertainment. But what about the prospects for more serious applications, for learning? The subject of this week’s blog post is the potential for the use of 3D content in learning / training.

The post will look at:

  • Factors converging to make more widespread use of 3D content possible
  • Potential learning benefits achievable through judicious and effective use of 3D content
  • Types of subject matter content that would be potentially amenable to instruction through 3D video
  • Challenges faced in making 3D content used in a more widespread fashion
  • Considerations for effective use of stereoscopic 3D in learning and training

First a note that when I talk about 3D here, I am making a distinction between older Pixar-style 3D animation (computer generated animation with lighting effects that make the images look “3-dimensional” but made for viewing on a 2D screen) and stereoscopic 3D, which uses two slightly different images (one for each eye), and glasses (either active with shutters or passive using polarized lenses) to simulate what the eyes and brain would experience looking at a real object, simulating an immersive and realistic experience. This article is focused on stereoscopic 3D.

Factors Leading to Increased Use of 3D

There are a number of developments causing a convergence toward an increased potential to use stereo 3D content for learning. These developments include:

  • Resurgence of interest in recent years 3D in cinema. A mixed bag for quality has slowed the growth of popularity of it in the theatres, but approximately 1/4 of box office dollars still come from 3D screenings. Some artists really have learned to work the distinct visual language of 3D in an artistic, compelling, integrated way and have made films oriented toward use of it that brings added value. Others tack it on as an afterthought, diluting the concept.
  • Increased accessibility and affordability of professional 3D camera rigs, as well as relatively affordable consumer-oriented dual-lens 3D camcorders. Basic 3D capable camcorders are currently in the hundreds of dollars to a little over a thousand range. For a while now, YouTube has allowed the uploading of user-generated 3D content.
  • Accessibility and affordability of TVs and monitors that can display 3D. Though 3D is no longer being used so much as a dominant selling point, the reality is that most newer TVs include 3D as an option by default. A quick search on the Future Shop website shows 3D capable HD LED TVs from top tier manufacturers like Samsung between $750 and $1000. And obviously as technology advances these prices will continue to fall.
  • Affordability of 3D blu-ray players. A similar search shows these devices selling for as little as a bit over $100.
  • Accessibility and affordability of office and school type projectors that can project 3D content. ($1500-2000)
  • Availability of software for editing 3D content (After Effects, e.g.).
  • Increased diffusion of expertise in 3D cinematography – understanding of principles behind stereoscopy and how to compose scenes for 3D video. This is an important factor for ensuring not only a quantity of content, but quality too.

Potential roadblocks

There are several potential roadblocks that need to be overcome, however:

  • Getting access to quality camera equipment
  • The need to develop knowledge of how to properly set up 3D shots and shoot in 3D
  • Hassle of glasses; active shutter type glasses need a power source and are expensive, polarized passive glasses by design filter out half the light, making for a dimmer, less vibrant picture
  • Expense of screens (though this is falling)
  • Limited amount and varied quality of content. Some 3D content is high quality, though much is riding the bandwagon, faddish. Few learn effectively how to use it, working with its strengths and weaknesses, and planning the visual composition of shots from the start with good use of 3D in mind, so as to make something that adds value from what would have been possible in 2D

Potential benefits of 3D

So why use 3D content for learning or training, beyond use as some sort of fad or gimmick? There are several potential benefits, given use of the technique in a meaningful way:

  • Immersiveness
  • Learner engagement
  • Increased level of connection with content
  • Sense of realistic “presence” of what is being seen
  • Enhanced ability to perceive and comprehend spatial relationships and depth in learning content

For learning that involves understanding a place or some sort of 3D structure, with 2D content, it takes more mental effort to interpret and understand the spatial relationships, to connect with the place. Good filmmakers and photographers know how to compose 2D imagery and video to help us with this, but there is still a need to take the 2D and mentally abstract from it, imagining mentally the 3D reality. With 3D, there is not so much imagination or abstraction; the mental processing of it in terms of the experience in the brain would theoretically be similar to how you would process the perceived image of the real thing if you were there.

Visit a location, or a good museum exhibit, and you can see the power of experiencing a location in all its spatial dimensions. It is one thing to read about something and look at a picture, another to experience it in an immersive fashion.

Potential areas of training application

Stereo 3D could be beneficial in a range of different subject matter areas, from hard sciences to life sciences to social science to fine arts. Some examples of where 3D could be useful:

  • Understanding 3D structures and relation to form. For example in hard sciences. For example to understand the spatial arrangement of stars in a galactic supercluster, to visualize a simulation of the formation of the solar system, to visualize the relation between structure and function for higher level protein structure, or of the enzymes that control DNA transcription, to visualize simulations of how weather interacts with geography like mountains.
  • In experiencing exotic or impossible to visit locales. For example, outer space, in Space Station 3D or Alfonso Cuaron’s upcoming Gravity. Or to virtually visit paleolithic cave paintings normally closed to the public in Werner Herzog’s Cave of Forgotten Dreams. Another would be out of the way natural parks or reserves.
  • In experiencing areas of natural splendor like the Grand Canyon. Or more remote areas of natural beauty. Ecologists and conservationists sometimes struggle to communicate the value and beauty of certain isolated natural locations and the value of their preservation. 3D can give a more meaningful sense of what the place is like and why it is special.
  • Along the same lines, to experience wildlife on video in a more realistic, lifelike manner.
  • In experiencing human spaces or manmade sites like famous areas in foreign cities
  • For history to better understand the spatial layout of an area to give a better grasp of how that shaped the experience of some historical event. Pass of Thermopylae, for example, or Normandy Beach, or Dieppe.
  • For anthropology in the study of different cultures and their ways of life and the spaces in which they live. A good part of social science and is in understanding the physical spaces in which the people live. Culture is, in a sense, formed at the intersection of a physical space with technology and climate. 3D transmits more powerfully that spatial dimension, allowing us to better bridge that  separation between ourselves and some exotic culture by virtually entering their living space.
  • Religious studies – to virtually experience the geography of the Sinai desert, or the Mount of Olives, to walk through a 3D archaeologists imagining of the Temple at Jerusalem, to virtually experience Mount A’rafat or the Kaaba in Hajj, or to virtually visit one of the more famous and spectacular Hindu or Buddhist temples
  • Fine arts: To experience art exhibitions, sculpture particularly in distant locations virtually for art education, to visit the studio of a prominent photographer or painter, or to be in the pit of an orchestral concert.
  • Experience of a first hand point of view for relatively recent history and contemporary human events as live footage gets captured in 3D
  • In architecture or structural engineering for collaboration in the design of structures by looking at 3D models in 3D

Important Training Considerations

In order to promote effective learning or training using 3D content, there are several issues to which attention will need to be given:

  • Ensuring effective cost-benefit. If investments of money and time are going to be put into buying equipment and learning how to use it, there needs to be a return on investment in the form of more efficient and / or more effective teaching / training. It has to be more than a fad or gimmick done “because the technology is there.” There has to be a compelling learning case for any particular usage.
  • Avoiding it being mere “edutainment” where it’s cool to watch but learners don’t gain anything toward the actual objectives. We need to recognize when 3D legitimately gives worthwhile added value that justifies it and when it’s just decoration. There is a difference between content simply being entertaining and content being educational. (Though of course, the ideal is for it to be both!)
  • Finding quality content. This is a major concern right now. There is currently a kind of catch 22 for example with 3DTV where people don’t watch a lot of it because there are hardly any channels with content and new channels are slow to appear because not many are actively watching.
  • A significant portion of the population (some estimates go up to 25%) experience negative effects / discomfort from watching 3D content, ranging from headaches to nausea. This comes from a mismatch between how the eyes must be oriented to experience the 3D effect in focus (both eyes looking straight ahead toward the screen, parallel to each other) and how the eyes naturally tend to orient themselves based on where the brain is telling them the image is (eyes rotate inward so that they converge on the object in between). This strain, however is usually experienced from watching a whole feature length 2 hour film in one stretch. This effect would likely be less noticeable in viewing sets of 3D clips with breaks in between, as would be the case with using 3D educational materials.
  • The need to establish a sound, rigorous research grounding of principles to guide best practice. There has been a relative shortage of hard research on establishing what works educationall in 3D. There are some research studies showing positive and significant differences in amount learned between groups when content was viewed in 3D as opposed to 2D. However, more rigorous research, as well as creative informal experimentation by training professionals and enthusiasts needs to be done. People need to generate content and play with different variables and see what types of content benefit in a meaningful way from a 3D treatment and which don’t, what factors augment or hinder the educational impact of 3D, how much of measured impact is attributable to simple novelty temporarily enhancing attention, and how much to real improvements in the ability to get ideas across. The increasing availability of affordable, quality 3D capable camcorders and screens on which to show the content should enable this by opening production of content up to education departments, training departments, and “amateur,” YouTube type enthusiasts.

In Conclusion

A number of factors are converging to produce an environment where stereoscopic 3D is increasingly a viable approach for some training content. Well constructed 3D content can potentially bring learning benefits as part of a properly-designed training and learning solution for a range of different subject areas. Proper attention has to be paid however to ensure that 3D content is used in a thoughtful, principle-based way so that it brings legitimate ROI rather than simply being a gimmick.

Interesting Links

http://jaredjared.com/wp-content/uploads/2011/10/Bendis-Stereoscopy.pdf

http://www.indeptheducation.com/wp/blog/

http://www.iste.org/connect/special-interest-groups/sig3d

http://planet3d.org/category/education-and-3d/

http://www.videomaker.com/videonews/2013/01/3d-video-boom-or-bust-in-2013

http://www.3dfocus.co.uk/3d-news-2/exclusive-3d-as-standard-as-the-power-button/11680

http://www.xpand.me/education/

Logical future directions of stereo 3D technology:

  • Combining 3D video capture with body movement based Natural User Interfaces (NUIs), interacting with a 3D imagery via physical gestures
  • Capturing and streaming 3D video to enable Live 3D virtual tele-presence. Some companies are already offering such setups for corporate clients.
  • Incorporation of 3D into mobile or wearable devices. Imagine a miniaturized dual lens 3D capture set-up in future versions of cellphones, tablet, or Google Glass-type augmented reality wearables with dual cameras and a 3D display in the glasses a few years down the road. Meta glasses are one example of projects working in this direction.

Augmented Reality and Wearable Computing: Possibilities for Google Glass in Training

Introduction

Hello, happy Friday and welcome to my  blog. One of my main objectives with this blog is to encourage innovation in training by taking a “skate to where the puck is going” perspective, looking at new technology coming in the not-too-distant future and looking at ways that the technology can enhance training. Today’s topic is augmented reality and wearable computing with a focus on Google Glass. To be enjoyed with a nice Friday morning coffee or tea at the desk. 🙂

NOTE: I am not affiliated with Google or Google Glass and this article is not intended as a promotion of Google Glass in particular. I have chosen to reference this particular technology as a relatively well known example of the technology that currently exists and will be available in the near future on the market.

Augmented Reality

One interesting contemporary trend in computing is so-called augmented reality. What is augmented reality? Augmented reality differs from virtual reality. Augmented reality provides an “overlay” that augments or extends one’s appreciation, understanding, or navigation of the real world rather than an immersion in an unreal virtual world. The user operates in the real world as normally, but with useful location or context dependent information feeds displayed in his field of view to assist him in whatever he is doing.

One example of augmented reality is the HUD, the heads-up display on aircraft. A projector behind the pilot projects display data onto a screen between the pilot’s head and the windshield. The artificial horizon line of the attitude display overlays the actual horizon and various pieces of key data are also projected onto this overlay layer.

An HUD also features prominently as futuristic concept art in the Ironman series of films. (Image included under fair use educational / commentary usage)

Some apps for mobile phones play with augmented reality as well showing overlay information about local points of interest in the immediate area overlaid on the camera preview image shown on the phone screen. The app uses data from the location, the compass, and the gyroscope to sense where the user is, and in which 3-dimensional direction the user is pointing the camera to update the augmentation.

Wearable Computing

Another trend is that of wearable computing. This involves devices with computing power and integrated sensor devices to collect data about the wearer and his environment (GPS position, orientation of body in space, velocity, acceleration/rotation, direction facing or direction of movement, as well as data feeds like local temperature and local points of interest ).

Some examples are Pebble Watch, Samsung Galaxy Gear watch.

Google Glass: Upcoming augmented reality wearable computer

 

Google Glass is an upcoming device (not yet commercially available, in beta) that is receiving favorable reviews in beta testing by participating individuals. Google Glass is a relatively lightweight pair of glasses with an integrated wireless internet connected computer. The device features embedded sensors and a small projector that projects imagery into the field of view of the wearer so that it overlays reality. It functions as both a wearable computer and an augmented reality device. There is a touch sensitive area on the side of the glasses near the user’s right temple to allow touching to initiate actions. Also, there are speakers to hear audio, a microphone to record audio and take voice commands, and a camera to take pictures or record video. You can see a video of the Google Glass in action here.

Glass features

The user can watch videos, see pictures, take pictures or record video or audio, make verbal commands or queries, search the internet, share pics/audio/video to social networks or emails, send dictated IMs, and enable video conferencing (with the user seeing the other person and the other person seeing the user’s POV (point of view) ). There is also the ability to access Google services such as Translate.

Training and performance support applications of Glass

In this post, I’d like to describe some of the conceivable training and performance support applications of this great new tool. Some of these possible applications are based on known out of the box capabilities of Google Glass, while others are reasonably foreseeable possibilities given the capabilities of the device and assuming creative effort on the part of app developers.

Technical Training, equipment maintenance

Google Glass could be very useful in technical / equipment maintenance training.

  • As a means to collect easy, hands-free POV  video of an expert / SME demonstrating how to fix an issue with the equipment or perform some procedure. This video could be streamed live, or could simply be a way to record video clips for use in online help or formal eLearning.
  • As a means to collect POV video capture of the trainee performing the task while streaming the video to an expert. The expert observes, and gives verbal, and possibly video feedback over an audio/video conferencing connection, possibly through another Glass, and potentially at a remote location.
  • As a visual support for component identification and access to more detailed information. Camera image recognition could recognize equipment, and overlay 3d Autocad or Ngrain image with labels on components. It could also potentially enable easy link through to online technical documentation formatted specially for viewing on the Glass’ screen.

Soft Skills training

On the other side of things, Glass could also be very useful in soft skills training.

  • POV video capture of user performance in a role playing simulation could be used for review in post-simulation debriefing sessions
  •  Or, to turn things around, in the same role playing simulation, you could have someone other than the trainee wear Google Glass and record. This would be good for client-facing skills training. The learner can see himself and his performance through the client’s direct POV. Either the video could be recorded and reviewed after the session, or both people could wear Glass, and establish a video-conferencing link. That way, the trainee could perform the simulated interaction in the scenario, while receiving live feedback of how the client experiences the interaction.

Performance support for someone working in a people-centric / client facing position

Certain professions have a much higher emphasis on meeting people, making and nurturing contacts, and all-around growing and maintaining a massive “mental rolodex.” This could include politicians, public relations or publicists, sales force, talent agents, etc. The people who go into these sorts of fields often have unusual talents for this, but everyone has his cognitive limits. We remember the face but not the name, for example, creating socially awkward moments.

A wearable eyeglass computer like Glass could help as a performance support. When a person comes into view, facial recognition could be carried out and the face checked against the contact database. This then brings up useful reminder data – photo, name, company, age, and any other useful or relevant information, allowing a smooth start to the conversation. Glass could also allow an easy way to photograph business cards and automatically (via OCR) extract information to import to contacts.

Language Learning

Augmented reality layers could make useful scaffolding for language learning support apps tying into using Google’s impressive tools for image recognition, speech to text, and text to speech. Text and audio overlays could provide helpful support information to the learner, and this could be used either in classroom practice, or out in the real world. The support could be optional as a scaffold with the ability to turn on or off as the learner feels the need. This could conceivably involve:

  • Live OCR (optical character recognition) and live translation overlay of signage or written material (reading store displays, street signs, restaurant menus, etc)
  • Live speech to text of foreign language to text translation overlay on screen
  • Live suggestions of phrases to use in conversation, with spelling and pronunciation cues

As well, Glass could enable course activities or assignments where the learner goes out into the community and records himself practicing the new language in a real situation (go buy something in a store and talk to clerks, ask for directions, etc). The conversations and interactions could be recorded and reviewed or graded afterwards. The world can become a language lab.

Historical site and museum interpretation

The technology could also find great usage in historical site or museum interpretation.

The user could borrow/rent a pair at the entrance or visitor’s center, and use them to experience a transparent overlay of video or 3d animation based on location. This would provide the experience of being there and then. This could be used at the sites of famous battles like Waterloo, Civil War, WWI and WW2 battlefields, Plains of Abraham, Revolutionary War, etc. Or at old historical ruins like the Colliseum or Acropolis. Or a natural history exhibition site could show what the location looked like in the Jurassic period, or give a visual sense of what the glaciers would have looked like in the past ice age.

Similarly, it could be used in museum exhibit interpretation. The user borrows/rents the glasses, comes to a display/exhibit, and the Glass detects the location and makes an audio-visual presentation available over wifi.

Operational performance support

Glass could also potentially offer performance support for operations of equipment like airplanes and cars.

In aviation, for example, this could enable hands-free, eyes straight ahead checklists as an alternative to glancing downwards at the checklist display on a cockpit Multi-function Display (MFD). Glass, receiving a wireless feed from the airplane, could display the current checklist item, upcoming checklist items, and any special cautions or warnings for steps. Visual annunciations could also potentially appear on the display.

It could also be used in driver training or driver performance support, for example performance support as a reinforcement for defensive driver training. The user can opt for supportive prompts or possibly the glass monitors the driver’s point of view and traffic via camera and gives scaffolding prompts until the learner reaches proficiency. For example: it could prompt the driver to check mirrors periodically, prompt to check the speedometer periodically,  provide a visible cue if the speed calculated from GPS and accelerometer exceeds what is known from GPS/Google Maps to be the posted limit by a some threshold (10-20 km/h above, say), provide a visible or aural prompt to encourage attention or slowing if red brake lights are seen far ahead or it is detected that a car far ahead is otherwise slowing or if the following distance behind the car in front falls below some set threshold.

Conclusion

These are just a few possible training or performance support applications that can be imagined for Google Glass. Doubtless others can and will be imagined and realized as the technology rolls out commercially.

Feel free to leave a reply to share your comments and your own ideas.