Quora: An online learning economy

Introduction to Quora

We are swamped these days with a variety of social networking sites and applications, all vying for our limited attention and time. Facebook, Twitter, Google+, Tumblr, LinkedIn, Pinterest, Reddit. Given this reality, people might groan a bit to have yet another one hoisted on them. Nevertheless, I want to introduce you however to another, not so well known social network and online community, Quora, that I think is well worth your time.



How does it work?

The basic mechanics of Quora are pretty simple.

People post questions. The questions typically consist of a short one sentence statement of the question, a more detailed description that establishes the question context, and user-added topic meta tags.

People see the questions, and can edit questions to improve them or they can provide answers.

Users upvote or downvote answers, and get credits for good answers.

You can choose to follow certain general topics. This will customize what sorts of content you want to see questions for, so that personally interesting content pops up to the top of your stream when you open the site or the mobile app.

follow_quoraThere is a social aspect where you can follow people’s posts or be followed, as well as exchange messages. You will initially discover interesting people through what surfaces on your main feed from your chosen topics. Some person has an interesting or insightful answer on a topic that interests you. You can check out their profile and see their other activity, including their questions and answers, what questions they are following, and which answers they have liked from other people. In this way you discover interesting people. When you follow them, the activity of these people also surfaces on your main feed, which can further point you toward interesting people, topics, and questions.

Quora also hosts very basic blogs where you can write longer form content that followers or other members of the Quora community might like.

A knowledge economy with reputation as currency

What makes Quora particularly fascinating to me as someone in the learning and training industry is that Quora is not just a vibrant learning community, but also a sort of “learning economy” with intellectual reputation as a sort of alternative virtual currency. (I recommend the book “Rethinking Money” for a fascinating take on the history, and potential future, of alternative currencies.)

This reputation currency is tracked in the form of credits. You can build these up by providing good answers that peoplequora_credits like. People can upvote answers they like, or downvote those they don’t. This gives you credits, ten for each time someone votes you up. This serves a number of purposes. First, it gives some reinforcement to encourage people to contribute quality posts, which benefits the community as a whole. Second, it also serves as a sort of “market signal” from the community giving you feedback that the particular post was appreciated by the community. You can analyze your post, see what sorts of answers work well. By learning what sorts of answers are in demand, you can supply more and earn more credits. But perhaps the most interesting thing, and what sets Quora apart for me, is what you can do with these credits.

These credits become a virtual currency within the site. You can then use, or “spend” these credits for different personally or altruistically useful things within the system:

  • You can pay to promote a question, whether your own or someone elses, to other Quorans. 1000 points, for example, will promote a question or answer to a few hundred people. This helps to get exposure for your question or answer, and also gives you a chance to promote someone else who is deserving of attention.
  • Paying a notable expert to answer a question, either your own or someone else’s. All users can set “prices” in credits to answer. Most users you can ask for free, but the pay to ask feature lets the more prominent and distinguished community members who would otherwise be inundated with question set a virtual price for their time. Often these prices are just to filter silly or trivial questions. Many users are pretty cool about this, and have a stated policy of refunding the price afterwards for legitimate questions.
  • Transfer credits to another person as a generalized gift or reward


Related to all this, Quora is also a wonderful networking opportunity, a way to make serendipitous contact with smart, interesting, accomplished people from all over the world. You can discover interesting people when they vote up your content and vice versa. You can trade follows. You can make initial contact with people, for example by posting an interesting question and asking specific people to answer. You can also privately message people when appropriate. This can lead to conversations and possible referrals to new opportunities or contacts in your field of work.

It is a useful personal branding or promotions tool. You can invest credits to promote people seeing your questions. You can invest credits to get knowledgeable people to answer your questions and gain visibility.

Tips for use

As with any tool, it takes some time to get familiar with how to use it. While I am by no means an expert at using Quora, I’ve discovered a few things from poking around with it and from tips others on the site have shared.

  • Follow a lot of topics, and a diverse mix. When you initially sign up, you can select from a basic list of topics, and can also search for particular topics. You can also add new topics to follow at any time.
  • Actively read a lot of questions and answers. You’ll learn a lot, discover fascinating people, and get a sense of how to write a good answer.
  • Actively vote up good answers that meet a certain level of quality you find high. This encourages good answers.
  • Follow interesting people you come across.
  • Reciprocate by following interesting people that follow you.
  • Ask good quality questions! That is, after all, the lifeblood of the community. Just make sure to run a search first to make sure it wasn’t already asked. That way you build the site rather than adding repetition. This can also perhaps help to resurface a question someone posed before, but which wasn’t answered.
  • Add topic meta tags when you submit your answer. Quora will automatically suggest some tags and you can add your own. This helps to assure that the question will be seen by people who will find it interesting.
  • Make sure to come back and fill out a detailed description for the question after submitting the one sentence question itself. The short form of the question is what will appear in people’s feeds, so work on making it brief but compelling. But the detailed description gives you more room to clarify the context and background for those who click through to see your question and perhaps answer.
  • Use @ mentions in answers to crosslink to other questions, answers, topics, or other people
  • Use the blogging feature to post longer form expressions of your views
  • Fill out your profile fully, and include links to any of your websites, to your LinkedIn, to your Facebook (so that they can read your public posts)/ Twitter/Tumblr. Tell people some background about yourself, to humanize you a bit, so that people get a sense of where you’re coming from. This helps people in making decisions as to whether to follow you.
  • Also, as you build up a good bank of answers, you can get “long tail” effects on your answers by curating a short list of what you feel are your best answers, and link to those top 5 say in your profile.
  • Consider injecting elements of your personal background and story into your answers.
  • Combine approaches with skilled timing. Ask a good and novel question in a topic, and at the same time, post up a longer form blog post about a related topic in the same general subject area. And then invest some credits for someone with influence and reputation to answer the question.
  • Be respectful and judicious with private direct messages. Some of the more prominent users (this does not include me, so please feel free to fire away) are swamped with ask to answer requests and other notifications. If someone has 10 000 followers, for example, probably better to start off with less direct contacts like upvotes, following, asking to answer, and then direct messages only if a conversation naturally develops.

Here is the link, by the way, to my Quora profile:


Some articles on MOOCs

I wanted to share some informative articles on Massive Open Online Courses (MOOCs) from University Affairs, a Canadian Higher Education publication. Has a nice balance between widespread accessibility and a nice critical approach. I’m including links to six articles here going back over the last two years.

The most recent one is probably the best of them. If you only have time for one article, read this one. It features an interview with Canadian researcher George Siemens of Alberta’s Athabasca University. Dr. Siemens, along with Dr. Steven Downes (formerly of Athabasca, now at Canada’s National Research Council, in New Brunswick) is considered to be one of the pioneers of the original MOOC concept. (Yes, MOOCs, a Canadian invention!)

It gives a lot of fascinating insights into the origins of the concept, where it has gone as it has gone more corporate and mainstream, and where it may be going in the future.

The other articles are great too, and give some insight into how the landscape has evolved over the last couple of years.

I hope to write something more substantive about MOOCs, with additional resources, including from research literature, sometime in the next couple of weeks. Currently reading a lot about this topic, and looking forward to sharing what I’ve found.

In the meantime, enjoy.








Concept: A different form of “Blended Learning”


For today’s post, I’d like to present an idea for an interesting approach to instruction blending aspects of e-Learning and Instructor-led classroom learning.

Instructor-led and e-Learning are two common methods of delivering instruction, each with their respective strengths and weaknesses. Because of these respective drawbacks, it’s common to use a “blended” approach that combines the best of both worlds while trying to avoid the respective drawbacks. The approach I’m describing today is a different take on the idea of blended learning.

As a refresher and introduction, though, I’d like to review a bit about the pros and cons of Instructor-led classroom and e-Learning. (Feel free to skip to the “Blended Learning” section if you’re already sufficiently familiar with these pros and cons.)

Instructor-led classroom


  • Learning costs less on the front end to design and develop and lesson plans and classroom aids.
  • Tends to make it easier to engage students and to read the student mood or level of attention and adapt accordingly, on the fly.
  • More flexibility in terms of assessment methods. Instructor can grade free-form texts and other presentations, and observe student demonstrations of skills, giving more fine-grained feedback.
  • Students have the benefit of face to face interactions with the teacher and other students.


  • More expensive per student to deliver the learning on a marginal cost per student basis. There is also the cost of the classroom space and whatever support staff are needed to keep the classroom in order.
  • The learner has no control over the pace of the lesson or the content taught.
  • The pace will often be determined by the slower students, who might take up time with questions about material others already understand, and hold back how much can be covered.



  • Lower marginal cost of delivery per student.
  • Easy to scale – it doesn’t make much difference whether the course files are served up to 50 students versus 5000, aside from added costs for server capacity and technical support demands. (And these additional costs are generally included in the per-student fee)
  • Can be entertaining and hold the students’ attention if the students if multimedia presentations with the right mix of flashy and informative are used.
  • Can engage the students if making use of strategies like storytelling of examples or using creative interactions like running through interactive case study scenarios or simulations.
  • Can potentially adapt to students or give flexibility so that they can skim faster or skip material they’re already familiar with.
  • The learner can go through the materials anywhere with a good internet connection and at any time they want.


  • Substantial potential upfront and yearly maintenance costs if using a commercial Learning Management System (LMS) to register students, serve up class content, and track student progress. (Part of the student fees would need to contribute to covering these costs).
  • Substantial development costs for new e-Learning materials. Good e-Learning can take on the order of a few hundred hours of labor per hour for full analysis, design, development, implementation, and evaluation.
  • Often the entertainment and engagement potential of the medium is not realized due to budget limitations.
  • Isn’t as adaptable to the learner’s mood and attention levels while the student is taking the material, because currently no way to effectively measure this over the web. (Though, potentially, in the future, tech such as learning software using input from devices like the MS Kinect sensor could track facial characteristics and body language of boredom or confusion and pause on the fly to pause, offer some remediation, offer to go back, go into a more detailed explanation, or suggest good web-based materials to review.
  • Student can feel alienated and alone if there are not good social elements allowing interaction electronically with someone for help or with other students.
  • Difficulty with holding motivation to complete the course.

Blended learning

The relative strengths and shortcomings of these two methods of course delivery led to the idea of combining the two approaches. This gave the idea of “Blended Learning.”

Blended learning is another approach which aims to try to get the best of both worlds by having some elements taught by a live instructor and other parts using e-Learning. When this works, you get a “whole better than the sum of parts” effect. Blended learning can take different forms.

For example, a learner might watch a video presentation at home online introducing the subject matter in an interesting way and then come to class and spend the class time participating in group discussions or other activities or to ask questions to the instructor. (The format known as the flipped classroom is a good example of this)

Or, theoretical portions of a training course could be covered in e-Learning, modules while hands on practical exercises could be carried out with a live instructor in the lab or classroom.

Another approach is to use the web as a communications and collaboration platform within which the instructor can do synchronous presentation of content to a live web audience. Learners are able to ask questions during the presentation and communicate and work together with other students. These web platforms are called “virtual classrooms. ” One examples of such a platform is BigBlueButton . The virtual classroom takes a lot of the functionality of the live classroom experience, but delivers it through an online platform.

Another variant of this uses some of the same tools as the last, such as Blackboard / WebCT and also Moodle. Here, the class is taught in the classroom by an instructor (sometimes, but not always with video recordings of the lectures archived online for later reference). The online platform is a space that houses a lot of the administrative aspects of the course. This is the place to review the class syllabus and class schedule, download pdfs of course readings, see documentation about assignments, complete computer graded assignment sets or quizzes, read course announcements, and even submit soft copies of assignments. Typically, there are also communication tools like discussion forums, mail systems, and internet messaging / chat, where students can discuss and interact with each other about course content. Usually these are called Course Management Systems or Learning Management Systems.

e-Learning within the classroom: An alternate approach

I want to talk here about another option that blends e-Learning and the classroom. Whether this option would strictly be called blended learning could probably be disputed, but I’m making the argument based on the loosest possible literal definition: a solution that blends elements of e-Learning material and classroom learning in a synergistic fashion. In any case, the name is less important than the idea.

The idea is to use the e-Learning content as a teaching aid or frame for the lesson plan within the classroom. The e-Learning lesson is presented using an internet connected computer hooked up to a digital projector and speakers. The e-Learning module can present information in an entertaining way and help to set up and support classroom activities led by the instructor.

The e-Learning content here would in effect become another form of classroom multimedia. Much like an instructor would present video or animation or audio content to liven the class and show something in a more effective way than speaking and using the blackboard, the instructor uses the e-Learning as a classroom presentation aid, framing and scaffolding the flow of the lesson.

The e-Learning module would have the advantage that once the module is open, it ties together all the media you want to use within the lesson flow in one presentation, one package, everything already sequenced and cued up, ready for use. It simplifies and reduces the number of things to fiddle with during the class. An instructor could alternatively do something superficially similar with a laptop and a bunch of open windows on the desktop to play the different media files one by one. But it would be an added load to deal with all this, with the hassles and time wasted in searching for and finding the right item to bring up, of switching windows and then starting it up. This is attention and energy an instructor could better spend watching and interacting with the students. In the setup I’m describing, the sequence of materials would already be set up, and the instructor could flow from one piece of media or instructor-led interactive activity simply by clicking “Next.”

The menus and the navigation and control elements of the lesson presentation would also allow some customization by the instructor. The instructor could pause a video or multimedia presentation to add some clarification or further details. If the facts on the ground on that particular day of class required a deviation from the lesson plan, the teacher could use the lesson table of contents or slide menu to skip over sections or go back to a particular slide.

In effect, it is a computer supported pedagogy (andragogy).

Some might say that a PowerPoint presentation could also be a possible container for this sort of presentation. However, some sort of e-Learning package would allow better control of embedded slide media through player controls and allow better options for navigation through the material.

An example of how it might work

So how would this work? There are multiple possible instructional strategies and lesson structures that could be used. Let’s take a look a couple of examples of how it could work for different subjects.

 Example One: A history classroom

The teacher welcomes students to class and starts the presentation. The courseware shows the students a video describing the background scenario of some key event during the time period the students are currently studying. The video would ideally include period film footage (if recent enough) or music or photographs of people or buildings, or paintings, with narration over top to tell the story and the basic facts. This would then set up some critical moment where a group or a leader had to make a decision that shaped the course of events. The video would end with a segue to an in-class activity where the learners role play different key figures.

One or two static slides would be presented establishing the instructions and ground rules for the assignment.

The teacher would take over and lead the students in the activity. The students would discuss what they, as different characters in the historical events, would have been thinking, what would have been their goals and motivations, and their thought processes of what they perceived of the others’ motivations, and on what to do next. The teacher facilitates this discussion.

Once the discussion wraps up, the instructor clicks next to resume the presentation, which would continue on with video discussing what actually happened, what the key actors actually did, and what they thought and said about it, as recorded in primary documents. The video would then segue toward another in class activity, where the students discuss how what they predicted or suggested differed from the actual reality. The teacher could facilitate the discussion.

The courseware could support the activity again by presenting static visuals of some key questions to ask and consider.

Example Two: A science classroom

A science classroom could also use this approach to handle the combination of presenting concepts and looking at applications. As well, for the combination of demonstrating problem solving concepts and applying these concepts.

Again, the teacher could welcome the students to class and then start up the presentation. The presentation would go to a video with multimedia explaining a particular science concept. The abstract concept could be explained in an engaging, visual, concrete fashion. The video could then segue into a live classroom demonstration set up previously by the teacher. One of the benefits of having the courseware present the concept is that the teacher can use that time to make sure all the final details are ready for the demonstration.

The teacher gives the demonstration presenting the concept. There could be interactivity here, with the teacher asking students for their ideas of what will happen, and then, after carrying out the demonstration, the class could discuss the differences between what they thought would happen and what actually happened.

Then, the teacher could click to restart the lesson. The presentation could transition back to the courseware leading the way, showing a video or animation introducing an important formula related to the concept. The video could also set up and then solve a word problem related to the concept. The video would then close with a segue to a classroom activity where the students can try out a few similar problems on their own.  The courseware could display the wording for the problems on the screen.

The learners would work on the problems, while the teacher circulates, and giving help and feedback.

Benefits of the Approach

If executed properly, this approach allows a seamless blending of these two modes of delivery, web and instructor led. The two would complement each other, and flow from one to the other and back again during the classroom presentation.


There are a few challenges to this approach.

First, and most obviously, the classroom needs a solid, dependable internet connection, particularly if video content is to be used within the presentation.

The e-Learning presentation also has to be set up with this instructional strategy and use case in mind. It should include introductory audio and images and/or video to segue into in-class activities and provide visual reinforcement of instructions for the activity. The presentation also has to include built-in “pause points” where it switches from watching a presentation to doing an activity.

Another criticism could be that using e-Learning as a classroom media or media and lesson container is that it would seem to miss the whole point of e-Learning. That is, you justify the usually exorbinant costs of the front end development of e-Learning by delivering to a large number of students. However, I’d respond to that on two fronts.

First, there are a number of rapid development tools out there that can make the process of e-Learning development quite straightforward, similar to that of making a nice PowerPoint, or even allowing the option to import and existing PowerPoint presentation and then use tools to augment the presentation. Some of the Articulate products, such as Studio and Storyline make things very simple and visual. A teacher or, for example, a subject based department in a high school could get a license for the software, and together build shared presentations. This would allow departments where multiple teachers with differing styles of teaching and differing levels of experience are teaching the same course to standardize a bit the presentation of the course material.

Second, another application would be for a company to build these sorts of presentations for different subjects in different courses and sell them to teachers or school boards as an off the shelf “lesson/unit/curriculum in a box.” This would give pre-packaged lesson plan structures, with media and in class activities ready to go.



Peer Assessment in MOOCs


As mentioned in some previous posts, I’m currently taking a MOOC (Massive Open Online Course) through Coursera. I’m overall very pleased with the course, enjoying it a lot, learning a lot. It’s my second MOOC course, and the first I’ve taken from one of the big three vendors (Coursera, Udacity, and EdX). In the process, I’m getting a lot of great insights in terms of thinking about design for this exciting new format of university education.

One area where I am pleasantly surprised is in how well the aspect of peer assessment in MOOCs works for assignments. For those who are not so familiar with the format, in the MOOC environment, there can be thousands of students per course (this is where the “Massive” part comes in). As a result, instructor or facilitator grading is not practical or cost effective. Now, for some types of assessments, such as multiple choice or short answer or matching or what not, these can be graded by machine. The challenge is grading of more open-ended work like essays, projects, problem sets, and what not. These more complicated types of submissions cannot readily be done by a machine.

As an alternative, therefore, “peer assessment” is used. That is, the grading is carried out by the students themselves.

Peer Assessment in MOOCs: How it Works

Each student gets randomly assigned five other student submissions (anonymous) to grade using an online grading form. A rubric guides the marker on areas to assess, and criteria for assigning or subtracting marks.

Here’s an example of a rubric used for a recent assignment on programming a simple virtual stopwatch (with the stopwatch itself on the right):

stopwatch_rubric  stopwatch


A web form based on the rubric is used to collect the assessment feedback. Drop down menus are used to select grades on each rubric item (0/2, 1/2, 2/2, e.g.) with criteria given for each possible mark for each item. Text boxes are given for more detailed feedback, to give the student some explanation when marks are taken off or to give some positive feedback for a truly impressive submission.


The rubrics help to focus the markers and to standardize the grading for improvements in the all-important Inter-Rater Reliability. For people to take MOOCs seriously, there needs to be that sense among students that the grading is as uniform and objective as whatever grading they would get in a normal university course. (I won’t get into the thorny question of how much that exists in current brick and mortar university classrooms 😉 ) Assigning multiple submissions to be marked by each student also means that each student will in turn get assessed by multiple peers. This adds another level of objectivity on top of the rubric; variance between markers can be washed out by dropping outlier grades (lowest and highest) and averaging over the remaining grades.

Overall this encourages objective, fair, and thorough grading. Extremely lax/superficial grading or abusive, over-strict grading tends to get dropped out when the low and high outliers are dropped. Also, there is a bit of a “do unto others” / karma effect here. The person you are grading this assignment could well be grading you the next assignment, so you don’t want to be abusive of the power. On the other hand, you know others are going to be marking down your errors if you made them, so you wouldn’t want to be over-lenient.

The student duty to contribute to marking is reinforced by making it part of the criteria for success; students lose grades on the next assignment if they fail to grade their minimum of five assignments. The option is also given to choose to mark more than the minimum. In the course I am taking, this is voluntary, and does not give any added benefit. I would probably suggest to the makers of the course or Coursera that some additional marks or offsets for errors in other assignments might be one way to incentivize learners in the desired behavior of going beyond the minimum.

Another neat touch is self-assessment. After grading the requisite five peers, the student is asked to evaluate his own work. This also becomes part of the grade. From a perspective of Adult Learning Theory, this gives a nice opportunity for the learners to reflect on their own work after a few days away from it, and in the light of what he has seen of the mistakes and highlights of what others have done, helping to further reinforce learning.

What we can learn from this in general

Overall, this is a system that works nicely, and helps to reinforce the idea that MOOCs can work in practice to replicate a university course online.

I think, however, we could learn some takeaway lessons from this format for our design of e-Learning in general. Assessment can often be a weak point in e-Learning / m-Learning due to the limitations in what can be graded by machine. This is why a lot of the assessment in e-Learning is in the form of simple, closed answer forms readily supported by authoring tools, things like True/False, multiple choice, short answer, drag and drop, and matching. Some interesting assessment options can be engineered for soft skills training by combining multiple choice questions with slide branching to create “Choose Your Own Adventure” style interactive scenarios. But open ended question answer types like essays, video responses, or project submissions are not so readily supported.

Perhaps we can take some inspiration from this technological implementation of peer assessment and self-assessment in MOOCs to enable simple grading of more complex assessments in e-Learning. At least for cases where we have a large number of students taking the course at the same time.

All that would be required from an Instructional Designer would be the design of easy to use and understand grading rubrics. In addition, the Instructional Designer would need to design web forms for the students to enter marks with drop down menus for each rubric item, and to enter open ended grader comments.

To handle all the magic the background, programmers would need to implement an engine for:

  • Receiving submissions and enforcing any soft/hard deadlines
  • Randomly assigning submissions to students for grading
  • Receiving and processing the submitted grades and calculating final assignment grades
  • Communicating the results to the learners

In summary, online peer and self-assessments based on rubrics and supported by the right IT functionality offer refreshing new possibilities for richer forms of assessment in e-Learning, given a large enough pool of students taking the course at the same time.

The importance of Learner and Context analysis for m-Learning design

I’ve talked before about tips for m-Learning. Mobile devices have become, or are becoming the dominant means through which people access online content, and the need for mobile delivered learning and support materials as part of an overall training solution has become increasingly apparent.

It’s become so much part of the culture now that whether to make materials available in a mobile-friendly format has ceased to be the most challenging question, because it’s no longer a question to puzzle over, but almost a given. Making training content available to mobile becomes a default expectation. It has gone mainstream as a component or means of delivery of training.

Mobile has also gone mainstream in terms of how we think about presenting content and how we structure interactions with e-Learning. The mobile app aesthetic has become a key influence in the design of look and feel of e-Learning presentations. This aspect of the design is becoming almost subliminal, just part of the culture. We start to just think in terms of stripped back, less visually busy screens with less text, and graphics that will look clear on a 5-10 inch screen.

Now the attention, in any training organization should be moving on to a more subtle and nuanced question. Namely, what parts of the training solution should be directed toward which of the multiple screens in the learner’s work environment and life?

A fully thought out and realized training solution designed for the specific combination of performer-job-context will consist of multiple components.

The multiple screens to which the learner has access, including work desktop, home desktop, smart phone, tablet, and connected TVs each have their own particular quirks in terms of portability, resolution, how you use it or interact with it, and where you can look at it. The primitive sort of approach to this in many organizations has often involved very blunt solutions. “Customers have iPads. customers like iPads,” says the upper management. “So put the training on iPads.” So the “mobile strategy” has often consisted of “put it on the iPad.”

Now, I don’t mean to slag this completely; at least it got development teams working on thinking about, and making, mobile friendly content. But a more nuanced solution is needed.

Different aspects of a training solution will fit best or more naturally with particular of these screens, depending on the needs of the worker, the work context, and the strengths and weaknesses of the various screens. As a result, different parts of the training may well need to be targeted toward different of the devices used by the learners. This underlines with renewed vigor the importance of a rigorous needs analysis, particularly learner and context analysis for m-Learning.

There are some questions that the designer needs to answer:

  • Where is the learner doing his job? At a desk with a desktop computer station? In a fixed location in a warehouse or factory floor? Is the person out of the office, for example, a salesforce professional, a sales engineer, or an onsite repair or installation technician? Does the worker spend a large percentage of the time travelling?
  • What sort of computer technology does the user have at his workplace provided by the employer? A desktop computer? A laptop? A tablet? A company cellphone? What sort of technology does the user himself have? Smartphone? Tablet?
  • Are there any data security or regulatory concerns that would preclude users accessing training materials on their own devices?
  • Does the workplace have training on the intranet and are the users openly and consistently encouraged to take training during appropriate moments on the company time?
  • Will the worker’s performance and transfer of learning benefit significantly from having a quick reference on the job?

Let’s highlight this with a few examples.

For example network access level on the job. If a worker has access to high speed wifi, then you can safely have more rich content coming through the mobile devices. If the workers are on their own cellular connections, you should stick to a less rich presentation for any mobile component. You need to be cognizant of bandwidth limitations. In the second case, perhaps the more useful solution for the mobile device is high level notes. Reminders that the worker can look up in a handy and simple fashion on the fly. The user can whip the phone out of the pocket, look up the simplified reference, put the phone away, and get back to work. Bullet lists or simple graphical reminders are ideal in this case.

Or for home tablets, is the worker going to want to watch work related training material on the couch on the iPad at the end of the day? For the average worker, maybe the motivation is not there. But if the person is a highly motivated individual interested in professional development training to improve or accelerate promotion prospects, then he quite likely would find the iPad availability attractive.

If an employer as a matter of policy allows and encourages workers to take professional development training during lulls in the work day, then a delivery through the work desktop would seem to be in order.

Is a person at a desk all day? If so, materials accessible on the desktop could well suffice.

If the person is in a job where he is constantly mobile, like a service or installation technician, mobile is the perfect platform for performance support tools.

Similarly someone in sales that has to travel to meet customers or potential customers. A mobile application can be an ideal way to top up product knowledge before meetings.

Also for employees who travel frequently. A tablet is the ideal size and form factor for use on a train or airplane or in a taxi.

On the other hand, if someone is at a fixed location in a factory setting, there may well be fixed stations with desktop computers, but the technical specs of the computers in that setting will probably be substantially lower than those of the phones in people’s pockets or a tablet. In this case, mobile support aids may be more usable.

So the basic takeaway is that we as designers have to understand the workers and their work flow and environment, look at what devices are available at what times, look at the training solutions we can offer and want to offer to help them learn and perform, and then decide which of these solutions are best delivered through which devices.

Rather than some across the board decision about desktop vs mobile, we need to take a more nuanced, component by component look at what to deliver, and where.

Concept: Using Natural User Interface technology to enable live adjustment of e-Learning presentation

Introduction: What good teachers do best…

One of the ways in which an instructor in the classroom soundly beats e-Learning is the ability to monitor the words, body language and facial expressions of the students. Skilled teachers are always, as it were, running multiple processes in parallel, paying attention, consciously and sub-consciously, to many things at once:

  • They are presenting the content of the lesson, coordinating modes of presentation, modifying emphasis and tone of voice to better explain a concept to that particular group.
  • They are monitoring the time used and time remaining and assessing in relation to where they are in the lesson
  • They are thinking about the part of the lesson they are in and preparing for the next
  • They are monitoring the class for disciplinary issues (in an elementary or secondary classroom)
  • They are monitoring the class for signs of interest, confusion, boredom, or frustration

Good teachers then use these cues to in real time calibrate and re-adjust the style and content of the presentation.

It’s a tiring load to balance, and takes a special skill set. But it’s the sort of task on which humans still far outperform machines. Multiple simultaneous tasks of a high level of dense, subtle, multi-faceted complexity in parallel. This is something that computers can not yet approach. (Though with the relentless exponential march of technology and research, AI experts would probably say it’ll get there sooner than we may think or be comfortable about)

Simulation of this using Natural User Interface technology

However, a very rough approximation of some of this sort of adaptive delivery could potentially be used in e-Learning in the relatively near future using available sensor technology. Learning software could potentially use input from natural user interface  devices like the MS Kinect sensor which comes with the Xbox One video game platform. This sensor has stereo-cameras to track user movements and face expressions and microphones to record user speech for game control.

When people are interested, bored, confused, or frustrated, these are all internal affective (emotional) states, but they manifest externally in facial expressions, spoken language, and body language in a relatively uniform fashion from person to person. This is what enables a human teacher to recognize these states in learners.

Software could use the SDK for the sensor and plug into APIs of the sensor. In that way developers could make applications that track user words, facial characteristics, and body language based on data read  by the sensor. These APIs are available  for professional use, and for research purposes. The learning delivery software could then analyze the input audio and video data based on certain criteria to try to identify (with a sufficiently high degree of probabilistic confidence) the emotional state of the learner.

Once the emotional state of the learner is identified, the software could respond by altering the flow of the presentation in an appropriate way.

This could take different forms.

  • The software could slow down the presentation, or speed it up/skim.
  • It could segue into a repetition of the confusing material or switch to a more elaborate explanation audio track.
  • It could pause the presentation.
  • It could present another optional example from a bank of examples to further clarify a topic.
  • It could ask one or two knowledge checks from a bank of questions. An NUI based system could also ask questions of the learner in a relatively natural voice (text to speech) based on an algorithm.
  • It could simply give a verbal prompt to confirm if the learners are following and if the material is moving too slowly or quickly and then wait for and interpret the user response.
  • It could offer some remediation, such as offering to take the user back one slide.
  • It could suggest good web-based materials to review.

Now, supposing such software were developed and incorporated in some way into learning management and delivery platforms, there is the question of what sort of efforts it would take for an Instructional Designer to develop some of these items (banks of examples, multiple explanations at different levels of depth / detail, questions, and narration tracks for all of these). As well, to what extent that development process could be streamlined or automated (using more improved future text to speech capabilities, for example) to prevent the development from being unwieldy. Also, if this software was incorporated into Learning Management Systems (LMS) or Learning Record Systems (LRS), which track learner session data, there might be concerns about privacy, as to whether at least some high level affective state data from the session should be kept on record. Designers might find this sort of data about how student reaction invaluable for evaluation purposes. But learners might find it creepy.

But, these questions aside, the fact is that the sensor technology with the capabilities to collect the needed data to support such a thing exists. At this point it would simply be a matter of a knowledgeable team or teams doing the hard work of writing the algorithms. And if used, it could potentially lead to much more adaptive and user-customized delivery of e-Learning, living up to the e-Learning marketing over-promises of yesteryear.

Microsoft Kinect for Windows Development

Microsoft’s Kinect for Windows development pages include access to the SDK (Software Development Kit).

The SDK with Kinect for Windows APIs is free to download and to use (no direct software licensing fee) to develop Windows apps that use the Kinect for Windows sensor.

Microsoft sells the special Kinect for Windows sensors (distinct from and somewhat more capable for commercial applications than the sensor used by the Xbox gaming platform) separately for $250. (There are apparently discounts for students and educational institutions.)

Microsoft also includes Human Interface Guidelines (HIG) with this package.




Research studies along these lines

In writing this article, I was pleased to discover that researchers are already looking into ways to use sensors and algorithms to automatically detect affective (emotional) states of learners relevant to attention and learning. Here is some further reading for those who are interested:







A curricula for teaching computer programming / software development


A lot of my posts tend to focus on new technologies and looking at how those could impact education and instructional design. But I’m going to switch gears for a bit today with a draft idea for a teaching plan for the classroom. This idea is for a curriculum of projects in a high school enriched or CEGEP/junior college level class on computer science and teaching computer programming.

Motivation behind this article

I’ve recently been enjoying a MOOC course that I’m taking from Rice University through Coursera. The course is an Introduction to Interaction-Programming in Python. It’s great. Mix of teachers gives a variety of styles of presentation. I’ve  come to realize more how important it is to give instructors in MOOC course the creative room to express their personality for their buy-in to the approach.

The course is nine weeks and has a nice project based approach where each week there is a new project. The projects start out simple and get more sophisticated as the weeks go by. The final project is a simple space shooter video game.


Anyhow, the material taught and the project based approach got me to thinking about how introductory computer science is taught and gave me the idea for an approach to teaching computer programming in the advanced high school or junior college classroom. (I took such a course myself in high school, and it was one of the more influential I took during that time period)

A note about programming / software development

Although computer science types are often stereo-typecast as loners without social skills, the reality is that modern software development is a highly social field. The largest software projects could not be completed in any reasonable amount of time by one individual, in some cases probably even in a lifetime. Tens, or at the high end, even hundreds or thousands of programmers can work together on one project. They need to communicate, coordinate, divide tasks/delegate, and collaborate. These are key team skills. Not to mention skills of decision-making and conflict resolution.

The open source programming movement is another good example where the emphasis is on sharing and collaboration.

The Course Design

Make the class be based on projects, with the project being some sort of computer program. The projects would be chosen so as to require larger sorts of programs to complete. Each will take hundreds of lines of code of complete. One of these every two weeks say. Would be cool too, ideally if each of these biweekly projects contributed to one large end of term project. That would take significantly more effort for the teacher to plan though. Should be one larger term project though, also collaborative.

These projects would be collaborative, built in groups. Divide the class into 3-4 groups, depending on class size. Want 6-8 students in each group.

The first few weeks could give some basic orientation, the chance for the class to read the first couple of chapters of the text, and go through some basic individual exercises. You can get some idea from this about who are the keeners and potential leaders, both for projects and for project sub-groups.

Once the class is ready, a few people could volunteer to be the main architect/lead developers for the program for each group or the teacher could assign. These leads could select say another one or two students to help design the higher level structure of the program. The teacher would support as needed. Those students would be able to get the highest potential mark, say an A+ or 100% because the task requires higher level thinking and work, and also because that’s the position that would get paid better in the real world. Then the other students in the team can be assigned to or choose different sections of the code. Those sections would be worth different amounts, based on the complexity of the task. One chunk or task might give a max of 85%, another 70% say.

Everyone can get something that gives them a solid pass.

Opposing teams test each other’s compiled executables (group # 1 grades 2, 2 grades 3, 3 grades 4, 4 grades 1, e.g.), trying to find bugs and writing bug reports. The teacher would show them how to perform bug testing and some templates for writing bug reports.They hand these back to the development group and give one chance to fix it. If this step is required, maybe the whole group gets reduced by a flat 5% for simplicity, whether one bug or many. (Aim for zero errors)

The team then verifies and tries to fix the bugs within a certain time window and then sends it for the second review. If the development group can’t reproduce the documented bugs, the team can appeal to the teacher, who will also investigate and cancel the deduction if he also can’t reproduce it. It is put through another full test again at this point. Bugs that remain are documented and a sliding grade reduction applies to the development team members based on what remains. (Not just the original bugs are tested – sometimes fixing the bug causes new bugs, so a full additional test is needed.)

The teacher calculates the grade based on this peer review. The peer review encourages teams to compete against each other in developing testing skills. The teacher makes his own review, and factors this into the grade as well. After each project, once evaluations are done, the teacher will take a class to review each teams projects together as a class, to discuss lessons learned.

Group leaders / architects need to develop testing routines based on reading and teaching from the text and the teacher and need to make sure others in the group apply testing on their own work to avoid losing marks.

The groups would not be permanent over the year, but would be picked anew for each project. Even the team leads might change as someone else within the groups shows an elevation in skills to a new level and the instructor and/or class feels he deserves a chance to try leadership. The teacher should instill values in those who regularly are team leads/project architects that it is important for leaders to step aside enough to give team members a chance to grow and shine.

Challenges: points still to resolve

The instructor would need to find a system to decide how the errors made by individual team members in their portion of the project affect the grade of everyone else. On the one hand you want to encourage the team leaders’ management skills development and the whole team’s teamwork skills. At the same time you want to prevent anyone from freeloading off the others in the knowledge the better students will do it for him out of concern for their own marks. (As is often a problem in group efforts, in school and in the workplace) Part of an individual’s mistake would reflect on the group, and part would be his own deduction.

Foreseeable critiques

One critique I would expect from educational circles would be that the competition and team rivalry aspect might be discouraging or off-putting to girls, who are already under-represented in the computer science classroom. On the other hand, the emphasis on the collaborative aspect of software development with it’s aspects of precise and effective communication, coordination, and collaboration might actually make things more attractive to the girls.

As well, the competitive aspect can be kept to relatively safe stakes. The instructor can emphasize that the focus in grading is on the objective absolute quality of the work each team does and not the ranking between the groups on a project. The recognition or honors given by the teacher are not an all or nothing thing lavished on the winning team each time. The lessons learned discussions post project also allow some of that essential soft skills aspect to come to the front.

Arguably, this approach actually brings out a balance in hard vs soft skills much better than the usual methods, and may be the sort of mix that is attractive to both genders.

Possible Modifications

One possible modification of this that I could think of that would subtly shift the tone would be to have the different teams collaborate rather than compete. Each group would get a different, but comparably difficult sub-task for implementing one overall programming task. This would require more time from the teacher to plan and coordinate all this, to think up a proper sized overall project and to evaluate multiple tasks rather than each group doing the same thing. The teacher would also probably have to take on the role of project architect, making the basic overall plan and structure and delegating well-defined but demanding sub-programs to develop to each group and monitoring what they are doing and that it fits in with everything else.

The other wrinkle is that ostensibly the different teams’ programs / modules would need to interact with the other teams’ modules to function. This could complicate testing.

Perhaps the best way would be for an instructor to try the competitive approach first a few times, and when he feels comfortable with it, then try the course collaboratively.