Category Archives: Instructor-Led / Classroom Teaching

Concept: Maintenance Training 2030


April 25, 2032 …

Ed, a Canadian aircraft maintenance training instructor located in Montreal, gets ready for his upcoming engines maintenance class.

Supposed to be a big class this week, 8 students.

  • 1 from Canada
  • 1 from the US
  • 2 from South America
  • 2 from India, and
  • 2 from China

Ed checks on the equipment in the classroom.

He puts on his Holo-Glasses, which come to life, softly glowing holographic data displays and icons popping up in front of him.  The device recognizes him, launching the virtual assistant to greet him. “Hello Ed! How are you? All set for your class?” “Just fine, thanks. Everything set?” “Yes, Ed. All the students are going to be attending; no cancellations. Everything looks good with the students. One was having some technical issues earlier, but I helped him through it.” Excellent,” thought Ed. “Everything looks alright with 15 mins to go.”

Ed begins cueing up the opening presentation notes, and the multimedia training manual. These pop up in their own windows in Ed’s field of view.

As Ed continues his preparations, the digital assistant relays notifications confirming the status of the students. The assistant is communicating with the students before class so Ed can focus on his preparation. Everything is looking good. Ed checks the 3D cameras and tests out his holopresence projection, seeing what his students will see.

“Loo-king good! Let’s do this!”

A few minutes later, the class begins. Ed welcomes the students as they holopresence in from their remote locations. Ed and the students, their Holo-glasses on,  take their places in the shared virtual classroom. The software places softly glowing holographic representations of the other participants in the shared visual space.  Ed looks out at the students’ faces, and the students see a holographic overlay of the same classroom and the same students from their own virtual perspective. At first, the experience is a little eerie, but as the class gets going, and all the students introduce themselves, the illusion takes hold and it feels like everyone is in the same classroom.

Ed presents the content, asks questions, and listens to the responses. Master teacher he is, he observes carefully, gets a sense of the learners’ body language and expressions, and, much like in a real class, adjusts as he goes. Ed brings up holographic 3D animations and models of the engine and components for the class to see. He zooms, rotates, and takes apart the holographic engine parts. The hologram also appears in the students’ fields of view, and Ed invites students here and there to come up and try for themselves and demonstrate actions to the class. Static images appear on screens in mid air, demonstrating schematics.

In the afternoon portion it is time for the virtual hands-on lab exercises. Ed and the students convene again, once again with beautiful, interactive 3D holographic models of the engine floating in the shared digital overlay. This time however everyone puts on their SureTouch(TM) haptic feedback gloves.

The gloves use sensors to read finger and hand position, the headset measures their hand positions in relation to the digital model’s virtual position, and actuators in the gloves give pressure feedback to simulate handling real objects with substance instead of just weightless holograms. It’s kind of weird at first, and it’s not quite the same as the real thing, but close enough for horseshoes and hand grenades, as they say. And definitely a hell of a lot cheaper than taking an actual engine offline to train.

As always, it took a few years for the technology to perfect itself and a lot of research and proofs of concept before the regulators really believed it could be as effective as the real thing. The Dutch Aerospace Lab did some great research as always, and once EASA signed off, the other regulators followed pretty swiftly after. Regulators came to appreciate virtual maintenance training, just as they came to appreciate the power of full flight simulators decades before.

The company definitely appreciates it too – they save a small fortune in flights, hotels, taxis, and per diems doing virtual classes like this over the course of the year. As do the students’ companies..

Ed for one, appreciates it too. No packing, no airport security, no  cramped 12 hour flight, no hotel room, no taxis, no jetlag, no traffic. Well … scratch that last one. This is Montreal, after all, where the seasons are winter … and construction. Even in 2032, there’s plenty of traffic. (You can’t win ’em all, I guess.) “Oh well, ” thought Ed. “Decent weather today, so at least could read a book on the way in while the autodrive on the car took care of all the unpleasantness.”  And all from the comfort of the Montreal office.

Ed loves it, and his family loves it too – less time away. And besides. even though he felt a little silly to admit it, irrational as it was, Ed had felt a littled weirded out by flying ever since they started the rollout of unpiloted commerical flights in the late 2020s. Hundreds of times safer than human pilots or not, it’s still kind of creepy to have algorithms flying you around instead of people.

“Or maybe I’m just getting old, ” Ed thought. Gets a little jarring after awhile to see the world transform itself before your eyes so quickly. The young seem to take it in stride, unphased, as they always do. And, Ed had to admit that the toys are pretty cool. All this change has its benefits.

Such is the stuff of life in a world of sci-fi dreams made true.

 

 

 

 

Concept: A different form of “Blended Learning”

Introduction

For today’s post, I’d like to present an idea for an interesting approach to instruction blending aspects of e-Learning and Instructor-led classroom learning.

Instructor-led and e-Learning are two common methods of delivering instruction, each with their respective strengths and weaknesses. Because of these respective drawbacks, it’s common to use a “blended” approach that combines the best of both worlds while trying to avoid the respective drawbacks. The approach I’m describing today is a different take on the idea of blended learning.

As a refresher and introduction, though, I’d like to review a bit about the pros and cons of Instructor-led classroom and e-Learning. (Feel free to skip to the “Blended Learning” section if you’re already sufficiently familiar with these pros and cons.)

Instructor-led classroom

PROS:  

  • Learning costs less on the front end to design and develop and lesson plans and classroom aids.
  • Tends to make it easier to engage students and to read the student mood or level of attention and adapt accordingly, on the fly.
  • More flexibility in terms of assessment methods. Instructor can grade free-form texts and other presentations, and observe student demonstrations of skills, giving more fine-grained feedback.
  • Students have the benefit of face to face interactions with the teacher and other students.

CONS:

  • More expensive per student to deliver the learning on a marginal cost per student basis. There is also the cost of the classroom space and whatever support staff are needed to keep the classroom in order.
  • The learner has no control over the pace of the lesson or the content taught.
  • The pace will often be determined by the slower students, who might take up time with questions about material others already understand, and hold back how much can be covered.

e-Learning

PROS:

  • Lower marginal cost of delivery per student.
  • Easy to scale – it doesn’t make much difference whether the course files are served up to 50 students versus 5000, aside from added costs for server capacity and technical support demands. (And these additional costs are generally included in the per-student fee)
  • Can be entertaining and hold the students’ attention if the students if multimedia presentations with the right mix of flashy and informative are used.
  • Can engage the students if making use of strategies like storytelling of examples or using creative interactions like running through interactive case study scenarios or simulations.
  • Can potentially adapt to students or give flexibility so that they can skim faster or skip material they’re already familiar with.
  • The learner can go through the materials anywhere with a good internet connection and at any time they want.

CONS:

  • Substantial potential upfront and yearly maintenance costs if using a commercial Learning Management System (LMS) to register students, serve up class content, and track student progress. (Part of the student fees would need to contribute to covering these costs).
  • Substantial development costs for new e-Learning materials. Good e-Learning can take on the order of a few hundred hours of labor per hour for full analysis, design, development, implementation, and evaluation.
  • Often the entertainment and engagement potential of the medium is not realized due to budget limitations.
  • Isn’t as adaptable to the learner’s mood and attention levels while the student is taking the material, because currently no way to effectively measure this over the web. (Though, potentially, in the future, tech such as learning software using input from devices like the MS Kinect sensor could track facial characteristics and body language of boredom or confusion and pause on the fly to pause, offer some remediation, offer to go back, go into a more detailed explanation, or suggest good web-based materials to review.
  • Student can feel alienated and alone if there are not good social elements allowing interaction electronically with someone for help or with other students.
  • Difficulty with holding motivation to complete the course.

Blended learning

The relative strengths and shortcomings of these two methods of course delivery led to the idea of combining the two approaches. This gave the idea of “Blended Learning.”

Blended learning is another approach which aims to try to get the best of both worlds by having some elements taught by a live instructor and other parts using e-Learning. When this works, you get a “whole better than the sum of parts” effect. Blended learning can take different forms.

For example, a learner might watch a video presentation at home online introducing the subject matter in an interesting way and then come to class and spend the class time participating in group discussions or other activities or to ask questions to the instructor. (The format known as the flipped classroom is a good example of this)

Or, theoretical portions of a training course could be covered in e-Learning, modules while hands on practical exercises could be carried out with a live instructor in the lab or classroom.

Another approach is to use the web as a communications and collaboration platform within which the instructor can do synchronous presentation of content to a live web audience. Learners are able to ask questions during the presentation and communicate and work together with other students. These web platforms are called “virtual classrooms. ” One examples of such a platform is BigBlueButton . The virtual classroom takes a lot of the functionality of the live classroom experience, but delivers it through an online platform.

Another variant of this uses some of the same tools as the last, such as Blackboard / WebCT and also Moodle. Here, the class is taught in the classroom by an instructor (sometimes, but not always with video recordings of the lectures archived online for later reference). The online platform is a space that houses a lot of the administrative aspects of the course. This is the place to review the class syllabus and class schedule, download pdfs of course readings, see documentation about assignments, complete computer graded assignment sets or quizzes, read course announcements, and even submit soft copies of assignments. Typically, there are also communication tools like discussion forums, mail systems, and internet messaging / chat, where students can discuss and interact with each other about course content. Usually these are called Course Management Systems or Learning Management Systems.

e-Learning within the classroom: An alternate approach

I want to talk here about another option that blends e-Learning and the classroom. Whether this option would strictly be called blended learning could probably be disputed, but I’m making the argument based on the loosest possible literal definition: a solution that blends elements of e-Learning material and classroom learning in a synergistic fashion. In any case, the name is less important than the idea.

The idea is to use the e-Learning content as a teaching aid or frame for the lesson plan within the classroom. The e-Learning lesson is presented using an internet connected computer hooked up to a digital projector and speakers. The e-Learning module can present information in an entertaining way and help to set up and support classroom activities led by the instructor.

The e-Learning content here would in effect become another form of classroom multimedia. Much like an instructor would present video or animation or audio content to liven the class and show something in a more effective way than speaking and using the blackboard, the instructor uses the e-Learning as a classroom presentation aid, framing and scaffolding the flow of the lesson.

The e-Learning module would have the advantage that once the module is open, it ties together all the media you want to use within the lesson flow in one presentation, one package, everything already sequenced and cued up, ready for use. It simplifies and reduces the number of things to fiddle with during the class. An instructor could alternatively do something superficially similar with a laptop and a bunch of open windows on the desktop to play the different media files one by one. But it would be an added load to deal with all this, with the hassles and time wasted in searching for and finding the right item to bring up, of switching windows and then starting it up. This is attention and energy an instructor could better spend watching and interacting with the students. In the setup I’m describing, the sequence of materials would already be set up, and the instructor could flow from one piece of media or instructor-led interactive activity simply by clicking “Next.”

The menus and the navigation and control elements of the lesson presentation would also allow some customization by the instructor. The instructor could pause a video or multimedia presentation to add some clarification or further details. If the facts on the ground on that particular day of class required a deviation from the lesson plan, the teacher could use the lesson table of contents or slide menu to skip over sections or go back to a particular slide.

In effect, it is a computer supported pedagogy (andragogy).

Some might say that a PowerPoint presentation could also be a possible container for this sort of presentation. However, some sort of e-Learning package would allow better control of embedded slide media through player controls and allow better options for navigation through the material.

An example of how it might work

So how would this work? There are multiple possible instructional strategies and lesson structures that could be used. Let’s take a look a couple of examples of how it could work for different subjects.

 Example One: A history classroom

The teacher welcomes students to class and starts the presentation. The courseware shows the students a video describing the background scenario of some key event during the time period the students are currently studying. The video would ideally include period film footage (if recent enough) or music or photographs of people or buildings, or paintings, with narration over top to tell the story and the basic facts. This would then set up some critical moment where a group or a leader had to make a decision that shaped the course of events. The video would end with a segue to an in-class activity where the learners role play different key figures.

One or two static slides would be presented establishing the instructions and ground rules for the assignment.

The teacher would take over and lead the students in the activity. The students would discuss what they, as different characters in the historical events, would have been thinking, what would have been their goals and motivations, and their thought processes of what they perceived of the others’ motivations, and on what to do next. The teacher facilitates this discussion.

Once the discussion wraps up, the instructor clicks next to resume the presentation, which would continue on with video discussing what actually happened, what the key actors actually did, and what they thought and said about it, as recorded in primary documents. The video would then segue toward another in class activity, where the students discuss how what they predicted or suggested differed from the actual reality. The teacher could facilitate the discussion.

The courseware could support the activity again by presenting static visuals of some key questions to ask and consider.

Example Two: A science classroom

A science classroom could also use this approach to handle the combination of presenting concepts and looking at applications. As well, for the combination of demonstrating problem solving concepts and applying these concepts.

Again, the teacher could welcome the students to class and then start up the presentation. The presentation would go to a video with multimedia explaining a particular science concept. The abstract concept could be explained in an engaging, visual, concrete fashion. The video could then segue into a live classroom demonstration set up previously by the teacher. One of the benefits of having the courseware present the concept is that the teacher can use that time to make sure all the final details are ready for the demonstration.

The teacher gives the demonstration presenting the concept. There could be interactivity here, with the teacher asking students for their ideas of what will happen, and then, after carrying out the demonstration, the class could discuss the differences between what they thought would happen and what actually happened.

Then, the teacher could click to restart the lesson. The presentation could transition back to the courseware leading the way, showing a video or animation introducing an important formula related to the concept. The video could also set up and then solve a word problem related to the concept. The video would then close with a segue to a classroom activity where the students can try out a few similar problems on their own.  The courseware could display the wording for the problems on the screen.

The learners would work on the problems, while the teacher circulates, and giving help and feedback.

Benefits of the Approach

If executed properly, this approach allows a seamless blending of these two modes of delivery, web and instructor led. The two would complement each other, and flow from one to the other and back again during the classroom presentation.

Challenges

There are a few challenges to this approach.

First, and most obviously, the classroom needs a solid, dependable internet connection, particularly if video content is to be used within the presentation.

The e-Learning presentation also has to be set up with this instructional strategy and use case in mind. It should include introductory audio and images and/or video to segue into in-class activities and provide visual reinforcement of instructions for the activity. The presentation also has to include built-in “pause points” where it switches from watching a presentation to doing an activity.

Another criticism could be that using e-Learning as a classroom media or media and lesson container is that it would seem to miss the whole point of e-Learning. That is, you justify the usually exorbinant costs of the front end development of e-Learning by delivering to a large number of students. However, I’d respond to that on two fronts.

First, there are a number of rapid development tools out there that can make the process of e-Learning development quite straightforward, similar to that of making a nice PowerPoint, or even allowing the option to import and existing PowerPoint presentation and then use tools to augment the presentation. Some of the Articulate products, such as Studio and Storyline make things very simple and visual. A teacher or, for example, a subject based department in a high school could get a license for the software, and together build shared presentations. This would allow departments where multiple teachers with differing styles of teaching and differing levels of experience are teaching the same course to standardize a bit the presentation of the course material.

Second, another application would be for a company to build these sorts of presentations for different subjects in different courses and sell them to teachers or school boards as an off the shelf “lesson/unit/curriculum in a box.” This would give pre-packaged lesson plan structures, with media and in class activities ready to go.

 

 

Peer Assessment in MOOCs

Introduction

As mentioned in some previous posts, I’m currently taking a MOOC (Massive Open Online Course) through Coursera. I’m overall very pleased with the course, enjoying it a lot, learning a lot. It’s my second MOOC course, and the first I’ve taken from one of the big three vendors (Coursera, Udacity, and EdX). In the process, I’m getting a lot of great insights in terms of thinking about design for this exciting new format of university education.

One area where I am pleasantly surprised is in how well the aspect of peer assessment in MOOCs works for assignments. For those who are not so familiar with the format, in the MOOC environment, there can be thousands of students per course (this is where the “Massive” part comes in). As a result, instructor or facilitator grading is not practical or cost effective. Now, for some types of assessments, such as multiple choice or short answer or matching or what not, these can be graded by machine. The challenge is grading of more open-ended work like essays, projects, problem sets, and what not. These more complicated types of submissions cannot readily be done by a machine.

As an alternative, therefore, “peer assessment” is used. That is, the grading is carried out by the students themselves.

Peer Assessment in MOOCs: How it Works

Each student gets randomly assigned five other student submissions (anonymous) to grade using an online grading form. A rubric guides the marker on areas to assess, and criteria for assigning or subtracting marks.

Here’s an example of a rubric used for a recent assignment on programming a simple virtual stopwatch (with the stopwatch itself on the right):

stopwatch_rubric  stopwatch

 

A web form based on the rubric is used to collect the assessment feedback. Drop down menus are used to select grades on each rubric item (0/2, 1/2, 2/2, e.g.) with criteria given for each possible mark for each item. Text boxes are given for more detailed feedback, to give the student some explanation when marks are taken off or to give some positive feedback for a truly impressive submission.

rubric_webform

The rubrics help to focus the markers and to standardize the grading for improvements in the all-important Inter-Rater Reliability. For people to take MOOCs seriously, there needs to be that sense among students that the grading is as uniform and objective as whatever grading they would get in a normal university course. (I won’t get into the thorny question of how much that exists in current brick and mortar university classrooms 😉 ) Assigning multiple submissions to be marked by each student also means that each student will in turn get assessed by multiple peers. This adds another level of objectivity on top of the rubric; variance between markers can be washed out by dropping outlier grades (lowest and highest) and averaging over the remaining grades.

Overall this encourages objective, fair, and thorough grading. Extremely lax/superficial grading or abusive, over-strict grading tends to get dropped out when the low and high outliers are dropped. Also, there is a bit of a “do unto others” / karma effect here. The person you are grading this assignment could well be grading you the next assignment, so you don’t want to be abusive of the power. On the other hand, you know others are going to be marking down your errors if you made them, so you wouldn’t want to be over-lenient.

The student duty to contribute to marking is reinforced by making it part of the criteria for success; students lose grades on the next assignment if they fail to grade their minimum of five assignments. The option is also given to choose to mark more than the minimum. In the course I am taking, this is voluntary, and does not give any added benefit. I would probably suggest to the makers of the course or Coursera that some additional marks or offsets for errors in other assignments might be one way to incentivize learners in the desired behavior of going beyond the minimum.

Another neat touch is self-assessment. After grading the requisite five peers, the student is asked to evaluate his own work. This also becomes part of the grade. From a perspective of Adult Learning Theory, this gives a nice opportunity for the learners to reflect on their own work after a few days away from it, and in the light of what he has seen of the mistakes and highlights of what others have done, helping to further reinforce learning.

What we can learn from this in general

Overall, this is a system that works nicely, and helps to reinforce the idea that MOOCs can work in practice to replicate a university course online.

I think, however, we could learn some takeaway lessons from this format for our design of e-Learning in general. Assessment can often be a weak point in e-Learning / m-Learning due to the limitations in what can be graded by machine. This is why a lot of the assessment in e-Learning is in the form of simple, closed answer forms readily supported by authoring tools, things like True/False, multiple choice, short answer, drag and drop, and matching. Some interesting assessment options can be engineered for soft skills training by combining multiple choice questions with slide branching to create “Choose Your Own Adventure” style interactive scenarios. But open ended question answer types like essays, video responses, or project submissions are not so readily supported.

Perhaps we can take some inspiration from this technological implementation of peer assessment and self-assessment in MOOCs to enable simple grading of more complex assessments in e-Learning. At least for cases where we have a large number of students taking the course at the same time.

All that would be required from an Instructional Designer would be the design of easy to use and understand grading rubrics. In addition, the Instructional Designer would need to design web forms for the students to enter marks with drop down menus for each rubric item, and to enter open ended grader comments.

To handle all the magic the background, programmers would need to implement an engine for:

  • Receiving submissions and enforcing any soft/hard deadlines
  • Randomly assigning submissions to students for grading
  • Receiving and processing the submitted grades and calculating final assignment grades
  • Communicating the results to the learners

In summary, online peer and self-assessments based on rubrics and supported by the right IT functionality offer refreshing new possibilities for richer forms of assessment in e-Learning, given a large enough pool of students taking the course at the same time.

Concept: Using Natural User Interface technology to enable live adjustment of e-Learning presentation

Introduction: What good teachers do best…

One of the ways in which an instructor in the classroom soundly beats e-Learning is the ability to monitor the words, body language and facial expressions of the students. Skilled teachers are always, as it were, running multiple processes in parallel, paying attention, consciously and sub-consciously, to many things at once:

  • They are presenting the content of the lesson, coordinating modes of presentation, modifying emphasis and tone of voice to better explain a concept to that particular group.
  • They are monitoring the time used and time remaining and assessing in relation to where they are in the lesson
  • They are thinking about the part of the lesson they are in and preparing for the next
  • They are monitoring the class for disciplinary issues (in an elementary or secondary classroom)
  • They are monitoring the class for signs of interest, confusion, boredom, or frustration

Good teachers then use these cues to in real time calibrate and re-adjust the style and content of the presentation.

It’s a tiring load to balance, and takes a special skill set. But it’s the sort of task on which humans still far outperform machines. Multiple simultaneous tasks of a high level of dense, subtle, multi-faceted complexity in parallel. This is something that computers can not yet approach. (Though with the relentless exponential march of technology and research, AI experts would probably say it’ll get there sooner than we may think or be comfortable about)

Simulation of this using Natural User Interface technology

However, a very rough approximation of some of this sort of adaptive delivery could potentially be used in e-Learning in the relatively near future using available sensor technology. Learning software could potentially use input from natural user interface  devices like the MS Kinect sensor which comes with the Xbox One video game platform. This sensor has stereo-cameras to track user movements and face expressions and microphones to record user speech for game control.

When people are interested, bored, confused, or frustrated, these are all internal affective (emotional) states, but they manifest externally in facial expressions, spoken language, and body language in a relatively uniform fashion from person to person. This is what enables a human teacher to recognize these states in learners.

Software could use the SDK for the sensor and plug into APIs of the sensor. In that way developers could make applications that track user words, facial characteristics, and body language based on data read  by the sensor. These APIs are available  for professional use, and for research purposes. The learning delivery software could then analyze the input audio and video data based on certain criteria to try to identify (with a sufficiently high degree of probabilistic confidence) the emotional state of the learner.

Once the emotional state of the learner is identified, the software could respond by altering the flow of the presentation in an appropriate way.

This could take different forms.

  • The software could slow down the presentation, or speed it up/skim.
  • It could segue into a repetition of the confusing material or switch to a more elaborate explanation audio track.
  • It could pause the presentation.
  • It could present another optional example from a bank of examples to further clarify a topic.
  • It could ask one or two knowledge checks from a bank of questions. An NUI based system could also ask questions of the learner in a relatively natural voice (text to speech) based on an algorithm.
  • It could simply give a verbal prompt to confirm if the learners are following and if the material is moving too slowly or quickly and then wait for and interpret the user response.
  • It could offer some remediation, such as offering to take the user back one slide.
  • It could suggest good web-based materials to review.

Now, supposing such software were developed and incorporated in some way into learning management and delivery platforms, there is the question of what sort of efforts it would take for an Instructional Designer to develop some of these items (banks of examples, multiple explanations at different levels of depth / detail, questions, and narration tracks for all of these). As well, to what extent that development process could be streamlined or automated (using more improved future text to speech capabilities, for example) to prevent the development from being unwieldy. Also, if this software was incorporated into Learning Management Systems (LMS) or Learning Record Systems (LRS), which track learner session data, there might be concerns about privacy, as to whether at least some high level affective state data from the session should be kept on record. Designers might find this sort of data about how student reaction invaluable for evaluation purposes. But learners might find it creepy.

But, these questions aside, the fact is that the sensor technology with the capabilities to collect the needed data to support such a thing exists. At this point it would simply be a matter of a knowledgeable team or teams doing the hard work of writing the algorithms. And if used, it could potentially lead to much more adaptive and user-customized delivery of e-Learning, living up to the e-Learning marketing over-promises of yesteryear.

Microsoft Kinect for Windows Development

Microsoft’s Kinect for Windows development pages include access to the SDK (Software Development Kit).

The SDK with Kinect for Windows APIs is free to download and to use (no direct software licensing fee) to develop Windows apps that use the Kinect for Windows sensor.

Microsoft sells the special Kinect for Windows sensors (distinct from and somewhat more capable for commercial applications than the sensor used by the Xbox gaming platform) separately for $250. (There are apparently discounts for students and educational institutions.)

Microsoft also includes Human Interface Guidelines (HIG) with this package.

http://www.microsoft.com/en-us/kinectforwindows/develop/

http://blogs.msdn.com/b/kinectforwindows/archive/2013/09/16/updated-sdk-with-html5-kinect-fusion-improvements-and-more.aspx

http://www.microsoft.com/en-us/kinectforwindows/Faq.aspx

Research studies along these lines

In writing this article, I was pleased to discover that researchers are already looking into ways to use sensors and algorithms to automatically detect affective (emotional) states of learners relevant to attention and learning. Here is some further reading for those who are interested:

http://www.emote-project.eu/

http://www.emote-project.eu/wp-content/uploads/2013/12/Affine.pdf

http://www.educationaldatamining.org/EDM2013/papers/rn_paper_09.pdf

http://ceur-ws.org/Vol-997/pale2013_paper_1.pdf

http://scholarworks.rit.edu/cgi/viewcontent.cgi?article=5772&context=theses

http://www.aaai.org/ocs/index.php/FLAIRS/FLAIRS12/paper/viewFile/4447/4843

A curricula for teaching computer programming / software development

Introduction

A lot of my posts tend to focus on new technologies and looking at how those could impact education and instructional design. But I’m going to switch gears for a bit today with a draft idea for a teaching plan for the classroom. This idea is for a curriculum of projects in a high school enriched or CEGEP/junior college level class on computer science and teaching computer programming.

Motivation behind this article

I’ve recently been enjoying a MOOC course that I’m taking from Rice University through Coursera. The course is an Introduction to Interaction-Programming in Python. It’s great. Mix of teachers gives a variety of styles of presentation. I’ve  come to realize more how important it is to give instructors in MOOC course the creative room to express their personality for their buy-in to the approach.

The course is nine weeks and has a nice project based approach where each week there is a new project. The projects start out simple and get more sophisticated as the weeks go by. The final project is a simple space shooter video game.

https://www.coursera.org/course/interactivepython

Anyhow, the material taught and the project based approach got me to thinking about how introductory computer science is taught and gave me the idea for an approach to teaching computer programming in the advanced high school or junior college classroom. (I took such a course myself in high school, and it was one of the more influential I took during that time period)

A note about programming / software development

Although computer science types are often stereo-typecast as loners without social skills, the reality is that modern software development is a highly social field. The largest software projects could not be completed in any reasonable amount of time by one individual, in some cases probably even in a lifetime. Tens, or at the high end, even hundreds or thousands of programmers can work together on one project. They need to communicate, coordinate, divide tasks/delegate, and collaborate. These are key team skills. Not to mention skills of decision-making and conflict resolution.

The open source programming movement is another good example where the emphasis is on sharing and collaboration.

The Course Design

Make the class be based on projects, with the project being some sort of computer program. The projects would be chosen so as to require larger sorts of programs to complete. Each will take hundreds of lines of code of complete. One of these every two weeks say. Would be cool too, ideally if each of these biweekly projects contributed to one large end of term project. That would take significantly more effort for the teacher to plan though. Should be one larger term project though, also collaborative.

These projects would be collaborative, built in groups. Divide the class into 3-4 groups, depending on class size. Want 6-8 students in each group.

The first few weeks could give some basic orientation, the chance for the class to read the first couple of chapters of the text, and go through some basic individual exercises. You can get some idea from this about who are the keeners and potential leaders, both for projects and for project sub-groups.

Once the class is ready, a few people could volunteer to be the main architect/lead developers for the program for each group or the teacher could assign. These leads could select say another one or two students to help design the higher level structure of the program. The teacher would support as needed. Those students would be able to get the highest potential mark, say an A+ or 100% because the task requires higher level thinking and work, and also because that’s the position that would get paid better in the real world. Then the other students in the team can be assigned to or choose different sections of the code. Those sections would be worth different amounts, based on the complexity of the task. One chunk or task might give a max of 85%, another 70% say.

Everyone can get something that gives them a solid pass.

Opposing teams test each other’s compiled executables (group # 1 grades 2, 2 grades 3, 3 grades 4, 4 grades 1, e.g.), trying to find bugs and writing bug reports. The teacher would show them how to perform bug testing and some templates for writing bug reports.They hand these back to the development group and give one chance to fix it. If this step is required, maybe the whole group gets reduced by a flat 5% for simplicity, whether one bug or many. (Aim for zero errors)

The team then verifies and tries to fix the bugs within a certain time window and then sends it for the second review. If the development group can’t reproduce the documented bugs, the team can appeal to the teacher, who will also investigate and cancel the deduction if he also can’t reproduce it. It is put through another full test again at this point. Bugs that remain are documented and a sliding grade reduction applies to the development team members based on what remains. (Not just the original bugs are tested – sometimes fixing the bug causes new bugs, so a full additional test is needed.)

The teacher calculates the grade based on this peer review. The peer review encourages teams to compete against each other in developing testing skills. The teacher makes his own review, and factors this into the grade as well. After each project, once evaluations are done, the teacher will take a class to review each teams projects together as a class, to discuss lessons learned.

Group leaders / architects need to develop testing routines based on reading and teaching from the text and the teacher and need to make sure others in the group apply testing on their own work to avoid losing marks.

The groups would not be permanent over the year, but would be picked anew for each project. Even the team leads might change as someone else within the groups shows an elevation in skills to a new level and the instructor and/or class feels he deserves a chance to try leadership. The teacher should instill values in those who regularly are team leads/project architects that it is important for leaders to step aside enough to give team members a chance to grow and shine.

Challenges: points still to resolve

The instructor would need to find a system to decide how the errors made by individual team members in their portion of the project affect the grade of everyone else. On the one hand you want to encourage the team leaders’ management skills development and the whole team’s teamwork skills. At the same time you want to prevent anyone from freeloading off the others in the knowledge the better students will do it for him out of concern for their own marks. (As is often a problem in group efforts, in school and in the workplace) Part of an individual’s mistake would reflect on the group, and part would be his own deduction.

Foreseeable critiques

One critique I would expect from educational circles would be that the competition and team rivalry aspect might be discouraging or off-putting to girls, who are already under-represented in the computer science classroom. On the other hand, the emphasis on the collaborative aspect of software development with it’s aspects of precise and effective communication, coordination, and collaboration might actually make things more attractive to the girls.

As well, the competitive aspect can be kept to relatively safe stakes. The instructor can emphasize that the focus in grading is on the objective absolute quality of the work each team does and not the ranking between the groups on a project. The recognition or honors given by the teacher are not an all or nothing thing lavished on the winning team each time. The lessons learned discussions post project also allow some of that essential soft skills aspect to come to the front.

Arguably, this approach actually brings out a balance in hard vs soft skills much better than the usual methods, and may be the sort of mix that is attractive to both genders.

Possible Modifications

One possible modification of this that I could think of that would subtly shift the tone would be to have the different teams collaborate rather than compete. Each group would get a different, but comparably difficult sub-task for implementing one overall programming task. This would require more time from the teacher to plan and coordinate all this, to think up a proper sized overall project and to evaluate multiple tasks rather than each group doing the same thing. The teacher would also probably have to take on the role of project architect, making the basic overall plan and structure and delegating well-defined but demanding sub-programs to develop to each group and monitoring what they are doing and that it fits in with everything else.

The other wrinkle is that ostensibly the different teams’ programs / modules would need to interact with the other teams’ modules to function. This could complicate testing.

Perhaps the best way would be for an instructor to try the competitive approach first a few times, and when he feels comfortable with it, then try the course collaboratively.

 

Observations on MOOCs and Teaching Styles

I’ve been spending some time in the past year taking MOOC courses. MOOCs, for those not familiar with this recent buzzword in education, are Massive Open Online Courses. They are courses built by universities, taught by university professors, but delivered through the internet to anyone who wants to take them, anywhere. For anyone who just wants to learn the material and go through the exercises, the courses are usually free to take. There are, increasingly, options to pay for the chance to take an exam for university credit or to get a certificate of completion. But if you just want to learn about some subject that you’re curious about at a university level, it is a great option.

University MOOCs are delivered, for the most part, through large consortiums such as Coursera, Udacity, and EdX, but the individual courses themselves are built by the universities themselves, and the universities maintain their own homepages through the consortium sites and maintain some university branding.

One course I took last summer was on Teaching with Moodle, put on by the folks at Moodle. Another one I’m taking now is from Rice University through Coursera on Introduction to Interaction Programming in Python.

https://www.coursera.org/course/interactivepython

This has been an illuminating experience. One of those things that is on my wishlist in terms of projects I would like to work on professionally is to work as an Instructional Designer helping university professors to make MOOCs. And over the past year as a student I’ve learned a few things. I hope to publish a longer post detailing all my observations once I’ve taken a few more courses and seen more variety in terms of how the concept is implemented well, and how it is implemented poorly.

I’d like to take the time now to share a couple of these early observations.

The first key observation is about the general need for buy-in from classroom instructors if you are either relying on them as subject matter experts (SMEs) for self-paced e-Learning or helping them to teach using an online platform, whether through asynchronous pre-recorded videos or a virtual classroom. They are a key stakeholder, and if they’re not comfortably on board, the process is going to be much harder and the results less successful than they could have been. This is the sort of thing they teach you in Educational Technology courses, but the understanding of this became much more concrete and visceral in practice. I learned this basic lesson well earlier in my career while working on a 15 month long project with a handful of courseware developers focusing on converting certain theoretical material in an Instructor Led Training course on maintaining Full Flight Simulators to self-paced e-Learning. This involved working with a number of instructors. One of the big challenges in working with the instructors was getting their full engagement and confidence. Some of them saw it as some sort of potential challenge to their livelihoods. Which is a hard mindset to deal with if you want to encourage them to give of their time.

The second observation is more specifically for Instructional Designers who would find themselves as Online Pedagogical Consultants in universities building MOOC content or helping instructors do so.  We want instructors to be comfortable with MOOCs and not them not as a threat. Rather, we want them to see it as just another media through which they can express themselves. To do this, though, you need to give them space to express their personality in the style of presentation.

The programming course I am taking gave a good chance to see this, as it uses four different Rice University Computer Science professors. The content presentation videos were short, ranging from about five minutes up to a max of about 15 minutes, and these videos were distributed between the different professors. Each of them has their own personal style of presentation. Their individual personalities come through. And as a learner, it was nice. I prefer some of them over others, but nevertheless, I like that it gives some variety. It makes it feel like the mix you see in a university.

This may be a challenge for some of us as Instructional Designers, as our usual instinct in online training is to try to find ways to standardize stylistic aspects of communication across courses. This is usually because we feel some pressure from marketing types with a stake in the programs, wanting to “establish a consistent brand.” There is something to that, and there is an element there where we have to standardize look and feel so that students can get a predictably good experience. This includes the look and controls of video/media players and the overall visual “container” template, e.g. MOOC delivery platform logo and login information, school logo, where the course title is, and where the high level navigation elements of the course are on the page. These are essential visual navigation and control elements that need to be standardized for better learner experience. As well, we do need to take the advantage of our time with the professors to give whatever coaching we can to impart some known tips for better lesson delivery, whether on video or in the classroom. But the point is that it can’t be too homogenized.

There is a value to the students, qualitatively and likely quantitatively too in allowing the professors to express their unique character. It brings the course to life more like a real classroom and makes it much less industrial and sterile. (Which may do something to help with student motivation to complete and help improve the dismal completion rates for MOOCs) There is perhaps sometimes a place for sterile, industrialized training, but I don’t think the university should give up the ghost just yet.