Buy this book on Lulu.

This book is sold at cost. If you would like to support this project, you can send $5 (or any other amount) via PayPal. All money will be used to pay for third-party editing and some much-needed graphic design work.


Copyright © 2017-18 Greg Wilson.
All content made available under the Creative Commons - Attribution License.

CC-BY License

Generated 2018-07-19.

5) Individual Learning

After reading this chapter, you will be able to

  • Explain what metacognition is and why it is important to learning.
  • Explain what near and far transfer are, and correctly identify which one occurs most often.
  • Name and explain six strategies learners can use to accelerate their learning.
  • Explain why working long hours reduces productivity.
  • Define calibrated peer review and explain its benefits for learning.
  • List common myths about computing education.

The previous three chapters have looked at what instructors can do to help their learners. This chapter looks at what learners can do for themselves by changing their study strategies and getting enough rest.

The key to getting more out of learning is metacognition, or thinking about one’s own thinking processes. Just as good musicians listen to their own playing, and good teachers reflect on their teaching (Chapter 8), learners will learn better and faster if they make plans, set goals, and monitor their progress. It’s difficult for learners to master these skills in the abstract—for example, just telling them to make plans doesn’t have any effect—but lessons can be designed to encourage certain study practices, and drawing attention to these practices in class helps them realize that learning is a skill that can be improved like any other [McGu2015,Miya2018].

The big prize is transfer of learning, which occurs when one thing we have learned helps us learn something else more quickly. Researchers distinguish between near transfer, which occurs between similar or related areas like fractions and decimals, or loops in different programming languages, and far transfer, which occurs between dissimilar domains—the idea that learning to play chess will help mathematical reasoning or vice versa.

Near transfer undoubtedly occurs—no kind of learning beyond simple memorization could occur if it didn’t—and instructors leverage it all the time by giving learners exercises that are close in form or content to what has just been presented in a lesson. However, [Sala2017] recently analyzed many studies of far transfer and concluded that while we might want to believe in it:

the results show small to moderate effects. However, the effect sizes are inversely related to the quality of the experimental design We conclude that far transfer of learning rarely occurs.

When far transfer does occur, it seems to happen only once a subject has been mastered [Gick1987]. In practice, this means that learning to program won’t help you play chess and vice versa.

5.1) Six Strategies

Psychologists study learning in a wide variety of ways, but have reached similar conclusions about what actually works [Mark2018]. The Learning Scientists have catalogued six of these strategies and summarized them in a set of downloadable posters. Teaching these strategies to students, and mentioning them by name when you use them in class, can help them learn how to learn faster and better [Wein2018].

Spaced Practice

Ten hours of study spread out over five days is more effective than two five-hour days, and far better than one ten-hour day. You should therefore create a study schedule that spreads study activities over time: block off at least half an hour to study each topic each day rather than trying to cram everything in the night before an exam [Kang2016].

You should also review material after each class (but not immediately after—take at least a half-hour break). When reviewing, be sure to include at least a little bit of older material: for example, spend 20 minutes looking over notes from that day’s class, and then 5 minutes each looking over material from the previous day and from a week before. (Doing this also helps you catch any gaps or mistakes in previous sets of notes while there’s still time to correct them or ask questions: it’s painful to realize the night before the exam that you have no idea why you underlined “Demodulate!!” three times.)

When reviewing, make notes about things that you had forgotten: for example, make a flash card for each fact that you couldn’t remember, or that you remembered incorrectly. This will help you focus the next round of study on things that most need attention.

The Value of Lectures

According to [Mill2016a], “The lectures that predominate in face-to-face courses are relatively ineffective ways to teach, but they probably contribute to spacing material over time, because they unfold in a set schedule over time. In contrast, depending on how the courses are set up, online students can sometimes avoid exposure to material altogether until an assignment is nigh.”

Retrieval Practice

Researchers now believe that the limiting factor for long-term memory is not retention (what is stored), but recall (what can be accessed). Recall of specific information improves with practice, so outcomes in real situations can be improved by taking practice tests or summarizing the details of a topic from memory and then checking what was and wasn’t remembered. For example, [Karp2008] found that repeated testing improved recall of word lists from 35% to 80%.

Research also shows that recall is better when practice uses activities similar to those used in testing; for example, writing personal journal entries helps with multiple-choice quizzes, but less than doing multiple-choice quizzes [Mill2016a]. This is called transfer-appropriate processing.

One way to exercise retrieval skills is to solve problems twice. The first time, do it entirely from memory without notes or discussion with peers. After grading your own work against a rubric supplied by the instructor, solve the problem again using whatever resources you want. The difference between the two shows you how well you were able to retrieve and apply knowledge.

Another method (mentioned above) is to create flash cards. In physical form, a question or other prompt is written on one side, and the answer is written on the other; in digital form, these are ideal for deployment on mobile devices like phones. If you are studying as part of a group, you can exchange flash cards with a partner; this also helps you discover important ideas that you may have missed or misunderstood.

A quicker version of this is read-cover-retrieve: as you read something, cover up key terms or sections with small sticky notes. When you are done, go through it a second time and see how well you can guess what’s under each of those stickies.

Whatever method you use, don’t just practice recalling facts and definitions: make sure you also check your understanding of big ideas and the connections between them. Sketching a concept map and then comparing it to your notes or to a previously-drawn concept map is a quick way to do this.

Hypercorrection

One powerful finding in learning research is the hypercorrection effect [Metc2016]. Most people don’t like to be told they’re wrong, so it’s reasonable to assume that the more confident someone is that the answer they’ve given in a test is correct, the harder it is to change their mind if they were actually wrong. However, it turns out that the opposite is true: the more confident someone is that they were right, the more likely they are not to repeat the error if they are corrected.

Interleaving

One way you can space your practice is to interleave study of different topics: instead of mastering one subject, then the next, then a third, shuffle study sessions. Even better, switch up the order: A-B-C-B-A-C is better than A-B-C-A-B-C, which in turn is better than A-A-B-B-C-C [Rohrer2015]. This is effective because interleaving fosters creation of more links between different topics, which in turn increases retention and recall.

How long you should spend on each item depends on the subject and how well you know it, but somewhere between 10 and 30 minutes is long enough for you to get into a state of flow (Section 5.2) but not for your mind to wander. Interleaving study will initially feel harder than focusing on one topic at a time, but that’s a sign that it’s working. If you are making flash cards for yourself, or doing practice tests, you should see improvement after only a couple of days.

Elaboration

Explaining things to yourself as you go through them helps you understand and remember them. One way to do this is to follow up each answer on a practice quiz with an explanation of why that answer is correct, or conversely with an explanation of why some other plausible answer isn’t. Another is to tell yourself how a new idea is similar to or different from one that you have seen previously.

Talking to yourself may seem like an odd way to study, but [Biel1995] explicitly trained people in self-explanation, and yes, they outperformed those who hadn’t been trained. An exercise that builds on this is to go through code line by line with a group, having a different person to explain each line in turn and say why it is there and what it accomplishes.

[Chi1989] found that some learners simply halt when they hit an unexplained step (or a step whose explanation they don’t understand) when doing mechanics problems in a physics class. Others who pause their “execution” of the example to generate an explanation of what’s going on learn faster. Instructors should therefore demonstrate the latter strategy to learners.

Explaining things to others even works on exams, though the extent of the benefits are still being studied. [Cao2017a,Cao2017b] looked at two-stage exams, i.e., a normal (individual) exam which is then immediately followed by a second exam in which students work in small groups to solve a set of problems. They found significant short-term gains for students doing exams collaboratively, but not long-term gains, i.e., the benefits visible a couple of weeks after the mid-term had faded by the final. They also found that students in the middle of the class benefited strongly, and that homogeneous-ability groups benefited, while heterogeneous groups did not.

Concrete Examples

One specific form of elaboration is so useful that it deserves its own heading, and that is the use of concrete examples. Whenever you have a statement of a general principle, try to provide one or more examples of its use, or conversely take each particular problem and list the general principles it embodies. [Raws2014] found that interleaving examples and definitions made it more likely that learners would remember the latter correctly.

One structured way to do this is the ADEPT method: give an Analogy, draw a Diagram, present an Example, describe the idea in Plain language, and then give the Technical details. Again, if you are studying with a partner or in a group, you can swap and check work: see if you agree that other people’s examples actually embody the principle being discussed, or which principles are used in an example that they haven’t listed.

Another useful technique is to teach by contrast, i.e., to show learners what a solution is not, or what kind of problem a technique won’t solve. For example, when showing children how to simplify fractions, it’s important to give them a few like 5/7 that can’t be simplified so that they don’t become frustrated looking for answers that don’t exist.

Dual Coding

The last of the six core strategies that the Learning Scientists describe is to present words and images together. As discussed in Section 4.1, different subsystems in our brains handle and store linguistic and visual information, and if complementary information is presented through both channels, then they can reinforce one another. However, learning is more effective when the same information is not presented simultaneously in two different channels [Maye2003], because then the brain has to expend effort to check the channels against each other.

One way to take advantage of dual coding is to draw or label timelines, maps, family trees, or whatever else seems appropriate to the material. (I am personally fond of pictures showing which functions call which other functions in a program.) Drawing a diagram without labels, then coming back later to label it, is excellent retrieval practice.

5.2) Time Management

I used to brag about the hours I was working. Not in so many words, of course—I had some social skills—but I would show up for class around noon, unshaven and yawning, and casually mention how I’d been up working ’til 6:00 a.m.

Looking back, I can’t remember who I was trying to impress. Instead, what I remember is how much of the work I id in those all-nighters I threw away once I’d had some sleep, and how much damage the stuff I didn’t throw away did to my grades.

My mistake was to confuse “working” with “being productive”. You can’t produce software (or anything else) without doing some work, but you can easily do lots of work without producing anything of value. Convincing people of this can be hard, especially when they’re in their teens or twenties, but pays tremendous dividends.

Scientific study of overwork and sleep deprivation goes back to at least the 1890s (see [Robi2005] for a short, readable summary). The most important results for learners are:

  1. Working more than eight hours a day for an extended period of time lowers your total productivity, not just your hourly productivity—i.e., you get less done in total (not just per hour) when you’re in crunch mode than you do when you work regular hours.
  2. Working over 21 hours in a stretch increases the odds of you making a catastrophic error just as much as being legally drunk.
  3. Productivity varies over the course of the workday, with the greatest productivity occurring in the first four to six hours. After enough hours, productivity approaches zero; eventually it becomes negative.

These facts have been reproduced and verified for over a century, and the data behind them is as solid as the data linking smoking to lung cancer. The catch is that people usually don’t notice their abilities declining. Just like drunks who think they’re still able to drive, people who are deprived of sleep don’t realize that they’re not finishing their sentences (or thoughts). Five eight-hour days per week has been proven to maximize long-term total output in every industry that has ever been studied; studying or programming are no different.

But what about short bursts now and then, like pulling an all-nighter to meet a deadline? That has been studied too, and the results aren’t pleasant. Your ability to think drops by 25% for each 24 hours you’re awake. Put it another way, the average person’s IQ is only 75 after one all-nighter, which puts them in the bottom 5% of the population. Two all nighters in a row, and their effective IQ is 50, which is the level at which people are usually judged incapable of independent living.

When You Just Can’t Say No

Research has shown that our ability to exert willpower runs out, just like our ability to use muscles: if we have to resist eating the last donut on the tray when we’re hungry, we are less likely to fold laundry and vice versa. This is called ego depletion [Mill2016a], and an effective counter is to build up habits so that doing the right thing is automatic.

“But—but—we have so many assignments to do!”, your learners say. “And they’re all due at once! We have to work extra hours to get them all done!” No: in order to be productive, they have to prioritize and focus, and in order to do that, they have to be taught how. One widely-used technique is to make a list of things that need to be done, sort them by priority, and then switch off email and other interruptions for 30-60 minutes and complete one of those tasks. If any task on a to-do list is more than an hour long, break it down into smaller pieces and prioritize those separately.

The most important part of this is switching off interruptions. Despite what many people want to believe, people are not good at multi-tasking. What we can become good at is automaticity, which is the ability to do something routine in the background while doing something else [Mill2016a]. Most of us can talk while chopping onions, or drink coffee while reading; with practice, we can also take notes while listening, but we can’t study effectively, program, or do other mentally challenging tasks while paying attention to something else.

The point of all this organization and preparation is to get into the most productive mental state possible. Psychologists call it flow [Csik2008]; athletes call it “being in the zone”, while musicians talk about losing themselves in what they’re playing. Whatever name you use, people produce much more per unit of time in this state than normal.

That’s the good news. The bad news is that it takes roughly ten minutes to get back into a state of flow after an interruption, no matter how short the interruption was. This means that if you are interrupted half a dozen times per hour, you are never at your productive peak.

5.3) Peer Assessment

Asking people on a team to rate their peers is a common practice in industry. [Sond2012] surveyed the literature on student peer assessment, distinguishing between grading and reviewing. The benefits they found included increasing the amount, diversity, and timeliness of feedback, helping students exercise higher-level thinking, encouraging reflective practice, and supporting development of social skills. The concerns were predictable: validity and reliability, motivation and procrastination, trolls, collusion, and plagiarism. However, while these concerns are legitimate, the evidence shows that they aren’t significant in class. For example, [Kauf2000] compared confidential peer ratings and grades on several axes for two undergraduate engineering courses and found that self-rating and peer ratings statistically agreed, that collusion (i.e., everyone giving their peers the same grades) wasn’t significant, that students didn’t inflate their self-ratings, and crucially, that ratings were not biased by gender or race.

One important variation on peer assessment and review is contributing student pedagogy, in which students produce artifacts to contribute to other students’ learning. This can be developing a short lesson and sharing it with the class, adding to a question bank, or writing up notes from a particular lecture for in-class publication. For example, [Fran2018] found that students who made short videos to teach concepts to their peers had a significant increase in their own learning compared to those who only studied the material or viewed the videos.

Another is calibrated peer review, in which a student reviews one or more examples using a rubric and compares their evaluation against the instructor’s review of the same work [Kulk2013]. Only once student’s evaluations are close enough to the instructor’s are they allowed to start evaluating peers’ actual work.

As long as evaluation is based on observables, rather than personality traits, peer assessment can actually be as accurate as assessment by TAs and other outsiders. “Observables” means that instead of asking, “Is the person outgoing,” or “Does the person have a positive attitude,” assessments should ask, “Does the person listen attentively during meetings,” or, “Does the person attempt to solve problems before asking for help.” The evaluation form in Appendix K shows a sample to get you started. To use it, rank yourself and each of your teammates, then calculate and compare scores.

5.4) Exercises

Learning Strategies (individual/20 minutes)

  1. Which of the six learning strategies do you regularly use? Which ones do you not?
  2. Write down three general concepts that you want your learners to master, and then give two specific examples of each. (This uses the “concrete examples” practice).
  3. For each of those concepts, work backward from one of your examples to explain how the concept explains it. (This uses the “elaboration” practice).

Connecting Ideas (pairs/5 minutes)

This is an exercise is an example of using elaboration to improve retention. Pick a partner, and have each person independently choose an idea, then announce your ideas and try to find a four-link chain that leads from one to the other. For example, if the two ideas are “Saskatchewan” and “statistics”, the links might be:

  • Saskatchewan is a province of Canada;
  • Canada is a country;
  • countries have governments;
  • governments are elected; and
  • people try to predict election results using statistics

Convergent Evolution (pairs/15 minutes)

One practice that wasn’t covered above is guided notes, which are instructor-prepared notes that cue students to respond to key information in a lecture or discussion. The cues can be blank spaces where students add information, asterisks next to terms students should define, etc.

Create 2–4 guided note cards for a lesson you have recently taught or are going to teach. Swap cards with your partner: how easy is it to understand what is being asked for? How long would it take to fill in the prompts?

Changing Minds (pairs/10 minutes)

[Kirs2013] argues that myths about digital natives, learning styles, and self-educators are all reflections of the mistaken belief that learners know what is best for them, and cautions that we may be in a downward spiral in which every attempt by education researchers to rebut these myths confirms their opponents’ belief that learning science is pseudo-science. Pick one thing you have learned about learning so far in this book that surprised you or contradicted something you previously believed, and practice explaining it to a partner in 1–2 minutes. How convincing are you?

Flash Cards (individual/15 minutes)

Use sticky notes or anything else you have at hand to make up a dozen flash cards for a topic you have recently taught or learned, trade with a partner, and see how long it takes each of you to achieve 100% perfect recall. When you are done, set the cards aside, then come back after an hour and see what your recall rate is.

Using ADEPT (whole class/15 minutes)

Pick something you have recently taught or been taught and outline a short lesson that uses the five-step ADEPT method to introduce it.

The Cost of Multi-Tasking (pairs/10 minutes)

The Learning Scientists blog describes a simple experiment you can do without preparation or equipment other than a stopwatch to demonstrate the mental cost of multi-tasking. Working in pairs, measure how long it takes each person to do each of these three tasks:

  • Count from 1 to 26.
  • Recite the alphabet from A to Z.
  • Interleave the numbers and letters, i.e., say, “1, A, 2, B, ” and so on.

Have each pair report their numbers: you will probably find that the third (in which you are multi-tasking) takes significantly longer than either of the component tasks.

Myths in Computing Education (small groups/20 minutes)

Working in groups of 3–4, vote on which of the following statements are true or false. When you are done, check your answers against [Guzd2015b] (which you can read online).

  1. The lack of women in Computer Science is just like all the other STEM fields.
  2. To get more women in CS, we need more female CS faculty.
  3. Student evaluations are the best way to evaluate teaching.
  4. Good teachers personalize education for students’ learning styles.
  5. A good CS teacher should model good software development practice, because their job is to produce excellent software engineers.
  6. Some people are just naturally better programmers than others.

Calibrated Peer Review (pairs/20 minutes)

  1. Create a 5–10 point rubric for grading programs of the kind you would like your learners to write that has entries like “good variable names”, “no redundant code”, and “properly-nested control flow”.
  2. Choose or create a small program that contains 3–4 violations of these entries.
  3. Grade the program according to your rubric.
  4. Have your partner grade the same program with the same rubric. What do they accept that you did not? What do they critique that you did not?

Top Ten Myths (whole class/15 minutes)

[Guzd2015b] presents a list of the top 10 mistaken beliefs about computing education. His list of things that many people believe, but which aren’t true, is:

  1. The lack of women in Computer Science is just like all the other fields of science, technology, engineering, and medicine.
  2. To get more women in CS, we need more female CS faculty.
  3. A good CS teacher is a good lecturer.
  4. Clickers and the like are an add-on for a good teacher.
  5. Student evaluations are the best way to evaluate teaching.
  6. Good teachers personalize education for students’ learning styles.
  7. High schools just can’t teach CS well, so they shouldn’t do it at all.
  8. The real problem is to get more CS curriculum into the hands of teachers.
  9. All I need to do to be a good CS teacher is model good software development practice, because my job is to produce excellent software engineers.
  10. Some people are just born to program.

Have everyone vote +1 (agree), -1 (disagree), or 0 (not sure) for each point, then read the full explanations in the original article) and vote again. Which ones did people change their minds on? Which ones do they still believe are true, and why?