January 2013: Is Working Memory Training Effective?

Overview

This review by Melby-Lervåg and Hulme (2012) is a meta-analysis of the effectiveness of instruction in and practice with working memory. A meta-analysis is a statistical summary of the findings from multiple studies that address the same topic. The authors found that working memory training improved working memory on tasks that aligned with those used for the training but that these effects became smaller as the time between training and assessment increased. Importantly, training in working memory was not found to relate to the kinds of improvements teachers would value in reading and mathematics. For the academic outcomes examined, effects were nonsignificant both immediately after training and at later follow-up.

There are several reasons for the high interest in working memory training studies. For example, because working memory is strongly associated with performance in mathematics and reading, and students who struggle academically often have low working memory, it follows that improving working memory might also improve academic skill learning or general cognitive skills that support learning, such as attention. In other words, because working memory may support learning across multiple domains, training in working memory may be an efficient means of targeting a range of desired outcomes, including academic ones. It is possible that whereas a computation intervention may improve arithmetic, a working memory intervention might improve not only arithmetic, but also mathematics word-problem solving and reading comprehension. Finally, this meta-analysis is also relevant to education because working memory training and similar interventions attract much attention both in the research literature as well as the lay media and because several programs to train working memory (e.g., Cogmed, Luminosity) are commercially available and are increasingly popular.

Working memory involves the simultaneous holding, processing, and updating of information in memory to accomplish particular goals, such as reading to understand a text or solving mathematical problems. However, in practice, working memory can have numerous meanings, and its relations to academic skills may vary according to how it is measured (Raghubar, Barnes, & Hecht, 2009). Working memory tasks typically involve the recall of information from memory in situations where it is difficult to rehearse the information; therefore, working memory is said to be of limited capacity. Working memory tasks are different from short-term memory tasks, which require repetition without active manipulation, dual processing, or updating. Working memory tasks can vary along a number of dimensions, such as auditory/verbal versus visual/spatial content.

Working memory tasks often require the following:

1. Concurrent storage/maintenance and manipulation (e.g., listening to a string of digits and repeating them in reverse order)
2. Concurrent processing and storage/maintenance (e.g., determining the truth of a series of sentences while also remembering the last word of each sentence)
3. “Keeping track” according to a rule while memory is being continually updated (e.g., hearing a string of letters while also noting each time the letter x repeats after one, two, or three intervening letters, as in “n-back” tasks).

Study Design

The Melby-Lervåg and Hulme study is a meta-analysis, meaning that it summarizes studies already conducted, rather than providing a new study. Meta-analyses are valuable because they cull from appropriate studies on a particular topic and statistically analyze findings. Why is this helpful? Because if, for example, you read only one study, and it had a positive outcome about memory training, you may conclude that memory training is valuable. However, when you look across a large number of studies, you may find that the prevailing findings are less positive. Reading and quantitatively summarizing many studies for you is the value of a meta-analysis.

The authors note several descriptive, or narrative (qualitative, not quantitative), reviews of working memory training. A meta-analysis quantifies such results through effect sizes (see our November column for a discussion of effect sizes) across a range of studies that meet specific inclusion criteria. Effect sizes in this study were computed as Cohen’s d, representing the difference in gain from pretest to posttest between the treatment and control groups. In this context, an effect size d of +1.00 means that the treated group improved one standard deviation more than the control group. The effect size is significant if it is large and consistent enough across studies to indicate that the findings are not due to chance; this consistency is captured in a range that is called a confidence interval, or CI (e.g., a CI of -1.00 to +1.00 indicates a very inconsistent effect, not favoring either group, whereas a CI of +.50 to +1.00 indicates a more stable and positive effect, favoring the treated group). If a CI includes 0 (i.e., it contains both positive and negative effect sizes), the effect size is nonsignificant.

The researchers electronically searched several databases, identifying 227 potential studies. Of these studies, 113 were excluded and 114 were further evaluated. Of the studies further evaluated, 23 were included because they compared a control group with a group that received working memory training for at least 2 weeks. The remaining studies were excluded because they did not have specific training programs or did not supply the necessary data to compute effect sizes. Moderators of effectiveness were also evaluated, including age, tutoring dose, design (randomized or not), type of control (treated or not), learner status (learning disabled or other), and intervention program (Cogmed or other).

Key Findings

Melby-Lervåg and Hulme arranged their results according to target outcomes. First, they explored training effectiveness for other working memory tasks, then for measures of verbal and nonverbal ability, and then for academics. The main findings suggest diminishing returns, with smaller and/or more nonsignificant findings as the resemblance of the target skill to the trained task decreases and as the time between training and assessment increases.

Working Memory Tasks

1. For verbal working memory, the average immediate effect (21 effects) was d = +0.79 (CI, +0.50 to +1.09) and significant, and the average delayed effect (6 effects) was d = +0.31 (CI, -0.19 to +0.80) and nonsignificant.
2. For visual working memory, the average immediate effect (18 effects) was d = +0.52 (CI, +0.32 to +0.72) and significant, and the average delayed effect (4 effects) was d = +0.41 (CI, +0.13 to +0.69) and significant.
3. There was one moderator effect for age, with younger samples producing larger effects for immediate verbal working memory.
4. There was one moderator effect for program, with Cogmed producing larger effects for immediate visual working memory.

Transfer to Other Tasks, Including Academic Ones

1. For nonverbal ability, the average immediate effect of working memory training (22 effects) was d = +0.19 (CI, +0.03 to +0.37) and significant.
2. For verbal ability, the average immediate effect (4 effects) was d = +0.13 (CI, -0.09 to +0.34) and nonsignificant.
3. For the Stroop task, which measures the efficiency with which irrelevant information can be ignored, the average immediate effect (10 effects) was d = +0.32 (CI, +0.11 to +0.53) and significant.
4. For word decoding, the average immediate effect (7 effects) was d = +0.13 (CI, -0.07 to +0.35) and nonsignificant.
5. For arithmetic, the average immediate effect (7 effects) was d = +0.07 (CI, -0.13 to +0.27) and nonsignificant.
6. There was one moderator effect for type of control, with larger effects for untreated versus treated controls for nonverbal ability.
7. A total of 16 delayed effects were reported across the above transfer tasks, and effects were low (d range was -0.06 to + 0.18) and nonsignificant.

Recommendations

Recommendation 1: Consider a key question to answer with regard to any type of training study: “What is the goal of the training?”

For example, the target outcome for training studies of working memory is typically improvement in working memory itself, though related outcomes might include other learning-related “general” abilities, such as fluid intelligence or attention, or day-to-day functioning skills, such as academic achievement. From an educational perspective, the interest is perhaps less whether working memory training is effective in general than whether working memory training improves academic outcomes directly. Further, establishing how much academic outcomes improve as a function of working memory training is important, given that well-studied interventions specifically address reading and mathematics. In other words, it would be useful to know whether working memory training improves reading more effectively than established research-based reading comprehension interventions. It is important to recognize that the meta-analysis found little evidence for transfer to academic outcomes in word decoding or arithmetic. Therefore, even if working memory training holds promise, it is premature for teachers and parents to expect that working memory training will directly and significantly improve students’ performance in these academic domains.

Recommendation 2: Consider for whom, or under what conditions, improvement occurs.

This study evaluated several moderators of the treatment effect (see our December column for a discussion of mediation and moderation). One interesting moderator was for nonverbal ability, where the overall effect was positive. However, the effect was larger for studies where controls did not receive anything other than pretests and posttests (which were all also nonrandomi;zed studies); studies with controls who received similar activities but without the key training components improved similarly to the active working memory training groups. Another moderator was for immediate verbal working memory, where younger samples showed stronger performance relative to controls than older samples. These moderators suggest that a simple “yes” or “no” is insufficient to appropriately address whether and how well working memory training “works.” The field needs further evidence about the mechanisms of action by which training improves which, if any, skills and the conditions under which, and for whom, improvement occurs. Furthermore, not yet enough studies evaluate effects of working memory training on aspects of academic function other than arithmetic and word reading.

Recommendation 3: Consider that research provides many tools that assist in evaluating whether a technique or method of training is effective and that research findings may differ from what appears in marketing materials for commercial products.

Sometimes, a particular program or technique is adopted without evaluating rigorous evidence. Meta-analyses such as this one, individual high-quality studies (especially those that involve random assignment, sufficient participants, and adequate measuring of short- and longer-term effects of training and transfer to untrained skills), and reviews (e.g., Shipstead, Hicks, & Engle, 2012) all communicate scientific results that are of use not only to researchers, but also to educators.

In sum, scientific evaluations, such as the one reported here on working memory training, communicate what is currently known with respect to the quantity and quality of evidence in support of or against a particular program of relevance to education. The results of the present study were pessimistic with regard to the effectiveness of working memory training, particularly for far-transfer outcomes, such as academics. This finding does not necessarily mean that such effects could not occur; instead, the evaluation shows that at present, there is little direct and/or robust evidence of such effects. Further research may better determine for whom, under what circumstances, and for what outcomes such training programs may or may not be effective.

Further Reading

Raghubar, K. P., Barnes, M. A., & Hecht, S. A. (2010). Working memory and mathematics: A review of developmental, individual difference, and cognitive approaches. Learning and Individual Differences, 20, 110–122.

Shipstead, Z., Hicks, K. L., & Engle, R. W. (2012). Cogmed working memory training: Does the evidence support the claims? Journal of Applied Research in Memory and Cognition, 1(3), 167–199.

Following the above article are commentaries with varying perspectives by Gathercole et al.; Gibson et al.; Jaeggi et al.; Klingberg; Logie; Morrison & Chein; and Shah et al.