We’ve been talking about adaptive learning for decades. At this point, some version of it has been built into just about every major learning management system, and the military has spent a kabillion dollars developing increasingly sophisticated ways to tailor instruction to individual learners. Yet, we still can’t seem to come to a consensus on whether it actually works.
Like most of the training technology effectiveness literature, the research around adaptive learning is kind of a mess. Adaptive learning can mean everything from adjusting difficulty to providing feedback to skipping material learners already know. And the control groups are rarely apples-to-apples. Some studies compare adaptive and nonadaptive versions of essentially the same instruction, while others compare sophisticated adaptive systems against traditional training programs that may not have been especially effective to begin with. So even when adaptive learning produces better outcomes, it’s sometimes difficult to know whether adaptation itself was responsible. In a recent meta-analysis, Fraulini and colleagues examined 30 publications and found that adaptive training produces small improvements in learning outcomes, although the benefits vary depending on what’s being adapted.
But here’s what I find particularly interesting about this paper. The authors acknowledge that some adaptive strategies, such as allowing learners to skip material they’ve already mastered, aren’t necessarily intended to improve learning outcomes; they’re intended to improve training efficiency. Yet their meta-analysis, like much of the literature, focuses on whether adaptive training improves learning rather than whether it reduces the time required to achieve proficiency. For those of us who actually build these things, that’s a pretty important distinction. After all, if a traditional course already produces highly proficient learners, why would we spend a bunch of money developing an adaptive version just to squeeze a few more points out of a posttest?
Back in 2015, my colleagues and I published a paper called that examined this exact question. We tested adaptive gunnery curriculum that used trainees’ previous performance to skip exercises covering skills they had already demonstrated. We compared it against a non-adaptive version of the same curriculum, which was already effective. Both groups achieved excellent scores, but the adaptive group completed training in 40% less time, using 57% fewer scenarios. When you consider the costs associated with running a large, complex simulator, that’s a big deal.
The question isn’t just whether adaptive learning works; it’s whether the benefits justify the investment. Since time is money, maybe we should spend a little less time asking whether adaptive learning makes people learn better, and a little more time asking whether it gets them where they need to go faster.