Results of a survey of interested researchers.

We have solicited input from the field on topics that researchers would like to be able to run inside of the ASSISTments Testbed. The Following table reflect Dr Heffernan categorization of the group of researchers listed in bold with a representive publication.  

Types of Feedback

  • What types of feedback cause the most learning (Shute, 2008)

  • Comparing types of hints provided to learners (Stamper, et al., 2013)

  • When is the best time to give feedback (Fyfe, Rittle-Johnson, & DeCaro, 2012)

Sequencing & Spacing

  • Schedules/procedures for practices & quizzes (Roediger & Karpicke, 2006)

  • Spacing skill content (Mozer, Pashler, et al., 2009)

  • The benefits of the "testing effects" over topics such as things like rereading  (Butler & Roediger, 2007)

Self-Regulated Learning & Metacognition

  • Teaching students to b better at self-regulation (Aleven et al., 2002)

  • Examining the effects of goal setting (Bernacki, Byrnes, & Cromley, 2012)

  • Metacognitive scaffolding incorporated into problem-solving (Roll, et al., 2012)

  • Testing Peer Learning (Ogan et al., 2012)

  • Email study (Kizilcec et al., DATE)

Social Context and Interaction

  • Adapting materials to students’ personal & peer interests (Walkington, 2013)

  • Embedding peer assistance (Walker, Rummel, & Koedinger, 2011)

  • How confidence impacts performance in algebra (Lang, et al., 2015)

  • Social context and social comparison across cultures in peer feedback (Choi, et al, 2019)

Interventions based upon Assessments

  • Computational models to diagnose the learner state (Rafferty & Griffiths, 2014)

  • Computational methods to assess affective states (Ocumpagh, et al., 2014)


  • Motivational videos from teachers (Kelly, Heffernan, D’Mello, et al., 2013)

  • Examining the effects of student planning (Kizilcec, Saltarelli, Reich & Cohen, 2017)

  • Changing incentive structures for appropriate usage of hints (Baker et al., 2006)

STEM Education Strategies

  • Effective presentations of worked examples in math (Booth, et al., 2013)

  • Examining strategies for learning fractions (Cordes, et al., 2007)

  • Learning the notation of mathematics (Hurst & Cordes, 2018)

  • The language of mathematics (Clinton, Walkington & Sparks, accepted)

  • How teachers frame problems (Boston. et al., 2020)

  • Cognitive demands of tasks, questions, and teacher’s follow-up (Boston et al., 2020)

  • Determining why erroneous examples are incorrect (Adams, McLaren, Durkin, Mayer, Rittle-Johnson, Isotani, & van Velsen, 2014)

Adams, D. M., McLaren, B. M., Durkin, K., Mayer, R. E., Rittle-Johnson, B., Isotani, S., & Van Velsen, M. (2014). Using erroneous examples to improve mathematics learning with a web-based tutoring system. Computers in Human Behavior, 36, 401-411.

Aleven, V.A., Koedinger, K.R. (2002) An effective metacognitive strategy: Learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive science 26 (2), 147-179

Baker, R.S.J.d., Corbett, A.T., Koedinger, K.R., Evenson, S.E., Roll, I., Wagner, A.Z., Naim, M., Raspat, J., Baker, D.J., Beck, J. (2006) Adapting to When Students Game an Intelligent Tutoring System. Proceedings of the 8th International Conference on Intelligent Tutoring Systems, 392-401.

Bernacki, M.L., Byrnes, J.P. & Cromley, J.G. (2012). The effects of achievement goals and self-regulated learning behaviors on reading comprehension in technology-enhanced learning environments. Contemporary Educational Psychology. 37 (2): 148-161.

Booth, J.L., Lange, K.E., Koedinger, K.R. & Newton, K.J. (2013). Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction. 25: 24-34.

Butler, A.C., & Roediger, H.L., III. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology. 19: 514-527.

Choi, H.N. Dowell, C. Brooks, S. Teasley (2019). Social Comparison in MOOCs: Perceived SES, Opinion, and Message Formality. 9th International Conference on Learning Analytics and Knowledge 2019 (LAK19). March, 2019. Tempe, AZ.

Clinton, V., Walkington, C., & Sparks, A. (2019). The effect of modifying the language of mathematics story problems on problem-solving measures. Presented at The 2019 Annual Meeting of the American Educational Research Association, Toronto, Canada.

Cordes, S., Williams, C.L., & Meck, W.H. (2007). Common representations of abstract quantities. Current Directions in Psychological Science. 16 (3): 156-161.

Hurst, M. A., Cordes, S. (2018) Labeling Common and Uncommon Fractions Across Notation and Education. Proceeding of Cognitive Science. 1841-1846 Retrieved from

Fyfe, E.R., Rittle-Johnson, B. & DeCaro, M.S. (2012). The effects of feedback during exploratory mathematics problem solving: Prior knowledge matters. Journal of Educational Psychology. 104: 1094- 1108.

Kelly, K., Heffernan, N., Heffernan, C., Goldman, S., Pellegrino, G. & Soffer, D. (2013). Estimating the Effect of Web-Based Homework. In Lane, Yacef, Motow & Pavlik (Eds) The Artificial Intelligence in Education Conference. pp. 824-827.

Kizilcec, R., Saltarelli, A., Reich, J. & Cohen, G. (2017) Closing global achievement gaps in MOOCs. Science 355. American Association for the Advancement of Science. Pp. 251-252. Retrieved from

Lang, C., Heffernan, N., Ostrow, K. & Wang, Y. (2015) The Impact of Incorporating Student Confidence Items into an Intelligent Tutor: A Randomized Controlled Trial. In the Proceedings of the 8th International Conference on Educational Data Mining EDM2015, Madrid, Spain. ISBN: 978-84-606-9425-0 pp 144-149.

Mozer, M, & Pashler, H. (2009) Predicting the Optimal Spacing of Study: A Multiscale Context Model of Memory. Neural Information Processing Systems Conference (NIPS). Retrieved from

Ocumpaugh, J., Baker, R., Gowda, S., Heffernan, N., Heffernan, C. (2014). Population validity for educational data mining models: A case study in affect detection. British Journal of Educational Technology. 45 (3): 487-501.

Ogan, A., Finkelstein, S., Walker, E., Carlson, R. Cassel, J., (2012). Rudeness and rapport: Insults and learning gains in peer tutoring. International Conference on Intelligent Tutoring Systems, 11-21. Retrieved from:

Rafferty, A.N. & Griffiths, T.L. (2014). Diagnosing algebra understanding via bayesian inverse planning.” In Proceedings of the 7th International Conference on Educational Data Mining. 351- 352.

Roediger, H.L. & Karpicke, J.D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science. 1 (3): 181-210.

Roll, I., Holmes, N.G., Day, J., & Bonn, D. (2012). Evaluating metacognitive scaffolding in guided invention activities. Instructional Science. Springer Netherlands. 40 (4): 691-710.

Shute, V.J. (2008). Focus on formative feedback. Review of Educational Research, 78, 153-189.

Stamper, J., Eagle, M., Barnes, T. & Croy, M. (2013). Experimental evaluation of automatic hint generation for a logic tutor. In Biswas, Bull, Kay, & Mitrovic Proceedings of the 15th International Conference on Artificial Intelligence in Education. Springer Berlin Heidelberg. 6738: 345-352.

Walker, E., Rummel, N., & Koedinger, K.R. (2011). Designing automated adaptive support to improve student helping behaviors in a peer tutoring activity. International Journal of Computer-Supported Collaborative Learning. Springer US. 6 (2): 279-306.

Walkington, C. (2013). Using learning technologies to personalize instruction to student interests: The impact of relevant contexts on performance and learning outcomes. Journal of Educational Psychology, 105 (4): 932-945.

Ostrow & Heffernan (2018) Results of a survey of interested researchers.