NBOME’s assessment FOR learning platform
NBOME’s assessment FOR learning platform
CATALYST is the NBOME’s assessment FOR learning platform. In addition to the learning opportunities that the CATALYST platform offers, assessments delivered on CATALYST provide both performance feedback and educational resources for the concept being assessed.
For individuals, CATALYST is convenient and easy to use. It presents immediate feedback with learning resources to help medical students, residents, physicians and health care professionals learn and retain relevant knowledge. Then learners can see their relative areas of strength and weakness, and where they need to focus efforts in learning such as independent study, reading, or Continuing Medical Education (CME) courses. Questions can be answered at a particular period of time or as schedules permit.
For organizations, unlike current “tests” that offer a single snapshot of a learner’s performance, assessments delivered on the CATALYST platform are designed to measure learning and progress over time. Collecting information over time allows an organization to feel secure that their learners are up-to-date with important concepts, not just once in a while. Delivering an assessment on the CATALYST platform could potentially take the place of a proctored examination, allowing more flexibility for the organization and its stakeholders.
Very importantly, with CATALYST, assessments can be customized with content for all levels of medical education and practice (students, residents, practicing physicians or other health care professionals) and for a variety of purposes (retaining content, learning new content, focused remediation).
Cramming, massed study immediately before testing, may seem like a good way to keep information fresh in order to pass. However, research has shown that cramming results in poor long-term retention of content.1,2 Studies in cognitive learning have shown that more complex and durable learning can be accelerated in a number of ways: by asking learners to retrieve what they learned previously (quizzing), by providing immediate feedback, by spaced repetition and by waiting to re-study new material until a little forgetting has set in.1,2 Learning and retention can also be enhanced by interleaving the practice of one skill or topic with another and by varying the content to generalize the learning.3 As an example, research has demonstrated that residents retain and transfer surgical skills better if taught in a distributed manner rather than a single multi-hour event.4
A traditional examination, such as COMLEX-USA, is focused on evaluating an individual’s performance following a program of learning, often measured in terms of passing or failing, and based on a predefined standard. In contrast, the CATALYST platform encompasses a learning methodology referred to as formative assessment. Formative assessment focuses on a learner’s development and providing ongoing feedback to help the learner identify strengths and weaknesses and target content areas that need attention.
While traditional examinations may only provide a single score as feedback, assessments on the CATALYST platform provide learners with immediate feedback with the correct answer, a rationale explaining why that answer is correct and the other choices are incorrect, as well as references to additional learning resources. After answering each question, learners may also be asked to rate their confidence in answering it correctly to help them reflect on how certain they feel about the correctness of their answer.
Assessments on the CATALYST platform are accessible 24/7 via computer, tablet, smartphone, or the CATALYST App.
Learners can follow their own progress on the CATALYST dashboard which includes the total number of questions to be answered and how long they will be available, the number of questions the learner has answered as well as those answered correctly and incorrectly. The CATALYST platform also provides learners with their strengths and weaknesses in different content areas and ongoing access to learning resources.
Supervisors will be able to follow progress for your registered learners in a number of ways, including by setting requirements over time, the number correct, and content coverage using the supervisory dashboard which will be available later this year.
For more information about using CATALYST for your assessment please contact Client Services at 866-479-6828 or firstname.lastname@example.org.
1 Roediger HL & Karpicke JD. Test-Enhanced Learning: Taking Memory Tests Improves Long-Term Retention. Psychol Sci 2006; 17: 249 – 255.
2 Brown PC, Roediger HL, & McDaniel MA. Make it Stick: The Science of Successful Learning. Cambridge MA: Harvard University Press, 2014.
3 Pan SC. The Interleaving Effect: Mixing It Up Boosts Learning. Scientific American, 2015.
4 Moulton CA. Dubrowski A, MacRae H, Graham B, Grober E, & Reznick R. Teaching Surgical Skills: What kind of Practice Makes Perfect? A Randomized, Controlled Trial. Ann Surg 2006; 244 (3), 400- 409.
Horber DT, Flamini J, Gimpel RR, Tsai H, Shrum K, Hudson K. CATALYST: Piloting a Longitudinal Assessment and Learning Program for Board Recertification and Continuous Professional Development. Journal of the American Osteopathic Association 2020; 120(3): 190-200. doi: 10.7556/jaoa.2020.031.