The End of Raw Scores, and the Benefits of Scale Scores
In 2018, the PAEA End of Rotation™ exams transitioned from raw scores to scale scores.
To give some historical background on the transition, it is important to understand the End of Rotation exams have at least two forms and in the case of Family Medicine, three forms. Each form is designed to measure the same content, but the specific set of questions differs from one form to the next.
While the exams are also built to be equivalent in difficulty, one form may be slightly more or less difficult than another. Programs should not use the raw “number correct” score for fair comparisons of performance on different forms of an End of Rotation exam. PAEA uses a statistical process called “equating” to compensate for the minor variations in difficulty between exam forms, resulting in scale scores that are comparable between exam forms.
There are numerous benefits to scale scores, all of which make exam score analysis and interpretation more accurate. Scale scores allow programs to fairly and most appropriately compare student- and cohort-level performance between different exam forms, years, and versions of the End of Rotation exams.
To support programs with the transition from raw scores to scale scores, PAEA provided a conversion calculator that converted an individual student’s scale score to a raw score or a whole class’s scale scores to raw scores. This was important in 2018 and 2019 when programs had students in the same cohorts, or back-to-back cohorts, that were administered End of Rotation exam forms both before and after we moved to scale scores.
Starting with the launch of the new platform and the publication of the new End of Rotation exam forms on August 23, PAEA will be removing the scale score conversion calculator and replacing it with a raw score look up table.
The raw score look-up table can still be used for converting individual scale scores back to corresponding scores. The difference is the table will be a static table compared to having a calculator where all student scores can be converted at once.
This look up table will be available through the summer of 2025 at which time PAEA will stop providing scale score to raw score conversion.
We recognize that the conversion calculator may have been beneficial to some programs; however, scale scores should be used for fair and accurate exam score analysis, interpretation, and comparison.
PAEA is committed to helping our members navigate scale scores and we will continue to offer education and resources on meaningful ways to use scale scores for grading purposes. You can also review how other programs have been using scale scores for the End of Rotation exam here.
If you have any questions, contact us at exams@PAEAonline.org.
