ÌÇÐÄvlog¹ÙÍø

In this article, we examine evaluation bias against Black, Latino, and Asian residents before and after a major change in the way internal medicine (IM) residents were evaluated that may have affected this bias. This change occurred in 2014, when the Accreditation Council for Graduate Medical Education, in collaboration with the American Board of Internal Medicine (ABIM), replaced the Resident Annual Evaluation Summary (RAES) rating system with the Milestone rating system (1–6). We focus on these groups because they report experiencing bias in medical education and are underrepresented in academic leadership positions in medicine (7–10). Understanding sources of bias is especially important for Black and Latino physicians because they are vastly underrepresented in medicine (URiM) (11–13). The Milestone evaluation system addressed several shortcomings in existing residency assessment and feedback methods, such as the lack of clear descriptions of performance expectations, an overreliance on program directors alone to evaluate residents, a lack of standardized evaluation criteria across residency programs, and little encouragement for improvement in physician competence during residency training. Changes introduced by the Milestone evaluation system included using Clinical Competency Committees to rate residents rather than relying solely on residency directors and inclusion of more descriptive and delineated rating categories than in the previous rating system (14). Although past research indicates that these types of changes have the potential to reduce evaluation bias, studies examining changes in bias across rating systems are sparse (2, 3, 15). To address this gap, we measured differences in medical knowledge ratings between Black, Latino, and Asian minoritized groups, stratified by nativity, versus U.S.-born non-Latino White (NLW) residents during the 5 years before and after adoption of the Milestone ratings. When assessing these differences, we accounted for underlying knowledge by controlling for initial ABIM certification examination score, a standardized and blinded measure of knowledge recorded soon after rating administration.