Numerical versus narrative: A comparison between methods to measure medical student performance during clinical clerkships

Background: Medical school evaluations typically rely on both language-based narrative descriptions and psychometrically converted numeric scores to convey performance to the grading committee. We evaluated inter-rater reliability and correlation of numeric versus narrative evaluations for students on their Neurology Clerkship. Design/Methods: 50 Neurology Clerkship in-training evaluation reports completed by their residents and faculty members at the University of Rochester School of Medicine were dissected into narrative and numeric components. 5 Clerkship grading committee members retrospectively gave new narrative scores (NNS) while blinded to original numeric scores (ONS). We calculated intra-class correlation coefficients (ICC) and their associated confidence intervals for the ONS and the NNS. In addition, we calculated the correlation between ONS and NNS. Results: The ICC was greater for the NNS (ICC = .88 (95% CI = .70–.94)) than the ONS (ICC = .62 (95% CI = .40–.77)) Pearson correlation coefficient showed that the ONS and NNS were highly correlated (r = .81). Conclusions: Narrative evaluations converted by a small group of experienced graders are at least as reliable as numeric scoring by individual evaluators. We could allow evaluators to focus their efforts on creating richer narrative of greater value to trainees.

Tags
Data and Resources
To access the resources you must log in

This item has no data

Identity

Description: The Identity category includes attributes that support the identification of the resource.

Field Value
PID https://www.doi.org/10.6084/m9.figshare.5350450
PID https://www.doi.org/10.6084/m9.figshare.5350450.v1
URL https://dx.doi.org/10.6084/m9.figshare.5350450.v1
URL http://dx.doi.org/10.6084/m9.figshare.5350450.v1
URL https://dx.doi.org/10.6084/M9.FIGSHARE.5350450
URL https://figshare.com/articles/Numerical_versus_narrative_A_comparison_between_methods_to_measure_medical_student_performance_during_clinical_clerkships/5350450
Access Modality

Description: The Access Modality category includes attributes that report the modality of exploitation of the resource.

Field Value
Access Right Open Access
Attribution

Description: Authorships and contributors

Field Value
Author Bartels, Josef
Author Mooney, Christopher John
Author Stone, Robert Thompson
Publishing

Description: Attributes about the publishing venue (e.g. journal) and deposit location (e.g. repository)

Field Value
Collected From Datacite; figshare
Hosted By figshare
Publication Date 2017-08-28
Publisher figshare Academic Research System
Additional Info
Field Value
Language Undetermined
Resource Type Dataset
system:type dataset
Management Info
Field Value
Source https://science-innovation-policy.openaire.eu/search/dataset?datasetId=dedup_wf_001::43ee41548af04b512bbddc031f17e0fe
Author jsonws_user
Last Updated 16 December 2020, 18:45 (CET)
Created 16 December 2020, 18:45 (CET)