Qualitative judgements of resemblance between road accident cases: what degree of agreement among experts?

In road accident research, judgements of overall resemblance between accident cases are sometimes used in order to group similar instances and to describe corresponding accident scenarios. This note deals with the degree of agreement among experts regarding such judgements of resemblance and case aggregations. We asked participants working as researchers or engineers in the field of road accident research to give their judgements on the resemblance of accident cases to one reference case, and then to say which cases they felt corresponded to a same accident scenario as this case. The results showed a strong agreement among these experts concerning the ranking of the cases by decreasing order of resemblance. A good agreement was obtained for cases considered by the experts as corresponding to a same accident scenario as the reference case, despite some disagreement for a few cases. These results are all statistically significant at the 0.05 level. Overall, these findings suggest that accident research using judgements of overall resemblance between accident instances can lead to results that are robust in terms of concordance among experts. Considering the limited size of the sample used, however, further research of the same type would be useful to confirm this conclusion.