Possible biases in risk comparisons

If one needs to compare the road safety level between countries, some measurements of road safety have to be compared. On that purpose, it is important to determine how accurate these measurements (approximately) are. In particular, the following issues have to be considered:

• Observations are likely to be biased: not all accidents may be counted and/or exposure may be under or over estimated. Moreover, no estimates for these biases may be available. If biases appear to be large and one is unable to correct for them, no reliable comparison can be made.

• The number of accidents is intrinsically variable: it is impossible, except for the case in which no accident can possibly occur, to predict the exact number of accidents. If one has to assess the potential variation in one observation, a Poisson approximation may be sufficient when the actual count is large enough. However, if two apparently equal areas need to be compared (or even the same area for a different time period), over dispersion issues have to be considered.

• The exposure figures are likely to be estimates themselves. This means that the variance in their estimates (i.e. the variance of the measurement error in the estimates) needs to be accounted for as well. Standard textbooks offer approximations to the variance of ratios, sometimes by means of linearization (delta method) or by means of simulation.

• In addition to the variance due to the fact that exposure figures are estimates (measurements), it also has to be considered that the exposure measures are approximations, proxies to the true exposure (e.g. one vehicle kilometre may not be the same as another one; the same number of vehicles may be used for more kilometres in a different time period).

The possible biases mentioned above have to be borne in mind and, when possible, corrected or accounted for. Sometimes knowledge of bias may prohibit further analysis. The consequences of unknown accident and exposure variations can only be assessed in the context of a statistical model, however no general model is available. It should be noted that the presentation of related models is not within the scope of this document. As far as exposure measurement errors are concerned, these should be accounted for in risk estimates, and this may be the most significant limitation in the use of exposure estimates. As discussed in detail in the next Chapter, the different methods for obtaining exposure estimates may account for measurement inaccuracies to a more or less efficient way.