Performance-based resource allocation – a cautionary tale

Performance-based allocation (PBA) frameworks are popular among funding agencies because of their perceived objectivity. Measurable criteria are thought to ensure that funds are directed to the most deserving recipients. Multilateral development institutions now disburse over $20 billion annually mostly using PBA. Here we raise a methodological concern that casts doubt over the objectivity and robustness of PBA frameworks. The problem arises when frameworks fail to distinguish between two types of performance assessments:(i) cardinal performance measurement, where differences between performance levels possess a meaningful quantitative interpretation; and (ii) ordinal performance ranking, where they do not. We demonstrate the risks of committing a ‘cardinal fallacy’where ordinal performance rankings are treated as if they were cardinal measurements. The real-world repercussions are substantial. Using a stylised variant of the World Bank IDA PBA framework we demonstrate that even a slight change in the arbitrarily chosen numerical scale for the ordinal performance rankings, say from the current 1-6 to 0-5, would in 2017, for example, have led to a reallocation of IDA funding of about $750 million. Countries like Afghanistan, Haiti and Yemen could see their IDA allocation change by more than a third.