Does the presence of neutral umpires mitigate bias in cricket?

A. Nilesh Fernando and Siddharth Eapen George, Out of Sight, Out of Mind? The Effect of Peer Evaluators on In-Group Bias (2022), University of Notre Dame and National University of Singapore (working paper).

In 1986, Imran Khan, captain of the Pakistan cricket team, invited Indian umpires V. K. Ramaswamy and Piloo Reporter to officiate Pakistan’s match against the West Indies in Lahore. It was the first time that neutral umpires stood in a Test match, prior to which only umpires of the hosting nation officiated matches. It was an era when home teams — Pakistan, more so — were often criticised of benefiting from the bias of these home umpires. Khan would later speak of the resistance from cricket traditionalists for introducing the idea of neutral umpires. And after years of toying with the idea, the International Cricket Council (ICC) relented and began appointing one neutral umpire per Test on a trial basis from 1992. In 1994, the practice was institutionalised for all Tests, and in 2002, the world body made it made it mandatory for both on-field umpires to be from neutral countries.

This continued until 2020, when the COVID-19 pandemic hit, and the ICC reverted to the home-umpire model to reduce the logistical strain in the attempts to restart international cricket. While there were no controversies since the home umpires took charge, former players like Sunil Gavaskar called for neutrals to be reinstated so as to leave no room for the perceived bias toward the hosts. And so, in April 2022, the ICC announced that with the relaxation of COVID-19 protocols, there would be one neutral umpire per Test from May, beginning with the Bangladesh vs Sri Lanka Test series. The move has reignited a debate on whether nationality-neutral umpires are necessary in cricket with improvement in umpiring standards and the advancement of technology such as the Decision Review System.

The necessity for neutral umpires

Now, a working research paper by two economics professors posits that the presence of one neutral umpire significantly reduces the bias towards a home team. The premise of cricket umpiring was used to study how the presence of peers influences in-group bias. The researchers found that panels with a neutral umpire award 14% more discretionary decisions against the home team and 23% fewer discretionary decisions against the foreign team. A discretionary decision, for the purpose of the study, is one that is more directly affected by umpiring bias, like an lbw, a run-out or a catch by the wicketkeeper.

Having scraped through the data of international matches from 1877 to 2011, the authors studied the behaviour of home and neutral umpires involving 1,09,996 decisions in 3,891 matches, both ODIs (One day International) and Tests. The findings show that despite the introduction of TV umpires and match referees to reduce bias of on-field umpires, the presence of a neutral standing umpire still had a more significant de-biasing impact on home umpires.

Neutrals do not monitor or evaluate their home colleagues and both umpires usually have a collegial relationship. “Nevertheless, neutral umpires may exert social pressure on home umpire colleagues to be impartial, as they stand on the field in the direct line-of-sight of the home umpire,” the study notes. “Overall, the introduction of neutral umpires reduced in-group bias by about 0.19 standard deviations, a slightly smaller effect than related interventions in other settings. This reduction in bias affected match outcomes: neutral umpires made matches more competitive and increased foreign teams’ odds of winning under favourable conditions.”

The de-biasing effect

However, it was also noted that the de-biasing effect was temporary. Home umpires who work with a neutral colleague are significantly less biased during that match, but their behaviour in subsequent matches, even those held just a few days later, is unaffected. The authors discuss the reasons for the de-biasing effect, noting that it could be due to factors such as less crowd pressure on home umpires when working with a neutral, and the home umpires valuing their colleague’s approval. The neutral’s presence serves as a visual reminder for the home umpire to be impartial.

There was another issue, though. The working paper finds that policies that succeed in mitigating bias along one dimension, such as nationality, may inadvertently increase disparities along other dimensions, such as race. “Because most elite umpires are white, the neutral umpire mandates often led to home country umpires being replaced by white neutral umpires. White-majority teams became significantly more likely to have an umpire of their own race, and this softened the impact of neutral umpires on white home teams,” the study points out. White umpires officiated 76% of matches, while only 52% of matches featured a white-majority team.

Takeaways from the study

The findings have implications for the design of bias-mitigation policies. By studying how the presence of peers affect in-group bias among professional evaluators, it was concluded that removing all conflicted evaluators may be unnecessary, especially when there is a shortage of requisite expertise. The authors call for further research to understand why peers reduce bias in some contexts but aggravate bias in others.

As for the ongoing debate on the neutral-umpire model, the ICC noticed that the performance of home umpires between July 2020 and February 2022 was “strong and had not affected games”. So, while T20s and ODIs will continue to have only home umpires for the 2022-23 period, the ICC wants to “utilise” more neutral umpires in Tests. The mandate for this period is that the on-field umpire, match referee and the third umpire for a Test match must be neutral, while the second umpire and fourth umpire can be from the hosting nation.