Scientists challenge Hawk-eye's decision-making credibility

Washington, June 13 : With Wimbledon just round the corner, scientists have attacked sports decision aids such as the Hawk-Eye system, insisting that such systems should come with a 'health' warning attached, as they may not always prove to be accurate.

Researchers from Cardiff School of Social Sciences have said that Hawk-Eye, and similar officiating tools, used to supplement or replace decision-making by umpires and referees, are becoming an increasingly common part of televised sports coverage. However, this new technology has also attracted its critics.

And one such critic is Wimbledon champion Roger Federer, who during his Final in 2007 against Nadal, claimed the technology was flawed. The flaw was noticed when Nadal hit a ball which appeared to television viewers, to the umpire, and to Federer as impacting well behind the baseline. But Hawk-Eye called it in. Though Federer appealed to the umpire, but the umpire accepted Hawk-Eye's judgment.

The new research is based on the claims made by Hawk-Eye Innovations' on its website about the disputed line call. The machine reported that the ball nicked the baseline by 1mm. However, Hawk-Eye Innovations also report that the average error of the machine is 3.6mm. Considering the Cardiff analysis to be correct, the errors can be even larger than 3.6mm on some occasions.

The International Tennis Federation, which tests the machines for use, would accept that Hawk-Eye had passed its test if it called the ball in by 1mm while the true position was out by 5mm. However, Cardiff researchers have challenged the credibility of the Hawk-Eye system.

Led by Professor Harry Collins and Dr Robert Evans, the researchers argue that such devices may cause viewers to blindly trust the ability of any technological devices to resolve disagreement among humans. They also advised that a detailed understanding about the working of the device may influence public opinion and spread the benefits to all technological decision making in the public domain.

"We used Hawk-Eye as the principal illustrative example as it is the most well-known of the commercial systems and is currently being used to make decisions in major tennis competitions such as Wimbledon. Technologies such as Hawk-Eye are meant to relegate line call controversies to the past, however, our analysis has shown that Hawk-Eye does not always get it right and should not be relied on as the definitive decision maker," said Professor Collins.

The researchers propose the Automated Decision Principle, which states that automated sports decision aids should not correct but should reproduce human systematic errors. This means that the typical errors made by human judges and viewers such as calling a tennis ball out' when it looks out to everyone even if the electronics suggest it might just have been in.

Also, the devices should be used as they are now to correct or reduce human random errors, which come from lapses of concentration, an obscured view or very fast action. However, one should also make it a point to consider that machine can also make mistakes.

If adopted, this Principle, would involve significant changes in the way devices such as Hawk-Eye are currently used in sport.

Collins said: "To avoid any chance of misleading the public we believe that Hawk-Eye's conclusion should be accompanied by statements and/or displays of the size of the possible errors, as is normal in science. This would ensure that the public is much better informed as to the limits and possibilities of technology.

"Although such systems have not have been designed for this purpose, they could actually become a useful way to educate the public of the limits and possibilities of technology and serve to ask questions about the degree of certainty the public can actually attribute to scientific measurement and technological tools. What's more, even perhaps create a new role for sports commentators whereby they would interpret and explain levels of uncertainty for the viewer."

The researchers summarised by saying that Hawk-Eye might be in danger of unnecessarily changing the traditional nature of certain games because it does not take account of traditional areas of human systematic error when making judgements. These systematic errors are an integral part of the sport. They include giving the benefit of the doubt to batters in the case of lbw or judging tennis balls to be out when skidding gives the appearance of their being out even if they have just clipped the line.

The Cardiff paper 'You cannot be serious! Public Understanding of Technology with special reference to Hawk-Eye', will be published in the journal Public Understanding of Science.

ANI