Reporting on Hawkeye

I emailed the ICC about the Hawkeye issue, and my email was directed to Paul Hawkins, who, according to his email reply is the Managing Director of HawkEye Innovations Ltd.. Mr. Hawkins argues that the quality of the prediction is more binary that one might think - i.e, either you can make a good prediction, or you cant. I quote his response in full.

Thank you for you interest. 

Hawk-Eye is right in both instances. We are already investing a great deal of unpaid time proving that we are right in every instance too the ICC, by providing additional information to them which they are able to analyse and verify. Therefore there is an independent source who is able to verify that we are right in every instance. Proving we are right requires us to reveal
intellectual property which differentiates our system from Virtual Eye - which we have strong evidence to suggest is insufficiently accurate.

It is too time consuming to provide a detailed answer to every member of the public who questions a specific instance for free.

If you are interested enough to pay for the time that it will take to provide a detailed response to the specific instances, and you are prepared to sign a NDA, then please let me know.

Otherwise, the generic information attached also available on our website may answer your questions. The piece of information in the attached most relevant to your specific question is that the predictive accuracy is more binary than you may imagine from a mathematical perspective - either you have sufficient data or you do not.

The document that Mr. Hawkins refers to is available here (pdf on the Hawkeye Innovations Website). From this document, Hawkeye's capability is described as follows:

1. Hawk-Eye is able to deliver a system which meets the following performance criteria:
2. Pitching point accuracy under 5mm (in MCC tests it was shown to be 2.6mm)
3. Interception point accuracy under 5mm (in MCC tests it was shown to be 2.6mm)

Prediction of where the ball passes the stumps:

In all “normal” LBW instances under 15mm and average error of 5mm
In “extreme” LBW instances under 25mm

An “extreme” LBW is one where there is less than 40cm of travel between pitching
point and interception point and the batsman is hit over 2 meters from the stumps.
The current protocol has a 45mm umpire call “margin”.

Later in the document, there is specific discussion of what happens in the case of full length deliveries.

For all of the LBW appeals in the matches where the referral system has been in place, there has been enough post bounce data to make an accurate prediction. On average there has been 3.58 meters of travel between pitching and hitting the batsman. The minimum amount of travel was an Anderson. Yorker on day 2 of the Centurion Test which had 35 cm of travel. Hawk-Eye predicted that delivery would hit the stumps half way up.

If you do some basic arithmetic, it is worth considering the following. The batting crease is 122 cms from the stumps. Hawkeye on TV gives us no inclination that since the batsman is forward, and the prediction is less certain when the batsman is on the front foot than when the batsman is on the back foot. A lot of front foot LBWs easily involve a 2m distance between the front leg and the stumps. I have never seen any hawkeye representation on TV distinguishing between an "extreme" and a "normal" LBW.

Even so, my problem is with the Hawkeye representation, not with it's computation. The representation involves how the information computed by Hawkeye is abstracted and communicated to the viewer through the animation on TV. This does not reveal details that are important from the point of view of the LBW  decision. In my view, if the quality of the computated trajectory is more binary than might be initially thought (and more binary than what the distinction between the "extreme" and "normal" situations might suggest)

Before I end, I want to reflect on that email that I received and the public relations note struck within it. By that i refer mainly to the large preamble in response to my emailed questions. I quote my email below

I have a couple of questions about Hawkeye which I've been thinking about for a few months now. Could you point me to someone at ICC with whom I can discuss these issues? My email is prompted by the Tendulkar and Dilshan LBW appeals ongoing Galle Test. I think the Umpires made the correct decision in both cases (Dilshan Not Out, Tendulkar Out). But I am totally bemused that Hawkeye can claim to predict both trajectories with equal certainty - and that this is what is presented to the viewers on TV.

If the predictive accuracy is substantially binary, is that not all the more reason to declare the margin of error in  the case of each delivery? I have asked the ICC for it's take on the matter, but haven't recieved any reply yet. I hope they send me a reply, as I think it will be an important component to the discussion, and I will report it if and when they do send me a reply.

Tweet