David M. W. Powers - resources relating to the bookmaker algorithm. The original Bookmaker paper and poster derives Informedness from the idea of an edge in gambling or trading.
244224 bmpaper.doc - ICCS technical paper July
2003
804297 BMPaper.pdf
1024512 bmposter.ppt - ICCS tutorial poster
July 2003
2382655 BMPoster.pdf
84480 BMExcel.xls – 2x2 case, 3x3 case, 13x13 worksheet
27136 bmsig.xls - shows 2x2 case + significance estimates
27136 bmsmall.xls - shows 2x2 case + mean F&G factors
28160 bmsym.xls - shows 2x2 case + misinformedness case
29184 bmtriple.xls - shows 3x3 case + mean F&G factors
28672 bmwtsym.xls - shows 2x2 case + weighted F&G factors
2603 bookmaker.m - matlab/octave script for bookmaker + F&G factors
Brief motivation Powerpoint (abstract as slide 5)
motivating Informedness, Markedness and showing the connection to Correlation
and Chi-squared Significance (HCSNet 2007, Abstracts p77 and Speedpapers p 29):
http://david.wardpowers.info
Draft showing full derivation and analysis of Informedness, Markedness and
relating them to Recall, Precision, Correlation and Chi-squared Significance
(draft to be submitted) as well as to ROC analysis (Receiver Operating
Characteristics), AUC (Area under the curve), DeltaP,
Regression, etc.
http://david.wardpowers.info/BM/Evaluation_
From Precision and Recall ....pdf
In summary, Precision reflects at chance level performance the Prevalence of the positive case in the dataset, and subtracting off the Prevalence and renormalizing as a probability gives the probability of an informed prediction (versus guessed prediction) – in the binary case this corresponds to DeltaP’ or to 2AUC-1. Conversely, Recall reflects at chance level performance the Bias towards positive labels by the predictor, and subtracting off the Bias and renormalizing as a probability gives the probability of a marked prediction (versus chance association) – in the binary case this corresponds to DeltaP. The Geometric Mean of Informedness and Markedness is the Pearson Correlation. All three can be regarded as different normalizations of the Chi-squared statistic.