| |
BGonline.org Forums
Eliminating Cherry Picking in Live XG PR Rating Lists
Posted By: Paul Weaver In Response To: Live PR Giants 2015 (Terje Pedersen)
Date: Monday, 22 February 2016, at 7:56 p.m.
Iancho Hristov of Bulgaria more than a decade ago pioneered the concept of creating a rating list based on the Snowie Error Rate of a set of recorded matches. I have used his name to create a new English word. "Ianchoing" means taking a set of matches, analyzing them with XG, entering them into the XG player profiles and compiling a PR rating list.
There are now various PR rating lists based on live matches. Last year Dmitriy, Iancho and I completed a project of Ianchoing several hundred matches, going back five or six years. Many of these matches were streamed at ABT events, but some were recorded privately. We never published our PR rating list because we had concerns about "cherry picking" and transparency. Some people who record their own matches will submit all of them, both the ones with low PRs and high PRs. However, there will be some who hide their bad matches and only submit their good ones. We call this "cherry picking."
In order to create a valid, accurate and meaningful PR list, all cherry picking must be eliminated. It is therefore necessary to disallow matches that have been privately recorded, transcribed and submitted, even though this policy will severely reduce the number of matches that are included in the PR list.
What matches should be included? All matches that are streamed at ABT events will be included. Some matches not streamed will also be included. For example, in the High Roller event at last month's NYMO, all first-round matches were recorded but not all were streamed. There is no cherry picking involved if all of these matches are used for the PR list, so they should all be included. Also, matches at BMAB events are all recorded but not streamed. These should be included. The decision about whether a given match will be included must be made before the match is played and not afterward.
In addition to eliminating cherry picking, there should also be total transparency. There should be a way to click on any player's name and see a complete list of all his or her matches, which can be easily downloaded. In other words, the system must be reproducible. Anyone should be able to Iancho his own matches, verifying his PR on the rating list.
Other issues must be resolved. What XG parameters should be used? What if a player rolls out some positions from a match, changing the PR? I encourage everyone will good ideas to tell me what you think would make a good PR rating system.
I have already begun the process of Ianchoing 2016 ABT streamed matches. The PR rating list from the NYMO will be included in my article in the next issue og PrimeTIme, the USBGF online magazine. I hope to update this list after every ABT or BMAB event in the USA. I would be happy to include results from other countries, but only if I am assured of receiving every single match that was streamed and none that were not streamed.
Eventually, I hope that someone will create an international PR rating list that will be transparent, respected and accepted.
| |
BGonline.org Forums is maintained by Stick with WebBBS 5.12.