BGonline.org Forums

Wild rollout results (bug?)

Posted By: Nack Ballard
Date: Thursday, 22 April 2010, at 1:05 a.m.

In Response To: 3 ply RO of 21$-54..over 100k (neilkaz)

21$-54

Thanks, Neil. So, now we have...

Snowie .. [S R12] 10k
GnuBG .. [S R3] 46k
XG 3-ply [S R13] 103k, and [S R40] 10k, [S R38] 10k
XG 4-ply [S R7] 19k

The question remains: "What is going on with Chase's two rollout results in bold?" They can be viewed here and here. (In case it matters, the first one is no Jacoby, and he attempted to use the same seed in both.)

I mentioned some (obvious) possibilities here, but maybe you or Xavier can see something amiss in the file output or someone else has an idea.

If Chase's results turn out to be extreme variance, then they're legit and his [S R39] 20k should be added to your [S R13] 103k, to bring the XG 3-ply total for 21$-54 up to [S R17+] 124k. Or if XG seeds are reproducible (I'm not sure I ever got an answer on that), then we count Chase's result as 10k and the total is [S R15+] 114k. You rolled so many trials (103k) that you managed to minimize the effect of his result on the average, but it's still good to be be aware that such wild results can and will occur.

OTOH, if Chase's two files combining to [S R39] 20k are bogus, the margin differs from that of your long result by .026. Does it matter, since we can conclude S is better anyway? For sure! What if you were rolling out a position and got play A beating play B by .013 when actually it is B that is better by .013? Or, worse, the rollout says A is better by .01 but actually B is better by .08? (That is, we don't know the nature or extent of the corruption or when it might strike.) You then misplay that position whenever it comes up and it throws off your intuition for all similar positions.

In other words, while I don't want to be an alarmist, if there is a bug in the XG program (marvelous tool that it is) or particular settings to avoid or some other problem to circumvent in its usage, it's better for all of us to find out sooner rather than later. [I can say that with assertion after fake results have led me down wrong analytic paths, then double-rerolling and eventually discovering several dozen corrupt Gnu files and still finding them; the downside is a lot more than just the cumulatively wasted CPU time.]

Nack

Messages In This Thread

• 4 ply RO of 21$-54
  neilkaz -- Monday, 19 April 2010, at 5:18 p.m.
  • 4 ply RO of 21$-54
    Nack Ballard -- Monday, 19 April 2010, at 6:18 p.m.
  • 4 ply RO of 21$-54
    Chase -- Monday, 19 April 2010, at 6:21 p.m.
    • 4 ply RO of 21$-54
      neilkaz -- Monday, 19 April 2010, at 6:48 p.m.
  • 4 ply RO of 21$-54
    Nack Ballard -- Monday, 19 April 2010, at 6:25 p.m.
    • 4 ply RO of 21$-54
      eXtreme Gammon -- Monday, 19 April 2010, at 6:32 p.m.
      • 4 ply RO of 21$-54
        Nack Ballard -- Tuesday, 20 April 2010, at 7:19 p.m.
  • ... and XG 3-ply RO of 21$-54
    Chase -- Monday, 19 April 2010, at 7:39 p.m.
    • ... and XG 3-ply RO of 21$-54
      Neil Robins -- Monday, 19 April 2010, at 8:38 p.m.
    • Currently [S R5]16k *NM*
      neilkaz -- Tuesday, 20 April 2010, at 7:41 p.m.
    • ... and XG 3-ply RO of 21$-54
      Nack Ballard -- Tuesday, 20 April 2010, at 7:54 p.m.
      • ... and XG 3-ply RO of 21$-54
        Chase -- Tuesday, 20 April 2010, at 8:39 p.m.
        • ... and XG 3-ply RO of 21$-54
          Chase -- Tuesday, 20 April 2010, at 8:46 p.m.
          • 41$-63, 21$-63, 51$-63 and 62$-63
            Nack Ballard -- Wednesday, 21 April 2010, at 4:35 a.m.
            • 41$-63, 21$-63, 51$-63 and 62$-63
              Chase -- Wednesday, 21 April 2010, at 10:43 a.m.
              • 41$-63, 21$-63, 51$-63 and 62$-63
                Matt Cohn-Geier -- Wednesday, 21 April 2010, at 10:51 a.m.
                • 41$-63, 21$-63, 51$-63 and 62$-63
                  Chase -- Wednesday, 21 April 2010, at 11:03 a.m.
                • 41$-63, 21$-63, 51$-63 and 62$-63
                  Nack Ballard -- Wednesday, 21 April 2010, at 2:11 p.m.
                  • 41$-63, 21$-63, 51$-63 and 62$-63
                    Chase -- Wednesday, 21 April 2010, at 4:53 p.m.
                    • 41$-63, 21$-63, 51$-63 and 62$-63
                      Nack Ballard -- Wednesday, 21 April 2010, at 5:44 p.m.
              • Nacbrac information
                Nack Ballard -- Wednesday, 21 April 2010, at 2:43 p.m.
                • Nacbrac information
                  Chase -- Wednesday, 21 April 2010, at 5:04 p.m.
                  • Nacbrac information
                    Nack Ballard -- Wednesday, 21 April 2010, at 5:25 p.m.
        • ... and XG 3-ply RO of 21$-54
          Nack Ballard -- Wednesday, 21 April 2010, at 3:39 a.m.
      • ... and XG 3-ply RO of 21$-54 w/Jacoby
        neilkaz -- Wednesday, 21 April 2010, at 3:07 a.m.
        • Will extend this 3 ply to 100k
          neilkaz -- Wednesday, 21 April 2010, at 12:38 p.m.
    • XG 3-ply RO of 21$-54 - w/ Jacoby
      Chase -- Wednesday, 21 April 2010, at 11:16 a.m.
  • 4 ply RO of 21$-54
    Nack Ballard -- Tuesday, 20 April 2010, at 8:19 p.m.
    • 4 ply RO of 21$-54
      neilkaz -- Tuesday, 20 April 2010, at 10:11 p.m.
  • 3 ply RO of 21$-54..over 100k
    neilkaz -- Wednesday, 21 April 2010, at 10:11 p.m.
    • Wild rollout results (bug?)
      Nack Ballard -- Thursday, 22 April 2010, at 1:05 a.m.
      • Wild rollout results (bug?)
        neilkaz -- Thursday, 22 April 2010, at 1:53 a.m.
        • Re-run of RO w/ random seed and Jacoby in progress
          Chase -- Thursday, 22 April 2010, at 2:08 a.m.
          • Re-run of RO w/ random seed and Jacoby in progress
            Nack Ballard -- Thursday, 22 April 2010, at 2:27 a.m.
            • Re-run of RO w/ random seed and Jacoby in progress
              Chase -- Thursday, 22 April 2010, at 2:42 p.m.
          • Completed RO w/ Jacoby and random seed
            Chase -- Thursday, 22 April 2010, at 3:05 p.m.
        • My 10k with seed 2
          neilkaz -- Thursday, 22 April 2010, at 3:09 a.m.
      • Wild rollout results (bug?)
        Chase -- Thursday, 22 April 2010, at 2:02 a.m.
      • Half as wild
        Daniel Murphy -- Thursday, 22 April 2010, at 2:37 a.m.
        • Half as wild
          Neil Robins -- Thursday, 22 April 2010, at 11:34 a.m.
        • Half as wild
          Timothy Chow -- Thursday, 22 April 2010, at 5:01 p.m.
      • 21$-54 rollout data (equities, links)
        Nack Ballard -- Thursday, 22 April 2010, at 7:50 p.m.
        • 21$-54 rollout data (equities, links)
          Daniel Murphy -- Thursday, 22 April 2010, at 8:34 p.m.
          • 21$-54 rollout data (equities, links)
            Nack Ballard -- Friday, 23 April 2010, at 9:03 a.m.
  • 4 ply RO of 21$-54 19k and finished
    neilkaz -- Thursday, 22 April 2010, at 12:08 a.m.
  • Snowie RO of 21$-54
    Nack Ballard -- Thursday, 22 April 2010, at 7:16 p.m.
    • Snowie RO of 21$-54
      Daniel Murphy -- Thursday, 22 April 2010, at 7:37 p.m.