N1SB Labs: Testing the Rolls of GW Starter Set Dice

[N1SB]

With how much time/money we spend on our armies, should we spend a bit on dice?

I did something yesterday and it was such an useful experience for a 40k Hobbyist like me (and perhaps you), I thought it'd be worth sharing not just the results, but also process I used in case you want to set aside 30 minutes and try this yourself.

I took 16 of GW's white dice from GW starter sets (i.e. Black Reach, Armageddon, etc.), rolled them 50 times as I would in a game, and just Excel'd the results. That's basically like 800 rolls, just to have some idea how bad my rolls were.

INB4 it's not scientific enough - I agree, but the goal is to answer, "Do I want to buy (better) dice for my 40k games?" For example, I didn't use a zero-variance robotic arm to throw these because I don't use a robotic arm during 40k games (although as an Iron Hands player that would be totally sweet). I didn't separate out the different types of GW white dice, which I'm sure got mixed with my friends' sets over years of playing, but that's what happens to many of us.

TL;DR - this was not a lab environment, but it is representative of what I do in-game, because I don't play 40k in a lab environment, therefore those imperfections are more relevant to my question because they would appear in actual 40k games.

+++ QUESTION(S) +++

Do I need to buy decent dice? Are the stories of GW dice rolling ones a third of the time true? How bad are cheap GW dice?

+++ BACKGROUND +++

My friend told me about this article: https://www.dakkadakka.com/wiki/en/That's_How_I_Roll_-_A_Scientific_Analysis_of_Dice and I noticed it was referenced here on B&C 10 years ago, but didn't want to necro the thread, plus I want to share the process, because it was valuable.

Long story short - on a six-sided die, any given result should come up one-sixth of the time. But the authour of the above article found, with most boardgame (non-casino) dice, Ones came up almost a third of the time! He attributed the problem to the imbalance of pips on dice and rounded edges (as compared to hard square casino dice), because dice-makers can save money on materials.

That sounded very logical to me, so I wanted to test it out for myself.

+++ HYPOTHESIS +++

Cheap six-sided dice give worse results, in that they'll give more statistically skewed, irregular, unevenly distributed results. Instead of giving any normal given result one-sixth or 16.6 repeating% chance, can be all over the place and rolls Ones about 30% of the time, like in the dakka article. The authour suggested, "Economics wins," meaning those problems were caused by cheap dice.

+++ METHOD +++

So I chose the cheapest dice I had! I used GW's starter set white dice, the ones they begrudgingly give you with your big box purchases, because it's not a draw/selling point. You buy sets for the minis, even the rules, but no one ever says they want the dice!

I scooped 16 readily available dice (I have more, but in different bags and I was lazy) in both hands, like I would in a 40k game. I'd shake at a random amount of time while listening to a television news program...because that's what happens. Sometimes you roll fast, sometimes you're listening to your friend/opponent explain a rule while you're still shaking them bones; it's not a lab environment but it's representative of what we actually do. Then I'd arrange the dice from ones to sixes in order, look at them and record the results on an Excel spreadsheet. I did that 50 times, equivalent to 800 rolls, then looked at the results at the very end, to avoid any additional biases.

The table I used was my painting/computer table, a wooden one. It MAY have a different result with a softer mat surface, like a game mat.

(For those that want a "control group" to compare the dice rolls with, the "control group" is actually the Mathhammer. On a six-sided dice, each side should come out on average once in six times, so 1/6=16.6% repeating, etc. That's the baseline we're testing against.)

+++ RESULTS +++

Imma post the spreadsheet first, then explain, easier to show then tell:

To quickly explain the table in case it's not organised well:

Ones to Sixes: these columns show how much of each result was rolled

Total: this is just an Autosum of all the previous columns to check, "Did I record all my dice?" Can ignore, but useful during data entry

Average: the mean average should be 3.5 on a six-sided die. That's why Leadership checks, on 2d6, were usually 7, as it's by far the most common combination you'll roll. It was just a quick way to eyeball variance while I entered each roll of data. EDIT - though I initially added this out of curiosity, this turned out to be an useful reference to see if your rolls skewed on the high or low side.

Bottom Blue LIne: this is basically the final results, an Autosum of how much each result came up, then the average of that sum

Percentages: this is the real bottom line, the end goal of this exercise, to see how the rolls were distributed. I just divided the sum of each roll result by the number of times I rolled (800 altogether).

+++ CONCLUSIONS +++

I mentioned 2 averages before: an average roll should have a mean of 3.5 and a number should come up 16.6% repeating of rolls.

Welp, after 50 rolls of sixteen dice, the average roll was around 3.50...that's pretty much as close as one can hope for to the expected. As for the percent of times any face comes up varies between 15.63% to 17.88%...which is roughly within the plus-or-minus.

I set out to test this theory the authour of the dakka article suggested, that ones come up almost twice as much as they should. That's obviously bad, as that's what causes plasmaguns to blow up, but my data do NOT suggest this is the case (for my dice at least).

Despite that initial assumption, my numbers came out pretty even-stevens, disproving the theory that Ones come up most often! While there WAS a slight skew towards ones (17.63%), but are even sixes (17.88%), directly contradicting that article. I didn't mean to, as I wasn't out to embarrass the guy, if anything I wanted to match his findings so I can blame everything on my dice.

Turns out nope, the dice were fine...the problem's probably me.

+++ 3 TAKEAWAYS +++

I want to note three key takeaways, just odd things I observed, during this process.

First, this was a small sample size and the results will be even better if we rolled more. 50 rolls of two handfuls of dice? As that number increases, i'm guessing the spread will even out more because of the law of large numbers. That band of 15.63% to 17.88% will get narrower with more rolls...which really brings them close to the 1/6 chance any face should come up on a 6-sided die.

Second, the most impressive thing was how neat the average roll of 3.50 (almost matching exactly the expected Mathhammer average), but it's also worth noting how that might be because the probabilities of "opposite faces" really seem to match up. In other words, ones and sixes seem to come up about the same number of times, while twos and fives also match each other in how often them come up, then threes and fours have about the same percentage with each other.

(That probably is by design, so each pair of opposite sides all have 7 pips, holes drilled in them, which causes the imbalance...but at least the dice roll results will then balance out. In fact, I remember when I first played boardgames, I noticed opposite sides always added up to seven...this was a strong reminder.)

Third, rolling and recording, THEN looking at your overall results at the end, is the biggest benefit of this exercise; it forces us to look at the total trends, NOT a snapshot view. At some points, I absolutely convinced myself I was rolling more ones i.e. "observer bias". Note my 4th test, where I rolled 7 ones. I really fixated on that at the time, thinking it validated every complaint I had about my dice. It was only by forcing myself to enter all the data 50 times in an Excel that I could see the overall picture, which surprised me.

+++ RECOMMENDATIONS +++

Try rolling your dice yourself while implementing your own system. It doesn't matter what method you use, as long as you have a method you stick with! Having an Excel in front of me kept me honest. The data entry process and looking at the total results at the end makes you look at things objectively.

(Seriously give it a shot with your own dice. It literally took me 30 minutes waiting for a TV show to come on. It took me far longer to share this info here than it did for me to actually roll the dice and do the data entry).

No need to buy fancy casino dice...el cheapo GW starter set white dice work out fine. In fact, I reckon they're way more evenly distributed (i.e. "good) then the fancy ones they're selling with Ultramarine symbols printed on their yin-yangs.

Reason for editing - really appreciated our fellow Fraters reading and commenting. Turns out, this might be an useful post for future reference, so I cleaned it up for easier reading/user-friendliness. Thanks a lot, Brethren.

Edited May 15, 2018 by N1SB

[Adeptus]

Quality post. Well done.

[Calyptra]

You can also test the balance of your dice by floating them in salt water.

 

http://makezine.com/2015/06/16/check-balance-gaming-dice/

[Ishagu]

Brilliant stuff :-)

[the jeske]

Median overlaps with the avarge?

[gitface]

Reminds me of something that happened in our roleplay group quite some years ago.

 

We always play in a living room without a table, so rolling the dice was done of the back of a book etc. meaning dice would try to run away :( 

 

At the time, my friend had recently connected a computer to the TV for a media center, long story short I ended up writing a programme to role various dice for the different games we did.  

 

One guy we always thought he liked to fudge the dice (one of the reasons I did the roller) and we'd not being using the roller long when the failed a role and in all seriousness that we do not think he realised what he said;

 

      "I'd have succeeded that role if I was using my own dice"

 

At the time we all looked at each other knowing that what the meant was that the could of lied and fudge the result.

[Chaplain Raeven]

You should be fine with monotone coloured or clear dice. Multi-coloured dice (like from Chessex) tend to roll worse. I trew so many quadruple ones with the multi-coloured ones, it wasn't even funny anymore. I rarely have this problem with my clear dice now.

[Brother Casman]

Bravo! Well done! An excellent use of the scientific method. I did my own test a few years ago, using GW's cube o' dice: rolling 25 dice at a time, for a total of 300 dice rolls - my results, from what I recall, were much the same as the ones in the OP. In other words, the dice are probably fair enough for gaming purposes.

[Trevak Dal]

I have some chessex uh...36 count dice (they are the same size as gw ones) and I got the slightly smaller than casino dice ones. I'm buying more of the big ones because they tend to roll better. Plus I like using my crown Royal bag to tote them around.

[SickSix]

Great post. I want to test my chessex vs GW dice now.

[Brother Lame]

I've been using some of those fancy numbered dice that came in the lasgun clip tin, I'd be interested to see how they work out.

 

One thing I've always noticed though is how much more dice rolls seem to be biased when you're only rolling one or two die at a time; I keep managing to roll the same number time and again even when they're bouncing around in my cupped hands for several seconds.

[Tyberos the Red Wake]

I'm always interested in dice analysis, because even though dice never roll perfectly randomly even in the real world, and even though slight skews may never affect your games in any way, it's still something I want to at least know about, and because so much theory on the game revolves around math based on perfectly random dice.

 

I've always felt that dice are like watches. You don't need super fancy expensive ones, even the cheapest ones will do the job just fine, assuming they aren't completely defective. Dice companies and watch companies cannot do business as such if they sell dice that are so bad they can't even function as dice. Similarly, a watch that can't even tell time properly due to accuracy-within-X-period-of-time isn't really a watch, it's a piece of junk. The difference between a $50 Casio or G-Shock and a $10,000 Rolex in terms of keeping accurate time on any given day is nearly zero. Similarly, I would assume a cheap Chessex die and a Game Science die on any given roll in a game of 40K is going to be identical.

 

One thing that I would note is that GW dice are not Chessex dice, and I believe the original laboratory grade experiment from Dakka Dakka used Chessex dice as well as old GW dice. The new GW dice are noticeably different, especially when they start adding designs. The most recent ones even moreso due to having designs on both the 1 and 6 face. I'm glad the GW dice aren't terribly unbalanced as I do tend to use them over other brands just for fun.

[Wrath of Bruinen]

Thanks Not 1 Step Backwards, really interesting stuff.

 

I'm interested that Chessex are getting a bit of a bad wrap in this post. Bit of a back story - I have a reputation within my gaming circle as having pretty bad luck, so one day I decided to invest in some new dice and bought some Chessex Blue Steel/White Gemini (CHX 26823 for anyone playing along at home). Since then I've always felt like my luck improved considerably.

 

So, I've decided to test mine out - for science! As a true advocate of the scientific method, I've tried to replicate OP's experiment as closely as possible - 16 dice, rolled 50 times on a wooden desk. Below is an interesting comparison.

 

 

 

             Ones     Twos     Threes     Fours     Fives     Sixes     Total     Avg

N1SB    17.63% 15.63% 17.00% 16.13% 15.75% 17.88%   800    3.50375

TWA      15.88% 15.63% 17.50% 17.63% 14.25% 19.13%   800    3.56125

 

 

My average comes out ever so slightly higher than OP's, probably because I rolled 10 more sixes over the course of 800 rolls. On the whole they, the results are broadly consistent, suggesting that Chessex dice are no better or worse than the GW ones.

 

 

**edit - apologies if the formatting bombs out - I had everything in a nice table and then when I went to post it all scrambled. It looks nice at my end, but your results may vary...

Edited August 24, 2017 by TheWeepingAngel