The concept of reliability - Google Sheets v0.1

[arigatous]

Update: I''ve started to actually work on Google Sheets Calculator, please refer to this post for details:

 

http://www.bolterandchainsword.com/topic/345132-the-concept-of-reliability/?p=5104674

 

...or why mathhammer usually doesn't work as intended

 

I noticed that whenever people talk of math here, they are always referring to average numbers:

- on average 9 assault marines backed up by a power sword equipped sergeant will kill 6.5 guards

- on average lascannon does 3.5 damage.

 

And when that doesn't happen, it is something which people call bad luck.

 

But in fact you can't expect to do average damage. The probability of that is only 50% - just like winning a coin throw. In order to do about average damage or more each and every time one would need to win each and every coin roll, which actually is extremely good luck.

 

Luckily, math has a lot of ways of calculating probabilities besides calculating average values.

 

Imagine the abovementioned unit of 9 chainsword-equipped marines attack the unit of guards equivalent (T3, AS 5+). What you can rely on and what really counts as good or bad luck? Look at the table below.

 

 

One can heavily rely on scoring at least 3 wounds. The probability of not scoring them is less than rolling 1 on d20 - something that doesn't really happen often.

One can somewhat rely on scoring at least 4 wounds. The probability of not scoring 4 or more is lower than rolling 1 on d6 - which nevertheless could happen here and there.

 

If you scored 5-8 wounds  - that's not a bad or good luck. That's somewhat expected.

 

If you scored 9 or more - your opponent can call you lucky bastard

 

What does it mean in terms of the game? If you need to kill that 6 guards reliably, you shouldn't charge with 10man squad as mathhammer advices. You should charge with at least 15 chainsword marines + power sword sergeant. That will give you the table below:

 

 

That's not a coin throwing anymore, that's a tactical decision. You need to kill them and you most probably will.

 

Is it worth trying to bring a calculator like that online or is the whole concept of scoring wounds reliably too heavy to use for most people? What do you think?

[Schlitzaf]

I’d be interested in Ari. Through for clarity sake do you mean 16 men or 15 men total. As an aside what you posted matches what I found the difference in 9-11 Man Crusaders and 13-15 Man Crusaders. Nice to see the math adds up.

[Gentlemanloser]

If you scored 5-8 wounds - that's not a bad or good luck. That's somewhat expected

The average of 5 to 8 is 6.5.

 

So basically you'd be somewhat expected to hit the average value, or around it.

 

If you scored 9 or more - your opponent can call you lucky bastard

But if you only got 3 you didn't have bad luck?

[arigatous]

 

But if you only got 3 you didn't have bad luck?

 

Exactly.

 

Say you need to wipe out a squad of 5 while average values suggest 6.5 average kills. If you wont, you will blame bad luck. But actually the probability of NOT killing even five is higher than rolling 1 on d6, which is not a rare case. So if you really need that tactically, you should plan ahead accordingly so you wouldn't blame luck later. Keeping in mind that average numbers are not the same as reliable numbers you could soften them in advance by a bit of shooting or keeping an msu squad in charge range so they could finish the job. But how soft should they become for a guaranteed finishing? That's what reliability table tells you. Shoot them down to 3 bodies and and you will have 97% instead of 80%.

[Kallas]

Doesn't that method of mathhammer also work out flawed? I mean, it's saying that the chance to cause one or more wounds is 100%, which is incorrect...

 

And I think that GML's point is that you can't say slightly above average is lucky, but slightly below is not unlucky. Because they're following the same statistical logic - so if above is lucky, then it follows that below is unlucky.

 

Also, regardless of method, I think this shows that Assault Marines are pretty crap at killing stuff :P

[CausticCossack]

I would definitely be interested in a calculator like this if it was available online.  Though, I must admit that I think I enjoy statistical analysis a little more than the average person...

[Gentlemanloser]

*cough* not Seahawk. ;)

 

Im also sceptical about 100%s and 'guaranteed'.

 

And 14% (chance of 9+) being lucky. While the smaller 9% (chance of not 4+) isnt unlucky.

 

At what size of chance does something qualify for luck? Either good or bad.

[Kallas]

*cough* not Seahawk. ;)

My bad, no idea how I made that mistake!!

[Gentlemanloser]

Hehe. Had that sig for years. ;)

[arigatous]

And I think that GML's point is that you can't say slightly above average is lucky, but slightly below is not unlucky. Because they're following the same statistical logic - so if above is lucky, then it follows that below is unlucky.

 

That's exactly the case. For some reason people think that luck and unluck are symmetrical around 50% but in warfare and any other planning it is not so (while probabilities distribution of course still is).

 

You can't fail a tactical plan just because you're bit unlucky. You need to have reserves to achieve a plan reliably. And at the same time when you're relying on luck for your tactical plans nobody can rely on tremendous luck. Being a bit lucky is already expecting the event which is less than 50% to happen.

 

That is why the border between luck and unluck generally lies lower than the average value.

 

Here is a very simple example from a real life planning:

 

Imagine one needs to catch a bus which departs around 9:00 AM. Could be up to 10 minutes earlier or up to 10 minutes later, the probabilities are symmetric.

 

Would we call the guy who came at 9:02 and still got on this bus lucky? Of course.

 

Would we call the guy who came at 8:58 and lost this bus unlucky? Not really, this is somewhat expected, 2 minutes don't give enough reliability.

[Ciler]

Yes, I think it would be an interesting thing to have this calculator. The key question being what value to set for a "reliable" roll or whether this could be adjustable. 90% seems a good value though. 

[Kallas]

Would we call the guy who came at 8:58 and lost this bus unlucky? Not really, this is somewhat expected, 2 minutes don't give enough reliability.

Um... the early guy is unlucky. Plain and simple.

 

That guy showed up on time: yes, the bus could arrive early but it shouldn't (because the listed time is 9, specifically for planning purposes), so he was unlucky that the bus was even earlier.

 

I strongly disagree that you can say that a result equidistant from the average is any less statistically lucky/unlucky than the other result.

 

Thing is, we are talking about probability/statistics. These aren't subjective, they have their values in these situations, and the percentages are fully representative of how lucky or unlucky they are.

 

-----

Edit:

 

Actually, let's look at this a slightly different way:

 

A single ASM attack against a Guardsman has a (1*0.66*0.66*0.66) 0.2875 chance of getting all the way to a kill.

 

Considering that, you can reasonably expect that two ASM should kill one Guardsman, as the calculations already account for the fact that three of the four will fail somewhere along the line: failing to kill one Guardsman with two ASM would be unlucky; and conversely, killing more than one is lucky.

 

The only difference between the two sides of that luck coin is the extent to which one can be un/lucky - since two ASM should be reasonably be expected to kill only one Guardsman, they can only be unlucky and lose one 'success'; on the other hand they can get lucky and successfully kill four. Here there is greater scope for them to succeed than to fail, and the greater their success then that is lucky in increasing degrees (that is to say, a large amount of luck is less likely than a small amount of luck: 1 kill is average, 0/2 kills are un/lucky, 4 kills is the most lucky).

 

In the specific case of ASM vs Guardsmen, a single attack has a 0.2875 chance of success.

 

One attack: 0.2875 kills

Two attacks: 0.575 kills

Three attacks: 0.8625 kills

Four attacks: 1.15 kills

Five attacks: 1.4375 kills

Six attacks: 1.725 kills

 

Three attacks is actually where one could reasonably expect to achieve a kill (0.8625 is comfortably over 50% likelihood). Considering that, losing a single attack (dropping chances down to 0.575; still more likely than not) does not reduce the chance from likely->not likely, and gaining one does not increase the likelihood of two kills to a statistically likely probability.

 

Adding or removing a second attack would push either result across the boundary: two attacks is a coinflip; six is more likely two kills than one.

 

So, I can agree that there are different degrees of luck. I would say that it is very much a case of perspective and that lower (ie, 'worse') rolls are not inherently less likely: each individual ASM attack has a low chance to actually kill a Guardsman, meaning that each attack can be correspondingly more lucky; but similarly, something that is extremely powerful (for example, a Captain with Relic Blade and a nearby Lieutenant vs the same Guardsmen, which works out to a 0.95 chance to kill per attack) would be extremely unlucky for every failed attack.

[arigatous]

While I'm following your point, that's not exactly what I'm talking about.

 

The trick with average values is that they become reliable if the number of experiments is very large. That is called "Law of large numbers". The more trials you have, the more chances you have to hit the expected value.

 

When the number of dice you have is low, LLC doesn't give you reliability anymore. You're now at the territory of "luck".

_______________

Now let's imagine you have 24 shots with 1/2 to hit, 1/2 to wound and no AS and 3 units of 2 enemies each against you. One of them - say unit A - is holding a control point. Other units - B and C - are just roaming.

 

Averages tell you to split fire evenly, because on average 8 shots "shall" score you 2 wounds (63% of scoring two or more, to be precise). But 8 shots don't comply with LLC, the number is too low. And that can lead to the following:

 

You scored 8 total wounds with your 24 shots, which is above average, so you're technically "lucky" according to LLC and they distributed as follows: 1 wound to unit A, 3 wounds to unit B and 4 wounds to unit C. Now you will consider yourself "unlucky", because you cared much more of killing unit A and on average you "should" have scored two wounds there. But that's not true - LLC doesn't cover small numbers and according to LLC you're lucky to score 8 wounds. You just fell in a trap of cognitive bias.

[Schlitzaf]

Ari, can you calculate how ‘average’ of two 5 Man Squads compare to 1 10 Man and 1 5 Man, then 1 15 Man? And how the average compares to “functional range”? Then while more for us Templars something I am curious about.

 

2 5 Man with Two Chain, Double Sword, and Double Special each

1 15 Man with Twelve Chain, Double Sword and Double Special.

 

They both come out to 200. But I find espaicially in melee, the 2 5 Man despite have more more Sword Attacks and better at softening targets, are notably worse than 1 15 Man. So my question, does the additional specials equate to more wounds than the 5 attack melee bros reliably over two wounds of combat.

 

I can do the average, but your chart here I think is far more interesting. Point of comparison for this circumstance. And if you can do the same comparison with 2 10 Man and 1 20 Man. See if the pattern holds.

[arigatous]

Here are the tables:

 

1. 2 squads of six (i decided to make 6 instead of 5 to make total point values equal) shooting and charging GeQ. 4 flamers, 8 pistol shots, 6 power sword attacks and 14 melee attacks. 208 points.

 

https://gyazo.com/2c4bb2341eecf3a326bcecb8041dd436

 

2. Same against MeQ.

 

https://gyazo.com/82ee14b98b8d3017b216c4dfb8e3913c

 

3. One squad of 15 (8 Ini, 7 Neo) shooting and charging GeQ. 2 flamers, 13 pistol shots, 3 power sword attacks and 25 melee attacks. 207 points.

 

https://gyazo.com/9089d74b38e5c152991902d91c82c699

 

4. Same against MeQ.

 

https://gyazo.com/0db1c517f81028a469a026617874dc46

 

So both options are pretty much the same, the difference while charging is hardly noticeable. However if we exclude flamers and add 3 casualties to simulate the subsequent round, the big squad will have an advantage of reliably scoring around one extra wound (two if lucky) vs GeQ. Both will continue to be equal vs MeQ.

[Schlitzaf]

Why Flamers if you don’t mind me asking? And why is the yellow box highlighted in each table? If we add rerolls of 1 to hit and to wound both together and separately do the two squads remain functionally comparable?

[arigatous]

Just because flamers are easier to count in my current table I don't have a complete calculator at my disposal as of now - the whole topic is about if I should spend several weekends to create it for the community.

 

In general re-rolls of 1s, while boosting the overall number of wounds, shouldn't change such a comparison a lot. Probably the unit with larger number of attacks will gain a slight advantage, but not something really noticeable.

 

What could potentially change the picture is the inclusion of probabilities of successful charges but that's too complex honestly

[BluejayJunior]

This is very interesting. I love the analysis. I know I would be interested in something like this. I know I have a habit if overestimating what my squads will accomplish in melee. I put too much reliance on Red Thirst. Something like this would definitely help me think of what I need to throw at the enemy to reliably kill it.

[Isolia]

I'd be interested too as I also find classical averaging unfit for game planning (it is however fit to compare units to acquire, as we assume a long term choice to use them and buy them , so we're moving to large datasets where averages start to be meaningful).

 

I was explaining just that the other day to someone in a hobby store that called his rolls unlucky, but it's not easy if people don't have a grasp of probabilities.

 

I like the bus analogy, as a 10 minute variation allowance means you can only call yourself unlucky if you arrived prior to 8:50 (or very close to it), since the variance is known :) I'll be implementing this in my next explanation if I'm unlucky and meet the bad dice guy again ;)

 

Keep up the good work, I hope to see this calculator that would be much, much more useful for tactical choices prior to a battle. Maybe it can be incorporated into an existing mathhammer base?

[Sheesh Mode]

I am very interested in this. 

It will be a useful tool to inform effective target prioritization. 

[arigatous]

I've started to create the whole thing in Google Sheets.

 

https://docs.google.com/spreadsheets/d/15Vv6eTKOU9PLcreTEafA_u4wXin2rq81nrel29sKFc8/edit?usp=sharing

 

Current limitations:

 

- 10 reliable wounds max (easy to extend but as a BT player I never needed more).

- max 2 weapon types at once (will probably make more in future).

- no option for re-rolling misses and such (plan to add that soon).

- Weapons will never have random shots and/or random damage characteristics e.g. D6 (that would add huge complexity to reliability calculations).

 

If you will find an error and/or will have a feature request, you're very welcome!

[arigatous]

Updated the table to 3 weapons and 15 wounds.

 

https://docs.google....dit?usp=sharing

 

That actually required adding custom scripts, which in turn required some Javascript studies  from me, so now I know that Mathammer forces its adepts to get hard skills

 

Re-rolling will probably be added today.

[M.Redbeard]

Well, it is good post and intetesting. I wamt to double check the math involved when I have access to the Google Doc.

 

But I want to point out that rolling average is not a 50% toss. It depends on the distribution of the roll. And not all rolling has the same distribution. Rolling a single dice has a homogeneous distribution and every result has the same probability. Rolling 2 dice though is not homogenous. Rolling a 7 in two dice has a probability 1/6 while rolling a 6 is of 5/36. If the distribution is symmetrical there is the same probability of rolling above or below average, but for example rolling om 2d6 since there is a normal distribution you can take the standard deviation to be sure and basically do the standard "sigma" test that you do in natural sciences, to determine your probability of doing something (since it is a finite amount of results, and a low count the amount of reliability is pretty easy in that sense). As such when playing by Mathammer and want reliability you have to check exactly what you are doing to know how reliable is to roll average. Amd to always remember that for a good roll there always will be a bad roll. Amd that nothing is ever warrantied, because there is always a probability of all 1s in your rolls.

[arigatous]

Basically the logic behind calculations is as follows:

 

1. Calculating the success probability of a single attack for a particular weapon vs. particular enemy as it's usually done in average calculations: like the basic chance of successful attack of a chainsword armed marine with rerolling 1s on hit vs GeQ would be:

[2/3+(1-2/3)*2/3]*[2/3]*[2/3]=32/81

 

2. Defining the probabilities of scoring exactly X successes from Z attacks with the abovementioned success chance for every X<=Z using the probability mass function for binomial distribution.

 

3. Calculating the resulting probabilities for up to 3 different weapons by using the following script:

 

function CalculateWounds(successes, input_B, input_C, input_D) { //Data format: Int, {},{},{}

 

var output = 0

 

for (var i = 0; i <= successes; i++) { 

  for (var j = 0; j <= successes; j++) {

    for (var k = 0; k <= successes; k++) {

      if (successes === i+j+k) { output = (output - (input_B * input_C[j] * input_D[k])); //The chance of making exactly 'successes' rolls using probabilities from {}

                        }

    }

  }

}

  

return output

     }

[arigatous]

 Amd that nothing is ever warrantied, because there is always a probability of all 1s in your rolls.

 

That was mentioned several times across the topic so probably I should answer that as well. Since the result is shown as a percentage it is, of course, rounded off. So 0.997 will be shown as 100% and 0.001 as 0%. I can't imagine the reason why someone would like to know if 100% is in fact 99.8% or 99.7% but if one really wants to know, one could change the display format to a more precise one using Format->Number from the top menu of Google Sheets.