The state of the game

[Beams]

One quick and easy way to help bring marines back into the game is to make their chapter tactics affect their vehicles. Why should eldar tanks and Tau battlesuits get subfaction bonuses when marines don't? Its cheap, its easy, and GW could errata it tomorrow. This wouldn't fix everything of coarse, especially since half of the chapter tactics are pretty lackluster, but it would help.

 

As far as Grey Knights go, they need a lot more help. They need massive overhauls on half the codex, price drops on all of their special weapons, and access to at least 1 more psychic table. The fact that Thousand Sons gets 3 psychic tables and GK only get 1 is a problem.

Treat all their psychic like smite, and give +1 attacks to troops, and they get a lot better. It's weird that your paying 22 pts for 1 attack. That said, great knights suffer from being 13 pts +4 for power weapon, +2 for stormbolter, +1 for deepstrike and +1 for psychic.

[BlackTriton]

Ok, I took some time with the data, cheking fthings out and I prepared my notes.

But "Commodore" over the flg site in the comment of the poscast already posted (in much, much more detail) what I found.

So here is his comment:

 

Okay, sorry I’m responding to this days late but I went and crunched a lot of numbers, and there’s a lot to get through so bear with me. The primary contention of a lot of people reading this is that this is evidence that some factions are *better* than others, but first we need to establish that some factions are actually winning more than others. To do this we do a categorical analysis of dependent proportions, using a Chi-squared test for difference in proportions, and get a test statistic of X=185 for a df=50 Chi-squared function, which gives a p-value so small it shows up as 0.000 in Stata.

 

***technical note: the reason we’re doing a chi-squared approximation even though I should be doing a Fisher’s exact test is that Fisher’s exact tests are extremely hard to compute when the numbers in the cells are large, which they are here, so my piddling 8gb ram core i-7 laptop was about to crap itself trying to compute it. Fortunately, a chi-squared approximation is very, very close in these circumstances so it’s fine.***

 

So, from this we can definitely say that some of the factions are performing better than others, but *why* are they doing better? There are a few possibilities:

1. The codices for the outperforming armies are stronger, so they win more.

2. Better players prefer playing some armies, so those armies win more.

3. Better players prefer armies that are perceived (rightly or wrongly) as being better, so those armies win more.

 

To check this I calculated the win rates of each army when it was a primary faction, as well as the win rates of those players when they chose alternative primary factions. For example, Sisters of Battle won 32.26% of games as a primary faction, but those same players won 51.67% of their games when Sisters of Battle were not their primary faction or not used at all. This helps give us a sense of how strong the players are when they are not affected by the codex in question. I will reproduce the key values below, along with a p-value for a Chi-squared test for difference in dependent proportions. Note that most of these differences have relatively large p-values so even though there might appear to be a large difference we cannot say, generally, that the codices overperform or underperform their players.

 

Win Rates By Faction and Faction’s Players When Playing Alternative Factions–

SoB: 32.26% – SoB Players: 51.67% (p=.145)

SM: 37.84% – SM Players: 44.03% (p=.352)

Custodes: 44.12% – stodes Players: 53.69% (p=.142)

Guard: 55.06% – Guard Players: 55.70% (p=.857)

Eldar: 49.11% – Eldar Players: 60.18% (p=.0088)***

BA: 38.06% – BA Players: 54.70% (p=.0016)**

Demons: 50% – Demons Players: 43.63% (p=.222)

CSM: 44.44% – CSM Players: 56.83% (p=.037)**

AdMech: 50% – AdMech Players: 43.89% (p=.562)

DA: 41.98% – DA Players: 40% (n too small)

DG: 43.61% – DG Players: 53.67% (p=.080)*

DW: 51.61% – DW Players: 33.33% (n too small)

DEldar: 54.08% – DEldar Players: 68.32% (p=.0014)***

GSC: 46.15% – GSC Players: 58.75% (p=.187)

Harly: 51.92% – Harly Players: 65.56% (p=.201)

Knights: 54.24% – Knights Players: 52.64% (p=.624)

Necrons: 40% – cron Players: 20% (n too small)

R.Knight: 54.84% – R.Knight Players:52.38% (p=.79)

SW: 32.26% – SW Players: 55.15% (p=.056)*

Tau: 50.78% – Tau Players: 16.67% (n too small)

TSons: 54.95% – TSons Players: 49.66% (p=.352)

Nids: 40.86% – Nid Players: 52% (p=.258)

Ynnari: 70.75% – Ynnari Players: 51.90% (p=.0128)**

 

*: significant at 10% level

**: significant at 5% level

***: significant at 1% level

n too small means that too few players played different primary factions for me to feel comfortable using an approximate test. Additionally, Orks and Grey Knights players, bless their hearts, couldn’t bring themselves to try other armies so they don’t get numbers.

 

From this we can see a few things:

1. Eldar, Dark Eldar, CSM and Blood Angels players significantly outperformed their codices, meaning when they chose to play another primary they won significantly more often. This indicates that these players may be stronger than their codex’s results would imply

2. Ynnari players statistically significantly underperformed their codex. This indicates that these players may be weaker than their codex’s results would imply.

 

My best guess is that Eldar and Dark Eldar players performed better by selecting Ynnari as their primary detachment instead. Additionally, this suggests that all of the other codices are not performing statistically better or worse than what would be expected based upon the win rates of the players who have selected those codices for their primary detachments. So, to directly address Reece’s argument about Space Marines, it would appear that the Space Marines codex does not attract much play from stronger or more competitively minded players. This may be because the codex is perceived as being weaker, but it does not significantly over or underperform the players.

 

We can now approach the question of whether certain codex players perform better than others. Without getting into every combination, because that would leave this post far too long, the answer is yes. Many of these groups statistically significantly outperform each other. For example, Harlequins players, when not playing Harlequins, win 65.56% of their games, compared to 44.03% for Space Marines players not playing Space Marines; which we can be 98.36% certain are statistically different from one another. This is further evidence that the codices have attracted players of different levels of strength, rather than suggesting that the codices themselves are strong or weak.

 

So lets reevaluate the initial three propositions

1. Are some codices are outperforming each other?

One and only one, the Ynnari codex wins more often than would be expected from the win rates of its players when those players are not playing Ynnari primary.

 

2. Do better players prefer some armies, causing them to win more?

Yes, Eldar, Dark Eldar, Chaos Space Marines and Blood Angels players underperform when using those armies as their primaries compared to their own performances on other primaries. Note, however, that Eldar and Dark Eldar are probably underperforming because their alternative is Ynnari, which we just showed is statistically overperforming. Consequently, the Ynnari-Eldar-Dark Eldar disparities are most likely all derivative of one problem.

 

3. Do better players prefer some armies because they’re perceived to be better, causing those armies to appear to be good?

Plausibly, but also unclear. Certainly stronger players prefer certain codices, and certainly those codices have a reputation for being strong, but generally speaking each codex is performing within the expected range of its quality of player. This is, in many ways, an extension of the previous question, but a much harder one to answer because it contains a cultural component. Overall, it is not clear what attracts good players to any particular codex.

 

Anyway, I know that this post is long, and you should give yourself a pat on the back if you bothered to finish it. If you take nothing else away from this post then remember the following: don’t listen to the podcast and hear that some army has a 47% win rate and think that it must be bad compared to an army with a 55% win rate. Statistically, it’s the player that makes the codex good or bad and you’re not going to change how good of a player you are by changing your codex.

[Ishagu]

Fantastical analysis.

 

There's a major thing to consider. The migration of players away from Marines. They fell from the 1st most played to 6th.

 

The low performance is in part explained by the better players seeking out better armies.

[BLACK BLΠFLY]

In the US most of the top competitive players typically shun SM.

[BlackTriton]

The "shun" of matine is accounted for in these number.

The conclusion here (and my own) is that the codex perform just as well as the player piloting the list.

 

My guess as to why they are shun would be how bland they are and / or that the codex dosent scale up well to the highest level of play.

 

Both these theory point out marine as a solid mid tier codex. The power gap between the top tier codexes and low tier is really not that big.

[BLACK BLΠFLY]

Great analysis !

 

/thumbs-up

[Ishagu]

I agree that the gap isn't as big as what we got in 7th between armies, but the marine book is definitely weak.

 

The top players who want to win will use the best stuff, and they aren't using the space marine codex.

[BLACK BLΠFLY]

Deathwatch is potentially top tier. It is really good.

[Prot]

As nice as the extra stats are I don’t understand if they account for some misleading data.

 

For example if I had a kick butt tournament with my Deathguard ( which I did at an ITC GT) and then I decide I’m throwing caution to the wind and decide I’m playing a goofy Dreadnoughts based Deathwatch list in my next two tournies and do terrible with that experiment, what have I just done to the numbers?

 

I actually do stuff like this all the time where I’ll take an event seriously then decide I want to do something off the wall. I’m sure I’m not the only one.

 

Also Astra is amazing imo, but we see Astra players who play non Astra are better off? That’s a head scratcher because there really aren’t many better factions unless they play Ynarri or something of that magnitude.

 

The author’s premise that the player makes the codex would appear to be disputed by Space Wolves players who appear to be very competent.... unless they’re using Wolves. The same is true of GSC players.

 

There’s also too many subtleties beyond the scope of the numbers that may be skewing this.... for example Death Guard players are marginally a ‘losing’ player with Death Guard primary however with a different Primary they become a ‘winning’ player by 10%. But for all we know this is the effect of a DG player increasing the Nurgle faction slightly and essentially playing the same army? ( this is pretty common). I’ve played guys claiming DG and the “Nurgle” Alpha Legion is truly doing the work. (As another example of this)

 

Also AdMech with that winning percentage baffles me.

 

Anyway, I certainly appreciate the effort, but it feels too grey for me.

[Captain Idaho]

I have seen many a 40K tournament with little LoS blocking terrain and that would skew results somewhat. Adeptus Mechanicus with those horrible kill robots standing there with 1st turn and nowhere to hide won't help things much either ;)

[Thorakitai]

What makes this data a bit misleading is the lack of consideration for allies, as some have noted. For example, pure admech is pretty terrible, doubly so when LoS blocking terrain exists. However, throw in a few allies (I use about 30% IG personally, which is far from the most competitive way to do it), and the army becomes quite good, which skews the numbers. I suspect there is a lot of this going on for the Imperial and Chaos factions.

 

And that is a huge difference for Tau. I have trouble believing that.

[Ipsen]

Yeah the data set is so small it is going to being skewed by all kinds of things. In particular i would have wanted to correct for the match ups. What was winning against what but its not included in the data unfortunately. They mentioned in particular, in the podcast, that capital city bloodbath had a lot of eldar for instance. If a significant number of the games played were eldar vs eldar it would push their numbers far closer to 50% than they would otherwise be.

[BLACK BLΠFLY]

If the tournament used the ITC Missions then there was prolly lots of terrain. Plus they tend to be quite competitive. There’s always going to be some people at the lower tables too so I don’t theres enough skew to have much impact on the numbers.

[BlackTriton]

There is a significant amount of uncertainty witin the player WR VS main faction WR.

 

Its more usefull to compare them to one another and get a feeling of where each faction belong.

Its also good to see what is outstanding, like blood angels performing highly as an ally and poorly as an main faction. This is important, otherwise one could see BA as a high performing faction, when it really is not.

[Honda]

The conclusion here (and my own) is that the codex perform just as well as the player piloting the list.

 

 

"An army of sheep, led by a lion, will always defeat an army of lions, led by a sheep." - Arab proverb

[BLACK BLΠFLY]

So true my brother !

[chapter master 454]

 

The conclusion here (and my own) is that the codex perform just as well as the player piloting the list.

 

 

"An army of sheep, led by a lion, will always defeat an army of lions, led by a sheep." - Arab proverb

 

 

"As an army marches on their stomach, one must remember that the army of sheep led by a lion is never hungry but the sheep leading lion starve because there isn't enough of their leader"

 

Sorry, but I find it just an amusing image (I do like the saying though).

After all, the sheep general only has "BAHHH-d" plans.

 

 

Regardless, yes the concept is simple: bad players still do bad with good armies. It applies everywhere in games unless it is incredibly brain-dead. A good player can make even weak codexes do well with good tactics and application of units. Reminds me of stomping net-deckers in card games with my own brand of decks no-one knew of because they were considered "bad". (also, feels bad when they don't even know their own army rules and abilities).

 

However, even if a good player makes a weaker option in a game work for them, it doesn't mean it is powerful. Just that the player piloting whatever it is knows how to work it. After all, we can all agree that marines are underpowered. Similar to how Eldar are overpowered. However when a skill marine player meets a new eldar player, things tend to go not so well for the eldar player in that case (which is good to see, skill should trump raw army power).

[Captain Idaho]

It's a case of when things are equal. Arguing that the extremes exist therefore the average majority doesn't need attention is inaccurate in game design.

 

If 2 players of roughly equal skill come up against each other but one has Custodes/AM soup and the other a combined arms Marine army, we all know who is going to win.

[Prot]

 

The conclusion here (and my own) is that the codex perform just as well as the player piloting the list.

 

 

"An army of sheep, led by a lion, will always defeat an army of lions, led by a sheep." - Arab proverb

  

 

You know just when I’m pretty sure I should never take tactical or public speaking advice from you, you go and post something like this. You are full of zeal my friend. :tu:

 

 

 

If the tournament used the ITC Missions then there was prolly lots of terrain. Plus they tend to be quite competitive. There’s always going to be some people at the lower tables too so I don’t theres enough skew to have much impact on the numbers.

This is kind of true.... if you’re at LVO or a tightly run ITC event. I played in an ITC event with DG against Ynarri on planet bowling ball. Ouch.

 

The ITC based data is okay, but I really only take the obvious from it : Codex Astartes is horrible. Other Astartes are marginally less horrible , chaos Overall is pretty terrible until you start minimizing the marine content which feeds back to our outstanding theme that Power Armoured units are pretty much the worst thing you can play in competitive 40k.

 

Now let’s be honest here... how many of us needed this big sack of stats to tell us that?

 

As a side note I turned on Warhammer TV to watch Nova coverage. Of course I happen to be watching Ynarri vs Chaos which was called Thousand Sons but actually contained allegiances that go 100% against background. Be that as it may I saw no way you would truly identify this as Thousand Sons aside from the highest cost units.

 

So this is a VERY common depiction of Thousand Sons in ITC. Would I call it Thousand Sons? Absolutely not I would call it a combination of strong models from three Chaos factions being cherry picked for maximum effect which is fine for ITC. But my point here is “technically “ this would be considered Thousand Sons for statistical ITC analysis. When I play Thousand Sons I have approximately a grand total of 2 models in common with this army.

 

Anyway, sorry about that rant... I just actually flat out asked one of the GW guys if they ever think of undoing all the early FAQ changes they did to Astartes back when they first came out.

 

The response was, “ I don’t know what you’re talking about”.

 

So I listed: flyers, assault cannons, Razorbacks, Hurricanes, Guilliman, and deep strike as some of those changes... no response. ( funny how massively this was Guilliman damage control in a game with 2-3 codexes at the time but the result absolutely destroyed Grey Knights).

 

Now to be fair a lot of people are just filling chat with stuff like “ For the Emperor” or “ I like chocolate chips” over and over so it’s really difficult to conduct conversation.

[Stofficus]

But what if I really like chocolate chips? 

In all seriousness though, nothing they're talking about there is some grand surprise, and as stated above, how they classify certain armies, which happen to have a couple good units, being used amidst soup isn't terribly helpful. 

I only ever play friendly games, and I've been tabled turn 2 by an all knight list, walked all over by Eldar until I made my own little fort of Leviathan Siege Dreadnoughts and noise marines, and watch regular codex: space marines armies get annihilated over and over. Even me using the CSM 'dex with only my 30k model collection, I still walk all over them, and it's not like Chaos is that much better when you're still using marine bodies and more or less the same vehicles. 

Guard though is an interesting case; on their own Guard can be fairly powerful, if limited in mobility outside a few outlier builds, so I can see them doing fairly well, but how much of their reporting is Guard CP batteries with three mortar infantry sections and a Company Comd with Kurov's Aquila? 

[BLACK BLΠFLY]

It's a case of when things are equal. Arguing that the extremes exist therefore the average majority doesn't need attention is inaccurate in game design.

If 2 players of roughly equal skill come up against each other but one has Custodes/AM soup and the other a combined arms Marine army, we all know who is going to win.

That is not true. I beat Knights and AM on a regular basis. I don’t see Custodes much at all anymore.

[Closet Skeleton]

My group made trees/forests block LoS after 3 games. We also tend to play with 30-50% terrain and maelstrom missions so we don’t see as much static gun lines as I see here at B&C.

 

Eldar benefit from terrain more than space marines do thanks to their mobility, you can't fix bad armies with terrain rules.

 

 

True in that Cruddance did a 1/2 arsed job but he did create a lot of new units that are quite good now.

 

 

Cruddance didn't create units, the miniatures design team did. He just wrote hit and miss rules for those units. Outside of the few terrible units the Cruddance books had some good internal synergy and were generally balanced with the books that came out around them, it was the power curve of subsequent books that made them useless.

 

7th ed Dark Eldar had a worse book by far.

[Stonehorse]

I've been playing GW games since Battle Master (1988 I think), so have played most of the editions for theor various game systems.

 

What we have to remember is that GW's aim isn't to write good balanced rules. Their aim is to sell miniatures and books that promote their sales.

 

One way of doing this is intentionally write bad rules for units/models, so their goal posts of what is good and bad are in a constant state of flux. Buy X because it is good, then X is now bad, so buy Y, Y is now bad so buy Z.

 

Plus it allows them to do the whole, 'this edition is now fixed' justification for a new edition. What else would we need 8 editions of 40k, and WHFB, and 2 editions of AoS?

 

It is similar to modern day capitalism, sell a product that has inbuilt redundancy, so people have to buy replacements.

 

I really like the models and setting, the rules leave a lot to be desired, the ease of finding an opponent is all that is keeping me. Having said that I am trying to convert people over to either OnePageRules, or introduce massive house rules to the core rules.

[Captain Idaho]

 

It's a case of when things are equal. Arguing that the extremes exist therefore the average majority doesn't need attention is inaccurate in game design.

If 2 players of roughly equal skill come up against each other but one has Custodes/AM soup and the other a combined arms Marine army, we all know who is going to win.

That is not true. I beat Knights and AM on a regular basis. I don’t see Custodes much at all anymore.

Come on man, it's disingenuous to suggest because I used Custodes/AM soup against Marines as an example my point is invalid.

 

1) The point was and still is; if both players are more or less equal, having one army head and shoulders above the other in capacity on the table will win out. The extremes might still be there but if the players are equal, having an army more powerful than the other player is a distinct advantage.

 

2) The example IS sound anyway. Pure Codex Marines, no Guilliman, will struggle against Custodes and AM soup. Nowhere in that post did I say Custodes were owning everyone else still in tournaments.