High level scoring analysis 2012-2014

[buchanan]

After reading much discussion about matches "decided by fouls" in the 2014 game, I wondered how the overall percentage of such games compared with past years. I found one early season report of "about 10%", and a few more detailed analyses. While I'm sure the latter contain the information, I wasn't up to going through the trees to find the forest I was looking for, so I wrote the attached program, with the results below. I defined a "decisive" scoring component (autonomous, teleoperated, endgame, or foul points) as one which would have changed the match outcome (won/lost/tied) had it been zero. Percentages are the percentage of matches for which each scoring component met this criterion. Matches where the individual components failed to add up to the total for either team were ignored.

Qualification matches:
> java -Dtype=Q CSVStats *csv
type Q

file 2012_Twitter_weeks_1_thru_8.csv
matches 5738 (6 ignored)
foul 385 7%
hybrid 1831 32%
teleop 1721 30%
end 1299 23%

file 2013_Twitter.csv
matches 6617 (5 ignored)
foul 359 5%
hybrid 2249 34%
teleop 2790 42%
end 1904 29%

file 2014_twitter.csv
matches 8300 (11 ignored)
foul 1057 13%
hybrid 3004 36%
teleop 4980 60%
end 0 0%


Eliminations:
> java -Dtype=E CSVStats *csv
type E

file 2012_Twitter_weeks_1_thru_8.csv
matches 1065 (13 ignored)
foul 63 6%
hybrid 449 42%
teleop 460 43%
end 449 42%

file 2013_Twitter.csv
matches 1236 (10 ignored)
foul 69 6%
hybrid 654 53%
teleop 821 66%
end 466 38%

file 2014_twitter.csv
matches 1533 (10 ignored)
foul 181 12%
hybrid 749 49%
teleop 1243 81%
end 0 0%


Both:
> java CSVStats *csv
type QE

file 2012_Twitter_weeks_1_thru_8.csv
matches 6803 (19 ignored)
foul 448 7%
hybrid 2280 34%
teleop 2181 32%
end 1748 26%

file 2013_Twitter.csv
matches 7853 (15 ignored)
foul 428 5%
hybrid 2903 37%
teleop 3611 46%
end 2370 30%

file 2014_twitter.csv
matches 9833 (21 ignored)
foul 1238 13%
hybrid 3753 38%
teleop 6223 63%
end 0 0%

[IKE]

So, the good news is that with the combined 2014 stats, 101% of points were scored without fouls, so folks should not be complaining..... Unfortunately for the refs, teams were giving 114% effort on average.

[echin]

So, the good news is that with the combined 2014 stats, 101% of points were scored without fouls, so folks should not be complaining..... Unfortunately for the refs, teams were giving 114% effort on average.

I think that's because this analysis is finding what percentage of matches would have been affected by the elimination of one of the components, so some matches could be counted multiple times if they could have been affected by multiple scoring aspects.

[AGPapa]

So, the good news is that with the combined 2014 stats, 101% of points were scored without fouls, so folks should not be complaining.

That's not what the percentages mean. Reread the post. It's percent of matches that would have a different result if that component was 0.

[Caleb Sykes]

It's amazing how balanced these percentages are for the 2012 eliminations.

[buchanan]

I'm not sure IKE was really serious, but as others have observed, the numbers don't have to add to 100%, and only would if every match had exactly one "decisive" factor. Many have more than one, e.g. a 40-35 match where the winner had 10/10/10/10, and many don't have any, e.g. a 40-20 match with the same breakdown.

Other definitions of "decisive" are possible, which is why for quantitative analysis it's important to pick one and be clear about what it is.

Another interesting distinction is between a decisive factor and the decisive factor. The same data can be sliced and diced many ways, one of which is to count matches decided by exactly one distinct scoring category. I did a bunch of that and won't bore you with it; for the most part it shows the same general characteristics with different numbers. One might be worth noting though:

Foul points "a deciding factor" (as in the earlier tables)
Matches Percentage
Year Q+E(Q/E) Q+E(Q/E)
2012 448(385/63) 7%(7%/6%)
2013 428(359/69) 5%(5%/6%)
2014 1238(1057/181) 13%(13%/12%)

Foul points "the deciding factor"
Matches Percentage
2012 113(112/1) 2%(2%/0%)
2013 67(66/1) 1%(1%/0%)
2014 429(405/24) 4%(5%/2%)


The lower table shows just how much worse AA fared against this metric than past games. This is 2x worse in the aggregate than past games, and maybe more importantly far worse than that in the elimination rounds. I think it likely that each and every one of those 429 matches represents a situation where three teams "walked away mad" - at themselves or an alliance partner for doing something they knew they shouldn't have done and were trying not to do, or at the game officials for making a call they couldn't understand. The numbers don't tell you which it was, but either way it's a Bad Thing we should all want to happen as infrequently as possible.

IMO, this is a measurable factor that goes a long way toward explaining some of the intense dislike of this game. Having matches decided by factors that involve necessarily inconsistent and often inscrutable human judgement calls is what's called in the world of quality analysis a "dissatisfier" - something that causes dissatisfaction (no matter what the product's other good qualities) if got wrong, though no one ever compliments anyone for getting it right.

No, not all foul points are questionable or hard to accept, but in the aggregate they're a good proxy measurement for the rate of occurrence of ones that are. Again IMO, I'll propose that the cold hard percentage rate of "matches decided by foul points" (either one, computed in this or a similar way of your choice) is a useful inverse "figure of merit" for a game. Keeping it low should be an explicit game design goal in the future.

[Pault]

Thank you for this. It is nice to see some objective analyse of the foul problems this year.

What I would be really interested to see is how this breaks down by week.

[Ether]

What I would be really interested to see is how this breaks down by week.

I did a week-by-week "Effect of Foul Points on Match Outcome" analysis for the following weeks in 2014:

Week6:
http://www.chiefdelphi.com/media/papers/2995

Week7:
http://www.chiefdelphi.com/media/papers/3000

Archimedes Curie Galileo Newton:
http://www.chiefdelphi.com/media/papers/3018


If there's interest I can run the script for weeks 1 through 4 individually.