Chief Delphi - High level scoring analysis 2012-2014

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)

- - High level scoring analysis 2012-2014 (http://www.chiefdelphi.com/forums/showthread.php?t=129265)

High level scoring analysis 2012-2014

After reading much discussion about matches "decided by fouls" in the 2014 game, I wondered how the overall percentage of such games compared with past years. I found one early season report of "about 10%", and a few more detailed analyses. While I'm sure the latter contain the information, I wasn't up to going through the trees to find the forest I was looking for, so I wrote the attached program, with the results below. I defined a "decisive" scoring component (autonomous, teleoperated, endgame, or foul points) as one which would have changed the match outcome (won/lost/tied) had it been zero. Percentages are the percentage of matches for which each scoring component met this criterion. Matches where the individual components failed to add up to the total for either team were ignored.

Qualification matches:

Code:

> java -Dtype=Q CSVStats *csv

type Q



file 2012_Twitter_weeks_1_thru_8.csv

matches 5738 (6 ignored)

foul    385     7%

hybrid  1831    32%

teleop  1721    30%

end     1299    23%



file 2013_Twitter.csv

matches 6617 (5 ignored)

foul    359     5%

hybrid  2249    34%

teleop  2790    42%

end     1904    29%



file 2014_twitter.csv

matches 8300 (11 ignored)

foul    1057    13%

hybrid  3004    36%

teleop  4980    60%

end     0       0%

Eliminations:

Code:

> java -Dtype=E CSVStats *csv

type E



file 2012_Twitter_weeks_1_thru_8.csv

matches 1065 (13 ignored)

foul    63      6%

hybrid  449     42%

teleop  460     43%

end     449     42%



file 2013_Twitter.csv

matches 1236 (10 ignored)

foul    69      6%

hybrid  654     53%

teleop  821     66%

end     466     38%



file 2014_twitter.csv

matches 1533 (10 ignored)

foul    181     12%

hybrid  749     49%

teleop  1243    81%

end     0       0%

Both:

Code:

> java CSVStats *csv

type QE



file 2012_Twitter_weeks_1_thru_8.csv

matches 6803 (19 ignored)

foul    448     7%

hybrid  2280    34%

teleop  2181    32%

end     1748    26%



file 2013_Twitter.csv

matches 7853 (15 ignored)

foul    428     5%

hybrid  2903    37%

teleop  3611    46%

end     2370    30%



file 2014_twitter.csv

matches 9833 (21 ignored)

foul    1238    13%

hybrid  3753    38%

teleop  6223    63%

end     0       0%

Re: High level scoring analysis 2012-2014

So, the good news is that with the combined 2014 stats, 101% of points were scored without fouls, so folks should not be complaining..... Unfortunately for the refs, teams were giving 114% effort on average.

Re: High level scoring analysis 2012-2014

Quote:

Originally Posted by IKE (Post 1381683)

I think that's because this analysis is finding what percentage of matches would have been affected by the elimination of one of the components, so some matches could be counted multiple times if they could have been affected by multiple scoring aspects.

Re: High level scoring analysis 2012-2014

Quote:

Originally Posted by IKE (Post 1381683)

So, the good news is that with the combined 2014 stats, 101% of points were scored without fouls, so folks should not be complaining.

That's not what the percentages mean. Reread the post. It's percent of matches that would have a different result if that component was 0.

Re: High level scoring analysis 2012-2014

It's amazing how balanced these percentages are for the 2012 eliminations.

Re: High level scoring analysis 2012-2014

I'm not sure IKE was really serious, but as others have observed, the numbers don't have to add to 100%, and only would if every match had exactly one "decisive" factor. Many have more than one, e.g. a 40-35 match where the winner had 10/10/10/10, and many don't have any, e.g. a 40-20 match with the same breakdown.

Other definitions of "decisive" are possible, which is why for quantitative analysis it's important to pick one and be clear about what it is.

Another interesting distinction is between a decisive factor and the decisive factor. The same data can be sliced and diced many ways, one of which is to count matches decided by exactly one distinct scoring category. I did a bunch of that and won't bore you with it; for the most part it shows the same general characteristics with different numbers. One might be worth noting though:

Foul points "a deciding factor" (as in the earlier tables)

Code:

        Matches         Percentage

Year        Q+E(Q/E)        Q+E(Q/E)

2012        448(385/63)        7%(7%/6%)

2013        428(359/69)        5%(5%/6%)

2014        1238(1057/181)        13%(13%/12%)

Foul points "the deciding factor"

Code:

        Matches         Percentage

2012        113(112/1)        2%(2%/0%)

2013        67(66/1)        1%(1%/0%)

2014        429(405/24)        4%(5%/2%)

The lower table shows just how much worse AA fared against this metric than past games. This is 2x worse in the aggregate than past games, and maybe more importantly far worse than that in the elimination rounds. I think it likely that each and every one of those 429 matches represents a situation where three teams "walked away mad" - at themselves or an alliance partner for doing something they knew they shouldn't have done and were trying not to do, or at the game officials for making a call they couldn't understand. The numbers don't tell you which it was, but either way it's a Bad Thing we should all want to happen as infrequently as possible.

IMO, this is a measurable factor that goes a long way toward explaining some of the intense dislike of this game. Having matches decided by factors that involve necessarily inconsistent and often inscrutable human judgement calls is what's called in the world of quality analysis a "dissatisfier" - something that causes dissatisfaction (no matter what the product's other good qualities) if got wrong, though no one ever compliments anyone for getting it right.

No, not all foul points are questionable or hard to accept, but in the aggregate they're a good proxy measurement for the rate of occurrence of ones that are. Again IMO, I'll propose that the cold hard percentage rate of "matches decided by foul points" (either one, computed in this or a similar way of your choice) is a useful inverse "figure of merit" for a game. Keeping it low should be an explicit game design goal in the future.

Re: High level scoring analysis 2012-2014

Thank you for this. It is nice to see some objective analyse of the foul problems this year.

What I would be really interested to see is how this breaks down by week.

Re: High level scoring analysis 2012-2014

Quote:

Originally Posted by Pault (Post 1381886)

What I would be really interested to see is how this breaks down by week.

I did a week-by-week "Effect of Foul Points on Match Outcome" analysis for the following weeks in 2014:

Week6:
http://www.chiefdelphi.com/media/papers/2995

Week7:
http://www.chiefdelphi.com/media/papers/3000

Archimedes Curie Galileo Newton:
http://www.chiefdelphi.com/media/papers/3018

If there's interest I can run the script for weeks 1 through 4 individually.