Hey yall, in the past week someone commented about the distribution of cargo getting returned to the field, so once TBA had some real matches in their data I took a look! This is from the “2022Week0” event. There were 210 lower cargo shots, 381 upper cargo shots, making 591 in total. The upper cargo was pretty even, but on the lower the CargoLowerRed was at 18.1%.
What do yall think is an acceptable “randomness” I think one exit being 18% is fine, but what if an event drops to 10% in one exit… 5%? I don’t see any guarantee in the manual for this distribution so I don’t think a field fault is possible… thoughts?
also, if this discussion started elsewhere and I haven’t seen it feel free to delete/close.
I’m less concerned with the near/far than I am even distribution of red/blue. I think 18% is a bit low, but not egregiously so. It’s an interesting distinction though.
I don’t know as I’d expect the return distribution to be completely random, as where the balls enter in probably factor into things too. That is, if a team is always shooting from the same spot in the same way, and more of their shots that go into the hub end up going out through one particular return than the other returns by some reasonable fraction, I don’t know as I’d be that surprised. It’s not completely like shaking a well-balanced 4-sided die.
I would love to track this… it would require someone to manually track each shot… probably not possible. It could totally explain some of the oddities we might see.
My prob stat isnt as good as it should be, is this helpful?
Well, with the data that’s there it might be possible to track if the exit distribution for red cargo are meaningfully different from the distribution for blue cargo. I don’t know enough statistics to determine what kind of difference would be “meaningful”, though.
Doing a simple binomial test on the lower red exit tells us that there is only a 1% chance that there would be 38 or fewer exits out of the 210 total exits assuming all exits have a 25% probability:
There’s definitely a better test to do with this data, I need to brush up on my statistics to know exactly the right one though. But we have moderately strong evidence that something was up with the lower red exit at week 0.
As do I, but the rudimentary test would be to split this data into ordered subsets and compare distributions between the various subsets. (i.e. Look at the first half of matches versus the second half)
1% is right if you are asking “what is the probability of no more than 38 out of 210 came through a particular exit” (e.g. “lower red”).
If you want to know “what is the probability of no more than 38 out of 210 coming through any single exit”, then you can’t use a binomial test and instead need to use a multinomial one. In a few minutes of looking I can’t find a good online implementation that can answer “at least” type queries for multinominal distributions, but as an approximation, we can use 1 - .98908504^4 = 4.3%. (p = . 98908504 because we want to know if “39 or more” balls came out of each exit).
This is an approximation/slight overestimate because it’s assuming the probabilities that each exit has at least 39 exits are disjoint, which they aren’t, but the outcome of <=38 is rare enough that I don’t expect this to affect the exact answer very much.
It’s been a while since I’ve taken stats, so I may be wrong in this, but I think it’s at least on the right track.
I believe in this case a chi-square test is a reasonable test, as there are 4 possibilities among 2 dimensions.
The assumptions are as follows:
Robots have influence over the upper and lower goal scoring. This means we cannot expect a certain number of balls in the lower or upper goal based on some percentage of total balls.
Balls scored must flow through an exit. This means we can expect a percentage of the total balls to flow through each exit.
Each exit has an equal chance of being taken. This means we’d expect each exit to have an equal percentage of balls flow through.
With these assumptions, and the raw data from week 0, we can build a chi-square test.
The expected balls through lower and upper is 25% of the total balls taken through lower and upper.
Building the tables…
Actual
Lower
Upper
Near
59
96
Far
55
93
Blue
58
89
Red
38
103
Sums
210
381
Expected
Lower
Upper
Near
52.5
95.25
Far
52.5
95.25
Blue
52.5
95.25
Red
52.5
95.25
(Actual-Expected)^2/Expected
Lower
Upper
Near
0.804761905
0.005905512
Far
0.119047619
0.053149606
Blue
0.576190476
0.410104987
Red
4.004761905
0.630577428
Chi Square Value: 6.604499
(4-1) * (2-1) = 3 Degrees of Freedom
P Value of 0.085631164
A P-Value of 8.5% is pretty low, and significantly influenced from the lower red goal, with a chi value of 4, over 5x the next highest.
Based on this, it appears there’s a preference to travel through a lower goal that isn’t red, but I don’t think it’s conclusive, it’s not outside random chance.
I agree that a Chi Squared test is the right test, but I’m not sure that upper and lower should be grouped into the same test, as we’re not interested in the distribution of high vs. low goals. The Chi Square value for just the lower goal is 5.5, which is a p-value >0.10, unpublishable in any reputable journal. For the upper goal the value is miniscule, indicating the outcome is likely decently random.
I feel very comfortable saying that there is no result here, as shocking as the 18% number seems with no further analysis. I suspect we’re just seeing the result of a very low number of trials, and this will significantly even out over the season. Especially so as I doubt any preferential outcome seen at a single event would be consistent across all fields at all events, so any errors would likely offset and result in extremely normal data. A more interesting question would be attempting to identify if specific fields or if specific FTA/Field staff that construct those fields result in less than normal distributions, but tracking fields/personnel across all events would likely be a significant effort for an ultimately boring outcome.
I would expect this to vary by event because each field will be assembled by a different team. However, the question is when at an event and we see a potential bias, what is the threshold at which the distribution is likely to be non random and thus should be addressed in some manner? I would say that 210 trials is a pretty good number to base an assessment on.
Whats really weird is that upper red didn’t have the same issue so my initial thoughts about unbalanced floors seem less likely. Though I’m unsure what could cause this
Chi-squared is definitely the correct test. Thanks for jogging my memory @Matt_G It should be used to compare the lower exits and upper exits against 0.25*(total exits) as you describe, not to compare the lower and upper exit distributions against each other.
This is roughly what I was thinking as well before I was re-introduced to the chi-squared test. It’s a dirty approach since it assumes all exits are independent, which they are clearly not. When <=38 balls come out of one exit, that will make it even less likely that <= 38 balls will come out of any other exit since there are more balls that have to come out somewhere. I even think the ~4% you calculated is still low, the value should be roughly twice as high to account for cases where more balls than expected leave an exit (2 sided binomial distribution). 8% is much closer to the chi-squared p-value of 14%. Of course, the chi-squared test is just better, but I think for cases where one exit is very extreme relative to the other exits, 8*(binomial p-value) is a decent approximation for the likelihood that a random distribution could be at least as extreme as the observed distribution. We’re playing fast and loose with probability rules here though, using chi-squared has a much better theoretical basis.
I ran 600 simulations of 210 balls leaving with a 25% chance of leaving each exit. In those simulations only 46/600, or about 8%, had a hub exit which had at least a 14 ball difference from 210/4 = 52.5 (<38.5 or >66.5). This lines up nicely with the ~8% crude estimation above, and also matches the chi-squared p-value of ~14% (since there can be extreme distributions that are still within those bounds (e.g. 40,40,65,65).
I agree, we need a larger sample size to say anything with certainty. Regular events should have much more data than week 0, so doing chi-squared tests even on a single event level could uncover biases. For now though, all we can do is wait and see.
Sounds fun, anyone want to start a tracking sheet of which events use which fields?
So, Week 0 NH was constructed almost completely by FTA’s. That was pretty much their construction staff. And they had almost all never assembled that field before. We aren’t going to see something like that again.
If you flip a fair coin, there is a chance of getting 10 heads in a row. Just because it is very unlikely doesn’t mean it’s not random. That’s sort of the definition of random.
As for multiplle cargo out the same exit. Let’s say that a cargo starts to exit on exit 1. But it doesn’t quite clear and the next cargo is partially occluded in it’s path. It should then be logical for it to have a higher likelyhood of taking the same exit.
