Considering the “random” uniform distributions of switches and scales in 2018 were not actually uniformly distributed, I’m not quite so confident as you are.
If you flip a fair coin, the result will be fair. In the real world and not textbook problems, there is also a possibility that the coin is not fair, and that is why we need statistical tests on distributions. Sometimes distributions “look” fair when in fact they are not, and sometimes (like in this case) distributions can “look” biased, when in fact we don’t have sufficient evidence to say that they are.
It’s also important to do the correct test, a one-tail binomial test as shown in my first post is misleading because it is only testing the possibility that fewer balls than expected will pass through the lower red exit, when we had no reason prior to week 0 to suspect the lower red exit would have fewer balls going through it. A fair test needs to look at the possibility that any exits could either have either more or less exits than average, which is where the chi-squared does good work.
I had an intuition about this that I think may explain the distribution. I’d argue that scoring the low hub is inherently less random because there is a much greater likelihood of placing the cargo very close to your scoring position. (If you’re on the red alliance, you’re more likely be be scoring on the “easy” side near the red alliance wall. This will create a natural bias on the low goal if one alliance happens to be scoring at a higher rate than the other.
I pulled data from TBA to see if this intuition was correct, and it seems like blue alliances scored many more low goals than red alliance. If we assume that most of these shots landed closer to the blue alliance side it would make sense that the red alliance output would show the lowest output as the cargo would have the largest distance to travel.
I also took the total for quals as I assumed they would be a little more randomized in terms of alliance pairings, but it still seems like the numbers favor blue for number of low goals scored. Though, I’d still be curious about the distribution percentages for just the qual matches.
Data below (combined low goals from auto + teleop):
What these results say to me is:
The fields should be built and distributed to regional hubs (such as District headquarters) around the world by Dec 31 each year, so they can be revealed on kickoff, used as training ground for KVs, and tested by teams throughout the build season.
In 2018, FIRST had an algorithm to create a “random” order for the scales. They advertised it as random, and as such you would think everything should average out over the different possibilities. The fact that there appeared to be a bias towards one outcome raised the very real indication that the algorithm, like all computer algorithms, wasn’t actually random!
In this case, we have a mechanical contraption in the middle of the field. This contraption has a certain symmetry to it. The contraption is also “dumb” - there is no logic or decision making involved. If we could set up a test that dropped a series of identical balls (identical diameter, pressure, wall thickness, weight distribution, surface condition, etc) in the exact same place with the exact same velocity, it would be reasonable to assume those balls would leave via the same exit. Of course, even in that situation there may be variables we aren’t controlling for that would change things (for example, wear on different parts of the hub).
Personally, I do NOT expect the output to be random, and I don’t expect it to be predictable. Shots will in no way be repeatable like in a controlled test. The fact that the distribution is as close as it is is certainly encouraging though!
I think the major concern from a gameplay standpoint is to avoid a bias towards one alliance color or another. If a Hub is built in such a way that 60% of the balls consistently go towards either the red or blue alliance side, that would be problematic.
Looking at the construction of the lower goal’s ball distribution system, I would suggest there is a strong correlation between where a ball enters and where it exits (ie. its not like a roulette wheel where the ball goes round and round and round). So to get a really idea of the “fairness” of it you would also need to record where the balls entered the hub.
After thinking about this some more, I’m pretty sure the chi-square test tests the claim that the hub provides random output of balls in general.
If we want to test the assumption that balls scored have a 25% chance of exiting any of the four exits, we need to test both the upper and lower hub. The chi square test only tests if the hub as a whole provides a reasonable distribution, which is all we can focus on without more information.
Besides that, we cannot perform a chi-square test on only the lower hub (at least in my configuration) as the degrees of freedom falls to 0.
Correct me if I’m wrong, but given my tables, I’m testing the expected distribution (25% of total scored in the upper/lower hub about each exit), against the actual distribution. This is exactly what you’re asking, isn’t it?
I think this is a reasonable argument.
Let’s say a ball scored from the blue is biased away from red and toward blue, and vice versa if scored from red.
Let’s say that bias sways the percentage 10% (Pulled out of thin air for the sake of argument).
This means there is 25% chance a ball will exit far or near, 15% chance to exit the opposite color, and a 35% chance to exit the same color. Applying that to the given totals means we expect a general percent chance of 30.5% toward the blue side, and 19.5% toward the red side.
The chi square value for that is 2.80753, a p value of .422 at 3 degrees of freedom.
I’ve attached the excel sheet I’m using if anyone wants to play around with this and check me on how I’m finding these values. It’s been fun exploring this and I’m enjoying the discourse around these topics as a community. Hub-Distribution.xlsx (10.4 KB)
Some more data to investigate this hypothesis:
Here are the full breakdown of exits in auto and teleop, quals and playoffs. These are split by exit and by alliance:
Cargo Lower Near
Cargo Lower Far
Cargo Lower Blue
Cargo Lower Red
Cargo Upper Near
Cargo Upper Far
Cargo Upper Blue
Cargo Upper Red
blue colored Cargo
34
31
35
18
44
43
40
40
red colored Cargo
25
24
23
20
52
50
49
63
If we group them according to the drivers’ perspectives (for example, cargo lower near for blue is equivalent to cargo lower far for red since both are on the right side relative to the drivers) we get the following:
Cargo Lower Right
Cargo Lower Left
Cargo Lower Own Side
Cargo Lower Opposing Side
Cargo Upper Right
Cargo Upper Left
Cargo Upper Own Side
Cargo Upper Opposing Side
58
56
55
41
94
95
103
89
So it was less common for a ball to go onto the opposing side when scored low, but the chi-squared value is 3.4, which gives a p-value of 0.33, not nearly enough to claim any deviation from a uniform distribution.
I took a different approach and retrieved ball exits per alliance and linked them to the teams playing in the alliance. I got the following for qualification matches:
Key:
Front - Blue port for blue, Red port for red
Back - Red port for blue, Blue port for red
Left - Far port for blue, Near port for red
Right - Near port for blue, Far port for red
Did this since the field is mirrored, and is easier to track
Ah, I think I see where we differ with the degrees of freedom.
I used the homogeneity test’s definition of degrees of freedom (rows - 1) * (columns - 1), so I run into issues when I remove the upper hub as a column.
You probably used the goodness-of-fit’s definition categories - parameters, which allows you to test the lower hub alone. Not sure which is most applicable in this case, however.
Edit: I’m pretty sure your goodness-of-fit test makes the most sense. We’re testing expected values against actual values, not anything regarding homogeneity or independence of exits.
As a sanity check, I compared your numbers for my team (1153) with my notes from scouting the video of our matches. I count that we scored 18 high goal shots in our four qualification matches, but you only have 3 high hub exits tallied for us. Your low goal numbers for our team appear to be correct, as we only shot low briefly in one match.
I am not sure if this is an artifact of your methodology, or perhaps because we didn’t have our team number on our bumpers. But it appears that you may be missing some data.
Lower hub has a chi-squared of 14.3. P-value of 0.0025. Very solid evidence balls are not exiting the lower hub uniformly. Looks to be mostly due to the red exit being favored over the blue exit.
Using the 3 regionals mndu, mndu2, and flwp, we are starting to see a consistent bias on the lower hub for the red exit, and a consistent bias on the upper hub for the near (scoring table side) exit:
All Balls Low Near
All Balls Low Far
All Balls Low Blue
All Balls Low Red
All Balls High Near
All Balls High Far
All Balls High Blue
All Balls High Red
mndu
197
186
149
221
326
272
278
310
mndu2
159
188
166
200
284
251
261
249
flwp
154
126
151
173
230
190
205
186
sum
510
500
466
594
840
713
744
745
Combining these three events, we have a chi-squared on the lower hub of 17.1, p-value = 0.0007, and a chi-squared on the upper hub of 11.9, p-value = 0.0076.
We now have quite a few events completed, so I thought I would update:
All Balls Low Near
All Balls Low Far
All Balls Low Blue
All Balls Low Red
All Balls High Near
All Balls High Far
All Balls High Blue
All Balls High Red
most common low exit
most common high exit
chi squared lower
chi squred upper
p-value lower
p-value upper
flwp
192
156
178
208
338
279
297
272
red
near
8.0
8.9
0.0470
0.0311
mndu
228
206
170
250
438
382
385
436
red
near
16.4
7.0
0.0010
0.0721
mndu2
175
202
183
211
382
369
371
344
red
near
4.3
2.1
0.2309
0.5501
bcvi
154
126
151
173
230
190
205
186
red
near
7.4
5.9
0.0601
0.1180
midet
113
123
164
135
182
183
167
196
blue
red
10.9
2.3
0.0121
0.5090
mifor
233
254
233
317
430
300
353
372
red
near
2.4
3.3
0.5024
0.3436
miket
282
309
292
273
309
298
299
262
far
near
2.5
4.4
0.4806
0.2248
nhgrs
270
239
309
300
358
340
371
351
blue
blue
10.8
1.4
0.0128
0.6996
orore
105
135
125
99
239
293
276
247
far
far
7.3
7.2
0.0617
0.0658
tant
53
68
40
57
233
265
223
205
far
far
7.4
8.2
0.0613
0.0420
txwac
109
99
88
103
168
178
147
157
near
far
2.4
3.3
0.5024
0.3436
sum
1914
1917
1933
2126
3307
3077
3094
3028
red
near
15.7
12.6
0.0013
0.0056
The lower exit at mndu is so far by far the distribution that deviates the most from the uniform distribution. There is only a 0.1% chance that a distribution at least as extreme as this could occur if the exits were all equally likely. The red lower, near upper most likely combo appears in 5/11 events.
I plotted a scatter gram of upper vs lower p-values. No p-values were between 0.12 and 0.21. Using these two natural breaks to define quadrants, 6 events had both below 0.12, 4 had both above 0.21, and 2 had lower < 0.12 and upper > 0.21, (the remaining quadrant was empty) so it appears likely that the magnitude of the two p-values are correlated.
Curiously, mapping the angle between the two most likely exits, 7 are at 90^\circ, 3 at 0^\circ, and 2 at 180^\circ. This tends to point away from the difference being due to a sloping floor or other directional bias linking lower and upper hubs. (The expected distribution if the two are rotationally independent is 6, 3, 3).
It would be massively more difficult to calculate, but I’d be interested in seeing if there is a correlation with where the balls are entering the hub.
At LSR I saw a number of cargo that would barely touch the agitator wheel in the upper hub. Plus I suspect that even cargo that touches the agitator in a “normal” fashion will follow a somewhat predictable path.
And at the same event I saw a couple of cargo that bounced around in the upper hub agitator for over 10 seconds. If you have laser guided cargo it may be possible to make a shot with a predictable exit, but the precision of shooting I’ve seen makes me think that the distribution should be quite random.
The lower hub can be very predictable. If you can place a single cargo so it rolls down the side of the hub, you can have it exit from the adjacent chute over 90% of the time. I know, I have one and have stood over it dropping balls into it. I’ve yet to see a robot that does that though.
One of the interesting things I noted was how successful upper hub shooters would deplete the cargo available to them over a match. Their ability to score went down over time unless they could successfully operate on the far side of the field.
