Chief Delphi - Newton 09

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)

- Championship Event (http://www.chiefdelphi.com/forums/forumdisplay.php?f=12)

- - Newton 09 (http://www.chiefdelphi.com/forums/showthread.php?t=76615)

Quote:

Originally Posted by artdutra04 (Post 848941)

Now that would be a sight to see - a Norm Abrams FRC robot! :yikes:

That thing would have some major class. :D

Quote:

Originally Posted by TKM.368 (Post 849075)

Thanks Joe! I don't suppose you intend to rerun the simulation with the addition of 687, will you? :ahh:

No promises. It takes about 24 hours to run a simulation (most of that is generating matches). I need to refactor the code to add another team, and I'm not sure if I'll have time to do that and run the simulation before the championship.

687's average OPR is similar to 33 and 768, so they should do similarly.

They won the engineering inspiration award at the Los Angeles Regional, which is how they qualified. Not sure how they added so late.

Great show of teams in Newton this year, I look forward to the competition.

http://www.youtube.com/watch?v=1eov0n4pukE

total coincidence our best score was at 1:21

Quote:

Originally Posted by Joe Ross (Post 849051)

Similar to what I did last year, I ran through 193 simulated qualification match schedules (all 102 matches, using the FIRST match generator). I used each team's average opr for their contribution to a match. I calculated the standings for each of the 193 simulated qualification schedules, and then looked at some statistics from the aggregate results.

When I went back and looked at last year's data, a team finished within 1 standard deviation of their average seed 60% of the time, and within 2 standard deviations 95% of the time. Only 3 teams seeded higher/lower then their min/max seed.

Code:

Team  Avg Seed        Median        Mode        StDev        Min        Max        #1 seed        Top 8

1625        6.3        4        1        7.5        1        48        40        155

121        7.4        5        1        7.7        1        47        46        131

1726        10.4        7        1        9.8        1        47        20        106

1507        12.6        9        3        12.6        1        69        11        96

2970        14.4        11        4        12.7        1        67        10        85

1701        17.6        14        3        13.6        1        78        7        56

234        18.5        14        5        15.3        1        65        7        60

16        21.8        17        7        16.2        1        76        4        47

1155        21.5        19        5        14.9        1        66        3        42

2377        22.4        19        5        15.8        1        76        2        44

102        21.2        17        7        16.1        1        67        3        49

191        21.9        19        2        14.8        1        64        2        43

364        22.6        19        31        15.5        1        67        2        40

1038        22.9        18        7        16.8        1        79        2        41

First, I must say I am impressed at your analysis - neat way to look at the available data and extrapolate a result. Unfortunately, as the stats suggest, the analysis really doesn't mean anything. The standard deviations are too large to lead any credance to the results, as would be expected in an analysis with this many variables. Keep at it, if you can refine this it will be a very cool way to predict team's success!

Quote:

Originally Posted by David Brinza (Post 848811)

Wait, Jim. I didn't see a miter saw, planer, router or jointer listed for 1730's pit. Or do you bring your own??;)

Let's put it this way at the Palmetto Regional Bilfred said that our team had everything but the kitchen sink. So you'll just have to come by and see what tricks we have up our sleeves ;)

Quote:

Originally Posted by Dan 1038 (Post 849337)

It worked better last year because there was a much wider distribution of scores. This year, there's isn't as big of difference between the 3rd or 4th best robot in a division and the 20th. The standard deviations last year were about half of what they are this year.

The other thing this demonstrates is how only 7 seeding rounds isn't adequate.

The final factor is that this year it's much harder to separate a single robot's performance in to any single number. For example, your partner that pins another robot so you can score doesn't score anything itself, but certainly contributes points to the alliance. Last year's game was much more seperable, and in that case OPR correlated with our scouting very well (0.9). This year it's still decent, but not as good (0.6).

For anyone interested I posted some fun signature userbars over here:
http://www.chiefdelphi.com/forums/sh...348#post849348

Support Newton! Or, if you can't pick, there's even one for every division!
FTC bars too, in case your team is competing in both!

[quote=BradMello;849216]Great show of teams in Newton this year, I look forward to the competition.

I agree,with a list of teams kike this 16 33 85 121 126 135 148 177 191 233 234 340 365 469 1625 1726 1732 1918 and im sure I missed some great teams ,Newton is rock solid!!!!!!!!!

Quote:

Originally Posted by Joe Ross (Post 849350)

The final factor is that this year it's much harder to separate a single robot's performance in to any single number. For example, your partner that pins another robot so you can score doesn't score anything itself, but certainly contributes points to the alliance. Last year's game was much more seperable, and in that case OPR correlated with our scouting very well (0.9). This year it's still decent, but not as good (0.6).

Agreed, Last year was perfect for this type of analysis as there was no significant defense contribution, as Simbotics so blatantly showed all of us :eek:. We had the (mis?)fortune of being against them at both Midwest and the championships... This year has so many variables, it is about impossible to analyze individual robots using alliance stats. At Buckeye, I saw a number of defensive bots with no scoring ability score huge - they had human players off the basketball team! That is a key to the game this year which is kind of new, since the HPs can make or break a close match. It is also about impossible to show the effectiveness of HPs without dedicating scouts to watch the robot and another to watch the HP!

All these predictions and rating systems are great, but the only score that matters is the one that shows up on the board after the match. All are good in that they make us think about what factors matter in this game, and they give us something to do while we wait for the real thing!

All of them seem to be in general agreement, with the same teams showing up in approximately the same place on each. I would caution against trying to get too precise with any of this. In fact, I think you are kidding youself if you have more than one or two significant digits in the rating.

We plan to use advance scouting for two things:
1) Qual match planning - who are the scoring threats, who is likely to come after us, and who should we send after their top threat?
2) Draft scouting priority - since we really can't do a good job of scouting every team. We plan to focus on about the top half.

Not to be left out of the fun, we came up with our own system. It is based mostly on the FIRST in MI system (win/loss record, draft selection, and elimination performance, all normalized to account for the different numbers of events) with a scaling factor based on the Simbots "+/- contribution" thrown in for good measure. Did we put the proper empahsis on the correct statistics? Who knows. It must be right because it puts us in the top 10 (just kidding). It agrees with a lot of the other ratings, but there are some good teams that didn't make the cut, so I wouldn't bet the farm on it. Here is what we came up with for the teams above the median:

Team # Rating
1625 49
2970 43
1155 42
121 42
1038 41
1726 41
1657 39
368 39
1507 38
85 35
1918 34
852 33
234 33
612 32
1086 32
3075 31
16 31
2344 31
832 29
148 28
469 28
1868 28
1569 27
292 27
768 27
365 27
1706 26
126 26
2866 25
2377 25
135 25
2004 25
102 24
2996 24
2609 23
2659 23
1714 22
1629 22
360 22
88 22
1511 21
233 21
364 20
1732 20
1701 20

Not sure how statistic driven your analysis is. You seem to have left off a good number of good teams and included a number of teams that were picked teams at regionals, not the pickers.

Quote:

Originally Posted by Joe Ross (Post 849051)

Code:

Team  Avg Seed        Median        Mode        StDev        Min        Max        #1 seed        Top 8

1625        6.3        4        1        7.5        1        48        40        155

121        7.4        5        1        7.7        1        47        46        131

1726        10.4        7        1        9.8        1        47        20        106

1507        12.6        9        3        12.6        1        69        11        96

2970        14.4        11        4        12.7        1        67        10        85

1701        17.6        14        3        13.6        1        78        7        56

234        18.5        14        5        15.3        1        65        7        60

16        21.8        17        7        16.2        1        76        4        47

1155        21.5        19        5        14.9        1        66        3        42

2377        22.4        19        5        15.8        1        76        2        44

102        21.2        17        7        16.1        1        67        3        49

191        21.9        19        2        14.8        1        64        2        43

364        22.6        19        31        15.5        1        67        2        40

1038        22.9        18        7        16.8        1        79        2        41

469        22.2        19        4        15.8        1        70        2        42

148        24.3        20        13        16.3        1        82        2        24

1732        25.4        22        19        16.1        1        68        1        27

135        24.2        22        30        15.2        1        77        6        31

1918        27.6        25        32        16.1        1        75        2        19

368        29.6        27        18        18.0        1        86        3        18

2344        28.7        27        6        17.3        1        86        3        26

1868        29.0        25        17        17.0        1        82        1        22

233        29.9        28        11        18.8        1        81        4        24

292        32.4        30        9        18.6        2        79        0        14

612        31.6        31        44        17.9        2        78        0        22

1714        33.4        31        22        17.9        3        85        0        11

852        32.5        30        8        19.2        1        87        1        17

360        31.2        27        51        19.2        1        78        2        22

85        30.9        28        25        18.3        3        84        0        15

126        30.4        29        22        16.4        1        77        1        14

122        35.9        34        43        18.9        1        84        1        13

1657        33.5        31        21        19.6        1        80        1        20

1629        33.8        32        32        19.2        2        86        0        17

88        37.8        35        19        20.8        2        86        0        15

846        35.1        34        30        18.7        1        83        1        13

365        36.7        34        14        20.6        2        83        0        12

768        37.9        36        35        17.4        2        79        0        7

33        36.9        36        32        19.2        1        81        2        11

1086        40.2        40        43        20.4        2        80        0        12

159        40.5        38        39        19.3        2        82        0        7

340        42.4        43        65        19.6        2        85        0        8

842        43.8        42        52        20.0        2        84        0        6

1350        43.9        45        46        18.9        2        82        0        4

3075        43.8        43        34        19.8        4        86        0        9

862        42.4        44        49        19.0        2        81        0        6

2890        44.1        46        55        19.9        2        85        0        9

2609        44.3        44        44        20.5        5        87        0        4

2866        46.0        47        34        19.1        6        87        0        4

177        46.6        51        53        20.7        6        84        0        3

2875        49.5        50        70        18.9        6        86        0        3

2283        51.7        53        68        18.0        10        83        0        0

2771        50.5        51        51        19.4        3        87        0        2

832        54.9        57        53        19.4        8        87        0        1

195        54.3        57        58        17.9        2        87        0        1

1138        54.3        57        61        17.9        12        87        0        0

578        53.4        56        56        18.0        6        86        0        1

1557        57.6        59        56        16.9        17        85        0        0

1730        53.9        55        38        20.0        6        86        0        2

228        56.3        58        71        18.7        1        87        1        2

1811        58.4        60        62        17.7        7        87        0        1

339        56.1        59        67        18.6        8        87        0        1

1516        57.2        59        55        17.8        10        87        0        0

1706        57.2        60        69        18.3        14        87        0        0

1098        56.8        58        72        19.1        7        87        0        1

358        59.3        61        70        18.5        7        87        0        1

2659        57.3        60        60        19.7        4        87        0        1

1569        60.1        62        73        16.8        12        85        0        0

980        58.5        60        66        17.7        11        87        0        0

2004        60.6        65        79        19.7        10        87        0        0

1511        60.2        64        58        18.4        8        86        0        1

1209        62.3        65        80        17.4        10        86        0        0

1458        61.9        67        56        17.6        12        87        0        0

2836        63.4        68        73        16.7        6        87        0        2

620        63.8        64        82        15.9        8        87        0        1

1506        67.3        72        75        16.2        25        87        0        0

2067        67.9        72        72        15.4        15        87        0        0

2702        67.5        72        81        15.6        11        87        0        0

2783        67.6        71        85        14.6        25        87        0        0

138        71.3        75        85        13.7        21        87        0        0

1547        69.4        73        83        14.0        16        87        0        0

1700        70.3        74        84        13.6        34        87        0        0

1311        70.4        74        74        13.7        25        87        0        0

1023        72.3        76        86        13.3        29        87        0        0

2996        73.2        76        84        12.0        22        87        0        0

2549        73.1        76        87        12.9        14        87        0        0

529        75.2        78        86        11.4        38        87        0        0

86        82.4        85        87        7.4        42        87        0        0

Boy oh boy do I love Stats <3 LOL! Coinicidence.... all 102 matches and 102 is there, doing pretty well... I always knew I liked that number :p lol. Seeing Stats. in general are just fun things so this is amazing :D

Quote:

Originally Posted by Wayne TenBrink (Post 849644)

All these predictions and rating systems are great, but the only score that matters is the one that shows up on the board after the match.

That's true, but you can't use that score to scout individual teams! There are 5 other robots on the field, that all affect the score! Making a scouting system based on that not only defeats the purpose of scouting at a regional, it's inaccurate because qualifiers aren't perfect.

Maybe I'm just critical because my team's not on the upper half of that list, though :D

Great work with all of the techniques so far.

One thing I like about the Simbots system is they look at each indiviual team and event. One important factor on this is to look for trends. Like anything else with time, some get better. Some do worse (because the field got better), some got lucky, some are consistently good.

The averaging systems can be dangerous because they do not pay attention to these trends. Some experienced teams had a great first weekend because there were a lot of dead bots at the regionals they attended that they could pick on.

Consistent top and bottom performers are easy to pick out. If a team only attends 1 event, it is very difficult to determine Lucky vs. Good.

Happy Data Crunching!

Rembmer always temper your data with common sense, and make sure your opinion reflects reality by using a data.

Quote:

Originally Posted by IKE (Post 849874)

One important factor on this is to look for trends. Like anything else with time, some get better.

The averaging systems can be dangerous because they do not pay attention to these trends.

This is a good point, some teams show up for competition with systems that need some work while others show up as ready as they will ever be.

340 showed up with some work to do. We started 0-3 & then went 9-1 before losing 2 in the finals. The record shows 9-6 but I see, streaks of 0-3 then 5-0 in quals, 4-1 then 0-2 in elims. We have also had weeks to work out the bugs & make some changes ;). I love the speculation leading up to Championship & I do know that prior performance plays a part but, things can change.