FRCTop25.com Week 2 *Results Broadcasted LIVE Tonight!* [Archive]

Mike Starke

16-03-2011, 14:06

Hey all! Justin and I have been totaling all the data, and we have the results of the FRC Top 25 teams through week two!! We will be releasing the results LIVE tonight (3/16) at 10:30PM EST via webcast at http://www.livestream.com/mikeandjustinitm Please join us as we will countdown the Top 25 teams.
If you have a question for us, please email us at mikeandjustinitm@gmail.com, as we will be answering questions live on the air! Also, if you are unable to see the show live, we will be archiving it so you can view it at a later time! Thanks for all your hard work voting this week! See you tonight!
- Mike and Justin

dodar

16-03-2011, 14:08

Did you guys with hold both polls till after the Expert unveil or are you releasing the public poll in a little bit?

Mike Starke

16-03-2011, 14:11

Did you guys with hold both polls till after the Expert unveil or are you releasing the public poll in a little bit?
We will be releasing the public poll live on the show, followed by releasing the expert poll after the show. You'll be able to see the results after the show at http://www.FRCTop25.com
Thanks!

Nathan Streeter

16-03-2011, 17:06

I'm really looking forward to the new rankings - should be exciting!

BrendanB

16-03-2011, 17:08

Yay, now I can get educated on all the amazing teams I didn't think of when voting! Bring a pad a pen kids!

mwtidd

16-03-2011, 20:44

These are a few teams from my top 25 with scores according to my new algorithm.

Next revision coming next week to separate robots and minibots in calculating the contribution. Should give a reading which is better. I'm happy with this one though...

the top 5 are truly in a league of their own.

My Estimated Team's Contribution
http://farm6.static.flickr.com/5174/5533595754_f726e04f14.jpg

vs OPR?
Team OPR
1114 62.9768
111 58.752
33 56.7075
148 56.5229
987 55.1992
217 54.4144
2056 52.7503
2512 49.8532
254 49.1796
1676 46.3775
2016 45.5644
1218 45.5229
876 44.9698
573 43.6522
359 43.5552
1126 43.2325
3284 41.255
179 40.8756
40 40.6288

lets see who wins week 2! :)
*note: my apologies to SD and Lake Superior (your events are the only 2 so far that didn't tweet so I don't have your data...)

JB987

16-03-2011, 21:23

These are a few teams from my top 25 with scores according to my new algorithm.

Next revision coming next week to separate robots and minibots in calculating the contribution. Should give a reading which is better. I'm happy with this one though...

the top 5 are truly in a league of their own.

My Estimated Team's Contribution
http://farm6.static.flickr.com/5174/5533595754_f726e04f14.jpg

vs OPR?

lets see who wins week 2! :)
*note: my apologies to SD and Lake Superior (your events are the only 2 so far that didn't tweet so I don't have your data...)

I would love to see your algorithm:confused:

Nathan Streeter

16-03-2011, 21:25

Ian Curtis

16-03-2011, 21:31

Out of curiosity, what is your new algorithm - I'm always intrigued by new algorithms for FRC statistics! :-)

It appears to be OPR-DPR (or SAA as Bongle calls it)

While OPR is relatively okay at predicting robot goodness, DPR is generally pretty bad at predicting a robot's defense capabilities. (especially in a game like this, how often does 1114 actually play defense?)

Nevermind, that appears to have been a happy coincidence for the test cases I tried. I'm curious too, what is it?

DPR is typically a good indicator of schedule difficulty though.

DPR tries to extrapolate how many points an opposing alliance scored when your robot was on the field. Typically, your robot doesn't much factor into that, especially in a mostly defense free game like this. If a team has low DPR, they could be a great defender, but the chances are much better that they just got an easy schedule with opponents who didn't put up many points. Even if you play great defense on a top tier team, they'll still score a lot.

mwtidd

16-03-2011, 21:38

It appears to be OPR-DPR (or SAA as Bongle calls it).

While OPR is relatively okay at predicting robot goodness, DPR is generally pretty bad at predicting a robot's defense capabilities. (especially in a game like this, how often does 1114 actually play defense?)

DPR is typically a good indicator of schedule difficulty though.

Out of curiosity, what is your new algorithm - I'm always intrigued by new algorithms for FRC statistics! :-)

Actually its neither.
Its what I call an ETC - Estimated Team's Contribution
As a qualifying match gets added to my DB I add the alliance's points to a running sum for a given team. Leaving each team with a TAQMP (total alliance qualifying match points), then divide by the # of matches played and you get ATAQMP(average TAQMP)
then based on AT* go through all of the matches again, comparing the 3 teams AT*s on an alliance, and figure out the % of that matches points attributed to a given team ETQMC (estimated teams qualifying match contribution)

add up all the ETQMCs for a given robot and divide by the number of matches and you are left with ETC (Estimate Team's Contribution) on a per match basis.

what OPR lacks is a match by match analysis, it more looks that big picture and misses some of the details. Also next week i will be separating the total score into (autonomous & teleop) and the (minibot). This should reduce the point inflation and get an even more accurate ETC

so next week I'll have an EMC (estimated minibot contribution), ERC (estimated robot contribution), and an improved ETC

Ian Curtis

16-03-2011, 21:45

Actually its neither. As a qualifying match gets added to my DB I add the alliance's points to a running sum for a given team. Leaving each team with a TAQMP (total alliance qualifying match points), then divide by the # of matches played and you get ATAQMP(average TAQMP)
then based on AT* go through all of the matches again, comparing the 3 teams AT*s on an alliance, and figure out the % of that matches points attributed to a given team ETQMC (estimated teams qualifying match contribution)

add up all the ETQMCs for a given robot and divide by the number of matches and you are left with ETC (Estimate Team's Contribution) on a per match basis.

what OPR lacks is a match by match analysis, it more looks that big picture and misses some of the details. Also next week i will be separating the total score into (autonomous & teleop) and the (minibot). This should reduce the point inflation and get an even more accurate ETC

So is the % for a given team/match that team's TAQMP/(sum of alliance's TAQMP?)? It'll definitely be neat to see it without the minibot. And by this algorithm, all teams will have a positive lineskier ranking?

mwtidd

16-03-2011, 22:05

So is the % for a given team/match that team's TAQMP/(sum of alliance's TAQMP?)? It'll definitely be neat to see it without the minibot. And by this algorithm, all teams will have a positive lineskier ranking?

its done by AT*, as different teams play different # of matches you have to take an average.

so lets say for the red alliance

r1c = AT* for red1

r1c/(r1c + r2c + r3c) = r1%c to match n
r1%c * redScore = r1mc (red1's match contribution)

also a teams AT* is calculated on a per event basis. So if you go to a second regional your teams match contribution is based on your performance at that regional

However your ETC is the average ETQMC over all matches at all regionals

There is no way to have a negative as points against are not factored. that being said, Just as I can calculate MC and RC I am about one days worth of coding away from a DC (defensive contribution). My hope was to present a madden-like scale for each team. So before a match you could plug in the 6 teams and see what you are in for (based on the #s rather than based on teams claims)

also I wanted to add an alliance selection app, that estimates a teams contribution to your alliance. For example if you have 2 minibots better than the 3rd then their MC would be dropped from their value. I think this would come in handy in picking out teams your may over look for your 2nd pick. Obviously you would want to check it against your scouting sheets...

Mike Starke

16-03-2011, 22:17

We're going live in about 15 minutes! Come join us!!!!
http://www.livestream.com/mikeandjustinitm

:)

JB987

16-03-2011, 22:20

Did you run the numbers for San Diego?

mwtidd

16-03-2011, 22:30

Did you run the numbers for San Diego?

Unfortunately SD was one of the two regionals that didn't post to the twitter feed. Supporting SD and Lake Superior would force me to parse the first site, and break the algorithms I plan on implementing later. Unfortunately when it came down to priorities, I have selected to abandon non tweeted events for now.

Great job with your video player though, I enjoyed watching the videos.

JB987

16-03-2011, 22:35

Unfortunately SD was one of the two regionals that didn't post to the twitter feed. Supporting SD and Lake Superior would force me to parse the first site, and break the algorithms I plan on implementing later. Unfortunately when it came down to priorities, I have selected to abandon non tweeted events for now.

Great job with your video player though, I enjoyed watching the videos.

Our pleasure! BTW, how are you going to differentiate which team scored which place with minibot...is there a data source for that somewhere without watching each match and hoping the camera catches all the action?

mwtidd

16-03-2011, 22:39

Our pleasure! BTW, how are you going to differentiate which team scored which place with minibot...is there a data source for that somewhere without watching each match and hoping the camera catches all the action?

no but by going through by each team's Average Minibot Score
I can calculate the % of their contribution similar to how i do the total contribution
So the minibot will still be an estimate, but after 10 matches it should be fairly reasonable.

The Lucas

16-03-2011, 23:49

r1c/(r1c + r2c + r3c) = r1%c to match n
r1%c * redScore = r1mc (red1's match contribution)

So do you add all those match contributions up and average it again to get your final number? If so, have you thought about another iteration to possibly get better numbers?

Interesting, there are always room for more metrics, particularly if you isolate mini bots and penalties. Also it would be interesting if you totaled up contributions across all regionals. I am not sure OPR would be to solve a sparse matrix like that well.

Just ideas, cool algorithm.

BrendanB

16-03-2011, 23:58

Thanks mike and justin for the edumakation! ;) ;)

mwtidd

17-03-2011, 00:00

So do you add all those match contributions up and average it again to get your final number? If so, have you thought about another iteration to possibly get better numbers?

Interesting, there are always room for more metrics, particularly if you isolate mini bots and penalties. Also it would be interesting if you totaled up contributions across all regionals. I am not sure OPR would be to solve a sparse matrix like that well.

Just ideas, cool algorithm.

Yup its an average of all the contributions. Actually a second iteration is an interesting idea, didn't think of that. But yeah it should definitely get a finer resolution than how I am doing it now. Its just trying to keep the server processing to a minimum (I don't want to kill it) :). But I think that a second iteration would definitely be worthwhile.

The ETC actually is totalled over all regionals. So 40s ETC is based on BAE and WPI.

Minibots will be isolated next. Penalties along with defense probably in a week or so.

I'm definitely happy with the results for this week. I still think 217 is being overlooked for PR reasons, one of the few robots you can't find much on. However every set of numbers I run they are at the top of the list.

I think OPR takes into account elims, which reflect more on the alliance rather than the teams, thats why i exclude elims from my calculations. If you look at one of the week 1 feeds for top25 I go through the motions of finding this out.

THANKS AGAIN MIKE AND JUSTIN!!!!
In spite of not not agreeing with me on stealing ubertubes I still love the show. :)

Justin Montois

17-03-2011, 00:05

Thanks everyone who watched this week. It's great to see the enthusiasm for what we're doing and it really makes all the work that goes into doing the show and the website worth it.

Congratulations to 1114, Simbotics for moving up to number one. You guys really deserve it.

If you have any questions or comments please feel free to email us at mikeandjustinitm@gmail.com or PM Mike or I. We monitor it all week and are always looking for topics to talk about on the show, ways to improve and any insight into the teams.

As always, thanks for your participation, we couldn't do it without the FIRST community. Good luck to all teams competing week 3, we'll see you next week!

PS: We apologize for the loss of signal towards the end of the show.

PPS: We failed AGAIN to record it. Third times the charm.....

Thanks everyone.

Basel A

17-03-2011, 00:08

The expert polls were made public last week.. Any chance of the same this week? Or am I just missing it completely?

mwtidd

17-03-2011, 00:10

The expert polls were made public last week.. Any chance of the same this week? Or am I just missing it completely?

They said they will be posting them later...

The Lucas

17-03-2011, 00:17

I think OPR takes into account elims, which reflect more on the alliance rather than the teams, thats why i exclude elims from my calculations. If you look at one of the week 1 feeds for top25 I go through the motions of finding this out.

OPR doesn't use elims for the same reason, It was considered a couple years ago and it just skews without adding value. This is why I like to call the bot with the highest OPR "Qualification Match MVP"

mwtidd

17-03-2011, 00:25

OPR doesn't use elims for the same reason, It was considered a couple years ago and it just skews without adding value. This is why I like to call the bot with the highest OPR "Qualification Match MVP"

Yeah I've thought about dropping a teams first 2 matches, but the logistics of doing that are a little difficult. I still would like to figure out a way, because many teams don't get the kinks worked out til their 3rd Q match or so.

Both ETC and OPR favor teams that worked from the very 1st Q match, which may not be the best team by the end of the regional

also now that I go match by match, I may be able to include elims into my calculations. Although they seem pretty sound right now.

The Lucas

17-03-2011, 00:30

Both ETC and OPR favor teams that worked from the very 1st Q match, which may not be the best team by the end of the regional

These stats are not a replacement for actual scouting, just another factor to consider.

XaulZan11

17-03-2011, 00:34

These stats are not a replacement for actual scouting, just another factor to consider.

Very true. Just by looking at teams from Wisconsin (which my team scouted and I saw nearly every match), it was pretty clear that 2826 was better than 234. 234 was very good, put 2826 was significantly due to their minibot. There is no subsitute for watching actual matches.

mwtidd

17-03-2011, 02:05

So is the % for a given team/match that team's TAQMP/(sum of alliance's TAQMP?)? It'll definitely be neat to see it without the minibot. And by this algorithm, all teams will have a positive lineskier ranking?

Actually after implementing the minibot calculations and what not, I found that all teams will not have a positive robot ranking. (robot is the # of points not scored by the minibot)

a robot can have a negative contribution if a minibot goes up but you get penalties. ie you get 30pts for the minibot, but theres a penalty. your final score is 27 with 30 minibot pts. leaving a -3 contribution to the robots in the match.

JamesBrown

17-03-2011, 08:53

Interesting how much 1519 dropped on an off week, a bunch of teams jumped them despite not playing this week. I am interested to see the individual expert poles and to see why 1519 dropped so much compared to other robots who were also off this week. All of their competition from Manchester improved in rankings compared to teams who also played in week 1 (40 played both weeks and moved up a spot, 175 only played week one, they dropped 8 spots but 10 new teams were placed above them so they actually improved compared to the same teams as last week, same for 131.)

Chris is me

17-03-2011, 09:22

PerpetualMotion

17-03-2011, 13:27

It's interesting to see what great teams are more easily "forgettable" (1519, 2016) and which are not (148).

I didn't vote this week as I was competing, but if I did, 40 would be in the top 3. They drastically improved from BAE.

177 is also quite underrated... but I always say that.

2016 only moved down two spots, and 4 teams were added above them. I don't think they're forgetable. Actually, they jumped three teams from the previous vote (16, 1676, and 1519). None of these teams compete last weekend. I don't understand how there's so much movement with little to no new information.

I agree 1519 seems to be forgettable as they dropped 12 spots. It's also interesting to note 118 remains in the top 25, while 5 teams that were previously ahead them are no longer on the list. None of those 6 teams played last weekend so there is also little to no new information there either.

JamesBrown

17-03-2011, 13:35

2016 only moved down two spots, and 4 teams were added above them. I don't think they're forgetable. Actually, they jumped three teams from the previous vote (16, 1676, and 1519). None of these teams compete last weekend. I don't understand how there's so much movement with little to no new information.

I agree 1519 seems to be forgettable as they dropped 12 spots. It's also interesting to note 118 remains in the top 25, while 5 teams that were previously ahead them are no longer on the list. None of those 6 teams played last weekend so there is also little to no new information there either.

Possible Chris actually meant 2137 who went from 13th to not even making the list of teams receiving votes.

Chris is me

17-03-2011, 14:51

Possible Chris actually meant 2137 who went from 13th to not even making the list of teams receiving votes.

That's the one. 2137 isn't exactly a household name in FIRST, yet their performance is being very quickly forgotten.

jwfoss

17-03-2011, 15:56

you're not suggesting that teams that are more well known will get more votes, are you?

IKE

17-03-2011, 16:56

At the end of week 2, there have been 15 events. Thus under 2 teams per event could be mentioned in a top 25. By the end of the season, there will be over 50 events. Many really really good teams will not make it into the top 25. The top 25 is just over the top 1% of all of FRC that means that statistically speaking, that top team at a regional of 50 teams stands about a 50% chance of getting a nod.

If the guys would be interested, forming a Top 100 list could be really cool. Top 25 ranked, then a 25-50, and then a 50-100 group would be really cool. this would be very interesting to later correlate to the elimination alliances at the World championship (24*4 teams).

sgreco

17-03-2011, 18:13

That's the one. 2137 isn't exactly a household name in FIRST, yet their performance is being very quickly forgotten.

This often happens in college football. When a team has a bye-week, they get passed by teams that won that week. Probably not much you can do about it, but in college football it's also easier to get a high rank if you are well known (LSU, Florida, Ohio State, or in our case 148, 217, 254 1114 etc). I guess over time teams just earn the right to be more easily recognized.

Paul Copioli

17-03-2011, 22:07

For the record, we dropped spots in both polls and didn't even play last week. So much for the theory regarding well known teams not dropping. I have to say that I love the polls (no, I do not vote in the expert poll), but they are as useful as the computers in the BCS.

Vikesrock

17-03-2011, 22:12

I don't understand how there's so much movement with little to no new information.

I know I changed a number of my relative rankings of Week 1 teams after watching additional video and examining some statistics such as OPR, selection position and others.

JVN

18-03-2011, 00:23

These rankings don't describe how good a team is, they describe how good the community thinks they are.

When you realize this, everything makes sense.

Mike Starke

18-03-2011, 09:06

I guess over time teams just earn the right to be more easily recognized.

These rankings don't describe how good a team is, they describe how good the community thinks they are.

Both excellent points. :)

EricH

18-03-2011, 11:03

Something I thought was nice last week and don't see this week is the chance to see the experts' individual polls. Is there a reason that that feature was removed?

sgreco

18-03-2011, 11:40

The polls are harder in FIRST because teams don't play nearly every week like they do in college football. I'm curious to see what will happen to teams that played in early regionals and don't play the final few weeks. I hope they don't get over looked.

The reason most teams that played week one went down is because the size of the field has nearly doubled. A team that had a high rank week one now has twice as many teams to compete against in week 2 for the same spot.

Mike Starke

18-03-2011, 12:08

Something I thought was nice last week and don't see this week is the chance to see the experts' individual polls. Is there a reason that that feature was removed?

Thanks for this post! We're working on it currently. We're at Chesapeake competing this weekend, so you'll be able to check those out probably on Sunday or early next week. Just didn't have enough time before we had to leave on thursday. Sorry for the inconvenience, but they'll be up soon!
:)

ks_mumupsi

18-03-2011, 12:31

Hmmm... I wonder why such a big issue... 469 doesnt even play till today, and remember I think everyone would have ranked them #1 last year as the team to beat... theres several top tier teams that have yet to play a regional.. I know we are looking forward to seeing 1086 and a few others in DC that we havent seen anything from yet.

And also remember teams will improve, im sure 233 and 217 and others will be making adjustments and gaining more practice to get better... things will be in a constant state of flux!!

Keep in mind this is a fun exercise not meant to be a guideline that this is how things should play out... enjoy it, for me as being on 2016 its an honor just to be a part of that list with some of the elites.

mwtidd

18-03-2011, 15:32