Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   General Forum (http://www.chiefdelphi.com/forums/forumdisplay.php?f=16)
-   -   Top 25 ETCs after Week 3 (http://www.chiefdelphi.com/forums/showthread.php?t=93809)

mwtidd 20-03-2011 15:54

Top 25 ETCs after Week 3
 
Here are the top 25 ETC | ERC | EMCs after week 3...

I'm curious to hear various opinions and insights :).

Top Minibots:
1676, 395, 2056, and 1507 (EMC 19)

Top Robot:
148 (ERC 35)

The bold number is ETC, then ERC and finally EMC.


Adam Freeman 20-03-2011 15:59

Re: Top 25 ETCs after Week 3
 
I assume this factors in the entire season...

Anyway you can pull out a team results by regional?

wynniethepooh 20-03-2011 16:08

Re: Top 25 ETCs after Week 3
 
I would also be curious to see this on a week by week basis. we played terribly at our first regional due to no deployment+limited driver practice, but at Peachtree our OPR was 57.8. It would be nice to see how we and everone else has improved.

mwtidd 20-03-2011 16:13

Re: Top 25 ETCs after Week 3
 
great points guys, I will work on changing it for an event by event basis, obvious need based on 67's second performance

Quote:

Originally Posted by Adam Freeman (Post 1042556)
I assume this factors in the entire season...

Anyway you can pull out a team results by regional?

67 at west michigan:
ETC: 45.5
ERC: 32.5
EMC: 12.8

apparently you guys were a monster.


Quote:

Originally Posted by wynniethepooh (Post 1042566)
I would also be curious to see this on a week by week basis. we played terribly at our first regional due to no deployment+limited driver practice, but at Peachtree our OPR was 57.8. It would be nice to see how we and everone else has improved.

unfortunately because of the tweet issue I dont have number for peachtree... sorry. Its as much a disappointment to me as it is for you.

I'll be working on changing the algorithm to only take the latest event ETC rather than overall ETC.

speedbuggy76 20-03-2011 16:21

Re: Top 25 ETCs after Week 3
 
Can you provide the algorithm to calculate it for a given team at a given regional? I realize that it would be too difficult to do for everyone at Peachtree since the data isn't on Twitter, but we'd love to calculate it for ourselves to see where we stand.

Thanks!

mwtidd 20-03-2011 16:28

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by speedbuggy76 (Post 1042572)
Can you provide the algorithm to calculate it for a given team at a given regional? I realize that it would be too difficult to do for everyone at Peachtree since the data isn't on Twitter, but we'd love to calculate it for ourselves to see where we stand.

Thanks!

Well to get your ETC you have to go through almost every team...

but in case your feeling ambitious:

As each match gets added to a DB, add the alliance points to a running sum for each team.

Team's total average = teams total / num matches

Then once all the matches have been added
loop back through all the matches again, but this time
calculate your portion of the contribution:

so say for red1:

red alliances average = ( red 1's avg + red2's avg + red 3s avg ) / 3
red 1's contribution = redScore * (red 1's avg / red alliances avg)

then add up all of your contributions and divide by the number of matches.

note I am working on a script to get the matches from the first site for events not tweeted...
however for those events you will only see etc
erc and emc can only be calculated with the twitter feed.

mwtidd 20-03-2011 18:04

Re: Top 25 ETCs after Week 3
 
Top ETCs using teams latest events:


wynniethepooh 20-03-2011 18:23

Re: Top 25 ETCs after Week 3
 
wow didnt expect to see us all the way up to 7th. that feels pretty good

mwtidd 20-03-2011 18:29

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by wynniethepooh (Post 1042674)
wow didnt expect to see us all the way up to 7th. that feels pretty good

just for the record, where these are estimates I would place you as a top 10 team... not necessarily #7 as there is only +-1 ETC point separates teams 4-10

If anything its more of an achievement because you are grouped with some truly amazing teams.


Tier 1: top 3
Tier 2: top 10
Tier 3: top 25

speedbuggy76 20-03-2011 18:55

Re: Top 25 ETCs after Week 3
 
Alright, I've spent some time trying to re-create your calculations using the data from Peachtree. However, I keep getting a different number for our team (2415) than you do.

Some sample data:
In match 5, blue alliance scored 39 points and was composed of 2415, 1683, and 3694
2415's average alliance score: 77.1
1683's average alliance score: 26.2
3694's average alliance score: 7.1

The average score of the teams on the alliance = (77.1+26.2+7.1)/3 = 36.8

2415's contribution = 39*77.1/36.8 = 81.7

Is this correct? I ran this calculation for all of our matches and found us to have an average contribution of 119.2. This disagrees with your value of 38. Any idea what I'm doing wrong/differently?

Also, how did you break up the Peachtree data for our robot into ERC and EMC? I'm just using the data set on the FIRST website which only contains total alliance scores.

Thanks for putting all this together and answering questions! You have my interest piqued.

mwtidd 20-03-2011 19:25

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by speedbuggy76 (Post 1042702)
Alright, I've spent some time trying to re-create your calculations using the data from Peachtree. However, I keep getting a different number for our team (2415) than you do.

Some sample data:
In match 5, blue alliance scored 39 points and was composed of 2415, 1683, and 3694
2415's average alliance score: 77.1
1683's average alliance score: 26.2
3694's average alliance score: 7.1

The average score of the teams on the alliance = (77.1+26.2+7.1)/3 = 36.8

2415's contribution = 39*77.1/36.8 = 81.7

Is this correct? I ran this calculation for all of our matches and found us to have an average contribution of 119.2. This disagrees with your value of 38. Any idea what I'm doing wrong/differently?

Also, how did you break up the Peachtree data for our robot into ERC and EMC? I'm just using the data set on the FIRST website which only contains total alliance scores.

Thanks for putting all this together and answering questions! You have my interest piqued.

Sorry, that's my mistake in the representation of the algorithm. You should use the total score for your alliance not the average...

so
(77.1+26.2+7.1)

instead of
(77.1+26.2+7.1)/3

*trying to run through it off the top of my head doesn't always work out :)

I have also thought about generating highs and lows, by removing a teams highest and lowest matches, to give an ETC window rather than a set value.

every match is tweeted to the FRCFMS twitter feed and it contains one extra piece of information the red and blue bonus, which is the points put up in the race. by knowing this I can use the same algorithm i use for etc, but use it on the bonus score and then the alliance score - the bonus score (robot score)

techvikesmom 20-03-2011 19:37

Re: Top 25 ETCs after Week 3
 
Just a note: Team 2054 has been renamed "TECHVIKES", not big blue crew. This is the second year under this name.

mwtidd 20-03-2011 19:41

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by techvikesmom (Post 1042743)
Just a note: Team 2054 has been renamed "TECHVIKES", not big blue crew. This is the second year under this name.

thanks! your name's been changed in my database... a lot of the names are a bit dated. at least we got one fixed :).

KrazyCarl92 20-03-2011 20:00

Re: Top 25 ETCs after Week 3
 
So where do the cheesy poofs or high rollers factor into all of this?

mwtidd 20-03-2011 20:09

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by KrazyCarl92 (Post 1042761)
So where do the cheesy poofs or high rollers factor into all of this?

see my new signature :) My system work off the twitter feed, if your regional is 1 of the 4 that didn't post, I don't have numbers for you. I will try to support untweeted regionals but I won't be able to supply ERCs and EMCs

techvikesmom 20-03-2011 20:09

Re: Top 25 ETCs after Week 3
 
thanks for the update!!!!

Dave Scheck 20-03-2011 23:08

Re: Top 25 ETCs after Week 3
 
Maybe it's just me, but I don't get it. You have us with an EMC of 15. We put our minibot up in first place in 15 of our 16 matches, and got second in the 16th to boost our RPs. Our real minibot contribution based on that data is 29.3751 ( (30 * 15 + 20) / 16). I realize that you're just making an estimate, but when the data is so far from the actual value I don't see how it can be useful.

Paul Copioli 20-03-2011 23:10

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Dave Scheck (Post 1042935)
Maybe it's just me, but I don't get it. You have us with an EMC of 15. We put our minibot up in first place in 15 of our 16 matches, and got second in the 16th to boost our RPs. Our real minibot contribution based on that data is 29.3751 ( (30 * 15 + 20) / 16). I realize that you're just making an estimate, but when the data is so far from the actual value I don't see how it can be useful.

I have the exact same questions. We have similar results from Detroit for our minibot.

Jim Zondag 20-03-2011 23:53

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Dave Scheck (Post 1042935)
Maybe it's just me, but I don't get it.

Me either, and I'm not convinced yet that I need any new acronyms or methods. Everyone keeps obsessing about who is the best, but this is irrelevant until Einstien. At a real tournament, you don't really need a spreadsheet to tell you who is the best. This is obvious; my mom can tell you who is the best with no data at all. A thousand varied analytical methods will be able to give you the top 8 teams in order.
You do, however, need good data and methods to tell you who is 16th-24th. This is where the real value add of competitve analysis is: not for the first pick, but for the second.

mwtidd 21-03-2011 00:02

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Dave Scheck (Post 1042935)
Maybe it's just me, but I don't get it. You have us with an EMC of 15. We put our minibot up in first place in 15 of our 16 matches, and got second in the 16th to boost our RPs. Our real minibot contribution based on that data is 29.3751 ( (30 * 15 + 20) / 16). I realize that you're just making an estimate, but when the data is so far from the actual value I don't see how it can be useful.

Quote:

Originally Posted by Paul Copioli (Post 1042936)
I have the exact same questions. We have similar results from Detroit for our minibot.

Hey guys, thanks for your posts. I'm glad to know how wrong it was. The reason your EMC is low is because of point inflation. Team's that had 2 minibots go up receive a greater emc than those who only have 1 go up.

Also I found a bug in my database, so it changed the values for some of the team's a little bit (111 being one of them). Please note the top minibot score in the nation is a 19, Dave 111 has a 17 so if you look at it in that context, it means you have one of the best minibots in FIRST.

Paul, 217 you also have a minibot with a rank of 17.

for example if you look at the new list below teams like 148, 40, and 1126 although they are top 25 teams, are lacking with regards to minibots.

Also one other thing, these are all calculated based on qualifying match scores only.

Right now minibot contributions are split among all 3 alliance members, I think I'm going to actually try to figure out the minibot finish order by the scores... shouldn't be too hard


mwtidd 21-03-2011 01:13

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Jim Zondag (Post 1042958)
Me either, and I'm not convinced yet that I need any new acronyms or methods. Everyone keeps obsessing about who is the best, but this is irrelevant until Einstien. At a real tournament, you don't really need a spreadsheet to tell you who is the best. This is obvious; my mom can tell you who is the best with no data at all. A thousand varied analytical methods will be able to give you the top 8 teams in order.
You do, however, need good data and methods to tell you who is 16th-24th. This is where the real value add of competitve analysis is: not for the first pick, but for the second.

Well hindsight is 20/20 but lets look back at the selection for western Michigan where using EMC could have resulted in a very different story, and it all starts with the very first pick.

2054 picks team 67.
2054: ERC- 25 EMC-17 (top minibot and robot)
67: ERC- 20 EMC-8

it looks like an obvious and smart pick but...
in my opinion they overlook one key player which would have changed everything for them.

2767: ERC- 3 EMC-16

Had 2054 picked 2767 they would have had a monopoly over the best minibots. essentially a guaranteed 50 pts each match.

1918 at R: 16 and M:9 would have picked team 67
but finishing 3rd and 4th in the minibot race would have a very tough time beating 2054.

so i think sometimes we even overlook a potential first pick, because we want to form a powerhouse...

XaulZan11 21-03-2011 02:07

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by lineskier (Post 1042982)
Well hindsight is 20/20 but lets look back at the selection for western Michigan where using EMC could have resulted in a very different story....so i think sometimes we even overlook a potential first pick, because we want to form a powerhouse...

Or maybe teams actually watched matches instead of relying on final scores and fancy formulas to rank teams...

I know I may be sounding like a jerk, but as someone who works on the picking list for my team, I would be pretty upset if I worked hard and scouted every match only to be second guessed by someone who likely didn't see any matches. OPR, ETC and various metrics are nice, but there is NO SUBSTITUTE for actually watching matches and seeing how good teams are.

Joe Ross 21-03-2011 02:28

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by lineskier (Post 1042982)
Well hindsight is 20/20 but lets look back at the selection for western Michigan where using EMC could have resulted in a very different story, and it all starts with the very first pick.

Maybe EMC isn't as good of indicator as OPR.

67 - 64.1978
2054 - 58.5124
2767 - 23.3331

Maybe 2767 didn't really have the second best minibot at the regional, but rather was the luckiest because they always had a 2nd minibot on their alliances. 16.67 is the maximum minibot score (50/3). However, to get that you need help from your alliance partners. If you had the absolute fastest minibot, but never had a 2nd minibot from your partners, you would get 10. As others said, only watching the matches lets you know the full story.

waialua359 21-03-2011 02:31

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by XaulZan11 (Post 1042993)
Or maybe teams actually watched matches instead of relying on final scores and fancy formulas to rank teams...

I know I may be sounding like a jerk, but as someone who works on the picking list for my team, I would be pretty upset if I worked hard and scouted every match only to be second guessed by someone who likely didn't see any matches. OPR, ETC and various metrics are nice, but there is NO SUBSTITUTE for actually watching matches and seeing how good teams are.

This is the absolute truth.
The data is a nice to know and cool to know, but it doesnt change how I go about selecting alliance partners, if I'm in the top 8.
In fact, I know who I'd pick this weekend in HI if I had the #1 pick, even before watching any matches at all.

ttakashima 21-03-2011 02:52

Re: Top 25 ETCs after Week 3
 
I tend to use both as a resource when helping my team chose its partners, numbers and watching matches. While having your scouter watch matches, interview teams, go over autonomous modes, who scored which object and how many times etc. In the end it comes down to both watching matches and using those numbers, weather it be this algorithm or your own method of analyzing different points. Both could be a useful resource in picking a team to be on your alliance.

The Lucas 21-03-2011 09:21

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by speedbuggy76 (Post 1042702)
the Peachtree data for our robot into ERC and EMC? I'm just using the data set on the FIRST website which only contains total alliance scores.

The Twitter feed includes Bonus (minibot) and Penalty points for each alliance, the FIRST site does not. FRC-Spy also uses the Twitter feed to get this info.

Mike could you do an OPR calculation on the Minibot data for comparison since you already have the bonus data from the twitter feed. I imagine it will skew toward the extremes, and work a little better at identifying the best minibot.

JamesBrown 21-03-2011 09:37

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by lineskier (Post 1042982)
Well hindsight is 20/20 but lets look back at the selection for western Michigan where using EMC could have resulted in a very different story, and it all starts with the very first pick.

2054 picks team 67.
2054: ERC- 25 EMC-17 (top minibot and robot)
67: ERC- 20 EMC-8

it looks like an obvious and smart pick but...
in my opinion they overlook one key player which would have changed everything for them.

2767: ERC- 3 EMC-16

Had 2054 picked 2767 they would have had a monopoly over the best minibots. essentially a guaranteed 50 pts each match.

1918 at R: 16 and M:9 would have picked team 67
but finishing 3rd and 4th in the minibot race would have a very tough time beating 2054.

so i think sometimes we even overlook a potential first pick, because we want to form a powerhouse...

I am not sure this is proving what you want to prove, 67 has one of the most misleading ETCs for the season. If you compare them to other teams using only their data from Western Michigan then they have a EMC of 13, much closer to 2767's 16 and have a much higher 33 for ERC. I don't think there is much debate that 67 was an excellent first choice.

mwtidd 21-03-2011 09:38

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by XaulZan11 (Post 1042993)
Or maybe teams actually watched matches instead of relying on final scores and fancy formulas to rank teams...

I know I may be sounding like a jerk, but as someone who works on the picking list for my team, I would be pretty upset if I worked hard and scouted every match only to be second guessed by someone who likely didn't see any matches. OPR, ETC and various metrics are nice, but there is NO SUBSTITUTE for actually watching matches and seeing how good teams are.

No no, once again, of course this is the case. Where I can't watch the matches, I was trying to gain insights from the #s. I am trying to make statements, to see people prove me wrong :). I can never improve without being corrected. These insights are phenomenal



Quote:

Originally Posted by Joe Ross (Post 1042999)
Maybe EMC isn't as good of indicator as OPR.

67 - 64.1978
2054 - 58.5124
2767 - 23.3331

Maybe 2767 didn't really have the second best minibot at the regional, but rather was the luckiest because they always had a 2nd minibot on their alliances. 16.67 is the maximum minibot score (50/3). However, to get that you need help from your alliance partners. If you had the absolute fastest minibot, but never had a 2nd minibot from your partners, you would get 10. As others said, only watching the matches lets you know the full story.

Thanks for these insights, and for the record overall I had 2767 ranked very low too. with an ETC of 19 compared to 2054s 43.

I am working on a new way of trying to calculate EMCs by actually trying to estimate which robots put minibots up, and how much they would be worth. So if you put up first and second you both get 25 points. as opposed to 16.

You are absolutely right that EMC can be very much swayed by luck. but that is why I picked 2767, I wanted to see what you guys had to say.

Maybe I would have been better off saying 2767 had a very reliably minibot, to some teams this is more valuable than the fastest.

Regarding reliability how did 2767's minibot compare to 67, 74 , 1918.

these are the 3 teams ERCs
67 - 20
2054 - 25
2767 - 3

again please disprove these claims but:
2054 and 67 had the 2 best robots or most reliable robots
1918 was very good or very reliable
74 was good and reliable

now these statements may be wrong, I'm looking for insights on them :)

speedbuggy76 21-03-2011 09:39

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by The Lucas (Post 1043057)
The Twitter feed includes Bonus (minibot) and Penalty points for each alliance, the FIRST site does not. FRC-Spy also uses the Twitter feed to get this info.

Yes, I understand that. I'm trying to understand how Lineskier determined a ERC and EMC value for our (2415) robot for our most recent regional - Peachtree, since it does not do a twitter feed. Just trying to see if I can re-create these calculations.

I'm working on an excel file to calculate ETC for a given regional, but my calculations are still a little different than those in this thread. I'll post a file later and maybe you guys can tell me what's different.

mwtidd 21-03-2011 10:08

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by speedbuggy76 (Post 1043062)
Yes, I understand that. I'm trying to understand how Lineskier determined a ERC and EMC value for our (2415) robot for our most recent regional - Peachtree, since it does not do a twitter feed. Just trying to see if I can re-create these calculations.

I'm working on an excel file to calculate ETC for a given regional, but my calculations are still a little different than those in this thread. I'll post a file later and maybe you guys can tell me what's different.

Turns out it was tweeted, sorry about that:
its short name was GA

Quote:

Originally Posted by JamesBrown (Post 1043060)
I am not sure this is proving what you want to prove, 67 has one of the most misleading ETCs for the season. If you compare them to other teams using only their data from Western Michigan then they have a EMC of 13, much closer to 2767's 16 and have a much higher 33 for ERC. I don't think there is much debate that 67 was an excellent first choice.

I'm not arguing that they weren't a great pick. I wanted to gain some insights. This isn't at all proving what I want it to :) but I'm learning.

An interesting combination is 2054 and 67.
2054's alliances put up 1050 points at MI over 12 matches
67's alliances put up 1131 over 13

2054's alliances put up 415 minibot points over 12 matches
67's alliance put up 360 points over 13

this may be a source of some problems, as expected those 50 point matches and 0 matches make a huge difference for EMC, I need to find a new way of calculating that.
Ill be looking closely at Q42, where 2054's alliance got 2nd and 4th.


2767's alliances put up 430 minibot points over 12 matches
one thing the EMC did though is deflated 2767's minibot contribution
cause their alliances put up the most minibot points at the regional.
but they had the second highest emc not the highest

mwtidd 21-03-2011 10:21

Re: Top 25 ETCs after Week 3
 
I want to thank everyone for the help in this thread.

Adam Freeman 21-03-2011 10:21

Re: Top 25 ETCs after Week 3
 
I think anyone that was at West Michigan or saw the webcast would agree that picking 67, even for just a minibot would have been a better choice. Ours was just as reliable and faster.

Our minibot was launched in 15 of our 18 matches...the other three where it did not were from other issues that resulted in failed launches early in the competition or being essentially disabled (last match). Plus the one elim match where it hit first but did not register.

No offense to 2767, but they were not carrying teams with scoring...in fact they were basically just a defender and minibot for most of the competition...so yes that had a very very dependable minibot. But not the "best" at the WM.

67 and 2054 had the fastest and most reliable minibots there. We would have launched them for #1 and #2 in any match we were both fully functioning in (my opinion, don't ask 1918 or 27 :p ) .

As for having another metric to judge teams on...I think it would be good if it was accurate, but more so for the later picks and only if you were actually watching all the competition matches.

I will have to pull our scouting data for what positions our minibot finished in each match and see what our actual minibot point contribution was.

I know in some matches we just launched late in the count and allowed our partners to finish first. We did not get on the trigger button quickly until late Saturday morning and into the afternoon.

speedbuggy76 21-03-2011 10:29

Re: Top 25 ETCs after Week 3
 
Here's my calculation file for Peachtree using the dataset from the FIRST website (thus there are no ERC or EMC calculations). Can you take a look at it and see why my values are different than yours? Most of the calculations are done using formulas and VBA, let me know if anything is unclear.

Also, how do you import data from Twitter to do your calculations? For my data, I had to copy/paste the table of values for rankings, match results, and playoff results into a text file, which I then imported into excel as a tab delimited file. Probably not the easiest approach, and I'd like to make it more versatile to handle other regionals.

Link to peachtree calculations: http://dl.dropbox.com/u/11470/peacht...lculations.xls

mwtidd 21-03-2011 10:32

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by speedbuggy76 (Post 1043087)
Here's my calculation file for Peachtree using the dataset from the FIRST website (thus there are no ERC or EMC calculations). Can you take a look at it and see why my values are different than yours? Most of the calculations are done using formulas and VBA, let me know if anything is unclear.

Also, how do you import data from Twitter to do your calculations? For my data, I had to copy/paste the table of values for rankings, match results, and playoff results into a text file, which I then imported into excel as a tab delimited file. Probably not the easiest approach, and I'd like to make it more versatile to handle other regionals.

Link to peachtree calculations: http://dl.dropbox.com/u/11470/peacht...lculations.xls

unfortunately I'm on a mac, so it looks like the VB macros aren't working quite right. i'm getting all div 0s

For the twitter feed I use java, to grab each tweet and parse the match data out of each tweet.

mwtidd 21-03-2011 10:38

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Adam Freeman (Post 1043085)
No offense to 2767, but they were not carrying teams with scoring...in fact they were basically just a defender and minibot for most of the competition...so yes that had a very very dependable minibot. But not the "best" at the WM.

Thanks for the insights!

What's interesting about EMC and ERC is that it does reveal single task teams. 2767 had a 3 for a ERC which is essentially a zero.

Two things probably happened with your EMC, first you probably played with better alliance partners so they probably stole a larger portion of the minibot points than 2767's partners. Also you had a few 0s in there which didn't help either. Team 2767 only had one 0.

Adam Freeman 21-03-2011 10:49

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by lineskier (Post 1043061)
again please disprove these claims but:
2054 and 67 had the 2 best robots or most reliable robots
1918 was very good or very reliable
74 was good and reliable

now these statements may be wrong, I'm looking for insights on them :)

67's auton was very good (15/18), with twice being knocked off track by partners.
67 scored on average ~6.5 tubes/match.
67's minibot was very reliable and fast, as long as the drive team gave the minibot the respect it deserved. :rolleyes:

2054 had an absolutely spot on autonomous in every match.
2054 scored on average ~3.5 tubes/match.
2054's minibot was very dependable, launching almost every match for 1st or 2nd place (have to check the data we have to be certain).

1918 auton was 50/50
1918 scored ~4.5 tubes/match. Capable of putting up 9 if given the chance. Had a couple of cRIO issues during qualifications and one match were a tube got stuck on their robot, so they couldn't score anymore.
1918 had troubles launching their minibot in early matches, seemed to get better as the competition went on. Very fast minibot...probably close to 2054 and 67 in raw speed, but slower in deployment.

74 was basically a clone of 2054....very dependable, very reliable. But, they worked up to that reliability quickly Friday morning.

27 was basically the same as 1918. Very good scorer (very dangerous if left alone), but had some arm issues in qualifications. Auton was spotty. Minibot was really good... I think they did not launch twice the entire weekend.

I will try to pull the data for these teams and calculate the final numbers. By far the top 5 teams at the competition.

226, 1718, and 2000 were the other teams that were pretty strong too.

mwtidd 21-03-2011 11:00

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Adam Freeman (Post 1043106)
67's auton was very good (15/18), with twice being knocked off track by partners.
67 scored on average ~6.5 tubes/match.
67's minibot was very reliable and fast, as long as the drive team gave the minibot the respect it deserved. :rolleyes:

2054 had an absolutely spot on autonomous in every match.
2054 scored on average ~3.5 tubes/match.
2054's minibot was very dependable, launching almost every match for 1st or 2nd place (have to check the data we have to be certain).

1918 auton was 50/50
1918 scored ~4.5 tubes/match. Capable of putting up 9 if given the chance. Had a couple of cRIO issues during qualifications and one match were a tube got stuck on their robot, so they couldn't score anymore.
1918 had troubles launching their minibot in early matches, seemed to get better as the competition went on. Very fast minibot...probably close to 2054 and 67 in raw speed, but slower in deployment.

74 was basically a clone of 2054....very dependable, very reliable. But, they worked up to that reliability quickly Friday morning.

27 was basically the same as 1918. Very good scorer (very dangerous if left alone), but had some arm issues in qualifications. Auton was spotty. Minibot was really good... I think they did not launch twice the entire weekend.

I will try to pull the data for these teams and calculate the final numbers. By far the top 5 teams at the competition.

226, 1718, and 2000 were the other teams that were pretty strong too.

Thanks!

and its pretty cool cause if you look at ETC for those last 3:

226- 15
1718- 15
2000- 14

It seems down matches really kill teams. Teams with a high ETC are essentially reliable every qualifying match or are really really good (having to make up for a zero is not easy).

2054 had a robot score of 25 where you had a robot score of 20. The difference there was probably the autonomous. I tried to get first to include autonomous in the tweet... maybe next year.

I'm going to try to figure out how the race was actually finished. Its entirely possible to figure out which alliance finished in which places, and then use that to predict who finished in which position for each team.

JesseK 21-03-2011 11:27

Re: Top 25 ETCs after Week 3
 
Oy, I must have missed the thread that said what ETC and EMC are and why they're all the rave right now. I mean, OPR in and of itself isn't a really good metric since minibots more of a binary thing than a linear scoring thing.

IMO, judging from what I watched in Weeks 1-3, Tubes may give a decent gauge for OPR as it relates to tubes if the OPR excludes minibot. Yet minibots themselves can only come from watching the field since the data doesn't give a specific team for each bonus.

mwtidd 21-03-2011 11:40

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by JesseK (Post 1043122)
Oy, I must have missed the thread that said what ETC and EMC are and why they're all the rave right now. I mean, OPR in and of itself isn't a really good metric since minibots more of a binary thing than a linear scoring thing.

IMO, judging from what I watched in Weeks 1-3, Tubes may give a decent gauge for OPR as it relates to tubes if the OPR excludes minibot. Yet minibots themselves can only come from watching the field since the data doesn't give a specific team for each bonus.

Maybe I should specify, ETC, EMC and ERC are a set of metric I came up with. They are still in their very early stages. Where they can tell you some things about some robots, for most robots its turning out to be more of a reliability metric. If you have bad match your ETC drops dramatically.

Also I don't mean to inflate the value of the ETC metric, they are simply something I have been working on and trying to get real feedback on.

I believe that by using my EMC I can estimate the way the race in each match finished, I am working on this right now. Hopefully this will give us a better reading on minibots. I also am working on a filter for the ERC that eliminates essentially non-contributors.

ETC stands for estimated teams contribution
ERC estimated robots contribution
EMC estimated minibot contribution

JesseK 21-03-2011 11:58

Re: Top 25 ETCs after Week 3
 
I see your point about ranking the teams relative to each other; yet there will still have to be a sort of 'confidence' factored in before we can rely on it any sort of relativistic data like that. For example, the teams ranked 4-12 may be within a Y% of each other, therefore we really can only say with X% confidence that their true rankings are correct. If the confidence levels are low across the board, then the algorithm itself isn't as useful as it may appear.

Since it's a learning exercise, I'd encourage you to lay out your assumptions, figure out which statistical metrics are better (mean versus median, and does std. dev. help explain anything?), as well as the methods of the algorithm (why average minibot over 3 teams instead of 2 poles?). It can only help you more correctly align the algorithm's results with the reality on the field.

Having a predicted match score based upon N-assumptions is much more valuable than having something that says "This team is 10% better than your team".

mwtidd 21-03-2011 12:06

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by JesseK (Post 1043132)
I see your point about ranking the teams relative to each other; yet there will still have to be a sort of 'confidence' factored in before we can rely on it any sort of relativistic data like that. For example, the teams ranked 4-12 may be within a Y% of each other, therefore we really can only say with X% confidence that their true rankings are correct. If the confidence levels are low across the board, then the algorithm itself isn't as useful as it may appear.

Since it's a learning exercise, I'd encourage you to lay out your assumptions, figure out which statistical metrics are better (mean versus median, and does std. dev. help explain anything?), as well as the methods of the algorithm (why average minibot over 3 teams instead of 2 poles?). It can only help you more correctly align the algorithm's results with the reality on the field.

Having a predicted match score based upon N-assumptions is much more valuable than having something that says "This team is 10% better than your team".

Thanks for the feedback. Where the EMC is calculated using percentages there is usually a team that gets a very small share of the overall EMC.

But after actually running through the race options, 30 is the only score that can happen in 2 different ways, and is very easy to check against, basically if the opposing alliance score is 45 you came in second and forth, if not you came in first alone.

Then I will be attributing the minibots score to the appropriate teams based on EMC.

Confidence metrics are not sometimes I have dealt with much, but I am definitely going to research them. Again I am a long ways away from being happy. But there are times where these statistics paired with actual scouting data present some interesting information, that OPR may leave out.

The Lucas 21-03-2011 12:22

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by JesseK (Post 1043122)
Oy, I must have missed the thread that said what ETC and EMC are and why they're all the rave right now. I mean, OPR in and of itself isn't a really good metric since minibots more of a binary thing than a linear scoring thing.

IMO, judging from what I watched in Weeks 1-3, Tubes may give a decent gauge for OPR as it relates to tubes if the OPR excludes minibot. Yet minibots themselves can only come from watching the field since the data doesn't give a specific team for each bonus.

I think Logomotion is a game of diminishing returns which is difficult for all the metrics. So really good alliances are often less than the sum of their parts. It is easiest to visualize this by looking at JVN's scoring matrix. Those Ubertubes you put up high in auton are worth big points (2-4 times more than teleop). Then you form a high logos over those Ubertubes, again big points. Now you are out of High pegs so each additional tube is only worth a fraction of the points. If you are really great, you have to start working on the low pegs, where you have to wonder: At 1pt/tube is this even worth it or should I get ready for the minibot race? Additionally, 3 great hangers would still have to share 2 racks, which limits each individual scoring potential.

Minibot race is the same way. if you have the fastest minibot out there it is worth 30 pts. Your partner's mini bot is a tenth of a sec slower but it is only worth 20 pts. Your 3rd partner my have an equally fast minibot but it is always worth 0 since they cant legally deploy it. Now lets say your opponents have ridiculously slow minibots (like 15 sec). They still can get 25 combined points as long as they deploy them. Chances are in some matches they would get 50 pts with their slow minibots because their opponents dont have any.

Metrics like OPR are a good jumping off point but you still need real scouting to make an educated decision. ETC is still a work in progress but it is interesting to see the minibot scores separated.

techvikesmom 21-03-2011 13:10

Re: Top 25 ETCs after Week 3
 
all this information is very interesting to see and read. One comment about our kids is that they picked what they saw in the 2 days of qualifying rounds. The kids picked their alliance partners, not the mentors, not the all the statistics!! They did a great job on picking alliance, and playing and they played a great alliance. It was an exciting and awesome weekend for all the robots and their teams. Keep up the great alliances at Niles this weekend.

mwtidd 21-03-2011 13:53

Re: Top 25 ETCs after Week 3
 
If everyone could take a look at this, and tell me what insights you get this would really help.

This is using my minibot prediction algorithm, and what I see here is HUGE minibot inflation. Basically I think one of these teams is stealing points from other teams...

it looks like to calculate minibots accurately you actually have to look at it starting from the match, rather than doing an initial minibot rating by total points.

values left to right
etc
erc
emc


Wayne TenBrink 21-03-2011 14:49

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Jim Zondag (Post 1042958)
Me either, and I'm not convinced yet that I need any new acronyms or methods. Everyone keeps obsessing about who is the best, but this is irrelevant until Einstien. At a real tournament, you don't really need a spreadsheet to tell you who is the best. This is obvious; my mom can tell you who is the best with no data at all. A thousand varied analytical methods will be able to give you the top 8 teams in order.
You do, however, need good data and methods to tell you who is 16th-24th. This is where the real value add of competitve analysis is: not for the first pick, but for the second.

Every word is true (as is always the case with Jim). We don't spend much time obsessing over scouting data for our first round pick - the second round is where we focus our efforts.

As for West Michigan:
- 67 was the obvious first pick. They had the best all around package there. Scrap any data from Kettering.
- As #2 seed, 27 was our obvious pick. Another great all-around package.
- There were other good minibots and other decent hangers, but few that could put it all together.
- Our minibot didn't climb nearly as quickly as 27, 67, or 2054 (we'll try to change that by MSC). On Friday we deployed about 2/6 attempts with our backup mini. On Saturday, we made some corrections on the deployment, got the #1 mini back together, and were about 7/8 attempts. EMC data won't pick up on that, but anybody watching from the stands will.
- As with everyone, we had different issues at different times, all of which hurt us on ETC. RUSH also had technical problems Saturday morning. Things came together by eliminations, and standardized rating systems won't reflect that (for us or anybody else). That's why being there and knowing the status of things matters.
- I understand why our EMC is low, but I can't understand an ERC of 16 for 1918. I believe we are competitive with anybody out there for acquiring and hanging tubes.
- Alliances are definitely not the sum of their components. There is a point of diminishing returns in this game, and it is very easy to get in each other's way. If you look at the elimination data at WMD, our #2 alliance outscored the #1 alliance (albeit slightly) and our opponents underscored theirs over the course of the quarters and semi's, despite the fact that we worked through some of the higher seeds.

In general, an observer can tell who is capable of doing what, and they instinctively know when to dismiss certain match data. We tend to rate teams based on their expected capabilities when things are working properly. Recent performance is generally a better indication of future performance than historical average because of all the continuous improvement that goes on in FIRST. I think that ETC's might better reflect true capabilities if they omitted one or two of the lowest matches, or perhaps looked at a 10 match running average, like some golfing handicaps.

Regardless of the imperfection of any system, they have their merits. Many thanks to those who do the work to make them possible.

Edit: Another random observation from West MI - There was one qual match between 27 & 67. Other than that, neither 27 nor 1918 ever played against 67 or 2054 until the finals.

mwtidd 21-03-2011 15:18

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Wayne TenBrink (Post 1043208)

In general, an observer can tell who is capable of doing what, and they instinctively know when to dismiss certain match data. We tend to rate teams based on their expected capabilities when things are working properly. Recent performance is generally a better indication of future performance than historical average because of all the continuous improvement that goes on in FIRST. I think that ETC's might better reflect true capabilities if they omitted one or two of the lowest matches, or perhaps looked at a 10 match running average, like some golfing handicaps.

Regardless of the imperfection of any system, they have their merits. Many thanks to those who do the work to make them possible.

Thanks for the insights on the golfing handicaps, and its definitely something I am trying to figure out. The logistics of removing the lowest match is a little difficult as I keep a running sum which is adjusted as matches are added. Definitely on my todo list this week.

Thanks!

mwtidd 21-03-2011 16:46

Re: Top 25 ETCs after Week 3
 
Found a bug....
I think you guys will like these new #s better :)


Adam Freeman 21-03-2011 18:07

Re: Top 25 ETCs after Week 3
 
Mike,

Here is some scouting data that I pulled on the teams that have been mentioned from the WM district:

* These are for qualification matches only, since I am pretty sure that is what your ETC, ERC, and EMC are calculated with.

Average Scores
Team Auto Teleop Minibot Total
67 5.00 39.75 23.33 68.08
2054 6.00 28.92 20.83 55.75
74 4.25 25.83 18.75 48.83
27 2.00 25.42 19.17 46.58
1918 0.50 27.83 11.25 39.58
2767 0.00 1.33 18.75 20.08

From the scouting data, I can see if/where a team scored an Uber-tube and what place their minibot scored. For teleop I can see how many tubes and where they scored them. Since I know where 67 scored tubes and created logos, I made an assumption that any tube scored by a team was part of a logo...so the team received the contribution of points for that tube as if it was part of a logo.

It's interesting to see how the raw data correlates not only to your estimates, but also to my subjective ratings for each of these teams.

Hopefully this helps further adjust your metrics.

mwtidd 21-03-2011 18:34

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Adam Freeman (Post 1043340)
Mike,

Here is some scouting data that I pulled on the teams that have been mentioned from the WM district:

* These are for qualification matches only, since I am pretty sure that is what your ETC, ERC, and EMC are calculated with.

Average Scores
Team Auto Teleop Minibot Total
67 5.00 39.75 23.33 68.08
2054 6.00 28.92 20.83 55.75
74 4.25 25.83 18.75 48.83
27 2.00 25.42 19.17 46.58
1918 0.50 27.83 11.25 39.58
2767 0.00 1.33 18.75 20.08

From the scouting data, I can see if/where a team scored an Uber-tube and what place their minibot scored. For teleop I can see how many tubes and where they scored them. Since I know where 67 scored tubes and created logos, I made an assumption that any tube scored by a team was part of a logo...so the team received the contribution of points for that tube as if it was part of a logo.

It's interesting to see how the raw data correlates not only to your estimates, but also to my subjective ratings for each of these teams.

Hopefully this helps further adjust your metrics.

Thanks for these #'s this helps out a ton.

I don't think EMC can get much better than this, an error of +/- 3 is something I'm happy with. I think I've killed most of the inflation that was going on there. It seems robot (auton + teleop) is still a little low, but i will be working on a filter to knock out teams who really don't contribute. As you found 1918 had a 1.33 but I had them at a 5. Obviously there's a bit of point inflation there. Hopefully I can filter that out and get a better read.

Thanks again!

railerobotics 21-03-2011 20:30

Re: Top 25 ETCs after Week 3
 
Take a look at Teams 935 and 932 from the OKC Regional.:)
935 Had and Average of 60 points per match and 30 point bonus off there bot 9 out of 10 in Qualification matchs. Then in the finals the 6 matchs they ran they never missed with 1.6 mini bot with a .5 deployment.

932 did not have mini bot but Scored about the same tube average.:) This being said keep in mind the OKC regional had a number of rookie and w 2nd year teams that did not score very many tubes.

mwtidd 21-03-2011 21:23

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by railerobotics (Post 1043392)
Take a look at Teams 935 and 932 from the OKC Regional.:)
935 Had and Average of 60 points per match and 30 point bonus off there bot 9 out of 10 in Qualification matchs. Then in the finals the 6 matchs they ran they never missed with 1.6 mini bot with a .5 deployment.

932 did not have mini bot but Scored about the same tube average.:) This being said keep in mind the OKC regional had a number of rookie and w 2nd year teams that did not score very many tubes.

932 had a EMC of 0 and a ERC of 56
935 had a EMC of 24 and a ERC of 43

looking for good minibots I see
2773
2352
476

2165 and 704's minibots look like they'd be worth checking out
probably 3660 too

as far as robots go it looks like you two dominated
1108 is up there with you guys though with a 31
the next ERC tier would start around 20 but theres quite a few robots there.

let me know how close I got :)

Debbie 21-03-2011 21:59

Re: Top 25 ETCs after Week 3
 
It is an honor to be listed among all of those great teams! Thanks for your time spent crunching numbers! We are looking forward to our second district next week and are hoping to improve on those numbers!

Pjohn1959 22-03-2011 07:52

Re: Top 25 ETCs after Week 3
 
OK, I give up.

No where is it described what an ETC, EMC, etc. is. Can you help out by explaining what you are talking about.

Thanks,

Debbie 22-03-2011 08:13

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Pjohn1959 (Post 1043618)
OK, I give up.

No where is it described what an ETC, EMC, etc. is. Can you help out by explaining what you are talking about.

Thanks,

HA! Glad I'm not the only one that had problems with that. It is either Estimated Team Contribution (or expected - not really positive) and Estimated Robot Contribution and Estimated MiniBot Contribution... I think...

mwtidd 22-03-2011 09:20

Re: Top 25 ETCs after Week 3
 
Quote:

Originally Posted by Debbie (Post 1043630)
HA! Glad I'm not the only one that had problems with that. It is either Estimated Team Contribution (or expected - not really positive) and Estimated Robot Contribution and Estimated MiniBot Contribution... I think...

:) Its supposed to be estimated, but you're right.

Based on the data I'm verifying on in West Michigan EMC is doing pretty well. +/-3 of their actual average contribution.

The ERC isn't quite as accurate, its about 10 pts off, but if you look at it as a scale I think its still useful. I worked yesterday to fix EMC and hopefully I'll have ERC better by the end of the day.

Debbie 03-04-2011 23:25

Re: Top 25 ETCs after Week 3
 
Are you posting a new one including this weekend's events? I am curious to see how your rating compares to the one we calculated with scouting. :)


All times are GMT -5. The time now is 16:05.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi