Veteran Teams have No Advantage

[Guy Davidson]

Originally Posted by The Lucas

If anyone needs raw data from last year, I attached a zipped xml file that has all the regional stats. It can be viewed in excel. I generated it with my FIRSTXML program. I really need to update FIRST XML for this years regionals.

I'll take a look it this soon, but you might have just saved me and another team member a lot of manual labor.

Thanks!

Guy

[Bongle]

Originally Posted by Nuttyman54

http://www2.usfirst.org/2006comp/eve.../rankings.html is the list for BAE last year. Replace the NH with your respective regional town (SJ for Silicon Valley, WI for Wisconsin, etc)

I found that guessing for the acronym was very difficult, so I went to google and did this:

<word from regional name> results site:www2.usfirst.org

And google will get it almost every time.

[dtengineering]

Originally Posted by Bongle

I did this last year. I found that older teams DID have an advantage in terms of scoring, if not seeding. This was based on all the week 1 and week 2 regionals in 2006.

That is very interesting data and analysis, and suggests that perhaps there is a slight advantage to be gained from experience (which one might well expect) but certainly not an insurmountable one. It also meshes well with the idea that -- particularly for a student centred team -- there is an upper limit to how much experience one can gain. Does it make sense that a ten-year team would have any advantage over a four-year team?

I think that changes to the KOP and the availability of COTS mechanisms have also greatly helped rookies and evened up opportunities. A two-speed gearbox or mecanum wheel four years ago was something that required a lot of technical expertise.... now it requires a credit card and some bolts. Which isn't a bad thing, but it does level the playing field.

Mentoring and support also helps balance the playing field. I know our rookie neighbours (2273, who did quite well and placed 20 spots ahead of us in Portland) are using a couple of our old motors and speed controllers in their system because they didn't have the inventory of spare parts on hand that we do. Their neighbours in the pits, team 114, not only loaned them a robot cart at first, but then built them one! (One of the many reasons 114 took home the J&J Sportsmanship award... well done, #114!) One of the great things about FIRST is that veterans help rookies, rather than pummel them.

So I look at this as follows... either:

1) Veteran teams have NO inherent advantage in terms of winning qualifying matches. The data posted on scoring from last year, however, suggests that this might not be the case.

2) The current scheduling system eliminates any advantage gained by experience by repeatedly pitting veteran teams against each other rather than by having them compete against a random selection of opponents. Assuming that this scheduling system is new this year, this would explain why last year veteran teams scored higher, but this year had not advantage in qualifying matches. (Yes, two slightly different measurments, but the best we've got right now.)

In any case I understand that FIRST is considering an adjustment to the match scheduling system. If they choose to eliminate the rookies vs. rookies and veterans vs. veterans part of the formula we can re-run the qualifying results from next weekend and see if there is more of a correlation between team number and seeding.

I think that would be really interesting to look at...

Jason

P.S. What a great idea to assign this as a stats project to an AP stats class... I might save this as an assignment for some of my math classes, too.

[Joe Ross]

Originally Posted by Bongle

I found that guessing for the acronym was very difficult, so I went to google and did this:

<word from regional name> results site:www2.usfirst.org

And google will get it almost every time.

Even easier: http://www.usfirst.org/community/frc...t.aspx?id=4188

[Bongle]

Originally Posted by Joe Ross

Even easier: http://www.usfirst.org/community/frc...t.aspx?id=4188

Wow, I spent half an hour looking through their site yesterday for those before I gave up and did my google method. Thank you.

[Rich Ross]

Two things:

1. Off topic a bit, but every time that a thread like this pops up, we are justifying the current failure that is the match sheduling system. By saying that it levels the playing field, people from FIRST think more and more that this is acceptable. I'm going to be blunt and say it: I ama against the current algorithm (spelling?) and expect first to come up with something better that will pair us with and against a bigger variety of teams

2. There are verteran powerhouses (someone mentioned 25 and 254) but there are also rookie powerhouses. Maybe they dont win a regional every year, but every year you will see rookies that mow down veterans. 1503 won 3 regionals last year, with 1114 (both relatively new teams). It's a fact that rookies and veterans are both FIRST teams, and given similar knowledge, they will both perform well.

Let's let FIRST know that we want to play against everyone, rookies and vets.

[Nuttyman54]

It's not veteran status. It's not rookies status. It's infrastructure. The teams that are well organized, well directed and are set up to ensure longevity are the ones that do well.

In order to be successful, you have to learn from experience. Rookies that have experienced mentors and enthusiastic members will do well (see 1902 for a prime example).

On the flip side, veteran teams that have no continuity from year to year are going to do poorly, or have mixed results. Not having any information on past robots can be very detrimental, as the team has to re-learn all of it's previous mistakes. Team handbooks, dedicated sponsors and supportive communities are the marks of successful teams.

[Matt Reiland]

One advantage that vetern teams DO have is their lower number and their team history. I have been at many a regional where a Big Name team didn't have the strongest robot of the litter but they were picked based on reasons other than that particular years robot performance. Sometimes it is the experience under pressure, sometimes it is the way they sell themselves, it could have even been that it was an incredibly robust machine and went out for every single match. Either way you don't have to have the #1 seed robot to be on the winning team, you may even have the worst robot at the regional (Based on the qualifying match scores) especially this year when you see some of these matches go so lopsided when the right robots are out there together against and unevenly matched oponent.

[Guy Davidson]

Originally Posted by Matt Reiland

One advantage that vetern teams DO have is their lower number and their team history. I have been at many a regional where a Big Name team didn't have the strongest robot of the litter but they were picked based on reasons other than that particular years robot performance. Sometimes it is the experience under pressure, sometimes it is the way they sell themselves, it could have even been that it was an incredibly robust machine and went out for every single match. Either way you don't have to have the #1 seed robot to be on the winning team, you may even have the worst robot at the regional (Based on the qualifying match scores) especially this year when you see some of these matches go so lopsided when the right robots are out there together against and unevenly matched oponent.

This argument only supports the reason for a new algorithm. The less teams one is exposed to, the less quality information he is likely to have, the more likely he is to revert to picking a low-numbered team. Hence, we should revert to a random algorithm, that will let each team compete with and against more teams.

-Guy

EDIT: Results from the inference I ran on last year's regionals results are coming tomorrow. I promise

[ScoutingNerd175]

Our scouting team took data on the number of tetras/balls actually scored by robots at the regionals we were attending. This may be more reflective of how teams are actually scoring. The only data I have access to at the moment is the data for the 2005 Chesapeake Bay Regional. So, out of 55 teams at the regional, 25 (45%) were numbered above 1000 (arbitrary number) with 9 (16.3%) being rookies. Looking at just the top 8 (scorers, not seeds), 2 (25%) were numbered over 1000, 1 (12.5%) was a rookie team. If I pick an arbitrary number again, like 2 tetras, 36% of teams that scored 2+ tetras were numbered over 1000 and 18% were rookies.



So I'm not sure if this data is in any way meaningful, but it may be more meaningful if someone keeps data on scoring, or something along those lines, to compare that to team number (in a more meaningful way than this) or to compare scoring to seed this year and last year (for example what % of the top 8 seeds were also in the top 8 scoring this years as compared to last year) to see if this algorithm is having an effect on where the highest scoring teams seed. (I can't do that here because I don't know the seeds for 2005 Chesapeake)

Or it might all be meaningless. But match results definitely do not equal scoring ability (the highest scoring robot in the Curie division at 2005 Championships was seeded 11th). Although, scoring ability definitely does not equal robot quality.

[Lil' Lavery]

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.
Also, I am in no way supporting the existing scheduling algorithm.

[65_Xero_Huskie]

I would not base anything off of this year. This year is just starting and the "veteran" teams have many competitions to go to. I would personally base any information off of the championships, for any year. If you are a good team then you will strut your stuff in the finals. All teams who have a good robot will have a good team. Not all veteran teams have good teamwork, but they do have the experience of doing this kind of thing over and over. I really want the rookie teams to shine this year which is what keeps the FIRST spirit strong.

[Ian Curtis]

At BAE, 58 scored the most game pieces (20) on Friday. 1276 scored the next most (18). Then came 1307 (16) and 1073 (13). Then there was 175 (12).

All of these teams are at least at the end of a high school cycle. 58 was a rookie just before '98. 1276 and 1307 were rookies in 2004. 1073 has at least five years experience (maybe 6), Buzz was a rookie in '96.

I don't see a whole lot of correlation here, other than all of these teams have at least 4 years under their belt. BAE is pretty heavy with old teams, so this data may be weighted. I would say veteran teams should have an advantage, put after your third year, you've had the opportunity to learn as much as you need to be competative (with experienced mentors, this knowledge should come even quicker).

[Taylor]

Originally Posted by Lil' Lavery

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.

Outstanding work.
Hypothetically, say two very low numbered teams (0.1 and 0.2) are in the same regional. They are also the best teams at the regional. They are very close to each other in competition, but far and away from any other team there. According to the algorithm, it would be safe to say that 0.1 and 0.2 play each other quite a bit, and it would not be out of the question that they both end up with a .500 record.
Given the current algorithm, that would mean the two best teams aren't even close to being captains.
Not fair by a long shot. (However, being a 4-digit team ourselves, we're not complaining)
Hopefully this year the scouts will do a good job and evaluate each team by their capabilities and potential rather than their win-loss record, and the cream will rise to the top come Elimination time.

[dtengineering]

Originally Posted by Lil' Lavery

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.
Also, I am in no way supporting the existing scheduling algorithm.

You raise a good point about the match scheduling system at the first weekend regionals ensuring that every match a low numbered team won, a low numbered team also lost, however that doesn't make the data meaningless. It may mean precisely what you suggest when you say "the desired effect of the system was achieved".... which is a meaningful conclusion.

If we run a similar analysis after the second weekend regionals and see a correlation between team number and qualifying success it will demonstrate that at first weekend regionals there was an artificial barrier to the win/loss record of some teams, and will have some more solid evidence to back up the "common sense" idea that veteran teams do have an advantage -- in general -- over newer teams. Something that -- frankly -- I'm okay with. There should be some benefit to years of hard work... not an insurmountable or overwhelming benefit, but some benefit at least.

Thanks to everyone who has joined in with ideas, thoughts and suggestions -- particularly the statistics and observations from previous years. I'm looking forward to seeing what this weekend brings as it looks like FIRST is moving quickly to resolve this particular issue.

Jason