Veteran Teams have No Advantage

[dtengineering]

A low team number means absolutely NOTHING when it comes to predicting the team's performance in qualification rounds.

Now that I've got your attention, allow me to explain. I was inspired by discussion in this thread "Random" match Schedules (http://www.chiefdelphi.com/forums/showthread.php?t=55178) discussing the apparent lack of randomness in assigning alliances for qualifying matches. It was stated that FIRST intentionally schedules matches so that alliances have roughly equal seniority, based on the assumption (presumably) that this will lead to a more balanced match.

That didn't seem fair to me....

So I downloaded the results from the first weekend's competitions, and dropped them into excel. Based on these 270 data points there is abolutely NO relationship between team number and seeding. The teams numbered below 300 had an average seed of 26, while teams numbered above 2000 had an average seed of 27.8. Hardly an advantage (although, admittedly I didn't calculate exactly how much difference would be required to be statistically significant here). Running a correlation coefficient over the whole data set shows a .007 coefficient of correlation between team number and seeding... and in some regionals the coefficient is negative (but also insignficant.)

This leads me to three possible hypothesis:

1) We are all mistaken about the advantage that senior teams have. Hey, I've done it... you've done it... and now it is alleged that FIRST is doing it. We're looking at a few very successful veteran teams and saying "wow... watch out for the teams with low numbers" and completely forgetting that for every veteran out there rocking the rack that there is another one struggling with their design, and a rookie team that is doing even better.

2) Veteran teams do have an advantage, but something is being done to prevent that advantage from helping them to win matches. It is possible that a scheduling system that pits veterans against veterans removes the legitimate advantage that comes from years of hard work and development.

3) My statistical analysis is incorrect or incomplete. I'm always willing to admit the possibility that I might be wrong. I challenge anyone, however, to prove that veteran teams have done signficantly better in qualification matches than newer teams. Math, here, please people... not anecdotes. We humans are really good at seeing relationships that don't really exist.

I'm open to suggestions, and to someone willing to examine finals matches and outcomes to see if veteran teams have any statistical advantage on Saturday afternoon... or to someone who can find some better predictor of success (perhaps previous year's rankings can be a reliable indicator) but until then.... I say that veteran teams have no advantage when it comes to winning matches and that a scheduling system that (allegedly) uses team numbers as a factor is not only unjustified, but unscientific.

Jason

P.S. Yes, veteran teams (my gosh... we're one now, I think...) have many advantages (and challenges) that junior teams might not have... but not ones that significantly affect the outcomes of qualifying matches.

[Guy Davidson]

Does anyone have, or know where I can get qualifying match data from last year? I'd like to do something similar, but for last year, so I can get some idea as to how much the new match algorithm matters.

Thanks.

[Nuttyman54]

Does anyone have, or know where I can get qualifying match data from last year? I'd like to do something similar, but for last year, so I can get some idea as to how much the new match algorithm matters.

Thanks.

http://www2.usfirst.org/2006comp/events/NH/rankings.html is the list for BAE last year. Replace the NH with your respective regional town (SJ for Silicon Valley, WI for Wisconsin, etc)

[nuggetsyl]

A low team number means absolutely NOTHING when it comes to predicting the team's performance in qualification rounds.

Now that I've got your attention, allow me to explain. I was inspired by discussion in this thread "Random" match Schedules (http://www.chiefdelphi.com/forums/showthread.php?t=55178) discussing the apparent lack of randomness in assigning alliances for qualifying matches. It was stated that FIRST intentionally schedules matches so that alliances have roughly equal seniority, based on the assumption (presumably) that this will lead to a more balanced match.

That didn't seem fair to me....

So I downloaded the results from the first weekend's competitions, and dropped them into excel. Based on these 270 data points there is abolutely NO relationship between team number and seeding. The teams numbered below 300 had an average seed of 26, while teams numbered above 2000 had an average seed of 27.8. Hardly an advantage (although, admittedly I didn't calculate exactly how much difference would be required to be statistically significant here). Running a correlation coefficient over the whole data set shows a .007 coefficient of correlation between team number and seeding... and in some regionals the coefficient is negative (but also insignficant.)

This leads me to three possible hypothesis:

1) We are all mistaken about the advantage that senior teams have. Hey, I've done it... you've done it... and now it is alleged that FIRST is doing it. We're looking at a few very successful veteran teams and saying "wow... watch out for the teams with low numbers" and completely forgetting that for every veteran out there rocking the rack that there is another one struggling with their design, and a rookie team that is doing even better.

2) Veteran teams do have an advantage, but something is being done to prevent that advantage from helping them to win matches. It is possible that a scheduling system that pits veterans against veterans removes the legitimate advantage that comes from years of hard work and development.

3) My statistical analysis is incorrect or incomplete. I'm always willing to admit the possibility that I might be wrong. I challenge anyone, however, to prove that veteran teams have done signficantly better in qualification matches than newer teams. Math, here, please people... not anecdotes. We humans are really good at seeing relationships that don't really exist.

I'm open to suggestions, and to someone willing to examine finals matches and outcomes to see if veteran teams have any statistical advantage on Saturday afternoon... or to someone who can find some better predictor of success (perhaps previous year's rankings can be a reliable indicator) but until then.... I say that veteran teams have no advantage when it comes to winning matches and that a scheduling system that (allegedly) uses team numbers as a factor is not only unjustified, but unscientific.

Jason

P.S. Yes, veteran teams (my gosh... we're one now, I think...) have many advantages (and challenges) that junior teams might not have... but not ones that significantly affect the outcomes of qualifying matches.

maybe the reason why your data is so close is because teams are playing numbers close to each other. Example 11 and 25 played each other 4 times. one team is going to be seeded lower then the other and the average will always be in the middle. Your excel sheet would be better if you used last years numbers.

[Jeremiah Johnson]

maybe the reason why your data is so close is because teams are playing numbers close to each other. Example 11 and 25 played each other 4 times. one team is going to be seeded lower then the other and the average will always be in the middle. Your excel sheet would be better if you used last years numbers.

I agree... you may also want to take into account the number of teams per regional. See how many closely numbered teams were playing other close numbered teams and then factor in the number of teams at the event. I don't know how to do that, I'm not a math person.

[dtengineering]

maybe the reason why your data is so close is because teams are playing numbers close to each other. Example 11 and 25 played each other 4 times. one team is going to be seeded lower then the other and the average will always be in the middle. Your excel sheet would be better if you used last years numbers.

I don't disagree, however my analysis is based on this year's data. Please see hypothesis two concerning it's interpretation.

I'm looking forward to seeing an analysis of last year's numbers (it sounds as if someone is running that right now...). If lower numbered teams did significantly better last year relative to newer teams than this year, and FIRST has implemented a new scheduling system, then I would say that the scheduling system has served to undermine years of hard work and development on veteran teams.

"IF", however, is a pretty big word. I'll wait to see the results. Personally, until I see otherwise I think the "low number=power house" hypothesis is based on the same irrational pattern recognition that makes us silly humans believe in lucky rabbit's feet and horseshoes.

Jason

[Guy Davidson]

I'm currently running data analysis on last year's data, and it seems to support the notion that lower number teams have no advantage. I think that notion comes due to the existence several low-number power houses - 25 and 254 serve as great examples. However, many other low numbered teams are not nearly as strong. I'll present some data tonight, and get some more online tomorrow.

[AdamHeard]

I remember seeing a thread a while back that compared '05 and '06 success at nationals. There may be some relevant data in the graphs in that thread if anyone remembers what thread I am referring to.

[Petey]

[U]
2) [B]Veteran teams do have an advantage, but something is being done to prevent that advantage from helping them to win matches. It is possible that a scheduling system that pits veterans against veterans removes the legitimate advantage that comes from years of hard work and development.

Or, it is possible that if you have two veteran teams and a younger team, the strength of that veteran team may be able to overcome the weakness of the younger team.

Perhaps a better statement would be that won/loss records mean nearly nothing when it comes to evaluating the strength of any robot. WPI had a much stronger robot than us this year and yet finished nearly 10 spots lower. Why? Because they had weak alliance partners and faced Buzz four times.

Why do you think you see so many teams keeping spreadsheets of individual robots performances? Why do you suppose that, after the usual top-8 incest (after all, if you're ranked #1, chances are you are dominant enough to carry both of your partners on your backs), we often see teams picking from the 20's or 30's instead of the teens?

It's because alliances, throughout which the strength and weakness is divided, =! teams.

Of course veteran teams have an advantage. They have infrastructure. They have institutional expertise. They have tried and failed and thus come out with more knowledge. You can have rookie teams that do quite well, but how often do you see a rookie team come out of the blue and dominate the field? A second year team? Even a third? If you could find the data, try seeing how many younger teams really dominate in any of the game components. I don't think it happens.

Your analysis isn't wrongheaded, it's just that the data is deceptive.

[RoboMom]

Of course veteran teams have an advantage. They have infrastructure. They have institutional expertise. They have tried and failed and thus come out with more knowledge.

Your analysis isn't wrongheaded, it's just that the data is deceptive.

There are many struggling veteran teams. Just because they have a low number doesn't mean there is an institutional advantage. Many of the lower number teams "recreate" themselves and go through any number of crises and evolve into a new team. Their number may be one of the few things that stays the same and the lessons learned don't necessarily get passed on. I don't have any hard data to back up what I just said; only personal experience.
For every statistic, there is another story.

[meaubry]

Measuring the true capability of any team (regardless of how long they have competed) requires more than simply analyzing the outcome of the matches that they participated in.

The results may mask many other things that a single participant in any alliance would have had to deal with. Not many veteran teams can overcome, incompatible alliance pairings, mechanical breakdowns, strategic mistakes, etc. - all the things that could go wrong or right that often determine the outcome of a match.

Unfortunately, the scores and match outcomes is what is often used to evaluate (statistically) these kinds of debates/discussions.

Yes, one would assume that more veteran teams would improve over time, utilize the things that they have witnessed, attempted, failed, and succeeded at.

There are far too many variables involved to make statements one way or the other, as to whom has an advantage over whom.

One can only assume, that veteran teams have learned to deal with the challenges better than the lesser experienced team, strickly due to level of exposure.

[mathking]

I think that one thing you definitely need to look at in addition to what you have done so far (which seems sound) is to look at how likely veteran teams are to place in the top 8 compared to younger teams. Which is different from the analysis you have done so far. It is an interesting question though. Gives me a good idea for an assignment for AP Statistics class while I am gone for the regional this weeked.

OK, I also just took a look at some of the data and it occurs to me that your result is exactly what FIRST would be trying to achieve by randomizing matches as it did.

[Dave Campbell]

What makes a veteran team? Participating in one year then returning the next should qualify that statement. Or maybe being one of the founding teams qualifies at team as a veteran.
Who gives a team this seniority? Our team for example is #1038. Our student population changes every two to three years. Are we less experienced than Team #45? We have mentors who are very active on our team that have been building FIRST robots since 1994, as a matter of fact, several of these robots have been very successful in the FIRST contests - (Sunny D, 1994 National Champs, 1996 National Semi + Chairmans Award, and the list for our former team - #144, goes on until we switched schools in 1998)
The students on some teams change every year...The mentors change occasionally, too. So, is there really an advantage to participation from year to year? It totally depends on the composition of individual teams and therefore cannot be controlled in an algorithm for match scheduling that is intended to have "veteran" teams play matches with other experienced teams, and "rookies" play more rookies. We sometimes have great years, followed by some "re-building" years. Sometimes we figure the game out, other times we totally miss it. That is part of life, part of FIRST. Your team number does not imply any "powerhouse" status. That is earned by consistant performance and should not be accounted for in any match scheduling.
I agree with the title of this thread. With returning mentors and students there are always lessons learned and re-applied to following years, but with a student-centric team, the personnel changes can be drastic from year to year and are not reflected accurately by a team number.

[Bongle]

I did this last year. I found that older teams DID have an advantage in terms of scoring, if not seeding. This was based on all the week 1 and week 2 regionals in 2006.

http://www.chiefdelphi.com/forums/attachment.php?attachmentid=4111&d=1142127063 - Average alliance team number versus score
http://www.chiefdelphi.com/forums/attachment.php?attachmentid=4114&d=1142139985 - Average alliance age in years versus score
And Here's the thread (http://www.chiefdelphi.com/forums/showthread.php?t=45270&highlight=week+2+statistics), with many more charts and things.

I remember seeing a thread a while back that compared '05 and '06 success at nationals. There may be some relevant data in the graphs in that thread if anyone remembers what thread I am referring to.
I think that was me as well. I downloaded the seed results of a bunch of teams from a bunch of regionals. I normalized them so that they were between 0 and 1, and the plotted the results. A point at (x,y) means that in 2005 a team got ranking x, and in 2006 they got ranking y. This graph shows that there is little year-to-year correlation for performance. Someone who does awful one year can get as high as the top 10% in the next year. Likewise, someone who wins regionals one year can go to the bottom 10% the next year. The powerhouse teams that win consistently from year to year aren't as widespread as people think.
http://www.chiefdelphi.com/forums/attachment.php?attachmentid=4724&d=1162868157

[The Lucas]

If anyone needs raw data from last year, I attached a zipped xml file that has all the regional stats. It can be viewed in excel. I generated it with my FIRSTXML program (http://www.chiefdelphi.com/media/papers/1832). I really need to update FIRST XML for this years regionals.

[Guy Davidson]

If anyone needs raw data from last year, I attached a zipped xml file that has all the regional stats. It can be viewed in excel. I generated it with my FIRSTXML program (http://www.chiefdelphi.com/media/papers/1832). I really need to update FIRST XML for this years regionals.

I'll take a look it this soon, but you might have just saved me and another team member a lot of manual labor.

Thanks!

Guy

[Bongle]

http://www2.usfirst.org/2006comp/events/NH/rankings.html is the list for BAE last year. Replace the NH with your respective regional town (SJ for Silicon Valley, WI for Wisconsin, etc)

I found that guessing for the acronym was very difficult, so I went to google and did this:

<word from regional name> results site:www2.usfirst.org

And google will get it almost every time.

[dtengineering]

I did this last year. I found that older teams DID have an advantage in terms of scoring, if not seeding. This was based on all the week 1 and week 2 regionals in 2006.


That is very interesting data and analysis, and suggests that perhaps there is a slight advantage to be gained from experience (which one might well expect) but certainly not an insurmountable one. It also meshes well with the idea that -- particularly for a student centred team -- there is an upper limit to how much experience one can gain. Does it make sense that a ten-year team would have any advantage over a four-year team?

I think that changes to the KOP and the availability of COTS mechanisms have also greatly helped rookies and evened up opportunities. A two-speed gearbox or mecanum wheel four years ago was something that required a lot of technical expertise.... now it requires a credit card and some bolts. Which isn't a bad thing, but it does level the playing field.

Mentoring and support also helps balance the playing field. I know our rookie neighbours (2273, who did quite well and placed 20 spots ahead of us in Portland) are using a couple of our old motors and speed controllers in their system because they didn't have the inventory of spare parts on hand that we do. Their neighbours in the pits, team 114, not only loaned them a robot cart at first, but then built them one! (One of the many reasons 114 took home the J&J Sportsmanship award... well done, #114!) One of the great things about FIRST is that veterans help rookies, rather than pummel them.

So I look at this as follows... either:

1) Veteran teams have NO inherent advantage in terms of winning qualifying matches. The data posted on scoring from last year, however, suggests that this might not be the case.

2) The current scheduling system eliminates any advantage gained by experience by repeatedly pitting veteran teams against each other rather than by having them compete against a random selection of opponents. Assuming that this scheduling system is new this year, this would explain why last year veteran teams scored higher, but this year had not advantage in qualifying matches. (Yes, two slightly different measurments, but the best we've got right now.)

In any case I understand that FIRST is considering an adjustment to the match scheduling system. If they choose to eliminate the rookies vs. rookies and veterans vs. veterans part of the formula we can re-run the qualifying results from next weekend and see if there is more of a correlation between team number and seeding.

I think that would be really interesting to look at...

Jason

P.S. What a great idea to assign this as a stats project to an AP stats class... I might save this as an assignment for some of my math classes, too.

[Joe Ross]

I found that guessing for the acronym was very difficult, so I went to google and did this:

<word from regional name> results site:www2.usfirst.org

And google will get it almost every time.

Even easier: http://www.usfirst.org/community/frc/content.aspx?id=4188

[Bongle]

Even easier: http://www.usfirst.org/community/frc/content.aspx?id=4188

Wow, I spent half an hour looking through their site yesterday for those before I gave up and did my google method. Thank you.

[Rich Ross]

Two things:

1. Off topic a bit, but every time that a thread like this pops up, we are justifying the current failure that is the match sheduling system. By saying that it levels the playing field, people from FIRST think more and more that this is acceptable. I'm going to be blunt and say it: I ama against the current algorithm (spelling?) and expect first to come up with something better that will pair us with and against a bigger variety of teams

2. There are verteran powerhouses (someone mentioned 25 and 254) but there are also rookie powerhouses. Maybe they dont win a regional every year, but every year you will see rookies that mow down veterans. 1503 won 3 regionals last year, with 1114 (both relatively new teams). It's a fact that rookies and veterans are both FIRST teams, and given similar knowledge, they will both perform well.

Let's let FIRST know that we want to play against everyone, rookies and vets.

[Nuttyman54]

It's not veteran status. It's not rookies status. It's infrastructure. The teams that are well organized, well directed and are set up to ensure longevity are the ones that do well.

In order to be successful, you have to learn from experience. Rookies that have experienced mentors and enthusiastic members will do well (see 1902 for a prime example).

On the flip side, veteran teams that have no continuity from year to year are going to do poorly, or have mixed results. Not having any information on past robots can be very detrimental, as the team has to re-learn all of it's previous mistakes. Team handbooks, dedicated sponsors and supportive communities are the marks of successful teams.

[Matt Reiland]

One advantage that vetern teams DO have is their lower number and their team history. I have been at many a regional where a Big Name team didn't have the strongest robot of the litter but they were picked based on reasons other than that particular years robot performance. Sometimes it is the experience under pressure, sometimes it is the way they sell themselves, it could have even been that it was an incredibly robust machine and went out for every single match. Either way you don't have to have the #1 seed robot to be on the winning team, you may even have the worst robot at the regional (Based on the qualifying match scores) especially this year when you see some of these matches go so lopsided when the right robots are out there together against and unevenly matched oponent.

[Guy Davidson]

One advantage that vetern teams DO have is their lower number and their team history. I have been at many a regional where a Big Name team didn't have the strongest robot of the litter but they were picked based on reasons other than that particular years robot performance. Sometimes it is the experience under pressure, sometimes it is the way they sell themselves, it could have even been that it was an incredibly robust machine and went out for every single match. Either way you don't have to have the #1 seed robot to be on the winning team, you may even have the worst robot at the regional (Based on the qualifying match scores) especially this year when you see some of these matches go so lopsided when the right robots are out there together against and unevenly matched oponent.

This argument only supports the reason for a new algorithm. The less teams one is exposed to, the less quality information he is likely to have, the more likely he is to revert to picking a low-numbered team. Hence, we should revert to a random algorithm, that will let each team compete with and against more teams.

-Guy

EDIT: Results from the inference I ran on last year's regionals results are coming tomorrow. I promise :P

[ScoutingNerd175]

Our scouting team took data on the number of tetras/balls actually scored by robots at the regionals we were attending. This may be more reflective of how teams are actually scoring. The only data I have access to at the moment is the data for the 2005 Chesapeake Bay Regional. So, out of 55 teams at the regional, 25 (45%) were numbered above 1000 (arbitrary number) with 9 (16.3%) being rookies. Looking at just the top 8 (scorers, not seeds), 2 (25%) were numbered over 1000, 1 (12.5%) was a rookie team. If I pick an arbitrary number again, like 2 tetras, 36% of teams that scored 2+ tetras were numbered over 1000 and 18% were rookies.



So I'm not sure if this data is in any way meaningful, but it may be more meaningful if someone keeps data on scoring, or something along those lines, to compare that to team number (in a more meaningful way than this) or to compare scoring to seed this year and last year (for example what % of the top 8 seeds were also in the top 8 scoring this years as compared to last year) to see if this algorithm is having an effect on where the highest scoring teams seed. (I can't do that here because I don't know the seeds for 2005 Chesapeake)

Or it might all be meaningless. But match results definitely do not equal scoring ability (the highest scoring robot in the Curie division at 2005 Championships was seeded 11th). Although, scoring ability definitely does not equal robot quality.

[Lil' Lavery]

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.
Also, I am in no way supporting the existing scheduling algorithm.

[65_Xero_Huskie]

I would not base anything off of this year. This year is just starting and the "veteran" teams have many competitions to go to. I would personally base any information off of the championships, for any year. If you are a good team then you will strut your stuff in the finals. All teams who have a good robot will have a good team. Not all veteran teams have good teamwork, but they do have the experience of doing this kind of thing over and over. I really want the rookie teams to shine this year which is what keeps the FIRST spirit strong.

[Ian Curtis]

At BAE, 58 scored the most game pieces (20) on Friday. 1276 scored the next most (18). Then came 1307 (16) and 1073 (13). Then there was 175 (12).

All of these teams are at least at the end of a high school cycle. 58 was a rookie just before '98. 1276 and 1307 were rookies in 2004. 1073 has at least five years experience (maybe 6), Buzz was a rookie in '96.

I don't see a whole lot of correlation here, other than all of these teams have at least 4 years under their belt. BAE is pretty heavy with old teams, so this data may be weighted. I would say veteran teams should have an advantage, put after your third year, you've had the opportunity to learn as much as you need to be competative (with experienced mentors, this knowledge should come even quicker).

[Taylor]

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.

Outstanding work.
Hypothetically, say two very low numbered teams (0.1 and 0.2) are in the same regional. They are also the best teams at the regional. They are very close to each other in competition, but far and away from any other team there. According to the algorithm, it would be safe to say that 0.1 and 0.2 play each other quite a bit, and it would not be out of the question that they both end up with a .500 record.
Given the current algorithm, that would mean the two best teams aren't even close to being captains.
Not fair by a long shot. (However, being a 4-digit team ourselves, we're not complaining)
Hopefully this year the scouts will do a good job and evaluate each team by their capabilities and potential rather than their win-loss record, and the cream will rise to the top come Elimination time.

[dtengineering]

Because the controversial scheduling system is in place, all this shows that the desired effect of the system was achieved. Every match a low number team won, a low number team lost.
This data is meaningless.
Also, I am in no way supporting the existing scheduling algorithm.

You raise a good point about the match scheduling system at the first weekend regionals ensuring that every match a low numbered team won, a low numbered team also lost, however that doesn't make the data meaningless. It may mean precisely what you suggest when you say "the desired effect of the system was achieved".... which is a meaningful conclusion.

If we run a similar analysis after the second weekend regionals and see a correlation between team number and qualifying success it will demonstrate that at first weekend regionals there was an artificial barrier to the win/loss record of some teams, and will have some more solid evidence to back up the "common sense" idea that veteran teams do have an advantage -- in general -- over newer teams. Something that -- frankly -- I'm okay with. There should be some benefit to years of hard work... not an insurmountable or overwhelming benefit, but some benefit at least.

Thanks to everyone who has joined in with ideas, thoughts and suggestions -- particularly the statistics and observations from previous years. I'm looking forward to seeing what this weekend brings as it looks like FIRST is moving quickly to resolve this particular issue.

Jason

[Doug Leppard]

It's not veteran status. It's not rookies status. It's infrastructure. The teams that are well organized, well directed and are set up to ensure longevity are the ones that do well.

In order to be successful, you have to learn from experience. Rookies that have experienced mentors and enthusiastic members will do well (see 1902 for a prime example).

On the flip side, veteran teams that have no continuity from year to year are going to do poorly, or have mixed results. Not having any information on past robots can be very detrimental, as the team has to re-learn all of it's previous mistakes. Team handbooks, dedicated sponsors and supportive communities are the marks of successful teams.

As a mentor of 1902 I agree. My previous team 1083 the school decided not to continue the team. A team mom on 1083 from another school decided to run it from their home without official shool sponsership and so at last minute started 1902.

But we pulled in experienced students, college mentors, adult mentors. George has 10 years experience, I have 5.

We are lucky to have deep infrastructure:
Strong hardware and partnered with another team to fill in our weaknesses
Strong software that has developed successful auto modes
Strong driver strategy and strong drive team
Strong non-engineers that raise money and awareness

It takes all that and more to make a succesfull team.

But we are still growing and learning what infrastructure it takes to run a team and be succesful. We are still making rookie mistakes but are learning and adjusting to those mistakes.

Big one last year when the hardware guys gave us a day to to auto mode and it put us ahead of others. This year they worked hard and delivered the bot over a week ahead so we had lots of driver practice, spent 60 hours on details of software and auto mode and worked out hardware problems.

Final thing is the team is fun, that stage being set by our team leader Wendy, we work together well.