[EWCP] Presents TwentyFour -- An FRC Statistics Blog

[Jared Russell]

Originally Posted by Andrew Schreiber

Just a heads up guys, we've released another post talking about the exploits of Dozer (the little plow bot) over the years.

Dang it feels good to be a Dozer

Great article. You give Dozer credit for 10 points during Rebound Rumble...I assume that's equating half a Co-op balance to 10 points (similar to what many did with OPR scores)?

[Ian Curtis]

Originally Posted by Jared341

Great article. You give Dozer credit for 10 points during Rebound Rumble...I assume that's equating half a Co-op balance to 10 points (similar to what many did with OPR scores)?

That is correct. Co-op is an annoying game mechanic when you are trying to back out robot goodness from match data because it could do obnoxiously perverse things to team incentives. But we can at least give Dozer the 10 pts.

[Kims Robot]

Originally Posted by Wayne TenBrink

Although many teams over-reach technically and would do better with less, it is still good to reach a bit beyond your comfort zone. A functioning, "simple" machine is best at early events where it is playing against non-functioning "complex" machines that haven't reached their full potential. They will eventually reach a plateau and struggle to remain competitive. I think a team should understand their technical limitation and design within them, but you should always strive to be competitive against the "great" teams, not the pack.

I like bits and pieces of both the original conclusions and this statement. I agree that teams should reach a bit outside their comfort zone (only those willing to fail will ever have great success), but to the original posts point, I think many reach WAY too far outside their capabilities. And it is also incredibly important for teams to decide what their goal is...
Are you building:
1. A Robot within your means
2. A Robot that can do everything
3. A Robot designed to win Matches
4. A Robot designed to win Regionals
5. A Robot designed to win Championships
Arguably these five points can be very different robots, and can be very different for each team. And in reality there are probably between 3 & 10 teams out of thousands that can do all five. I think the data presented shows us that very well. Each team needs to do an analysis of the numbers, the ways to score, and what they think they are capable of, and then figure out how to match that to what they want to do.

And another point that has been discussed before but I've only seen hinted at here, is that a HUGE factor in any of the "winning" strategies is driver time. Even an average robot with great drivers who have lots of practice and good strategy can very easily seed high and win events.

[techtiger1]

I want to thank EWCP for putting this blog together. I love statistics. My team is considered one of those that pushes the envelope as far as design and tries to be competitive to a chamionship level. I'll admit we have completely missed the mark sometimes and bit off way more than we could chew. Strategy alone can win you alot of matches in FRC and I think this blog more than anything else shows that.

[Andrew Schreiber]

Some thoughts on the 3 Day Robot.

http://twentyfour.ewcp.org/post/4007...he-3-day-robot

As an aside, the Robot in 3 Days guys will be on the EWCPcast this Sunday at 9pm EST.

[Sandvich]

These are very good posts, please keep it up.

[Ian Curtis]

Another entry -- Do Teams Get Better at Events? The answer is unsurprising, but the magnitude of the improvement and what teams get better at may come as a surprise.

Has anyone ever done scouting with the time value of points? I'm not sure the result would be worth the effort, but it is an interesting thought.

[Nuttyman54]

Originally Posted by Ian Curtis

Another entry -- Do Teams Get Better at Events? The answer is unsurprising, but the magnitude of the improvement and what teams get better at may come as a surprise.

Has anyone ever done scouting with the time value of points? I'm not sure the result would be worth the effort, but it is an interesting thought.

Very interesting, I'm looking forward to the further results on the topic.

Regarding the time-value of points scouting, you are proposing to determine the "value" of points throughout the competition, and then modify a team's worth based on the "valued" points they score? Thus, a robot which consistently scores 3 points every match and does not improve or decline is worse at the end of the competition than at the start, because 3pts is worth less at the end?

It's an interesting theory. If you could get your hands on some hard scouting data from someone (maybe 1114?) for one of their events, you could test it out and see how much of an effect it has on the final results.

[Andrew Schreiber]

Originally Posted by Nuttyman54

Very interesting, I'm looking forward to the further results on the topic.

Regarding the time-value of points scouting, you are proposing to determine the "value" of points throughout the competition, and then modify a team's worth based on the "valued" points they score? Thus, a robot which consistently scores 3 points every match and does not improve or decline is worse at the end of the competition than at the start, because 3pts is worth less at the end?

It's an interesting theory. If you could get your hands on some hard scouting data from someone (maybe 1114?) for one of their events, you could test it out and see how much of an effect it has on the final results.

If anyone is willing to send scouting data our way let us know and we'll see what we can do.

Just a heads up guys, don't expect this pace of updates all the time. We've got a lot of data we want to get out to teams while it can still be useful to them.

[Jon Stratis]

I'll see what sort of data we still have sitting around on computers... we've been using an excel-based scouting system for several years aimed at quantitative analysis of points scored by each individual robot for our events. If we have those spreadsheets somewhere, they might provide what you're looking for, at least for the Minnesota competitions.

[IKE]

Originally Posted by Ian Curtis

Another entry -- Do Teams Get Better at Events? The answer is unsurprising, but the magnitude of the improvement and what teams get better at may come as a surprise.

Has anyone ever done scouting with the time value of points? I'm not sure the result would be worth the effort, but it is an interesting thought.

Very interesting. This seems to correlate well with the multiple event and the "practice makes perfect" mantra. IE, the more you play, the better you get.

An interesting experiement to observe. Take someone who does not play video games, and give them a game that needs some controller level skill. Say racing. Mario cart is particularly good. Have them keep practicing the same level. You will find a drastically different driver at the end of 1 hour of drive time...
This would be a really interesting experiment at the next Extra Life 24 hour video game a thon. My guess is the trends you see in that sort of experiement would correlate pretty well with amoutn of play time and scoring.

[Akash Rastogi]

Originally Posted by Ian Curtis

Another entry -- Do Teams Get Better at Events? The answer is unsurprising, but the magnitude of the improvement and what teams get better at may come as a surprise.

Has anyone ever done scouting with the time value of points? I'm not sure the result would be worth the effort, but it is an interesting thought.

Nice another very interesting blog post, Ian!

[Ian Curtis]

Originally Posted by IKE

Very interesting. This seems to correlate well with the multiple event and the "practice makes perfect" mantra. IE, the more you play, the better you get.

An interesting experiement to observe. Take someone who does not play video games, and give them a game that needs some controller level skill. Say racing. Mario cart is particularly good. Have them keep practicing the same level. You will find a drastically different driver at the end of 1 hour of drive time...
This would be a really interesting experiment at the next Extra Life 24 hour video game a thon. My guess is the trends you see in that sort of experiement would correlate pretty well with amoutn of play time and scoring.

That is actually how I initially phrased the article, that practice is super important since teams get so much better and you don't want to short-change your drivers. Someone pointed out though that perhaps that improvement was coming from good teams getting even better than unpracticed teams improving.

After some cursory glances it looks like the bottom tier does not really improve over the course of the event (low scoring events do not see the same rise the "typical" event sees). Looking at some traditionally stronger regionals you see the "typical" upward trajectory, but I haven't gone through to prove to myself who drives that. I sincerely hope it is the majority of the field, but since scoring is so concentrated in the "elite" tier it is possible for maybe the top 20% or less to drag the event average up by themselves.

[Nuttyman54]

Originally Posted by Ian Curtis

After some cursory glances it looks like the bottom tier does not really improve over the course of the event (low scoring events do not see the same rise the "typical" event sees). Looking at some traditionally stronger regionals you see the "typical" upward trajectory, but I haven't gone through to prove to myself who drives that. I sincerely hope it is the majority of the field, but since scoring is so concentrated in the "elite" tier it is possible for maybe the top 20% or less to drag the event average up by themselves.

There are several factors at work here. Certainly practice and play time will make anyone better, and teams have a chance to make changes and improvements to the robot over the course of the competition. But good teams automatically give themselves room for improvement.

A team whose robot has a fundamentally flawed mechanism that never works is not going to see much improvement over the event, because no matter how good their drivers get, the mechanism is their limitation. A better team that is working on accuracy might have more success as their drivers improve their aim. A great team who has good accuracy will improve as their drivers get faster and can make more shots.

The tricky part is that you're inherently focusing on offensive capabilities that scale. A robot who is only designed to play defense and hang for 10pts this year won't see any "improvement" in this scheme, even if their mechanism works perfectly, because they can only score 10pts maximum ever. Their defense may improve drastically over the course of the event, or how quickly they hang, but we don't have an accurate way to statistically evaluate that.

[Ian Curtis]

Originally Posted by IKE

Very interesting. This seems to correlate well with the multiple event and the "practice makes perfect" mantra. IE, the more you play, the better you get.

An interesting experiement to observe. Take someone who does not play video games, and give them a game that needs some controller level skill. Say racing. Mario cart is particularly good. Have them keep practicing the same level. You will find a drastically different driver at the end of 1 hour of drive time...
This would be a really interesting experiment at the next Extra Life 24 hour video game a thon. My guess is the trends you see in that sort of experiement would correlate pretty well with amoutn of play time and scoring.

So, I kept thinking about this & Evan's comments and realized I was probably being too pessimistic. I'm not sure if any self respecting statistician would ever use a rolling median, but I plotted one anyways and it says rather optimist things about the improvement of the 50th percentile over the course of the event. Based on the linear regression line, the median match starts at 12.6 points and rises to 21.2 over the course of the event, a roughly 70% improvement.



Then I plotted the evolution of the quartiles over the event and it turns out that the 25th percentile and 75th percentile also improve over the course of the event. Seeing as penalties remained basically constant I was expecting to see the bottom quartile stagnate -- instead they got twice as good! I'm going to look at all the data before opening my big fat mouth in the future.

(I'll clean up the plots and post it in a few days. I'm never sure if I love or hate Excel, because it usually has a function to do what I need that is fairly idiot-proof, but the plotting tools always assume what I don't want when trying to set up the graphs.)