Average score per match and cycle times

[Citrus Dad]

Originally Posted by Mike Schreiber

How did you come to these predictions? Some sort of math or model? "Engineering Judgement"?

Math and observations. We were counting the number of cycles and so we could see how many were happening for different levels of performance. In 2013 some teams got up to 7 and 8 cycles in Champs! So we see cycles of 20-30 seconds in most games.

[dardeshna]

Originally Posted by Andrew Schreiber

I typically worry less about predicting absolute scores and prefer to focus on analyzing the impact that doing actions at different rates has on the outcomes. For example - 2014 if you could pass about as fast as you could acquire a ball there was a distinct advantage but if it took you a lot longer there was less advantage. By finding how long you CAN take to do things before they become less valuable you can drive your strategic design.

Another good example is when you have multiple goals with differing point values (such as 2013 or 2016, but I only have the 2013 model built) Obviously hitting the 2 point goals was easier so your accuracy was increased, but we needed to figure out how much different the accuracy had to be to make up the point difference. This is another case where you're not looking exactly for a raw cycle speed but instead looking at points where the plot of scores reaches a local maximum.

I've recently started using an online tool called guesstimate for building these models. It's reasonably easy. Here are links to 2013, 2014, and 2015. Mind you, these are not really complete models, they were built to tell me what I needed to know about the game based on our discussions. Other folks may have different needs.

How did you determine the different distributions and timing of various actions? Other folks have mentioned having humans simulate the action, so just curious.

Another thought that someone on our team brought up was the idea of numbers in the game manual hinting at how many cycles would be done per match. For example, a capture requires (required) 8 goals, which might point to an average of 3 high goals per bot (which is about where we ended up at least).

[Andrew Schreiber]

Originally Posted by dardeshna

How did you determine the different distributions and timing of various actions? Other folks have mentioned having humans simulate the action, so just curious.

Another thought that someone on our team brought up was the idea of numbers in the game manual hinting at how many cycles would be done per match. For example, a capture requires (required) 8 goals, which might point to an average of 3 high goals per bot (which is about where we ended up at least).

Gut feel and experience? Obviously for things like travel times you can compute that.

Distributions - Again, gut feel. I really alternate between 3. Normal in which I'm specifying the 95th percentile edges and the average is between them. Long tail in which I assume it'll take about the minimum number but could have a longer tail out to the max. And equal where it's just an equal distribution between all possible values (I don't use this one much for obvious reasons, though if I wanted to have a non-constant lift time for totes in 2015 I could have done that)

But, as I said, I don't so much use them to compute scores I focus more on interactions between actions and outcomes.

Edit: Numbers in manuals are based on what the GDC thinks things should be worth. They may be based on their simulations but, let's be honest, this is a group that didn't see the rain of frisbees at the end of the match in 2013... so, uh, use with caution.

[dardeshna]

Originally Posted by Andrew Schreiber

Gut feel and experience? Obviously for things like travel times you can compute that.

Distributions - Again, gut feel. I really alternate between 3. Normal in which I'm specifying the 95th percentile edges and the average is between them. Long tail in which I assume it'll take about the minimum number but could have a longer tail out to the max. And equal where it's just an equal distribution between all possible values (I don't use this one much for obvious reasons, though if I wanted to have a non-constant lift time for totes in 2015 I could have done that)

But, as I said, I don't so much use them to compute scores I focus more on interactions between actions and outcomes.

Ah okay that makes sense. I like your idea of looking at when a higher scoring option becomes less effective. Could definitely see how that could have been very useful this year with high/low goals.

Now I think about it from that perspective, our off season robot that was a low goaler was significantly quicker in cycles than our high goaler build season robot, so it did end up doing better.

Edit: Numbers in manuals are based on what the GDC thinks things should be worth. They may be based on their simulations but, let's be honest, this is a group that didn't see the rain of frisbees at the end of the match in 2013... so, uh, use with caution.

Rain of frisbees at the end of the match in 2013? Before my time

[Andrew Schreiber]

Originally Posted by dardeshna

Ah okay that makes sense. I like your idea of looking at when a higher scoring option becomes less effective. Could definitely see how that could have been very useful this year with high/low goals.

Now I think about it from that perspective, our off season robot that was a low goaler was significantly quicker in cycles than our high goaler build season robot, so it did end up doing better.



Rain of frisbees at the end of the match in 2013? Before my time

So, it's not all about less effective. Sometimes it's finding things that give you more bang for your time (2014 w/ passes) or things that have more room for delays because you have to do things fewer times (2015 w/ cans) or even things more tolerance for screwing up.

The way I try to evaluate games is look at the inputs I can control (how fast I do things, accuracy, which scoring methods, etc) and what outputs exist (usually raw score but sometimes other things like limited cans in 2015). Then it's all about finding how to change inputs to get the outputs I want.

EDIT: Oh gosh, I keep forgetting that that 2013 was a "long time ago"

[Dylannn]

Originally Posted by dardeshna

Generally, at the end of the season, we look back on these estimated values and laugh, knowing that they were totally unrealistic.

In my two seasons of FRC, I have definitely seen this to be true on my team But I think it is definitely important to set these high goals so that the team is always striving for more during the season.

[dardeshna]

Originally Posted by Dylannn

In my two seasons of FRC, I have definitely seen this to be true on my team But I think it is definitely important to set these high goals so that the team is always striving for more during the season.

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

[Rangel(kf7fdb)]

Originally Posted by dardeshna

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

It all comes down to understanding your team and what you guys are capable of. The general rule that many successful teams follow is to be the best they can at one particular task instead of the master of none. There are and will always be teams that can do it all but even so, being really good at one particular task can make your team competitive enough to go head to head with these teams. The 2013 world championship alliance is a great example of this design strategy.

[RoboChair]

Originally Posted by dardeshna

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

The real trick to that is to be able to do the minimum everything required to win a match assuming similar opponents. Then on top of that be really good in a defined role for the game being played.

Look at our 2015 robot, a lot of teams made fun of the fact we could only do stacks of 5. But we had our can grabbers and those were our focus, that was our high end role, beat the other guy to the cans every time without fail.(admittedly maybe not the best example, but still)

Don't rely on everyone all the time, but don't do everything all the time either.

[Lil' Lavery]

Originally Posted by dardeshna

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

Prioritize what your robot needs to be able to accomplish, and then stick to that priority list as best as possible when designing and fabricating your robot. That's not to say you can't also grab some "low hanging fruit" from your lower priority items, but try to avoid designing around lower priority items. Don't compromise the tasks you NEED to be good at in favor of adding in additional low priority functionality.

[Dylannn]

Originally Posted by dardeshna

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

On 3481, after analyzing the game we talk about setting realistic goals based on what we know we can actually accomplish. This year we challenged ourselves to be a high goal shooter. After perfecting the shooter we just kept trying to increase the amount of high goals we could score per match. So I guess it is important to specialize in one aspect of the game and work on it until you perfect it, and always try to better your bot.

[BrendanB]

Originally Posted by dardeshna

I guess that begs another question, how do teams avoid falling into the "carry" strategy trap and having their robot attempt to do everything, ending up with a situation where it does nothing well? Obviously there are some teams that can pull this off but not everyone for sure.

Always prioritize tasks and stick to them. You can re-prioritize as you go along depending on how your strategies changes or any major updates that would shift your team, but you should always have a unified top priority typically starting from basic game play up.

It is important to step back from the chaos of the season and look at the bigger picture and do so without being emotionally attached to a strategy or concept. Setting goals like being the best at high goal shooting is a focused strategy, but you have to remember all of the other sub-systems you need to have perfected if you can be the best at high goal shooting. If your drivebase is just "okay" chances are you'll struggle at crossing defenses or navigating around the field. If you focused on making a simple collector you'll have trouble acquiring game pieces which will reduce the number of shots you can take. If your shooter controls have too many variables your shots will be inconsistent. If you are working up until the end of build season without time to test and debug your drivers will struggle on the field learning how to use your robot.

Many of these lessons I've learned over the years from mistakes we've made by not keeping ourselves prioritized and lacking a cohesive effort to field the best robot WE can as a team. Many times fielding your best robot means it won't be the best robot on the field. This isn't a bad thing. If everyone fielded the best robot they could at their events and kept improving their machine FRC would be more fun & exciting. Always field YOUR best machine and never stop improving.

We had to scale back our efforts drastically in Week 4 of build season so we could compete at our first event with a completed machine. It was a little deflating knowing younger teams in our area were doing more than us but we perfected what we had and never stopped improving it. What we started our season with was far from what we ended our season with.