[FAF] - June 28, 2013 - Game Design Committee

[rsisk]

Frank Answers Fridays - Game Design Committee

Frank Answers Fridays: June 28, 2013

Blog Date:
Friday, June 28, 2013 - 10:17
This Frank Answers Fridays feature has been getting positive reviews, which is great. Unfortunately many of the positive comments have been directed toward me personally, when it was actually one of our fine FRC Staffers, Rose Kue, who suggested the concept, came up with the name, and even worked with the FIRST Marketing department to develop the snappy new banner. Her suggestion fit so well with our new approach to communication, my answer was an immediate ‘Yes!’. This is yet another example of the great ideas coming from the FRC Staff. We’ll work to keep the new ideas coming – not all will be hits, but even the failures will teach us lessons that will help to move us forward.

Today’s question comes from Eric Browning, a mentor from Team 3289 in Utah.

****

Question:

Hello from Team 3289.

Since FRC 2011 FIRST has started outrageously scored alternate events like the pole climbing minibots in FRC2011 and the pyramids this year. Smaller teams usually only have time to focus on the main event which has been continually diluted by these side activities. Go onto YouTube and you'll find wheelless robots that were built for climbing only because the reward is so much more it's not worth their time to gather and shoot frisbees (which was much harder than climbing). If the side events score is decreased, maybe we can get back to all-around robots vs. specialized "point bots". Will FIRST see the light on this problem?

Do not answer unless you are going to print my entire question, no editing please.

Answer:

Eric, thanks for the question. This gives me the opportunity to talk a little about one important aspect of our game design process.

In FRC, we have a mix of teams with widely varying resources and experience playing on the same field in the same matches. This creates a challenge for game design. Our mission to inspire students to become science and technology leaders requires us to deliver ‘just right’ challenges to every team. A just right challenge stretches you past your current limits so you can grow and earn – with your hard work – a legitimate sense of accomplishment and a more expansive view of your true potential. Too much of a stretch, though, and the challenge becomes discouraging rather than inspiring. How can one game provide the right experience to our wide variety of teams?

On the Game Design Committee, we solve this problem by attempting to create games that have a low floor but a high ceiling. In other words, we try to build in tasks that very basic robots can support, along with tasks that require more sophisticated designs. We want simpler robots to be able to contribute in a substantive way to the match, while still challenging even our most experienced and well-resourced teams to the point where no team feels they have mastered the game and have nothing more they could have done with their design. Since every team is different, we present teams with a palette of game options, from which they can assemble their own just right challenge, considering their resources and intended game strategy. Whether a team’s limitations are driven by experience, technical knowledge, or other resources, the question as it relates to the challenge is the same – what’s the most you can do with what you have?

We don’t always get the mix of tasks and their values right in the game, of course, but in the post-event surveys we sent to teams for 2013, with 2,289 respondents, 78% said the game challenge was ‘Just Right’. (2 individuals – 0.09% - said it was ‘Far Too Easy’. Maybe they misread the question?) Also, last night I was at Dean’s home talking to John V-Neun, a mentor from Team 148, the Robowranglers, a team with many years - and trophies - under their belts. John said he found this year’s game very challenging. It sounds like even the Robowranglers had to stretch to meet the challenges of the 2013 game.

I expect that some individuals will disagree with the approach we’re taking, and I won’t say this way of doing things is set in stone, but my hope is that everyone can understand the reasoning behind it.

[Sidebar: John V-Neun, along with Karthik Kanagasabapathy and Charlie Wensel, were invited to Dean’s home last night as representatives from IFI at this week’s Supplier Summit. Every year, a team of FIRST staffers, led by Kate Pilotte, Kit of Parts Manager, puts on a Supplier Summit here in New Hampshire that starts with a lobster dinner at Dean’s home and ends late the next day after a series of meetings and workshops. This gives us a chance to thank our many generous suppliers and get ideas to enhance our Kit of Parts for the following season.]

I’ll blog again soon.

Frank

[Madison]

I think it's interesting to see the different types of language people use to describe FRC games, events, etc.

Here, the original question uses the phrase "main event" to describe a single portion of the game. I wonder what series of events led him to believe that the frisbees were the 'main event'. Further, I wonder why he didn't realize that, for most teams at most events, the most straightforward path to victory was to excel in that 'main event'.

I think this points out that, despite the inference made in the question, the problem isn't with the game design, necessarily, but with how it is communicated to teams.

[Chris is me]

Originally Posted by Madison

I think it's interesting to see the different types of language people use to describe FRC games, events, etc.

Here, the original question uses the phrase "main event" to describe a single portion of the game. I wonder what series of events led him to believe that the frisbees were the 'main event'. Further, I wonder why he didn't realize that, for most teams at most events, the most straightforward path to victory was to excel in that 'main event'.

I think this points out that, despite the inference made in the question, the problem isn't with the game design, necessarily, but with how it is communicated to teams.

My thoughts exactly. I left a comment on the blog (exactly 600 characters) attempting to express this thought, but this post does a better job than I could. It seems like this mentor is frustrated about his team's performance and is lashing out at the game design for causing that. Yet, I bet if you counted every regional winner this year, less than 1/4 of them would have the ability to climb past 10 points, and less than half of them could floor load.

There is no "main event". There are a series of tasks you can do - you pick the ones that have the best effort to reward ratio for your team's resources and then excel at them. That's the formula to win, as long as you're good enough at whatever you choose to do. Choosing more tasks than you can handle is a common mistake that leads to a lot of overcomplicated robots and unfortunate seasons.

Perhaps FIRST could communicate better to teams that *no task is required*, but I personally didn't think it was at all ambiguous...

Honestly, the 2012 and 2013 games were extremely balanced. 2011 is a different story, obviously, but it's clear that the GDC learned a lot from that year.

[Pault]

Originally Posted by Chris is me

It seems like this mentor is frustrated about his team's performance and is lashing out at the game design for causing that.

I did a little research. The only regional 3289 attended was the Utah regional. They were the #3 alliance captain. They were defeated in the semi-finals by the #2 alliance. This video shows 1891 on the #2 alliance make climbing look easy (literally). This combined with the fact that a disproportional number of robot reveal videos on YouTube are 30 point climbers could create the illusion that the asker is believing. Also, there is only 1 member of Chief Delphi who lists 3289 as their team, and he/she has only 1 post. So although CD isn't the sole medium of communication between teams, its very likely that the asker has very few connections to teams outside his area.

I don't see any problem with the GDC's communication of how the game should be played: this year was the first year where the vast majority of teams aimed for a strategy that was within their means and executed well. The complaints in this question were caused because a person was put into an unusual set of circumstances where so many things that he encountered led him to believe that climbing was both easy and unreasonably valuable; he could not see the full picture. And there is not much we can do about it; if teams choose to not use chief delphi or some other way of interacting with teams from around the world, we can't force them to change that. All we can do is keep trying to encourage it.

[Siri]

Originally Posted by Madison

...Further, I wonder why he didn't realize that, for most teams at most events, the most straightforward path to victory was to excel in that 'main event'...

Exactly. Like, you know, on Einstein.

It does seem to be very much a communication issue, with communication being very much 2-party (albeit not so much bilateral). Personally, I think it's mostly an interpretation issue, but I'll yield to the general case. There were quite a few teams this year that felt better to "...buil[d] for climbing only because the reward is so much more it's not worth their time to gather and shoot frisbees (which was much harder than climbing)" [all emphasis mine], and there were quite a few teams that neglected the minibot in 2011. I wouldn't blame the GDC for this, though: my question is how we as the community can better prepare each other (pre-season) to assess how future games will actually play out.

[GaryVoshol]

I'm guessing the question defined "main event" as being the part of the game in between autonomous and the end game. While this year's end game didn't have a specified time limit, it still was an end game because you had to be on the pyramid at the end of the game.

[Caleb Sykes]

Sometimes, when looking at a complaint about scoring values, it is useful to look at the team's context. Team 3289 competed at the Utah regional, they went 10-4 overall. The lost one qual match to the Hawaiian Kids (enough said, you guys are awesome). In their other 3 losing matches, they played against 1891, the best high-level climber there.

Believe me, I know how it feels to lose matches because there are other robots that can do things that your robot cannot, it stinks. Afterward, all that I think is what ifs.
What if we had spent our time designing a minibot deployment system.
What if we had had different wheels so that we could balance more easily.
What if we had built a climber instead of a floor pick-up device.

This is the one statement that I have trouble with:

If the side events score is decreased, maybe we can get back to all-around robots vs. specialized "point bots".

Personally, I would much rather see 10 FCSs, 10 ground pick-up bots, 10 3rd-level climbers, 10 cyclers, and 10 defenders that are all specialized than 50 all-around so-so robots when I go to a competition. Then, when we all play, it will become apparent what the strongest strategy is.

Engineering is optimizing and trade-offs. FRC is the same. Choose the best strategy that your team can manage, then optimize until you run out of ideas, brainstorm some more, and optimize again. If the GDC wanted, they could make games where only one thing happens, and we would all become excellent at optimizing. But they don't, because they want us to make trade-offs.

I hope the best for you guys, 3289. You appear to have done well this year. My one suggestion would be to spend 1 more day in the early build season analyzing the game and determining trade-offs.

EDIT: Pault added in 3289's context, beat me to it

[Mr. Lim]

This year's game was REALLY hard. Arguably the hardest FRC game I've ever been involved with, dating back to 1998.

It would be too easy to tell this mentor "Hey! None of the 3 World Championship winning teams could climb past 10 pts. They were dedicated Main Event-ers! They were pretty darned successful!"

The irony is, because the game was so hard, it actually worked to the advantage of the non-elite teams. If they had the cajones to pick just one thing to do well, and spent the entire build season dedicated to doing that one thing at a World Class level, they had a shot at winning big this year. I saw a lot of teams in Ontario make huge strides with their on-field performance, because they took this philosophy.

Even the best of the elite teams struggled to "do everything" and do it really well this year. There's a long list of historically great FRC teams who didn't have as much success this year because they built robots that tried to do too much, and ended up doing everything poorly. In fact, the two robots who put up the single biggest individual scores in matches this year (469 and 1310) notably couldn't climb past 10 pts. I don't recall any robot that even came close in individual robot scoring (~140 points in a single match) that could 30 point climb.

I actually think the game needs to be "too hard" to give teams like 3289 the best chance at winning. To take advantage of this, it means teams have to bite the bullet, and focus on building robots that do one or two things really really well, then lean on great strategy and scouting to piece a winning alliance together.

It probably also means the end of an era of dominant robots that can "do everything," but I too think this is probably a good thing...

[rachelholladay]

GDC, don't get self-conscious and feel bad. We like you. You've made two great, fun, exciting games. Keep 'em coming.

[nicholsjj]

Part of this should make people realize how hard the GDC's job is when they design a game. Point Evaluation is such a difficult requirement for any GDC due to them not being able to see what the teams and robots are capable of. The person who asked the question asked why the climb value was so high, but I saw many cimbs either removed or not picked due to their respective value compared to the overall field. I would wish I could PM this guy to tell him that my personal favorite climber this year did not get picked at champs(1421).

Back to the overall game design for FIRST. I also believe that providing multiple strategic options for teams to focus on in the game is very important. I believe teams would be very scared if they were forced to compete in one specific task against teams such as 67,111,254,469, and 1114( ). Teams need variability in order to be competitive(IMO). The GDC did a fantastic job this year in particular for providing a game with so many multiple ways to become a valuable alliance partner. Just look at the Division Elims(as well as IRI next month) to get a true sense of how many different strategies were used.

The one item I didn't like about this year's game was the cost to build a "close" practice field. To be either a good FCS or pyramid climber teams would usually have to shell out around quite a bit of cash. This isn't a bad thing, but I would personally like a game with an end game field structure that is cheaper to build like 2011 instead of 2012/13.

[ttldomination]

Originally Posted by nicholsjj

Back to the overall game design for FIRST. I also believe that providing multiple strategic options for teams to focus on in the game is very important. I believe teams would be very scared if they were forced to compete in one specific task against teams such as 67,111,254,469, and 1114( ). Teams need variability in order to be competitive(IMO). The GDC did a fantastic job this year in particular for providing a game with so many multiple ways to become a valuable alliance partner. Just look at the Division Elims(as well as IRI next month) to get a true sense of how many different strategies were used.

Now, I'm playing devil's advocate, but I also understand where OP (of the question) is coming from.

Let's say that we took this year's game and removed the climbing portion entirely. Do you believe that there would be much difference in the abilities of the top tier machines? Even a difference between the mid-tier and top-tier teams?

If anything, the strategic *variety* plays to the strengths of resource-heavy teams that can allocate resources to accommodate a majority of strategies.

However, rather than attempt to justify this disparity, I choose to embrace it. I know that my students will play in scenarios in which they are outgunned by teams who didn't have to "stretch" as much (or so it seems), and it's a scenario that my students and I have grown to expect and to learn from.

Originally Posted by nicholsjj

The one item I didn't like about this year's game was the cost to build a "close" practice field. To be either a good FCS or pyramid climber teams would usually have to shell out around quite a bit of cash. This isn't a bad thing, but I would personally like a game with an end game field structure that is cheaper to build like 2011 instead of 2012/13.

+1 here.

- Sunny G.

[cmrnpizzo14]

Originally Posted by nicholsjj

The one item I didn't like about this year's game was the cost to build a "close" practice field. To be either a good FCS or pyramid climber teams would usually have to shell out around quite a bit of cash. This isn't a bad thing, but I would personally like a game with an end game field structure that is cheaper to build like 2011 instead of 2012/13.

Why do you need a great practice field to be a good FCS? As Paul said above, we were a good FCS (not to toot our own horn) and all we built was a feeder slot and the 3pt goal. We later built a 2pt goal after the first regional but I wouldn't say we spent more than $50 constructing the necessary field elements this year.

[nicholsjj]

Originally Posted by cmrnpizzo14

Why do you need a great practice field to be a good FCS? As Paul said above, we were a good FCS (not to toot our own horn) and all we built was a feeder slot and the 3pt goal. We later built a 2pt goal after the first regional but I wouldn't say we spent more than $50 constructing the necessary field elements this year.

How much did you pay for Frisbees? I was looking more at the cost to build a pyramid for the price of the field elements. I also think it hurts our team not having a gym in order to get the 54 foot length, but thats usually not a big deal overall for most teams.

[cmrnpizzo14]

Originally Posted by nicholsjj

How much did you pay for Frisbees? I was looking more at the cost to build a pyramid for the price of the field elements. I also think it hurts our team not having a gym in order to get the 54 foot length, but thats usually not a big deal overall for most teams.

You need 50+ feet to get realistic testing or practice as an FCS.

We only had 6 frisbees this year so I think we spent $30 then? I don't remember the pricing on them.

We did not have a gym either, we used the school's cafeteria and simply taped out a field. it wasn't perfect, but it was close enough.

[z_beeblebrox]

Originally Posted by cmrnpizzo14

Why do you need a great practice field to be a good FCS? As Paul said above, we were a good FCS (not to toot our own horn) and all we built was a feeder slot and the 3pt goal. We later built a 2pt goal after the first regional but I wouldn't say we spent more than $50 constructing the necessary field elements this year.

You need 50+ feet to get realistic testing or practice as an FCS.