Crazy Idea, looking for comments

Posted by Joe Johnson, Engineer on team #47, Chief Delphi, from Pontiac Central High School and Delphi Automotive Systems.

Posted on 5/4/99 7:58 PM MST

I posted this message while back, but got no takers. Thought perhaps folks might have missed it.

Here is another chance.

What do you think about the idea of having 3 team alliances always (specifically, during the Qualifying Matches)?

Each QM would have 6 team ‘on deck.’ Shortly before the match, the 6 teams would be divided into 2 groups of 3 and two ‘alliance captain’ teams would be randomly selected.

In this way, teams would have practice for the finals in negotiating not only the strategy but the robot choices as well.

In addition to this, it would help to even up 2 on 1 problem created by non-functional or barely functional robots (the chance of 2 dead robots on one alliance is much less – therefore there would be fewer 2 on 1 QM’s).

It could cause a problem or two also, not the least of which is that a team may NEVER have their robot play a match, but perhaps this could be worked out.

Thoughts?

Joe J.

Posted by Mike Kulibaba, Student on team #88, TJ², from Bridgewater-Raynham Regional and Johnson and Johnson.

Posted on 5/4/99 8:13 PM MST

In Reply to: Crazy Idea, looking for comments posted by Joe Johnson on 5/4/99 7:58 PM MST:

Joe,

I’d go for the idea as long as we make sure one team doesn’t sit for all there matches, And another thing, if my team had to sit down for a match i might be a little upset if I know that we can do a better job, I think there might be alot of arguments to who plays if there are 3 good robots. But hey I’d be willing to give it a try.

Kuli TJ² Team 88

Posted by P.J. Baker, Engineer on team #177, Bobcat Robotics, from South Windsor High School and International Fuel Cells.

Posted on 5/4/99 8:31 PM MST

In Reply to: Crazy Idea, looking for comments posted by Joe Johnson on 5/4/99 7:58 PM MST:

I like the idea of avoiding 2 on 1’s, but I agree with Mike that there might be arguments about who plays. How about this crazy idea instead. Keep the seeding based on average q points, and allow teams to volunteer to play extra matches. SOme system would have to be worked out to organize this but I think it could work. If you can not at least make your bot move when it’s time for your match, you get 0 QP’s and go back to the pits. Which ever volunteer bot is on deck then gets thrown in and has an extra match going towards their QP average.

I haven’t put much thought at all into this idea (in fact I think it might be stolen from another post a while back), so I’ll stop here and see if anybody wants to run with it.

P.J.

Posted by Daniel, Student on team #192, Gunn Robotics Team, from Henry M Gunn Senior High School and NASA Ames.

Posted on 5/4/99 9:15 PM MST

In Reply to: Crazy Idea, looking for comments posted by Joe Johnson on 5/4/99 7:58 PM MST:

I know I’ve said this before, but sit tight cuz I’m gonna say it again.

It is a well known statistical fact that luck is eliminated with increased trials. We need increased trials!

If one 540 match can put a lucky robot into the select sixteen, and similarly one 8 point round can pull a select sixteen team from that coveted position, there is obviously a problem. You have all acknowledged this in your posts. The solution is to increase the trials!

How do we fix it? More arenas. More time.

I saw someone suggest an extra day. This costs more, but it may not be such a bad tradeoff. We all want more matches. Don’t you think those 80+ hour all-nighters would be better requited with more than six matches? Seriously, we put our lives into this thing.

More arenas would mean a big headache for FIRST and less time between matches. Unless another day is added. Once again, it may be worth the costs.

Personally, I think adding a third alliance partner is merely a way to avoid the issue. It works to a point, but it reduces the amount of matches some team gets to play. In essence, it makes the situation worse. No mater how well a teams bot performs, they deserve some play time. We all deserve to play. Remember what FIRST stands for…

And as for practicing for the finals, that also seems a little contrived. It’s human nature to try to find reasons to support our opinions so I don’t blame you, but it didn’t really seem like an issue to me. It’s not something you need too much practice for. I know my team didn’t have a problem with it.

Just some thoughts from the guy who’s got too many…
-Daniel

Posted by Dan, Other on team #247, da Bears, from Berkley High and PICO/Wisne Design.

Posted on 5/5/99 3:56 AM MST

In Reply to: Or we could try the easy way… posted by Daniel on 5/4/99 9:15 PM MST:

I agree that increased trails would be one way to solve the problem.
However, I think the way q-points are calculated is the problem. I say
improve the ranking system and six rounds would be sufficient.

Flaws that I see with the system:
It favors offensive, high scoring rounds only.
It discriminates against good defensive rounds which turn out to
be important in the elimination tourney.
As a result, sometimes losing outweighs winning.

I have been trying to come up with some ideas for a better ranking system.
If I get something concrete to post, I will.

Dan
#247 da Bears

Posted by Andrew Trax, Coach on team #180, S.P.A.M., from Southfork,Martin Co. High and UTC.

Posted on 5/5/99 4:25 AM MST

In Reply to: Re: Or we could try the easy way… posted by Dan on 5/5/99 3:56 AM MST:

Simplest way to simplify is to only rank by wins. One win - one point.
Point totals in a round could remain as is. But for ranking, count
the victories. Then a low-scoring, but well-fought win would be
weighted the same as a high-scoring, stroll-in-the-park against a weaker
opponent.
The teams with the most wins go to finals. Say, 5 out of 6 to qualify.
Simple, easy, fair.

Mrs. Trax

Posted by Daniel, Student on team #192, Gunn Robotics Team, from Henry M Gunn Senior High School and NASA Ames.

Posted on 5/5/99 7:58 AM MST

In Reply to: Re: Or we could try the easy way… posted by Andrew Trax on 5/5/99 4:25 AM MST:

Exactly. If ranking was done as seeding was last year, wins would mean a lot more.

Average QPs can be the tie breaker.

To point out the difference, my team was 4 of 4 last year and we ended up seeding at 6th place. This year we were 6 of 6. Although you’d naturally think that would be even better, we ended up ranking in the mid-30s.

Funny that.

A win-based ranking system would reward both defensive and offensive strategies, and would hold both at similar values as in the finals. It makes too much sence!

Still, this would bring robots that lose a match due to bad luck way down in seeding. In either case, more matches are needed.

My position still stands =)

-Daniel

Posted by Andrew Trax, Coach on team #180, S.P.A.M., from Southfork,Martin Co. High and UTC.

Posted on 5/5/99 5:21 PM MST

In Reply to: Do it like seeding last year… posted by Daniel on 5/5/99 7:58 AM MST:

: Exactly. If ranking was done as seeding was last year, wins would mean a lot more.

: Average QPs can be the tie breaker.

: To point out the difference, my team was 4 of 4 last year and we ended up seeding at 6th place. This year we were 6 of 6. Although you’d naturally think that would be even better, we ended up ranking in the mid-30s.

: Funny that.

: A win-based ranking system would reward both defensive and offensive strategies, and would hold both at similar values as in the finals. It makes too much sence!

: Still, this would bring robots that lose a match due to bad luck way down in seeding. In either case, more matches are needed.

: My position still stands =)

:
: -Daniel

But the bad luck you’d run into in this ranking system would be
mechanical malfunctions, human player falls off Space Mountain,
your bus driver gets lost and you end up in Pahokee. Stuff you
can’t blame on any other team. Sweet, huh?

Mrs. Trax

Posted by P.J. Baker, Engineer on team #177, Bobcat Robotics, from South Windsor High School and International Fuel Cells.

Posted on 5/5/99 10:42 AM MST

In Reply to: Or we could try the easy way… posted by Daniel on 5/4/99 9:15 PM MST:

: It is a well known statistical fact that luck is eliminated with increased trials. We need increased trials!

 I agree, increased trials would reduce the luck factor.  But there is another way (I think).  At one point I thought FIRST was actually doing this, but I changed my mind.  Randomly break up the teams into 8 groups (this year there would have been 26 teams per group.  Play a round robinish tournament with the smaller group.  The top two teams from each group comprise the 'sweet 16' and then you conduct the draft from there.  As long as there is some minimum # of Q mathces per team (let's say at least 4), luck is actually a function of the ratio of the number of matches played to the number of teams in your group.  With 6 matches and 208 teams, luck is a much bigger factor than 6 matches ad 26 teams.

: If one 540 match can put a lucky robot into the select sixteen, and similarly one 8 point round can pull a select sixteen team from that coveted position, there is obviously a problem. You have all acknowledged this in your posts. The solution is to increase the trials!

 I'm still not sure that this is true.  Give a 'lucky' team one 540 point win and 5, 100 point losses (a very good losing score).  Their average QP is 353, decent but below this year's top 16 (We were 14th with about a 460 average).  

There has been a lot of discussion about defense not being rewarded in the seeding rounds. This is obviously true, but … FIRST gave us all the same set of rules on day one. It was obvious then that they wanted to reward scoring in matches. This makes sense. People want to see high scores, it’s more fun. Defense is exciting in the elimination rounds, because winning is so much more important. Think about pro hockey. It’s hard to get people to watch because of the low scoring. In the post season though, there is nothing more exciting (in all of pro sports, IMHO) than sudden death in the Stanley Cup playoffs. I don’t think FIRST screwed up this year by placing so much emphasis on scoring in the Q rounds. What I feel they did screw up was in placing so much importance on scoring for a game where it was very difficult to tell the score until after it is all over. They paired a game that is more exciting to watch when played defensively with a system that rewarded offense. Rather than lobbying FIRST to do one thing or the other with the scoring, I think that we should be asking them to make sure that there is a good fit between whatever the game is and whatever the seeding formula is.

Any Thoughts?

P.J.

Posted by Chris, Coach on team #308, Walled Lake Monster, from Walled Lake Schools and TRW Automotive Electronics.

Posted on 5/5/99 12:50 PM MST

In Reply to: Re: Or we could try the easy way… posted by P.J. Baker on 5/5/99 10:42 AM MST:

: I agree, increased trials would reduce the luck factor. But there is another way (I think). At one point I thought FIRST was actually doing this, but I changed my mind. Randomly break up the teams into 8 groups (this year there would have been 26 teams per group. Play a round robinish tournament with the smaller group. The top two teams from each group comprise the ‘sweet 16’ and then you conduct the draft from there. As long as there is some minimum # of Q mathces per team (let’s say at least 4), luck is actually a function of the ratio of the number of matches played to the number of teams in your group. With 6 matches and 208 teams, luck is a much bigger factor than 6 matches ad 26 teams.

:
: Any Thoughts?

: P.J.

I think the luck factor would just shift to a different area. The luck factor would be in the strength of the group of 26 that you’re in. A decent team could get lucky and be put in with a group of lesser robots and make it to the dance while the same team might get put in a group with a bunch of good teams and finish poorly. It’s a lot like (oh no, another sports reference) Major League Baseball a few years ago when 4 teams in the American League East had better records than the champion of the American League West. The A.L West champion made it to the playoffs and only one of the 4 better teams in the A.L. East made it. That system isn’t necessarily fair either.

-Chris

Posted by P.J. Baker, Engineer on team #177, Bobcat Robotics, from South Windsor High School and International Fuel Cells.

Posted on 5/5/99 2:11 PM MST

In Reply to: Re: Or we could try the easy way… posted by Chris on 5/5/99 12:50 PM MST:

: I think the luck factor would just shift to a different area. The luck factor would be in the strength of the group of 26 that you’re in. A decent team could get lucky and be put in with a group of lesser robots and make it to the dance while the same team might get put in a group with a bunch of good teams and finish poorly. It’s a lot like (oh no, another sports reference) Major League Baseball a few years ago when 4 teams in the American League East had better records than the champion of the American League West. The A.L West champion made it to the playoffs and only one of the 4 better teams in the A.L. East made it. That system isn’t necessarily fair either.

: -Chris

This is a good point. I have a feeling that my way does reduce the luck involved, but since I haven’t done the math, let’s assume I’m wrong. All we need now is a way to reduce the “luck” involved in the assignment of teams to the small groups. Guess what, we have one! The regional tournaments. Take the seeding results form all of the regionals and divide them into quartiles (the top 25% from each regional are lumped together, the second 25% from each, etc.). Then fill each of the small groups for the national tournament like this:

Randomly assign the top quartile teams, then the second quartile, then the third quartile, and then the fourth quartile.

For teams that go to more than 1 regional, their average quartile rank would be used, rounding up for .5’s and higher, down for below .5 (1.33 goes to 1st quartile, 1.5 goes to 2nd quartile)

Certainly there is some luck involved in where you seed in regionals, but it is less so. There is also an argument that there will be some regionals with much stronger fields than others, but I think this goes further towards reducing the luck factor. We can not eliminate it, nor would I really want to. But there are ways to minimize it to the point where most people feel there is a direct correlation between effort, ability, and results.

P.J.

Posted by Joe Johnson, Engineer on team #47, Chief Delphi, from Pontiac Central High School and Delphi Automotive Systems.

Posted on 5/5/99 8:02 PM MST

In Reply to: Re: Or we could try the easy way… posted by P.J. Baker on 5/5/99 2:11 PM MST:

I disagree that the round robin of smaller groups lessens the luck of the draw, but I think that the idea of using the regional results is a great idea.

I propose that the averages carry over from the regionals. This would help to give us more rounds without adding to the time at the national. The scores from each regional may have to be normalized so that all regionals have the same average QM points in order to account for regional differences in offensive/defensive strategies.

I could go for it. It is not as desireable as having more QM’s but it is perhaps a good compromise between an invitation only Nationals and what we currently have now.

What do others think?

Joe J.

Posted by Tom Vanderslice, Student on team #275, ORHS/AST/Hitachi, from Academy of Science and Technology and Hitachi.

Posted on 5/5/99 10:10 PM MST

In Reply to: Yes, use the regional results… posted by Joe Johnson on 5/5/99 8:02 PM MST:

: I disagree that the round robin of smaller groups lessens the luck of the draw, but I think that the idea of using the regional results is a great idea.

: I propose that the averages carry over from the regionals. This would help to give us more rounds without adding to the time at the national. The scores from each regional may have to be normalized so that all regionals have the same average QM points in order to account for regional differences in offensive/defensive strategies.

: I could go for it. It is not as desireable as having more QM’s but it is perhaps a good compromise between an invitation only Nationals and what we currently have now.

: What do others think?

: Joe J.

Don’t really see this working for a couple reasons…

  1. Some teams don’t/can’t (although i don’t know why you wouldn’t unless
    you couldn’t) go to regionals…you can say oh well…they just get
    6 QM’s…well that leads right into #2… (convienently :wink:

  2. You’d have a lot of teams w/ a lot of different numbers fo QM’s
    and the proverbial ‘lucky/unlucky round’ gets magnified or shrunk (sorry
    i couldn’t think of a smart sounding -ed word to mean shrunk…sue me…)
    depending on your QM’s…at least in this year’s system everyones
    lucky/unlucky rounds counted equally…but you could have one team use
    their skill and score 3 perfect rounds at various regionals…but the
    effect is shrunken down so much by the fact that they had 33 QM’s…but
    the team that gets ‘lucky’ with one perfect score at nationals and those
    are their only 6 QM’s is virtually guaranteed a spot in the finals…
    and you think people complain about it (oh sorry…discuss it) now… :wink:

Tom
Team 275

p.s. ifyou can’t tell i’m slightly cynical about the whole ‘lucky/unlucky’
round thing…before nationals (my team didn’t get to go to a regional…
we’re poor and in houston) i was thinking…wow just gotta get lucky once
and we’re in…after the competition i now know that it takes a lot more
than just getting ‘lucky’ to get a really good score…unlucky rounds are
a little more likely…but still…‘lucky’ perfect scores aren’t gonna come
up too often…and any team who has the ability to get a perfect score
whether they are paired with a box w/ wheels or the #1 seed deserves a spot
in the finals…b/c your robot has to be able to do something special to
get a perfect score…

sorry…i didn’t mean to write all that…i just needed to say it… :slight_smile:

Posted by P.J. Baker, Engineer on team #177, Bobcat Robotics, from South Windsor High School and International Fuel Cells.

Posted on 5/6/99 5:29 AM MST

In Reply to: Yes, use the regional results… posted by Joe Johnson on 5/5/99 8:02 PM MST:

I can’t argue too convincingly about the smaller groups thing. It feels that way in my gut, but I wouldn’t trust my statistical instincts much farther than I could throw a robot.

I want there to be a way to use the regional results, but I think that directly carrying over QM averages to the National tournament would be too much of a penalty for teams that didn’t quite have it together at the 1 regional they attended. Of course, there should be some reward for performing well at regionals, and yes, a penalty for shipping a robot that doesn’t work. How about this: Normalize the results of the regionals and then count a team’s regional average as a pecentage of their seeding rank at Nationals, say 25%-33%. That way, you couldn’t rack up a 500 pt average in 30 regional Q matches and coast into the National finals with a very mediocre performance there, and a sub-par performance at a regional wouldn’t sink you chances at Nationals.

Now, what about the teams that don’t go to any regionals. Personally, I think that this should be discouraged, but I haven’t been around long enough to develop a really strong opinion about it. I see three options: Let them use just their average at Nationals, Give them the Regional average going into the National Tournament, or give them a fraction (3/4) of the regional average going into the national tournament. I’ve no idea which is best …

I think this is being productive, more ideas please!

P.J.

Posted by Daniel, Student on team #192, Gunn Robotics Team, from Henry M Gunn Senior High School and NASA Ames.

Posted on 5/6/99 8:04 AM MST

In Reply to: Yes, use the regional results… posted by Joe Johnson on 5/5/99 8:02 PM MST:

I know it seems like a good thing to use regional results, but as of now, it’s not a consistent measure. Some teams go to many regionals, others go to few or none at all. It’s a great idea and I’m sure FIRST will work something similar into the games of the future, but we’re not ready yet. We won’t be ready until all robots go to the same number of regionals. If you think about it, a team that goes to many regionals doesn’t get affected by a bad round in Florida, whereas a robot in Florida that gets unlucky for a match or two, experiences large effects on their average QP. I’ve always been an advocate of teams going to however many regionals they want, but under those changes, I’d have to move to the other side. You have to always remember there are teams that can’t even afford to go to one regional.

We always need to strive to become MORE fair, not less (and I know it will never be ‘fair’, but it’s always a positive change to move towards that ideal).

I like the idea, but we’re not ready.

The calculations would be messy, I think that’s a sign.

Agree/Disagree?
-Daniel

Posted by P.J. Baker, Engineer on team #177, Bobcat Robotics, from South Windsor High School and International Fuel Cells.

Posted on 5/6/99 11:05 AM MST

In Reply to: Not ready…yet. posted by Daniel on 5/6/99 8:04 AM MST:

I agree. If you read my post below, you’ll see a potential solution to the problem of teams having different #'s of Q matches from the regionals. It goes like this:

Let every teams regional average be weighted the same in the seeding matches at Nationals, regardless of whether they played 6 or 60 regional matches.

Posted by Daniel, Student on team #192, Gunn Robotics Team, from Henry M Gunn Senior High School and NASA Ames.

Posted on 5/6/99 12:04 PM MST

In Reply to: Re: Not ready…yet. posted by P.J. Baker on 5/6/99 11:05 AM MST:

Even so, it’s such an unconstant variable. Some teams don’t even go to any…

I guess I can live with being a skeptic… =)

-Daniel

Posted by Joe Johnson, Engineer on team #47, Chief Delphi, from Pontiac Central High School and Delphi Automotive Systems.

Posted on 5/6/99 7:55 PM MST

In Reply to: Not ready…yet. posted by Daniel on 5/6/99 8:04 AM MST:

Okay, how about these (not always fair) suggestions for making it more fair for teams that go to more or less than 1 regional:

Give every team a starting Qualifying Point (QP) total equal to 6 times their average from all the regionals they attended of their average QP/match for each regional. Teams that go to no regionals get 6 times the average from all the regionals of the average QM/match.

For example, if a team scored 100 QP/Match at Chicago, 300 QP/Match in Philly, and 400 QP/match in Ypsi, then they would start the Nationals with 1200 QP (or 200 QP/Match). NOTE: ((100+200+300)/3)*6 = 1200

Further if
CA average QP/Match = 101,
IL average QP/Match = 99,
FL average QP/Match = 105,
PA average QP/Match = 95,
CT average QP/Match = 150,
NJ average QP/Match = 75,
MI average QP/Match = 75,

Then every team that went to no regionals would get 600 QP at the start of the nationals (or 100 QP/Match). NOTE: ((101+99+105+95+150+75+75)/7)*6 = 600

Assuming 6 QM’s at the Nationals, then a team’s average QP/match for the Nationals would then be weighted of 1/2 from the Qualifying Matches at the Nationals and 1/2 from Qualifying Matches from the Regionals (except for the no regional teams which 1/2 get an unfair lift and 1/2 get an unfair drag, life’s not fair :wink:

Continuing with the example above if the team that went to the 3 regionals was paired with at team that went to no regionals and they won with 300 points, then their new QP/Match would be 300 and 214 respectively. NOTE: (1200+3300)/7 = 300, (600+3300)/7= 214

The actual QP/match would probably have to be normalized themselves to take out differences in scoring from regional to regional, but that is only a minor detail.

By doing this, everyone’s QM’s at the Nationals would be worth the same in terms of how much a particular bad or good match moved a teams QP/match, but some of the information gained from the regional would be incorporated into the National performance.

Itis a bit complicated to explain, but I don’t think that it would be any trouble to implement and I think that it is a way to address some of the concerns folks have bought up.

What do you think?

Joe J.

Posted by Daniel, Student on team #192, Gunn Robotics Team, from Henry M Gunn Senior High School and NASA Ames.

Posted on 5/6/99 8:11 PM MST

In Reply to: some ‘unfair’ modifications to make it ‘fair’ posted by Joe Johnson on 5/6/99 7:55 PM MST:

I’m sorry but I still can’t see what problem you’re trying to solve here. If you’re trying to make the game more fair, I can’t see how that does the job. Teams that don’t go to regionals could get cheated. In fact, it is most likely going to be the rookie teams that don’t go to regionals. Those are exactly the people who we don’t want to feel cheated. We need this thing to grow, not shrink.

Is it maybe something else?

Perhaps I’m just confused…

Lemme know.
-Daniel

Posted by Joe Johnson, Engineer on team #47, Chief Delphi, from Pontiac Central High School and Delphi Automotive Systems.

Posted on 5/6/99 8:52 PM MST

In Reply to: Re: does that really make it more fair? posted by Daniel on 5/6/99 8:11 PM MST:

This is a long metaphor, but stick with me.

Suppose you have 200+ bags of gold and you want rank each bag of gold from heaviest to lightest.

But… You don’t have very good scale: It is just a balance. Further, you must weigh 4 randomly selected bags at a time, 2 on each side of the balance. Even worse, suppose the balance is not necessarily on a level base, but rather the base is randomly tipped for each measurement, adding even more uncertainty to each result.

The only real way to accomplish the task is by making lots of measurement.

It is clear (at least to me) from the Nationals, that 6 measurements per bag of gold is not sufficient to accurately rank the bags of gold. Many light bags are going to be estimated to be heavy simply because they were randomly weighed with 6 heavy bags or because they were weighed against 6 pairs of light bags (or even against one heavy bag and an empty bag!). The same is true for heavy bags that would be estimated to be light simply because of the luck of which bags was on their side of the balance or which bags happened to be on the other side of the teeter-totter. In addition, there is that random tipping of the base to deal with (a broken arm, a bad battery, a blown fuse, etc.).

It isn’t exacty the most efficient means of ranking bags, but it is kind of exciting to watch.

The road to more accurate bag ranking runs right though the the town of More Weighings, but with 200+ bags to rank we don’t have time to even look in that direction yet alone go there.

But… as it turns out, we have already weighed some of these bags against each other at the regionals. Wouldn’t it be nice to be able to use these measurement somehow?

What many folks are proposing (myself included) is that we use some of these measurements, but try to do so in a way that does not totaly take all the excitement and uncertainty out of the important weighings at the Nationals.

As to it being fairer, I can’t say.

I do think that such proposals would be more likely to produce a heavier group of ‘Heaviest 16’ bags of gold than would otherwise be the case.

To me, this seems more fair. To a particular bag of gold that would have been judged in the Heaviest 16 except that it didn’t participate in an earlier weighings, it would not seem fair at all.

While I sympathize with such bags of gold, I think that the improved odds of getting a more rational result is worth it.

My 2 cents.

Joe J.