Michigan Regional System: Who is asking the hard questions around here?

[Katy]

Originally Posted by Lil' Lavery

Rather than a discussion of the specific merits and issues with the michigan structure (which has been discussed ad naseum in other threads), I think Katy's point was to lay out a discussion of what constitutes success.
Regardless of whether or not FIRST puts out an official "criteria for success" of the Michigan district pilot, the community needs to weigh in on what should be considered successful. This is especially imperative since FIRST has not issued such criteria publicly.

Lil'Lavery couldn't have said it better! As a matter of fact as it turns out "Big Lavery" said it better than ever I did or could have! This is a post well-worth reading and if I had known about it before starting this you had better bet I would have quoted it quite a bit. http://www.chiefdelphi.com/forums/sh...d.php?p=761241

Originally Posted by Mike Q.

we competed in traverse city this weekend. we played 12 qualifying matches even though there was quite a few field problems. most of the field problems were communication errors. the competition level was a little down from a standard Michigan regional.

Thank you Mike Q! Can you please go into some detail: do you have any more information about the field problems (the issue, count of number of times it delayed a match, number of times it effected match outcomes) etc. Do you have any other information that answers any other questions that have been posted so far?

As our discussion is continuing it is becoming apparent that we will need data in general to compare Michigan to! It is not fair to compare them to an ideal, we should compare it normal larger regionals. If you have data compiled from other regionals for comparison purposes please post it!

Also we're going to need some way to organize this data probably if we ever do get it in serious volume. A google doc...I am not sure how to make it so anybody can edit it. I guess I could paste in an invite and check "invites can be used by anybody" or something but that is somewhat awkward. Thoughts?

Originally Posted by WEHickok

I have been involved with this since our first meeting in Francois' living room last April or May. Our every move has been monitored by FIRST and the FIRST Board of Directors. This has not been done in a secret as some have stated.

Dude! Does that mean party at Francois' house and we're all invited? I approve! Let's go make some robot policy!

No seriously, I do not think "a meeting in somebody's living room" particularly supports your proposal that this process was "transparent policy making."

Originally Posted by WEHickok

Brandis University (I hope I spelled that correctly) has already created a survey that will be used at the close of the season...asking teams, parents, students, volunteers, etc., for their reaction. This data will be studied.

Can we see this survey? Maybe it can help us create some of our criteria! Will we be allowed to see the raw unprocessed result data?

Originally Posted by WEHickok

So, Please give this a try. Let us see where this goes.

I don't think anybody is "not giving this a try." We just want to evaluate the try scientifically. We're engineers, what do you expect?

All trolling aside, I really appreciate the data you are bringing to the table for this discussion. You have a perspective and access to data that most of us do not. Whatever data that perspective provides for our evaluation we will be quite thankful for.

[RMiller]

This is how I have come to view the goal: more chances to play for a lower cost without lowering quality.
So, first some objective questions: Has this increased the number of times teams play? (I think this is relatively straightforward)
How many teams have had increased (non-robot) costs? How much have they increased? Remember, there may be teams that would only have gone to one regional and now have two sets of travel costs to pay.
Now, the subjective part: has the quality of play decreased?
Has the quality of "production" decreased?
If so for either of these: How so? Which district events? Is it acceptable? Is there ways to improve without raising the cost?

Questions for future consideration/upon expansion: can teams opt out into a different structure (prohibitive travel costs perhaps)? Some areas don't have enough teams to support a structure like this, what are the options for those places?

I want to say that I certainly like the idea. As a mentor on a team that spent under $500 beyond registration (granted we are rookies that started the year late), any chance to increase play time while reducing costs would be appreciated.
Oh, and after spending the weekend at Midwest, if part of any lower quality "production" is a lower volume on music that would be much appreciated as well! Without fail, I come away with awful headaches due to this (a lack of sleep doesn't help either), but I never remember ear protection until later.

Edit: I guess I type too slow as it appears most of this has now been said in one form or another.

[XXShadowXX]

Originally Posted by Mike Q.

we competed in traverse city this weekend. we played 12 qualifying matches even though there was quite a few field problems. most of the field problems were communication errors. the competition level was a little down from a standard Michigan regional.

I must first respond to this comment; In my mind, this was to blame on the new FMS, which seams to be kinda buggy, and inexperience operators(sorry if a operator of the FMS reads this but, really how much experience did the average operator have with the old IFI based system last year). I truly didn't have much hope for a smooth district. In a closing remark when the FMS was running at full tilt the 1.5 hours we were behind was made up by lunch/alliance selection.

[Mike Q.]

at the beginning of the day on friday before the first match there was about an hour delay. That was Definitely the longest delay about every other match or so there would be a 15 second to 1 minute delay. about every delay was because of communication between the robot and the field. by the end of qualifications the had it running good and we were able to get back on Schedule.

Thanks to all of the controls people at traverse city for all there hard work

[XXShadowXX]

Wasn't the initial delay caused by teams not have their firmware upgrade to the required level?

Oh on gratz on the epic win.

[Mike Q.]

I think it was because the teams didn't have there firmware updated. I'm not the most Knowledgeable when it comes to that type of stuff.

[Katy]

Originally Posted by XXShadowXX

Now to the Michigan District System discussion:

That would be in another thread please! This discussion is about how we might evaluate the system and to collect data with which to evaluate it, not its theoretical merits! I find it incredible that anybody is still discussing the theory of such a system when there are hard facts available for the collecting! What does anybody's opinion on something count for against hard experimental data? We don't debate on why gravity should or shouldn't exist, we don't vote on astrophysics, and I'm stumped as to why we should try to debate our way out of this!

I'm not trying to pick on you here, I know other people are doing it, but please knock it off.

Originally Posted by XXShadowXX

Wasn't the initial delay caused by teams not have their firmware upgrade to the required level?

That is very important data! Since we do not have Thursday matches in this system maybe we need to make running the robot on the field as part of the inspection process.

[XXShadowXX]

I should say that i am sorry, i quickly went to say my view, of the delay that was made, and then jumped into a topic that i wasn't prepared to talk about, it will be deleted.



The delay that i know about;
Firmware upgrades; Well teams didn't keep up on the Firmware, some teams have issues with net connectivity, and at regional events were not allowed to broadcast wifi, so i don't think that extra Thursday would help.
Rebooting; The FMS was rebooted after every single match, is this standard procedure?
Communication; A catch all, a team gets disabled and goes starts autonomous code/Two teams press disable at the same time and disable the whole alliance/A team randomly goes in autonomous while the MC is on the field/ect.

[Nawaid Ladak]

my questions

1: to Michigan Teams: Do you guys miss traveling out of state to compete; (this is mainly to those big teams that choose not to travel this year).

2: to teams outside of Michigan: has the competition dropped off a bit compared to last year, where last year Michigan Teams competed?

3: to Michigan Teams: how much has the competition level dropped compared from the regional last year to these division events this year.

4: to teams who are outside of Michigan and traveling to Championships: do you believe that the district system in Michigan gave these teams any advantage/disadvantage over the other teams attending the Championship Event? Do you believe the competition level displayed by the Michigan teams matched your expectations, or was it varied?

Success can be measured in different ways; better competition for everyone is good. this is just a simple matter of this question

is Michigan turning to Chinese ethics (quality < quantity)?

[Alex Golec]

So, let's play the numbers game with this, starting with team growth:

How has Michigan's growth rate changed, relative to prior years and other parts of the US/Canada? (I don't mean to exclude the rest of the world, but the data is too small there) Are more rookie teams forming? Are more veteran teams being retained?

There is also a question of costs:

Can teams use smaller budgets because of cost savings? How much money that used to go to regional fundraising is now going to teams? Was the $1000 reduction in registration a factor in the formation of some rookie teams?

Now a question of publicity:

Is the travel divide between Michigan and non-Michigan teams significantly impacting the quality of given events? (Do recall that the Michigan boundary was established out of convenience of geography) Is the increase in local events helping teams get more press coverage? Is the glamor-reduction at events causing negative media coverage? On the topic of travel: what is the likelihood that a team will leave FIRST because of local competitions versus what is the likelihood that a team will start because of a local competition?

Now, to answer the questions:

Growth rates can be calculated by dumping data out of FIRST and meticulously sorting it, and then analyzed given economic conditions of the US/Canada. It can be done eventually.

Costs are tougher to determine - most teams don't publish their budgets. Event costs may be made available through FiM - as I understand, 7 districts are operating for the cost of one regional. In addition to cost, one must eventually consider the concept of "value" - how much bang did teams get for their dollar? Two local events for $5000 versus one event for $6000 seems seems huge.

Publicity will be much harder to quantify - counting news articles might not be feasible or even relative. Surveys might not yield enough data unless all teams respond (truthfully). Value of data might also become a question - are rookie team's inputs more important than veterans? How much do we care about what non-Michigan teams have to say?

What makes matters worse, is that these aren't even all of my questions. But its a start.

[Andrew Schreiber]

After attending the Rookie Regional, and Traverse City I feel that FiM is a step in the "correct" direction. That is just my opinion, some criteria I feel need to be evaluated.

Are some events considerably weaker than others? Yes some events are going to be weaker, this could be due to timing, first week events are generally less competitive than 6th week ones. But I am talking about geographical areas causing weaker winners. The winners from one event should be roughly equal to the winners from another event.

Does it benefit teams? Do we see more teams playing at higher levels? Higher levels is based on the skill of the team in question, obviously I don't expect every team to be playing at the level of 217 but I expect gradual improvement through the course of the season, that would be a true measure of this tests impact. A team that barely moves their first event needs to at LEAST score one ball in their second, that is a measurable improvement and I feel we should all strive to improve between events.

Long Term, one concern I have is that because we are organized by geographical districts (more or less) we will become very separate. For example let us say that the next year 85 comes out and wins Traverse City AGAIN, Im sure BOB is a great team (in fact after playing with and against them I KNOW they are) but I do not want to get into a situation where the State Championship becomes what is essentially the same teams over and over. With luck this structure will allow teams that are not perennial powerhouses to gain some ground.

A small concern of mine is that the depth of field will diminish, by the 4th alliances third pick teams should still be picking teams they WANT not just whoever is the least crappy machine left (Not saying they do or dont now, just saying it is something I never like to see) If this situation occurs frequently then there is a failing in the system.

I have some concerns about the structure, but I feel the goals are in the right direction. Best of luck and I cannot WAIT to compete instead of just watch.

[Wayne TenBrink]

FIRST is about inspiring young people to think about careers in Science, Technology, etc. How do you objectively measure the affect of the MI format on inspiration? In my opinion, it would be important to measure the format's affect on the expansion of the program (perhaps measured by the percentage of students in a region that have the opportunity to participate). This would take some time to do.

There is a concern about "quantity vs. quality", but what qualities are we talking about? If the new format led to a situation that drove students away from science and technology that would be a bad thing. If the average quality of the robots was diluted by a bunch of newbies and under-resourced teams, so what? This program isn't about us mentors and it isn't about the robots. The major learning benefits come during training, planning, and build - which isn't directly affected by the competition format.

As for the two day format, the lack of Thursday practice day increases the risk of schedule delay due to undiscovered field issues, and increases the liklihood of "no-shows" for unresolved robot issues. Those could be measured easily enough.

I don't think it is valid to associate field issues and the MI format. Any delays and communication problems at TC were apparently no worse than anywhere else. For the most part, they were associated with the new control hardware.

We were at Traverse City this weekend. Thursday evening check-in and inspection was a big help. We initially had communication problems due to firmware updates (thanks to Jim Sontag for fixing us up). Again, not specifically a MI format issue. It would be nice for teams attending their first event to be able to do a functional test on Thursday evening.

Our students don't seem to care about missing the out-of-state travel experience (not that we had money to go to a second event anywhere with the standard format!). Anything that involves getting out of town, staying in a hotel, eating in restaurants, and hanging out with other friends is OK by them.

I like the MI program. Anything that gets us into 3 competitions for less money than 1 is alright by me. Last year, we only had money for one regional. Our robot was badly damaged on Friday morning, and wasn't fully back up until Saturday morning - that squelched most of our "competition season". It wasn't particulary inspirational. The MI format would have helped us a lot.

[Andy L]

I haven't read every post here, but I thought I'd throw in my 2 cents at what I'm seeing that I like from the Michigan events that may be overlooked.

Our team participates in two regionals every year, both an hour away from our school. Convincing our school's administration, students, teachers, and parents to drive an hour to watch a robotics competition is extremely hard. District events would give us more local competitions, so we could get much more interest from our local area.

[Analog]

Originally Posted by Mike Q.

I think it was because the teams didn't have there firmware updated. I'm not the most Knowledgeable when it comes to that type of stuff.

Yes, this is correct. It could have been easily avoided if the teams understood the requirements BEFORE entering the arena. Our driver station firmware was also not up to date, but we caught it before opening ceremonies.

[GaryVoshol]

One success already, and I realize this is only anecdotal, not firm facts.

I heard one of the host coordiators say the Traverse City District cost $7,000 to put on. Compare that to the estimated $150,000 - $250,000 it would have cost to stage another regional; that's a real success right there. Most of this was due to the hard work of the organizing committee and the support of the community. For example, we volunteers were fed very well by a different catering company every meal. The caterers also sponsored the event; our exellent lasagne supper cost the committee $2.00 per head.

I still share some of the concerns brought up in this thread, and a few others. I'm willing to withhold judgement until sometime after the season is over.