Einstein Field issues Handled correctly?

[jblay]

FIRST did what they have done for four years when these kind of issues have occured. Give maximum one round of replay and then proced with the matches regardless of the result.
These actions are not fair to the teams on the field, winners and losers, it is unfair to the teams that competed and have already been eliminated, but most importantly it is not fair to the kids who worked so hard all season just to have it come to an end like that.
As great it is to say that we use this cool crio from NI, it is not worth this. I fear that spreading the word of FIRST has become the word of FIRST instead of teaching kids life lessons. What happened on Saturday was a result of a decision made by FIRST 4 years ago that was not in the spirit of FIRST.
It is my opinion, and I know I am not alone, that it is time for a serious change to the control system because enough is enough. It is also my opinion that FIRST needs to remedy what happened on einstein by awarding the championship to all 12 teams on einstein, but this is something that I don't think there is a chance they will do.
A big move needs to be made by FIRST here and if it isn't done I suspect much fallout in the frc community with more teams just going over to do vex.

Once again, I'd like to kindly remind people the problem is most likely not the NI C-RIO. The C-RIO's job is to process the information given to it from the sensors and inputs from the radio and use that to control the various aspects of the robot.

The problem most likely is the radio. The C-RIO does not connect to the driver station at all. It's the D-link radio that does.

The C-RIO was introduced 4 years ago in 2009 to help in handling the vision processing that was introduced more heavily in 2009 than any other game, and to accommodate for the old IFI technology. The radio, on the other hand, wasn't chosen too well, and as a result left many teams without connectivity to the field.

tl;dr: It's most likely the radio's fault, not the C-RIO's.


EDIT: I am making these accusations because while it certainly is possible it is something else, multiple robots in every match had comms issues. Some of the teams never had comms issues before. I don't have a field diagnosis, but this is my result from troubleshooting the problem with what I have.

[techhelpbb]

Originally Posted by jblay

A big move needs to be made by FIRST here and if it isn't done I suspect much fallout in the frc community with more teams just going over to do vex.

As a mentor I think that the current control system (the entire control system including the radio) offers the opportunity to teach about troubleshooting. The problem is that while the opportunity to learn troubleshooting appears the tools, access and freedom to do that troubleshooting does not.

When I worked as the spare parts person at MAR Mount Olive I brought my oscilloscope. It surprised me the gulf between the teams that actually used it and the teams that neither knew what it was or even had any idea of what to do with it on a FIRST robot. The difference in levels of understanding was not between the veteran teams and the new teams were you'd think it would be.

I think that FIRST needs to very carefully reconsider their process for troubleshooting. Otherwise I suspect that the only thing a highly integrated control system does is remove students from the equation in favor of placing the responsibility directly on FIRST.

I respect either aspect of the matter. I'd just hate to make FRC operate on the equivalent of the LEGO NXT system (and I'm not knocking LEGO NXT I have been an FLL judge for years).

[ticoloco12]

Hello all,

Now that the dust has settled, if only a tiny bit, I'll share what I saw from Einstein. You may know me as Will, I work with team 207's strategy and drive team. I am a FIRST Alumni and have been mentoring for 4 years now. let me make it clear that Team 207 is home and nothing but thankful to team 233 and 987 for an amazing time at world champs and our first trip to Einstien. Through the chaos that was the Finals, they showed that they are pure class and of the highest examples FIRST has. (341, 254,and 78 as well)

Yes, many, an unforgivable number in fact, of robots were either dead or experienced significant periods of losing coms. Some, clearly game changing. This sucked the emotion and prestige of the finals, clear out of every team on the field. Not one of the drive teams, or pit crews in our alliance cheered as we made it into the finals after semi's. It was a feeling of disbelief, that FIRST was allowing such a thing. It would have taken 30 minutes, to replay all of Einstein back to back in my mind. but that did not happen.

As a programming mentor for Pink said earlier, this is not how ANY FIRST team wanted to win or lose. I remember being on the drive team myself and dreaming of one day coming back to mentor my team there, to feel that rush, to cheer to the absolute maximum the human lungs can. To have been dead silent, and not even be able to rally our team between F1 and F2 because everyone's sentiment was, whoever works longer will win, is the largest disappointment I have ever had in my FIRST life. Even if making it there after an EPIC curie division Finals. From what I see in finals videos and scores, 341's alliance in the division finals was as competitive if not more so than any alliance on Einstein.

However, the most important feeling and experience to get away from this, is the spirit of FIRST teams. They are completely separate from fields and external chaos such as sever weather, curses, coms issues and all else. Every team on Einstein was united in their dislike of what happened, and treated each other graciously and professionally. When 987 needed time to inspect their robot and wanted to call a timeout (and I still don't know why we were denied one) 180 came to offer any assistance and tried to call theirs as well. both finalist alliances congratulated each other, on the field, and in the pits. The students were proud and even 118, and the Canadian teams did not hesitate in spreading hugs and handshakes to us all. The students themselves began making plans to replay all matches at IRI with robots in the same condition as they left the field Saturday. Dean Kamen was on the field, as was Dave and the look on their face said it all. They were disgusted and Dean was almost in tears as a Girl from a team I cannot remember pleaded the cause of her team. Dave was gone it seemed by the time the finals were over, and Woodie's face from the stage looked like he was viewing somebody drown and could not help. I insist, the truest qualities of FIRST were never seen in such bright colors and they were here. Technology failed us and perhaps the field crew have a huge catastrophe to correct. These finals will be remembered as the day the field chose the winner. BUT IN NO WAY, SHOULD THESE EVENTS TARNISH THE IMAGE OF FIRST, ITS MISSION, ITS STUDENTS, TEAMS, OR FOUNDERS.

Some of the greatest Robotics matches I have ever seen took place on Curie field in the finals. And I speak for every member of team 207 when I say we are beyond proud of all we did, and all our alliance partners did. We would put money on our alliance anyday, and if we lost a fair match, we'd cheer so loud for the alliance that beat us that our lungs would explode. Regardless of what trophies or banners the fields gave us, the embraces, laughs, comments, and connections with amazing FIRST-ers are sooooooo much more valuable. I truly hope the Einstein replay at IRI happens, not to play these matches, as much as to share them with such amazing teams and individuals, like we should have been able to.

I hope a lot changes next year, but only as far as the field goes with technology, and in organization and error management. the Human aspect of FIRST and it's competitors is something we can be prouder today of than ever.

-Will

[Deetman]

Originally Posted by SuperNerd256

Once again, I'd like to kindly remind people the problem is not the NI C-RIO. The C-RIO's job is to process the information given to it from the sensors and inputs from the radio and use that to control the various aspects of the robot.

The problem is the radio. The C-RIO does not connect to the driver station at all. It's the D-link radio.

The C-RIO was introduced 4 years ago in 2009 to help in handling the vision processing that was introduced more heavily in 2009 than any other game, and to accommodate for the old IFI technology. The radio, on the other hand, wasn't chosen too well, and as a result left many teams without connectivity to the field.

tl;dr: It's the radio's fault, not the C-RIO.

Be careful making blanket assumptions and statements like that without any evidence supporting it. This is really a general statement... we can all speculate as much as we want but I am not aware of anyone (possibly even FIRST) that has hard data that points to one component or another. Anecdotal evidence seems to be pointing many in one way, but there are multitudes of layers to our complex control system in which any one of them could be the cause.

[jblay]

Originally Posted by SuperNerd256

Once again, I'd like to kindly remind people the problem is not the NI C-RIO. The C-RIO's job is to process the information given to it from the sensors and inputs from the radio and use that to control the various aspects of the robot.

The problem is the radio. The C-RIO does not connect to the driver station at all. It's the D-link radio.

The C-RIO was introduced 4 years ago in 2009 to help in handling the vision processing that was introduced more heavily in 2009 than any other game, and to accommodate for the old IFI technology. The radio, on the other hand, wasn't chosen too well, and as a result left many teams without connectivity to the field.

tl;dr: It's the radio's fault, not the C-RIO.

If it is so simple then why didn't FIRST make a change in radios after the immediate issues of week 1?

[techhelpbb]

Originally Posted by jblay

If it is so simple then why didn't FIRST make a change in radios after the immediate issues of week 1?

It's not that simple.

Some robots have power issues.
Some robots have radio issues.
Some robots have code in the cRIO that causes communications problems.
Some robots have bad components.
Some robots have bad wiring (that sometimes doesn't show up unless it's under load or getting impacted just the right way).
Some robots have issues with video swamping.

I can say that I personally have used the NI cRIO 8 module PLC in industrial use and it's a nice product. A little expensive but nice.

This said there are many, many examples of the D-Link AP's power connector being picky and a possible source of issues. That could be fixed by soldering wires for power to the PCB inside the D-Link (if only FIRST would allow it or provide some like that at each event).

It's not fair to say that we have sufficient quantifiable evidence to lay blame on any one piece of hardware. Good troubleshooting dictates that quantifiable evidence must exist for this to be more than speculation.

[216Robochick288]

I honestly hesitate to answer. Maybe switching fields just simply wasnt an answer. It is hard to say for sure, but I anxiously await what FIRST might have to say about this. I certainly would hope that this gets solved for next year. Still, congrats to The Bomb Squad, Raider Robotics and SPAM.

Originally Posted by Brandon_L

Also one question that arose on the ride back from a few people: Why would the teams agree to play on a broken field?

In a responce ot this, what were they supposed to do? I see little that the teams could do.

[Ether]

Originally Posted by ticoloco12

Hello all...

Thanks for sharing your thoughts. It's better to post a link instead of re-posting the exact same text to multiple threads (especially for lengthy messages).

[Tim Reddersen]

"This is where gracious professionalism comes into play." Quote from my son Daniel, one of the drive team members of 2194 Fondy Fire. I have no doubt that the FIRST organization will handle this well. I saw Dean Kamen squatting by the field, looking at things, with concern written all over his face. And I am very gratified to see that the posts here on Chief Delphi are so supportive. You are all awesome and we are still happy beyond words to have been the first Wisconsin team to make it to Einstein. Our most sincere thanks to our Alliance members 548 Robostang and 118 Robonauts.

[Johnny_5]

Originally Posted by steverk

Has anybody stopped to consider the lighting on Eistein?

The lighting was very different on Einstein than it was on any other field. It was much more involved. It had moving spotlights. It was routinely dimmed.

Theatre lighting is controlled by a several pieces of equipment called "dimmer packs." In recent years, they went to wireless controls on the dimmer packs, so ugly or lengthy control wires would not need to be strung to the lighting racks.

To quote wikipedia directly:

Recently, wireless DMX512 adapters have become popular, especially in architectural lighting installations where cable lengths can be prohibitively long. Such networks typically employ a wireless transmitter at the controller, with strategically placed receivers near the fixtures to convert the wireless signal back to conventional DMX512 wired network signals...The first commercially marketed wireless DMX512 system was based on frequency-hopping spread spectrum (FHSS) technology using commercial wireless modems.[8] Somewhat later some venders used WiFi/WLAN technology. Other later generation systems still used frequency-hopping spread spectrum (FHSS) technology, but at higher bandwidth. FHSS systems tend to disturb other types of wireless communication systems such as WiFi/WLAN. This has been solved in newer wireless DMX systems by using adaptive frequency hopping and cognitive coexistence, a technique to detect and avoid surrounding wireless systems, to avoid transmitting on occupied frequencies.[9]

(http://en.wikipedia.org/wiki/DMX512)

So could this be part of the problem? Was the transmitter near the wireless access point? Hopefully someone familiar with the field can answer some of these questions.

I own a theater company and I manage a few theaters out of my local hometown and use a very similar setup to what they use in the rigging on Einstein. Wireless DMX is very behind in the amount of channels it can support and the lights that move, the intelligent lighting takes up many channels as every single aspect takes one channel. It would not have made any sense for them to have used wireless DMX. Using W-DMX would also mean flying all of the dimmer packs for the lighting, then the rigging isn't balanced on one side if those packs are up there.

I have used 2.4GHz and 5Ghz for intensive applications such as media streaming in theaters that were using wireless DMX at the same time and experienced no interference.

[koolbob23]

Originally Posted by Johnny_5

I own a theater company and I manage a few theaters out of my local hometown and use a very similar setup to what they use in the rigging on Einstein. Wireless DMX is very behind in the amount of channels it can support and the lights that move, the intelligent lighting takes up many channels as every single aspect takes one channel. It would not have made any sense for them to have used wireless DMX. Using W-DMX would also mean flying all of the dimmer packs for the lighting, then the rigging isn't balanced on one side if those packs are up there.

I have used 2.4GHz and 5Ghz for intensive applications such as media streaming in theaters that were using wireless DMX at the same time and experienced no interference.

Agreed. Using wireless DMX would be way too much work for this kind of setup as well as I thought I saw some DMX wires on the metal bridges they were using to hold everything up. Getting all the lights to work correctly is enough work without factoring in wireless well at least for me.

[Duke461]

Originally Posted by Holtzman

How exactly would we go about rebooting our crio mid match when we have no communication with our robots? Walk out on the field and cycle the main breaker?

We will share our full thoughts and experiences in the coming days.

Well, I'm in no way an electronics guy, so I may not have worded this correctly. However, there's something you can do on/through your driver station to do something similar to "restarting the cRio/radio/something". We had no comm's or code and we clicked something on our Driver Station during the match to fix it (I was not the one doing this so I'm not 100% sure what they did). If someone knows what I'm talking about and could explain what I mean further, that would be awesome.

-Duke

[basicxman]

Originally Posted by Duke461

Well, I'm in no way an electronics guy, so I may not have worded this correctly. However, there's something you can do on/through your driver station to do something similar to "restarting the cRio/radio/something". We had no comm's or code and we clicked something on our Driver Station during the match to fix it (I was not the one doing this so I'm not 100% sure what they did). If someone knows what I'm talking about and could explain what I mean further, that would be awesome.

-Duke

There is a button to soft restart a connected robot's cRio. What you're describing likely didn't affect the cRio.

[robostangs548]

I have been a student and FIRST mentor since I was 8 years old. I am now 21, and with school and work now consuming my life, I will be announcing my resignation from the team that I have grown to love and hold as my number one priority over the last few years. I have grown with this team from the "new-beginning" where we lost all sponsorship my freshman year, and paid mostly out of pocket to build and compete in the FIRST Robotics Competition.

Since this was what I considered the "last year" I decided to dedicate most of my time, mentoring and helping build both an amazing team, and robot. Along with the a dedicated team, thousands of hours, and a hard earned budget, this was the first experience I have had on Einstein. I can remember watching all of the great teams that I grew up with on Einstein from 1998-2011, and dreaming that with enough dedication, fund-raising, and overall team effort, we would one day be able to compete with those teams. Our team worked to find sponsors, mentors, and build a dream robot. After many all-nighters and extreme dedication, finally, that opportunity came with a near perfect alliance of 118, 2194, and 548.

Though it is still unknown what the root cause of these issues were, I am amazed that with a overall financial contribution as large as those in which large corporations and other donors have contributed, along with the number of engineers and other volunteers involved, FIRST was still unable to fix an issue that was known week one.

My "pit-crew", along with others, sat field-side disheartened by the fact that technology was failing not only our alliance partners, but also all of our opponents. An image that sits burned into my brain is the "This is FIRST" video that used to be on the FIRST website, where one of the mentors says "Months of Work.... Sat there.... Dead...." and really brings an emotional side out of me that can't be put into words. I can say with a large level of confidence, that this issue had nothing to do with the robots, but rather the field. I think the scores of these Einstein matches were a clear indicator of how messed up the outcome of this really was.

A team like 118, who is mentored by engineers that work on projects that literally leave our planet, who had a robot that not once had an issue through our entire Newton competition, without even a slight modification in code, were handicapped with a robot that "Sat there.... Dead...." every match on Einstein with the only reason being that of field coms.

118 proceeded to be lectured by an FTA that it definitely was not field issues, but rather issues with their robot. They pulled up a report from their robot only to see that the field could definitely be the reason for the loss of communication at the start of the match. The FTA said it could be their cRio having an issue, so they swapped out to a new cRio only to have results matching those from matches prior.

I was amazed how teams on the field pulled together, worked as a team, and almost every team said "we did not want to win this way." All teams were upset at one point or another with the outcome of these Einstein matches. From the field, we heard the lack of cheering, and could see the look of utter shock in the faces of everyone in the audience. Though the FTA's did work hard to help solve this issue, I believe that FIRST had no idea what was going on. Do I think they made the right decisions in how they handled the situation? Far from it. Do I think they knew what the issue was? Absolutely not. But after that entire disheartening fiasco, I would assume they wont let it happen again. I think that all the teams were amazing, and though it was upsetting, it was an honor to leave FIRST surrounded by everyone I grew up admiring.

Congrats 16, 26, 180.