Should FIRST Release the 2009 Control System beta Libraries and information?

[Elliot Swart]

The beta test teams currently have access to beta versions of the libraries for the compactRIO. While the limited number of compactRIO controllers obviously limits the size of the beta test, FIRST is compounding the advantage by limiting access to the programming libraries, and other controller information, to those teams.

While the explaination given for this is that they don't want teams to panic/plan based on inaccurate or unfinalized information, I think it is patronizing of FIRST to presume that the non-beta test teams are incapable of handling this.

I may be in the wrong about this, and so I am putting this poll up to see what the rest of the FIRST community thinks.

[pogenwurst]

I think the "panic" concern is legitimate, but at the same time I think that's outweighed by the need to keep the playing field level. I'm still a bit put-off about the way the beta test is being handled, but the time for debate on that matter is passed.

I'd at the very least like a systematic overview of what capabilities are/are not offered and the general structure, if we can't have code.

[EricH]

You know, it's called "beta test" for a reason. If a video game is in the beta test stages, maybe a few thousand people are playing it out of millions, and NOBODY else has access.

The "test" part means that those libraries are most likely untested or not tested under "battle conditions".

FIRST may be assuming that the other teams can't handle it, but probably not. I would actually assume that they know that the teams (for the most part) can handle it, but choose not to so that there aren't 1500+ solutions to a problem which could be more easily solved by fifteen teams finding a better solution.

I would actually say that by controlling the release of those libraries, they are doing us a favor by making sure that said libraries are complete. Which is worse: Knowing that a library is there and not having access, or having access only to find out the hard way that it doesn't have what you need?

Or, for another example, say FIRST accidentally released part of the Manual early. One of the parts talking about the game, say, in early December. Maybe it's missing a subsection; maybe it isn't quite clear. Teams start planning. Then they release the game in January and lo and behold, there's an extra subsection, it's a whole lot clearer, and there's a major change. Would you be confused? That's probably why they're holding it back.

The beta test teams' advantage will disappear quickly as teams get their units and build on the beta testers' data.

[BornaE]

Originally Posted by Elliot Swart

The beta test teams currently have access to beta versions of the libraries for the compactRIO. While the limited number of compactRIO controllers obviously limits the size of the beta test, FIRST is compounding the advantage by limiting access to the programming libraries, and other controller information, to those teams.

One of the reasons for the limited number of beta test teams was the limited availability of the cRio's. a bigger reason is that the FIRST developers and NI developers working on this project can only answer questions to a limited number of ppl.
Even with the 18 teams, there is a lot of discussion going on and i am having a hard time keeping up with all of it, lots of bugs are being fixed.

at this stage, the codes are not very useful if you cannot ask questions from the developers and they cannon answer questions to everyone.
As soon as the code is in a stage that is usable by the FIRST commiunity, it will be released.

[Elliot Swart]

The primary issue with having access to the libraries is being aware of code structure. Just as the 2005-2008 controllers all had code that required a certain structure (user_rountines.c, different places for autonomous and teleoperated code), the CompactRIO is almost certainly getting a structure with a simliar function. While even if we cant program with the libraries, we can still design template code.

Current Poll Status:
16 For
4 Against

[EricH]

And if the libraries have a flaw? What happens to that code? Do you fix it? If so, do you publish the fix? What if 1500+ teams do the same thing?

If the libraries have a flaw, and teams design around the libraries, even in template, would you want to be responsible for the resulting confusion, failed code, etc.? I wouldn't. Neither, apparently, does FIRST. Hence, the beta test.

I have one other question. Would you rather have had no "guinea pigs"? No beta testers to make sure the system works? Or would you rather have 1500+ teams having trouble?

Seeing as FIRST is doing a beta test, we have to accept the conditions. I'm sure that FIRST will release the libraries as soon as possible.

[Jonathan Norris]

Does anyone know if the controller will be available early for rookie teams, from what I remember it will be available early for veteran teams. Just wondering if it will be the same for rookie teams.

[Elliot Swart]

The beta test is an important part of integrating the new controller into FIRST and I have no problem with that.

While I have no doubt that many teams will choose to sit out the beta test and wait for the results, there should be a way for teams who want to see the libraries to do so. To ensure that teams who look at beta code and information know about the risks of using beta code, perhaps they could register to get read-only access to the beta test forum. This would also prevent the problem of having 1500+ solutions generated for each problem.

Even with the libraries, without the CompactRIO the non-beta test teams are not going to be activly debugging them. We will just get an overview of how the robot is controlled. While we need a beta test, there is no reason why non-beta test teams shouldn't have access to the beta information if they want it.

[EricH]

Originally Posted by Jonathan Norris

Does anyone know if the controller will be available early for rookie teams, from what I remember it will be available early for veteran teams. Just wondering if it will be the same for rookie teams.

I think it's first-come-first served... I seem to remember that it's any team can ask for it to be sent early, but they pay shipping.

[Pat Fairbank]

As a member of 1114's beta-testing team, I've seen the C/C++ code for the new controller, and in my opinion it's still somewhat rough around the edges. I think releasing it to teams now would just result in confusion, and dozens of people pointing out the same flaws to the code's maintainers.

[EricH]

Originally Posted by Elliot Swart

...

While I have no doubt that many teams will choose to sit out the beta test and wait for the results, there should be a way for teams who want to see the libraries to do so. To ensure that teams who look at beta code and information know about the risks of using beta code, perhaps they could register to get read-only access to the beta test forum. This would also prevent the problem of having 1500+ solutions generated for each problem.

Even with the libraries, without the CompactRIO the non-beta test teams are not going to be activly debugging them. We will just get an overview of how the robot is controlled. While we need a beta test, there is no reason why non-beta test teams shouldn't have access to the beta information if they want it.

Tell that to FIRST. FIRST, for reasons known only to them, has chosen not to release the information. If this is to change, teams must ask for it. Also, there is the public forum.

Once again, if we want a change in FIRST, we must ask FIRST to change.

[Elliot Swart]

That's what I am trying to do. I just plan to let the poll sit for a couple of days, and if the % in favor of releaseing seems impressive enough, I'll write to FIRST asking them to release the information.

p.s. The public forum has very little information


I think one problem with the beta test are the group applications. Essentially, teams that applied together are getting access to all the information and code, even though they only have 1 compactRIO. Many more teams than the 18 beta testers have access to the code. Why shouldn't every team.

It is notable that many of the disenters are either from Beta Teams or in the beta testing groups. Your teams obviously feel themselves prepared to deal with the perils of beta code and information, why do you want to deprive that right from others.

While I think many people in the beta teams disagree with me about the release of information, please remember that FIRST is a competition, and they are being given an advantage. Thier programming and electronics people will be much better prepared at the start of the competition than the rest of the teams. While I don't resent them for being accepted into the beta tests, and I understand this test is necessary, as part of the FIRST community and pillars of the FIRST competition, they should be using thier influence to level the playing field, not fight against it.

When a beta tester says that he doesn't think the other teams are ready to see what his team is allowed to, he either being condesending, or trying to retain his teams competative advantage.

[EricH]

Originally Posted by Elliot Swart

p.s. The public forum has very little information

Give them time. The teams got their units earlier in the week. You don't expect teams to do ALL the testing in one week or less, do you?

It is notable that many of the disenters are either from Beta Teams or in the beta testing groups. Your teams obviously feel themselves prepared to deal with the perils of beta code and information, why do you want to deprive that right from others.

I have not been on the team I graduated from for two years. This year, I will have no team by my own choice. And it isn't just the beta teams, it's WFA winners. Remember 159 teams applied, but only 18 were accepted. I'm sure FIRST would have liked 159 teams, but that's the way things are.

When a beta tester says that he doesn't think the other teams are ready to see what his team is allowed to, he either being condesending, or trying to retain his teams competative advantage.

Or, he sees something that he thinks is not right and is buying time for a fix to be implemented.

Remember, the beta teams are required to teach other teams about what they learn. If they don't, FIRST could have them send their unit back.

EDIT: Elliot, I think someone beat you to it.

[Mike Schroeder]

okay to all of you who think the code and stuff should be released early, take a second, go back and read some of the stuff in the technical forum here on chiefdelphi. find a thread with more than 50 posts in it, got one good, now read all the replies, I will give you a sec.... good no sort through all those replies, and tell me how many were useful...... good now out of those useful replies tell me how many were someone repeating that information over again, now tell me how many contratict each other, now how many are just someone saying something like this is great or i agree or i dont like it or just a random post not relaying any useful information. now imagine that x 10 and that is what you will get if you open all this early to everyone so Dave Lavery has said it before, and although its raining in NJ right now i'll reitterate GO OUTSIDE get some fresh air enjoy life, then worry about robots.

[BornaE]

Originally Posted by Elliot Swart

When a beta tester says that he doesn't think the other teams are ready to see what his team is allowed to, he either being condesending, or trying to retain his teams competative advantage.

I think that in this stage a limited number of people should have access to the codes and clean it up. no one meant any specific team neither they intended to be condescending.

FIRST has a limited number of developers working on this new control system and if they are bombarded by hundreds of email per day asking almost the same questions, they cannot efficiently isolate problems and fix it, nor they can answer the questions.