[FRC Blog] Event Results and API Data
 

Posted on the FRC Blog, 2/28/15: http://www.usfirst.org/roboticsprogr...s-and-API-Data

Re: [FRC Blog] Event Results and API Data
 

Very, very, very frustrating, but at least glad FIRST still has all the data...

Re: [FRC Blog] Event Results and API Data
 

Do they?
http://www.chiefdelphi.com/forums/sh...4&postcount=23

Re: [FRC Blog] Event Results and API Data
 

If it is stored locally (which it clearly is, as the event needs to data to generate rankings and such), then there is no reason it has to be lost.

Re: [FRC Blog] Event Results and API Data
 

Yes, it is stored locally. I volunteered as a scorekeeper for an off season event last October, and the match data (and settings) were backed up automatically after each match. There are also means of exporting it as well.

Re: [FRC Blog] Event Results and API Data
 

New software this year, though. If we were running last year's software and API, we wouldn't be having these problems.

Also, even if the data is stored locally, that's no guarantee it's going to be pushed out and made available via the API after these events are over.

Re: [FRC Blog] Event Results and API Data
 

Yes, it is a different FMS software this year. However, each version of FMS seems to be based off of the previous season's version (with improvements and game specific updates). I don't think they would have eliminated the backup system, since that is a pretty important feature.

It is true that there is no guarantee that the data will become publicly available, but at the very least the district results need to be available. This data is vital to calculating the district rankings for each district team that competed this year.

Re: [FRC Blog] Event Results and API Data
 

This is based on my recollection of last year's FMS software...

I think that getting the data off would have to be a manual task. The database backups will definitely exist on each scorpion case, but I don't know if the functionality exists to automatically upload old data (or if you can even do it at all, once the event wizard is complete). The solution might be to remote into each field (ugh) and copy of the backup files. Or, have the Scorekeeper at each week 2 event export a bunch of csv reports, which can be imported. Doesn't seem like very good options, especially when the blog post didn't give a time frame for a fix. I wouldn't be surprised if it's not up for Week 2, either...

Re: [FRC Blog] Event Results and API Data
 

To my knowledge, FMS has been completely rewritten since last year.



I'd be surprised if there were any FTA across the country that wasn't aware that these problems were occurring. I would hope that they'd have the presence of mind to throw the data on a flash drive and/or send it manually to FIRST for reporting/archival.

Re: [FRC Blog] Event Results and API Data
 

The software is "new" in the sense that it has been updated since last year, but the core functionality is mostly the same as last year. They started with last year's code and updated it for this year.

I'm not sure about your comment about last year's API- The only "API" available last year from FIRST was a webpage that third parties could scrape (which isn't really an API). This is the first year a true API has existed.

I'm not sure what the plan is for data that wasn't already posted, but I wouldn't be surprised if FIRST put in the effort to upload it.

Re: [FRC Blog] Event Results and API Data
 

First opensourcing all the code they own would increase the quality of a lot of things.

Re: [FRC Blog] Event Results and API Data
 

That's a mighty bold statement. Pretty much totally wrong, but mighty bold.

Open source is essentially useless in an environment with forced deadlines.

Re: [FRC Blog] Event Results and API Data
 

Please tell us more about how allowing more about how the code getting more scrutiny ahead of time would make it worse. Can I sign up for your newletter?

Re: [FRC Blog] Event Results and API Data
 

There is a difference between open for scrutiny and open source.

If I put a document in a PDF and send it to you for you to review, that is open for scrutiny. If, however, I simply put the same document in GDocs and send you the link--with full editing permissions--that is open source.


In code, open source could be pretty bad, DEPENDING on a number of factors. Particularly, comments and other documentation, work breakup, and even "code styles".

But having the code open for scrutiny--that might help.

Re: [FRC Blog] Event Results and API Data
 

Pretty much this. Because the FMS has to be built and fixed on a deadline, an open source model, where you are depending on people who have no reason to meet an arbitrary schedule or work on "uninteresting" bugs to work to a schedule and fix the uninteresting bugs.

Re: [FRC Blog] Event Results and API Data
 

I think you're ignoring countless fundamental open source projects that fit all these criteria and far more.

Re: [FRC Blog] Event Results and API Data
 

Many open source projects do all these things very well...

Re: [FRC Blog] Event Results and API Data
 

You're falseoy equating open-source with community-built. I can name a number of open-source projects that were started, designed, and maintained by paid employees of a company but have code that is available for the public to view and modify at will. FIRST could still be the primary maintainer of the code while still allowing public scrutiny and pull requests.

Re: [FRC Blog] Event Results and API Data
 

Looks like some of the data is online at the http://frc-events.usfirst.org/ website now.

However at least 2 of the districts that I've looked at the data is showing up for the wrong alliance. (Red is Blue, and Blue is Red). The 2 that I have looked at are FiM - Southfield and FiM - Standish. I was trying to watch a few matches of teams so looked them up to find out what matches they played in on FRC Events. Then watched the video on youtube and they are on the opposite alliance in the video.

Not sure if they just transposed these the colors or what. But thought I would give everyone else a heads up. I also did not look at the Finals data or scores yet as I was just trying to do some scouting on teams that will be at our next event and see what they are capable of doing.

Re: [FRC Blog] Event Results and API Data
 

Isn't there a pretty swell alternative to the FMS that is open source, or am I going crazy?

Re: [FRC Blog] Event Results and API Data
 

So, if they make the real FMS open source, does that mean everyone is going to run out and spend thousands (probably 10's of thousands) of dollars on servers, touch panels, routers, displays and all the other hardware required to make it actually work? As an experienced Software engineer, I can tell you that the system is most likely too complicated to accurately and reliably reproduce on a simple computer, at least to the point where someone could contribute to updating it. Isn't that why they released FMS Lite? It's a simple, low cost alternative to the FMS that actually CAN be run on a normal computer with no specialised equipment.

Re: [FRC Blog] Event Results and API Data
 

The playoff display is too complex. It's difficult to determine who actually won the playoffs.
http://frc-events.usfirst.org/2015/TXDA/playoffs

They need to simplify to something similar to the TBA format.

http://www.thebluealliance.com/event/2015txda

Re: [FRC Blog] Event Results and API Data
 

Cool, guess I can tell my bosses that because our projects are open source I don't have to meet deadlines or work on uninteresting bugs.

You clearly have never worked on an open source project for a company. Please try to actually have experience with something before you comment on it.

Source: I work full time developing open source software. Last year I worked heavily on a project to compute clinical quality measure for the Veterans Health Administration. It uses the same computation engine as the verification tool for Meaningful Use Stage 2, a tool named Cypress, also developed as open source software. I'm currently working on tools to validate FHIR servers and to build a tool for patient risk management for doctors assessing geriatric patients. All of these have deadlines and the "boring bugs" get fixed.

Re: [FRC Blog] Event Results and API Data
 

Darn. Inland Empire, Georgia Southern Classic, and PNW Oregon City are still missing data. I wonder if they'll upload the data for these events manually so that we can see it.

Edit: Oops. Now I see there is only partial info for a lot of other events. The ones I mentioned have nothing.

Re: [FRC Blog] Event Results and API Data
 

I expect most of the FMS servers are in transit between events right now, so if the data didn't get fully uploaded it may be a day or two before the FTA's backup is used to backfill the data or the FMS servers get back online.

I wouldn't despair though, the data for Northern Lights was incomplete most of yesterday but now appears to be fully correct. The missing info could be stuck in a queue somewhere in the cloud.

Re: [FRC Blog] Event Results and API Data
 

Cheesy Arena is indeed open source. I didn't make it to Chezy Champs this year, but from what I hear, it performed flawlessly. No surprise given the all-star engineers that developed it.

FMS Lite has not been updated in years and is not open source.

Re: [FRC Blog] Event Results and API Data
 

No, it's not open source, but I wasn't saying it was. I was saying it was designed to be used on cheap, easy to obtain hardware, unlike the real FMS.

And also, it hasn't been years since it was updated. It was updated just last year for the 2014 season, and I would assume it will be again this year. http://www.usfirst.org/roboticsprogr...lite-available

Re: [FRC Blog] Event Results and API Data
 

Thanks for that link, I wasn't aware of the 2014 version; if I recall correctly, there was a gap of several years prior to that (2009?) which is where I got that idea.

Re: [FRC Blog] Event Results and API Data
 

It has been stated that a new FMS lite is planned so that in future years you do not need to select the FMS/DS communication within the driver station.

Re: [FRC Blog] Event Results and API Data
 

Allow me clarify my statement:

Given the talent of our community, I know putting the FMS and STIMS/VIMS on Github would result in useful, bug fixing, and feature adding pull requests. It wouldn't take away control from FIRST but would allow motivated mentors and students to help.

Re: [FRC Blog] Event Results and API Data
 

As a remote spectator, I was disappointed that I could not find a qualification schedule for my former team, and as a result I watched less than I have in years past.

Re: [FRC Blog] Event Results and API Data
 

The FRC website is actually doing this more "correctly," though neither are providing an accurate picture of what is happening. In qualifications, quarterfinals, and semifinals, neither alliance "wins." It is all based on average score for these rounds.

It would be nice if the FRC website highlighted who won the finals though, and both should more clearly highlight the teams that move on to the next round throughout playoffs.

As for Cheesy Arena:

Keep in mind that Cheesy Arena is designed for offseason events that don't have the team number signs, the team lights, emergency stop buttons, alliance station lights, SCC (alliance scoring cabinets), any game-specific field hardware (e.g. goal lights in 2014, weight sensors in 2013), ref panels, or any other miscellaneous field hardware that typical comes with a FIRST event.

It may be open source, but it also has far fewer moving parts to deal with.

Re: [FRC Blog] Event Results and API Data
 

Except the FRC website doesn't show the average scores, but The Blue Alliance does.

The bolded "winning" team on The Blue Alliance will be removed for non-finals matches.

Re: [FRC Blog] Event Results and API Data
 

Ah, I was just looking at the match results and somehow missed the playoffs breakdown on TBA. That is definitely more useful.

Re: [FRC Blog] Event Results and API Data
 

Cheesy Arena handles most of this stuff and was designed around using readily available hardware to display things such as team numbers. It even improves upon the game specific elements like the hot goal lights, which were actually timed correctly in Cheesy Arena.

Re: [FRC Blog] Event Results and API Data
 

Yes, it does provide most of things in forms other than what we are used to at FIRST competitions.

However, FIRST is unlikely to completely replace the existing hardware in the very near future. Additionally, while Cheesy Arena is an excellent substitute for the FMS for offseason events and the cost of the overall system is lower, FIRST has a different set of requirements for the official field.

For example, the GitHub page for Cheesy Arena explains that Chezy Champs used cheaper consumer-grade LEDs instead of the professional grade LEDs used on the official field.

I am not privy to the decision making process that goes into selecting the components for the field, and I'm not making any arguments for which software or hardware is better. What I'm saying is that Cheesy Arena isn't designed to be a replacement for the FMS during competition season, and isn't likely to become a replacement there.

Re: [FRC Blog] Event Results and API Data
 

I'm not sure whether or not open source FMS is a good idea yet. Certainly I think the community would be very on board with helping out. One additional point: wouldn't open-sourcing FMS open FIRST up to 2012-style sabotage?

Re: [FRC Blog] Event Results and API Data
 

The best security is achieved when everyone can see the problems and then fix them, not through obfuscation. Security achieved because "no one knows how our internals work" is bad practice.

Re: [FRC Blog] Event Results and API Data
 

Although it's not a perfect analogy, this sounds like various Wikipedia debates I've heard. On one hand, letting anyone edit pages leads to incorrect/biased information, sabotaged pages, and edit wars. On the other hand, Wikipedia is one of the largest, easily accessible sources of information--and most of the time other editors fix incorrect information.

Without any relevant experience, my guess on this would be that if FIRST decides to open source it, they open up the possibility that experienced mentors and students are able to help. However, they also open it to the risk that anyone can sabotage it if they like. Of course, there are far worse consequences to this than just a sabotaged Wikipedia page, which FIRST has to consider carefully.

Re: [FRC Blog] Event Results and API Data
 

It's important that to note that "open source" does not mean anyone can make edits without review, potentially allowing the system to be "sabotaged." Changes must still be reviewed and then accepted by the owner.

Re: [FRC Blog] Event Results and API Data
 

This. It's surprising how some people on this thread don't seem to understand how typical open source development works. It's not like we're suggesting FIRST make the repo public, accept all pull requests, and then just build and use whatever's there come competition season.

FIRST is likely very limited in the number of software engineers they can hire. Based on some of the technology stack being used, it's likely that FMS continues to utilize legacy code that's been around for a while. As Cheesy Arena demonstrates, a fresh approach using modern technologies can have great results.

Open sourcing FMS and creating a community advisory board could allow recognized developers to contribute to FMS and improve quality more than FIRST's in house developers could do on their own.

The biggest potential obstacle in making FMS an open source project might be the risk of potential game leaks. One possible approach could be to keep source closed closed during initial development and release the source once kickoff occurs. Then, feedback and community improvements could be put towards the next release. Android works in a similar way; each new version is developed internally by Google, then released to AOSP when it is announced publicly. OEM partners and community members can submit fixes which are then rolled into point updates or put towards the next version of Android.

Re: [FRC Blog] Event Results and API Data
 

I echo the calls for open sourcing FMS.

This and this.

Open Source does not mean there is not a core group of developers committed to working on it for a deadline. It just opens it up to the possibility that extra pairs of hands/eyes can be volunteered.

Open-sourced software does not make it more vulnerable to security flaws; in fact, if the community is strong enough, it can lead to even better security. Again, another set of eyes/hands to spot and fix those security, performance and scalability issues.

One could argue that FIRST should make FMS Free, not just Open Source, Software, but one step at a time I guess.

Re: [FRC Blog] Event Results and API Data
 

I also agree with the idea of a greater open-source initiative on the part of FIRST.

Open source has become huge in the CS world recently; and one reason is because of all the advantages it provides. FIRST's development team can remain "benevolent dictators" of their code, just like Linus is with the Linux kernel - contributions may be submitted by anyone, but only the worthy get merged in. I think that the community is dedicated enough to make meaningful contributions back upstream - and this benefits everybody.

Many people also regard the field software as a "mystery box" and have no idea how any of it works. An open code base will let people understand how the field functions, because they can peruse the code themselves. I think this might reduce some of the anger we've seen in recent years - this weekend's API issues and last year's delayed pedestals and hot goals come to mind. Who knows, maybe somebody in the community would have spotted an optimization that could have fixed the problems quickly. Often, a new set of eyes is all you need to spot errors (my favorite tactic is to Rubber Duck Debug with a stuffed tux toy on my desk).

I can, however think of a few drawbacks. Mainly developer time. I can only imagine how stressful being a developer at FIRST (it's a pretty small team, too) is this time of year, and adding in a responsibility to watch a public repository can be a lot. However, most of the main contributions can come back in the offseason, when they have more time. An open source repo also shouldn't contain any advance information about the next game. I liked the suggestion above about following the Android model - code doesn't have to be pushed to the repo until kickoff.

To me, the benefits of a greater open source involvement far outweigh the cons. I know that if the FMS API was open source, I would have spent a significant chunk of time this weekend helping debug it, and I would have been happy to help contribute fixes. Plus, FIRST could set a great example for teams by promoting open code; something all computer science students should have experience with when the enter the real world. FRC already gives so many positive experiences, this is another one that could make a great difference.

Plus, GitHub has a much better UI than TeamForge...

Re: [FRC Blog] Event Results and API Data
 

This alone should be enough of a reason to switch to a different release model. TeamForge is one of the worst pieces of software I've had the misfortune to work with

Re: [FRC Blog] Event Results and API Data
 

The model used by WPI during control system beta tests would be another possibility. In this model, access to source is granted to a select group of vetted individuals during development in order to move fast and avoid buggy or incomplete code from making its way into teams' hands. Once a stable version is available (such as at the beginning of the season), source is released to all comers, and bugs may be filed or patches proposed using online tools.

Re: [FRC Blog] Event Results and API Data
 

Greater than the FRC event API? That's pretty darned open. Anyone can take the data from the events and massage it and display it anyway they want. I myself have plans to work with it but no time yet.

Open source for the FMS is pretty silly. You cannot test and validate it without all of the specialized hardware required for a field and you cannot do any coding on the part that changes yearly (the scoring) without knowing the details of a game that's kept secret.

Re: [FRC Blog] Event Results and API Data
 

The scoring part isn't really the issue here; it seems to be data storage and transport that needs a lot of work. A properly designed system would let you work on all of that independent of the scoring components. And, as it's already been stated, the scoring components can still be released on kickoff for other individuals to review.

And no, the FRC event API is not open. It couldn't get any more closed. Public data != open-source software.

Re: [FRC Blog] Event Results and API Data
 

I think you're misunderstanding what we mean by open source, and are confusing it with a publicly available API. From Wikipedia:
For some more information about open source development models, read through this page

The FRC API is not open source, because I can not access the code running on FIRST's servers. They have published an interface for the API, but not the implementation details.

For example, it would have been beneficial to have the API open source this past weekend, when it went belly up. If members of the community had access to the source code, they could have been able to find the root cause of the issue and helped collaborate with FIRST on a fix. This would result in faster iteration and higher quality code.

A popular project that embodies many benefits of open source is the Linux kernel. It's the basis for a totally free (as in libre, not gratis) operating system, and is very heavily used in development circles. It is community written (over 4,500 different people have contributed), and is (in my opinion, but debate on this can be for another thread) the best OS out there.

As for your comments about FMS - it may not be feasible to run the software without access to the field hardware, but there is still value in making it open source. Namely, increased transparency. If people have the ability to go through the source and understand how it works, a lot of the mystery is taken out of plugging into the field (when you know exactly what is happening behind the scoring table). It would also let the community understand how and why some of the bugs in FMS (and believe me, they exist) are there, and possibly contribute to a fix, which means the bug could be patched faster than it would be otherwise.

Re: [FRC Blog] Event Results and API Data
 

It's also pretty darned broken.

And on the topic of testing without knowing the game, that is just wrong. It's absolutely possible to test data transport without knowing the game. And a properly architected system should just have game specific actions as part of a workflow anyway.

Re: [FRC Blog] Event Results and API Data
 

The API is somewhat "open" in a sense that they are taking suggestions from the community (in the team forge discussions) for the implementation of the API. However, the source code is completely closed to the public.

You are definitely right, just because public data is available doesn't make it an open source project. I think some people are confusing the difference between data and code.