On the quality and complexity of software within FRC

[faust1706]

Originally Posted by sforsyth

You most likely have not worked in the real world

This is my second summer interning as a software developer. The code base is cleaner than anything I have ever written and it is highly organized. Extremely navigable, highly commented, and the code you submit has to get approval from your ENTIRE team before it gets pushed. The company deals with massive amounts of data for analysis. Optimization is the name of the game. More functions than not have their corresponding big O with it, and many have big theta as well. Anyways, irrelevant.

Originally Posted by SoftwareBug2.0

I agree there's a lot of poor quality code in FRC and little higher-level discussion.

A lot of high schoolers do poorly with programming

Studies suggest that roughly 40 percent of people aren't even capable of learning to program. Just look at the fizz bizz test. Really good read.

Originally Posted by gblake

I am surprised by how few people have challenged the OP's assertion that I'll paraphrase as "most FRC software stinks"; and by how many respondents seem to embrace that assessment.

First agree on what you want a given collection of software to do and be. Then have a proper conversation about quality.

Because it is the general consensus it seems. I did expect some backlash. While robots vary greatly year after year, the software generally does not. I forget where I read this, but it basically said that software is not considered good if it isn't able to adapt to an unforeseen use of it.

Imaging if google crashed when you tried to find a route from london to toyko,

Making adaptive code is something that needs to be a goal from the start, not an after thought.

How about we all just real Zen and the Art of Motorcycle Maintenance and call it a day in regards to a conversation of quality?

Originally Posted by SoftwareBug2.0

Reading code is not very fun.

I love reading code, especially elegant and concise code. I think I'll make an off season "competition" of a code standard challenge from the team's code this year. I'll begin working on the details tomorrow with a friend and anyone else who is interested.

[artK]

I've also given the quality and development of FRC controls and software a fair bit of thought. I have also thought for a while about the amount of software discussion versus mechanical, strategic and even business discussions on Chief Delphi.

I think part of it has to do with the lack of robotics programmers, both on CD and in the FRC community as a whole. A lot of teams don't have dedicated software mentors, who can teach the students the principles of software engineering (especially design patterns which allow for easily expandable code), and control theory (at the very least some feedback control, but the more the merrier). Most teams would be lucky to get a computer science teacher who generally teaches the basics (which are important, but doesn't necessarily lead to good software design). This leaves the upperclassmen, who may not have years of programming experience joining the team, to teach the lowerclassmen, with leadership changing every year or two.

A reason for a lack of mentors with experience in robotics software may be due to the lack of exposure of computer science and software engineering students to the field, usually until grad school or dedicated robotics programs like WPI or Waterloo have, which don't have a huge output of people. And the difference between robotics software from what CS and SE students typically work with that plunging in head first is intimidating enough that they never give robotics much thought.

Additionally, I bet a lot of students use other websites not only to help debug their code, but to copy and paste code from a website, which doesn't lead as much learning. I would like to see somebody try and copy/paste a CAD model.

[gblake]

Originally Posted by faust1706

Because it is the general consensus it seems. I did expect some backlash. While robots vary greatly year after year, the software generally does not. I forget where I read this, but it basically said that software is not considered good if it isn't able to adapt to an unforeseen use of it.

Imaging if google crashed when you tried to find a route from london to toyko,

Making adaptive code is something that needs to be a goal from the start, not an after thought.

How about we all just real Zen and the Art of Motorcycle Maintenance and call it a day in regards to a conversation of quality?

Whether or not a general impression about low-quality exists, what is probably more important to think about is whether or not it is accurate.

The notion of adapting to unforeseen use is interesting to ponder. What does it have to do with software that isn't required to adapt to unforeseen circumstances? In that circumstance creating adaptable software is a waste of resources.

I read that book long ago. No need to repeat that.

Instead why don't we discuss what our software is supposed to do and be *before* we decide that it is a failure. The software development industry is no stranger to the importance of putting a good specification in place early.

I understand your thesis; but I don't think you have proven it. Until you can put together a good argument that the majority of FRC Software doesn't satisfy teams' requirements, put me in the camp that both says there is always room for improvement, and says that a general indictment is unwarranted.

[gblake]

Help me out folks, if a team's software isn't made public, isn't it supposed to be rewritten from scratch every year?

And even if all of a team's software is made public from year to year, isn't the general notion of the annual FRC learning experience that software, like the physical robot, will be largely developed anew, during the season, each year?

I realize that there is a broad spectrum of ways teams actually do their software development; but isn't the clean sheet of paper, plus some true off the shelf code, sort-of the big picture thrust of the rules?

[Necroterra]

Originally Posted by gblake

Help me out folks, if a team's software isn't made public, isn't it supposed to be rewritten from scratch every year?

And even if all of a team's software is made public from year to year, isn't the general notion of the annual FRC learning experience that software, like the physical robot, will be largely developed anew, during the season, each year?

I realize that there is a broad spectrum is ways teams actually do their software development; but isn't the clean sheet of paper, plus some true of the shelf code, short-of the big picture thrust of the rules?

I'm pretty sure that you are right and all reused code is supposed to be public. I think a lot of teams throw it on their website or a github repo, but it isn't under a easy to find name. However, it is also really hard to enforce this rule and it usually isn't that big of a deal, so it gets ignored as far as I know.

I don't see why you would ever start completely from scratch - things such as path planning, vision code, state machines, interfaces, dashboard tools, etc. can be developed and refined over years, and generally can't be done well in 6 weeks. Re-deriving your algorithms every year might be a learning experience, but it won't teach you as much as refining and expanding your codebase will.

I think that there are some situations where following this rule and being an effective FRC programmer don't necessarily align. Here are a few theoretical examples where the rule is technically broken.

1. I wrote a spline generator in the offseason / last year, and copy pasted it into my project this year without putting it on Github. This is technically against the rules, but at least the student is still learning.
2. I had my friend / parent / mentor give me a spline generator they wrote for college, and copy pasted it without making it public. This is bad, since you neither shared it nor wrote code yourself.
3. I opened my code from last year, and used it as a reference for how I did certain things such as getting joystick inputs. This to me is fine, since while you might retype certain lines verbatim, a single line or method call barely counts as code - in this case, really you are just not having to look things up in the documentation as much.

I love reading code, especially elegant and concise code. I think I'll make an off season "competition" of a code standard challenge from the team's code this year. I'll begin working on the details tomorrow with a friend and anyone else who is interested.

I would love this, so would it be like you provide a description of the robot and it's sensors / components, and people submit code that should make it run?

[efoote868]

On a one-off project, if I have a specific set of tasks and I'm writing serial code that must execute once every 20 milliseconds, why should I care if it takes 2 microseconds or 10 milliseconds to execute?

If I have 512 MB of memory available, why should I care if I use 10 kb or 300 MB, so long as it holds everything I need and works the way I want it to?


Particularly in one-off projects where I'm not designing to value, more important than smallest, absolute fastest, most efficient is code readability and "it just works." If I have 10 hours to write and test code to fit a spec, you'd better believe I'm spending about 8-9 hours testing my code to make sure it works. If I spend any time optimizing, that is time not spent testing.

I understand that if I'm manufacturing 100 million widgets and I can save a fraction of a cent on manufacturing PCB or hardware specs I'm doing it, but if I'm doing just one prototype, cost is not my motivating factor.

[Michael Hill]

Originally Posted by faust1706

Sorry for lack of quotes, on mobile. The blame is also on us. My mentor had me calculate the big O of every algorithm I designed, and he questioned every algorithm design I had. If I didn't have a well thought out answer, I wasn't allowed to use it. We've been using that code base ever since, haven't changed a line of it, and it has been able to adapt to the 2013 game, a 3 camera system in 2014, and 3d imaging this year.

Part of our partnership with nvidia will be promoting good code in the community and getting people excited about programming. We thought of the idea of hosting a class at worlds, but we feel even if we had it, not a lot of people would show up.

I do not have a solution to this deeply rooted problem as it ultimately comes down to each individual team. Addressing it as a problem is the first step. Another idea we had with nvidia is to have an award for quality of source code, teams that want to be a contender would submit their code online before competition began.

Here's one of the interesting things about FRC. Your mentor has taught you some really good skills that are applicable at a much higher level than what's normally done in FRC. A bit academic, but from your other posts I've seen on here, it looks like it's made you a better developer. While code may have been developed faster for the robot, it doesn't really matter because you learned something. While you may not calculate the Big O for every algorithm you do, you will probably also keep it in mind in any future algorithms your write.

[marshall]

Originally Posted by efoote868

I understand that if I'm manufacturing 100 million widgets and I can save a fraction of a cent on manufacturing PCB or hardware specs I'm doing it, but if I'm doing just one prototype, cost is not my motivating factor.

Even with large scale manufacturing, cost is no longer the limiting factor thanks in large part to companies like these: http://www.arm.com/ and http://www.intel.com/

Processing power is cheap and there are so many layers of abstraction these days that optimizing for Big O isn't necessarily where you'll see the most performance gains. From my experience in the manufacturing and software development industries, there is a lot more to be gained from network optimizations than there is from Big O these days. You can optimize an algorithm all day long but if you send it via a crappy protocol then the information might not get there or, more likely, it will arrive late.

My point being:

Originally Posted by gblake

There is more than one way to skin most cats, including software cats

Meow.

[artK]

Originally Posted by gblake

Help me out folks, if a team's software isn't made public, isn't it supposed to be rewritten from scratch every year?

Yes. The rules usually state this fairly early on in the manual. I think Frank even made a post telling teams this rule early so people would publish it.

And even if all of a team's software is made public from year to year, isn't the general notion of the annual FRC learning experience that software, like the physical robot, will be largely developed anew, during the season, each year? I realize that there is a broad spectrum is ways teams actually do their software development; but isn't the clean sheet of paper, plus some true of the shelf code, short-of the big picture thrust of the rules?

Yes, new software is supposed to be written each year. The real question is what new software needs to be written. A number of teams, from my general impression, will start essentially from scratch each year. I believe this may contribute to the problem. Since most teams don't/can't reuse the code from the previous year, they have to start from what they know, and usually without help. It's like asking a designer to CAD up a drivebase who doesn't necessarily know what goes into a drivebase. Yeah, it might work, but it's probably really heavy, slow, and doesn't turn well.

By comparison, other teams will reuse the framework from the previous year, and adapt it to the new robot. It's like starting the season with enough COTS parts to build a drivebase, and the parts increase in quality and quantity each year. Is it ready for competition? No. Does it save development time, allowing programmers to work on other things? Yes. Is their an easily available framework for teams to use? Not really, at least not now. You could borrow code from teams like 254 or 236 who post their code every year, or try and use the scripting languages I've seen 1902, 987, and 4334 post (I think those teams have did that in the past).

[Greg McKaskle]

I have two pennies.

I do not think that FRC code is bad across the board. I think the range of code quality is similar to that of wiring and mechanical construction. In other words it ranges from impressively good to -- "are you actively trying to sabotage your team" -- bad.

To me, much more important than the code on the robot is the knowledge gained by the students on the team. If students wrote code that has issues, but learned how to identify the issues and improve their performance the next time, then FRC has provided a successful challenge. If they have learned how to communicate and write SW as part of a team, how to break down a problem, complete their solution, meet a deadline, evaluate their results, ask questions, and accept mentoring, then they are well prepared for college, training, and career.

To paraphrase, it isn't really about the robot code. That is only the campfire ...

And by the way, communication with nonprogrammers is super important. Most programmers don't write code for themselves or for people like them. They write code as part of a product team that includes EEs, MEs, business and marketing, writers, perhaps even doctors or physicist. When you learn to explain what your code does and doesn't do to a nonprogrammer, when you learn to understand someone else requirements, expectations, resources, tolerances, and deadlines in a common domain language, you are better prepared to become a successful programmer. This goes for all engineering disciplines, not just programmers.

Finally, I see FRC as a multidisciplinary engineering challenge. It incorporates strategic decision making, logistical efficiency, interaction with real-world materials and tolerances, and interaction with five other robots (most years). And it isn't just a controlled demonstration of capabilities. It is a timed sport-like competition that incorporates human drivers and players , and it also places value on presentation, aesthetics, and outreach.

What I'm getting at is that FRC is BIG and complex and really challenging. Code is but one part of the robot. It is necessary for teams to make decisions as to what they will spend time on. There are many many ways to successfully navigate the big FRC ocean. I'd highly encourage the programmers to enter other contests and take on additional challenges. I'd also encourage everyone on the team to learn more about programming. While CS/CE specialists may always be needed, you may also find it very useful as an ME/EE/whatever. I think computers are here to stay.

Greg McKaskle

[Ari423]

Originally Posted by artK

Yes, new software is supposed to be written each year. The real question is what new software needs to be written. A number of teams, from my general impression, will start essentially from scratch each year. I believe this may contribute to the problem. Since most teams don't/can't reuse the code from the previous year, they have to start from what they know, and usually without help. It's like asking a designer to CAD up a drivebase who doesn't necessarily know what goes into a drivebase. Yeah, it might work, but it's probably really heavy, slow, and doesn't turn well.

THANK YOU!!! On my team, we generally re-write the code each year, but we model it off of previous year's code. This allows us to use parts of the code that worked and find better ways to do things that didn't. Once they learn how to program, all of our programmers have access to view the robot code so they can learn good practices from it and become a better programmer before they take on the role of head programmer.

I think I'll make an off season "competition" of a code standard challenge from the team's code this year. I'll begin working on the details tomorrow with a friend and anyone else who is interested.

I would definitely be interested in doing this. Either have contestants interpret what already made code does (and maybe optimize it) or give them a task and have them make code to make it work. Only problem will be different programming languages. I have some "interesting" to read through code that might be a fun challenge without comments for this competition.

4 years. Every 4 years, there is a complete turnover. (I intentionally exclude mentors from this.) Given that many teams have limited programmers in the first place (and, given some of those threads, it's tough to get a programmer to step up to replace a lone programmer who is moving on), there isn't really a good progression... so even if you do get a programmer or two who are starting to force the team towards high quality, or do more complex things, right about the time they hit that point they're gone, and someone else is starting from near-zero.

As long as you have at least one year of cross-over to teach the new programmer, we have always been fine. When I joined the team, the head (and only) programmer was a senior, and he spend most of his time working on the robot, while I spent most of my time learning how to program. He only actually taught me how to program the robot in two hours after the end of the season. Sure, I wasn't as good my first year as head programmer as I am now, but I was still able to make neat, efficient code that works. So either every programmer on my team who has been in this situation is a genius, or other people can do it too.

Again, I cannot stress enough how much it helps to have a version of code finished before the robot is built. Sure you might not be able to test most of it, but you can test that state machine that would take you hours to make and test while you have better things to be doing. Having code done beforehand allows you to make more advanced code.

[faust1706]

side note that I don't want to create another thread about: I need help deciding on a task of sorts that would interest programmers enough to do it. PM me(or shoot me an email: hunter.park11235@gmail.com) if you're interested in helping me design a task.

[AlexanderTheOK]

I have personally been following and teaching this guide.

[wireties]

Not all teams have poor software practices. But it is rarely "professional" looking. While helping other teams at competitions I find software all over the place. 1296 splits the application up into tasks/threads for each subsystem and uses messages to synchronize everything. This way the students are all working on smaller self-contained behaviors. We also emphasize good coding standards (including testablilty, reduced complexity, Hungarian notation, etc), draw it then code it, test and retest etc.

Somewhere in this thread it was asserted FRC robots are not all that complex because they "do not process much data". That is kind of a IT way of looking at it rather an engineer's approach. The complexity in FRC robot software centers around temporal constraints, control algorithms and quality assurance. Handling the data to and from the driver station and dashboard is the simple part.

Several persons in this thread have remarked about the value of tested code. Code that works is crazy valuable but can be even more so if good practices are followed from day zero. Software has a "life cycle". That life cycle ends if the code is unreadable or not maintainable even if it works. So the holy grail is to follow good practices to begin with, test the heck out of it and always keep a copy of the code that worked! It can be done, even in a 6-week time frame.

HTH

[GeeTwo]

I'll admit that I've just skimmed the responses, so I'm probably going to repeat a few points:

• Selection Bias: Most of the posts of code on CD are either code that is not working, and the writer doesn't know why, or software that the coder believes is well written (often justifiably so), and is being posted in a GP spirit. I'll wager that neither is typical FRC code!
• Excepting mentor coders, an average FRC coder's experience is probably better measured in months than years. Many high schools (including Slidell High) do not offer a single programming class. We don't despair at how clumsily the bear dances, but marvel that it dances at all.
• Most robot designs don't settle down in time for the coders to have a chance to really test the code. This is true at the rookie end, where the robot gets bagged before it's working, and at the powerhouse end, where the robot is redesigned at least twice between stop build and CMP, and at most points in between.
• How many posts have we seen lamenting that six weeks and three days is not enough time to build a robot? Guess which department gets the shortest part of that too-short stick? Anyone?
• Except in mandatory high-reliability environments (e.g. medical, space, perhaps automotive), code that works 95% of the time delivered by the deadline is "better" than code that works 99.999% of the time but comes along a few months too late. FRC is NOT a high-reliability environment that grants enough time to "get it right".
• Git is great when you have access to the internet, when the head programmer has the training and discipline to manage forks and when every programmer has the discipline to sync up after every couple of hours of coding. High school students at competitions typically meet none of these requirements.
• Test code on the practice field? Don't make me laugh! We can't get enough time to test mechanical systems on a field that doesn't properly simulate the competition field.
• While we don't have a general rule of "don't fix code unless it's broken", a basic risk/reward analysis frequently results in code lock down during competition.