On the quality and complexity of software within FRC

[faust1706]

McCall quality factors are lacking, or in some cases non-existent. in FRC and it is about time to address it.

Year after year, the vast majority of teams are writing poorly written, inefficient, and disorganized code. Their code base fails to generalize for multiple years due to poor design, but no one seems to care or talk about this.

Someone posts a cad drawing of a robot that they did for practice. People ask questions asking about the foundations of the design ("How thick is the G10?"), but no one asks questions when code is released. No one asks, "what is the *big-O notation of this function?" No one cares enough to truly scrutinize someone else's code. No one cares that an on board vision program isn't threaded. No one cares about how inefficient a routine is so long as it works in a match. A lot of teams forbid the altering of code once it works, which is a disgusting practice. Teams that get 10-15 fps on a vision program and say "it's good enough" when they haven't even done their research on optimization techniques. The vast majority of code would get a 'C' at best in an intro to programming class in a high school. So why is no one talking about this?

[gblake]

Originally Posted by faust1706

... So why is no one talking about this?

I think you exaggerate a bit when you write "no one"; and I think it would be (should be) unusual to generalize about "the vast majority of teams" FRC software's condition, without first agreeing with your audience what requirements that software is supposed to satisfy.

There is more than one way to skin most cats, including software cats.

Blake

[Anupam Goli]

I'll bite.

Year after year, the vast majority of teams are writing poorly written, inefficient, and disorganized code. Their code base fails to generalize for multiple years due to poor design, but no one seems to care or talk about this.

Year after year the vast majority of teams fail to build a robot that can play the game effectively. Just like there are teams that have great designs but not so great software, there are teams with horrible designs but great software (I was on one of these teams in high school, the only thing I could brag about doing in high school was writing good software for a terrible robot).

People ask questions asking about the foundations of the design ("How thick is the G10?"), but no one asks questions when code is released. No one asks, "what is the *big-O notation of this function?"

FIRST gives us restrictions on weight and height. The only software restriction we have is the ports we can use and the hardware limitations. I don't think many of us are concerned about the efficiency of our code with the hardware in the roboRIO. A 120 lb limit is a much harder limitation to work with.

Also, as a firmware engineer (intern), my first priority is proving the concept before actually applying it to the hardware I work with. I don't care about the efficiency until I have to ship it on very limited hardware, I want to see the concept work before I start trimming my variables, deallocating memory, etc.

A lot of teams forbid the altering of code once it works, which is a disgusting practice.

I'm not so sure it's disgusting, but it's not a practice i'm fond of. Just like our mechanical design, our software design should be constantly evolving to include more automation and autonomous capability. However, software changes are harder to see physically than mechanical changes. Of course if you write code and deploy, and it doesnt work, someone's going to yell at you to change it back. However, with a mechanical design, you can see the points of failure easily. I agree, this is a bad perception to have. There are merits to having software not left alone (incompatible libraries in future updates, etc.) but those are few and far between I suppose.

The vast majority of code would get a 'C' at best in an intro to programming class in a high school. So why is no one talking about this?

You vastly overestimate the rigor of a high school programming class. If you saw the MATLAB code I wrote for my Computing for Engineers introductory college course, I'm sure you'd have a heart attack. When you're not graded on efficiency and don't have to time to make it efficient, why make it so?


Just like the top teams have amazing designs, they also have amazing software. I marvel over 254's software releases and always learn something about their software design. I suppose the majority of CD's talk is mechanical design-oriented, so it's tough to see all of the software discussion going on (and there's a lot).

[Pault]

I can agree that compared to the CD ME community, the CD programming community really does not do nearly as good of a job at encouraging and helping teams improve their code beyond just making sure they understand the fundamentals and their code works. There is very little talk about making code flexible, or features that could be added, etc. All of the threads are something like either "What programming language should I use?", "Where do I go to learn how to code in X language?", "Why does this code not work?", "What is a PID?", etc. I think it could really benefit the teams on here if there was more in depth conversation.

Your being very harsh on the average team, though. Just like with mechanical and electrical, the goal of many programming teams is just to "make it work well enough for us to get picked for eliminations." Many of them dont even have a coding mentor, and rely pretty much exclusively on the resources online to figure out how to write basic code. I don't think we need to be criticizing them because their code is not as robust as we would expect from our own teams, they are struggling to get it to work at all.

[kylestach1678]

Originally Posted by Pault

I think it could really benefit the teams on here if there was more in depth conversation.

This is very true. Only a very small portion of CD contains anything related to programming, and of that, as you said, very little is anything more than stack overflow-style "why does this not work" problems. It would be fantastic if there was more discussion on CD about the fantastic code that some teams have created such as 971's control code, 1706's and 900's vision programs, and 254's motion planning to name a few.

[Ari423]

I'm afraid I have to agree with OP. I am the head (and only) programmer on my team, and have been so for the past 3 years, so I have a lot of experience with programming for FRC.

I will be the first to admit programming is hard to do well, but I think some teams just aren't putting enough effort to make their code efficient. Just today on CD I saw a piece of code (which I will leave unnamed) that almost brought my eyes to tears in its inefficiency. Even without a programming mentor and with a 6 person team, I have always put an emphasis on code efficiency and design as well as effectiveness. Sometimes I feel as if the FRC community has taken all of the focus from programming and moved it to mechanical, and left programming as an afterthought: something that you just throw together in a few hours to make it work and then never look at again. I understand that there are multiple ways to do a single task, but some ways ARE better than others.

A good example of this is dashboards. I cannot speak to programming a Java or C++ dashboard, but I know that making a dashboard in LabVIEW is easy to do, relatively speaking. Every year, after I finish the robot code and control panel, while I am waiting for the robot to be ready to test on, I program a dashboard that shows the important information coming from the robot, and adds other inputs than those on the control panel (trims, resets, etc). It usually looks something like this:

It doesn't take long to make, makes debugging much easier, and generally looks nice. We don't always use all of the displays, but they is at least one display for every aspect of the robot code. However, I often see much larger and more advanced teams who use LabVIEW but either don't have a dashboard, use the default dash, or make a custom dash with only a few numerical outputs. I don't understand; maybe someone can explain this to me. With the customizability given to you, why not make a dashboard suited to your needs that is easy to read and looks nice?

tl;dr Why is programming becoming an afterthought in the FRC community?

[connor.worley]

IMO once your code works, you get more diminished returns improving it than you do improving your mechanisms. Efficiency doesn't really matter because there's no requirement to scale.

Requiring teams to submit their code to the judges for review might fix this, but I don't know if the volunteer capacity exists.

[MikLast]

Originally Posted by arimb1999

I don't understand; maybe someone can explain this to me. With the customizability given to you, why not make a dashboard suited to your needs that is easy to read and looks nice?

Maybe the only thing you really need is three auto buttons.
(that's all we needed. the driver station console gave us everything else we needed.)

[cjl2625]

Originally Posted by arimb1999

I'm afraid I have to agree with OP. I am the head (and only) programmer on my team, and have been so for the past 3 years, so I have a lot of experience with programming for FRC.

Yup, exact same for me.

Most of our code for this year is pretty gross, which is probably a result waiting for the mechanical team to make something, and then me scrapping together something that makes it move. There's plenty of automation, but it's perhaps not the most efficient.

In general, it's just rushed through to make something that works. I usually don't improve it, perhaps because I'm afraid of breaking the code, and then getting yelled at by those darn wrench-swinging monkeys of the mechanical subteam (). But mainly it's a result of time. I'm usually so busy that it's not really worth my time to make these improvements that won't really have a worthwhile effect. Perhaps if the programming team was more than one person, it would be more manageable to produce beautiful, efficient code.
I'm determined to expand the programming team for next year, so we'll see what happens with the code then.

Though over the summer, when I actually have time, I do like to reiterate some of the code that may be used in the future. I've remade our swerve drive code a few times, and now it's significantly more efficient and organized than the original version. And I know there's still room for improvement, which I'll try to tackle this summer.

[faust1706]

Sorry for lack of quotes, on mobile. The blame is also on us. My mentor had me calculate the big O of every algorithm I designed, and he questioned every algorithm design I had. If I didn't have a well thought out answer, I wasn't allowed to use it. We've been using that code base ever since, haven't changed a line of it, and it has been able to adapt to the 2013 game, a 3 camera system in 2014, and 3d imaging this year.

Part of our partnership with nvidia will be promoting good code in the community and getting people excited about programming. We thought of the idea of hosting a class at worlds, but we feel even if we had it, not a lot of people would show up.

I do not have a solution to this deeply rooted problem as it ultimately comes down to each individual team. Addressing it as a problem is the first step. Another idea we had with nvidia is to have an award for quality of source code, teams that want to be a contender would submit their code online before competition began.

[teslalab2]

I can say this, I was the programming team and half of the mechanical team, and though I spent a lot of time on the code, it worked and that was about it... when I wanted to finally get around to fixing the spaghetti in my code, the robot mechanically suicided and I didn't have time to fix my code, this went on constantly until build was over.

[Necroterra]

Throwing in my 2 cents about some of the reasons teams might have poor software.

• Not enough manpower: I think that many teams have only a single programmer. If so, it is not trivial for one person without any support to learn and apply everything that goes into buiilding a good project.
• Not enough support: I also wouldn't be surprised if many teams had no dedicated software mentors. Additionally, even here on CD we don't have much of a focus on software, and the WPILIB documentation, while decent, doesn't teach anything about how to design good software (Git, how to organize code, patterns, etc.)
• Not enough visibility: Almost every other aspect of a team is immediately present in their robot and team - you can see the mechanical design, the team branding, the manufacturing, etc. but aside from the occasional code release on CD, I would bet that many teams' programmers haven't really even looked at anyone else's code.
• Not enough incentive for good software: aside from innovation in controls, there is very little incentive for medium or low performing teams to put a lot of resources into their code. Most teams don't have a robot that performs well enough for things like extendable/flexible code or vision to actually matter. The roboRIO is more than fast enough for pretty much everybody, and with the CANTalons, the vast majority of functionality is achieved by very simple code.
• Team's attitude towards software: I think some teams have a very negative or accusatory attitude towards the software subteam. I've definitely seen some shirts / signatures / quotes / jokes that seem to indicate some (probably not conscious) lack of respect towards programmers. I think our team had some bad experiences with these sorts of things before I joined, but I can't myself comment on it.
• Limited robot access: It is inherently difficult to develop software for a machine that isn't built yet. Design/fabrication teams will always want to improve the robot for all six weeks, meaning the programmers have to fight for time with the robot.
• Time limitation: six weeks is a really short timeframe for software. Implementing a good workflow, doing code reviews / refactors, etc. just doesn't really make sense on such a short timescale. I think many teams don't do these sorts of things during the offseason either, either because they just don't work much on robots outside of the main season or for any of the reasons above.

I just think overall, the structure and culture of FRC isn't honestly that great for learning software unless the students are really motivated. I've definitely gotten a lot out of it, but I also have been doing a lot of research and outside work. In my one year of experience, I ran into many of the above issues.

Also, forbidding the modification of release code (that is, working code at competition) is a great idea. Aside from hotfixes / emergencies / changing something trivial like autonomous values, or controller inputs, messing with code at competition is bad practice.

[EricH]

Originally Posted by Pault

All of the threads are something like either "What programming language should I use?", "Where do I go to learn how to code in X language?", "Why does this code not work?", "What is a PID?", etc. I think it could really benefit the teams on here if there was more in depth conversation.

Caution: "Not a Programmer" Talking in Programming Thread.

I'm going to take the above as kind of a "trigger point". Mainly because, it's what got my attention.

Who is asking those questions?

No, seriously, who is asking those questions? And, just for grins, why?

Think about it.


Now that you've thought about it, from what I've seen an awful lot of those threads are your first-time programmers. Not 2nd, 3rd, or 4th year programmers, but first-timers. The exception is the "why is this not working", which can come from anybody who is trying something new, or just wants a code review.

The other factor is that many teams don't give their programmers a robot until Week 5.75 if the programmers are lucky. Then they expect the robot to work in the first match on the field. See: hotfix the code and pray it works. Then fix some other bug. Repeat.


1197 does have a good programming team--but I know enough not to ask questions. (I leave that to the programming mentors.)

[connor.worley]

Originally Posted by faust1706

Another idea we had with nvidia is to have an award for quality of source code, teams that want to be a contender would submit their code online before competition began.

If you do this, allow submissions after competition. Lots of cool features get added between competitions.

[Pault]

Originally Posted by EricH

Caution: "Not a Programmer" Talking in Programming Thread.

I'm going to take the above as kind of a "trigger point". Mainly because, it's what got my attention.

Who is asking those questions?

No, seriously, who is asking those questions? And, just for grins, why?

Think about it.


Now that you've thought about it, from what I've seen an awful lot of those threads are your first-time programmers. Not 2nd, 3rd, or 4th year programmers, but first-timers. The exception is the "why is this not working", which can come from anybody who is trying something new, or just wants a code review.

The other factor is that many teams don't give their programmers a robot until Week 5.75 if the programmers are lucky. Then they expect the robot to work in the first match on the field. See: hotfix the code and pray it works. Then fix some other bug. Repeat.


1197 does have a good programming team--but I know enough not to ask questions. (I leave that to the programming mentors.)

Yes, I completely realize that. I never said it was a bad thing that those threads exist. I was just pointing out how few threads actually provoke deeper conversations about coding than that. Imagine if nearly every thread about mechanical topics were "What CAD software should we use?", "Where can I learn to use X CAD software?", "Will this tank drive design function without breaking?", "How do gear ratios work?", etc. The technical side of CD would get boring really fast.