On the quality and complexity of software within FRC

[faust1706]

McCall quality factors are lacking, or in some cases non-existent. in FRC and it is about time to address it.

Year after year, the vast majority of teams are writing poorly written, inefficient, and disorganized code. Their code base fails to generalize for multiple years due to poor design, but no one seems to care or talk about this.

Someone posts a cad drawing of a robot that they did for practice. People ask questions asking about the foundations of the design ("How thick is the G10?"), but no one asks questions when code is released. No one asks, "what is the *big-O notation of this function?" No one cares enough to truly scrutinize someone else's code. No one cares that an on board vision program isn't threaded. No one cares about how inefficient a routine is so long as it works in a match. A lot of teams forbid the altering of code once it works, which is a disgusting practice. Teams that get 10-15 fps on a vision program and say "it's good enough" when they haven't even done their research on optimization techniques. The vast majority of code would get a 'C' at best in an intro to programming class in a high school. So why is no one talking about this?

[gblake]

Originally Posted by faust1706

... So why is no one talking about this?

I think you exaggerate a bit when you write "no one"; and I think it would be (should be) unusual to generalize about "the vast majority of teams" FRC software's condition, without first agreeing with your audience what requirements that software is supposed to satisfy.

There is more than one way to skin most cats, including software cats.

Blake

[Anupam Goli]

I'll bite.

Year after year, the vast majority of teams are writing poorly written, inefficient, and disorganized code. Their code base fails to generalize for multiple years due to poor design, but no one seems to care or talk about this.

Year after year the vast majority of teams fail to build a robot that can play the game effectively. Just like there are teams that have great designs but not so great software, there are teams with horrible designs but great software (I was on one of these teams in high school, the only thing I could brag about doing in high school was writing good software for a terrible robot).

People ask questions asking about the foundations of the design ("How thick is the G10?"), but no one asks questions when code is released. No one asks, "what is the *big-O notation of this function?"

FIRST gives us restrictions on weight and height. The only software restriction we have is the ports we can use and the hardware limitations. I don't think many of us are concerned about the efficiency of our code with the hardware in the roboRIO. A 120 lb limit is a much harder limitation to work with.

Also, as a firmware engineer (intern), my first priority is proving the concept before actually applying it to the hardware I work with. I don't care about the efficiency until I have to ship it on very limited hardware, I want to see the concept work before I start trimming my variables, deallocating memory, etc.

A lot of teams forbid the altering of code once it works, which is a disgusting practice.

I'm not so sure it's disgusting, but it's not a practice i'm fond of. Just like our mechanical design, our software design should be constantly evolving to include more automation and autonomous capability. However, software changes are harder to see physically than mechanical changes. Of course if you write code and deploy, and it doesnt work, someone's going to yell at you to change it back. However, with a mechanical design, you can see the points of failure easily. I agree, this is a bad perception to have. There are merits to having software not left alone (incompatible libraries in future updates, etc.) but those are few and far between I suppose.

The vast majority of code would get a 'C' at best in an intro to programming class in a high school. So why is no one talking about this?

You vastly overestimate the rigor of a high school programming class. If you saw the MATLAB code I wrote for my Computing for Engineers introductory college course, I'm sure you'd have a heart attack. When you're not graded on efficiency and don't have to time to make it efficient, why make it so?


Just like the top teams have amazing designs, they also have amazing software. I marvel over 254's software releases and always learn something about their software design. I suppose the majority of CD's talk is mechanical design-oriented, so it's tough to see all of the software discussion going on (and there's a lot).

[evanperryg]

FIRST gives us restrictions on weight and height. The only software restriction we have is the ports we can use and the hardware limitations. I don't think many of us are concerned about the efficiency of our code with the hardware in the roboRIO. A 120 lb limit is a much harder limitation to work with.

As you said, a big reason programming restrictions may be so small is that school programming courses are very low-caliber. the most advanced programming taught in my school is very, very simple VBA, or a little RobotC. The "programming class" is literally just a semester of ALICE, the biggest joke in the programming world. However, most other technical aspects have some sort of advanced class. Physics, autos, woods, any technical class will teach you some amount of mechanical engineering. Schools with PLTW have a rigorous electronics course that goes as far as to explain the workings of FPGAs and other highly integrated digital systems, and most schools have an introduction to electrical class that will get into basic digital logic. Programming? Not so much. The "problem," if you can even consider there to be a problem here, is that most students already have some kind of base for other technical aspects of the robot, but none for programming.

Originally Posted by faust1706

Studies suggest that roughly 40 percent of people aren't even capable of learning to program. Just look at the fizz bizz test. Really good read.

Seems like another reason why FRC programming is generally very rough. If it's inherent that only a limited number of people can program at all, then the fact that FRC is getting lots of people to make code that even works is pretty impressive.

Originally Posted by gblake

It's become so trendy that I worry that folks are losing sight of the differences between efficiently operating on small datasets and efficiently operating on large datasets. For small datasets, low-overhead brute force will often beat the pants off of manipulating some fancy data structure that is appropriate for larger datasets. There is more to writing efficient code than learning the big O characteristics of various data structures and algorithms.

FRC scouting in a nutshell.

Originally Posted by EricH

That's also not its goal. Plain and simple. FRC is aimed at inspiration.

Inspiration through science and technology. OP's prerogative seems to be making the science and technology in FRC more advanced, thereby advancing the inspiration. Advancing programming in students is certainly in line with FIRST's goals, but it has diminishing returns and is much more difficult to implement than other modes of FRC-related inspiration.

Originally Posted by connor.worley

IMO once your code works, you get more diminished returns improving it than you do improving your mechanisms. Efficiency doesn't really matter because there's no requirement to scale.

We've begun to encounter the issue that WPIlib for Java is poorly coded itself. The libraries relating to vision code are particularly messy. This may be why so many teams opt to build their own libraries entirely. Sometimes, optimization is good. But, usually, it isn't all that necessary for FRC because we have a magical beast of processing power called the roboRIO. Since there's no real limit on how inefficient your code can be (unless you try vision code, then good luck) there's usually no reason to optimize your code. Outside of FRC? Yeah, optimization is important. In FRC? Not really, because 2 minutes out of your 2:15 match, your robot is an RC stacking machine, not an autonomous robot.

[Thad House]

Originally Posted by evanperryg

We've begun to encounter the issue that WPIlib for Java is poorly coded itself. The libraries relating to vision code are particularly messy. This may be why so many teams opt to build their own libraries entirely. Sometimes, optimization is good. But, usually, it isn't all that necessary for FRC because we have a magical beast of processing power called the roboRIO. Since there's no real limit on how inefficient your code can be (unless you try vision code, then good luck) there's usually no reason to optimize your code. Outside of FRC? Yeah, optimization is important. In FRC? Not really, because 2 minutes out of your 2:15 match, your robot is an RC stacking machine, not an autonomous robot.

This is a very true fact. You can tell the Java libraries for this year were rushed to get finished in time. Also, some of the hacks needed to interface with the native c++ code for the FPGA just look like they could cause more issues. This is the same thing that causes the vision libraries to be slow. It wastes alot of time marshalling the structs between java and c++, and this looks to be what is so slow. We've been trying to alleviate alot of these issues in the DotNet port, and its easier since working with native code is easier, but its still a challenge trying clean up the code to make it faster, yet keeping it running the same code.

[connor.worley]

Originally Posted by Thad House

This is a very true fact. You can tell the Java libraries for this year were rushed to get finished in time. Also, some of the hacks needed to interface with the native c++ code for the FPGA just look like they could cause more issues. This is the same thing that causes the vision libraries to be slow. It wastes alot of time marshalling the structs between java and c++, and this looks to be what is so slow. We've been trying to alleviate alot of these issues in the DotNet port, and its easier since working with native code is easier, but its still a challenge trying clean up the code to make it faster, yet keeping it running the same code.

I wish there was a middle ground, something that could run natively like C++ but be like Java in the sense that it lacks a lot of the pitfalls C++ learners fall into.

[Thad House]

Originally Posted by connor.worley

I wish there was a middle ground, something that could run natively like C++ but be like Java in the sense that it lacks a lot of the pitfalls C++ learners fall into.

Too bad having a garbage collector and running natively don't work together super nice, without a lot of work. Because that's basically what that would be. I think later this summer, I want to do a runtime shootoff between LV, C++, C#, Java and Python (all the currently working languages) and see how much performance you gain or lose with each one.

[gblake]

Originally Posted by Thad House

This is a very true fact. You can tell the Java libraries for this year were rushed to get finished in time. Also, some of the hacks needed to interface with the native c++ code for the FPGA just look like they could cause more issues. This is the same thing that causes the vision libraries to be slow. It wastes alot of time marshalling the structs between java and c++, and this looks to be what is so slow. We've been trying to alleviate alot of these issues in the DotNet port, and its easier since working with native code is easier, but its still a challenge trying clean up the code to make it faster, yet keeping it running the same code.

Some brave and enterprising young soul might want to try creating libraries that use JNI to implement these interfaces ... if JNI isn't being used already.

[Thad House]

Originally Posted by gblake

Some brave and enterprising young soul might want to try creating libraries that use JNI to implement these interfaces ... if JNI isn't being used already.

They are all JNI. The issues come from not being able to pass by reference, so passing structs, such as the ones used for vision, still has trouble even with the JNI.

Some of the things that I know are fine, but still make me cringe, involve the use of generics and enums. Since Java on the CRIO did not support either of these, most of the Java code doesn't have them. However, you can tell that some of the new classes do use generics and enums. I know this is fine, but you can tell there is a disconnect between the old and the new code, and something that would be nice would be for somebody to take a month and thoroughly clean it up. I bet with some cleanup to include new features, and maybe some algorithm refactoring, we could get the code much nicer and easier to work with. Maybe they could fix all the spelling and punctuation errors in the comments too

[evanperryg]

Originally Posted by Thad House

They are all JNI. The issues come from not being able to pass by reference, so passing structs, such as the ones used for vision, still has trouble even with the JNI.

Some of the things that I know are fine, but still make me cringe, involve the use of generics and enums. Since Java on the CRIO did not support either of these, most of the Java code doesn't have them. However, you can tell that some of the new classes do use generics and enums. I know this is fine, but you can tell there is a disconnect between the old and the new code, and something that would be nice would be for somebody to take a month and thoroughly clean it up. I bet with some cleanup to include new features, and maybe some algorithm refactoring, we could get the code much nicer and easier to work with. Maybe they could fix all the spelling and punctuation errors in the comments too

We're working on it. Among other things, we've also removed three nested while(true)s with no delays, and the notorious error message that swears at you.

[Thad House]

Originally Posted by evanperryg

We're working on it. Among other things, we've also removed three nested while(true)s with no delays, and the notorious error message that swears at you.

I would love to see this when you guys get done, or close to done. Maybe some of the fixes would be useful in the other ports as well.

Also, which nested while loops? I haven't noticed any that have caused issues so far, but maybe I just don't remember. I've read too much code lately.

[Ether]

Originally Posted by evanperryg

...

Hi Evan. If you've got a couple of minutes, there's some unfinished business over on your "Blown CIM" thread.

[evanperryg]

Originally Posted by Ether

Hi Evan. If you've got a couple of minutes, there's some unfinished business over on your "Blown CIM" thread.

I'm sorry... what are you talking about?

[kylestach1678]

Originally Posted by evanperryg

I'm sorry... what are you talking about?

I believe that he may have got the wrong Evan.

[evanperryg]

Originally Posted by kylestach1678

I believe that he may have got the wrong Evan.

yep, found the other evan...