|
On the quality and complexity of software within FRC
McCall quality factors are lacking, or in some cases non-existent. in FRC and it is about time to address it.
Year after year, the vast majority of teams are writing poorly written, inefficient, and disorganized code. Their code base fails to generalize for multiple years due to poor design, but no one seems to care or talk about this. Someone posts a cad drawing of a robot that they did for practice. People ask questions asking about the foundations of the design ("How thick is the G10?"), but no one asks questions when code is released. No one asks, "what is the *big-O notation of this function?" No one cares enough to truly scrutinize someone else's code. No one cares that an on board vision program isn't threaded. No one cares about how inefficient a routine is so long as it works in a match. A lot of teams forbid the altering of code once it works, which is a disgusting practice. Teams that get 10-15 fps on a vision program and say "it's good enough" when they haven't even done their research on optimization techniques. The vast majority of code would get a 'C' at best in an intro to programming class in a high school. So why is no one talking about this? |
Re: On the quality and complexity of software within FRC
Quote:
There is more than one way to skin most cats, including software cats. Blake |
Re: On the quality and complexity of software within FRC
I'll bite.
Quote:
Quote:
Also, as a firmware engineer (intern), my first priority is proving the concept before actually applying it to the hardware I work with. I don't care about the efficiency until I have to ship it on very limited hardware, I want to see the concept work before I start trimming my variables, deallocating memory, etc. Quote:
Quote:
Just like the top teams have amazing designs, they also have amazing software. I marvel over 254's software releases and always learn something about their software design. I suppose the majority of CD's talk is mechanical design-oriented, so it's tough to see all of the software discussion going on (and there's a lot). |
Re: On the quality and complexity of software within FRC
I can agree that compared to the CD ME community, the CD programming community really does not do nearly as good of a job at encouraging and helping teams improve their code beyond just making sure they understand the fundamentals and their code works. There is very little talk about making code flexible, or features that could be added, etc. All of the threads are something like either "What programming language should I use?", "Where do I go to learn how to code in X language?", "Why does this code not work?", "What is a PID?", etc. I think it could really benefit the teams on here if there was more in depth conversation.
Your being very harsh on the average team, though. Just like with mechanical and electrical, the goal of many programming teams is just to "make it work well enough for us to get picked for eliminations." Many of them dont even have a coding mentor, and rely pretty much exclusively on the resources online to figure out how to write basic code. I don't think we need to be criticizing them because their code is not as robust as we would expect from our own teams, they are struggling to get it to work at all. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
1 Attachment(s)
I'm afraid I have to agree with OP. I am the head (and only) programmer on my team, and have been so for the past 3 years, so I have a lot of experience with programming for FRC.
I will be the first to admit programming is hard to do well, but I think some teams just aren't putting enough effort to make their code efficient. Just today on CD I saw a piece of code (which I will leave unnamed) that almost brought my eyes to tears in its inefficiency. Even without a programming mentor and with a 6 person team, I have always put an emphasis on code efficiency and design as well as effectiveness. Sometimes I feel as if the FRC community has taken all of the focus from programming and moved it to mechanical, and left programming as an afterthought: something that you just throw together in a few hours to make it work and then never look at again. I understand that there are multiple ways to do a single task, but some ways ARE better than others. A good example of this is dashboards. I cannot speak to programming a Java or C++ dashboard, but I know that making a dashboard in LabVIEW is easy to do, relatively speaking. Every year, after I finish the robot code and control panel, while I am waiting for the robot to be ready to test on, I program a dashboard that shows the important information coming from the robot, and adds other inputs than those on the control panel (trims, resets, etc). It usually looks something like this: Attachment 19123 It doesn't take long to make, makes debugging much easier, and generally looks nice. We don't always use all of the displays, but they is at least one display for every aspect of the robot code. However, I often see much larger and more advanced teams who use LabVIEW but either don't have a dashboard, use the default dash, or make a custom dash with only a few numerical outputs. I don't understand; maybe someone can explain this to me. With the customizability given to you, why not make a dashboard suited to your needs that is easy to read and looks nice? tl;dr Why is programming becoming an afterthought in the FRC community? |
Re: On the quality and complexity of software within FRC
IMO once your code works, you get more diminished returns improving it than you do improving your mechanisms. Efficiency doesn't really matter because there's no requirement to scale.
Requiring teams to submit their code to the judges for review might fix this, but I don't know if the volunteer capacity exists. |
Re: On the quality and complexity of software within FRC
Quote:
(that's all we needed. the driver station console gave us everything else we needed.) |
Re: On the quality and complexity of software within FRC
Quote:
Most of our code for this year is pretty gross, which is probably a result waiting for the mechanical team to make something, and then me scrapping together something that makes it move. There's plenty of automation, but it's perhaps not the most efficient. In general, it's just rushed through to make something that works. I usually don't improve it, perhaps because I'm afraid of breaking the code, and then getting yelled at by those darn wrench-swinging monkeys of the mechanical subteam (:P). But mainly it's a result of time. I'm usually so busy that it's not really worth my time to make these improvements that won't really have a worthwhile effect. Perhaps if the programming team was more than one person, it would be more manageable to produce beautiful, efficient code. I'm determined to expand the programming team for next year, so we'll see what happens with the code then. Though over the summer, when I actually have time, I do like to reiterate some of the code that may be used in the future. I've remade our swerve drive code a few times, and now it's significantly more efficient and organized than the original version. And I know there's still room for improvement, which I'll try to tackle this summer. |
Re: On the quality and complexity of software within FRC
Sorry for lack of quotes, on mobile. The blame is also on us. My mentor had me calculate the big O of every algorithm I designed, and he questioned every algorithm design I had. If I didn't have a well thought out answer, I wasn't allowed to use it. We've been using that code base ever since, haven't changed a line of it, and it has been able to adapt to the 2013 game, a 3 camera system in 2014, and 3d imaging this year.
Part of our partnership with nvidia will be promoting good code in the community and getting people excited about programming. We thought of the idea of hosting a class at worlds, but we feel even if we had it, not a lot of people would show up. I do not have a solution to this deeply rooted problem as it ultimately comes down to each individual team. Addressing it as a problem is the first step. Another idea we had with nvidia is to have an award for quality of source code, teams that want to be a contender would submit their code online before competition began. |
Re: On the quality and complexity of software within FRC
I can say this, I was the programming team and half of the mechanical team, and though I spent a lot of time on the code, it worked and that was about it... when I wanted to finally get around to fixing the spaghetti in my code, the robot mechanically suicided and I didn't have time to fix my code, this went on constantly until build was over.
|
Re: On the quality and complexity of software within FRC
Throwing in my 2 cents about some of the reasons teams might have poor software.
I just think overall, the structure and culture of FRC isn't honestly that great for learning software unless the students are really motivated. I've definitely gotten a lot out of it, but I also have been doing a lot of research and outside work. In my one year of experience, I ran into many of the above issues. Also, forbidding the modification of release code (that is, working code at competition) is a great idea. Aside from hotfixes / emergencies / changing something trivial like autonomous values, or controller inputs, messing with code at competition is bad practice. |
Re: On the quality and complexity of software within FRC
Quote:
I'm going to take the above as kind of a "trigger point". Mainly because, it's what got my attention. Who is asking those questions? No, seriously, who is asking those questions? And, just for grins, why? Think about it. Now that you've thought about it, from what I've seen an awful lot of those threads are your first-time programmers. Not 2nd, 3rd, or 4th year programmers, but first-timers. The exception is the "why is this not working", which can come from anybody who is trying something new, or just wants a code review. The other factor is that many teams don't give their programmers a robot until Week 5.75 if the programmers are lucky. Then they expect the robot to work in the first match on the field. See: hotfix the code and pray it works. Then fix some other bug. Repeat. 1197 does have a good programming team--but I know enough not to ask questions. (I leave that to the programming mentors.) |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Really simply, most teams' programmers get the robot really late in the season--and most teams' programmers are then trying to get the robot running under pressure from mechanical who wants drive time. Ugly code that works is of far, far greater value to the team than really nice, standards-compliant, reusable year-after-year code. That is the perception, at least. And it is very difficult to break out of that without a determined effort by one or more programmers to force the issue. And then there's another problem. 4 years. Every 4 years, there is a complete turnover. (I intentionally exclude mentors from this.) Given that many teams have limited programmers in the first place (and, given some of those threads, it's tough to get a programmer to step up to replace a lone programmer who is moving on), there isn't really a good progression... so even if you do get a programmer or two who are starting to force the team towards high quality, or do more complex things, right about the time they hit that point they're gone, and someone else is starting from near-zero. You can argue that if they were doing it right that wouldn't be an issue, but getting to the point of doing it right can take YEARS. As far as complexity...Let's just say that some mechanical mentors aren't willing to trust the programmers with more than basic sensors, and have to be poked, prodded, and otherwise convinced to (allow the programmers to) use the more advanced items. (I'm not one of them--but I prefer the simpler ways of doing things over the more complex ways.) OTOH, when the complex stuff works right, they suddenly want a lot more... Just got to get them there--again, that usually takes a programmer who insists on it until... oops, graduated another one, time to start over. ;) |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
The simple version is that FIRST is an engineering competition, not a science fair or research project. It's the difference between an engineer at a company that designs and manufactures really cheap CD players and a college professor. The engineer designing the CD player realizes that their solution isn't perfect, and shouldn't be, due to its limited parts cost and design time. The college professor has years and years to conduct research and strives to achieve perfection. It's not always about doing things in a very academic, well documented, extremely optimized way, but instead it's about getting things done that work as well as they need to, but no better. Why shoot for 60 fps when 2 fps works just as well? Why learn about optimization techniques for things that are efficient enough? Why change code that already works when there is other work to be done? To many, FIRST is an engineering competition. Teams must engineer a solution to a problem in a limited amount of time, just like real engineers do in the "real world". There is a finite amount of time to optimize an infinite number of things, so engineers must make tradeoffs. Should they give the programmers 10 hours with the robot to write vision software, or should they iterate on our hook so that we can lift totes easily? I didn't see a team this year that seriously benefited from having computer vision on their robot. Should a programmer spend time optimizing and threading their code when they could spend time practicing driving the robot? Most people (myself included) judge robots by their ability to score points, prevent opponents from scoring, win matches (if applicable), seed high, and progress in the tournament structure of the competition. A really cool mechanical thing that isn't very effective in a match is unimpressive - for instance, some 3D printed parts, many carbon fiber parts, many crazy magnesium alloy parts... A really simple mechanical thing that is very effective in a match is very impressive. Look at the robots from 118, 254, 1114, and other really great teams. They use traditional methods of manufacturing with traditional materials because it works. A really cool software thing that isn't very effective (vision processing this year) is unimpressive to the crowd. A piece of software that just works is impressive, regardless of its efficiency, documentation, or organization. I never saw a robot and thought "Gee, that team would be so much better off if there software was more efficient", or "that team could beat the Poofs if their vision system had a tiny bit less lag". Instead, I thought things like "if only their elevator was a little faster". Quote:
What if 10-15 fps is enough? What if 1 frame per second is good enough? I guess what I'm saying is that most teams software is 'good enough' and is rarely is what causes a team to be unsuccessful. There are many teams with very well written software and ineffective robots, but there aren't many teams with effective robots limited by bad software. Rarely are the best robots filled with extremely complex algorithms and parts, but rather, well engineered simple solutions. The code posted in Chief Delphi whitepapers often deals with concepts more advanced than anything found on any world champion robot. |
Re: On the quality and complexity of software within FRC
Quote:
I couldn't have said it better. |
Re: On the quality and complexity of software within FRC
Quote:
The Gitflow workflow is, in my opinion, perfect for FRC Teams. Work on features and develop throughout build season, then once drive practice gets close, start a release branch, and put out a master update for the drivers to practice on (or of course just have them practice on develop... it depends on how much driver practice you would want to do). Then, a few days before each competition, start another release and clean code, disable any extraneous or nonworking features, decide on autonomous routines, etc. Changes at competition should be hotfix branch only. |
Re: On the quality and complexity of software within FRC
Quote:
So far, the closest thing to useful vision processing in those 3 more years I spent programming has been cheesy vision. While it is an ingenious solution to an unexpected but consistent field failure, it is not robot vision processing, it was a simpler, quicker to develop, and more effective alternative to it. This year, at my last regional as a student ever, Ventura, there were two teams that were able to consistently stack the 3 yellow totes, 1717 and 696. Both robots accomplished this task without cameras, because cameras were not needed. As others have said, it's an engineering competition, where we are making a robot that scores points. When code efficiency scores teams points, you'll see a lot of teams start making efficient code. |
Re: On the quality and complexity of software within FRC
I don't know why, but this thread reminds me of a joke me and one of the mechanical mentors started making this year about deciding to forget the code and just wind up the robot before every match. It's a mechanical team's dream...
|
Re: On the quality and complexity of software within FRC
I agree with many of the above posters about code efficiency, but not about the relative gains of code improvements. While "getting it to work" is a bare minimum that most teams accept, automating small tasks can help a driver tremendously (place stack, shoot once in position), as can sophisticated control like maintaining lift position rather than open loop lift control by the driver. Like anything else there are diminishing returns, but I do think most teams would do better if they put a higher priority on their code/controls.
In my mind, there are two ways an individual can work to change the status quo of "no one cares about code." The first is obvious: work with your team to do what you believe is important. Different teams differently value scouting, CAD, marketing. There are teams that are known for their excellence in their preferred areas. Be the team that excels in code and spreads the word. The second way is also obvious. If you think people should critique code on Chief Delphi the same way they do CAD, then you do that. It only takes two people to have a conversation. Perhaps you'll start a larger trend, maybe you'll just give some good advice. Teams quite often post their codebases on CD, and usually receive no response. Start the conversation you want to see. I do think FRC would better prepare students for 21st century engineering if FRC games placed a higher value on code, but I also recognise the difficulty of effectively doing that while maintaining watchability and other priorities. Not to mention that it's not FIRST's chief goal to prepare students but rather to inspire them. |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
This year, I had our team's programmers assemble a document including the code, with full comments, and some flowcharts and rationales. It helped them better understand their decision making process, realize its advantages and weaknesses, and identify ways to improve. It's also nice to have another deliverable for judges. P.S. getting programmers to document their work can be difficult. |
Re: On the quality and complexity of software within FRC
You most likely have not worked in the real world... ahhhh to have a fresh college mind where everything is perfect and there is plenty of time to make everything perfect and just the way *I* want it :p
There are others that have said it in this thread I'm sure... the perfect code is whatever you think it is, your perfect code is not necessarily mine. If it works, then it is probably perfect to the person that wrote it. Most of these kids are brand new to programming and it is better served just to learn the language, leave the perfection to College courses if that is what they decide to go into. That being said, I'm sure there there are many ways to help them learn to write clean code and that really should be what you are asking... how can we help the kids? |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Aside from vision (and possibly elaborate real time spline generation) what even deals with enough data in FRC to worry about O complexity?
|
Re: On the quality and complexity of software within FRC
Quote:
-A lot of high schoolers do poorly with programming: Even though the AP computer science test is ridiculously easy over 30% of students who took it last year received a 1 (for those not familiar with AP tests, 1 is the lowest possible grade). And out of all the AP tests given that was the highest percentage of ones awarded. -Reading code is not very fun. I know you've released a couple of interesting projects and it's cool to know what they do but I'm pretty much not going to read their source code unless I want to reuse them. -The code libraries that you're given in FRC don't set you up to do fancy things well. It's non-modular in a way that makes unit testing hard. It's hard enough that I don't think I would have figured out a way to do it when I was in high school. Sure there are teams that have worked around this and built things that you could build on, but they're not well known. For example, my team publishes our code which includes a very different way of using C++ but I don't know of any other team that's even looked closely at it. Similarly, 1510 has published an alternate framework for Java which they've been using but I only know of one other team that has bothered to try it. -Optimization and efficiency are ignored because they can be. To do what the average team does the processor is about two orders of magnitude faster than it needs to be and you have 3 or 4 orders of magnitude more RAM than needed. I always think that I'm going to get to teaching the students on my team about efficiency when we get to the point where there would be some benefit to doing so, but things always end up 'good enough'. -FRC games allow success with limited software functionality. Every year there are robots that have no autonomous program at all and win tournaments. This is silly. The game should start with each robot in a box taped on the floor and you shouldn't get driver control until it leaves the box. One thing that I must disagree with you about is that restricting changes to code can be rational. Even if your code is correct it does not follow that your system will behave as expected. Minor code changes can and do trigger bugs in other parts of the system. You might now be triggering a different (and buggy) code path in a library you're using. We've all seen compiler bugs before. And there's the famous "download silently failed and left system in an unusable state". When one of these other problems occurs it would be nice to find out why but right before a match is neither the time nor place. Also, there is some interesting coding going on even if it's not going on a robot. For example, here: http://www.chiefdelphi.com/forums/sh...hreadid=137451. |
Re: On the quality and complexity of software within FRC
I am surprised by how few people have challenged the OP's assertion that I'll paraphrase as "most FRC software stinks"; and by how many respondents seem to embrace that assessment.
It is completely impossible to make a judgment like that without knowing what the software is required/expected to do. You can easily make a case for McCall's criteria or other rather abstract notions of what is/isn't high-quality software being completely irrelevant if the software is satisfying all requirements placed on it. McCall, et al published their important work to take a shot at being able evaluate software written for use in a certain general context. However, outside of that context, other criteria often apply. First agree on what you want a given collection of software to do and be. Then have a proper conversation about quality. Don't put the cart before the horse. Don't let the tail wag the dog. Blake |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
Imaging if google crashed when you tried to find a route from london to toyko, Making adaptive code is something that needs to be a goal from the start, not an after thought. How about we all just real Zen and the Art of Motorcycle Maintenance and call it a day in regards to a conversation of quality? Quote:
|
Re: On the quality and complexity of software within FRC
I've also given the quality and development of FRC controls and software a fair bit of thought. I have also thought for a while about the amount of software discussion versus mechanical, strategic and even business discussions on Chief Delphi.
I think part of it has to do with the lack of robotics programmers, both on CD and in the FRC community as a whole. A lot of teams don't have dedicated software mentors, who can teach the students the principles of software engineering (especially design patterns which allow for easily expandable code), and control theory (at the very least some feedback control, but the more the merrier). Most teams would be lucky to get a computer science teacher who generally teaches the basics (which are important, but doesn't necessarily lead to good software design). This leaves the upperclassmen, who may not have years of programming experience joining the team, to teach the lowerclassmen, with leadership changing every year or two. A reason for a lack of mentors with experience in robotics software may be due to the lack of exposure of computer science and software engineering students to the field, usually until grad school or dedicated robotics programs like WPI or Waterloo have, which don't have a huge output of people. And the difference between robotics software from what CS and SE students typically work with that plunging in head first is intimidating enough that they never give robotics much thought. Additionally, I bet a lot of students use other websites not only to help debug their code, but to copy and paste code from a website, which doesn't lead as much learning. I would like to see somebody try and copy/paste a CAD model. |
Re: On the quality and complexity of software within FRC
Quote:
The notion of adapting to unforeseen use is interesting to ponder. What does it have to do with software that isn't required to adapt to unforeseen circumstances? In that circumstance creating adaptable software is a waste of resources. I read that book long ago. No need to repeat that. Instead why don't we discuss what our software is supposed to do and be *before* we decide that it is a failure. The software development industry is no stranger to the importance of putting a good specification in place early. I understand your thesis; but I don't think you have proven it. Until you can put together a good argument that the majority of FRC Software doesn't satisfy teams' requirements, put me in the camp that both says there is always room for improvement, and says that a general indictment is unwarranted. |
Re: On the quality and complexity of software within FRC
Help me out folks, if a team's software isn't made public, isn't it supposed to be rewritten from scratch every year?
And even if all of a team's software is made public from year to year, isn't the general notion of the annual FRC learning experience that software, like the physical robot, will be largely developed anew, during the season, each year? I realize that there is a broad spectrum of ways teams actually do their software development; but isn't the clean sheet of paper, plus some true off the shelf code, sort-of the big picture thrust of the rules? |
Re: On the quality and complexity of software within FRC
Quote:
I don't see why you would ever start completely from scratch - things such as path planning, vision code, state machines, interfaces, dashboard tools, etc. can be developed and refined over years, and generally can't be done well in 6 weeks. Re-deriving your algorithms every year might be a learning experience, but it won't teach you as much as refining and expanding your codebase will. I think that there are some situations where following this rule and being an effective FRC programmer don't necessarily align. Here are a few theoretical examples where the rule is technically broken.
Quote:
|
Re: On the quality and complexity of software within FRC
On a one-off project, if I have a specific set of tasks and I'm writing serial code that must execute once every 20 milliseconds, why should I care if it takes 2 microseconds or 10 milliseconds to execute?
If I have 512 MB of memory available, why should I care if I use 10 kb or 300 MB, so long as it holds everything I need and works the way I want it to? Particularly in one-off projects where I'm not designing to value, more important than smallest, absolute fastest, most efficient is code readability and "it just works." If I have 10 hours to write and test code to fit a spec, you'd better believe I'm spending about 8-9 hours testing my code to make sure it works. If I spend any time optimizing, that is time not spent testing. I understand that if I'm manufacturing 100 million widgets and I can save a fraction of a cent on manufacturing PCB or hardware specs I'm doing it, but if I'm doing just one prototype, cost is not my motivating factor. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Processing power is cheap and there are so many layers of abstraction these days that optimizing for Big O isn't necessarily where you'll see the most performance gains. From my experience in the manufacturing and software development industries, there is a lot more to be gained from network optimizations than there is from Big O these days. You can optimize an algorithm all day long but if you send it via a crappy protocol then the information might not get there or, more likely, it will arrive late. My point being: Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Quote:
By comparison, other teams will reuse the framework from the previous year, and adapt it to the new robot. It's like starting the season with enough COTS parts to build a drivebase, and the parts increase in quality and quantity each year. Is it ready for competition? No. Does it save development time, allowing programmers to work on other things? Yes. Is their an easily available framework for teams to use? Not really, at least not now. You could borrow code from teams like 254 or 236 who post their code every year, or try and use the scripting languages I've seen 1902, 987, and 4334 post (I think those teams have did that in the past). |
Re: On the quality and complexity of software within FRC
I have two pennies.
I do not think that FRC code is bad across the board. I think the range of code quality is similar to that of wiring and mechanical construction. In other words it ranges from impressively good to -- "are you actively trying to sabotage your team" -- bad. To me, much more important than the code on the robot is the knowledge gained by the students on the team. If students wrote code that has issues, but learned how to identify the issues and improve their performance the next time, then FRC has provided a successful challenge. If they have learned how to communicate and write SW as part of a team, how to break down a problem, complete their solution, meet a deadline, evaluate their results, ask questions, and accept mentoring, then they are well prepared for college, training, and career. To paraphrase, it isn't really about the robot code. That is only the campfire ... And by the way, communication with nonprogrammers is super important. Most programmers don't write code for themselves or for people like them. They write code as part of a product team that includes EEs, MEs, business and marketing, writers, perhaps even doctors or physicist. When you learn to explain what your code does and doesn't do to a nonprogrammer, when you learn to understand someone else requirements, expectations, resources, tolerances, and deadlines in a common domain language, you are better prepared to become a successful programmer. This goes for all engineering disciplines, not just programmers. Finally, I see FRC as a multidisciplinary engineering challenge. It incorporates strategic decision making, logistical efficiency, interaction with real-world materials and tolerances, and interaction with five other robots (most years). And it isn't just a controlled demonstration of capabilities. It is a timed sport-like competition that incorporates human drivers and players , and it also places value on presentation, aesthetics, and outreach. What I'm getting at is that FRC is BIG and complex and really challenging. Code is but one part of the robot. It is necessary for teams to make decisions as to what they will spend time on. There are many many ways to successfully navigate the big FRC ocean. I'd highly encourage the programmers to enter other contests and take on additional challenges. I'd also encourage everyone on the team to learn more about programming. While CS/CE specialists may always be needed, you may also find it very useful as an ME/EE/whatever. I think computers are here to stay. Greg McKaskle |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
Again, I cannot stress enough how much it helps to have a version of code finished before the robot is built. Sure you might not be able to test most of it, but you can test that state machine that would take you hours to make and test while you have better things to be doing. Having code done beforehand allows you to make more advanced code. |
Re: On the quality and complexity of software within FRC
side note that I don't want to create another thread about: I need help deciding on a task of sorts that would interest programmers enough to do it. PM me(or shoot me an email: hunter.park11235@gmail.com) if you're interested in helping me design a task.
|
Re: On the quality and complexity of software within FRC
I have personally been following and teaching this guide.
|
Re: On the quality and complexity of software within FRC
Not all teams have poor software practices. But it is rarely "professional" looking. While helping other teams at competitions I find software all over the place. 1296 splits the application up into tasks/threads for each subsystem and uses messages to synchronize everything. This way the students are all working on smaller self-contained behaviors. We also emphasize good coding standards (including testablilty, reduced complexity, Hungarian notation, etc), draw it then code it, test and retest etc.
Somewhere in this thread it was asserted FRC robots are not all that complex because they "do not process much data". That is kind of a IT way of looking at it rather an engineer's approach. The complexity in FRC robot software centers around temporal constraints, control algorithms and quality assurance. Handling the data to and from the driver station and dashboard is the simple part. Several persons in this thread have remarked about the value of tested code. Code that works is crazy valuable but can be even more so if good practices are followed from day zero. Software has a "life cycle". That life cycle ends if the code is unreadable or not maintainable even if it works. So the holy grail is to follow good practices to begin with, test the heck out of it and always keep a copy of the code that worked! It can be done, even in a 6-week time frame. HTH |
Re: On the quality and complexity of software within FRC
I'll admit that I've just skimmed the responses, so I'm probably going to repeat a few points:
|
Re: On the quality and complexity of software within FRC
Quote:
Also, maybe this is just me but it seems like paying attention to what happens with different forks of code should be the easy part of a software mentor's job. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
#/media/File:Git_operations.svg) |
Re: On the quality and complexity of software within FRC
Quote:
So, if student A uses computer A to work on fixing bug A; and student B uses computer B to work on fixing bug B; and computer C is used to transfer code into the robot; how do the students'results get into computer C (and shared across computers A & B) (during the scarce time available during a tournament) if computers A, B, and C aren't linked by some sort of LAN? Do the students pass around a memory stick? I think this is the scenario GeeTwo is envisioning. |
Re: On the quality and complexity of software within FRC
Quote:
If most software teams have no adult guidance at all that might be the definitive answer to "what's up w/ FRC code quality?" |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
OBTW, I don't see the image. |
Re: On the quality and complexity of software within FRC
Quote:
The thought of having multiple students on multiple computers fixing robot code DURING THE COMPETITION... *shudders* All joking aside, I really don't see why that many people need to be fixing robot code during the competition. There really should be one or two people who know everything well enough to fix it. Heck. One or two people can develop most robot code on their own. It's a good idea to have two programmers on one laptop in a coding pair to fix bugs quicker, but more than that you end up having too many laptops and people in the pits. |
Re: On the quality and complexity of software within FRC
This year, 20 had it's best programming team yet. And they were a team, for once, whereas in previous years we often had one person who did most of the programming on the robot.
We had most functions during teleop that were automated, and advanced controls on our elevator , forks, and drive, and it was exceptional to see them take a less-than-ideal mechanical design and optimize it the way they did. However, I wish we chose a simpler mechanical design, because there was only so much they could do to fix the robot before returns started to diminish. I'm generally a fan of making the mechanical design support the programming. Making the programming the easy part. Making the programming be "When the driver presses the button, shoot", and not "When the driver presses the button, activate the aiming system, then shoot". For example, 20's 2013 robot was built to be as simple a floor loader as possible. We shot from one position on the field, and that was right in front of the pyramid. We had some limit switches hooked up so that when the robot was in position, they would cause some lights to light up, and our drivers knew they were locked in and shot. However, if your team has more programming resources, they could persue a design with a higher ceiling due to programming, like 195's 2013 robot. They were a full-court shooter, and they went to Connecticut with a very good full court shooter and won the event with Team 20. At champs, they were faster, more accurate, and could shoot from more places. At IRI, they had perfected their bag of tricks to include shots through the opponent's pyramid and from anywhere on the field. There is definitely a place in FRC for more complex, elegant programming, but it's not necessary to succeed. Complex programming is a resource, in the same way that sponsorships, machinery, and CAD knowledge is a resource. If you're able to use that resource, that's fantastic for you and your team, you have more options en route to success. If you don't have that resource, you have a limited set of options for success. I know that in 2016, 20 is likely to lean more heavily on our programming team than ever before, because we know they're extremely capable, but in 2017 we might not. |
Re: On the quality and complexity of software within FRC
We use SVN for version control and take our server with us to each competition. This allows us to have as many or as few programmers working on code, if needed, as we want. While we don’t make a huge number of changes it does vary from year to year. Mostly we are changing the robot configuration file to change autonomous mode.
Another reason for taking the SVN server to competition is to keep track and store the robot configuration and log files from each match. Our drive team does a commit to the SVN server after each match, and then anyone interested can do an update and immediately have access to the last match configuration and log files. The students are very capable and able to keep track of using version control. I try very hard to teach them the right way and once they get use to this tool it is one of those things you wonder how you lived without it. Even if you only have one programmer on the team you should be using version control. I tell people it is like having a giant edit undue button. One can revert the files to any previous commit. This is a wonderful feature. We even have our CAD, electronics, & scouting teams using SVN. Why anyone would not use version control is beyond me. It is not that hard and is a tool every student should have in their knowledge base. -Hugh |
Re: On the quality and complexity of software within FRC
I just finished my last year of high school, and have been a programmer for my team during my sophomore, junior, and senior years. I took 2 years of Java courses at my school, including the AP CS exam. We gained our first programmer mentor in 8 years at the start of my junior year, and he was a college freshman at the time.
We've won engineering awards at nearly every single competition (maybe not in one or two of the competitions) we've been to since 2012, which is 8-10. No national champs. So with some decent experience, I'll ask you a question that may give you some insight on the 'state' of FRC programming: What is Big O? I've never heard of it. _______________________________ The fact is - many, many teams do not have any insight to advance programming techniques at all. Many teams don't even have access to the basic fundamentals. For gods sake, this year we started using abstract stuff for the first time. We knew what how it all worked, but we never bothered implementing it. We are one of the few high schools lucky enough to have programming classes at all. A close friend from another nearby team said his school doesn't have any programming classes at all, and they didn't have a programming mentor for multiple years. I'm not sure if they even have one now, but I know they didn't several years ago. How can you expect kids to figure out these 'standards' when they're teaching themselves basic syntax and struggling to understand the concept of the networking system being used in FRC? _______________________________ I think there is a very, very large disconnect between mentors on CD and the students on their teams. On some teams - like many in this thread - there are very, very large expectations to meet software and programming 'standards', whereas other teams do not even have the slightest clue on the 'norms' of this stuff. Sure, you can say 'Get your programming teachers at your school to mentor!' Well, yes. But it's not even that simple. The teachers who teach programming courses here are math teachers and know Java and only Java. Additionally, expecting them to dedicate incredible amounts of time to mentoring is a ridiculous expectation. Mentoring is a LOT of work. We're struggling to find someone who will be able to handle the paperwork we go through each year. We aren't exactly in an area with a booming tech industry, and I'm sure many other teams are in the same situation. Additionally, many teams struggle with building the robot in the first place. Sometimes teams are lucky to test their bot before week 6 begins. It's an immense amount of pressure to get a robot near fully functional in less than a week. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Teachers are one possible source of mentoring, but so are engineers and technicians.
FRC is quite different from how I think of CS classes are normally taught in high schools. FRC is part realtime, part feedback control, and the remainder being composed of scientific/mathematical/engineering tasks such as cleaning up noisy sensors, building state machines, and mathematical transforms to match mechanical or electrical constructions. The robots I see within FRC don't have a database, don't draw circles and squares using a turtle, and don't have grids of rocks, flowers, and whatnot that morph over time. Also, the issues that happen on the robot are rarely isolated. "Our robot code has a problem", invariably involves a wiring glitch, mechanical bind, toasted motor, missing jumper, scratched encoder disc, and sometimes a logic or race condition mixed in. This is incredibly similar to the type of SW written by NI customers, the majority of whom are not CS. So don't limit yourself to computer scientists or computer science teachers. Greg McKaskle |
Re: On the quality and complexity of software within FRC
You know Maslow's hierarchy of needs?
Code:
Maslow's Hierarchy of NeedsCode:
Jared's Hierarchy of FRC Needs for Repeatable, On-Field SuccessTL;DR: Software in robotics is hard because not only is it hard to create good software - doing so is also heavily influenced by every other aspect of a team and robot. There are very few instances of a robot or team where I've thought, "The only thing holding them back is software quality", so I have to agree with Blake that for the majority of teams, the software meets the requirements (though everything can always be better). |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
If this is a serious issue, there may be other version control programs that suit your needs better. I am simply most familiar with Git and its features. Quote:
 |
Re: On the quality and complexity of software within FRC
Quote:
For those who haven't noticed or cared (and... it's legitimate to be proud of not needing to notice or care ;)), I have noticed that the buzzworthiness of knowing and discussing the big O characteristics of various data structures and algorithms for various operations (insert, sort, etc.) has been on the rise for a while because of the money to be made in big data (business analytics, online shopping, social networking, searching, databases in clouds, intelligence/spying, etc. etc.), and it's now quite trendy. It's become so trendy that I worry that folks are losing sight of the differences between efficiently operating on small datasets and efficiently operating on large datasets. For small datasets, low-overhead brute force will often beat the pants off of manipulating some fancy data structure that is appropriate for larger datasets. There is more to writing efficient code than learning the big O characteristics of various data structures and algorithms. FRC has opportunities to for programmers to experience both sides of this fence. |
Re: On the quality and complexity of software within FRC
The one thing people harping about code efficiency and big O functions need to hear is this:
The most important resource isn't CPU time, or even the money to run a team. It's time. There are only so many man hours a person can spend doing something. (obviously varying from person to person) Money can be acquired through time. (People can spend time fundraising) CPU time can be bought with money (Just buy another onboard computer for the second camera.) If it's going to take 50 hours of work to get your vision tracking program optimally threaded for all 4 cores, for the sake of saving 200 dollars on an onboard computer, you are literally working for less than minimum wage. Now if you're writing code for mass produced industrial equipment, you're no longer looking at 200 dollars, likely 200,000 dollars, or 2,000,000 dollars. Then it would be worth it. You might also want to learn how to write threaded code well. In that case, it becomes worth it for the educational value. FRC robots are not mass produced, and most FRC programmers are still worried about getting EVERYTHING working in the several hours they have with the robot. Out of all the ways FRC programmers can spend their time, code efficiency is the least efficient way to spend it. |
Re: On the quality and complexity of software within FRC
It took me less than 30 minutes to properly thread my vision code in 2013 on a quad core sbc....
But that's not the point. It's that frc ultimately fails at demonstrating what computer science really is. |
Re: On the quality and complexity of software within FRC
|
Re: On the quality and complexity of software within FRC
Quote:
...and know the math behind all that, which can be worse than the math used to compute the various items I just mentioned. Oh, and the electricals would need to be able to understand the guts of the electrical devices, which as I recall can get into Diff. Eq. As I recall, that's barely touched on in H.S. calculus. Plain and simple: Part of any (and I do mean ANY) engineering project is to deliver a product that works, on time and on budget (and on weight). Comp Sci is very much an engineering field. If you have the time to go above and beyond, nobody is stopping you from going above and beyond, provided that the product (your code) works (read: runs the robot) properly. I think Randall puts it very nicely in 664... |
Re: On the quality and complexity of software within FRC
OK, I'll bite.
Quote:
Quote:
What is computer science, really? |
Re: On the quality and complexity of software within FRC
Quote:
I do not have a firm definition of computer science, as I do not have one of mechanical engineering either. But I'll give it a shot by combining various definitions I've heard from my professors and read over the years. Computer Science is concerned with information in much the same sense that physics is concerned with energy; it is devoted to the representation, storage, manipulation and presentation of information. Computer scientists deal mostly with software and software systems, which includes theory, design, development, and application. Let's dissect this in terms of FRC: Representation: An interesting survey would be to see what percentage of frc programmers know how various data types are stored in the computer (and the rough range of them) Storage: Beyond the occasional class, data structures such as queues and trees are not used in FRC to my understanding Manipulation: Manipulating ints is as trivial as tightening a bolt, manipulating complex data structures is another thing Presentation: Besides graphs and printing to console, how are teams presenting the data they have when they want to? Theory: Do teams take to the calculate the complexity of their problem before they solve it? Design: (Serious question): Do teams take to the whiteboard and draw out their algorithm, like this famous one? Development: There are many theories about software development so I won't touch this as to not anger people. Application: Does it solve what it is suppose to? I feel in general this is the one area FRC meets. Side note: For those who say why bother about efficiency if what is being done isn't complex to begin with. It's about learning and developing good habits. Programming habits are some of the hardest to break in my opinion and it takes self discipline to do things right, and not what is easy (Insert jfk speech). When you are constantly worried about optimization, you begin thinking like a computer scientist. |
Re: On the quality and complexity of software within FRC
Quote:
Why this matters: Quote:
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Using a red/black tree to sort a list if ten numbers is less efficient than bubble. Coding it is only more efficient if you're regularly going to sort hundreds or thousands of numbers. When we started the team, Gixxy (our first program lead, now a CSCI major, and fascinated with computers since before he was two) was planning to write this big code base to carry from year to year. However, most of the FRC reusable code is already in WPIlib or LabView. Unless you reuse hardware (or at least hardware design), the software is also going to be a fresh start. In four years, I believe that we have reused vision processing code (running on a raspberry pi), the threaded connection to the pi, and an xBox controller front end. We have borrowed old drive system and manipulator code to build new, but we followed up with such massive modifications that I wonder whether we saved more time by reusing than we spent in fixing "old code" problems. That is, the main reason most programmers practice good coding (that is, someone will have to maintain it ten or more years from now, maybe me) is not a valid concern for deck plate FRC programming. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
On top of a lot (probably most) teams not having a programming mentor of some sort, I would venture to say that most of the programming mentors out there have no formal training in controls.
|
Re: On the quality and complexity of software within FRC
Quote:
All this talk of optimization is great but I haven't seen anyone in this thread talk about profiling. It's been a while since I took the compiler optimization class that I took in college but as I recall, there were some choice readings by Donald Knuth that pointed out that pre-mature optimization is the root of all evil or something similar. Profiling helps to determine where to optimize systems and with FRC robots containing a physical component, profiling would entail looking at the code as well as the robots output and the speed/efficiency of motors and gearboxes... which as others have pointed out, is more likely to provide substantial efficiency gains. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
For passenger airplanes, spacecraft, nuclear plants and warheads, and medical equipment... not so much. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
"That'll never happen" has become an inside joke. |
Re: On the quality and complexity of software within FRC
Quote:
Quote:
But I sure hope you guys aren't suggesting that should be the "hear no evil, see no evil" attitude of the lead project engineer of a project where human life hangs in the balance. It's certainly not the way I ran my projects. |
Re: On the quality and complexity of software within FRC
Quote:
I'm saying that you shouldn't be spending a bunch of extra time optimizing loops and building adaptable code with the assumption that you'll need that optimization or need to adapt it in the future. In most (not all) cases it will be a wast of time and money. |
Re: On the quality and complexity of software within FRC
Quote:
... never. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Quote:
Quote:
Quote:
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
"Waste of money" sounds like you are talking about something other than just an FRC robot. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
OP wanted to raise the bar for the FRC software quality.
What are some ways to do that without being too preachy (and without getting dragged into the weeds by topics like what-is-the-the-one-true-code-formatting-style, or the-one-true-way-to-use-a-Hungarian-naming-convention? Publish some reference designs (several... the number of good ways to do things will be legion) that guide students, and lead them to ask good questions about details, but that don't hand them answers on a silver platter? Students are given physical kit-bot parts. Maybe the kit-bot BOM should include some software parts they can put together to form a basic FRC software system (does this already exist?)? Perhaps put a few good examples of software requirements specifications in the Kit of Parts? Create simulators (that expose the appropriate APIs) that students can use when their own team's real equipment is unavailable, or during off-season practice sessions, thereby giving them more development time during build season, and more practice time before build season? Something else? Blake |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Simulation encompasses a wide spectrum of approaches, from mocking speed controller class interfaces all the way to doing a full dynamics simulation. The former is useful for debugging logic errors; the latter is required (to some level of fidelity) to actually do closed-loop testing of the program. This year 254 did a little of both for developing and debugging our control algorithms and designing our can grabbers (however, our approach was strongly tied to our use of Java...we built a "fake" WPIlib JAR and swapped it out to do simulated tests). The problem with simulation beyond just mocking low level interfaces is that teams now need a way to specify their robot configuration to the simulation. This is tedious and error prone in most cases, and very difficult to do accurately (e.g. estimating friction, damping, bending, or inertial properties of robot mechanisms is hard). Even professionally, I've watched many PhDs lose hours of work having to debug configuration issues in their URDF files (a common format for expressing robot topologies). The best solution for FRC would be to provide examples for common FRC mechanisms and COTS drivetrains and let teams go from there...but I worry that the complexity gets large so quickly that if a team can navigate that, well, they are probably not the ones who REALLY need programming help. |
Re: On the quality and complexity of software within FRC
Quote:
|
Re: On the quality and complexity of software within FRC
Quote:
Some of the things that I know are fine, but still make me cringe, involve the use of generics and enums. Since Java on the CRIO did not support either of these, most of the Java code doesn't have them. However, you can tell that some of the new classes do use generics and enums. I know this is fine, but you can tell there is a disconnect between the old and the new code, and something that would be nice would be for somebody to take a month and thoroughly clean it up. I bet with some cleanup to include new features, and maybe some algorithm refactoring, we could get the code much nicer and easier to work with. Maybe they could fix all the spelling and punctuation errors in the comments too :D |
| All times are GMT -5. The time now is 01:41. |
Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi