How do YOU develop software?

I thought it would be fascinating to look at the different processes people use to develop software, to see how it varies.
IMPORTANT: I’m not talking about what you would recommend as a developing process. I’m asking about what you actually use. Think about a recent project you’ve done - what were the steps or stages you went through between when you set out to develop the software, and when you considered it a finished product?

Recently I set out to test some SHARP IR sensors. I was having trouble getting accurate distance based off the curve on the datasheet, so I decided I’d test the the voltage-distance curve to see what it actually was. (This was also an excuse to try out my new NI myDAQ.)

The first thing I did was define the features I wanted:

  • once-click sample
  • show history of sample: mean, max, min (graphed)
  • sample 100 times per datapoint
  • show graph of most recent samples
  • auto-increment option
  • log data to file

I then built my user interface (front panel): [first thumbnail]

Then I had to figure out the structure of my top-level program. I made a table to describe this. [second thumbnail]

Because some user actions cause multiple program actions, I chose a queued state machine for the structure, using an event structure to catch the user events and translate them into program actions.

In actually coding this, I make the structure first. I made the event structure, and set it up to catch all the events I wanted. I made an enumerated variable with one value for each program action, and a queue to pass that data. Then I actually set up each vase in the event structure and the case structure. There were only two instances where I felt I needed subVIs: in the data acquisition, and in the data logging.
So, I put up flat sequence structures as dummies, and finished the rest of my top-level VI.
Here’s what it looked like: [third thumbnail]

So from there I made the subVIs. The logging subVI is actually not yet existing in this initial version, but it was complete enough that I could start improving the UI and catching bugs. The reason I still have this initial version at all is because it was the point at which I committed to my Git repository.

So, in between the initial version and the final version, I made some changes. I’ll be perfectly honest - I did not update the planning documentation when I did so. I use the planning documentation to guide me in how to create the software, but I don’t usually create end-user documentation.
I did some minor changes, like adding key bindings to the “sample buttons”, and getting rid of the “log name” in favor of just using the “log filename”. However, my major changes rooted from making logging more practical. I had two different SHARP IR sensors: a short-range one and a long-range one. I wanted to try an algorithm to use them together, to get more reliable results (their voltage-distance curves do not pass the horizontal line test - for most voltages, there are two possible distances). I had been trying to figure out a good logging scheme where I could combine the logs of the two sensors in one file. Ideally I would be able to read them together. I then remembered that I could simply log them together, so each would have data at the same distance points. That was the first major change - to test both sensors at the same time.
The second major change was in how I stored the data. I had been storing it as a cluster of arrays - well organized, but very inneffecient, because each array can be a different length. I realized I had to convert it to a two-dimenional array to log it anyways, and so I changed that to a two-dimensional array. This time, I made it a typedef, so that if I made further changes, I wouldn’t break my subVIs.
I think the only change I made after this was to let the user keep trying to save the log until it worked.

NOTE: I programmed this in LabVIEW 2010, because that’s what I got with my NI myDAQ. It is backsaved to LV 8.6. I can open it on my computer, but there may be some missing dependencies. If so, just click ignore, and you should be able to see most of the code.
Also, I actually do my documentation on paper. I can think better that way. However, for the purpose of posting this on a forum, I opted to type it in, rather than scanning my messy notes.

So there you have it: a very small project, completed in about 6 hours spread over two days. I think it’s a fair representation of my software development process.
But what about you? What steps do you go through when you develop software?


table_for_planning.png

initial version.zip (131 KB)
final version.zip (147 KB)



table_for_planning.png

initial version.zip (131 KB)
final version.zip (147 KB)

Depending on the application …

  1. Identify Hardware
  2. Identify Requirements
  3. Build Back End Modules
  4. Build Front End Modules (not gui)
  5. Build GUI
  6. Build Top Level Modules

1 - Simple. Identify what you need or at least what you plan to use.
Some questions I ask myself:
How is it going to be used? What makes it tick? What do I need to note from a program standpoint?

2- Identify what the program needs in order to work properly. Not “Item A needs to be wired to item B” But more along the lines of “I need to sample this at this interval”.
Some questions I as myself:
What are my inputs? What are my outputs? How do my inputs affect outputs? What needs to be done on a dumb level? What requires more brains or objects to program?

3 - Decide what is a back end function AKA building blocks. And start building them. As I build each piece I write a test program(s) to test the range and all functions associated to it. Once each block is built I write a master tester that tests everything AT THE SAME TIME to make sure it functions properly with everything else.

4 - Build front end modules. These are more along the lines of Input A causes behavior B. As I build once again I test to make sure behavior is met. Typically this takes the longest time to build. Once all is built I TRY TO BREAK IT (eg mashing buttons, providing false/corrupt input, disconnecting essential things [WHERE DID MY JAG GO!])
Some questions I ask myself:
What can break? If A breaks what can it affect? How to bypass bad behavior from broken inputs? What functions should be automated? What functions should have manual override?

5 - Usually ask the end user for items they want or decide for them. Although uncommon there are modules built here that regulate the front end modules as well.
Some questions I ask myself:
How can this be made the easiest for the user? How can X be taken care of automatically? How many presses/clicks does it take to do X? What information does the user ALWAYS need to see? What information does the user NEVER need to see? What information COULD the user see if it is out of range? (For example I could have a sensor value popup ONLY if it out of range)

6 - Usually along the lines of AI
Questions I ask myself:
What would the user do in the situation? Could I just record a response for this situation? How should the program change behavior if X Y Z? What can the program do FASTER then human? How can that be used to its advantage?

From a robotics standpoint:

I have everyone as a group do 1 and 2.
3 is left to newbies (teaches responsibility) with guidance of vets.
4 is usually done by vets with newbies learning.
5 is usually done by vets/mentors if time allows.
6 normally happens at competition :yikes:

I typically will start out with a ‘dream list’ which is where I define everything that the code should ever do. Then I cut that down to what is core functionality - the stuff that absolutely cannot be dropped.

THEN, I actually look at what hardware I have available and start engineering the interfaces to handle the functionality in the code. Lastly, I start fighting windows and bugs (in that order). That last portion usually takes up about 90% of the time allotted for the project.

Could you go into a little more detail here?
What’s the process you use to make the code once you know the specifications? (Making the specifications appeared to be your first step, and your last step appeared to be “find bug, fix bug”)

I personally need a lot of paper when designing code, especially robotics code. The order I think in goes as listed:

  1. Get the goal, what is the final product expected? What is the hardware and how will it be used?
  2. Write (yes write) pseudo, you will be shocked how many errors I find when I write out a rough map of what im doing. In this rough map, I dedicate a page to describe variables and methods with a little reason as to WHY I need it. I can’t expect myself to come back to work after a break and have the same mindset as when I left, so I need a refresher.
  3. Collect data. Chances are, the task has a little hole where I say “I need a tad more info here.” Like in the case of calibrating a PID with pots, or maybe my knowledge about a structure or rebuilt method isn’t 100%. Naturally, I do static tests and research to plug these holes. Graphs, charts, you name it and I probably have used it to collect data and weigh options.
  4. Fill the pseudo with real code, WRITTEN in whatever language im doing.
  5. Break my code into small test bites and hand check each bite before testing the whole. I determine bites by ordering the final code’s functions from really basic and easy to test with less risk to very risky and hard to test. As an example in robotics, communication first, then drive, etc.
  6. Type the code in order from previous step and test after each addition. During these tests, I write down what happened, in case I forget or whatever.
  7. When the code is all done, release code as first version and implement into final product. Repeat for patching.

Personally, this all breaks down when I use labview because it is difficult for me to put my code into the pictorial form. I know how and why things work, I have done java for 4 years and have recently been exploring the boundaries of c++, but even after 3 years of robotics, labview is still hyroglyphics!

Like a boss.

Actually, I’m the pseudo coder, so i can’t say anything about actual coding, but in the role of a pseudo coder, the first thing I do is read over the rules for limits. I like to know how high I can make the robot go, how far extended, and all of the info needed to write a perfect pseudo code.

Egg, your process seems pretty adaptable to LabVIEW.

For most good LV users, the design process generally mixes pseudo dataflow and pseudo procedural descriptions. The design is often done at a whiteboard, but paper works fine. You start with where the data starts, and write that on the left, then think about what happens to that data next and draw an arrow from the source to the processing step, …

The text descriptions will often take on the form of (process array comparing to previous, calculate arm setpoint, etc. ) and that will later turn into a subVI or a loop of code.

For FRC, the teleop would start with a joystick read with specifics of joystick number, axis, button numbers, etc. The next box of description might be about identifying button combos, latching, timeouts, and mapping joystick 0-1 values to other ranges. Next it would either update setpoints on motors, push them to other loops running independently, or store values for the next iteration.

As with pseudocoding in other languages, syntax isn’t important, short-hand is good, and the semantic comments, structure info and literals are some of the more important values to hilite. A potential benefit of dataflow languages is their focus on the data being processed, stored, etc. Those transfer mechanisms and flow are a validation that the processing is working on the right values. Finding that in procedural langauges is also important, and often more tedious to locate.

After doing this for awhile, I normally don’t go to the whiteboard. I just lay out a big LV diagram and use the comment tool and some of the loops and single frame sequences to describe the structure. And of course I pseudocode less than I used to. These days, it is generally to document or explore an algorithm with others.

Hope this helps to see past the hieroglyphics.
Greg McKaskle

  1. Identify the primary reason for the software, and the flow of how it works
  2. Identify any ‘fringe’ things (GUI, file IO, etc) that are usually already developed as other tools in some capacity, yet will need to be integrated in
  3. Develop a proof of concept for (1), with plenty of debug statements.
  4. Iterate over the flow, using refactoring techniques, until the primary purpose of the software is complete and efficient. Create plenty of hooks, even unused ones, that allow flexibility in control of when
    methods are called (getter methods, calculate methods). 1. Develop the structure and control of the main program that the user will see, if it’s meant for someone else.
  5. Develop fringe things. Basic GUI, FileIO to save settings, etc. This is actually the most time-consuming and least-fun part, yet it’s typically what the end-user complains the most about.

For example, I’m developing a (soon-to-be) open source scheduling algorithm that is supposed to be flexible enough to schedule matches for FLL, FTC, and FRC. Its primary purpose is to develop a schedule subject to parameters such as # of teams per alliance, # of alliances, & min/max match spacing. Honestly, if match spacing wasn’t an issue it’d be trivial to make a scheduling algorithm ><. The fringe features are “Alliance Insertion” (pair up titans versus titans, subject to their consent) & scheduling for judging after the match schedule is done. It will have FileIO in case the program quits in the middle of a competition, and a GUI with which to interact.

  1. argue with build team for an hour trying to figure out what they want me to program

  2. (if starting from scratch) get the drive train programming done and turn as much as i can into a subvi

  3. attept to test drivetrain

  4. argue with build team again for giving me wrong specs

  5. fix drive train

  6. repeat steps 2 through 5 up to 9 times

  7. start making code for any other manipulators and turning any similar code into a subvi

  8. test and argue with build team again

after that it’s just fixing everything they decide to change XD

This is copy-paste-modified from a chat log I had with a teammate about the exact same process:

I normally start by defining the problem - for robot code, I make a list of all the different sensors, motors, etc. and group them according to functionality - the four drive motors would be one section of the code, for instance. Then I would go create the “essentials” - the bare minimum needed to run. After that I would work on ISOLATED subsystems - that way they won’t affect each other, so your code (shouldn’t) break other code. Then I would fill each subsystem out to “working”, although more like “working and safe”, and then finally advance to “polish, enhance”.