Software Team Organization

[Orborde]

After a year off of FIRST, I'm back, now as a mentor at Purdue University (assigned to team 1747) and I'm pretty enthusiastic. Given how disorganized software team tends to get, I wrote up the following thoughts on how to keep the software team organized and in good communication with the rest of the team:

Proposed Software Team Organization Stuff

As a programmer on 1024 for two years, I ran into a lot of problems keeping the software team updated to the current robot state. Also, code got disorganized, misplaced, impossible to locate when someone needed it, resulting in someone with only a dim idea what they're doing trying to hack something together 5 minutes before a regional starts, and general chaos. The following is a proposal to keep things organized and in the realm of engineering rather than randomness.

1. Set up a source code repository (probably using Subversion) on either its own server at the school or on one of my own machines in my dorm room (which have globally visible IP addresses).
- Each of the programmers will have a username on this system. There will also be a "general team" account for the people that will inevitably want to change things around on the robot without telling anyone. The repository may be configured to automatically email the entire programming team whenever a commit is made from the "general" account.

2. The revision control repository will be automatically mirrored and backed up nightly to several different places, including:
- A backup server somewhere (possibly the mentor's laptop, if nothing else)
- The mentor's school home drive
- A full copy of the repository, along with an archive of the latest trunk version and other branches, will be put up every night at a certain web site (likely the mentor's or the lead programmer's), and the address made known to EVERYONE. This will ensure that, as long as we can get internet somewhere, we can get our current code.

3. Print up "information from the programming team" flyers and post them conspicuously around the build zone. These flyers may include:
- "General commit" username, password, and the hostname of the repository.
- Since we understand that the non-programming people may be less than enthusiastic about messing with the revision control system, there will also be a "programming team official contact" email (either that of the programming mentor or of the lead programmer).
- Programming team cell numbers
- There will be a notice in VERY VERY LARGE LETTERS that ALL CHANGES TO THE WIRING OF THE ROBOT MUST BE COMMUNICATED TO THE PROGRAMMING TEAM SOMEHOW; a simple email to one of the "contact" addresses, at the very least.
- The address of the web site containing the globally-visible current source tree archives mentioned above, along with a description of what to do with it.

4. The interface to the robot hardware will be abstracted through a header file and DOCUMENTED. The "interface" header file will contain macros mapping things like LEFT_TREAD to pwm01 (for example), along with an explanation of how to use it (such as 254=forward, etc.). There will be NO direct access of any variables (such as p1x) ANYWHERE in the user code.
- The interface information will be printed up (possibly in a spreadsheet) and placed next to #3, with a warning in very large letters written on it that anyone who deviates from the listed interface without telling the software team will, in polite terms, be shot.

5. The software team will meet regularly with the heads of the other technical teams to maintain good communication.

The above are the primary goals I would like to see fulfilled in the organization of the software team. Some other ideas that may or may not come to fruition:

6. Automate, automate, automate. The mentor (and, with luck, the lead programmer) has enough UNIX capability to implement many scripts to keep things in order in the code tree. Some things that could be done automatically are:
- Scan through every commit to make sure that the "no direct hardware access" requirement in #4 is not violated. If it is, the system will allow the commit but send an email to the person who made the commit pointing it out. If it is made from the "general commit" account, an email will be sent to the programmer elected to fix these sorts of mishaps.
- Run a nightly build of the source tree (as many real software houses do) and email someone if it fails.
- Write "testbed" C programs that pull in chunks of the source tree for unit testing against various inputs and report the results.
- (This one borders on fascism) Keep file-by-file reports of "lines of comments" vs. "lines of code", and have the mentor look into files that have low comment-to-code ratios.
- Any other clever automated testing or checking we can think of. Obviously, we don't have much need for memory leaktesting; what else?

I decided I'd run this by you folks here at CD before inflicting it on the students. So what do y'all think? Is this overkill for a FIRST project? Has anyone else tried any of the things I've mentioned on their own teams? Have any suggestions?

Thanks,
-William

[Orborde]

EDIT: I concluded that this was off-topic. Now where's that delete button...?
EDIT OF EDIT: Whoops...someone replied!

[jerry w]

this is very well written. i have been trying for 5 years to create this nearly identical organization with our team. last year we came close. the returning students this year are actively working to implement this organization.
we did not have warning banners. this year we must warn the electrical team to avoid the constant changes!

thanks for the clearly written statements.

jerry w

[Orborde]

Originally Posted by jerry w

this is very well written. i have been trying for 5 years to create this nearly identical organization with our team. last year we came close. the returning students this year are actively working to implement this organization.
we did not have warning banners. this year we must warn the electrical team to avoid the constant changes!

thanks for the clearly written statements.

jerry w

Does your team keep a centralized code repository? If so, is it on a school server or someone's box at home? How do you deal with keeping the code somewhere near reality?

[Cuog]

Im not sure if you mentioned this already but it deserves repeating(I only scimmed after reading half). MODULAR. Use alot of small Functions written by members that do very specific tasks. Don't be affraid to have call trees that are very long. It will save time and recoding later.

Heres an example:

User Routines();
>void Map_Joystick_to_PEM();
>>unsigned char Lookup_Map_Curve(unsigned char Joystick_Value);

And all that stuff because if anyone needs that calculation one call
can save each person the time to write the code that does that same thing three different ways. This again saves on debug time, since we all don't have enough time anyway.

[Dave Flowerday]

Originally Posted by Cuog

Use alot of small Functions written by members that do very specific tasks. Don't be affraid to have call trees that are very long. It will save time and recoding later.

Be careful with this. Having very deep call trees can cause you to overflow the stack, which is not very large in an embedded environment like the IFI controller (and this is a real pain to debug). Also, remember that there is overhead with each function call, so if you go overboard with small functions not only do you risk stack problems but you're also likely using up more RAM and CPU cycles than necessary.

Most experienced software people will tell you that there is a 'sweet spot' for complexity of functions. If you don't simplify enough, you have huge functions full of spaghetti code. If you simplify too much, the code can also be difficult to read because there's no easy way to see what's going on overall. I'd suggest that functions which contain the primary logic of your program should be somewhere in the vicinity of 25-75 lines or so, with various 'worker' functions which are frequently used which might only be 5 or 10 lines. And as always, remember that comments are free Use them often so when you return to the code next season you have some idea of what you were thinking at the time.

[Orborde]

Originally Posted by Dave Flowerday

And as always, remember that comments are free Use them often so when you return to the code next season you have some idea of what you were thinking at the time.

Assuming you can FIND the code next season...

I see your point about the call trees - on the other hand, we shouldn't be writing any recursive functions here; have you ever actually hit this wall in practice?

I thought the MCC18 compiler supported an "inline" keyword; does it, or am I just hallucinating?

Thanks for the additions. Any ideas on the "improving communications" aspect?

[Donut]

This is a good idea that's been tossed around on our team before, but unfortunately none of us have the time or experience to really implement it.

If you're looking for simple alternative methods (or anyone in this thread is), here's what worked quite well last year for us (note this is on a team where all of the coding was done on 3 laptops, since we couldn't install programs on the school computers):

1. Make a new project everytime you have a major code change (in case it ends up not working and you have to revert). Try to keep a simple naming system that makes sense; we had "Base 01", "Base 02", etc. for our beta builds, and "Release 101", "Release 102", etc. for our official tested and working builds.

2. Try to divy out the different tasks to a few different groups, so that each of you can produce different files and then merge all of them together in one super project. This eliminates alot of the over-writing each others code problem since you work in entirely different files for most of it.

3. Make sure the code is updated on all systems at the end of every day. In our case, we uploaded the most recent version of each file onto a flash disk in one updated project, then uploaded this project back onto all the computers. As an alternative, have each group work in different builds (group programming autonomous in base 01, group programming drive in base 02, etc.), and upload the most recent version of each project onto each computer.

4. Have one person focus on making sure everyone elses' code will fit together and that everyone is in a group doing something, a "project manager". They might also want to keep a running Read Me file as to what has been added to a project since the previous version.

5. Set a meeting with the electrical team heads, and make sure all wiring decisions are settled and documented in something both groups have a copy of. Create an alias file that is very easily changable in case someone doesn't adhere to this code, and you end up having to change a wire or 2. If you have the programs to, try to have a visual wiring diagram that shows what everything is going to (if desperate, powerpoint works nicely).

That's a general gist of what we did last year, and we should be using about the same method this year. Not being able to use school computers can be very restrictive upon your team, but it also forces you to really know where everything is since you can't have a network sitting there to store everything.

[Dave Flowerday]

Originally Posted by Orborde

Assuming you can FIND the code next season...

I think you already have the answer to that. Use Subversion. We keep all our code in there. As a rule of thumb, for the most part commits are only made when the code is tested and working (exception is when two different people are working on separate parts and need to merge them, then one might commit non-working code so that the other can merge and commit the merged working version right after).

The other thing we do is keep what we call the "motor spreadsheet" in Subversion. This is an Excel sheet that contains information on all the motors, sensors, etc. in the system and indicates what inputs or outputs they are hooked to. The electrical team maintains this sheet and notifies software if they change anything. Also we tend to stick with the same assignments year-to-year as much as possible (for example, drive motors are PWMs 1-4, etc). Once a motor or sensor is assigned to an input or output, it usually stays there for the season unless there's a really good reason to change it.

[chris31]

Originally Posted by Dave Flowerday

The other thing we do is keep what we call the "motor spreadsheet" in Subversion. This is an Excel sheet that contains information on all the motors, sensors, etc. in the system and indicates what inputs or outputs they are hooked to. The electrical team maintains this sheet and notifies software if they change anything. Also we tend to stick with the same assignments year-to-year as much as possible (for example, drive motors are PWMs 1-4, etc). Once a motor or sensor is assigned to an input or output, it usually stays there for the season unless there's a really good reason to change it.

We did that last year and I must say it is very helpfull to the programmers to know what is what after the electrical has moved stuff.

[Qbranch]

Originally Posted by Orborde

Another idea I just had:
Most of us here know very well how the IFI compiler doesn't promote calculations to INT automatically and how many bugs it has caused. Would it be better to simply throw the "always convert to INT" flag in the build options at the outset and not think about the problem any more?

To clarify, the compiler is in no way associated with IFI. The only ifi-written component in the software writing process is the IFI bootloader.

The Microchip Compiler for 18F series parts (MCC18), works very close to the hardware and isnt like the average compiler you run in to for targeting JRE/windows/linux/unix operating systems. There is no flag to 'always convert to int' BUT you could write a compiler macro to always typecast all your calculations into the 2byte (integer) format.

Just be advised that this will waste memory, and unless the variables are unsigned will make bit shift division difficult.

PM me if you have any questions.

-Q

EDIT: The post i replied to is no longer in this thread. I realize this post is now off topic.

[Astronouth7303]

Even with our small team (myself), I have been doing a lot of that for some time. Especiall #1, 4, 5.

It's notable that you almost have to make an exception for #4 for interrupts. (Still use the alias for value access, but you don't have to alias all the configuration bits.)

The lines of communication between the software group (me) and head of electrical (Biff) have been open. We have always worked closely.

One thing we have gotten to doing is to define what each of the IO pins are, and how the sensors are wired to those pins. eg, have pwm01 and pwm02 go to the driveline and digIn1 and digIn2 go to the encoders. Define which way pots go. (I recomend the same direction for the pot, the motor that drives it, and any control board for it.)

BTW, Subversion can fail commits with helpful messages. (See the mime types commit hook floating about.) You don't have to email a team member to notify them of bad practices.

All-in-all, very good.