Hi CD! I am the lead programmer of Team 3019 Firebirds, and I have a few questions about software such as GitHub and Jenkins- Our team has only recently (starting last year) been using GitHub to keep track of revisions throught EGit. One of the issues we had was that we couldn’t really have multiple people working on code at the same time, and since both me and the previous head programmer were almost always available to work on the code at the same time, it wasn’t a huge issue. Now, some of my programming oriented mentors are bringing up software like Jenkins. I don’t know much about the Github process and only really used it rudimentarily to keep track of versions and update code gradually, as well as make it public. Can anyone give me a rundown of how GitHub/Jenkins are useful specifically with robot code organization? My programming “experience” mostly consists of just programming the robot in Java with Eclipse, so I have barely any concept of revisioning and how GitHub/Jenkins works, and why they would be helpful to the development of robot code. Thank you all for the advice!
I"m happy to hear you are using version control. In addition to letting multiple people work on the code base, it provides history. Which saves your bacon if there is a problem!
Jenkins is a Continuous Integration server. What you do is set up a job on Jenkins to automatically build the code every time someone pushes to the repository. Building consists of compiling, running any tests you have and optional other steps. There are two ways to configure this. You can use polling where Jenkins asks git periodically if there has been a push. Or you can use a git hook to have github inform Jenkins about the change.
The words I underlined in the previous paragraph are concepts and apply to other tools; not just Jenkins.
Consider using Travis CI instead of Jenkins. Travis is free for open source projects. Many FIRST teams share their code and you even said yours is public; so you’d qualify to use it for free. The advantage of Travis over Jenkins is that you don’t have to install a CI (Continuous Integration) tool nor do you have to maintain a server or pay for cloud hosting. Travis also integrates with github out of the box. It’s not hard to configure Jenkins to do that regardless.
Jenkins is a product that keeps builds in one place and self tests them to make sure they’re working properly to put it simply. It’s not something I have much experience with either. However, I’m not sure why only one person at a time could edit code? GitHub offers free organization accounts with unlimited private repos and seats for FRC through their education program which I would recommend taking advantage of (included in the virtual KOP). I still think GitHub is quite a bit more useful than Jenkins in an FRC setting personally, which is why my team still uses it. Both can be used simultaneously, but Jenkins requires use of a server to run its tools.
Edit: As Jeanne mentioned, Travis CI is an awesome tool too that’s free. She’s likely much more knowledgeable in this area as well, so I’d direct questions at her and not me.
I interpreted the one person at a time thing to refer to before they used GitHub. I absolutely agree there is no user limit for GitHub repos (regardless of whether using public/private/organization repos.) And yes, I do a lot of DevOps work at my real job. So I do have above average knowledge in this area :).
Aventek: Note that Jenkins is not a source code repository (nor is Travis). You use them in addition to Git/GitHub - not instead of.
Jenkins is a “Continuous Integration/Continuous Deployment” environment/tool. It is usually set up such that it will monitor your source control system of choice (Git, Subversion, etc.) and when it sees changes it will sync those changes and build the source code, and if you have them it can run tests against what it builds, and if all of that succeeds, then Jenkins can be configured to deploy the resultant build. Even if you only configure Jenkins to build changes as they come in, that alone is useful in making sure check-ins to your source control system of choice are actually buildable and not breaking.
I have experimented with Jenkins in the last couple years with FRC C++ code, and have found that the way things build (at least for C++) is a bit cumbersome and awkward in Jenkins. I did have this set up before FRC moved to the RoboRio, and it was much easier.
Git is a source control system, that should allow multiple people to work on the same code base at the same time. Other source control systems are Subversioin, CVS, Perforce. They each have their advantages and disadvantages and won’t go into them here. The obvious benefit to a source control system is that it tracks change history so that if you change something, and things break, and you can’t remember what you change, you can look back to see what changed and who made the change. I am not sure why you couldn’t have multiple people work on the code base at the same time.
Other people have done a great job of explaining what Jenkins is, so I’ll skip that part Instead, I’ll explain how we use Github and Travis, and what our development process is like.
We have ~15 robot programmers, and one programming mentor. Of the programmers, 5 are experienced, and the rest just joined the team this year. We usually are developing somewhere between 5-10 features at any given time.
We use GitHub to store our code. Every programmer has a fork of our robot code repo. When someone starts to add a feature, they checkout the latest code from our main repo, and create a branch on their fork (In actuality, branches don’t “belong” to a specific fork, but that’s a kind of technicality). They add whatever files are needed to add the feature, and commit them to their branch. With so many people working on things at the same time, splitting up files is very important! We have 265 files in our repo, not including libraries!
Another thing that we do is write tests for our code (This is one of the main uses of Travis/Jenkins). Our tests check that the code does the correct thing. For example, we’ll write a test that simulates a mechanism on our robot, and runs our code with that simulation as an input. It’ll then check that the output is what we expect. This can be confusing at first, so ask if you have any questions about this!
Once the person is done with their code and tests, they’ll submit a pull request to our main code repo. A pull request is basically a request to merge code into our main repo. We then have other people review the code, ask questions about it, and submit feedback on it. Once all of the comments have been addressed, and enough people have reviewed it, we merge it into our main repo.
That’s basically what we use git/github for. We use Travis (basically the equivalent to Jenkins) to run our tests and make sure that the code compiles. The main use of this is for pull requests - we have github set up to not allow merging until the tests all pass on Travis. This lets us make sure that the code in our main repo will always build properly and pass all the tests.
That’s a summary of how we work. It’s not necessarily a good idea to try to implement this all at once, but hopefully it gives you an idea of what this looks like at a slightly larger scale.
If you want to get started doing something like this, I’d suggest two things to start out with:
1.) Make your git commits as small as possible. This makes it easier to collaborate with other people, and is nicer to use. Additionally, learn git really well. I’d recommend using the command line interface for git. It will help you gain an understanding of what git is actually doing, and once you learn it, it’s much nicer than any GUI that I’ve seen. Start using branches, if you don’t already.
2.) Start writing automated tests. The one thing that makes this hard is that WPILib can’t be compiled to run on most computers, so you need to separate all of the calls to WPILib from anything that you want to test. This means that you need to think more about how you structure your code, but it’s worth it!
If you have any questions about how we work, or about how you could go about implementing some of this on your team, feel free to ask
Thank you all so much for the help and feedback, and sorry for confusing anyone about having only one person working on the code at a time- I primarily meant that because I am the only one on the team experienced with the Java FRC libraries and structure enough to be constantly working on the robot’s code. We only have about 3 programmers on the team anyways, and the 2 besides me are still being trained. I will definitely keep using Git, and try to smoothen/better my use of it, as well as introduce the branches system mentioned above. Keeping a main build separate from test build is very useful and I hadn’t thought of it before; previously we used a single main branch and only used GitHub for the history feature to keep track of and comment changes. Again, thank you all so much for the advice!