I am a member of team 1360 a relatively new team in Ontario and we wanted to start building some vision libraries but want to ensure we do it properly. We are pretty good at OpenCV but aren’t sure the best way to integrate it into the robot. We were thinking that the Jetson might be the best way to do it since it is small and powerful while allowing easy use of the Nvidia vision framework. We were also looking at running OpenCV on an Android although aren’t sure about how difficult it may be. We have also tried running vision off a raspberry pi but the lack of power became an issue. We also have run it off the rio but also found similar power issues. Could anyone who has experience with the Jetson please give us pointers or other off-board development platforms. Thanks 
Hey 1360! Our team started around the same time as yours - our George and your George talk a lot
This isn’t really an answer to your question but to give you an idea of what at least one other team does, we use a Raspberry Pi 2 running a custom openCV wrapper we call TrackerBox2. It talks to the RoboRio via a TCP protocol. We can get about 10fps with a simple detection workflow, which was enough to help us track the lift targets.
This fall we definitely want to try alternative options so that we know what’s out there and can pick the best solution, rather than just assume that what we’ve got will serve us the best.
We used the Jetson TX1 this year on our robot. The dev board is pretty big so we used a different carrier board (http://connecttech.com/product/orbitty-carrier-for-nvidia-jetson-tx2-tx1/). We used OpenCV on the Jetson and after we did all the math and whatnot, we used ZeroMQ (http://zeromq.org/) to send those values back to the roboRIO. The camera we used is the Microsoft Lifecam HD3000.
Team 2471 has tried numerous vision systems during my time on the team. We have tried all manner of systems, from on board processing, driver station processing, co-processors of many varieties, USB cameras, ethernet cameras, and more! During my sophomore year alone we tried three or four different systems from the ground up, just during build season.
What we learned from all of this, is that vision is easy (like, really easy), but creating a system that works quickly and reliably on the field during the match is a hole different ball game.
Once again, this year, we had tried our best to get a new visions system working for three straight competitions. Of course it worked at home, and it even worked in some of our matches. However, it hardly ever seemed to work for the entire match, let alone two matches in a row. Finally, we decided to try the Cheesy Poofs vision from last year, because other teams had implemented it successfully, and we liked the idea of using a phone, because it has an internal battery, and reduces the number of connections in the system.
This decision was made after the PNW championships, and before worlds, so we only had a few days to get it up and running. Thankfully our lead programmer took it on in full force, and got it working in about 3 days, including tuning.
Let me tell you, this was a hole different vision experience! We got to the competition, deployed code, went to the sensor tuning session, and it worked on both goals. Furthermore, it proceeded to work for all of our matches on Carver. No issues at all.
10/10 would recommend.
So I have not used a Jetson, but I used an android for vision this year. Set up is relatively easy. A good resource is FRC-2016-Public/vision_app at master · Team254/FRC-2016-Public · GitHub from last year. This is actually the library we use for communications.
Essentially all you are doing is creating a virtual port through the ADB interface on the Rio. Then with the phone, use opencv library to process the video feed and send the data (distance, angle, etc.). through a JSON object over that port. Then on the Rio, read from that object.
This year my team used labview vision code on the roborio. There are a lot of great examples in the labview install. I am also working on a prototype/side project for future years written in Java that uses a Raspberry Pi 3, GRIP, and the wpilib vision samples (see links below). As for communicating the data back, NetworkTables seems to me to be the best option as it is already included in the robot already and is very easy to do. I have seen quite a few teams use python as well
ScreenstepsLive documentation/tutorial: http://wpilib.screenstepslive.com/s/4485/m/24194/l/687863-off-board-vision-processing-in-java
wpilib vision samples:
Thanks for all the suggestions, I will definitely lol into 254s vision system.
I been watching the different links and chatter about Android vision. What I’m missing is setting up the phone side.
What exactly do you need help with?
We actually did the same as this. Jetson with ZerMQ as the communication library.
What do you need help on? I’m currently working on a Vision Framework where you can customize for a Pi, Jetson, or other co processors. Its in C++ but I’m planning on making a python one in the future.
We had the exact same setup except we sent the values back with NetworkTables. May I ask why your team used ZeroMQ instead? Is it a performance benefit?
Is there an apk to install or app to launch on the phone. I’m not a power programmer but enjoy reverse engineering working solutions that i can tweek to our needs
For our team, getting NetworkTables up and running was a pain. It threw a lot of CMake errors and I could not get it to work. Now, I feel like the issue might have been the CMakeLists but we just decided to forget it and use ZeroMQ. ZeroMQ was really easy to setup and took very minimal lines of code. Performance-wise I never compared the two. What did you do for building NetworkTables on the Jetson. Were you guys using C++ or pynetworktables?
I know a couple teams like 254 have used Android phones for their vision and maybe you could check out there code. I’m not familiar on doing it on an android but the process would be similar. Getting OpenCV on the phone and the running the vision script. Communication would have to be UDP/TCP unless you figure out a way for NetworkTables.
Check Cheesy Poofs’s CheezDroid here
I looked at that github code. I don’t see how the system installs and runs on the phone.
So first step is to get an IDE for android. Your main options are Android Studio(I prefer this) or IntelliJ. Once you do that, download Cheezdroid, and open up the project in the IDE. The project is configured for Android development, and should be ready to be deployed once built. From there, the app just runs on the phone and sends back JSON files through the USB connector to the RIO, which reads them.
Yeah, that part is really vague. As for the OpenCV and code this is pretty helpful if you have not seen it already. Then the app would have to integrate ADB for communications and running the script. Sorry, I have not really gotten into Android Vision Systems yet but I’ll definitely look into it.
Actually, if you follow the guide on ScreenSteps live here with the example/template code here you have the option of using a java binding of NetworkTables which I find very intuitive and easy to use. If you look through the gradle file you can find some lines which add it as a dependency and you could apply it to whatever project you need
I looked up the carrier board you mentioned but I am confused as to what it is. Does it connect to the Jetson? Or is is a smaller version of? What does it do? Our goal this summer is to work on vision. We are familiar with the PI but just bought an Odroid Xu4 which is faster than PI. We considered the Jetson but the size was a turn off. Any insight on the carrier board you mentioned and opinions about Odroid Xu4 is appreciated.
I think I found my answer. I didn’t realize you could remove the Jestson processor from the board. No I get it you can make the foot print smaller.
