Vision improvements

Right now we are running basic vision code on our robot(only start capture and set fps to 24). While it is effective, our team is hoping to step it up for the next event, is there any obvious improvements that could be made to improve the quality of the stream. We tried bumping up the resolution but that drastically lowered the fps. I know a lot of teams use the raspberry pi, is that worth buying. Pretty much any general ideas would be much appreciated. Thanks for any help.

Buy a limelight. According to our programming people and many others I’ve talked to they’re amazing.

3 Likes

we have one already lol, but yeah it is really good. I was thinking more of a way to improve camera quality for sandstorm

Definitely use a Raspberry Pi. If you want to bump up video quality try to use h264 instead of mjpeg.

okay I’ll see if I can convince my team to buy it then, thanks!

Keep in mind that you have to stay within bandwidth limits: running a high resolution with high FPS could easily cause you to exceed your allotted bandwidth and get penalized.

And by get penalized you mean simply not being able to not achieve your expected FPS right? As you said it’s an allotted bandwidth. You cannot exceed it even if you wanted to.

I believe you are correct in that it will simply cap the FPS. However, it would be very easy possible to exceed the bandwidth limit “if you wanted to” but it would be very much against the rules (not 100% sure but somewhere along the lines of robot disabled, cards, or disqualification). I guess you could just keep your resolution and FPS high but I wouldn’t recommend it just in case you lose some more important information instead. I’m not sure what the control system does as far as data prioritization but I’d rather play it safe than sorry.

No, it’s not. The radio firmware rate-limits to the prescribed bandwidth limit, and you can’t connect to the field without using that radio.

1 Like

I’m pretty sure they don’t penalize you with cards or points, at least in my experience. Iv’e accidentally sent a large video stream to the driver station before, and was never penalized for it. You are hard limited to that bandwidth and communication with the driver station will suffer if you exceed the limit during a match, which is punishment enough.

Alright, I guess you would have to modify the firmware (or something similar) which would be quite a bit harder than “very easy.” My point was more that it is (theoretically) possible to exceed the hard limit and you would most likely be harshly penalized if you attempted it. Definitely not worth spending the time to do.

Does the limitation effects components with in the robot (like Ethernet camera connected to the radio which Jetson use for example) or only the com between DS and radio?

As noted here:
https://wpilib.screenstepslive.com/s/currentCS/m/getting_started/l/144986-programming-your-radio
The limit is only on the outbound side of the wireless interface.

1 Like

Oh I see, that makes sense, thanks!

You might be “effectively” penalized by your robot being very difficult to control. The system is supposed to prioritize control packets but we had severe control issues in some matches that had high bandwidth usage.

Well yes, but actually no. The FMS gives you a little more than 4 Mbps, but not a whole lot.

The FMS/your radio apparently simply drops packets when you exceed your bandwidth limit. This is what punishes you - you will be unable to control your robot (or, since the control protocol is some parts TCP, have significant lag in doing so.)

2 Likes

I think it is listed in the FMS Whitepaper, but things like the camera and networktables get lower priority than Field and Driverstation data.

It took us several matches, but we had about 50% of our matches where we had issues with controls, namely, the the joysticks getting their values properly into the network so the robot could consume them.

I think it was due to the high bandwidth used by our limelight with the secondary camera stream plugged in. I ended up setting the stream mode to low, and our software lead ended up re-writing the control loop of our default driving command concurrently. So I don’t have A,B testing to know for sure what could be going on there but it 100% appeared to be the cause.

The FTA/FMS data never showed any network problems on our side, but it only ever happened on the field never in the pits or in practice. I’m 100% convinced it has to do with network traffic not allowing the controls to update.

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.