[more developers wanted] PhotonVision H264 streaming!

[thatmattguy]

So continuing my trend of accidentally half implementing features and then running out of time, I’ve got a half baked POC implementation of web video streaming using gstreamer in Photon! I’m looking for some eager Java devs (no experience required) to bring this the last 30% of the way home. If we can push this through it would be absolutely huge for teams — we have been hitting limitations with mjpeg robustness for years now, not to mention bandwidth problems.

I’ve got a fork of thirdparty opencv with gstreamer building, and some hacky code. We just need to get this fully into the photon backend and add the streams to the web UI.

If you’ve been wanting to contribute, but haven’t found a chance to yet, here it is!

GitHub - mcm001/photonvision at 2023_10_15-gstreamer

[mdurrani834]

@mdurrani834

Just to add on to Matt’s post on a more general note, I just wanted to highlight what a great opportunity this is for people to start getting involved in open source software. Through contributing to PhotonVision, you’re able to contribute to something that is used by teams all across the world and gain valuable experience. On top of the technical experience, you gain important, programming adjacent skills that directly translate to your classes in college or future job (if you go into software development), like how to actually use Git properly, developing in a collaborative software environment, bug hunting, navigating a large codebase, and a lot more. From a career standpoint, it also looks pretty good to employers as open source contributions, collaborative software development, and everything else you learn are valuable to them. I know this experience helped @bankst land one of their jobs working in a similar computer vision related space and is something I’ve had the chance to talk about again and again at career fairs, interview, networking events, etc. So overall, a good time where you learn a lot and can help yourself and others!

Crossposting.

[crueter]

At the risk of sounding dumb how does this help PhotonVision?

[vladb]

In general, mjpeg is less efficient bandwidth-wise than H264 at encoding video [1]. Because the FMS restricts robot communication speeds to 4 Mbps, bandwidth efficiency is key to getting better quality streams to your driver station laptop.

Whether the computational overhead is worth it is another question, though it probably is with faster and faster coprocessors.