We have two limelights on our bot. Works beautifully at home, and awful when connected to FMS. Looking for a solution. FTAs blame the cameras and say no one can get them to work. With no solution, this will be the biggest waste of money we’ve dumped into a product. $800 on two cameras that aren’t even reliable enough for the driver to navigate by, let alone use the vision alignment we were relying on. While acknowledging this could be completely our fault, right now I’m pretty pissed at Limelight. Has anyone else in the country successfully used two limelights? V2 Any tips?
What specifically is happening with the Limelights? Have you configured them for static IP use? You really shouldn’t be bandwidth limited unless you’re streaming at high resolution (we run a Limelight and a USB camera through our roboRIO with no issues).
I’ll grab our software student…
(student using my phone): We have configured both cameras to a static ip address with low resolution set. We are also running bonjour in the background. We have both cameras streaming to our shuffleboard on the driver station laptop which we have tried changing out. One camera works nearly perfect with some targeting issues that could be caused by the field lighting. The other will not even load the camera feed when connected to the FMS.
Paging @Brandon_Hjelstrom …
Is it always the same camera that’s failing? Have you tried leaving only the failing camera connected while on the field? Is one of the cameras connected by an Ethernet switch that may be the problem? Finally, does the camera work in your pits with the competition radio configuration?
I’ll come by your pits with a software student after this match to see what we can do.
Does accessing the stream directly from 10.23.63.X:5800 through a web browser work?
Are the limelight’s configured to two separate static IP addresses? (e.g. one should be set to 10.xx.xx.11 and the second should be 10.xx,xx.12). Are the limelight’s configured to use two separate names so that you can get targeting info from both separately? (In the same IP configuration page, there should also be an option to name the limelight’s. By default, a limelight is named “limelight” and outputs all the data through the same table under NetworkTables. If you have two limelights, at least one should be configured to a different name like limelight and limelight2)
Out of curiosity, is it just the streams that are having issues, or is the limelight also not publishing data to networktables?
We’ve been using 2 on our robot at Del Mar Regional this weekend without any connection issues, though we did find that if we powered on our robot before the FMS switched team numbers, the Limelights would fail to stream. To resolve this power cycle once the team numbers on the field switch or wait to power on.
If there’s an ordering issue on the FMS switching over, that’s most likely due to something being DHCP. My advice to anyone using a coprocessor is to set both the RoboRIO to static 10.TE.AM.2 and the coprocessor to static (.10 or similar). The former so that NT is sure to connect and the latter so that a DHCP renewal doesn’t happen at a bad time.
The DS can be dynamic, but I also like to set it to the traditional .5 static address just so it doesn’t need to wait to pick up DHCP.
1 Like
ToddF are there any more details you can provide on the problems you’re having? Have they been resolved today? Did any of the ideas in this thread solve the problem? Feel free to contact us directly at [email protected] and we will help you out.
We have found that steaming to Shuffleboard can be unreliable at times. Then a team recommended using Chrome, which has worked flawlessly ever since
1 Like
Have you tried plugging in a secondary camera to a single limelight module? Having a regular webcam plugged into the one limelight box?
It would be good to know the symptoms you were seeing with Shuffleboard so we can try to fix this issue. The most common reason Shuffleboard would not get the stream would be because it’s not getting published to NT (e.g. the Limelight doesn’t establish a NT connection). Is your RoboRIO set to the static .2 IP?
Yes, the standard 10.TE.AM.2, limelight on .10 and DS on .5. We would find the camera might get a few second of image before the match began – after the FMS swapped numbers – and then freeze as auto started. Granted we have not yet been to a competition this year and have no idea if it is still an issue.
Todd,
It sounds like your team may have been using the driver mode option with both of your Limelights. If your streaming rate hadn’t been reduced before this, this may have caused you to hit the new bandwidth limit.
As the exposure of one of your cameras increases, the streaming bandwidth is going to increase along with it due to increased amounts of detail in the image. We have recently found that in a crowded environment with the normal streaming and max exposure rate bandwidth for one camera can approach 2mb/s with driver mode.
The fix for this is to use a “driver pipeline” with an exposure of 30-50ms rather than 100ms (use the lowest number you are comfortable with). The visual difference is sometimes imperceptible, while the bandwidth is sometimes cut in half. An additional benefit of this is that your streams will become even more responsive than before.
Have you ever measured your bandwidth usage?
Driver mode is actually on a list of features to remove after this season. We will work on a best practices list to prevent failures on the field.
If this wasn’t the issue, we will need to know what the ip addresses and hostnames of your LLs were, your entire networking layout, which dashboards you tried, etc.
Congratulations on your win today. Please send us an email if you would like some help diagnosing the issue.
3 Likes
We ran 2 limelight v2s with 2 elp usb cams all comp long with 0 problems (besides the moderate latency that sends out driver into the side wall durring sandstorm, but we get that in the room also).
Both limelight cams are always in the driver mode and we keep the exposure way down and leds off to keep the bandwidth down.
1 Like
The software team beat on it until they found a solution. They communicated it to the FTAs, who were able to get many of the other failing Limelights at the event to work. I don’t know the details myself, but I’ll get them to do a writeup at our debrief meeting tomorrow night. Or at latest, later this week. I do know it was a simple tweak to the networking settings, nothing to do with bandwidth.
Happily, we got them working in time for eliminations, and used our vision assisted alignment to help go on and win the event. It’s only fair that I offset my original frustration (since I can’t edit the original post (CD should really allow editing for longer than 24 hours)) with an endorsement. When they work properly, Limelights are a terrific product, and are instrumental in our gameplay strategy this year. We designed our bot around having vision assisted fast cycling. Even with the simplest vision assistance, using limelights can mean the difference between winning and losing an event. (We won both F2 and F3 by a single point). Some big caveats, though: they are finicky to network properly with the FMS, and there is no way to trouble shoot them at an event other than randomly changing things before matches and hoping it works. The FTAs aren’t able to provide much help, and getting a 3rd party camera working with FMS isn’t going to be their highest priority. Hopefully our solution will work for others, too, and they can avoid this frustration.
Hi
We (GaCo 1629) were one of those other teams that managed to get the Limelight working at the same event, with a very delicate work-around (reported here Limelights disconnect from roborio when on field)
I’m eager to see your final problem/solution write-up.
Thanks for all your team’s help at the event. I hope these issues can be resolved.
Phil.
1 Like
Hi,
I am the software student and driver for Triple Helix that has been working with our mentors to get both of our limelight 2’s up and running in our matches. We have both cameras streaming to our shuffleboard. When we first started competing only one of our two cameras was working when connected to the FMS. When the FMS would connect to our driver station our Limelight on the front of our bot would always boot up immediately and work the entire match with almost no problem. The second camera would never connect and if it did, it would crash almost immediately and just show a black screen. We tried to recreate this issue in our pit but when we connected, both cameras worked perfectly fine, it was only when we connected to the FMS. The steps we took that finally seemed to fix it were resetting the static IP address and changing the network mask. Before the IP addresses were 10.23.63.3 and 10.23.63.4. The camera that worked every time was the 10.23.63.4 so we changed the second Limelight (10.23.63.3) to 10.23.63.5. Then we also changed the network mask which had been 255.255.0.0 to 255.255.255.0. With these two changes both cameras worked perfectly for the rest of the competition. Below I have created a step-by-step of our entire configuration process.
- We configured all of our Limelight 2’s with the latest image (2019.5.1). The latest image can be found at https://limelightvision.io/pages/downloads.
- Download and Install Bonjour to aid in the networking process. Before this was done it was common for our cameras to crash almost instantaneously and they were unusable.
- After the camera was imaged, we loaded the web dash for it in Chrome where we
a. Set the team number
b. Set the stream rate to low
c. Changed the IP address to static. Because we were using two Limelight’s we set their static IP addresses to 10.23.63.4 and 10.23.63.5.
d. Set the networking mask to 255.255.255.0
e. We also then set the name of each camera running on our robot which by default is “limelight.local:5801” to “limelight-front.local:5801” and “limelight-back.local:5801”.
f. We had to do the above steps through a direct ethernet connection because after the initial reimaging the webpage would not load. To access the camera this way we searched “limelight.local:5801” unless you have already set a name for it which you would have to input that instead. Because we connected over Ethernet and not wireless, we had to add our gateway which for us (Tripe Helix 2363) was 10.23.63.1.
g. To make these changes you must power cycle the limelight. While this is being done you should also close chrome and reopen the page because we had issues where it wouldn’t reload otherwise. Sometimes the limelight will not set to static the first time and we would have to try that process again. - After this is set, reopen the page and underneath the camera page make sure that they camera is streaming at the lowest possible rate so that you don’t exceed the field limit of 5 Mbps. With these settings, and with two cameras running at the same time, we were still under 1 Mbps during our matches.
- Along with all these changes, also check that the any laptop that you plan on using has the windows firewall disabled and an the IPv6 disabled.
- The last thing that we had to do was the FTA informed me that after we leave our pit and before we connect to the field, we must restart our driver station laptop. We had to do this every match. I never tried to skip this step and see if both cameras worked anyway because I didn’t want to risk not having both cameras so late in the competition.
3 Likes