Finally getting around to working with PV for 2023.
First question: after install of 2023.4.2 limelight image to limelight v2, the LL led had both left and right segments lit. After downloading the LL hardwareconfig.json file and uploading it, the LL only lights the left segment when turned on. Is this expected?
Second question: I have had a devil of a time getting PV on the LL to identify the reflective tape targets for the 2023 game. I have read the docs, CD posts and fooled with the settings extensively and after a lot of that fooling around I finally got PV to sort of recognize the tape targets on the scoring posts. But, PV constantly switches back and forth between the high and low targets or no target at all. I blocked one of the tape targets and it still switches between have a target and not target. It switches so fast and returns no target so often, our code can’t make sense of it. I could live with switching between targets but switching between “I have a target” and “I don’t have a target” when a target is in the field of view does not work.
I expect to be able to configure PV to return a steady target list so our code can determine when we have or don’t have a target. This is how it worked for past games using the limelight firmware. I did not have anywhere near this much difficulty getting the LL to work.
I will get the configuration as soon as I can get back to the robot lab.
Interesting about the LED brightness. I will try setting it as you suggest and do the tuning process again. Will be middle of next week before I can test this.
Sorry, apparently you don’t have dimmable LEDS Limelight 2+ users have access to an “LED Brightness” Slider which allows for LED dimming. Although I suppose it’s possible for PV to support this and LL2 software not. You’ll have to go with low exposure times which is desirable.
You didn’t mention the conditions for the jitter between targets. Near the target, far from target, robot motion, robot stopped? Bright ambient lighting, darkened room? Sunlight, ceiling lights?
The PV web console has an adjustment slider for LED brightness but I left it at 100%. I did set the exposure and brightness very low. This yields a black raw view with the targets very clearly visible in green.
The robot is about 6 feet from target, not moving. The environment is a high ceiling shop space with overhead lights. The room is not that bright. There is some sunlight coming in from high windows.
It seems like the exposure and brightness yield a “box” in the shape of the target on the processed view, that box is black outlined with a white edge. PV seems to switch back and forth identifying the box as a target and the outline as a different target. I will get a screen shot when I get to the lab next.
I think this is what you are describing as faulty and what you desire. I couldn’t get my LL with PV to connect to the browser (of course it worked the last time I had it on)
but here’s what I see on RPi PV. In this instance the HUE range was the tiniest bit too tight. (Obviously not the game poles just two pieces of tape hanging on a box in my basement. The little dots between the large tape and smaller tape is the model of the 2022 hub high target still available for use.)
And there is a lot of jitter with just an outline detected.
Yes. My input looks like your upper left picture though much smaller since the targets are smaller. The picture on the upper right is what one would expect. I had to work hard to get a processed image that looks like your lower right picture. The red outline never appears and the target identification with target number is constantly flashing as it ids the shape as a target and then changes its mind.
I will get the settings as soon as I get back into the lab.
Shouldn’t the color picker set the hue correctly? I used it to set my hue values.
Color Picker could be helpful to get you on the path to the correct threshold but you have to select pixels carefully and still not know exactly which pixel you selected. I’d probably click set to average on a “typical” looking pixel then expand range a couple of times again on pixels that look like what should be included for min and max. Then use the slider bars gently to get the best results.
Because of those uncertainties and multiple steps I consider the color picker more of a novelty feature since the slider bars with the colored cues are excellently done and I can tune faster with them than fussing with color picker.
My test strip right now with one set of the the LimeLight 2+ LEDs has hues 54-70 and the color picker covered that range with 51-71 and 54-74 on two successive tries with careful pixel selection.
Picker did values 200-220 with the correct range being 174-255. I guess I can’t judge typical min and max values as well as I can hues. I got only part of the tape until I adjusted with the slide bar.
The saturation is 255 and the color picker covered that easily with 238-255.
I’ll test those numbers when I get back to the lab.
You mention one set of the LEDs lit. What is up with that? Why only 1 set? It was using both sets with PV image before uploading the hardwareconfig file.
I should clarify the LED question…you may be using 1 set on purpose with the 2+. As far as I know, I have done nothing to cause only one set to be lit on my LL. It went from 2 sets to 1 set with the upload of the config file. I don’t know if this is expected or if that is another problem in our case.
I understand the question about the LEDs not working right but I don’t know the answer. Sorry, I didn’t mean to imply an answer - I used 1 set for my tests because the distance is about 2 feet and both sets on was too bright. I don’t have an LL2 to test.
I know what this is going to sound like, but I had an out-of-robot experience today. Back in the lab working on PV and LL2. Started with the same problems I have documented here. Really poor tracking and no adjustment seeming to make any real difference. I also tried the settings suggested by SLAB-Mr.Thomas and saw little change to the problem. I have the config and pictures of the PV console of that situation…but
At a point in testing, the PV web console views of the camera became unresponsive. I switched to the settings tab and clicked Restart PhotonVision. When PV restarted, an amazing thing happened. The PV console reflected a completely different configuration and the target idenfitication was working just as I had expected when I started. Target identification was accurate, solid and repeatable. I have no idea where the settings came from or how the PV config changed on restart, but I will upload the config files from when it was failing to when it started working shortly.
With PV working as expected, I started testing our code to move the robot in response to the data returned in the PV api. This went ok except for this new strange issue where PV (I assmue on the camera but maybe on the RoboRio) has a network table communication problem and the PV library on the RIO reports no new data sent by the camera and I am unable to control the LEDs. Restarting the code or the LL seems to clear this up for a bit but then it comes back. It got worse to the point I could not really do any testing as the communication between our code and PV on the LL failed regularly…and then it work for a few minutes. Using outlineviewer you can see the tabel entry for photonvision/cameraname is fully fleshed out and other times there is no data in that sub table. Really frustrating.
I get errors on the console suggesting the camera name I am using is not found in the network tables but then moments later there is a message saying the camera name was found. Defintately something going on with the network tables link between PV on the RIO and PV on the LL but after lots of testing I could not find a cause. I will be back in the lab tomorrow and collect more information.
This is the config that appeared after restart of PV on the LL. I have no idea where these settings came from. But with them, PV worked as expected in terms of target identification and tracking. photonvision-settings (1).zip (81.6 KB)
Long-shot but easy thing to try is use static ip addressing. Some people say it’s the only way to go and others say shouldn’t ever be necessary but try it if you have network problems.
I’m a “developer” (don’t have much time to help debug today) but yes, please join the discord. It’s linked in our documentation under contact. Much less friction with debugging and much more active on there.
Testing again today, PV working in terms of locating target in a reliable fashion. However, the network tables “connection” issue remains. Here are some notes:
PV working and then this error:
NT: DISCONNECTED NT4 client 'photonvision@3' (from 10.44.50.72:57672): remote end closed connection
NT: NT4 socket error: operation canceled
NT: Got a NT4 connection from 10.44.50.72 port 39246
NT: CONNECTED NT4 client 'photonvision@3' (from 10.44.50.72:39246)
NT: WARNING: client photonvision@3 publish '/photonvision/ledModeState' conflicting type 'int' (currently 'double') (ServerImpl.cpp:2003)
NT: DISCONNECTED NT4 client 'photonvision@3' (from 10.44.50.72:39246): remote end closed connection
NT: NT4 socket error: operation canceled
NT: Got a NT4 connection from 10.44.50.72 port 41162
NT: CONNECTED NT4 client 'photonvision@3' (from 10.44.50.72:41162)
NT: WARNING: client photonvision@3 publish '/photonvision/ledModeState' conflicting type 'int' (currently 'double') (ServerImpl.cpp:2003)
After PV restart on web console got this:
NT: DISCONNECTED NT4 client 'photonvision@3' (from 10.44.50.72:55612): remote end closed connection
NT: NT4 socket error: operation canceled
PhotonVision coprocessor at path /photonvision/LL-4450 not found on NetworkTables. Double check that your camera names match!
But the names do match...
Followed by this:
Found the following PhotonVision cameras on NetworkTables:
LL-4450
Error at org.photonvision.PhotonCamera.verifyVersion(PhotonCamera.java:357): Found the following PhotonVision cameras on NetworkTables:
LL-4450
Then a repeat of the first message above. The result is very intermittent connection to PV on the camera.
I also see this at times:
PhotonVision coprocessor at path /photonvision/LL-4450 is not sending new data.
Warning at org.photonvision.PhotonCamera.verifyVersion(PhotonCamera.java:365): PhotonVision coprocessor at path /photonvision/LL-4450 is not sending new data.
The message above happened right after a LL restart and one successful run of robot alignment command. At second attempt the above error (no data) had
occurred and there is no new target data but old target data is still returned by PV library and the robot drives off into the distance until stopped.
Then triggered robot code again and it is getting fresh data and moved correctly. Seems to recover from no data on it's own.
The main issue is whatever is causing the drop and reconnect by the network tables code on the LL. It regularly disconnects (above) and
reconnects, complains about ledstate type mismatch then does the 'can't find camera then finds camera'. When these disconnects happen the PV
web console may lose it's camera pictures but they come back when it reconnects. When this happens, the LED on the LL flashes.
Finally, I changed to static IP for PV on LL and it made no difference.
All this repeats so its hard test the robot code as PV only works intermittently. But it does seem to recover and work a bit before failing again, so I was able to test the robot code with some success.
Seems to boil down to some issue with network tables, appearing to be NT on the LL. I am attaching a Riolog file from one of my tests. RioLog.txt (69.6 KB)
I think you should double check the versions of PV on the LL and on the Rio. The library on the Rio needs to match what is on the coprocessor (your LL in this case). Given the message about the data type mismatch for the LEDs, a version mismatch seems possible.
