We ran our complicated vision processing - dual cameras, a stereo and a mono - and 3 OpenCV generated images in addition to the 2 camera images.
It works perfectly! No changes to our program required.
We use our own compiling code process on Visual Studio Code and only had to copy into our VS Code folder the new jar files from the Java example and change the name of the OpenCV jar file reference in the build.
There is only one trivial incompatibility between the new and the old and we found it because we are lazy and when doing 2019 testing with some cameras, we left off the pixel format, height, width, etc for the Vision setup that goes into the frc.json file. The 2019 image worked leaving everything blank. The 2020 image requires the camera settings for the OpenCV generated images to display other than black screen.
Nice that the older equipment still works right. And I’m looking forward to receiving my new RPi 4 next week since it is supported by FRCVISION.
Thanks to Peter Johnson for another job well done!
While the 2020 FRCVISION works perfectly on the RPi 3 model B it was no end of trouble with the RPi 4 model B I got today.
First the kit came with the mini HDMI connector for the 4 but the other end didn’t match my monitor so that was a hassle.
Then the mini HDMI wouldn’t plug into the RPi 4 and both were pretty much destroyed in the fussing. Hard to tell which part ruined the other but the RPi will work with a different cable after I straightened its pins. Good thing I bought 2 RPi kits at once. The other mini HDMI cable was slightly different (surprisingly) and worked OK but the RPi connection was loose so I used the 2nd RPi. The 2nd kit with RPi and cable worked OK but it was a little tricky finessing in the mini HDMI - angle the connector down a little to start. Connections were tight and reliable after that.
My Ethernet connector wouldn’t snap into the 1st RPi, either. I’m not impressed with the connectors that they used. The 2nd Ethernet worked OK.
Finally I got to fire up our vision code. Bad news - 9 times out of 10 tries our OpenCV generated images would not show - nothing but a black screen (the 2 camera images were OK , though). No error messages and why it worked once I have no idea.
Slipped the microSD card and cables back into the old RPi 3 Model B and all is OK again. (Except maybe for the $160 pile of RPi 4 model B clutter on my table.)
I’m so glad the image worked so well for you on the 3B, but also sorry to hear about your bad experiences with the 4. The image is literally identical between the two, so I’m surprised that things didn’t “just work” (software wise at least!) on the 4. That being said, if the 3B works and meets your performance needs, just stick with it!
If you can PM me your code I can do some testing locally to see if I can figure out what might be happening.
Thanks for the offer. First, maybe you can help me with creating a RPi startup log that I could see what is happening. It takes a lot of your time so I hate to get you deep into my code which may or may not be correct.
The program structure is 5 images on 5 ports from 5 different threads are being created.
Camera 1 starts in a thread and starts its “GRIP” pipeline in another thread.
Camera 2 starts in a 3rd thread and starts its “GRIP” pipeline in the 4th thread.
An image merge 5th thread starts to combine 2 previous images into one view.
When the RPi starts from POWER UP the 2 camera threads start and work - we see the images.
The 3 OpenCV generated threads start as seen in the “top” command" but there is no output - messages in the loop don’t print and there is no image. I changed the “infinite loop” from !thread.interupted to true but that didn’t help.
So now the processing has started and is working (incorrectly) so I click Terminate on the vision dashboard controls (it Downs and Ups). Now everything works perfectly - all images appear and messages print as expected.
I’ve changed the code somewhat to be very rigorous about the threads starting in the right order at the right time to make sure the inputs are being produced. “synchronize” is used in setters and getters for the images. Any errors from camera grab as handled. The code is (almost?) perfect!
If I could see messages from power on startup and when our program first starts maybe I could see something to change.
Thanks for your help.
Logging of the user process is done via the /service/camera/log/run script. By default this script calls exec /usr/local/frc/bin/netconsoleTee -u, which copies the console output to localhost port 6666 (the webserver then listens to this port). There’s a couple of options to get logs from startup; probably the most failproof in your instance is to change this to something like exec /usr/bin/multilog t /tmp/mylog, which will put the output in /tmp/mylog/current (with each line timestamped). Note this change won’t take effect until you either restart the Pi or kill the netconsoleTee process.
I added a “sleep 10” as the second line of frcvision’s /service/camera/run.
I figured it was a timing problem but where? There!
Extra sleep makes the manual or webserver restarts lethargic so maybe the waiting 5 seconds in the other script isn’t needed. I didn’t experiment with that yet nor see how little sleep is necessary.
Thanks for the tip on multilog - I learned something but the output was identical to what I saw on the webserver - all messages at the beginning were missing until well into our camera program. And that hint got me into the system a little deeper and enabled me to strategically place the sleep after a bunch of other flailing away, of course. Thus I still have no idea what is wrong but a little sleep does the trick. Thanks for your help.
Putting the thread sleep at the beginning of our Java vision program is equally effective as changing the script /service/camera/run. I’m glad I don’t have to change the FRCVISION image to get this to work.
Here’s close to the minimum I figured out is reliable and the explanation as best as I’m going to know.
In the RPi 3 we had a 3 second wait in front of a flash drive mount to use for logging.
In the RPi 4 we had to add a 5 second wait in front of an InetAddress.getByName(URL) to make the OpenCV generated image threads work and prevent an error concerning not having an address for the datagram. We could remove the previously installed sleep in front of the mount since that was after the Inet address get and covered by its sleep.
I wonder if use of run level at boot up could be used to make sure the camera job doesn’t run before the system is fully functional and capable of handling its requests of the system?
This FRCVISION RPi image is a huge addition to our robot team experiences as there was no time for students to create it and they get the benefit of concentrating on the image processing. We have been running low on roboRIO resources and the RPi is a great addition to the robot processing capacity.
Thanks
-rkt
I’m sorry (or how computers make good people look foolish). I tested the 5 second sleep configuration in front of the Inet.address 15 times without a failure and declared victory and moved on. I was wrong!
I ran the RPi 3 for some hours [I have only one FRCVISION image microSD card at the moment] then returned to the RPi 4 and it would not run!
The only thing that makes the RPi 4 work right for me is a thread sleep for 10 seconds (maybe a little less) as the first thing in the program. class Main{{Thread.sleep(10000); [try/catch etc]} …} Other sleeps in the program are not needed as they are covered by the initial one. Tomorrow may be a different story or I might take a break from all this pain.
And so far the CPU processing on the RP4 seems a little faster than the RPi 3 but it’s hard to tell because with the better USB on the RPi 4 the stereo camera, bandwidth hog, is running at twice the frames per second but the mono camera is running exactly the same.