Unable to Read Serial from JeVois

jevois
#1

We have been trying to implement our vision code using the JeVois cam this year, and we are running into some trouble reading data from the serial connection. To test this, we created a very simple Python module for the JeVois cam that sends the string “text” repeatedly over the serial line. Here’s what we are seeing:

  • When we run the module through the JeVois Inventor program, we are able to see the test message in the console as expected.
  • The video stream itself can be viewed, either through jeVois’s program, a webcam viewer, or through the SmartDashboard (when using CameraServer).
  • In any other serial monitor we have tried, we are unable to see this message. We have tried Screen, Minicom, the Arduino serial monitor, and the RoboRIO SerialPort class (Java).
  • In all other serial monitors, we are still able to run commands and view their output, along with log information from serlog.

We have tried several things in an attempt to fix this problem, but so far nothing has worked. Here’s what we’ve tried so far:

  • Selecting our module using the setmapping command.
  • Selecting our module using the setmapping2 command.
  • Setting the initial video mode to NONE and then using streamon and streamoff to control the module’s behavior.
  • The reload command.
  • The restart command.
  • Disconnecting/reconnecting the JeVois.
  • Reimaging the JeVois cam’s SD card.
  • Testing our module on multiple JeVois cams.
  • Running setpar serout All and setpar serout USB manually and through the .cfg scripts on the JeVois.

We have been following the official JeVois documentation, as well as this excellent guide written by @billbo911.

At this point we’re not sure what to try next. Are we missing something? Has anyone run into this issue, or does anyone have any suggestions as to how we might resolve it?

#2

Are you putting your serial output statements (jevoid.SendSerial()) in the “Process” function, or the “ProcessNoUsb” function? When running “headless”, it calls the ProcessNoUSB function instead of the Process function.

Baud rate could also be an issue.

To get serial data in the RoboRio, I configured it for headless operation by including the following in my config file on jevois.

setmapping2 YUYV 640 480 30.0 Lame Sandbox2

setpar serout USB

streamon

I put my code for detection and output in the ProcessNoUSB function. After pressing “go headless” in the System tab of Jevois Inventor, it started spitting out the serial messages from proscessNoUSB to the console window. (Before that, it was spitting out the serial messages from the Process function.)

I then disconnected from my PC, hooked it up to the top USB port of the RoboRio, and started a serial port object, using kUsb1 and 115200 as parameters, and the data arrived to the RoboRio.

(Now if I could just get my colors working on the Jevois. No matter what settings I make, it just seems like everything is too dark, or the colors aren’t right. The yellows end up white.)

#3

Thanks for your feedback! That got us reading serial data, but now it seems like we have to choose between the video stream or the serial data; we haven’t been able to send both at once. Any ideas?

#4

If you are running “Headless” , the precessNoUSB will not send video, but serial over USB is available.
If you are running from a “process” call, both video and serial over USB are available.

We would need to see your code to help figure out what’s happening.
PM me if you would like to show me your code and I will see what we can figure out.

#5

Hmmm…what I found was that when I sending a “process” call, (i.e. not headless), I could see the serial output in the console in inventor, but when I plugged it into the Roborio, I got nothing. To get something on the Rio, I had to go to headless mode.

Operator error is a distinct possibility here.

#6

To further elaborate on why I think “headless” operation was somehow significant to my, and probably aryker’s, problem, it has to do with what script is running.

I plug my Jevois camera into a USB port on the Roborio. It boots up. So…what will it run? There’s nothing telling it to do anything, so that sounds pretty “headless” to me. So, it will run whatever was called in its config file (in the case of the config file I posted earler, that module was Sandbox2) and it will run it in headless mode, calling the processNoUSB function instead of the process function.

And that’s what was happening. I got the messages from Sandbox2’s processNoUSB function.

When I run Grip, or when I run Jevois Inventor, it starts in hosted (“headful”?) mode, and I can’t convince it to run anything other than DemoSaliency.

I don’t know how I would convince it to run in hosted mode while connected to the RoboRio. I’m guessing, and guessing is the correct word here because I haven’t tried it, that I would start a WPILib CameraServer object. At that point, somewhere in the magic of that library, it would probably send some sort of command that would force the camera into a certain set of parameters, i.e. resolution and frame rates, and it would select a module based on that? But whatever it came up with (I’m guessing DemoSaliency, because that’s what happens when I select the camera in Grip) wouldn’t have the serial messages.

My current quest is to try and find out how I can convince the thing to run the program I want it to run, using the camera parameters I want to run on it, while I’m connected to Grip. Once I do that, maybe I can get it to work streaming data (i.e. not headless) when I’m on the RoboRio, with the camera parameters I want.

#7

Hi David,
Hopefully I can hit on enough suggestions to get y’all running. Before I go any further, let me just state that I have exactly ZERO experience with GRIP. So for that piece, I will have to step aside.

Quick aside; “Process” is the method that is called repeatedly by the JeVois environment within your STREAMING script to acquire and process images. “processNoUSB” is the method called when you run headless.

OK, there are basically two ways the JeVois operates: Streaming, Not Streaming (ie. HEADLESS).

To launch either automatically, there are two different approaches. The Streaming scripts are started from the videomappings.cfg file by designating them with a “*” at the end of the line. ie " MJPG 320 240 15.0 YUYV 320 240 15.0 Jevois yourscriptname * ".

There is a tutorial on Headless here.

To launch a “Headless” script you need to call the script in the initscript.cfg file. You must have an entry for “setmapping2” (note: no ending “s” in setmapping2. Ask me how I know :sob:) .
You will need an entry in the videomappings.cfg file for your headless script as follows:

NONE 0 0 0 YUYV 320 240 15.0 Jevois yourscriptname

You will also have to have and entry to start the stream. For example:

setmapping2 YUYV 320 240 15.0 Jevois yourscriptname
streamon

Now that the scripts can run automatically, make sure you designate where you want the tracking data to flow out. This can be done manually, or automatically. I suggest you verify you get the results you want by doing it manually first, then configure JeVois to do it automatically for you.

Commands you need to know:

setpar serout USB # send the serial out the USB port
setpar serout Hard # send the serial data out the Hardware port
setpar serout Both # yep, send it out both ports

You can configure your scripts to automatically use these configurations by adding the appropriate commands in the script.cfg file in the folder with your script.

Now here is an “Eagle Tip!”.
We do all of our testing of serial communications with the hardware port connected through an arduino pro micro set up with a serial passthrough script. Once we see the data we want, we then adjust the code to stream it out the USB port. With that said, now the JeVois Inventor is MUCH MORE USABLE, we are starting to use it almost exclusively. The console tab allows you to see the serial data without actually connecting to the hardware port.

Alrighty, if there is anything I missed, or any new questions that this did not answer, please post it up and I will try to help.

#8

If you are using OpenCV, I can’t imagine not using Grip to visualize your filters and other common operations. It’s that good, and it’s free.

But for purposes of this thread, it’s a windows program that allows you to see the output of a webcam, but doesn’t allow you to set any parameters of the webcam. Somewhere between the webcam and the PC, bits must be flowing to make the connection and probably tell each other a bit about the properties of the camera and/or application, but the user doesn’t have any control over that.

I took the asterisk off of the line that identified DemoSaliency as the startup application, and I put it at the end of the line for my Sandbox2 module. When I started up again, it ran DemoSaliency. Then I completely erased or commented out every line in the whole file except the one for Sandbox2 (with the asterisk). It started Sandbox2, in streaming mode. I’ve run it before in headless, so I’m sure that would still work, although I didn’t try it out again.

However, the camera parameters weren’t saved. The device still selected a set of parameters it liked a bunch more than mine. Unfortunately, those parameters don’t work worth a hoot. The gain is too high and there are vertical lines running all through the image, and the colors are all wrong. I now have these lines at the top of initscript.cfg

setcam autoexp 1

setcam autogain 0

setcam autowb 0

setcam brightness 1

setcam contrast 3

setcam saturation 2

setcam redbal 119

setcam bluebal 134

setcam gain 56

setcam hflip 0

setcam vflip 0

setcam powerfreq 2

setcam sharpness 6

setcam absexp 427

setcam presetwb 0

but it seems to have no effect. (I don’t know exactly which of those parameters are actually used if you aren’t running headless. It seems like nothing in that file has any effect except headless, but that leaves me with no way to save camera parameters, and the image is completely unusable at the settings that are coming up.)

Thanks for the assistance.

#9

Oh, I’m familiar with what GRIP is, I just never have had a need to use it. I have been using OpenCV for several years now and have found I am very comfortable with using it.

It sounds like you may not be using the script.cfg in the folder with your headless script. The content of that file will override the content of initscript.cfg.
Yours may not have anything of value in it, but it is worth looking into to see if it is causing a configuration conflict.

#10

I never use Grip to actually generate the OpenCV code, or run the pipeline the way it can be used, although a student programmer did use that method one year. I just use it to visualize the results of various filters.

And you were correct. Script.cfg was indeed the key. My yellows now look yellow.

I will now resume my extreme praise for the Jevois camera. Thank you for your assistance. Much appreciated.