Human Vision Camera compatiable with FMS & Java ?
 

Anyone know a specific camera model (probably Axis) that is known to work seamlessly as a human vision camera with

1) A Java programmed robot
2) FMS
3) Windows say 8.1 or 10 laptop with smart dashboard and
4) IE or Chrome (with no special kludges)

We are now told the Microsoft LifeCam 3000 is incompatible with FMS (at least in a java environment). Our Dlink IP camera will not work with IE or Chrome possibly with FMS a possible cause contributor.

I am told FIRST recommends a specific AXIS camera but no amount of searching on their site finds that document.

Re: Human Vision Camera compatiable with FMS & Java ?
 

We used a LifeCam 3000, with few problems at Bayou in Week 3. We also use Java. We also used a Genius 120 degree camera on our intake side as well and had a button to switch the video feeds. The only problem we had was a single match where SmartDashboard wouldn't connect to the robot for some reason, but I don't think that had anything to do with the camera.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Could you give details on who is telling you that the Lifecam 3000 is no longer supported for Java?

https://wpilib.screenstepslive.com/s...ifecam-hd-3000

That URL makes it sound ... supported. The other camera setup page for Axis calls out the Axis 206, Axis M1011 and Axis M1013 as being supported. In reality any Axis camera that supports their Vapix API would work with WPILib API. And Driver Cameras don't even need any of this. They will most likely support an MJPEG stream or H.264 stream or similar, and the compatibility is then up to the dashboard viewer to connect to the stream. I've seen a number of teams use a web browser opened to the stream.

By the way, lots of USB cameras will also work. I've had good luck with the Logitech 310 and 270. They cost far less than Axis and typically less than an IP camera. They do place a bit processing load on the roboRIO CPU, but that isn't typically an issue.

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

Just a small bit of info, I've heard that no USB web cams were working over FMS at North Bay.

Re: Human Vision Camera compatiable with FMS & Java ?
 

We were told at the North Bay regional when our 3000 would not work when connected to FMS but worked otherwise that FIRST knows of an issue with FMS and the 3000. I can't remember for certain if they said it was related to Java or the smart dashboard. We had the same problems at Greater Toronto Central Regional in week one.

Its critical we have a reliable trustworthy camera feed going for champs.

Re: Human Vision Camera compatiable with FMS & Java ?
 

What was your Microsoft LifeCam HD-3000 connected to, that connected it to the network, that interfaced with FMS?

Was it on a co-processor like the Kangaroo or something running Linux?

I totally agree with Greg as I see no reason to believe that a Microsoft LifeCam HD-3000 camera won't work with FMS.

I do see reasons why the Microsoft LifeCam HD-3000 camera might not work very easily with Linux in general. The RoboRIO runs a distribution of Linux so therefore there are issues I have seen and documented in another topic on ChiefDelphi between the Microsoft LifeCam HD-3000 and the RoboRIO. These issues very easily could be annoying to deal with when using Java on the RoboRIO even directly to Video4Linux. They are likely as bad if you use Java-to-OpenCV-to-Video4Linux or something like GRIP-to-Java-to-OpenCV-to-Video4Linux.

I also agree with Greg - Logitech makes a lot of similar and cheaper webcams that work just fine with the entire FIRST control package with the current distribution of Linux. The Logitech C920 USB camera works just fine to the default FIRST dashboard (you need a bit of Java code to get the RoboRIO to stream to the driver's station) and you can switch the camera that sends if you have multiple in Java I've tested it (there is a short delay). However, as a USB camera, the Logitech C920 doesn't work to a plain old web browser all by itself (you need to add more software to make that happen).

The Axis cameras are fine webcams and any model Greg also listed will do the job. Just be aware you may need to give it a static IP to get it working on a real field and that may make testing in the pit interesting. More than one Axis camera might also be a challenge.

Re: Human Vision Camera compatiable with FMS & Java ?
 

But IP cameras were?

How were you told of this? Were they an official of some sort?

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

As I posted earlier, the LifeCam was tested during beta as were a number of other USB cameras. This doesn't mean that every aspect worked perfectly, or that the tests were super thorough, but I do not believe that any official channels have made any such statements about the LifeCam 3000 and FMS.

So, can you give details on what worked, what was tried, what error messages you received, etc?

The default dashboard will open a TCP connection on port 1180 to the roboRIO's IP address. It will then read the stream that encodes a jpeg preceded by a version string. This code is identical to last year, and was even used since 2009 for IP cameras that were forwarded through the cRIO. I assume that the SmartDashboard does something very similar.

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

The 3000 is a usb camera directly plugged into the roborio. No offboard processor etc. Absolutely vanilla installation. There were other teams there with 3000s that would not work either.

Our Dlink IP camera set on a fixed IP would not work in IE or chrome on the driver station but would work with some special kludges in Firefox.

Re: Human Vision Camera compatiable with FMS & Java ?
 

For a working USB camera, our team had two Logitech C920 cameras working at two competitions so far.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Did you guys put the Java code into the RoboRIO to start the camserver?
Did you make sure to put the code into the correct place in the code to insure it started running at the correct moments?

There are some examples of how to do this, which do work, in the sample work for Eclipse.
I can provide some pretty detailed instructions to get to this sample if anyone wants them.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Two FIRST tech people (CSA and I believe the FTA) spent a lot of time, twice with us on the field, the second time during lunch (20 minutes). I believe they must have called FIRST central support. At the end they said FIRST knows of a technical compatibility issue with the 3000 and FMS

I will shortly contact FIRST tech support.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Did your team program in Java ? Did you make use of the smart dashboard ?

Re: Human Vision Camera compatiable with FMS & Java ?
 

I will add that FRC11 has a Logitech C920 on their robots, running in Java, connected to the RoboRIO and no one has come to me with any issue. We even hosted a week 1 district event where I was a CSA and there was no issue at all.

Fair note: if your Java code has an issue starting the camera server correctly it may not matter what USB camera you use it may not work correctly. When FRC11 first connected the Logitech C920 they didn't put the Java code and in there so it did not work. I can also see it being possible someone could fail to start the camera server on the RoboRIO the way it should be and make a problem at that point. The sample code for Java in the Eclipse editor gives a great place to start so that you don't say - try to instantiate the camera server in teleop ;).

Re: Human Vision Camera compatiable with FMS & Java ?
 

If you could send me some links, I will verify everything with the lead programming mentor.

Everyone: Please remember the 3000 worked fine via wifi and tethered at all times outside of a regional. It only did not work when connected to FMS.

Re: Human Vision Camera compatiable with FMS & Java ?
 

I am going to direct quote my directions from an e-mail I sent to my students on FRC11:

Do you want me to upload the sample code?
I would like to see your code, as I am sure Greg would as well.

Re: Human Vision Camera compatiable with FMS & Java ?
 

I didn't forget this information, and I don't think others have either.

I think the primary mechanism for the FMS to break your USB camera is by changing the timing of when the client and server apps start and finish their connection. I saw very similar issues with a team at the Bayou regional and who developed a process to work around the issues.

There is, by the way, no official statement about what USB cameras are compatible with or incompatible with the FMS. The FMS should have no impact on whether a particular USB camera works. To the FMS, the images are simply data in a TCP stream.

I'd also like for you and all teams to have a reliable camera for championships -- and all the events. In order to look into this better, can you summarize which dashboard you are using and how the camera is being served on the roboRIO?

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

Greg and techhelpbb: Thanks for the input. Sorry I was away. Our team has a quite high level of technical expertise. We have been at it for over a decade. We did do (AXIS) camera based target tracking a few year ago. We are pretty good at following installation instructions and the MS 3000 worked fine everywhere except on the field. We program in java and have a fairly advanced smart dashboard with the low res camera view in a window. We have heard of a number of other teams also having usb camera issues when connected to the field but then we have heard of teams with 2 cameras working fine. We attempted to contacted FIRST a week ago but have not heard anything back. We have decided to focus on a supported AXIS IP camera for the worlds as we just cannot afford to risk what happened at our two regional s.

For anyone, if your USB camera worked at a regional, it would be nice to know
1) did you program in Java
2) do you use the smart dashboard
3) what version of windows did your laptop use
4) what resolution you used for the camera

Re: Human Vision Camera compatiable with FMS & Java ?
 

Since this thread touches on FMS networking, I'll post a link to the most recent FMS Whitepaper that explains how all of it works:

https://wpilib.screenstepslive.com/s...fms-whitepaper

The main point is that FMS is "just a network" and only communicates with your DS (not the robot).

Re: Human Vision Camera compatiable with FMS & Java ?
 

Can your team provide the robot code? Private message would be fine.
If you send it to me I will only forward to Greg or FIRST.

It is no skin off my back if you do not want to use a USB camera at all.
I would, however, like to find a root cause for the issue.

Re: Human Vision Camera compatiable with FMS & Java ?
 

I'm not doubting your level of expertise. I was helping a team just this weekend doing C++ with a mentor who does video processing for a living. But reliable client/server video serving is not easy. There have been many bugs in the video stuff in WPILib too, and I suspect that is what we are talking about here.

My main point is that FIST didn't declare that the MS Lifecam is not supported, but if your code's timing has uncovered a bug in the camera server, you can likely reproduce it by changing timing between connections or similar modification in how the client and server start up. Once you can reproduce the bug, you can likely work around it and/or ensure that the Java folks can make progress on it.

Axis cameras have a built-in server and are another good way to get a reliable transmission. If you can't get the USB to work, Axis is a good backup plan.

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

re "I'm not doubting your level of expertise."
Sorry just wanted to give you guys some additional background info.

re "My main point is that FIRST didn't declare that the MS Lifecam is not supported"
Just a reminder that the CSA and I believe it was the FTA at Northbay told us specifically that FIRST knows of an issue with the MS 3000 and FMS. It might have been Java related. I am now trying to contact FIRST HQ to find out exactly what the issue is.

I really appreciate the high level of interest in resolving this both you guys are showing. I need a few more days to replicate our environment in my own test environment and hopefully hear what FIRST has to say re what the CSA said.

Re: Human Vision Camera compatiable with FMS & Java ?
 

As a member of the FRC Control Systems Team, the author of some of the camera support code, and a frequent CSA, I'm trying to squash this statement. The FMS has no involvement with the DS/DB/camera comms. The field network is involved. It blocks some ports and affects connection timings, just as almost any ethernet or wifi network will. But it has no idea what is on your encrypted TCP/UDP stream between your devices. Waiting for FIRST to fix the FMS so that this camera works is not a good use of your time.

I'm happy to help you identify your issue, but pointing at the FMS is not productive. Instead, we need to look at which feature of the Java libraries you are using, which server, which client, etc. We can either look at the specifics of your USB solution, to help identify what is likely a timing issue in WPILib or how it was being used, or we can look at your proposed IP camera to help ensure that it works for you at your next event.

Greg McKaskle

Re: Human Vision Camera compatiable with FMS & Java ?
 

We have successfully used TWO LifeCam HD 3000. One is connected directly to the roboRIO (programmed in Java) and streamed to the smart dashboard using the CameraServer object. The other is connected to a raspberryPi and piped through mjpeg-streamer to grip. We did not feed the second camera stream to the driver station, but the first camera went to the driver station. We had this configuration at both Palmetto and Smoky Mountains regionals with no issues.

Re: Human Vision Camera compatiable with FMS & Java ?
 

As an FTAA and CSA myself, like Greg said,
After another quick skim through this thread, I also haven't seen you clarify exactly what "doesn't work" on the field. Does the stream not appear at all on the Dashboard? Is it choppy or frozen on an image? Do you see any logs? What kind of things did you try and troubleshoot with the FTA/CSA at your event? All of those things can help us understand the problems you're facing and give better ideas on what to try next.

That being said, I'll spitball a few ideas

• Not all network ports are open on the field, see R60 this year
  
  Quote:

  Originally Posted by R60 Communication between the ROBOT and the OPERATOR CONSOLE is restricted as follows: A. Network Ports: i. TCP 1180: Camera data from the roboRIO to the Driver Station (DS) when the camera is connected the roboRIO via USB, bi-directional. ii. TCP 1735: SmartDashboard, bi-directional iii. UDP 1130: Dashboard-to-ROBOT control data, uni-directional iv. UDP 1140: ROBOT-to-Dashboard status data, uni-directional v. HTTP 80: Camera connected via switch on the ROBOT, bi-directional vi. HTTP 443: Camera connected via switch on the ROBOT, bi-directional vii. UDP/TCP 554: Real-Time Streaming Protocol for h.264 camera streaming, bi-directional viii. UDP/TCP 5800-5810: Team Use, bi-directional Teams may use these ports as they wish if they do not employ them as outlined above (i.e. TCP 1180 can be used to pass data back and forth between the ROBOT and the DS if the Team chooses not to use the camera on USB).

• QoS limitations. The radio will limit your connection to 7Mbit/sec (see R60B). If you have the camera configured to stream super high quality video at 60fps, you may run into trouble. This document explains how you can test your bandwidth usage at home. Additionally, the 7Mbit limit is a theoretical ceiling, wireless conditions at your specific event may preclude such speeds. In any case, robot control packets are prioritized over video streaming, so packets from your camera may get dropped, causing issues with your stream.
  
• This one is a real long shot, but I vaguely remember a discussion on a CSA call about a Java WPILib bug with the CameraServer code, I believe it's this on the tracker. I don't know enough about this to recommend further troubleshooting to see if this is what you're experiencing though, I'd have to defer to someone else on the subject.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Thanks Phil.

We did notice our cpu loading drop from 90% to I believe 50% when we removed the usb camera. We had been concerned about our 90% cpu usage since our first regional.

We are pretty certain we are in compliance with all the port and bandwidth limitations but I will be rechecking that personally this week (I am not the programming mentor who is busy with enhancements for champs). And the CSA was looking at it at the regional.

I am pretty sure we were running 320x200 resolution but certainly no more than twice that.

Re: Human Vision Camera compatiable with FMS & Java ?
 

Hi Greg - thanks for the clarification re your involvement. I'll pm you shortly with more info and code (it may take a day to get to the code that used the USB camera which has since been updated).

Re: Human Vision Camera compatiable with FMS & Java ?
 

Sounds like this bug.

You can either recompile wpilibj or you can copy cameraserver.java into your project and make the change there.

I saw this bug on 2 teams this past weekend and it sounds identical to your issue.

Re: Human Vision Camera compatiable with FMS & Java ?
 

This may be what we should have tried...
What I can tell you... same code worked flawlessly in 2013-2014 and worked using 1180... using 554 only worked in one match at Lubbock. The code is essentially ffmpeg... ffplay that has been modified to pass frames... so all connection code is deep within ffmpeg itself. My point is that its a well defined solution, worked flawlessly in the pit... can withstand cold cycles, and passes a simulated fms environment.

I'm replying here, because there may be a chance that someone may have similar issues, and someone may have some insight.

There is one other thing... our URL is
rtsp://FRC:FRC@10.34.81.11:554/axis-media/media.amp

Not sure if the login and password are an issue...


I've used this to mitigate the race condition of the connection and this has worked fine in outside FMS environment, and previous years (using 1180).

The thing I find most frustrating with this problem is that I cannot pursue it, and that makes it very frustrating. I'm hoping we can simulate FMS enough to reproduce it, or find some insight as to what it can be. We can only speculate, and wait until the last moment to test again. :(