2009 - Live camera feed to drivers during a match?
 

People seem to ask this question every year, and in the past, the answer has always been "NO", but this is the first year a video signal can be transmitted back through the wireless bridge for display on a LabVIEW dashboard program, so...................

I was looking through Section 8 of the Manual, and I could not find anything that prohibited displaying the live video feed from the camera on the PC dashboard at the operator's station during a match.

Enlightenment requested.

Re: 2009 - Live camera feed to drivers during a match?
 

From which camera? Standard rules are in place for non-KOP cameras (<R69>).

However, the KOP camera is classed as a vision sensor (<R02>C refers to vision sensors as remote sensing capabilities, and the camera is certainly a vision sensor). I can't find anything that would prohibit that one being sent back to the operator station. I would ask in Q&A. I expect that the answer would be "yes" for that camera only.

Re: 2009 - Live camera feed to drivers during a match?
 

Yes, I was specifically referring to the Axis camera.

Re: 2009 - Live camera feed to drivers during a match?
 

In which case, I refer you to <R81>. You can connect a computer for viewing feedback, which could easily include information from a vision sensor (aka Axis camera).

I would still ask in Q&A, but I am 99% sure that the answer will be something to the effect of, "If the camera is the Axis camera, go right ahead. If it is another camera [possible exception: CMUCamII], you won't get past inspection."

Re: 2009 - Live camera feed to drivers during a match?
 

I have seen no rule to prevent such an action, and it is at least technically possible with the new control system. As for any benefit to be had from it, I would think it would be negligible, especially given the ~5-7 fps framerate on the display over wifi.

Re: 2009 - Live camera feed to drivers during a match?
 

Actually, unless something has changed the answer is, "NO. During match play, all ports except those used by the Drivers Station are blocked by the field management system. This rules out being able to get camera feedback, since getting camera feedback is not a primary function of the Driver Station but instead is done on a separate port."

The issue is bandwidth, and guaranteeing bandwidth, IIRC. At least, this has been the answer every time I've ever asked the FRC development team here at NI, even up to 2 weeks ago. Then again, the Game Design Committee has the final say, so it'll have to come from them.

-Danny

Re: 2009 - Live camera feed to drivers during a match?
 

OK cool, thanks. I do recall that restriction being mentioned on the forums sometime late last year - just wanted the "official" answer - I feel this is close enough, and I think the GDC has more important questions to answer anyway. ;)

Re: 2009 - Live camera feed to drivers during a match?
 

I remember seeing that as well.

It seems to me that this is actually a rather important question. Our design-in-process will be much easier to operate with camera video available. If the GDC does prohibit us from sending camera video to the laptop, hopefully they'll still allow us to send graphical feedback. My system will work fine as long as I can use data from the camera to draw a picture.

Perhaps we have to find the intensity and color of each pixel and send that to the DS so we can turn it back into video at the laptop. (Hopefully not...)

Re: 2009 - Live camera feed to drivers during a match?
 

Woah,
Does that mean I cant show the rpms of my wheels? or the status of limit sensors? getting more feedback from the robot was one of the things I was really looking forward to, especially in a match where it may be hard to see some of your robot.

Re: 2009 - Live camera feed to drivers during a match?
 

I agree that this is an important question, and I was surprised it wasn't explicitly addressed in the competition manual.
It was mentioned as being unlikely we would be able to receive live video feed during competition at the WPI control system seminar and the same point has been repeated since. My understanding is that "live" video is sent back as a distinct stream from the standard driver station data. The latter does include some user data so reporting RPM etc should be OK. I haven't tried sending images back this way, so I don't know what the user data limitations are.

Our experience is that the update rate & transmission delays make it unlikely that the video on the dashboard could be the sole means of driver feedback - at least for a maneuverable robot on carpet - but maybe it would be OK for this years skidfest!

However it is up to the GDC, so we should know soon after Q&A is open tomorrow.

Re: 2009 - Live camera feed to drivers during a match?
 

Rule <R81> in the robot section of the manual says

<R81>
Teams are permitted to connect a portable computing device (Laptop computer, PDAs, etc.) to either of the Ethernet ports on the Driver Station for the purpose of displaying feedback from the ROBOT while participating in competition MATCHES. Portable computing devices may only connect to the Driver Station through one of the Ethernet ports – they shall not connect to the Driver Station through any other port. Portable computing devices may only connect to the Driver Station – they must not directly connect to any ARENA ports or equipment. Please note that AC power will not be available at the playing field so these devices will have to run on internal batteries.

The bolded section will solve your problem:]

Re: 2009 - Live camera feed to drivers during a match?
 

When I went to control system training at NI in Austin this very question was brought up.

You CANNOT stream video back for viewing on the dashboard, THIS YEAR. FIRST is worried about the load 6 robots streaming up 640x480 video would put on the field network. So for now the answer is no.

As for other feedback from encoders, switches, etc... It's fair game. If I understood things correctly you can also pull information from the FMS such as time remaining, field status and operator mode.

Re: 2009 - Live camera feed to drivers during a match?
 

Is it possible to bring data back simply to the driver station for display or use there?

I see that the driver station has the potential to hook up multiple input devices to the digital and analog ports. Neat. BUT - is there a way to send information back to the driver station and use that information? Either for display on the device (if there is room) or through digital output.

In Labview I see something called 'Set User Data'- but it basically says you can send 984 bytes from the host computer (I'm assuming that means cRio) to the driver station.

Crazy thought - think about enabling the shaker on a joystick under certain conditions...

Re: 2009 - Live camera feed to drivers during a match?
 

Then they might want to put that out in public, via an update or Q&A. Sorry, Jon, but if it isn't in the rules, it isn't in the rules, and right now, it isn't in the rules (or Q&A, the interpreters of the rules).

Next up: someone with a Q&A account asks this question. (I don't have one.)

EDIT: Double apologies to Jon, but the GDC has made the ruling that it is legal. Practical remains an unknown factor.

Re: 2009 - Live camera feed to drivers during a match?
 

*warning:theoreticallness ahead*

Since streaming of video isn't explicity prohibited in the rules(it's just that the port is firewalled off),it might theoretically to possible to have a thread on the cRio that would "tunnel" video data over the user data interface, and code your dashboard program to reconstruct the video, similar to how you would set up a TCP/IP tunnel on a computer. Of course, I don't know what kind of data rate you could get, or whether it would interfere with your controls

Re: 2009 - Live camera feed to drivers during a match?
 

See my above post.

Re: 2009 - Live camera feed to drivers during a match?
 

From the GDC, "However, limitations on the data transfer rate provided by the wireless communication system may provide a practical limit on the effectiveness of the implementation."

Does this mean they are implementing limitations on the data transfer rate? Using their Cisco equipment they could easily cap the bandwidth to make it impossible for a team to transfer live video to the dashboard.

Re: 2009 - Live camera feed to drivers during a match?
 

I think it's more complete to say that the GDC has stated that it is legal, but they reserve the right to make it impractical :)
I wonder if "limitations on the data transfer rate provided by the wireless communication system" will become less onerous throughout the competition season?

Re: 2009 - Live camera feed to drivers during a match?
 

I think most of you do not understand... the data transfer limit is already in place. You will NOT be allowed to stream from the camera directly to the driver's station. As posted a few times in other threads, ALL network traffic other than the robot control & dashboard traffic will be firewalled at the field. This means the only way you can get an image back from the camera is to take the camera data on the CRIO, pack it into the dashboard data structure that is allowed to be sent back, and unpack it at the other end. From what I understand, the amount of data that can be packed into the dashboard packet is a little under 1000 bytes. I don't know offhand how many times per second this packet is sent.

I think a lot of people don't realize (and FIRST has not made this very clear) that you will not be allowed to transmit whatever network traffic you like. You can do it at home for practice, but once you get to the field it will be blocked.

Re: 2009 - Live camera feed to drivers during a match?
 

I'm glad that update 5 makes up for the GDC's previously vague answers.

Re: 2009 - Live camera feed to drivers during a match?
 

Bandwidth from 6 robots while managing a field has always been the concern with going to the new WIFI system. Many factors including possible interferrence from venues and overall security weigh on this. Especially when looking at situations like Atlanta with 4 fields running at once. While the new fields are using the latest State of the Art Wifi, there has not been enough characterization done of what adding a real time video stream from each robot will do to the overall performance of controlling the bot over the WIFI simultaniously. We will be collecting data from all regionals through this season to see what the possible impacts are.

Begrudgingly, the decision was made to put off being able to stream video from bots through this current regional timeline. I have faith that we will prove it can be done reliably after getting data from all venues this year and using robot streaming video in the OFF SEASON so it can be utilized in future regional events. :cool:

BotBash BOB Pitzer
4FX Design

Re: 2009 - Live camera feed to drivers during a match?
 

Streaming video to the teleoperators will only contribute to pilot workload and doesn't take advantage of the robot's onboard processing capability. My team plans to use a single bit head-up display as we did in 2006. When the robot's locked on (distance and azimuth) and ready to shoot, an LED in the co-pilot's safety glasses will light up and the co-pilot can fire away.

Re: 2009 - Live camera feed to drivers during a match?
 

You are given 984 bytes 50 times a second to do what ever you please with. Testing has already shown this to be very very very doable, and has been available to you from day 1.

If you want to send back sensor readings, go for it. If you want to send back internal state, go for it. If you want to send back a stream of robotics related limericks that are generated programmaticly in response to field conditions, go for it. You can even use it to send back video if you'd like.

However, 984 bytes at 50Hz is what you get. That won't fit full motion video, but it might be able to fit heavily compressed / decimated video. For example, you could easily send back a black/green/pink thresholded stream. If you are a total nutter, you could send back info on object positions to reconstruct a picture of what things should be on your laptop.

For reference, this is about 7 times faster than a 56k dial up modem. Enough to do a lot of cool stuff, not enough to be wasteful.

Re: 2009 - Live camera feed to drivers during a match?
 

This can lead to an interesting thought experiment.

* To be most useful, the images sent to the driver should be sent at reasonable video speeds (say 10 Hz or so)

* This means we need to be able to compress a single frame down to 5*984 bytes = 4920 bytes

* At 640x480x24 bits per pixel (standard RGB color), a single uncompressed image is 921,600 bytes.

* At 160x120 at 1 bit per pixel (binary black and white), a single uncompressed image still needs 2400 bytes. Hmm... We could even have 2 bits per pixel (4 shades of gray) and still fit under the limit.

Think about this one possible implementation: We have 5 different types of 984-byte packets. Each packet needs a header telling the dashboard PC which type it is, along with a frame number (to correlate out-of-order packets and drop old ones). It would also be helpful to have target positions (for, say, up to 3 targets we are tracking). Then comes the payload of raw image data (I want to avoid compression for now).

Implementation:

Packet 1:
1 byte for correlation (high 4 bits are the packet id (1); low 4 bits are the frame number that rolls over ever second and a half or so)

3 bytes for target #1:
+1 byte indicating which type of target, and whether or not it is visible
+1 byte indicating Y position in the frame
+1 byte indicating X position in the frame

3 bytes for target #2:
+1 byte indicating which type of target, and whether or not it is visible
+1 byte indicating Y position in the frame
+1 byte indicating X position in the frame

3 bytes for target #3:
+1 byte indicating which type of target, and whether or not it is visible
+1 byte indicating Y position in the frame
+1 byte indicating X position in the frame

974 bytes left over for payload data (first 1948 pixels of the image)...

Packets 2-5 would only need the 1 byte ID/frame #, and would have plenty of room left for the rest of the raw image data.

There are various methods for generating your 2-bit thresholded image; many are fairly inexpensive in terms of processing time.

---

I'm not sure about the utility of it, but I think that this shows that you might be able to get away with some "video" this year.