Go to Post There is nothing that says 'gracious professionalism' like making brunch for your opponents. - Monochron [more]
Home
Go Back   Chief Delphi > Technical > Programming
CD-Media   CD-Spy  
portal register members calendar search Today's Posts Mark Forums Read FAQ rules

 
Closed Thread
Thread Tools Rate Thread Display Modes
  #1   Spotlight this post!  
Unread 24-03-2010, 10:25
Geek 2.0 Geek 2.0 is offline
Registered User
FRC #0107
Team Role: Programmer
 
Join Date: Sep 2009
Rookie Year: 2008
Location: Holland, MI
Posts: 120
Geek 2.0 will become famous soon enough
Post What does the Camera Slow Down?

I've heard from a few people now that the camera slows down the dashboard. However, I'm not exactly sure what, how, or why it slows down. So...

If the dashboard slows down, is it only the dashboard? (For instance, the dashboard does update as frequently, but the robot still responds the same)

If the robot slows down, why and how? Is there a way around this?

Is the problem because people are sending the image to the dashboard? What if we don't do this?

I think you can get the general idea of what I'm looking for. We're trying to decide if the camera is worth the time/effort.
  #2   Spotlight this post!  
Unread 24-03-2010, 10:50
apalrd's Avatar
apalrd apalrd is offline
More Torque!
AKA: Andrew Palardy (Most people call me Palardy)
VRC #3333
Team Role: College Student
 
Join Date: Mar 2009
Rookie Year: 2009
Location: Auburn Hills, MI
Posts: 1,347
apalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond reputeapalrd has a reputation beyond repute
Re: What does the Camera Slow Down?

A few things
Generally, things slow down because there is either a lack of RAM, CPU power, or in some cases bandwidth.

1. The camera has to send complete images over WiFi 15 times per second. That's a lot of bandwidth, something WiFi can't always handle as fast as we would like.

2. The Dashboard Data is updated at the same frequency as the robot comm packets, 20hz or every 50ms. The Camera image is different, updating as fast as it can (which isn't very fast, usually around 10hz max)

3. If the robot slows down, either you are sending too much data and the field is throttling bandwidth, or your code is using more than the CPU can provide. I have seen both. If you only have a problem on the field, it's the first. Otherwise, it is probably the second. The solution is to compress the image more and send less (smaller image), or lessen use on CPU.

4. We have had problems with the Dashboard image, but we didn't use it anyway so we disabled it.
__________________
Kettering University - Computer Engineering
Kettering Motorsports
Williams International - Commercial Engines - Controls and Accessories
FRC 33 - The Killer Bees - 2009-2012 Student, 2013-2014 Advisor
VEX IQ 3333 - The Bumble Bees - 2014+ Mentor

"Sometimes, the elegant implementation is a function. Not a method. Not a class. Not a framework. Just a function." ~ John Carmack
  #3   Spotlight this post!  
Unread 24-03-2010, 10:53
Geek 2.0 Geek 2.0 is offline
Registered User
FRC #0107
Team Role: Programmer
 
Join Date: Sep 2009
Rookie Year: 2008
Location: Holland, MI
Posts: 120
Geek 2.0 will become famous soon enough
Re: What does the Camera Slow Down?

Are there any suggestions for what you can do in code (we use Java) to make image processing more efficient so that it doesn't affect robot performance?
  #4   Spotlight this post!  
Unread 24-03-2010, 11:26
TubaMorg TubaMorg is offline
Programmermechanicalelect ricalcoach
AKA: Dan
FRC #1480 (Robatos Locos)
Team Role: Mentor
 
Join Date: Jan 2006
Rookie Year: 2005
Location: Houston
Posts: 450
TubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond repute
Re: What does the Camera Slow Down?

Quote:
Originally Posted by Geek 2.0 View Post
Are there any suggestions for what you can do in code (we use Java) to make image processing more efficient so that it doesn't affect robot performance?
We use this and haven't noticed any slow down:

Code:
cam = AxisCamera.getInstance();
cam.writeResolution(AxisCamera.ResolutionT.k320x240)
cam.writeCompression(30);
The image size/resolution and compression are the things to pay attention to as obviously the less compressed and larger the picture, the more bandwidth would be used up.
__________________
I don't need a signature.
  #5   Spotlight this post!  
Unread 24-03-2010, 11:53
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Is there any way to disable the sending of the video over the wire to the dashboard? We used the stock dashboard and only enabled the camera (at 320x240), and found that our controls were ~1 second delayed when we got on the field at portland.

We ended up just disabling all our camera code and just leaving the initialization and still saw a huge slow down, so we just decided to just give up ont eh camera entirely.
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
  #6   Spotlight this post!  
Unread 24-03-2010, 12:06
Dave Scheck's Avatar
Dave Scheck Dave Scheck is offline
Registered User
FRC #0111 (WildStang)
Team Role: Engineer
 
Join Date: Feb 2003
Rookie Year: 2002
Location: Arlington Heights, IL
Posts: 574
Dave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond reputeDave Scheck has a reputation beyond repute
Re: What does the Camera Slow Down?

Quote:
Originally Posted by ErichKeane View Post
Is there any way to disable the sending of the video over the wire to the dashboard? We used the stock dashboard and only enabled the camera (at 320x240), and found that our controls were ~1 second delayed when we got on the field at portland.

We ended up just disabling all our camera code and just leaving the initialization and still saw a huge slow down, so we just decided to just give up ont eh camera entirely.
When you instantiate the AxisCamera object, it creates the video server task behind the scenes. If you comment out the instantiation the server won't be started and video won't be sent.

EDIT: After rereading your post it sounds like you might be trying to use the camera onboard without sending video back. If that's the case you would need to specifically update the WPILib code to not start the server automatically.
  #7   Spotlight this post!  
Unread 24-03-2010, 12:07
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Quote:
Originally Posted by Dave Scheck View Post
When you instantiate the AxisCamera object, it creates the video server task behind the scenes. If you comment out the instantiation the server won't be started and video won't be sent.
I guess my point is: I want to be able to use the data from the camera on the cRio without streaming it to the Dashboard, which I suspect is the reason for the huge delay we saw when we got on the field.
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
  #8   Spotlight this post!  
Unread 24-03-2010, 13:09
synth3tk's Avatar
synth3tk synth3tk is offline
Volunteer / The Blue Alliance
AKA: David Thomas
no team
Team Role: Alumni
 
Join Date: Jan 2007
Rookie Year: 2007
Location: Ohio
Posts: 2,005
synth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond reputesynth3tk has a reputation beyond repute
Re: What does the Camera Slow Down?

Don't know if you saw this yet:
Quote:
Originally Posted by Dave Scheck View Post
EDIT: After rereading your post it sounds like you might be trying to use the camera onboard without sending video back. If that's the case you would need to specifically update the WPILib code to not start the server automatically.
  #9   Spotlight this post!  
Unread 24-03-2010, 13:11
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Quote:
Originally Posted by synth3tk View Post
Don't know if you saw this yet:
I didn't, thanks! He edited after I replied and was done. I was afraid of that answer... At least it'll give me something to do next year
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
  #10   Spotlight this post!  
Unread 24-03-2010, 14:13
TubaMorg TubaMorg is offline
Programmermechanicalelect ricalcoach
AKA: Dan
FRC #1480 (Robatos Locos)
Team Role: Mentor
 
Join Date: Jan 2006
Rookie Year: 2005
Location: Houston
Posts: 450
TubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond reputeTubaMorg has a reputation beyond repute
Re: What does the Camera Slow Down?

You really should be able to get it to work fine. Take a look at these threads:

http://www.chiefdelphi.com/forums/sh...ht=java+camera

But, seriously, if you haven't tried compressing the image, you will find that it will solve most lag issues (unless there are problems elsewhere in your code).
__________________
I don't need a signature.
  #11   Spotlight this post!  
Unread 24-03-2010, 14:17
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Interesting points in that Tuba, thanks.

I doubt it was the rest of the code, it was extremely simple code, particularly since the camera stuff wasn't there. Basically we got to Portland for a practice round, found it was really slow, so we disabled OUR camera code (left the initialization) and it was still really slow. So we removed the initialization and everything was fine.

We had compression at 80 (i think that was the recommended number last year?).

We were fine with the rest of the stuff on that list (other than resolution), since we didn't ever even use the developer account.

It worked perfectly either wired or on the practice field, but once we got to the actual field, there was a huge delay according to the driver.
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
  #12   Spotlight this post!  
Unread 24-03-2010, 14:28
slavik262's Avatar
slavik262 slavik262 is offline
We do what we must because we can.
AKA: Matt Kline
FRC #0537 (Charger Robotics)
Team Role: Alumni
 
Join Date: Jan 2007
Rookie Year: 2007
Location: Sussex, WI
Posts: 310
slavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to behold
Send a message via AIM to slavik262
Re: What does the Camera Slow Down?

Quote:
Originally Posted by apalrd View Post
1. The camera has to send complete images over WiFi 15 times per second. That's a lot of bandwidth, something WiFi can't always handle as fast as we would like.

2. The Dashboard Data is updated at the same frequency as the robot comm packets, 20hz or every 50ms. The Camera image is different, updating as fast as it can (which isn't very fast, usually around 10hz max)

3. If the robot slows down, either you are sending too much data and the field is throttling bandwidth, or your code is using more than the CPU can provide. I have seen both. If you only have a problem on the field, it's the first. Otherwise, it is probably the second. The solution is to compress the image more and send less (smaller image), or lessen use on CPU.

4. We have had problems with the Dashboard image, but we didn't use it anyway so we disabled it.
There's more than enough bandwidth to send the video. The main reason for the slowdown is how data is sent, not how much.

Networking 101:

In the modern world of networking, there are two main network protocols, UDP and TCP.

TCP (Transmission Control Protocol) comes first. It is an extremely common and robust protocol (http, the "internet protocol", runs on top of it). It works by forming a connection between two devices and then sending data across the connection. The protocol is managed: code running behind the scenes makes sure that the data arrives to the program using a TCP socket undamaged and in order. It does this in the following ways:
  • If a packet doesn't make it across the network intact, TCP stops all other incoming packets from being received and requests the bad packet to be resent.
  • If a packet arrives out of order, the TCP socket waits to deliver it until the packets that come before it in the stream arrive.
This can make TCP bad for use in real-time applications where getting new data (such as frames in a video stream) is more important than waiting for the old data to come in undamaged.

UDP (User Datagram Protocol) is the second type, and the simpler of the two. It is a connectionless protocol. Unlike TCP, sending data over a UDP socket just "shouts" it at the target IP without first forming a connection. There is no confirmation that the packets are received. The header of each packet sent across the network contains only its source, its destination, a packet length, and a checksum to confirm that this data hasn't been mangled. Because of its connectionless nature and small header, UDP provides no guarantee that the packets arrive intact or in order. It is up to the programmer to assure these things using data they send in the packets.

However, the less-structured nature of UDP can be an advantage in real-time applications where getting new data (such as frames in a video stream) is more important than waiting for old data to be resent or be put into order. Video chat programs such as Skype usually use UDP because of this.

How it relates:

The robot comm packets and dashboard packets are sent in UDP, which makes sense since up-to-date information is the critical factor here. However, for a reason that I cannot understand in any way, the video is sent using TCP. Consider why this is bad:
  1. If a frame of video is bad, it would make the most sense for the dashboard program to just skip it and move onto the next frame. However, this is impossible in TCP, as the connection itself stops until it can retrieve the bad frame, slowing down the dashboard.
  2. Not only is the dashboard slowed down, but the robot is slowed down too. The networking task in charge of sending the video has to stop sending new information in order to resend the bad data. This can cause performance issues in the robot as more time is spent "catching up" and less time is spent executing other code.

TheDominis and I appealed to the GDC some time ago to be allowed to develop our own UDP video system, but for whatever reason we were shut down. I'm still at a loss for why, but for now we're stuck with the TCP video feed.
__________________

Last edited by slavik262 : 24-03-2010 at 14:35.
  #13   Spotlight this post!  
Unread 24-03-2010, 16:31
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Useful post for many of the students here I'd suspect. All you caused for me is flashbacks of the 7 layer OSI model which I learned about for years. I'm intimately familiar with TCP vs UDP

I'd realized that the video here was sent via TCP (fired up Ethereal while dianosing issues during the build season).

What I figured was that there wasn't enough bandwidth for 6 robots running the video and other traffic at the competition, which is why it runs fine on the practice field/elsewhere, but not on the competition field.

I was really hoping there was an switch I could set in the initialization code that would disable the server, so I would be able to do my normal analyzation, and not the dashboard view (since I knew about the no UDP stuff form the GDC).
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
  #14   Spotlight this post!  
Unread 24-03-2010, 16:53
slavik262's Avatar
slavik262 slavik262 is offline
We do what we must because we can.
AKA: Matt Kline
FRC #0537 (Charger Robotics)
Team Role: Alumni
 
Join Date: Jan 2007
Rookie Year: 2007
Location: Sussex, WI
Posts: 310
slavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to beholdslavik262 is a splendid one to behold
Send a message via AIM to slavik262
Re: What does the Camera Slow Down?

The problem seems to have much more to do with the Classmate having issues rendering video and working the TCP stream at the same time due to its absolutely [sarcasm]wonderful processor[/sarcasm]. It seems that the best/only solution at present time is to compile WPILib yourself and not initialize the PCVideoServer when the camera is initialized.
__________________

Last edited by slavik262 : 24-03-2010 at 16:56.
  #15   Spotlight this post!  
Unread 24-03-2010, 16:54
ErichKeane ErichKeane is offline
Registered User
FRC #3210
Team Role: Mentor
 
Join Date: Nov 2003
Rookie Year: 2004
Location: Hillsboro, OR
Posts: 113
ErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really niceErichKeane is just really nice
Send a message via AIM to ErichKeane
Re: What does the Camera Slow Down?

Quote:
Originally Posted by slavik262 View Post
It seems that the best/only solution at present time is to compile WPILib yourself and not initialize the PCVideoServer when the camera is initialized.
Yep, seems like that is the case. Perhaps it would give me a reason to fix the WPILib like I've been promising myself for years
__________________
2004: Team 219- Programmer/Designer/Builder
2009: Team 2374- Mentor
2010: Team 3210- Mentor
Closed Thread


Thread Tools
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[FTC]: Slow Down Max Speed!! JohnFogarty FIRST Tech Challenge 10 16-11-2009 13:24
IR Sensor: Does Slow Loop Polling Work. de_ Programming 2 07-02-2008 23:49
Slow down teh motors! brennerator Programming 21 18-01-2007 23:30
What does your camera do? Qbranch General Forum 23 22-04-2006 02:26
Cheapest and easiest way to slow down a motor sanddrag Technical Discussion 41 21-12-2005 07:26


All times are GMT -5. The time now is 10:37.

The Chief Delphi Forums are sponsored by Innovation First International, Inc.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi