Image processing on the driver station laptop

[cbf]

Is there anything in the rules preventing us from doing our rectangle recognition on the driver station laptop rather the cRIO? Even in Hybrid mode?

Just want to make sure..

[Alan Anderson]

I don't see anything in the rules that prevents processing the camera image on the computer running the Driver Station.

[AndyB871]

I think it's legal as long as you keep in mind the ports that are available on the field.

If you look at one of the more recent team updates:



The Arena
The Player Stations

Once plugged in to the Field Management System via the Ethernet cable provided, the ports that the teams will be able to access on the playing field are as follows:

– TCP 1180: This port is typically used for camera data from the cRIO to the DS when the camera is connected to port 2 on the cRIO. This port is bidirectional on the field.

– TCP 1735: SmartDashboard, bidirectional

– UDP 1130: Dashboard-to-Robot control data, directional

– UDP 1140: Robot-to-Dashboard status data, directional

– HTTP 80: Camera connected via switch on the robot, bidirectional

– HTTP 443: Camera connected via switch on the robot, bidirectional



All these ports are open on the playing field, so a team can use them as they wish if they do not employ them as outlined above (i.e. TCP 1180 can be used to pass data back and forth between the robot and the DS if the team chooses not to use the camera on port 2).

We took this to mean that we could freely use ports 1130 and 1140 to communicate data via UDP from our Dashboard to our robot and back.

In fact, this is how we're implementing our vision tracking algorithms right now. Our PC receives the Image, processes it and generates a list of targets, then that list of targets is prioritized and sent back to the robot for tracking.

-Andy

[moc601]

I've been looking for a definitive rule in the manual and other docs without success.

Are the Driver Station and Dashboard permitted to be separate laptops?
Are two laptops permitted to be used during the contest?

I'm wondering how a team would make use of the web services interface otherwise.

[moc601]

Quick followup to refine my question.

I've read about the $400 component limit that COTS laptops appear to be subject to.
The location of the laptop would be in the pit, next to the provided netbook drivers station. I seem to remember some teams having two laptops in the pit with them in our rookie year 2011.

[AndyB871]

a definitive rule

|-> See my post above about port #'s I think that's pretty definitive at least on the network level.

Are the Driver Station and Dashboard permitted to be separate laptops?
Are two laptops permitted to be used during the contest?
|-> That's a good question. Maybe you should make a post at a higher level in the forum about this. I'd guess that it's okay, because you're still restricted as to what ports you can use and how the robot must be controlled, etc.

At the same time however, It's possible that the Field system is designed to disallow traffic from any IP address not 10.xx.yy.6 , in which case the second laptop (whichever one does not have the *.6) wouldn't be able to communicate, that's just me throwing a possibility out there. I've got no way to back that one up.

Edit:
You're not restricted to any number of laptops in the pit. You're only restricted on the playing field, anyone please correct me if I'm wrong there. I sure hope not.... we bought ourselves a ~$350 laptop with a nice big display to use for our DS/Dashboard

[moc601]

Edit:
You're not restricted to any number of laptops in the pit. You're only restricted on the playing field, anyone please correct me if I'm wrong there. I sure hope not.... we bought ourselves a ~$350 laptop with a nice big display to use for our DS/Dashboard[/QUOTE]

Just to be painfully clear:)
So if we're in the pit with the KOP netbook and also a COTS $350 laptop we are not going to commit a foul on that basis alone.

[AndyB871]

No. You're fine in the pit; I don't know if that is OK on the field though.

[moc601]

Thanks!

[sjspry]

The rule about the usable ports should be last years. This year, there seems to be no restrictions.

Also note that last year's enumerated ports weren't necessarily the only ones open on the field in practice, but I wouldn't've relied on it.

[mutantblond]

you mean to say that we're restricted to a computer for our driver's station that cost less than $400?

[mutantblond]

just answered my own question: rule 13 says that the operator console is excluded from the $3500 maximum to spend on the robot.

[cbf]

In fact, this is how we're implementing our vision tracking algorithms right now. Our PC receives the Image, processes it and generates a list of targets, then that list of targets is prioritized and sent back to the robot for tracking

Andy --

Are you using Labview? It looks to me like UDP Port 1130 is the only UDP port one can definitely depend on being open from the Driver Station to the robot. If that's the case and you're sending your own custom packet with the list of targets, how do you get the robot code to distinguish it from the normal driver station control packet?

If you're not using UDP Port 1130, how can we be sure that any other port will actually be open at the competition?

[WizenedEE]

The rule about the usable ports should be last years. This year, there seems to be no restrictions.

oh? http://frc-manual.usfirst.org/viewSingleItemMap/694

Make sure to update your manual for the team updates.

[AndyB871]

cbf -
Yep, we're using labView.

The robot code doesn't have to distinguish anything here. The Driver station packets are being transmitted over a different port (1165 I believe). I'm not sure how familiar you are with networking but since they're different ports you can think of them as disjoint channels. A UDP receive on port 1130 will never see a packet from port 1165 and vice versa (sp? I never get that one right)

So in a nutshell, we just open a new socket on 1130 on the Dashboard, and pack the data and ship it out to the robot IP address; and in the Robot code We create a new socket to listen for data on 1130, and do a UDP receive followed by some error checking and an unflatten.

-Andy

[Chris Hibner]

I ran a test using UDP two days ago and I wasn't very happy with the results - perhaps one of the NI guys could say if what I saw is typical or not.

I set up a test program to communicate via UDP from a laptop to the cRIO. The laptop was set up to countinuously count up and the value of the counter was passed to the cRIO via UDP. I read the received counter value while in debugging mode in LabVIEW.

The issue is this: if the data wasn't immediately received on the cRIO, the data was queued and was then read eventually in the original order that it was sent. For example, the counter on the laptop would be counting through 200 and the received packet on the cRIO would be paused at 180 for a second. Then, when the cRIO started receiving packets again, it would start counting up again from 181 even though the laptop was now on 220. For a real-time system, I would expect that all missed/delayed packets would be ignored and the most recent packet would be read.

This is a real problem if I'm trying to receive target aim and distance data from the image processing on a remote computer. I have no idea how much delay there is from what I'm receiving to when it was sent. If some delay occurs, I could be off by X seconds for the remainder of the match.

I ran the same experiment with TCP and did not see this issue.

[Alan Anderson]

Chris, what caused the packets to be missed/delayed in the first place? I've been using UDP to send the vision processing results to the robot without noticeable issues. Can you show the code you used to test it?

[Chris Hibner]

Chris, what caused the packets to be missed/delayed in the first place? I've been using UDP to send the vision processing results to the robot without noticeable issues. Can you show the code you used to test it?

As soon as I saw that behavior, I deleted the code and replaced it with the TCP version. I was reading that TCP has error checking for delayed and missed packets while UDP didn't, so I thought I would give TCP a try immediately.

The code was basically the example UDP client / UDP server code from the example folder, just reworked to pass the counter. On the cRIO side, the receiving was being done in the fast loop of Periodic Tasks (which we have set to 20 ms). The laptop was running a bare-bones dashboard project, and the transmit was being done in a 20 ms loop.

I'm pretty sure the problem on the receive side was we were over loading the CPU occasionally (sometimes more than occasionally). We have a lot of test code in there and running in debug mode slows things down even further. However, the behavior seen is still troublesome because I can almost guarantee there will be occasions that we'll overrun the CPU this season.

The troublesome part was that the transmission is a straight-forward FIFO queue. In the example of my previous post, if we had stopped the laptop when the counter was at 250 and the cRIO was showing the counter at 210, the cRIO would show the counter counting up 211, 212, etc as if the laptop was still alive and sending, even though it was stopped. The cRIO would continue to count up until it reached 250 (the value that the laptop was at when we stopped it), and THEN it would notice that it was stopped. The cRIO side didn't notice an issue until about 2 seconds AFTER stopping the laptop from transmitting.

[dellagd]

Yes! Careful with UDP. The difference between UDP and TCP is a pro vs con thing. UDP is faster, but you do trade of accuracy. From where im sitting (Hehe..Apollo 13), UDP just really transmits data. thats is. What happens to said data is not the senders problem. It just transmits and whatever gets there gets there. (We see this alot with the NetConsole(Which is UDP based) when some print statements will loose a letter in the beginning sometimes). With TCP, It actually, Im not sure how this part works, but it makes sure the data gets there. This process takes time, and is therefore slower.

Pro vs Con , as I said before.

[Chris Hibner]

UDP just really transmits data. thats is. What happens to said data is not the senders problem. It just transmits and whatever gets there gets there.

Yeah, I read that in the documentation. I was keeping my fingers crossed that the receiver side would be a little more like SPI or async PAS5 communication: if the receiver didn't want to use the data, it just gets overwritten by the next message. I didn't realize that when the receiver finally decided to get the message, you don't know how many messages ago you were getting. Oh well, TCP seems to be working for us. However, I may find a way to break that as well.

[Greg McKaskle]

I'm a little surprised by what you describe about the UDP test, but that is also written a bit differently that I'd normally write it. Rather than delay the UDP read loop, I'd have the loop be throttled by the UDP Read timeout parameter. As for whether the UDP buffer size, I'm sure the buffers are limited in size, but I'm not sure of the size or policy.

Greg McKaskle

[Chris Hibner]

I'm a little surprised by what you describe about the UDP test, but that is also written a bit differently that I'd normally write it. Rather than delay the UDP read loop, I'd have the loop be throttled by the UDP Read timeout parameter. As for whether the UDP buffer size, I'm sure the buffers are limited in size, but I'm not sure of the size or policy.

Greg McKaskle

Thanks for the response. I haven't tried changing the buffer or playing with the timeout. I might try that if I get some time.

[wireties]

The classic illustration if that UDP is like sending a letter. You write it, put it in an envelope and send it to the destination. It probably gets there, especially if the destination is local. BUT there is no way to be sure unless you build an acknowledgement into your application protocol.

TCP is more like a phone call. There is a distinct connection phase followed by the exchange of information (acknowledged by each party) and a disconnect phase.

UDP is faster - on highly reliable links. TCP is made to work between any two endpoints no matter the delay and reliability of the link. TCP will keep counts of packets going back and forth, ack them all and automagically retry if something is dropped. UDP just sends and forgets.

In this case I would try UDP and enumerate the packets. If you are missing a packet and/or one is out of order just throw that image away and wait for the next. Like Alan said, it should not happen very often.

And the socket address is a combination of the IP address AND the port number. So the ports to and from your robot to the driver station should always be available to you. Their availability have little to do with other robots on the field.

HTH

[cbf]

AndyB871 --

Yes, I'm very familiar with networking. (I'm a mentor, not a student.) I had assumed since the rules said port 1130 was the only one available for Operator station to Robot communication that they meant it and therefore the driver station must use that port. (I tried reading the DrvrStn COM vi, but the important stuff all seems to be buried in the opaque NETCOM library.)

Having that port available for our own purpose will make this much easier. I was getting ready to Wireshark this to see what was going on, so thanks for saving me the trouble!

Chris --

Yes, you cannot assume that subsequent UDP packets will simply replace earlier UDP packets that have not yet been read. Most operating system network stacks (and I presume that would include VxWorks) helpfully provide a certain amount of buffering to held several packets. If your packets are really short, then the network stack will hold more of them for you!

I wouldn't switch to TCP -- that just adds more overhead. The answer is either to throttle (would require back traffic), or have your code on the cRIO flush packets to catch up.

[Alan Anderson]

It's very likely that the 20 ms sampling of Chris's receiving UDP socket was at the root of the problem. If the Driver Station's 20 millisecond UDP send was running just slightly faster than the one on the cRIO -- which it almost certainly would be if the cRIO code were being run in debug mode -- then packets would start to stack up. Since the cRIO was only retrieving one packet each 20 milliseconds, it could never catch up once it got behind. If the UDP receive function itself were used to control the loop rate, with a few seconds (or more) timeout, it wouldn't fall behind the way the original code can.

[Greg McKaskle]

I think we are all getting on the same page. If you were to use WireShark, and please do if you have the time to expose the kids to it, you will see that the heartbeat of the system are UDP packets flowing from DS to robot and back every 20ms. If they glitch for more that 100ms, the robot gets disabled because the NETCOMM library doesn't feed the FPGA with the next encoded cookie.

I can't say how much UDP buffering there is, but there is definitely some. I'm sure it can be adjusted, but I'd think of that as being helpful in making the OS smaller, not more realtime. The key to having all the other loops doing UDP is to allow all of the readers to be clocked by arriving packets. Nothing else throttles them -- most have a timeout of one second in order to set some defaults. The writer loops are the ones that have the delay, and for that, I use a timed loop which is critical priority. It only takes one timed loop for the whole system and everything else responds to the sent packet. Other writing loops such as for the Kinect use regular loops.

If you want to see the packet statistics, the Charts tab on the DS now displays the lost packets per second and the trip time. Approximately 50 are sent, and the scale for total trip time to and from the robot is 100ms.

Please let me know if you see other odd behaviors or have questions.

Greg McKaskle

[Chris Hibner]

I thought about this a little more, and the behavior of the UDP seems logical to me now. The protocol is made for transferring contiguous data (like strings or files) where missing a packet is a bad idea (it would make your string nonsense or corrupt your file). Therefore, making sure all of the data is received is more important than the timing of the data.

Keep in mind that prior to this experiment, my knowledge of TCP or other comm protocols aimed at the PC world is virtually nil. My experience is more in the real-time sensor protocols where the data is usually single sensor readings that change rapidly. In my world, receiving everything that is sent is usually much less important than making sure you have the most recent data (although in an ideal world, you like to have both).

I guess we're kind of trying to use a square peg to fit through a round hole. I'm seeing that you may have to whittle the peg down a bit, but eventually you can get it to fit. I think I have what I need to make it work.

[Joe Ross]

I thought about this a little more, and the behavior of the UDP seems logical to me now. The protocol is made for transferring contiguous data (like strings or files) where missing a packet is a bad idea (it would make your string nonsense or corrupt your file). Therefore, making sure all of the data is received is more important than the timing of the data.

Both TCP and UDP are typically implemented with multiple levels of fifos (buffer in the library at the send side, buffer in the os in the send side, buffer in the ethernet chip in the send side, buffer in the ethernet chip at the receive side, etc). This does complicate only reading the most recent data. I have not looked at the LabVIEW UDP libraries, but (based on experience with UDP on other embedded systems) I would expect there is a way to peek into the buffer to see how much is waiting, read all of it, then only use the latest data in your code.

If you research UDP vs TCP, you'll actually find the opposite conclusion (UDP loses data, TCP doesn't). However, this comes from use over a lossy connection, such as the internet. UDP just blasts out the data. If one packet gets routed through Russia, it will be received after a later packet that gets routed more directly. It can also get lost. It's used for protocols where missing or out of order information is not critical, but low latency is. TCP on the other hand is a handshaked protocol where if a packet is lost, it will be re-transmitted or if a packet is delayed, it will be re-assembled in the correct order. This is what you use if want to transfer contiguous data. Neither tries to lose data, but only TCP guarantees you don't lose data.

Now, in a low utilization local link like the robot network, in reality packets don't get lost or delayed, so there is little difference between TCP and UDP.

I'm not sure why you see different results with tcp. If anything, the results should be worse, given the same algorithm for reading data.

[Chris Hibner]

I'm not sure why you see different results with tcp. If anything, the results should be worse, given the same algorithm for reading data.

I'm thinking I would attribute that to sample size (i.e. I didn't do enough testing to see it yet).

[Jon236]

I'm a little surprised by what you describe about the UDP test, but that is also written a bit differently that I'd normally write it. Rather than delay the UDP read loop, I'd have the loop be throttled by the UDP Read timeout parameter. As for whether the UDP buffer size, I'm sure the buffers are limited in size, but I'm not sure of the size or policy.

Greg McKaskle

That worked for us.