Distinguish Serial Data?

[6600gt]

How can you distinguish a group of bytes(data) form another?
Other than using a unique byte in front a group of bytes.

I just can't figure out how to do it.

For example, I am writing a program that sends a group of bytes periodically. I need to write a program that can distinguish between this data and data that I would type into the terminal window(not periodical). These are different programs so there is no way to syncronize them.

Now I know that only one program can access the serial port at any given time. The program I am writing allows you to turn the serial port on and off. This allows the terminal window to access the serial port without exiting my program.(Might need to restart the IFI loader again)

The only thing I could think of was to send a unique byte in front of the group of byte that I am about to send to the RC, so it can distinguish it from another group. What if I coincidentally typed that same unique starter byte in to the termial port-> ? What do I do now?

Thanks.

[TimCraig]

[quote=6600gt]How can you distinguish a group of bytes(data) form another?
Other than using a unique byte in front a group of bytes.
QUOTE]

If all you have is a stream of bytes coming through the serial port, there's no way other than try to come up with a unique "signature". IFI uses a two byte scheme of hex FFFF for the dashboard packets to synchronize them AND is careful to prevent most of their single byte data from taking on the value FF. This is why, if I remember right and I think there's one packet you have to be careful with, that the PWM values are interwoven in the packet since they're limited to values from 0 to FE (0 to 254).

If your data can take on all values, then using a one byte signature doesn't work because, in general, any byte in the stream has some probability of having that value. In a uniform distribution, this would be 1/256. A two byte signature reduces this to 1/65536 that any two byte sequence matches, etc.

IFI gets away with a little since their data is single byte and a large part of it is range limited. If you expanded their data to 16 bits AND allow signed numbers, FFFF becomes much more likely since it's really -1.

Going to ASCII increases your overhead but can aleviate the problem if your data is all numeric or the nonnumeric entities have limited values. Then you can just "name" your packets.

[KenWittlief]

If the other device is known, like a keyboard, you could pick a starting flag byte that wont be used by the other device.

or the number of bytes you use for the flag (start characters) will determine the probability that stream will happen by random chance.

If you use an 8 bit start flag, the probability is 2^8 = 256:1

two bytes: 65,536:1

four bytes: 4.29 billion to one

your SW would then have to keep track - once you see the start flag you take the rest of the data as data. This implies you know how long the data packet will be.

[Mark McLeod]

Another possibility, depending on the data your program generates, is to use the high order bit in each byte as your id flag.

Anything you type into your terminal window will be ASCII. ASCII never uses the most significant bit in a byte, so in every byte originating from your terminal window, bit 8 will be zero.

Your program generated stuff can always set bit 8 to one.
Of course this limits the program data you send to 7-bit values, unless you split them over multiple bytes.

[Astronouth7303]

You may also be able to use the status lines in RS-232, although synchronization may be an issue.

Brief:
RS-232 was made for computers to communicate with modems, so there is a number of status lines available: RTS, DTR, CTS, DSR, RI, DCD. The first 2 are computer to device, the last 4 are device to computer.

Depending on the API available, you may not be able to accurately detect when exactly the status lines change. If this is the case, you will need to put breaks (technical term) in the data. A break is simply when no data is transmitted for a while (ie, not continuously).

Of course, if you're just transmitting data, and not using full RS-232, this will be of no use to you.

Another option would be to use 9-bit or larger words (word=a single transmitted frame). This may or may not be available depending on your hardware.

[6600gt]

Thanks everyone

I am using .NET 2.0 Framework and programming using Visual C++ 2005 Express Edition.

Right now I have three things connecting to the program port: the program I am working on, the terminal window, and an external pic.
Camera will always be on the TLL port.

My best option seems to be to use unique two byte starter for the external pic and my program, but to distinguish if it is the termial window I will have to detect the ASCII characters.

Do the printf strings use ASCII characters? How do you trasmit a 16 bit variable? Is it easier to just integrate the termial window into my program?

[Astronouth7303]

Originally Posted by 6600gt

Do the printf strings use ASCII characters? How do you trasmit a 16 bit variable? Is it easier to just integrate the termial window into my program?

Yes. Varies. Yes.

Most of the time (ie, in every instance I have seen), two applications cannot share a serial port. Therefore, you cannot use both IFI Loader's terminal and your app.

It is notable that, depending on what implementation you are using, the line endings may not be standard. (I don't know how well Microsoft's stuff handles \r.)

I'll leave the 16-bit question to your own pondering; the solution depends on your use for it.

[6600gt]

Ok I have a new problem now...

How demanding are serial port interrupts and communication in general on the RC?

I am writing an application that communicates to the RC through the program port. Some of the data that I am trying to send can be compressed into a byte...for example say I want to tell which pwms I want to be controlled by my app, I can either send 16 bytes(if I want to control all 16) numberd 1-16 or I can send 2 bytes(16 bits) to tell me which pwms to control. Now the bits require extra processing after the data has be recieved( you have the open up the bytes and go through an if statement then the data follows after in a sequence). On the other hand the bytes can be immedietly followed by the byte data for the particular pwm(thus a for loop can take care of all of them).

Using the bits requires 18 total bytes to be sent while using the bytes requires 32 total bytes(if I want to control all 16). I want to do this about 40 times a second. And I couldn't care less if the PC was doing all 64 bit floating point math...

Which one is more efficient and less demanding on the RC?

Thanks again.

[Tom Bottiglieri]

Originally Posted by 6600gt

How demanding are serial port interrupts and communication in general on the RC?

From experience on my team, the interrupts are more "interrupting" than you may think. Last season we ran a serial LCD screen on the robot. Between this, multiple encoder interrupts (@ 128ppr), timer based interrupts for analog sensors, and a decent sized application, we managed to crash the RC a few times, via IFI's master processor's watch dog timer. We then added a switch to disable the LCD during matches, which stopped the crashing all together. I know this wasn't the most scientific of methods to find the real problem, but the crashes stopped after we shut down all serial communication.

Originally Posted by 6600gt

Which one is more efficient and less demanding on the RC?

I guess this depends on your application. If you only plan on changing one or two output values per communication cycle, you may want to send the 8 bit output value attached to an 4 bit ID number. You would only send the "new" data when it is needed, and the RC application would parse everything out and assign motors new output values.

For example...

If output #2 needs to change, your serial string could be: 02160N (where N is your Null termination key.)

If output #3 and #12 need to change, your string could be: 0312712254N (in this #3 is changed to 127 and #12 is changed to 254)

If nothing changes, just send a null.

[Astronouth7303]

Originally Posted by Tom Bottiglieri

I guess this depends on your application. If you only plan on changing one or two output values per communication cycle, you may want to send the 8 bit output value attached to an 4 bit ID number. You would only send the "new" data when it is needed, and the RC application would parse everything out and assign motors new output values.

For example...

If output #2 needs to change, your serial string could be: 02160N (where N is your Null termination key.)

If output #3 and #12 need to change, your string could be: 0312712254N (in this #3 is changed to 127 and #12 is changed to 254)

If nothing changes, just send a null.

It's far faster if the RC doesn't have to perform string parsing. (Kevin knows this, too; the first thing his code does is put the camera in raw mode.)

"03127\0" is six bytes (30 33 31 32 37 00h). "\x03\x7F" is 2 bytes (03 7Fh). You can use "\xFF\xFF" (FF FFh) for synchronization.

If you're worried about over-taxing the processor, stay away from strings.

Also, serial data doesn't have to be continuously streaming. If there's no change, you just don't transmit anything.

As for that big block to sort through the PWM selection, you can just use an array. If you look at tx_data_record in ifi_default.h, you'll see that all the PWMs are contiguous. So you can use this:

char* pwms = &pwm01; // pwm01 => txdata.rc_pwm01
// elsewhere...
pwms[port] = value;

Of course, if that doesn't work like I think it should (ie, the pointers are screwed up), you'll never know it except by strange, erratic, and just puzzling behavior. Use the dashboard during testing; if user byte 2 or user cmd are anything but 0, something's off.

[Tom Bottiglieri]

Originally Posted by Astronouth7303

It's far faster if the RC doesn't have to perform string parsing.

Also, serial data doesn't have to be continuously streaming. If there's no change, you just don't transmit anything.

As for that big block to sort through the PWM selection, you can just use an array. If you look at tx_data_record in ifi_default.h, you'll see that all the PWMs are contiguous. So you can use this:

char* pwms = &pwm01; // pwm01 => txdata.rc_pwm01
// elsewhere...
pwms[port] = value;

Of course, if that doesn't work like I think it should (ie, the pointers are screwed up), you'll never know it except by strange, erratic, and just puzzling behavior. Use the dashboard during testing; if user byte 2 or user cmd are anything but 0, something's off.

Great point Jamie.

I probably should have worded my post in a different way. I didn't want to get across so much the method (string vs. hex vs. bin vs. whatever), but rather the idea of only updating certain array values when necessary. Now that I've thought about it, this probably isn't the best solution, but this kind of data syncing is what has been beat into my brain lately with some of the ajax stuff I've been doing.

So pretty much 6600gt just needs to decide whether he wants a static or dynamic serial stream. Static will be easier to parse, but may send a whole bunch of non changing data values that are just taking up space and transmission time. But then again, we're talking about data on such a small scale it probably won't make a huge difference in overall processing time. (Transmission + parse + execution)

[6600gt]

Originally Posted by Tom Bottiglieri

Great point Jamie.

I probably should have worded my post in a different way. I didn't want to get across so much the method (string vs. hex vs. bin vs. whatever), but rather the idea of only updating certain array values when necessary. Now that I've thought about it, this probably isn't the best solution, but this kind of data syncing is what has been beat into my brain lately with some of the ajax stuff I've been doing.

So pretty much 6600gt just needs to decide whether he wants a static or dynamic serial stream. Static will be easier to parse, but may send a whole bunch of non changing data values that are just taking up space and transmission time. But then again, we're talking about data on such a small scale it probably won't make a huge difference in overall processing time. (Transmission + parse + execution)

Thanks everyone

Dynamic serial stream seem real interesting because the mouse can only change 1 value at a time, thus only one value will be modified at any given time. This could help vastly to reduce the load on the RC, while increaseing the load on the PC...so I will do it.

Edit: I just realized its more complicated that this. There is a checkbox in front of each one of the 16 pwm sliders. If it is checked then I am sending the RC new values otherwise the RC will be sending me its current values for that particular pwm(not checked ones). So everytime I need to tell the RC which pwms I want to control and which ones I don't. Do you think static will work better here?

[yongkimleng]

Hi,

One method i've used before is to use 7 out of the 8 bits for data with the 1st bit to mark that the byte is the first byte in the sequence. The recieving end (computer) would do the byte counting.

Of course this may not be suitable if you require 8 bits of data in each byte sent

[6600gt]

Originally Posted by yongkimleng

Hi,

One method i've used before is to use 7 out of the 8 bits for data with the 1st bit to mark that the byte is the first byte in the sequence. The recieving end (computer) would do the byte counting.

Of course this may not be suitable if you require 8 bits of data in each byte sent

Thanks, but I need all 8 bits. This Dynamic serial data is hard...