Chief Delphi

Chief Delphi (http://www.chiefdelphi.com/forums/index.php)
-   Programming (http://www.chiefdelphi.com/forums/forumdisplay.php?f=51)
-   -   New compression method (http://www.chiefdelphi.com/forums/showthread.php?t=30187)

Aalfabob 11-09-2004 18:29

Re: New compression method
 
Quote:

Originally Posted by rbayer
A power point presentation? Why not just the decompressor/compressor programs. Or at least one of them. Based on your earlier posts/graphs, I'm guessing you've already written the compressor and decompressor, so if you could just attach them to an email, that would be great.

More on why you can trust me not to steal your ideas:
1. Eagle Scout
2. Well-respected around here
3. Other teams have trusted me (on numerous occasions) with their top-secret code for their robot in order to get some feedback on it. In ALL cases, I respected their wishes and never divulged any details. In fact, even if a team came up with a really cool idea (which many did), I didn't even incorporate it into my own team's code out of respect for the original programmers wishes.

Please, if you want people to believe you about this new compression scheme, I'm your guy. Send me a working compressor and/or decompressor and I promise to provide an honest opinion as well as keeping everything completely confidential. I will NOT, however, endorse a compression scheme simply because you know how to use Excel and make lots of pretty graphs.

Also, if all you haven't actually programmed the compressor/decompressor yet, then just send me a short (1-2 paragraph) description of how you plan to do it and I will give you as much feedback as I can on the feasibility of your scheme.

Rob

I have a full running compressor, but it was really messy, slow, and had no options as in selecting a file name and what not. I have been working on a new compressor portion that used bit operations that i really hadnt thought about when making the first one. So the new one is like 500 lines shorter and alot faster but not finished yet because it does not output it to a file to be recompressed.

I can give you build 1 of the compressor and a word document of how it works if you like. The document pretty much shows how it can reverse and how it compresses / uncompresses. Ill try to send you it by tonight because I have to leave in 30 minutes for a couple of hours but when i come back I should have plenty of time.

rbayer 11-09-2004 18:34

Re: New compression method
 
Quote:

Originally Posted by Aalfabob
I have a full running compressor, but it was really messy, slow, and had no options as in selecting a file name and what not. I have been working on a new compressor portion that used bit operations that i really hadnt thought about when making the first one. So the new one is like 500 lines shorter and alot faster but not finished yet because it does not output it to a file to be recompressed.

I can give you build 1 of the compressor and a word document of how it works if you like. The document pretty much shows how it can reverse and how it compresses / uncompresses. Ill try to send you it by tonight because I have to leave in 30 minutes for a couple of hours but when i come back I should have plenty of time.

Sounds great. If you need anything from me between now and then, either email me (FIRSTprograms@mn.rr.com) or IM me (FIRSTisLife) or PM me here. For the sake of everyone else reading these boards, let's try to avoid just posting to the forum if it's something that only concerns you and me.

Rob

Adrian Wong 11-09-2004 23:47

Re: New compression method
 
...and the plot thickens...

rbayer 12-09-2004 14:38

Re: New compression method
 
An update:

Last night, Aalfabob sent me his compressor along with a short description of how it works. I then proceeded to take 5 files from my computer (1 text file, 1 JPG image, 1 exe, 1 rar file, and 1 file of random data) and run them through his compressor. All files ended up being right around 800 bytes. I then emailed the compressed files back to him, and am currently waiting for him to decompress them and send them back to me. I'll let you guys know if they come back the same as the originals as soon as I get them.

-Rob

MikeDubreuil 12-09-2004 15:15

Re: New compression method
 
Quote:

Originally Posted by rbayer
I'll let you guys know if they come back the same as the originals as soon as I get them.

It would be a good idea to use an MD5 checksum to determine if the files decompressed are identical to the originals.

Fulcrum2000 12-09-2004 15:33

Re: New compression method
 
Quote:

Originally Posted by MikeDubreuil
It would be a good idea to use an MD5 checksum to determine if the files decompressed are identical to the originals.

Also make sure you rename(!) or move the original files before receiving the decompressed ones from Aalfabob... It would be even better to compare the files on a different PC then the one you used for compression.

Rafi A 12-09-2004 15:34

Re: New compression method
 
Quote:

Originally Posted by Adrian Wong
...and the plot thickens...

then it congeals

Ryan M. 12-09-2004 17:24

Re: New compression method
 
Quote:

Originally Posted by rbayer
An update:

Last night, Aalfabob sent me his compressor along with a short description of how it works. I then proceeded to take 5 files from my computer (1 text file, 1 JPG image, 1 exe, 1 rar file, and 1 file of random data) and run them through his compressor. All files ended up being right around 800 bytes. I then emailed the compressed files back to him, and am currently waiting for him to decompress them and send them back to me. I'll let you guys know if they come back the same as the originals as soon as I get them.

-Rob

How large where each of the respective files? 801 bytes? ;)

Aalfabob 12-09-2004 19:12

Re: New compression method
 
Alright theres something going on with my C++,
A variable is changing after a certain amount of time when im not even referring to it in the hole program except for the start. Ive checked to see if I misplaced a line somewhere and I havent, the compiler says that the variable doesnt even come up in the area where its changing. Any ideas? It seems very odd that its doing this unless another program is somehow editing the memory when its running.

Ryan M. 12-09-2004 19:22

Re: New compression method
 
Quote:

Originally Posted by Aalfabob
Alright theres something going on with my C++,
A variable is changing after a certain amount of time when im not even referring to it in the hole program except for the start. Ive checked to see if I misplaced a line somewhere and I havent, the compiler says that the variable doesnt even come up in the area where its changing. Any ideas? It seems very odd that its doing this unless another program is somehow editing the memory when its running.

It sounds like a stray pointer to me, but it's hard to know for sure. Does it happen consistently every time (IE is this variable unexpently changed every time, or just once in a while) and does it get changed in the same way? If there are any pointers, make sure you initialize each one before use. I know it sounds obvious, but it happens.

Aalfabob 12-09-2004 19:32

Re: New compression method
 
Quote:

Originally Posted by Ryan Morehart
It sounds like a stray pointer to me, but it's hard to know for sure. Does it happen consistently every time (IE is this variable unexpently changed every time, or just once in a while) and does it get changed in the same way? If there are any pointers, make sure you initialize each one before use. I know it sounds obvious, but it happens.

It happens every time and changes to a different value. I was using this variable to hold the file size and it is not used in any functions. It also seems that it is happening at about the same time from the start of the program. I ran a debug and nothing seems to be wrong but the one variable changing. The variable stays the same through the hole program (Which runs in a loop) for a while and then it just changes.

Aalfabob 12-09-2004 21:29

Re: New compression method
 
RBayer checked over my code a little and found a little problem in it. In one of my if statements i put an = instead off an == which somehow through off the results. With this fixed it seems that the files are getting larger. But in another build ive made without this error they seem to get smaller. There probebly is an error in that one to, but ill check it another day just incase it can get some files to really small sizes.

The problem wasnt really with the ability to uncompress the data to normal state, but the way the data comes out as when it compresses each time. So say you compress it once and it gains compression, the way it is put back into the file makes it impossible to gain any more compression, and will ussually loss some. I dont know how this actually happens but there must be a law to keep random data from being compressed (Not the way stated before with the n-1 or whatever it was).

And as stated before it is impossible to do this lol, if anyone wishes to take a look or anything about how I tryed to do this just leave me your email and ill send you a .doc or I could just post it here if anyone wishes. Hopefully noone has wasted to much time with this thread.

FizMan 12-09-2004 21:39

Re: New compression method
 
Post it here please!

Alan Anderson 12-09-2004 22:00

Re: New compression method
 
Quote:

Originally Posted by Aalfabob
RBayer checked over my code a little and found a little problem in it. In one of my if statements i put an = instead off an == which somehow through off the results.

That single feature of C syntax has probably caused more programs to fail than any other bug in the history of computing.
Quote:

...there must be a law to keep random data from being compressed (Not the way stated before with the n-1 or whatever it was).
I actually thought that explanation was quite well presented.

The "law" is pretty simple: you need a certain number of bits in order to represent a certain amount of information. If a file has redundancy, or repeating patterns, it contains less information than if it had no such patterns. The redundancy can be removed and the file can be represented in a smaller form. Text tends to exhibit statistical patterns. Executable programs and pictures and audio have other patterns. Truly random data by definition has no redundancy, and thus can't be compressed without losing information.

Max Lobovsky 12-09-2004 22:15

Re: New compression method
 
Quote:

Originally Posted by Alan Anderson
If a file has redundancy, or repeating patterns, it contains less information than if it had no such patterns. The redundancy can be removed and the file can be represented in a smaller form. Text tends to exhibit statistical patterns. Executable programs and pictures and audio have other patterns. Truly random data by definition has no redundancy, and thus can't be compressed without losing information.

That isn't entirely true. Of course, as stated many times in this thread, no compression scheme can reduce the size of all possible files, or, alternately, consistently reduce the size of a stream of random data. The problem with Mr. Anderson's explanation is that the idea of "truly random data" doesn't really have any effect on the matter. Given a finite piece of data, how does one decide if its random or has patterns? Well, it's impossible, but one way we can sort finite pieces of data by randomness is to choose a compression scheme and check if it can compress the piece of data or not. Of course, this will not yield the same classification of randomness as with any other compression scheme.

The reason we can sort of decide if data is random is that we can easily some types of patterns and use that as our "compression scheme."


All times are GMT -5. The time now is 13:31.

Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2017, Jelsoft Enterprises Ltd.
Copyright © Chief Delphi