How to "zip" a big string to small ?

pysome has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks
I have a huge big string to store into DB.
But you know, it take too much space.So i need to zip(or compress) the big string at first,then store it to DB.
e.g.

my $a = 'big big string here ....';
my $b = zip($a);
[download]

if the length($a) is far less than length($b) ,then it will be OK.
What's way can be used? Thanks

Comment on How to "zip" a big string to small ? Download Code

Replies are listed 'Best First'.
Re: How to "zip" a big string to small ? by dHarry (Abbot) on Oct 28, 2008 at 08:41 UTC
First of all, how big is a "huge big string"? Secondly, what DB do you use; sometimes the DB can do the compression for you. Third as suggested Compress::Zlib could be an option (in general fast and good compression). But there are many others on CPAN . In general it depends on what you try to compress (structure of the string), and the size of it. You don't give info on those. I don’t think Archive::Zip is of any use to you, unless you want to store zip-files in your DB.	[reply]
Re^2: How to "zip" a big string to small ? by pysome (Scribe) on Oct 29, 2008 at 03:29 UTC
Thanks your reply. 1)The big string ,i mean the string is large enough,about 8-10k 2)I use the SqlServer ( varchar(max) ) to store the string. 3)I just compress common string ,eg.: `my $string = " aaaaaaaaaaaaaaaaaaaaaaaa sdfsdf bbbbbbbbbbb sdfsdfsdf sdfffffffff sdfsdf < tabl> sfsdfffffffffffffffffffffffffffffffffffffffffff ... much and more .... "` [download]	[reply] [d/l]
Re^3: How to "zip" a big string to small ? by dHarry (Abbot) on Oct 29, 2008 at 08:07 UTC
1)The big string ,i mean the string is large enough,about 8-10k This doesn’t strike me as "big". 2)I use the SqlServer ( varchar(max) ) to store the string. SQLServer should have not trouble handling those strings. 3)I just compress common string ,eg.:... Any of the suggested compression alternatives should work for you. Based on the example you provide I assume that you will obtain excellent compression ratios. I would like to draw your attention to one more alternative: RLE which stands for Run Length Encoding. It doesn’t give you a great compression ration but it’s very simple and therefore fast. I have found a Perl implementation for it which can be found here (I did not tested it though)	[reply]
Re^4: How to "zip" a big string to small ? by pysome (Scribe) on Oct 29, 2008 at 09:10 UTC
Re: How to "zip" a big string to small ? by ccn (Vicar) on Oct 28, 2008 at 08:06 UTC
perldoc Compress::Zlib perldoc Archive::Zip	[reply]
Re: How to "zip" a big string to small ? by zentara (Archbishop) on Oct 28, 2008 at 21:45 UTC
I don't know how it compares to compression, but you can try a base 128 conversion. `#!/usr/bin/perl use warnings; use strict; use Math::BaseCnv; #fast my $time = time; print "time -> $time\n"; my $time_128 = cnv( $time, 10, 128 ); print "time_128 -> $time_128\n"; #this will sometimes print hidden newlines exit;` [download] I'm not really a human, but I play one on earth Remember How Lucky You Are	[reply] [d/l]

Back to Seekers of Perl Wisdom