~hruske Hruške, jabuke, jablane, čežane. » debian
Home Contact Sitemap

Hruške, jabuke, jablane, čežane.

Ste se gdaj vprašali, zakaj Najboljšega soseda nikoli ni doma, ko pridete na obisk?

A replacement for bzip2 compression

Posted on Maj 15th, 2008 in debian, linux |

I was looking for a replacement for the old and slow bzip2. Not to say it doesn’t work, it’s just too slow for use case I have, and gzip just doesn’t bring enough space savings. And, after all, why settle for less than you’re able to achieve?

So, after checking out Linux compression utilities, the only one that fits my requirements:

  • it should be included in Debian repositories
  • it should compress approximately as good as bzip2 does in similar time, but should decompress faster
  • if possible, it should be free software

The only real contestant is 7-zip, which uses the super efficient LZMA algorithm for compression. It can be quite slow, though. So, the idea is to try to fine tune the utility, to use at most the space bzip2 would or less, and be faster when decompressing. p7zip, the Unix port, has similar compression settings as gzip has, ranging from 1 to 9. I tested some of them, to find optimal settings for my use case and made some benchmarks. I used three different test files, all of which were tar files, but with different contents. Test case 1 was 40MB of text, test case 2 was about 200MB of a recent Haiku OS image and test case 3 were essentially a 70MB bunch of Java JAR files.

This is how the archives compressed. Numbers are normalized to bzip2, for comparison.
7-zip vs. bzip2: Archive size comparizon

Time needed for compression:
Time needed for compression by algorithm and test case

Time needed for decompression:
Time needed for decompression by algorithm and test case

You can see that 7zip always decompresses faster, and that in general, higher 7z compression makes the archive decompress faster. Interesting.

Some more info:

  • 7zip was Debian package p7zip-full 4.57~dfsg.1-1
  • bzip2 was Debian package 1.0.5-0.1
  • test machine was 2.16GHz Macbook with 2GB RAM, doing only the tests
  • frequency scaling was off
  • all files were first cached in RAM by doing “cat file > /dev/null” so disk I/O was not the bottleneck
Komentarji so izklopljeni

Disconnecting the hard drive, the software way

Posted on Maj 8th, 2008 in debian, linux |

A hard drive recently died, taking away lots of data and filling the logs with failed ide requests. Annoying in too many ways. So, how do you make the disk stop yelling?

The easiest is to make the kernel not see it. There’s a neat trick that enables you to do just that, and it’s called module unbinding. So, basically you unmount the drive (or what’s left of it), then you only need to figure out what device to disconnect from which driver, which isn’t that hard. If you do “ls -al /sys/block/sda/”, you’ll see something like this:

drwxr-xr-x  8 root root    0 maj  8 21:47 .
drwxr-xr-x 20 root root    0 maj  8 21:47 ..
-r--r--r--  1 root root 4096 maj  8 21:47 capability
-r--r--r--  1 root root 4096 maj  8 21:47 dev
lrwxrwxrwx  1 root root    0 maj  8 21:47 device -> ../../devices/pci0000:00/0000:00:1f.2/host2/target2:0:1/2:0:1:0
drwxr-xr-x  2 root root    0 maj  8 21:47 holders
drwxr-xr-x  2 root root    0 maj  8 21:47 power
drwxr-xr-x  3 root root    0 maj  8 21:47 queue
-r--r--r--  1 root root 4096 maj  8 21:47 range
-r--r--r--  1 root root 4096 maj  8 21:47 removable
drwxr-xr-x  4 root root    0 maj  8 21:47 sda1
drwxr-xr-x  4 root root    0 maj  8 21:47 sda2
-r--r--r--  1 root root 4096 maj  8 21:47 size
drwxr-xr-x  2 root root    0 maj  8 21:47 slaves
-r--r--r--  1 root root 4096 maj  8 21:47 stat
lrwxrwxrwx  1 root root    0 maj  8 21:47 subsystem -> ../../block
-rw-r--r--  1 root root 4096 maj  8 21:47 uevent

The line of our interest is

lrwxrwxrwx  1 root root    0 maj  8 21:47 device -> ../../devices/pci0000:00/0000:00:1f.2/host2/target2:0:1/2:0:1:0

where you can see, that the device id is “2:0:1:0″. The last step is to actually unbind the device:

cd /sys/block/sda/device/driver
echo -n "2:0:1:0" > unbind

Et voila, your hard drive is no more as far as the kernel knows.

Komentarji so izklopljeni

Forgot your Kopete password?

Posted on September 11th, 2007 in debian, linux, python |

I sure did, and I you didn’t use KWallet, just like me, then you are probably going to find this script a bit helpful when retrieving your forgotten password. Just save the code as a file, eg. getpass.py, and run python getpass.py userid@jabber.org. You will get your password in no time. Be sure to run it in your home dir.

from ConfigParser import ConfigParser
import sys
from kdecore import KStringHandler

config = ConfigParser()
cryptedpass = config.get("Account_JabberProtocol_" + sys.argv[1], 'Password')
print KStringHandler().obscure(cryptedpass.decode("utf-8"))

Komentarji so izklopljeni

New Egenix Python modules

Posted on Junij 9th, 2007 in comp, debian, pootle, python |

Great news! Egenix has released a new version (3.0.0) of mxTextTools. The most important improvement is probably Unicode support. I have considered mxTextTools as a candidate for speeding up Pootle, but previous versions lacked unicode support. And the new version is already in Debian unstable.

Now that this is solved, I might even give it a try when I find some time.

Komentarji so izklopljeni

NFS prek IPv6

Posted on April 24th, 2007 in debian, linux |

Še ena dobra IPv6 novica. Z Linux kernelom 2.6.21 bo NFS v jedru dobil podporo za IPv6, kar v praksi pomeni, da NFS do Kiberpipe od doma ne bo več tako neverjetna stvar.


Komentarji so izklopljeni