View Single Post
Posts: 1 | Thanked: 0 times | Joined on Feb 2009
#48
Hi guys,

I'm going to try one of those wikipedia reader as well. I just noted that all of them use bzip2 compression. There is maybe a better solution using LZMA or more specifically Lzma Utils ( new version beeing named xz)

http://tukaani.org/lzma/
http://tukaani.org/xz

A quick check on file enwiki-20081008-pages-articles.xml

Code:
bzip2 4.1GB
xz    3.3GB


and decompression is faster than with bzip2 (they claim 2-5x faster, haven't tried by myself)

Last edited by fik; 2009-02-10 at 13:41.