Gobby

gobby.debian.org debconf12/bof/shrink-repository

Intro
 - speaker = Hideki Yamane https://wiki.debian.org/HidekiYamane/
   not good at English, so please speak clear and slow, guys :)

Links
 - http://penta.debconf.org/dc12_schedule/events/930.en.html
 - http://www.slideshare.net/henrich_d/lets-shrink-debian-package-archive


**If you have a question, please add it NOW :) **

-------------------------------------------------------------------------------
Agenda
-------------------------------------------------------------------------------
* How large is Debian Repository
  -> ???GB
* solution for that
  -> xz
 * Is it really effective?
  -> Yes/No
 * Problem on slower Arch
  - mips, arm, etc...
 * How much can we shrink it?
  - ???GB -> ???GB 
 * other benefit
  - download traffic
  - download time
  
-------------------------------------------------------------------------------
problem?
-------------------------------------------------------------------------------
 - Uses too memory hungry xz compression??
 - compression
    + much CPU time
    + much RAM (Bug#679774 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=679774?)
    + not scale with multi core
 - decompression time

 please also see debian-CD and disscution in -devel

Adam Borowski says:
 https://lists.debian.org/debian-devel/2012/07/msg00157.html
> * XZ is almost always better:
>   • xz -0 is twice as fast as gzip, with 78% the size (tested on a random
>     92MB unstripped amd64 executable)
>   • xz -3 is at par with gzip -9's compression speed
>   • xz -6 (the default) is a lot slower when compressing, fast when
>     decompressing, needs only 10MB memory, 58% size
>   • xz -9 has very slow compression, takes gobs of memory, 56% size
>     (Obviously, the "size" numbers are dragged down by uncompressible files
>     when you look at the whole archive.)
>   • It has somewhat slow start: small files may compress better with gzip,
>     but never to a degree that would justify the complexity of switching
>     compression algorithms.

 As I tested, xz -9 shrinks 1/3 archive (all, *i386 and *amd64 (+ia64))

> * After recompressing the whole archive, it turns out compressing only
>   biggest packages is not a good idea: the bulk of space is taken by
>   medium-sized packages, which recompress almost as good.  Thus, for best
>   effects, xz should be dpkg-deb's default.

 Almost yes, but there's a little problem with decompression time.
 
 
-------------------------------------------------------------------------------
Pros vs Cons
-------------------------------------------------------------------------------
Pros)
 -
 -

Cons)
 -
 -