Saturday, December 18, 2010

Compress and split large files under linux

One of the common problems we used to face is transferring large files from Linux through mediums that doesn't support large files, especially when it comes to more than 4 GB files, usually that is a virtual machine hard disk, or backup file.

To compress and split files under linux, this command can be used:


$ tar -cvj big-file.dvi | split -b 650M -d - "small-file.tar.bz."

This command will compress and split the "big-file.dvi" to many files with size 650MB for each, giving it the name "small-file.tar.bz.0, small-files.tar.bz.1, etc...."

To join the files together again, this command can be used:

$ cat small-file.tar.bz.* > big-file.tar.bz

and to extract the output file :

$ tar -xvj large-file.tar.bz