Pipeing (e.g. tar with ssh)

From wiki.network-crawler.de

Copying/Moving files with two tar processes

/data/files/foo/* will be copied to /data/files/bar/

(cd /data/files/foo/ && tar cf - . )|(cd /data/files/bar/ && tar xvf - ) 

or change into the source directory e.g.:

cd /data/files/foo/
tar cf - . |(cd /data/files/bar/ && tar xvf - )

After changing to the source directory the first part compresses the files and writes them to stdout ( f - ). The second tar is started in a subshell with "( <command> )" and redirects stdout to stdin ( f - ) of the subshell and extracts it directly.

With ssh

same in green with ssh:

(cd /data/files/foo/ && tar cf - . )| ssh <serverip/name> "cd /data/files/bar/ && tar xvf -"

Experiments with buffers:

# time tar cf - 1GB | (cd /data/tmp/ && tar xf - )
real    1m20.494s
# time tar cf - 1GB | buffer -m 519200 | (cd /data/tmp/ && tar xf - )
real    1m16.462s
# time tar cf - 1GB | buffer -m 10m | (cd /data/tmp/ && tar xf - )
real    1m12.377s

special thanks to Raphael Becker for the information