Idea
A fast and unencrypted data transfer for local network usage. The
idea is to send a tar stream over TCP by using the
nc tool.
The pipe viewer (pv) is optional but allows tracking the
transfer rates and total data transmitted. By comparing with the
original data sizes, this can act as a progress indicator.
Local Data Copying with Tar
tar -C SOURCE-DIR -c . | pv | tar -C DEST-DIR -xNetwork Data Copying
To copy data as-is, be sure to invoke the tar commands
as root such that they can read and write all data including permissions
etc.
The --numeric-owner flag is useful to transfer data
as-is even on systems with different user and group IDs.
Sender Side
The sender acts as a client:
tar -C SOURCE-DIR --numeric-owner -c . | pv | \
nc -q 10 192.168.2.1 2244Receiver Side
The receiver acts as a server on address 192.168.2.1 and port 2244. Change as required.
nc -w 10 -l 192.168.2.1 2244 | pv | tar -C DEST-DIR --numeric-owner -xVerifying Integrity
To check if the file contents match on both ends, compute
sha256sum and compare it on the target. Note that if the
paths are not equal on source and destination, it may be required to
edit checksums.sha256 before using it on the receiver
side.
Sender Side
find SOURCE-DIR -type f -exec sha256sum {} + | tee checksums.sha256Receiver Side
sha256sum -c checksums.sha256 | tee checksum-results.txt
grep -v OK checksum-results.txtData Copying and Checksum Computation in One
Requires shell capable of process substitution.
Sender Side
tar -C SOURCE-DIR -c . | pv | tee >(sha256sum > sha256-sent.txt) | \
nc -q 10 192.168.2.1 2244Receiver Side
nc -w 10 -l 192.168.2.1 2244 | pv | \
tee >(sha256sum > sha256-recv.txt) | tar -C DEST-DIR -xAfterwards, compare contents of sha256-sent.txt and
sha256-recv.txt. They should be identical!
See Also
nc(1), pv(1), rsync(1), tar(1), tee(1)
Article with similar basic ideas: