Idea
A fast and unencrypted data transfer for local network usage. The idea is to send a tar stream over TCP by using the nc tool.
The pipe viewer (pv) is optional but allows tracking the transfer rates and total data transmitted. By comparing with the original data sizes, this can act as a progress indicator.
Local Data Copying with Tar
tar -C SOURCE-DIR -c . | pv | tar -C DEST-DIR -xNetwork Data Copying
To copy data as-is, be sure to invoke the tar commands as root such that they can read and write all data including permissions etc.
The --numeric-owner flag is useful to transfer data as-is even on systems with different user and group IDs.
Sender Side
The sender acts as a client:
tar -C SOURCE-DIR --numeric-owner -c . | pv | \
nc -q 10 192.168.2.1 2244Receiver Side
The receiver acts as a server on address 192.168.2.1 and port 2244. Change as required.
nc -w 10 -l 192.168.2.1 2244 | pv | tar -C DEST-DIR --numeric-owner -xVerifying Integrity
To check if the file contents match on both ends, compute sha256sum and compare it on the target. Note that if the paths are not equal on source and destination, it may be required to edit checksums.sha256 before using it on the receiver side.
Sender Side
find SOURCE-DIR -type f -exec sha256sum {} + | tee checksums.sha256Receiver Side
sha256sum -c checksums.sha256 | tee checksum-results.txt
grep -v OK checksum-results.txtData Copying and Checksum Computation in One
Requires shell capable of process substitution.
Sender Side
tar -C SOURCE-DIR -c . | pv | tee >(sha256sum > sha256-sent.txt) | \
nc -q 10 192.168.2.1 2244Receiver Side
nc -w 10 -l 192.168.2.1 2244 | pv | \
tee >(sha256sum > sha256-recv.txt) | tar -C DEST-DIR -xAfterwards, compare contents of sha256-sent.txt and sha256-recv.txt. They should be identical!
See Also
nc(1), pv(1), rsync(1), tar(1), tee(1)
Article with similar basic ideas: