Rsync command used to copy and sync files, directories locally or remotely. Learn rsync with different examples and options.
Rsync i.e. Remote Sync is the command used for copying and synchronizing files or directories locally or remotely on Linux Unix based systems. In this article we will learn how to use rsync command. different switches of it and examples.
As you know Rsync not only copies but also synchronizes files and directories, it is best for mirroring data between two machines and data backups. Rsync is an efficient way than any other copy command since it takes bandwidth and time only for the first time of data copy. In consecutive sync cycles, it only transfers data which is changed. This requires less bandwidth and time, but you still have updated data at the destination! Hence it is best for remote data mirrors and backups.
Lets see Rsync command with different uses and their examples.
Rsync command should be available on your system in plain vanilla installation. If not, install the ‘rsync’ package. Basically without any switch you can run the command with just source and destination path. The path can be local or remote.
# rsync testfile /tmp/
This is a very basic simple example of rsync command. It won’t show any output and copy/sync data from source file ‘testfile‘ to destination file under /tmp directory.
Rsync command with verbose mode
-v switch can be used for verbose mode. Information like byte sent, bytes received, transfer speed, the total size of the source is displayed on the console when command run.
# rsync -v testfile /tmp/directory9/
created directory /tmp/directory9
testfile
sent 79 bytes received 31 bytes 220.00 bytes/sec
total size is 7 speedup is 0.06
If you watch closely, Rsync will create a destination directory if it does not exist!
Rsync command to copy directories
-r (recursive copy) the switch helps in copying directories recursively to destination.
# rsync -rv testdir /tmp/
sending incremental file list
testdir/
testdir/test2
testdir/test4
testdir/test1/
testdir/test3/
sent 194 bytes received 62 bytes 512.00 bytes/sec
total size is 0 speedup is 0.00
In the above example we used verbose mode as well. Directory testdir has two files named test2, test4. It also has directories named test1, test3. All these 4 entities copy over to destination as you see their listing in output.
Preserve permissions and ownership while rsync copy
-a (archive mode) the switch used to instruct rsync to preserve file directory permissions and ownership at the destination.
# ll test1
-rw-r--r--. 1 root root 0 Jul 23 22:40 test1
# rsync -av test1 /tmp
sending incremental file list
test1
sent 69 bytes received 31 bytes 200.00 bytes/sec
total size is 0 speedup is 0.00
# ll /tmp/test1
-rw-r--r--. 1 root root 0 Jul 23 22:40 /tmp/test1
You can observe source and destination permissions and ownership is preserved after using -a switch with rsync.
Rsync with compression
One of the important factors while copying data is bandwidth. Bandwidth utilization can be reduced with compressed data. Compressed data transfer is more speedy, takes less time and bandwidth. Rsync allows us to compress data during transfer using -z switch.
# rsync -azv test1 /tmp
sending incremental file list
test1
sent 66 bytes received 31 bytes 194.00 bytes/sec
total size is 0 speedup is 0.00
If you observe we synced the same file test1 from previous example output (obviously after removing /tmp/test1). Without compression it sent 69 bytes and now with compression 66 bytes! I know, 3 bytes is no big difference but these are empty test files. Compression makes a big difference with a bunch of huge files.
Rsync to copy from local to remote
For copying file/directory from local to remote, specify the destination as remote in notation username@hostname_or_IP:/path/
# rsync -azv test1 root@10.10.1.23:/tmp/
root@10.10.1.23's password:
sending incremental file list
test1
sent 66 bytes received 31 bytes 21323.00 bytes/sec
total size is 0 speedup is 0.00
Rsync to copy from remote to local
It will work like the above example only you have to reverse source and destination.
# rsync -azv root@10.10.1.23:/tmp/ test1
root@10.10.1.23's password:
sending incremental file list
test1
sent 66 bytes received 31 bytes 21123.00 bytes/sec
total size is 0 speedup is 0.00
I am transferring empty file here hence you see total size as zero.
Limiting bandwidth use in Rsync
Another important aspect of network file copy is to limit the bandwidth of transfer so that it won’t take whole available bandwidth, choke network, and degrade the network performance for other services/devices using it. Rsync has the ability to limit bandwidth with --bwlimit switch. The switch should be supplied to specify a maximum transfer rate in kilobytes per second.
# rsync -razv --bwlimit=100 httpd-2.4.25 /tmp/
sending incremental file list
httpd-2.4.25/srclib/apr/build/
httpd-2.4.25/srclib/apr/build/ltmain.sh
-----output clipped------
sent 1214030 bytes received 10136 bytes 97933.28 bytes/sec
total size is 83112573 speedup is 67.89
You can see speed limited to 97kbps (97933 bytes/sec) as per our specification 100kbps in command switch --bwlimit.
Show progress while transferring in Rsync
If you have a list of file directories with big size then its good to see the progress of each transfer so that you are assured that transfer is in progress and not hung. Specify --progress switch so that transfer details shown for each file transfer in output.
# rsync -razv --progress httpd-2.4.25 /tmp/
sending incremental file list
httpd-2.4.25/
httpd-2.4.25/.deps
0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=1312/1314)
httpd-2.4.25/.gdbinit
10566 100% 0.00kB/s 0:00:00 (xfer#2, to-check=1311/1314)
httpd-2.4.25/ABOUT_APACHE
13496 100% 2.57MB/s 0:00:00 (xfer#3, to-check=1310/1314)
httpd-2.4.25/Apache-apr2.dsw
65980 100% 10.49MB/s 0:00:00 (xfer#4, to-check=1309/1314)
----- output clipped -----
In the above output you can see after each file copied, transfer speed details, the duration is flashed on the screen.
Apart from above all switches, there is the whole list of switch which can be used with Rsync command. All of them can be found under the Rsync man page on your terminal. I am listing a few important one below for your quick reference.
-h: Output numbers will be shown in a human-readable format--include: Include files. Wild cards can be used--exclude: Exclude from the copy. Wild cards can be used-q: Quiet mode. Suppress non-error messages--backup: Make a backup-l: Copy links as well-H: Preserve hard links-t: Preserve modification times--dry-run: Perform dry run. No changes actually made.--delete-after: Delete files after transfer
Let us know if you want additions of examples of any specific switch in the below comments. I will add them accordingly.
A very good stepping stone, thank you