Rsync command used to copy and sync files, directories locally or remotely. Learn rsync with different examples and options.
Rsync i.e. Remote Sync is the command used for copying and synchronizing files or directories locally or remotely on Linux Unix based systems. In this article we will learn how to use rsync command. different switches of it and examples.
As you know Rsync not only copies but also synchronizes files and directories, it is best for mirroring data between two machines and data backups. Rsync is an efficient way than any other copy command since it takes bandwidth and time only for the first time of data copy. In consecutive sync cycles, it only transfers data which is changed. This requires less bandwidth and time, but you still have updated data at the destination! Hence it is best for remote data mirrors and backups.
Lets see Rsync command with different uses and their examples.
Rsync command should be available on your system in plain vanilla installation. If not, install the ‘rsync’ package. Basically without any switch you can run the command with just source and destination path. The path can be local or remote.
# rsync testfile /tmp/
This is a very basic simple example of
rsync command. It won’t show any output and copy/sync data from source file ‘
testfile‘ to destination file under
Rsync command with verbose mode
-v switch can be used for verbose mode. Information like byte sent, bytes received, transfer speed, the total size of the source is displayed on the console when command run.
# rsync -v testfile /tmp/directory9/ created directory /tmp/directory9 testfile sent 79 bytes received 31 bytes 220.00 bytes/sec total size is 7 speedup is 0.06
If you watch closely, Rsync will create a destination directory if it does not exist!
Rsync command to copy directories
-r (recursive copy) the switch helps in copying directories recursively to destination.
# rsync -rv testdir /tmp/ sending incremental file list testdir/ testdir/test2 testdir/test4 testdir/test1/ testdir/test3/ sent 194 bytes received 62 bytes 512.00 bytes/sec total size is 0 speedup is 0.00
In the above example we used verbose mode as well. Directory testdir has two files named test2, test4. It also has directories named test1, test3. All these 4 entities copy over to destination as you see their listing in output.
Preserve permissions and ownership while rsync copy
-a (archive mode) the switch used to instruct rsync to preserve file directory permissions and ownership at the destination.
# ll test1 -rw-r--r--. 1 root root 0 Jul 23 22:40 test1 # rsync -av test1 /tmp sending incremental file list test1 sent 69 bytes received 31 bytes 200.00 bytes/sec total size is 0 speedup is 0.00 # ll /tmp/test1 -rw-r--r--. 1 root root 0 Jul 23 22:40 /tmp/test1
You can observe source and destination permissions and ownership is preserved after using
-a switch with rsync.
Rsync with compression
One of the important factors while copying data is bandwidth. Bandwidth utilization can be reduced with compressed data. Compressed data transfer is more speedy, takes less time and bandwidth. Rsync allows us to compress data during transfer using -z switch.
# rsync -azv test1 /tmp sending incremental file list test1 sent 66 bytes received 31 bytes 194.00 bytes/sec total size is 0 speedup is 0.00
If you observe we synced the same file test1 from previous example output (obviously after removing /tmp/test1). Without compression it sent 69 bytes and now with compression 66 bytes! I know, 3 bytes is no big difference but these are empty test files. Compression makes a big difference with a bunch of huge files.
Rsync to copy from local to remote
For copying file/directory from local to remote, specify the destination as remote in notation
# rsync -azv test1 firstname.lastname@example.org:/tmp/ email@example.com's password: sending incremental file list test1 sent 66 bytes received 31 bytes 21323.00 bytes/sec total size is 0 speedup is 0.00
Rsync to copy from remote to local
It will work like the above example only you have to reverse source and destination.
# rsync -azv firstname.lastname@example.org:/tmp/ test1 email@example.com's password: sending incremental file list test1 sent 66 bytes received 31 bytes 21123.00 bytes/sec total size is 0 speedup is 0.00
I am transferring empty file here hence you see total size as zero.
Limiting bandwidth use in Rsync
Another important aspect of network file copy is to limit the bandwidth of transfer so that it won’t take whole available bandwidth, choke network, and degrade the network performance for other services/devices using it. Rsync has the ability to limit bandwidth with
--bwlimit switch. The switch should be supplied to specify a maximum transfer rate in kilobytes per second.
# rsync -razv --bwlimit=100 httpd-2.4.25 /tmp/ sending incremental file list httpd-2.4.25/srclib/apr/build/ httpd-2.4.25/srclib/apr/build/ltmain.sh -----output clipped------ sent 1214030 bytes received 10136 bytes 97933.28 bytes/sec total size is 83112573 speedup is 67.89
You can see speed limited to 97kbps (97933 bytes/sec) as per our specification 100kbps in command switch
Show progress while transferring in Rsync
If you have a list of file directories with big size then its good to see the progress of each transfer so that you are assured that transfer is in progress and not hung. Specify
--progress switch so that transfer details shown for each file transfer in output.
# rsync -razv --progress httpd-2.4.25 /tmp/ sending incremental file list httpd-2.4.25/ httpd-2.4.25/.deps 0 100% 0.00kB/s 0:00:00 (xfer#1, to-check=1312/1314) httpd-2.4.25/.gdbinit 10566 100% 0.00kB/s 0:00:00 (xfer#2, to-check=1311/1314) httpd-2.4.25/ABOUT_APACHE 13496 100% 2.57MB/s 0:00:00 (xfer#3, to-check=1310/1314) httpd-2.4.25/Apache-apr2.dsw 65980 100% 10.49MB/s 0:00:00 (xfer#4, to-check=1309/1314) ----- output clipped -----
In the above output you can see after each file copied, transfer speed details, the duration is flashed on the screen.
Apart from above all switches, there is the whole list of switch which can be used with Rsync command. All of them can be found under the Rsync man page on your terminal. I am listing a few important one below for your quick reference.
-h: Output numbers will be shown in a human-readable format
--include: Include files. Wild cards can be used
--exclude: Exclude from the copy. Wild cards can be used
-q: Quiet mode. Suppress non-error messages
--backup: Make a backup
-l: Copy links as well
-H: Preserve hard links
-t: Preserve modification times
--dry-run: Perform dry run. No changes actually made.
--delete-after: Delete files after transfer
Let us know if you want additions of examples of any specific switch in the below comments. I will add them accordingly.