Rsync to EC2 linux server on AWS

Learn how to rsync to EC2 with the help of SSH protocol authenticated using a private key file. The process can be used for Rsync from and between EC2.

Rsync to EC2 Linux instance

We learned about Rsync in our last post. We learned how Rsync helps in a data backup or mirroring by using less bandwidth, time on the second run. Since it syncs only changes in later executions after the first fresh copy operation. Now many traditional data centers are moving to cloud services like AWS. Rsync can be useful to sync data from your local server to AWS hosted EC2 instance (if the data size is not huge).

In this article we will learn about how to rsync to EC2 server in AWS. Since you know EC2 Linux instances don’t use a conventional used id-password combination for authentication, Key pairs need to be used in Rsync for authentication EC2. For the Rsync setup, your EC2 instance must be launched with public-private key pair and you should have a private key file with you.

Get started

  • To start with making sure your EC2 instance is launched with a key pair.
  • Upload private key file on the source server (from where you are going to Rsync to EC2)
  • Make sure key file set with 400 permission
  • Get public IP or public DNS name of EC2 server from AWS EC2 console web page
  • Confirm you are able to connect from source to EC2. (verify AWS security groups and firewall settings)

Execute Rsync to EC2

We have testfile.tar for testing copy and private key file (mykey.pem) ready on the source server.

[root@kerneltalks ~]# ll /root/mykey.pem
-r--------. 1 root root 1675 Jul 24 01:01 /root/mykey.pem
[root@kerneltalks ~]# ll testfile.tar
-rw-r--r--. 1 root root 39198720 Dec 19  2016 testfile.tar

Now, use below Rsync command :

[root@kerneltalks ~]# rsync -avz -e "ssh -i /root/mykey.pem" testfile.tar ec2-user@ec2-13-126-114-120.ap-south-1.compute.amazonaws.com:/tmp/
sending incremental file list
testfile.tar

sent 8520069 bytes  received 31 bytes  3408040.00 bytes/sec
total size is 39198720  speedup is 4.60

Where –

  1. -a: Archive mode preserves permission and ownership
  2. -v: verbose mode
  3. -z: compress
  4. -e: Choose remote shell of execution
  5. ssh -i keyfile: Use the private key for authentication on destination using ssh protocol
  6. source (testfile.tar)
  7. Destination: Public DNS name of EC2 instance

That’s it! Your file is copied over to EC2. This can be done vice versa as well. You can sync files from the EC2 server to the local server as well. Just switch source-destination paths and you are all set to go.

Rsync between two EC2 servers

Rsync can be executed between two EC2 servers i.e. from one EC2 server to another. The same above command can be used. If you are doing it for EC2 instances within the same region then the Internal DNS name can be used in a command.

Conclusion :

Rsync is possible from, to, and between EC2 servers. Key file authenticated SSH protocol should be used in the Rsync command to achieve this.

4 thoughts on “Rsync to EC2 linux server on AWS

  1. Vishal Alhat

    Is it possible to do bidirectional syncing like server1 to server2 and vice-a-versa? I tried with Rsync or lsyncd but it works unidirectional only. I have a data directory in which data gets uploaded through code or end user so files uploaded on server1 get copied over to server2, but files uploaded on server2 are not synchronized with server1. I and using both instances through load balancers, so user hits are randomly distributed to both servers. Is there any way to overcome this problem?

    Reply
    1. Shrikant Lavhate Post author

      You should be using single storage or S3 bucket as store container for uploaded files. Make it available on both servers. In a way you dont have to sync files from server to server since no matter which server user is connected to, his/her file will always be uploaded to backend one storage!
      And yup, rsync is one way sync only. .

      Reply
  2. Kinsen Siu

    I used rsync as root and it seemed to work fine connecting from target server (aws) to source server (ibm cloud) over ssh but then i couldn’t access my aws instance. Any thoughts on what I might have done wrong?

    Reply
    1. Shrikant Lavhate Post author

      As I understand from your comment is :
      From target server on AWS you can rsync to source server on IBM cloud.
      But you are not able to login to target server on AWS using SSH ?
      or am I missing something here.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.