How to transfer data between two EC2 Linux instances

Small tutorial to learn how to transfer data between two EC2 Linux instances on AWS. Explains the use of key authentication in the SCP command.

EC2 instance in AWS is a server instance that uses key-based authentication for login. Now, beginners or first time EC2 users wonder how to copy files from one EC2 server to another? or how to transfer data between two EC2 instances? You can achieve it using key files in scp command.

Few pre-requisites are :

• Source and destination EC2 instances should be in the same region (can be in different availability zones)
• Both instances should be able to communicate over port 22. Configure security groups accordingly.
• EC2 instances built with specifying key pairs at the time of launch.

Lets get into actual stuff now. For example consider below setup :

1. Two EC2 servers in the same region installed with Red Hat Linux
2. Security groups allow port 22 both direction communication
3. Key pairs used at the time of launch for both servers.
4. The private key file is ready with me (.pem file). It’s part of the key pair used during launch.

In this tutorial, we will be copying file testfile.tar from server kerneltalks1 to kerneltalks2. Look at the file on kerneltalks1.

[root@kerneltalks1 ~]# ls testfile.tar
-rw-r--r--. 1 root root 39198720 Dec 19  2016 testfile.tar

Now, you need to use secure copy scp command with key file for authentication like below :

[root@kerneltalks1 ~]# scp -i /root/mykey.pem testfile.tar ec2-user@172.31.24.59:/tmp

Here,

• -i switch used to specify the key file
• Then source file path
• followed by destination in userid@hostname/ip:/path/ format.

You have to use the Private IP of the destination EC2 instance. You can get this from your AWS EC2 console web page. Here kerneltalks2 has private IP 172.31.24.59. You can use hostname if you make an entry in /etc/hosts file.  Remember different Linux distros on AWS have different login id to be used.

# scp -i /root/mykey.pem testfile.tar ec2-user@172.31.29.79:/tmp
The authenticity of host '172.31.29.79 (172.31.29.79)' can't be established.
RSA key fingerprint is 66:c4:ce:37:c6:e6:a1:6c:2f:f9:9b:f2:f5:05:e3:38.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.31.29.79' (RSA) to the list of known hosts.
testfile.tar                                                                                                               100%   37MB  37.4MB/s   00:00

In the above output, you can see kerneltalks2 authenticated using key file and file transfer was completed.

[root@kerneltalks2 ~]# ls /tmp/testfile.tar
-rw-r--r--. 1 root root 39198720 Dec 19  2016 testfile.tar

Rsync is another good way to copy data between two EC2 instances. You can learn about it in our other article: Rsync for EC2 on AWS.

Conclusion :

Files and directories can be transferred between two ec2 instances using the same Linux scp command. Only the authentication part is to be done using a key file with -i switch.