Ansible Copy and other Ways to Copy Files to Remote Hosts

In Ansible, copying files is a common task most users perform every day. The Ansible copy module is a familiar way to copy files and copy directories with Ansible, but it’s definitely not the only way. Are you picking the right method?

In this tutorial, you’re going to learn how to copy files in Ansible using techniques like the copy module, template module, synchronize module, and more!

Prerequisites

If you’d like to follow along, please be sure that you have the following:

• At least 2 Linux servers. Any Linux distribution will work as long as it has Python 3 installed. One server will be acting as the Ansible control node and the other as the managed node. The Ansible host will be using a placeholder IP address of X.X.X.X.
• Python 3 and pip installed on the Ansible control node. If you need help installing Python 3 or pip, you can check out this article.
• The ability to SSH into the Ansible control node and the managed node. If you need help with getting logged into a Linux server, you can check out some articles online.
• Sudo access on each node to install the software needed. This tutorial will be using a user account called someuser.

Sudo is a Linux command that allows you to run commands as a privileged user. It’s the “Run as Administrator” for Linux. Check out this article to learn more about the sudo command.

Setting up the Ansible Control Node

If you already have Ansible installed, you can skip this section.

Before you can copy files with Ansible, you must set up an Ansible control node. The control node is where all the Ansible commands are run from. When a command is executed, the control node will connect to the remote machine via the SSH protocol and execute commands to configure the managed host.

Installing the Ansible Package

For the first task in setting up the Ansible control node, you must first install Ansible itself. To do that:

1. Open your favorite SSH client and connect to the Linux server that will be your Ansible control node.

2. Create a Python virtual environment named ansible.

python3 -m venv ansible

A virtual environment is an isolated environment for Python. Virtual environments let you install specific versions of Ansible without requiring the Linux distro packaging tools. You can install Ansible with the dnf or apt command, but the version of Ansible that gets installed may not match the version in the tutorial. Newer versions of Ansible may not work with this tutorial in the future.

3. Once you’ve created the virtual environment, run the command below to initialize the virtual environment. Initializing the virtual environment sets up the PATH variable in Linux to use your isolated Python environment. This allows you to install Python packages in your isolated environment instead of affecting the entire system.

source ansible/bin/activate

4. To ensure that the pip package is up to date, run the command below to update pip

pip install -U pip

5. Finally, run the pip command to install Ansible 2.10.6. Ansible is a Python package that installs with Python’s package manager, pip.

This tutorial uses Ansible 2.10.6. Using this version of Ansible will guarantee that examples in this tutorial will work as expected. Other versions may work but cannot be guaranteed.

pip install ansible==2.10.6

Creating the Ansible Inventory File and Testing Connections

Once you’ve installed Ansible, you now need to define a list of managed hosts to target with Ansible. These hosts will be the machines you’ll copy files to. In Ansible, managed hosts are defined in an inventory file.

Assuming you’re still connected to your Ansible managed node, open your inventory with your favorite editor and add the line shown below.

managed_node ansible_host=X.X.X.X ansible_user=someuser ansible_ssh_extra_args='-o StrictHostKeyChecking=no'

Once Ansible knows about your managed hosts, execute the ping module to test the connection between the Ansible controller and the managed node.

ansible managed_node -i inventory -m ping

The copy Module

The most common method to copy files with Ansible is via the copy module. This Ansible module serves one purpose and one purpose only; to copy files as-is to managed hosts.

Let’s dig in and walk through out to use the copy module to copy a file to a managed host.

On the Ansible control node:

1. Create a file called text.txt. Ansible will copy this source file to the managed host.

echo "Hello World" > text.txt

2. Now, using Ansible’s copy module, copy the file to the managed host using the command below.

You’ll see that the -a parameter has a src and dest key. The src value refers to the file on the Ansible control node, while dest refers to the path where Ansible will copy the file.

ansible managed_node -i inventory \ ## tells Ansible which inventory file to use
 -m copy \ ## tells Ansible which module to load,
 -a "src=test.txt dest=/tmp/test.txt" ## attributes that are passed to the module

After the command completes, you will see the following output. Notice that the text is yellow and the top line of the output says “CHANGED.” This output lets you know that Ansible has changed the managed host state (added a new file called test.txt to the /tmp directory).

3. Hit the up arrow and rerun the command. Now notice that the output text is green and the top line of the output says “SUCCESS.” This output indicates that there is already a /tmp/test.txt file on the managed node and the managed host is in the desired state.

Now validate that Ansible actually copied the file to the managed host with the shell module by running the cat command on the remote host.

ansible managed_node -i inventory -m shell -a "cat /tmp/text.txt"

If all is well, you should see the output below.

The template Module

Although the copy module is great for copying existing files, what about files that need values changed at runtime? In that case, you can use the template module.

The template module is similar to the copy module, but it lets you define variables inside a text file, transform it, and then copy it to your managed hosts.

Related:How to Create Ansible Templates to Save Configuration Time

1. First, create a Jinja2 template which is a text file containing various variables. To do so, run the commands below.

## Create a text file called template.txt.j2 with "Test Message" as contents
 echo "Test Message" > template.txt.j2
## Append the text {{ ata_message }} to the file
 echo "{{ ata_message }}" >> template.txt.j2

In the Jinja2 template file, notice a line that has the string ata_message wrapped in curly braces. This string is a Jinja2 variable, allows you to change the contents of the file at runtime.

2. Copy the template.txt.j2 template to the managed host by running the command below setting the variable ata_message to Hello World Template.

Although the command below looks similar to the previous step, you’ll see one additional argument (-e). The -e argument stands for extra variables and is a way to define variables at runtime.

# Run the Ansible template module against the managed_node in inventory file "inventory"
# The template module will copy and transform the template.txt.j2 file to the
# managed host in the /tmp directory
 ansible managed_node -i inventory -m template \ 
 -a "src=template.txt.j2 dest=/tmp/template.txt" \ 
 -e "ata_message='Hello World Template'"

3. Finally, execute the shell module again to read the file with the cat command on the managed host.

ansible managed_node -i inventory -m shell -a "cat /tmp/template.txt"

The synchronize Module

Similar to the copy module covered above, the synchronize module is a bit different. Instead of performing a simple file copy, the synchronize module uses the rsync utility.

The rsync utility is a great option for copying many files at once. It uses less bandwidth and uses the remote-update protocol to speed up file transfers.

To use the synchronize module, you must ensure you have rsync installed on both the control and managed nodes.

To use Ansible’s synchronize module to copy this tutorial’s test file, invoke the synchronize module as shown below.

# Run the Ansible synchronize module against the managed_node in inventory file "inventory"
# The synchronize module will copy the test.txt file to the
# managed host in the /tmp directory

ansible managed_node -i inventory \
-m synchronize \
-a "src=test.txt dest=/tmp/test.txt"

By default, the synchronize module uses rsync’s “push” mode to transfer files which copies files from the control node to the managed host. You can configure rsync to use “pull” mode as well. When the pull mode is used, the managed node pulls the files from the control node.

The output you can see below is much different from the output of the copy module. Since the synchronize module uses the rsync utility, the output will contain the rsync utility syntax used to copy the file.

The get_url Module

Perhaps the file(s) you’d like Ansible to copy to managed nodes isn’t on the Ansible control node. Instead, the file is hosted on a web server somewhere and is available via URL. In that case, use the get_url module. The get_url module pulls a file from a URL and copies it to managed hosts.

To copy files with the get_url module, invoke the get_url module as shown below. This command downloads the file stored at https://file-examples-com.github.io/uploads/2017/02/file_example_CSV_5000.csv and copies it to /tmp/test.csv.

# Run the Ansible get_url module against the managed_node in inventory file "inventory"
# The get_url module will copy the "file_example_CSV_5000.csv" file from https://file-examples-com.github.io
# to the managed host in the /tmp directory
 ansible managed_node -i inventory \
 -m get_url \
 -a "url='https://file-examples-com.github.io/uploads/2017/02/file_example_CSV_5000.csv' dest=/tmp/test.csv"

The git Module

If you have files store in a source code management (SCM) tool like Git and need Ansible to copy files from that repo to a managed host, use the Git module.

To use the git module, you must have Git installed on the managed node.

To demonstrate using the Git module to copy files to a managed node, invoke the Git module as shown below. This command invokes the git clone command on each managed node in the inventory file cloning the Ansible GitHub repo into the /tmp/ansible directory.

# Run the Ansible git module against the managed_node in inventory file "inventory"
# The git module will copy Ansible git repository from GitHub
# to the managed host in the /tmp directory

ansible managed_node -i inventory
-m git
-a "repo=https://github.com/ansible/ansible.git dest=/tmp/ansible"

Now view the contents of the Git repository on the managed host. To do so, invoke the shell module to execute the ls command to read all files in the /tmp/ansible directory. If successful, you should see the contents of the Ansible repository in the output.

ansible managed_node -i inventory -m shell -a "ls -ahsl /tmp/ansible"

Conclusion

Ansible provides several ways to copy files to a remote host. These modules provide a declarative and easy understanding of building your automation workflow so you can share easily share with your team or organization.