## Using Python Fabric to automate GNU/Linux server configuration tasks

Fabric is a Python library and command-line tool for automating tasks of application deployment or system administration via SSH. It provides tools for executing local and remote shell commands and for transferring files through SSH and SFTP, respectively. With these tools, it is possible to write application deployment or system administration scripts, which allows to perform these tasks by the execution of a single command.

In order to work with Fabric, you should have Python and the Fabric Library installed on your local computer. Within this article, we consider using a Debian-based distribution on the local computer (such as Ubuntu, Linux Mint and others).

On the local computer, you may not need to install Python as it is shipped by default on most of the GNU/Linux distributions. To install the Fabric library, you may use pip. On Debian-based distributions, pip can be installed with apt-get through the python-pip package:

After installing it, you may update it to the latest version using pip itself: After that, you may use pip to install Fabric:

To work with Fabric, you must have SSH installed and properly configured with the necessary user’s permissions on the remote servers you want to work on. In the examples, we will consider a Debian System with IP address 192.168.250.150 and a user named “administrator” with sudo powers, which is required only for performing actions that require superuser rights.

One way to use Fabric is to create a file called fabfile.py containing one or more functions that represent the tasks we want to execute, for example:

File: fabfile.py

In this example, we have defined two tasks called remote_info and local_info, which are used to retrieve local and remote systems information through the command uname -a. Also, we have defined the host user and address we would like to use to connect to the remote server using a special dictionary called env.

Having this defined, it is possible to execute one of the tasks using the shell command fab. For example, to execute the task local_info, from within the directory where fabfile.py is located, you may call:

which gives:

Similarly, you could execute the task called remote_info, calling:

In this case, Fabric will ask for the password of the user “administrator”, as it is connecting to the server via SSH:

There are lots of parameters that can be used with the fab command. To obtain a list with a brief description of them, you can run fab --help. For example, using fab -l, it is possible to check the Fabric tasks available on the fabfile.py file. In this example, it outputs

As in the previous example, on the file fabfile.py, the function run() may be used to run a shell command on a remote server and the function local() may be used to run a shell command on the local computer. Besides these, there are some other possible functions to use on fabfile.py:

• sudo('shell command'): to run a shell command on the remote server using sudo;
• put('local path', 'remote path'): to send a file from a local path on the local computer to the remote path on the remote server;
• get('remote path', 'local path'): to get a file from a remote path on the remote server to the local path on the local computer.

Also, it is possible to set many other details about the remote connection with the dictionary env. To see a full list of env vars that can be set, visit http://docs.fabfile.org/en/1.6/usage/env.html#full-list-of-env-vars. Among the possible settings, its worth spend some time commenting on some of them:

• user: defines which user will be used to connect to the remote server;
• hosts: a Python list with the addresses of the hosts that Fabric will connect to perform the tasks. There may be more than one host, e.g.,
• host_string: with this setting, it is possible to configure a user and a host at once, e.g.

As it could be noticed from the previous example, Fabric will ask for the user’s password to connect to the remote server. However, for automated tasks, it is interesting to be able to make Fabric run the tasks without prompting for any user input. To avoid the need of typing the user’s password, it is possible to use the env.password setting, which permits to specify the password to be used by Fabric, e.g.

If the server uses SSH keys instead of passwords to authenticate users (actually, this is a good practice concerning the server’s security), it is possible to use the setting env.key_filename to specify the SSH key to be used. Considering that the public key ~/.ssh/id_rsa.pub is installed on the remote server, you just need to add the following line to fabfile.py:

It is also a good security practice to forbid root from logging in remotely on the servers and allow the necessary users to execute superuser tasks using the sudo command. On a Debian system, to allow the “administrator” user to perform superuser tasks using sudo, first you have to install the package sudo, using

and then, add the “administrator” user to the group “sudo”, which can be done with the command

Having this done, you could use the sudo() function on Fabric scripts to run commands with sudo powers. For example, to create a mydir directory within /home, you may use:

File: fabfile.py

and call

which will ask for the password of the user “administrator” to perform the sudo tasks:

When using SSH keys to log in to the server, you can use the env.password setting to specify the sudo password, to avoid having to type it when you call the Fabric script. In the previous example, by adding,

would be enough to make the script run without the need of user intervention.

However, some SSH keys are created using a passphrase, required to log in to the server. Fabric treat these passphrases and passwords similarly, which can sometimes cause confusion. To illustrate Fabric’s behavior, consider the user named “administrator” is able to log in to a remote server only by using his/her key named ~/.ssh/id_rsa2.pub, created using a passphrase, and the following Fabric file:

File: fabfile.py

In this case, calling

makes Fabric ask for a “Login password”. However, as you shall notice, this “Login password” refers to the necessary passphrase to log in using the SSH key:

In this case, if you specify the env.password setting, it will be used as the SSH passphrase and, when running the create_dir script, Fabric will ask for the password of the user “administrator”. To avoid typing any of these passwords, you may define env.password as the SSH passphrase and, within the function that uses sudo(), redefine it as the user’s password:

File: fabfile.py

Alternatively, you could specify the authentication settings from within the task function:

File: fabfile.py

On this example, the command ‘:’ does not do anything. It only serves as a trick to enable setting env.password twice: first for the SSH passphrase, required for login and then to the user’s password, required for performing sudo tasks.

If necessary, it is possible to use Python’s with statement (http://www.python.org/dev/peps/pep-0343/), to specify the env settings. A compatible create_dir() task using the with statement, would be:

File: fabfile.py

The fab command is useful to performing system administration and application deployment tasks from a shell console. However, sometimes you may want to execute tasks from within your Python scripts. To do this, you may simply call the Fabric functions from your Python code. To build a script that runs a specific task automatically, such as create_dir() shown above, you may do something like:

File: mypythonscript.py

As we have seen, with Fabric, it is possible to automate the execution of tasks that can be done by executing shell commands locally, and remotely, using SSH. It is also possible to use Fabric’s features on other Python scripts, and perform dynamic tasks, enabling the developer to automate virtually anything that can be automated. The main goal of this article was to show Fabric’s basic features and try to show a solution to different scenarios of remote connections, regarding different types of authentication. From this point, you may customize your Fabric tasks to your needs using basically the functions local(), run() and sudo() to run shell commands and put() and get() to transfer files.

To conclude, we show a more practical example of a Python script that uses Fabric to deploy a very basic HTML application on a server. The script creates a tarball from the local HTML files at ~/website, sends it to the server, expands the tarball and moves the files to the proper directory (/var/www/website) and restarts the server. Hope this article helped you learning a bit about Fabric to automate some tasks!