Ansible for DevOps: Server and configuration management for humans (2015)
Chapter 6 - Playbook Organization - Roles and Includes
So far, we’ve used fairly straightforward examples in this book. Most examples are ad-hoc for a particular server, and listing all tasks in one long listing makes for a fairly long playbook.
Ansible is very flexible when it comes to organizing your tasks in more efficient ways so you can make your playbooks more maintainable, reusable, and powerful. We’ll look at two ways to split up tasks more efficiently: using includes and roles. Finally, we’ll explore Ansible Galaxy, a repository of some community-maintained roles that help configure common packages and applications.
Includes
We’ve already seen one of the most basic ways of including other files in Chapter 4, when vars_files was used to place variables into a separate vars.yml file instead of inline with the playbook:
- hosts: all
vars_files:
- vars.yml
Tasks can easily be included in a similar way. In the tasks: section of your playbook, you can add include directives like so:
tasks:
- include: included-playbook.yml
Just like with variable include files, tasks are formatted in a flat list in the included file. As an example, the included-playbook.yml could look like:
---
- name: Add profile info for user.
copy:
src: example_profile
dest: "/home/{{ username }}/.profile"
owner: "{{ username }}"
group: "{{ username }}"
mode: 0744
- name: Add private keys for user.
copy:
src: "{{ item.src }}"
dest: "/home/.ssh/{{ item.dest }}"
owner: "{{ username }}"
group: "{{ username }}"
mode: 0600
with_items: ssh_private_keys
- name: Restart example service.
service: name=example state=restarted
In this case, you’d probably want to name the file user-config.yml, since it’s used to configure a user account and restart some service. Now, in this and any other playbook that provisions or configures a server, if you want to configure a particular user’s account, add the following in your playbook’s tasks section:
- include: example-app-config.yml
We used {{ username }} and {{ ssh_private_keys }} variables in this include file instead of hard-coded values so we could make this include file reusable. You could define the variables in your playbook’s inline variables or an included variables file, but Ansible also lets you pass variables directly into includes using normal YAML syntax. For example:
- { include: user-config.yml, username: johndoe, ssh_private_keys: [] }
- { include: user-config.yml, username: janedoe, ssh_private_keys: [] }
To make the syntax more readable, you can use structured variables, like so:
- include: user-config.yml
vars:
username: johndoe
ssh_private_keys:
- { src: /path/to/johndoe/key1, dest: id_rsa }
- { src: /path/to/johndoe/key2, dest: id_rsa_2 }
- include: user-config.yml
vars:
username: janedoe
ssh_private_keys:
- { src: /path/to/janedoe/key1, dest: id_rsa }
- { src: /path/to/janedoe/key2, dest: id_rsa_2 }
Include files can even include other files, so you could have something like the following:
tasks:
- include: user-config.yml
inside user-config.yml
- include: ssh-setup.yml
Handler includes
Handlers can be included just like tasks, within a playbook’s handlers section. For example:
handlers:
- include: included-handlers.yml
This can be helpful in limiting the noise in your main playbook, since handlers are usually used for things like restarting services or loading a configuration, and can distract from the playbook’s primary purpose.
Playbook includes
Playbooks can even be included in other playbooks, using the same include syntax in the top level of your playbook. For example, if you have two playbooks—one to set up your webservers (web.yml), and one to set up your database servers (db.yml), you could use the following playbook to run both at the same time:
- hosts: all
remote_user: root
tasks:
...
- include: web.yml
- include: db.yml
This way, you can create playbooks to configure all the servers in your entire infrastructure, then create a master playbook that includes each of the individual playbooks. When you want to initialize your entire infrastructure, make changes across your entire fleet of servers, or just check to make sure their configuration matches your playbook definitions, you can run one ansible-playbook command!
Complete includes example
What if I told you we could remake the 137-line Drupal LAMP server playbook from Chapter 4 in just 21 lines? With includes, it’s easy; just break out each of the sets of tasks into their own include files, and you’ll end up with a main playbook like this:
1 ---
2 - hosts: all
3
4 vars_files:
5 - vars.yml
6
7 pre_tasks:
8 - name: Update apt cache if needed.
9 apt: update_cache=yes cache_valid_time=3600
10
11 handlers:
12 - include: handlers/handlers.yml
13
14 tasks:
15 - include: tasks/common.yml
16 - include: tasks/apache.yml
17 - include: tasks/php.yml
18 - include: tasks/mysql.yml
19 - include: tasks/composer.yml
20 - include: tasks/drush.yml
21 - include: tasks/drupal.yml
All you need to do is create two new folders in the same folder where you saved the Drupal playbook.yml file, handlers and tasks, then create files inside for each section of the playbook.
For example, inside handlers/handlers.yml, you’d have:
1 ---
2 - name: restart apache
3 service: name=apache2 state=restarted
And inside tasks/drush.yml:
1 ---
2 - name: Check out drush master branch.
3 git:
4 repo: https://github.com/drush-ops/drush.git
5 dest: /opt/drush
6
7 - name: Install Drush dependencies with Composer."
8 shell: >
9 /usr/local/bin/composer install
10 chdir=/opt/drush
11 creates=/opt/drush/vendor/autoload.php
12
13 - name: Create drush bin symlink.
14 file:
15 src: /opt/drush/drush
16 dest: /usr/local/bin/drush
17 state: link
Separating all the tasks into separate includes files means you’ll have more files to manage for your playbook, but it helps keep the main playbook more compact (meaning it’s easier to see all the installation and configuration steps the playbook contains), and also separates tasks into individual, easily-maintainable groupings. Instead of having to browse one playbook with twenty-three separate tasks, you now maintain eight included files with two to five tasks, each.
It’s much easier to maintain a more granular set of tasks than one very long playbook. However, there’s no reason to try to start writing a playbook with lots of individual includes. Most of the time, it’s best to start with a monolithic playbook while you’re working on the setup and configuration details, then move sets of tasks out to included files after you start seeing logical groupings.
You can also use tags (demonstrated in the previous chapter) to limit the playbook run to a certain include file. Using the above example, if you wanted to add a ‘drush’ tag to the included drush file (so you could run ansible-playbook playbook.yml --tags=drush and only run the drush tasks), you can change line 20 to the following:
20 - include: tasks/drush.yml tags=drush
You can find the entire example Drupal LAMP server playbook using include files in this book’s code repository at https://github.com/geerlingguy/ansible-for-devops, in the includes directory. |
You can’t use variables for task include file names (like you could with include_vars directives, e.g. include_vars: "{{ ansible_os_family }}.yml" as a task, or with vars_files). There’s usually a better way than conditional task includes to accomplish conditional task inclusion using a different playbook structure, or roles, which we will discuss next. |
Roles
Including playbooks inside other playbooks makes your playbook organization a little more sane, but once you start wrapping up your entire infrastructure’s configuration in playbooks, you might end up with something resembling Russian nesting dolls.
Wouldn’t it be nice if there were a way to take bits of related configuration, and package them together nicely? Additionally, what if we could take these packages (often configuring the same thing on many different servers) and make them flexible so we can use the same package throughout our infrastructure, with slightly different settings on individual servers or groups of servers?
Ansible Roles can do all that, and more!
Let’s dive into what makes an Ansible role by taking one of the playbook examples from Chapter 4 and splitting it into a more flexible structure using roles.
Role scaffolding
Instead of requiring you to explicitly include certain files and playbooks in a role, Ansible automatically includes any main.yml files inside specific directories that make up the role.
There are only two directories required to make a working Ansible role:
role_name/
meta/
tasks/
If you create a directory structure like the one shown above, with a main.yml file in each directory, Ansible will run all the tasks defined in tasks/main.yml if you call the role from your playbook using the following syntax:
1 ---
2 - hosts: all
3 roles:
4 - role_name
Your roles (each one as its own directory, where the directory name is used by Ansible as the name of the role) can live in a couple different places—in the default global Ansible role path (configurable in /etc/ansible/ansible.cfg), or in a ‘roles’ folder directly within the same directory as your main playbook file.
Another simple way to build the scaffolding for a role (complete with all the available options/directories, a README file, and a structure suitable for contributing the role to Ansible Galaxy (we’ll get to Galaxy in a little bit!) so it can easily be shared), is to use the ansible-galaxy init command. Running this command creates an example role in the current working directory, which you can modify to suit your needs. |
Building your first role
Let’s clean up the Node.js server example from Chapter four, and break out one of the main parts of the configuration—installing Node.js and any required npm modules.
Create a roles folder in the same directory as the main playbook.yml file like we created in Chapter 4’s first example, and inside that folder, create a new folder nodejs (which will be our role’s name). Create two folders inside the nodejs role directory, meta and tasks.
Inside the meta folder, add a simple main.yml file with the following contents:
1 ---
2 dependencies: []
The meta information for your role is defined in this file. In basic examples and simple roles, you just need to list any role dependencies (other roles that are required to be run before the current role can do its work), but you can add much more to this file to describe your role to Ansible and to Ansible Galaxy. We’ll dive deeper into the meta information later. For now, save the file and head over to the tasks foler.
Create a main.yml file in this folder, and add the following contents (basically copying and pasting the configuration from the Chapter 4 example):
1 ---
2 - name: Install Node.js (npm plus all its dependencies).
3 yum: name=npm state=present enablerepo=epel
4
5 - name: Install forever module (to run our Node.js app).
6 npm: name=forever global=yes state=latest
The Node.js directory structure should now look like the following:
1 nodejs-app/
2 app/
3 app.js
4 package.json
5 playbook.yml
6 roles/
7 nodejs/
8 meta/
9 main.yml
10 tasks/
11 main.yml
You now have a complete Ansible role that you can use in your node.js server configuration playbook. Delete the Node.js app installation lines from playbook.yml, and reformat the playbook so the other tasks run first (in a pre_tasks: section instead of tasks:), then the role is included, then the rest of the tasks (in the main tasks: section). Something like:
pre_tasks:
# EPEL/GPG setup, firewall configuration...
roles:
- nodejs
tasks:
# Node.js app deployment tasks...
You can view the full example of this playbook in the ansible-for-devops code repository. |
Once you finish reformatting the main playbook, everything would run exactly the same during an ansible-playbook, with the exception of the tasks inside the nodejs role being prefixed with nodejs | [Task name here].
This little bit of extra data shown during playbook runs is useful because it automatically prefixes tasks with the role that provides them, without you having to add in descriptions as part of the name values of the tasks.
Our role isn’t all that helpful at this point, though, because it still does only one thing, and it’s not really flexible enough to be used on other servers that might need different Node.js modules to be installed.
More flexibility with role vars and defaults
To make our role more flexible, we can make it use a list of npm modules instead of a hardcoded value, then allow playbooks using the role to provide their own module list variable to override our role’s default list.
When running a role’s tasks, Ansible picks up variables defined in a role’s vars/main.yml file and defaults/main.yml (I’ll get to the differences between the two later), but will allow your playbooks to override the defaults or other role-provided variables if you want.
Modify the tasks/main.yml file to use a list variable and iterate through the list to install as many packages as your playbook wants:
1 ---
2 - name: Install Node.js (npm plus all its dependencies).
3 yum: name=npm state=present enablerepo=epel
4
5 - name: Install npm modules required by our app.
6 npm: name={{ item }} global=yes state=latest
7 with_items: node_npm_modules
Let’s provide a sane default for the new node_npm_modules variable in defaults/main.yml:
1 ---
2 node_npm_modules:
3 - forever
Now, if you run the playbook as-is, it will still do the exact same thing—install the forever module. But since the role is more flexible, we could create a new playbook like our first, but add a variable (either in a vars section or in an included file via vars_files) to override the default, like so:
1 node_npm_modules:
2 - forever
3 - async
4 - request
When you run the playbook with this custom variable (we didn’t change anything with our nodejs role), all three of the above npm modules will be installed.
Hopefully you’re beginning to see how this can be powerful!
Imagine if you had a playbook structure like:
1 ---
2 - hosts: appservers
3 roles:
4 - yum-repo-setup
5 - firewall
6 - nodejs
7 - app-deploy
Each one of the roles would live in its own isolated world, and could be shared with other servers and groups of servers in your infrastructure.
· A yum-repo-setup role could enable certain repositories and import their GPG keys.
· A firewall role could have per-server or per-inventory-group options for ports and services to allow or deny.
· An app-deploy role could deploy your app to a directory (configurable per-server) and set certain app options per-server or per-group.
All these things become very easy to manage when you have small bits of functionality separated into different roles. Instead of managing 100+ lines of playbook tasks, and manually prefixing every name: with something like “Common |” or “App Deploy |”, you now manage a few roles with 10-20 lines of YAML each.
On top of that, when you’re building your main playbooks, they can be extremely simple (like the above example), enabling you to see everything being configured and deployed on a particular server without scrolling through dozens of included playbook files and hundreds of tasks.
Variable precedence: Note that Ansible handles variables placed in included files in defaults with less precedence than those placed in vars. If you have certain variables you need to allow hosts/playbooks to easily override, you should probably put them into defaults. If they are common variables that should almost always be the values defined in your role, put them into vars. For more on variable precedence, see the aptly-named “Variable Precedence” section in the previous chapter. |
Other role parts: handlers, files, and templates
Handlers
In one of the prior examples, we introduced handlers—tasks that could be called via the notify option after any playbook task resulted in a change—and an example handler for restarting Apache was given:
1 handlers:
2 - name: restart apache
3 service: name=apache2 state=restarted
In Ansible roles, handlers are first-class citizens, alongside tasks, variables, and other configuration. You can store handlers directly inside a main.yml file inside a role’s handlers directory. So if we had a role for Apache configuration, our handlers/main.yml file could look like the following:
1 ---
2 - name: restart apache
3 command: service apache2 restart
You can call handlers defined in a role’s handlers folder just like you would handlers included directly in your playbooks (e.g. notify: restart apache).
Files and Templates
For the following examples, let’s assume our role is structured with files and templates inside files and templates directories, respectively:
1 roles/
2 example/
3 files/
4 example.conf
5 meta/
6 main.yml
7 templates/
8 example.xml.j2
9 tasks/
10 main.yml
when copying a file directly to the server, add the filename or the full path from within a role’s files directory, like so:
- name: Copy configuration file to server directly.
copy: >
src=example.conf
dest=/etc/myapp/example.conf
mode=644
Similarly, when specifying a template, add the filename or the full path from within a role’s templates directory, like so:
- name: Copy configuration file to server using a template.
template: >
src=example.xml.j2
dest=/etc/myapp/example.xml
mode=644
The copy module copies files from within the module’s files folder, and the template module runs given template files through the Jinja2 templating engine, merging in any variables available during your playbook run before copying the file to the server.
Organizing more complex and cross-platform roles
For simple package installation and configuration roles, you can get by with placing all tasks, variables, and handlers directly in the respective main.yml file Ansible automatically loads. But you can also include other files from within a role’s main.yml files if needed.
As a rule of thumb, I like to keep my playbook and role task files under 100 lines of YAML if at all possible (that way it’s easier for me to keep the entire set of tasks in my head while working through any issues). If I start nearing that limit, I usually split the tasks into logical groupings, and include files from the main.yml file.
Let’s take a look at the way my geerlingguy.apache role is set up (it’s available on Ansible Galaxy and can be downloaded to your roles directory with the command ansible-galaxy install geerlingguy.apache; we’ll discuss Ansible Galaxy itself later).
Initially, the role’s main tasks/main.yml file looked something like the following (generally speaking):
1 - name: Ensure Apache is installed (via apt).
2
3 - name: Configure Apache with lineinfile.
4
5 - name: Enable Apache modules.
Soon after creating the role, though, I wanted to make the role work with both Debian and RedHat hosts. I could’ve added two sets of tasks in the main.yml file, resulting in twice the number of tasks and a bunch of extra whenstatements:
1 - name: Ensure Apache is installed (via apt).
2 when: ansible_os_family == 'Debian'
3
4 - name: Ensure Apache is installed (via yum).
5 when: ansible_os_family == 'RedHat'
6
7 - name: Configure Apache with lineinfile (Debian).
8 when: ansible_os_family == 'Debian'
9
10 - name: Configure Apache with lineinfile (Redhat).
11 when: ansible_os_family == 'RedHat'
12
13 - name: Enable Apache modules (Debian).
14 when: ansible_os_family == 'Debian'
15
16 - name: Other OS-agnostic tasks...
If I had gone this route, and continued with the rest of the playbook tasks in one file, I would’ve quickly surpassed my informal 100-line limit. So I chose to use includes in my main tasks file:
1 - name: Include OS-specific variables.
2 include_vars: "{{ ansible_os_family }}.yml"
3
4 - include: setup-RedHat.yml
5 when: ansible_os_family == 'RedHat'
6
7 - include: setup-Debian.yml
8 when: ansible_os_family == 'Debian'
9
10 - name: Other OS-agnostic tasks...
Two important things to notice about this style of distribution-specific inclusion:
1. When including vars files (with include_vars), you can actually use variables in the name of the file. This is very handy for a variety of use cases, and here we’re including a vars file in the format distribution_name.yml. For our purposes, since the role will be used on Debian and RedHat-based hosts, we can create Debian.yml and RedHat.yml files in our role’s defaults and vars folders, and put distribution-specific variables there.
2. When including playbook files (with include), you can’t use variables in the name of the file, but you can do the next best thing: include the files by name explicitly, and use a condition to tell Ansible whether to run the tasks inside (the when condition will be applied to every task inside the included playbook).
After setting things up this way, I put RedHat and CentOS-specific tasks (like yum tasks) into tasks/setup-RedHat.yml, and Debian and Ubuntu-specific tasks (like apt tasks) into tasks/setup-Debian.yml. There are other ways of making roles work cross-platform, but using distribution-specific variables files and included playbooks is one of the simplest.
Now this Apache role can be used across different distributions, and with clever usage of variables in tasks and in configuration templates, it can be used in a very wide variety of infrastructure that needs Apache installed.
Ansible Galaxy
Ansible roles are powerful and flexible; they allow you to encapsulate sets of configuration and deployable units of playbooks, variables, templates, and other files, so you can easily reuse them across different servers.
It’s annoying to have to start from scratch every time, though; wouldn’t it be better if people could share roles for commonly-installed applications and services? Enter Ansible Galaxy.
Ansible Galaxy, or just ‘Galaxy’, is a repository of community-contributed roles for common Ansible content. There are already hundreds of roles available which can configure and deploy common applications, and they’re all available through the ansible-galaxy command, introduced in Ansible 1.4.2.
Galaxy offers the ability to add, download, and rate roles, and you can register either using a social account or a normal account on the site (though you don’t need an account to install and use roles from Galaxy).
Getting roles from Galaxy
One of the primary functions of the ansible-galaxy command is retrieving roles from Galaxy. Roles must be downloaded before they can be used in playbooks.
Remember the basic LAMP (Linux, Apache, MySQL and PHP) server we installed earlier in the book? Let’s create it again, but this time, using a few roles from Galaxy:
$ ansible-galaxy install geerlingguy.apache geerlingguy.mysql geerlingguy.php
The latest version or a role will be downloaded if no version is specified. To specify a version, add the version after the role name, for example: $ ansible-galaxy install geerlingguy.apache,1.0.0. |
Ansible Galaxy is still evolving rapidly, and has already seen many small improvements. There are a few areas where Galaxy could use some improvement (like browsing for roles by Operating System in the online interface, or automatically downloading roles that are included in playbooks), but most of these little bugs or rough areas will be fixed in time. Please check Ansible Galaxy’s About page and stay tuned to Ansible’s blog for the latest updates. |
Using role requirements files to manage dependencies
If your infrastructure configuration requires have five, ten, fifteen or more Ansible roles, installing them all via ansible-galaxy install commands can get very tiring. Additionally, if you host roles internally (e.g. via an internal Git repository), you can’t install the roles through Ansible Galaxy. The ansible-galaxy command, however, can be passed a ‘requirements’ file that it will parse and use to automatically download all the included dependencies—whether on Ansible Galaxy or in some other role repository.
There are two simple syntaxes, the first which follows Python’s pip requirements convention. Generally, you can create a requirements.txt file (I usually add one in the root of my project directory), and include a line for each role, using the same syntax we used in the install command earlier:
1 geerlingguy.apache,1.3.1
2 geerlingguy.mysql
3 geerlingguy.php,1.4.1
Specify one Ansible Galaxy role per line either by itself, or with a particular version number, and then run ansible-galaxy install -r requirements.txt. Ansible will download all the roles from Galaxy into your local roles path.
If you want to be able to install roles from other sources, like GitHub, an HTTP download, BitBucket, etc., or if you’d like to specify the path into which the roles should be downloaded, you can use a requirements.yml file with the following syntax:
1 # From Ansible Galaxy, like the earlier requirements.txt example.
2 - src: geerlingguy.firewall
3
4 # From GitHub, into a particular path, with a custom name and version.
5 - src: https://github.com/geerlingguy/ansible-role-passenger
6 path: /etc/ansible/roles/
7 name: passenger
8 version: 1.0.2
9
10 # From a web server, with a custom name.
11 - src: https://www.example.com/ansible/roles/my-role-name.tar.gz
12 name: my-role
Installing roles defined in this file is similar to the .txt example: ansible-galaxy install -r requirements.yml. For more documentation on Ansible requirements files, read the official documentation: Advanced Control over Role Requirements Files.
A LAMP server in six lines of YAML
With the Apache, MySQL, and PHP roles installed, we can quickly create a LAMP server. This example assumes you already have a CentOS-based linux VM or server booted and can connect to it or run Ansible as a provisioner via Vagrant on it, and that you’ve run the ansible-galaxy install command above to download the required roles.
First, create an Ansible playbook named lamp.yml with the following contents:
1 ---
2 - hosts: all
3 roles:
4 - geerlingguy.mysql
5 - geerlingguy.apache
6 - geerlingguy.php
Now, run the playbook against a host:
$ ansible-playbook -i path/to/custom-inventory lamp.yml
After a few minutes, an entire LAMP server should be set up and running. If you add in a few variables, you can configure virtualhosts, PHP configuration options, MySQL server settings, etc.
We’ve effectively reduced about thirty lines of YAML (from previous examples dealing with LAMP or LAMP-like servers) down to three. Obviously, the roles have extra code in them, but the power here is in abstraction. Since most companies have many servers using similar software, but with slightly different configurations, having centralized, flexible roles saves a lot of repetition.
You could think of Galaxy roles (which typically install and configure common software like Apache or MySQL, or set up security rules or frameworks) as glorified packages; they not only install software, but they configure it exactly how you want it, every time, with minimal manual labor. Additionally, many of these roles work across different flavors of Linux and UNIX, so you have better configuration portability!
A Solr server in six lines of YAML
Let’s grab a few more roles and build an Apache Solr search server, which requires Java and Apache Tomcat to be installed and configured.
$ ansible-galaxy install geerlingguy.java geerlingguy.tomcat6 geerlingguy.solr
Then create a playbook named solr.yml with the following contents:
1 ---
2 - hosts: all
3 roles:
4 - geerlingguy.java
5 - geerlingguy.tomcat6
6 - geerlingguy.solr
Now we have a fully-functional Solr server, and we could add some variables to configure it exactly how we want, by using a non-default port, or changing the memory allocation for Tomcat6.
I think you might get the point. Now, I could’ve also left out the java and tomcat6 roles, since they’ll be automatically picked up during installation of the geerlingguy.solr role (they’re listed in the solr role’s dependencies).
A role’s page on the Ansible Galaxy website highlights available variables for setting things like what version of Solr to install, where to install it, etc. (as an example, view the geerlingguy.solr Galaxy page).
Using community-maintained roles, you can build a wide variety of servers with minimal effort. Instead of having to maintain lengthy playbooks and roles unique to each server, Galaxy lets you build a list of the required roles, and a few variables that set up the servers with the proper versions and paths. Configuration management with Ansible Galaxy becomes true configuration management—you get to spend more time managing your server’s configuration, and less time on packaging and building individual services!
Helpful Galaxy commands
Some other helpful ansible-galaxy commands you might use from time to time:
· ansible-galaxy list displays a list of installed roles, with version numbers
· ansible-galaxy remove [role] removes an installed role
· ansible-galaxy init can be used to create a role template suitable for submission to Ansible Galaxy
You can configure the default path where Ansible roles will be downloaded by editing your ansible.cfg configuration file (normally located in /etc/ansible/ansible.cfg), and setting a roles_path in the [defaults] section.
Contributing to Ansible Galaxy
If you’ve been working on some useful Ansible roles, and you’d like to share them with others, all you need to do is make sure they follow Ansible Galaxy’s basic template (especially within the meta/main.yml and README.md files). To get started, use ansible-galaxy init to generate a basic Galaxy template, and make your own role match the Galaxy template’s structure.
Then push your role up to a new project on GitHub (I usually name my Galaxy roles like ansible-role-[rolename], so I can easily see them when browsing my repos on GitHub), and add a new role while logged into galaxy.ansible.com.
Summary
Using includes and Ansible roles organizes Playbooks and makes them maintainable. This chapter introduced different ways of using include, the power and flexible structure of roles, and how you can utilize Ansible Galaxy, the community repository of configurable Ansible roles that do just about anything.
_________________________________________
/ When the only tool you own is a hammer, \
| every problem begins to resemble a |
\ nail. (Abraham Maslow) /
-----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||