This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.
Ansible: Repository Hierarchy¶
Tip
This document describes details about Ansible being used to manage installations of Tocco. For documentation about managing servers via Ansible, have a look at the docs/ directory in the Ansible repository.
Tip
Setup instructions can be found in Setup Ansible.
Repository¶
The Ansible configuration is stored in a Git repository in the /tocco directory.
The root directory, /, is used for server management.
Overview of the repository structure:
tocco
│
├── config.yml # Definition of existing installations
│ # and parameterization.
│
├── filter_plugins # Filter plugins for use with Jinja2.
│ ├── crypto.py #
│ ├── format.py # For instance, in {{ "a_string"|to_camelcase }},
│ └── parse.py # `to_camelcase` is the filter
│
├── global.yml # Global variables
│
├── inventory.py # Script used to parse config.yml/global.yml and
│ # convert it to a proper Ansible Inventory
│
├── library # Custom Ansible module
│ ├── backoffice_installation.py #
│ ├── cloudscale_s3.py # Example showing module defined in teamcity_parameters
│ ├── teamcity_parameters.py # being used:
│ ├── teamcity_project.py #
│ └── vshn_openshift.py # - name: TeamCity - set parameter
│ # teamcity_parameters:
│ # user: ansible
│ # password: '{{ secret }}'
│ # id: ProjectId
│ # params:
│ # branch: master
│
├── playbook.yml # Starting point defining which roles
│ # to execute for which installation.
│
├── roles
│ └── tocco
│ ├── files # Files used in tasks
│ │ └── history_db.sql #
│ │
│ ├── tasks # Instructions for how to setup and configure
│ │ ├── database.yml # installations.
│ │ ├── mail_domains.yml #
│ │ ├── main.yml # `main.yml` the starting point and everything
│ │ ├── route.yml # else is included from there as needed.
│ │ └── teamcity.yml #
│ │
│ └── templates # Templates using the Jinja2 templating
│ ├── deploymentconfig_nice.yml # language. This templates are used
│ └── rolebinding_ansible_edit.yml # within tasks.
│
├── secrets2.yml # Ansible Vault containing passwords
│ # and other secrets in encrypted form.
│
└── test_plugins # Custom test for use in Jinja2
└── basics.py #
# For instance, in {{ 5 is even }},
# `even` is the test.
Configuration (config.yml/global.yml)¶
Structure¶
global.yml:
# Global variables
db_server: db1.tocco.cust.vshn.net
s3_endpoint: https://objects.cloudscale.ch
config.yml:
abc: # Customer "abc"
s3_bucket: nice-abc # Customer variables for "abc"
mail_relay: mxout1.tocco.ch #
installations:
abc: # Installation "abc"
db_name: nice_abc # Installation variables for "abc"
solr_core: nice-abc
abctest: # Installation "abctest"
db_name: nice_test # Installation variables for "abctest"
solr_core: nice-test #
Variable Precedence¶
Variables from highest to lowest priority. Higher priority precedes lower priority:
Installation variables
Customer variables
Global variables
Example:
global.yml:
db_server: db1.tocco.ch
config.yml:
abc:
db_server: db2.tocco.ch
abc: # <= db_server is "db3.tocco.ch"
db_server: db3.tocco.ch
abctest: # <= db_server is "db2.tocco.ch"
xyz:
xyz: # <= db_server is "db1.tocco.ch"
xyztest: # <= db_server is "db4.tocco.ch"
db_server: db4.tocco.ch
Merge Variables¶
By default, variables are replaced rather than merged:
Example, merging dictionaries¶
global.yml:
env:
nice2.request.limit: '1000'
config.yml:
abc:
env:
nice2.history.enabled: 'true'
abc:
env:
nice2.pool_name: 'test'
abctest:
In the above example, the result will be:
Installation |
Resulting Value |
|---|---|
abc |
env:
nice2.pool_name: 'test'
|
abctest |
env:
nice2.history.enabled: 'true'
|
This behavior can be changed using the !merge type:
global.yml:
env:
nice2.request.limit: '1000'
config.yml:
abc:
env: !merge
nice2.history.enabled: 'true'
abc:
env: !merge
nice2.pool_name: 'test'
nice2.request.limit: null
abctest:
env: !merge
nice2_request.limit: '2000'
In the above example, the result will be:
Installation |
Resulting Value |
|---|---|
abc |
env:
nice2.history.enabled: 'true'
nice2.pool_name: 'test'
# setting the value to null removes the item
# nice2.request.limit: null
|
abctest |
env:
nice2.history.enabled: 'true'
nice2_request.limit: '2000'
|
Tip
In addition to null the string "__null__" can be used
to remove a value. Using this string may be required in
Jinja2 expressions as there null can’t be used:
env:
nice2.history.enabled: "{{ '__null__' if condition else false }}"
Example, merging lists¶
global.yml:
mail_allowed_recipients:
- tocco.ch
- frank@example.com
config.yml:
abc:
mail_allowed_recipients: !merge
- fritz@example.com
- frank@example.com
abc:
mail_allowed_recipients: !merge
- example.net
abctest:
In the above example, the result will be:
Installation |
Resulting Value |
|---|---|
abc |
mail_allowed_recipients:
- tocco.ch
- frank@example.com
- fritz@example.com
# - frank@example.com # duplicates are ignored
- example.net
|
abctest |
mail_allowed_recipients:
- tocco.ch
- frank@example.com
- fritz@example.com
|
Limitations:
This is only implemented for dictionaries and lists defined directly on the customer or installation.
Implementation:
The !merge type is implemented within the inventory script (tocco/inventory.py). It
handles merging the dictionaries and lists, and hands the variables over to Ansible afterwards.
Templating with Jinja2¶
The templating language Jinja2 can be used in variables as well as on templates and in tasks.
Documentation:
Example:
global.yml:
db_name: nice_{{ installation_name }}
history_db_name: '{{ db_name }}_history'
db_server: |-
{% if location == 'blue' -%}
db1.blue.tocco.ch
{%- else -}
db1.red.tocco.ch
{%- endif %}
config.yml:
abc: # <= db_name is "nice_abc"
location: red # db_server is "db1.red.tocco.ch"
# history_db_name is "nice_abc_history"
abctest: # <= db_name is "NICE2_ABCTEST"
db_name: NICE2_{{ installation_name|upper }} # db_server is "db1.blue.tocco.ch"
location: blue # history_db_name is "NICE2_ABCTEST_history"
Evaluation:
Junja2 templates are evaluated for every installation independently. Thus, {{ installation_name }} always correspond to the name of the installation being processed.
Also, all expressions and statements are only evaluated when used. Thus, when setting these variables …:
is_production: "{{ not is_test }}"
is_test: "{{ installation_name.endswith('test') }}"
db_user: "{% if location == 'nine' %}{{ installation_name }}_user{% else %}{{ installation_name }}{% endif %}"
… they are not evaluated until used. Here for instance by passing them to the debug module:
- name: print debug info
debug:
msg: '{{ db_user }}'
when: is_production
{{ not is_test }}, {{ installation_name.endswith('test') }} and
{% if location == 'nine' %}…{% endif %}, defined in the variables above, are only evaluated now,
and will be evaluated again when used again. Consequently, the variables installation_name,
location and is_test used in the expressions/statements can be referenced before they exist. This
delayed evaluation is used extensively throughout the Ansible playbooks. It allows the use of global,
customer, installation and run time variables without having to worry whether they have been set
at that point.
Special variables:
A bunch of special variables are set transparently based on the definitions in config.yml and can
be used anywhere in a playbook. These variables are set by the inventory script (inventory.py).
customer_name |
The customer to which the installation belongs. |
installation_name |
Name of the installation. |
sibling_installations |
Names of all other installations belonging to the same customer. |
Ansible itself has built-in special variables that can be used too.
Ansible does not understand the concept of customers or installations. For Ansible to be able make sense of it, installations are translated to hosts and customers to groups. This means, for instance, hostvars, contains the variables belonging to all installations and groups contains the names of all customers.
Hint
In Yaml, quotes have to be used for any value starting with {{:
db_server: {{ var }} # Invalid, the first { is consider a start of
# dictionary by Yaml.
db_server: '{{ var }}' # ok