This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.

Migration

Tip

See Introduction to Deployments and Migrations for a simpler introduction to migrations.

Legend

This document describes the DevOps part of updating from one major or minor version to another.

Pre-Migration / Non-Migration State

When no migration is in progress, there is exatly one production and one test system.

Migration of Test

First, a testnew system is created and upgraded. During these phase hotfixes can still be deployed independently by deploying them to test first and then production.

Instructions

1. Create an installation named ${customer}testnew via Ansible. Copy the config from ${customer}test to ensure custom configurations are preserved. You should end up with something like this in config.yml:

    example:  # <-- customer
    # ...
    installations:
      example:  # <-- production
        db_server: db1.example.net             # <-- ${prod_db_server}
        # ...
      exampletest:  # <-- pre-existing test
        branch: releases/1.0
        env:                                    #
          custom_option: example                #        COPY THIS
        db_server: db1.example.net              #
      exampletestnew:  # <-- new test system
        branch: releases/1.0                    # <-- set this to correct (target) version
        env:
          custom_option: abc
        db_server: db1.example.net              # <-- ${testnew_db_server}
   

   For details see Create new Customer or Installation.

2. If you don’t want to replace ${customer}, ${prod_db_server} and ${test_db_server} in the following steps manually, set them as variables in your terminal:

   customer=${customer}                #
   prod_db_server=${prod_db_server}    # Replace ${…}
   test_db_server=${test_db_server}    #
   

3. Check if DNS record exists:

   dig ${customer}testnew.tocco.ch
   

   If it doesn’t (no output from dig), create records for both, ${customer}newtest and ${customer}testold [1]. See ${INSTALLATION_NAME}.tocco.ch.

4. Create installation ${customer}testnew:

   cd ${GIT_ROOT}/tocco
   ansible-playbook playbook.yml -l ${customer}testnew --skip-tags monitoring
   

   Note

   If you see errors that don’t make sense to you, make sure that your local ansible setup is according to the latest tocco docs ansible setup guide.

5. Copy prod database:

   tco cp ${prod_installation} ${test_installation}/nice_${customer}testnew
   

6. Deploy installation ${customer}testnew

   Warning

   If the customer has a customer specific action or widget, it is recommended to create the “testnew” installation in the back office already one day earlier. Then the customer bundle is automatically release for the new version during the night. Else you must Manual trigger releasing with the variable CI_RUN_AUTO_RELEASE set to 1 and CI_RELEASING_PACKAGES set to CUSTOMER-bundle (e.g. for the test customer it would be test-bundle)

   See Deliver (Simple)

7. Enable monitoring:

   cd ${GIT_ROOT}/tocco
   ansible-playbook playbook.yml -l ${customer}testnew -t monitoring
   

Migration of Production

During this phase, the old test system is renamed testold and testnew renamed test. Once this is done, production can be upgraded to the new version.

Instructions

1. Update config.yml:

   1. Rename ${customer}test to ${customer}testold

   2. Rename ${customer}testnew to ${customer}test

   3. Update or add branch for production

   4. Add branch for testold if missing

   Example:

   example:
     # ...
     installations:
       example:
         branch: releases/1.0    # <-- set to target version
          # ...
       exampletestnew:    # <-- rename to 'exampletest'
         # ...
       exampletest:       # <-- rename to 'exampletestold'
         # ...
         branch: release/1.0     # <-- add if missing
         # ...
   

2. If you don’t want to replace ${customer} and ${db_server} in the following steps manually, set them as variables in your terminal:

   customer=${customer}      # Replace ${…}
   db_server=${db_server}    #
   

3. Create ${customer}testold and update configs of ${customer}test and ${customer}:

   cd ${GIT_ROOT}/tocco
   ansible-playbook playbook.yml -l ${customer}test,${customer}testold,${customer} --skip-tags monitoring
   

4. Set downtime for ${customer}test and ${customer}testnew.

   See Schedule Downtime

5. Stop test systems:

   oc -n nice-${customer}test scale --replicas 0 dc/nice
   oc -n nice-${customer}testnew scale --replicas 0 dc/nice
   

6. Rename databases:

   tco db-rename ${customer}test ${customer}testold
   tco db-rename ${customer}testnew ${customer}test
   

7. Start ${customer}test again:

   oc -n nice-${customer}test scale --replicas 1 dc/nice
   

8. Remove ${customer}testnew

   Skip removal of DNS record [1].

   See Remove Installation/Customer

9. Deploy installation ${customer}test

   See Deliver (Simple)

10. Deploy installation ${customer} (production)

    See Deliver (Simple)

11. Deploy installation ${customer}testold

    See Deliver (Simple)

12. Enable monitoring for testold:

    cd ${GIT_ROOT}/tocco
    ansible-playbook playbook.yml -l ${customer}testold -t monitoring
    

Complete Migration

Weeks later, when sure it isn’t needed anymore, testold can be removed.

Instructions

1. Remove ${customer}testold

   Skip removal of DNS record [1].

   See Remove Installation/Customer

Footnotes

[1] (1,2,3)

DNS records for ${customer}testnew and ${customer}testold are needed on every migration. To avoid additional work, they are created but once and kept.