This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.

CI

Setup

Gitlab is used as host provider and CI of the tocco-client project.

The triggered pipelines can be inspected here: Gitlab CI Pipelines.

Stages and Jobs

Alls jobs can be found in the .gitlab-ci.yml file in the root.

There are a set of jobs that are triggered for every merge request and push to master. The jobs ensure, that the linting command and all unit tests run successfully and storybook can be build. Also the builded storybook gets pushed to the Gitlab Storybook Repository and a link to this storybook deployment is written as comment to the merge request.

There is also a scheduled pipeline that runs daily on master testing all the above and also runs the more time consuming end-to-end tests with cypress.

Also there are two jobs that support the development workflow. One job handles the auto-releasing of packages on the release branches (if the changelog is not empty a package is released) on each commit. The last job handles cherry-picking across different versions and is described here Cherry-pick job

Caching

All node_modules folder are getting cached since a installation of all dependencies from scratch takes very long. The yarn.lock files act as keys. That means if a dependency changes, a new cache is created. Each job should define if it pull, pushed or pull and pushed to the cache to speed things up.

Image

As docker image the linux alpine version is used to speed up the image initialization phase by several seconds. As downside of this optimization, some dependencies must be installed by a job with apk.

Storybook

On release branches and master the storybook is daily built. For merge requests per default no storybook is built. However the storybook job can be manual triggered in the Gitlab UI by clicking on the play button of the job:

../../_images/storybook-mr-gitlab.png

Gitlab merge request pipeline

Releasing

Manual trigger releasing

Using schedule (releasing all packages)

If all packages should be released open Pipeline Schedules and run the job of the corresponding branch.

Using pipeline (releasing only some packages)

Open Run Pipeline on Gitlab. Select the branch where you want to manually release (either master or nice-releases/VERSION). The following variables can be passed:

  • CI_RUN_AUTO_RELEASE (mandatory): must be set to 1 that the releasing is started

  • CI_RELEASING_PACKAGES (optional): per default the releasing is started for all releasable packages. if a comma-separated list is passed (e.g. admin,devcon) only these packages are release candidates

  • CI_RELEASING_FORCE (optional): per default a package is only released if the changelog is not empty. if true is passed as value the package is always released

Fix failing job

If the releasing job fails on the Gitlab CI, some packages are already published to NPM but the new version is not committed to the release/master branch. First you must manually change these versions (check log to know which packages were already released). Commit these changes and merge them. Last you must move for each package the git tag. You must execute:

git checkout ${COMMIT_ID}
git tag ${PACKAGE_NAME}/${RELEASE_TAG} -f
git push origin ${PACKAGE_NAME}/${RELEASE_TAG} -f

  • COMMIT_ID: commit id on the release/master branch of your created commit on Gitlab

  • PACKAGE_NAME: npm package name which always has the tocco prefix (e.g. tocco-admin)

  • RELEASE_TAG: npm release tag name (e.g. nice34). On the master the tag is named like the upcoming version

Cherry-pick job

Fix failing cherry-pick job

For example there could be an auto merge conflict. In such a case there is log message like this:

CONFLICT (content): Merge conflict in packages/entity-browser/src/routes/list/containers/ListViewContainer.js

Workflow for fixing (in the example the source branch is nice-releases/227 and the target branch nice-releases/228) :

  • Check log which commits are cherry-picked (see line 2 und 3):

Order of commits to cherry-pick:
- 952f1e0fd57f000f9bed1a0969efceaaed43e10c
- 00f9bed1a0969efceaaed43e10c952f1e0fd57f0

  • Checkout the target branch

  • Cherry pick the commits with git cherry-pick ${COMMIT_ID} -x and fix conflicts

  • Create a MR

  • Optional: if the MR is approved release all necessary packages (all necessary packages will be released automatically every night)

  • Move the auto merge tag to the head of the source branch

git checkout ${HEAD_COMMIT_ID}
git tag -f auto-merge/${SOURCE_BRANCH_NICE_VERSION}
git push origin auto-merge/${SOURCE_BRANCH_NICE_VERSION} -f

Merge Request

There is one job that automates the creation of merge requests after a successful release of a package (or cherry picking), the branch containing the release (or cherry picked) commits must be rebased to its target branch using a merge request.

CI scripts on different versions

In version 2.27 many client CI scripts were introduced or adjusted such as auto-releasing or cherry-picking. If a CI feature is added or modified this should be done ideally on version 2.27 if possible. However between 2.27 and the latest version are some differences in the scripts which are documented here. The command git diff origin/nice-releases/34 .gitlab-ci.yml scripts ':!scripts/nice2' ':!scripts/widgets' (on branch nice-releases/35) is used to extract the difference (Folder nice2 and widgets are excluded as the are only used by Cypress which runs on the master branch).

Changed in version 3.2

.gitlab-ci.yml

  • yarn_setup job packages/*/ replaced with packages/*/*/

  • test job artifacts storage added

  • cypress job expire_in changed

  • sonarqube job before_script added

helpers.sh

  • in method getDevDependenciesGreps command adjusted to get dev dependencies

  • method setPackageDir added

  • method checkPackage adjusted

release-message.js

  • script added

release-package.sh

  • adjust navigation to folder

  • add slack message for package release => release-message.js

release-packages.sh

  • adjust iterating over folders

tag-all-packages.sh

  • script added

Changed in version 3.3

.gitlab-ci.yml

  • cypress job

    • upgrade cypress image

    • CYPRESS_INSTALL_BINARY version changed

helpers.sh

  • setPackageDir supports now bundle packages

tag-all-packages.sh

  • add registry to npm dist-tag add ... command

Changed in version 3.4

.gitlab-ci.yml

  • variables NICE_VERSION, DOCKER_NICE_VERSION, NICE_TIMEOUT_SECS, S3_SECRET_ACCESS_KEY, CYPRESS_API_KEY, CYPRESS_PASSWORD, CYPRESS_USER, CYPRESS_USER_API_KEY_HASH, CYPRESS_USER_PASSWORD_HASH added

  • .nice-as-service added

  • yarn_setup job runs now also on schedules

  • cypress job

    • extends .nice-as-service

    • CYPRESS_INSTALL_BINARY version changed

    • variables HIBERNATE_MAIN_SERVERNAME, HIBERNATE_MAIN_DATABASENAME, POSTGRES_USER, CYPRESS_BASE_URL, BACKEND_URL

    • replace scripts

msg-validation.js

  • add command

  • remove unused code

Changed in version 3.5

.gitlab-ci.yml

  • variable NICE_VERSION removed

  • variable POSTGRES_VERSION added and Postgres updated to v15

  • test job

    • artifacts are no longer saved

    • command yarn test:ci instead of yarn test --ci

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

    • add command yarn db:create-template

    • run cypress with --browser chrome

  • sonarqube job no longer depends on test job

helpers.sh

  • in method getDevDependenciesGreps command adjusted to extract dev dependencies after updating lerna

Changed in version 3.6

.gitlab-ci.yml

  • image from node 16 to node 18 updated

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

Changed in version 3.7

.gitlab-ci.yml

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

    • yarn cypress:run --browser chrome replaced with yarn cypress:run

  • add customer sonarqube job

Changed in version 3.8

.gitlab-ci.yml

  • removed tagging script

  • adjust command for releasing

  • adjust command for branch protection

  • remove gitlab job storybook-link and do command in storybook-mr

  • add stage cleanup and job to handle failed cherry-pick pipelines

else

  • rename protect-branch.js to protectBranch.js

  • rewrite of releasing script

    • added files helpers/*, cherrypick.js, releasePackages.js, releasePackagesHelpers.js, releasePackagesHelpers.spec.js, storybookBranch.js, storybookHelpers.js, storybookHelpers.spec.js, storybookMergeRequest.js, validateCommitMessage.js, validateCommitMessageHelpers.js, validateCommitMessageHelpers.spec.js and variables.json

    • removed files auto-release.sh, build_storybook_mr_target.sh, cherrypick.sh, deploy_storybook_gitlab.sh, git.sh, helpers.sh, msg-validation.js, release-package.sh, release-packages.sh, storybook_link_gitlab.sh and tagging.sh

  • send slack message when cherry-pick merge request could not have been created (handleFailedMergeRequest.js)

Changed in version 3.9

.gitlab-ci.yml

  • use latest node 20 alpine image

  • removed SPRING_PROFILE as variable of .nice-as-service

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

  • sonarqube job

    • updated from jdk11 to jdk17

Changed in version 3.10

.gitlab-ci.yml

  • adjusted before_script of .nice-as-service (no longer use asc files)

else

  • remove google.asc and postgres.asc

  • sonarqube job

    • -Dsonar.login=$SONARQUBE_KEY replaced with -Dsonar.token=$SONARQUBE_KEY

Changed in version 3.11

.gitlab-ci.yml

  • use latest node 22 alpine image

  • cypress job

    • use node 22 cypress image

    • CYPRESS_INSTALL_BINARY version changed

  • define abstract yarnv4 and extend all existing gitlab jobs. if the job has a before_script it must manually extend the before_script of yarnv4

releasePackages.js

  • adjust git adding files

  • adjust yarn publish for yarn v4 (renamed method writeYarnrc and new command for yarn publish)

  • use yarn version for version update (replaces --new-version argument)

releasePackagesHelpers.js

  • complete rewrite of method writeNpmrc / writeYarnrc

else

  • update from yarn v1 to yarn v4