This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.

CI

Setup

Gitlab is used as host provider and CI of the tocco-client project.

The triggered pipelines can be inspected here: Gitlab CI Pipelines.

Stages and Jobs

Alls jobs can be found in the .gitlab-ci.yml file in the root.

There are a set of jobs that are triggered for every merge request and push to master. The jobs ensure, that the linting command and all unit tests run successfully and storybook can be build. Also the builded storybook gets pushed to the Gitlab Storybook Repository and a link to this storybook deployment is written as comment to the merge request.

There is also a scheduled pipeline that runs daily on master testing all the above and also runs the more time consuming end-to-end tests with cypress.

Also there are three jobs that support the development workflow. One will create git tags for each release based on the release commit. This has to be done since tags created by yarn get lost after a rebase of the release branch. The next job handles the auto-releasing of packages on the release branches (if the changelog is not empty a package is released) on each commit. The last job handles cherry-picking across different versions and is described here Cherry-pick job

Caching

All node_modules folder are getting cached since a installation of all dependencies from scratch takes very long. The yarn.lock files act as keys. That means if a dependency changes, a new cache is created. Each job should define if it pull, pushed or pull and pushed to the cache to speed things up.

Image

As docker image the linux alpine version is used to speed up the image initialization phase by several seconds. As downside of this optimization, some dependencies must be installed by a job with apk.

Releasing

In the CI pipeline the script is executed with the --auto argument

  • setup variables (last release tag, last version, changelog, nice version, release tag)

  • define next version of package (increase hotfix version)

  • check if git tree is clean (else stop)

  • check if changelog of package is not empty (else stop)

  • commit changelog

  • run yarn publish which creates a new chore commit

  • move npm dist tag to published hotfix version

Cherry-pick job

To explain this rater complex job.

Naming

  • source branch: branch on which the CI job is running (e.g. nice-releases/227)

  • target branch: branch to which the commits should be cherry-picked (next release branch or master; e.g. nice-releases/228)

  • auto merge tag: each release branch has a git tag like auto-merge/${VERSION}. All commits before this tag (including the commit itself) are already cherry-picked (if necessary)

Workflow

  • setup variables (auto merge tag, source branch, nice version)

  • configure git and clone repo (only source & target branch and all tags of these branches)

  • get commit id which belongs to the auto merge tag of the source branch (e.g. tag auto-merge/227)

  • checkout source branch

  • retrieve commits which should be cherry-picked. Such a commit has a Cherry-pick:Up in the commit message body and are after the auto merge tag in the history (=not already cherry-picked)

  • stop if nothing should be cherry-picked

  • checkout target branch

  • cherry pick commit by commit

  • create and push new branch cherry-picking/_TARGET-BRANCH_-auto-xxxxxxxx

  • move auto merge tag to head of source branch

Fix failing cherry-pick job

For example there could be an auto merge conflict. In such a case there is log message like this:

CONFLICT (content): Merge conflict in packages/entity-browser/src/routes/list/containers/ListViewContainer.js

Workflow for fixing (in the example the source branch is nice-releases/227 and the target branch nice-releases/228) :

  • Check log which commits are cherry-picked (see line 2 und 3):

c2b6b34cdca570fd36bdef75682b702a9702938b is commit id of tag auto-merge/227
00f9bed1a0969efceaaed43e10c952f1e0fd57f0 has 'Cherry-pick: Up' in commit message
952f1e0fd57f000f9bed1a0969efceaaed43e10c has 'Cherry-pick: Up' in commit message
Switched to branch 'nice-releases/228'

  • Checkout the target branch

  • Cherry pick the commits with git cherry-pick ${COMMIT_ID} and fix conflicts

  • Create a PR

  • Optional: if the PR is approved release all necessary packages (all necessary packages will be released automatically every night)

  • Move the auto merge tag to the head of the source branch

git checkout ${HEAD_COMMIT_ID}
git tag -f auto-merge/${SOURCE_BRANCH_NICE_VERSION}
git push origin auto-merge/${SOURCE_BRANCH_NICE_VERSION} -f

Merge Request

There is one job that automates the creation of merge requests after a successful release of a package (or cherry picking), the branch containing the release (or cherry picked) commits must be rebased to its target branch using a merge request.

CI scripts on different versions

In version 2.27 many client CI scripts were introduced or adjusted such as auto-releasing or cherry-picking. If a CI feature is added or modified this should be done ideally on version 2.27 if possible. However between 2.27 and the latest version are some differences in the scripts which are documented here. The command git diff origin/nice-releases/34 .gitlab-ci.yml scripts ':!scripts/nice2' ':!scripts/widgets' (on branch nice-releases/35) is used to extract the difference (Folder nice2 and widgets are excluded as the are only used by Cypress which runs on the master branch).

Changed in version 2.28

tagging.sh

  • exit 0 removed

Changed in version 2.29

helpers.sh

  • in method setNiceVersion command adjusted

release-package.sh

  • return statement adjusted

Changed in version 3.1

.gitlab-ci.yml

  • image from node 14 to node 16 updated

  • formatting job added

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

    • execution command adjusted

    • artifacts storage added

  • sonarcube job added

msg-validation.js

  • code reformatted

Changed in version 3.2

.gitlab-ci.yml

  • yarn_setup job packages/*/ replaced with packages/*/*/

  • test job artifacts storage added

  • cypress job expire_in changed

  • sonarqube job before_script added

helpers.sh

  • in method getDevDependenciesGreps command adjusted to get dev dependencies

  • method setPackageDir added

  • method checkPackage adjusted

release-package.sh

  • adjust navigation to folder

release-packages.sh

  • adjust iterating over folders

tag-all-packages.sh

  • script added

Changed in version 3.3

.gitlab-ci.yml

  • cypress job

    • upgrade cypress image

    • CYPRESS_INSTALL_BINARY version changed

helpers.sh

  • setPackageDir supports now bundle packages

tag-all-packages.sh

  • add registry to npm dist-tag add ... command

Changed in version 3.4

.gitlab-ci.yml

  • variables NICE_VERSION, DOCKER_NICE_VERSION, NICE_TIMEOUT_SECS, S3_SECRET_ACCESS_KEY, CYPRESS_API_KEY, CYPRESS_PASSWORD, CYPRESS_USER, CYPRESS_USER_API_KEY_HASH, CYPRESS_USER_PASSWORD_HASH added

  • .nice-as-service added

  • yarn_setup job runs now also on schedules

  • cypress job

    • extends .nice-as-service

    • CYPRESS_INSTALL_BINARY version changed

    • variables HIBERNATE_MAIN_SERVERNAME, HIBERNATE_MAIN_DATABASENAME, POSTGRES_USER, CYPRESS_BASE_URL, BACKEND_URL

    • replace scripts

msg-validation.js

  • add command

  • remove unused code

Changed in version 3.5

.gitlab-ci.yml

  • variable NICE_VERSION removed

  • variable POSTGRES_VERSION added and Postgres updated to v15

  • test job

    • artifacts are no longer saved

    • command yarn test:ci instead of yarn test --ci

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed

    • add command yarn db:create-template

    • run cypress with --browser chrome

  • sonarqube job no longer depends on test job

helpers.sh

  • in method getDevDependenciesGreps command adjusted to extract dev dependencies after updating lerna

Changed in version 3.6

.gitlab-ci.yml

  • image from node 16 to node 18 updated

  • cypress job

    • CYPRESS_INSTALL_BINARY version changed