This is an internal documentation. There is a good chance you’re looking for something else. See Disclaimer.
CI¶
Setup¶
Gitlab is used as host provider and CI of the tocco-client project.
The triggered pipelines can be inspected here: Gitlab CI Pipelines.
Stages and Jobs¶
Alls jobs can be found in the .gitlab-ci.yml file in the root.
There are a set of jobs that are triggered for every merge request and push to master. The jobs ensure, that the linting command and all unit tests run successfully and storybook can be build. Also the builded storybook gets pushed to the Gitlab Storybook Repository and a link to this storybook deployment is written as comment to the merge request.
There is also a scheduled pipeline that runs daily on master testing all the above and also runs the more time consuming end-to-end tests with cypress.
Also there are two jobs that support the development workflow. One job handles the auto-releasing of packages on the release branches (if the changelog is not empty a package is released) on each commit. The last job handles cherry-picking across different versions and is described here Cherry-pick job
Caching¶
All node_modules folder are getting cached since a installation of all dependencies from scratch takes very long. The yarn.lock files act as keys. That means if a dependency changes, a new cache is created. Each job should define if it pull, pushed or pull and pushed to the cache to speed things up.
Image¶
As docker image the linux alpine version is used to speed up the image initialization phase by several seconds. As downside of this optimization, some dependencies must be installed by a job with apk.
Releasing¶
Manual trigger releasing¶
Using schedule (releasing all packages)
If all packages should be released open Pipeline Schedules and run the job of the corresponding branch.
Using pipeline (releasing only some packages)
Open Run Pipeline on Gitlab. Select the branch where you want to manually release (either master
or nice-releases/VERSION
).
The following variables can be passed:
CI_RUN_AUTO_RELEASE
(mandatory): must be set to1
that the releasing is startedCI_RELEASING_PACKAGES
(optional): per default the releasing is started for all releasable packages. if a comma-separated list is passed (e.g.admin,devcon
) only these packages are release candidatesCI_RELEASING_FORCE
(optional): per default a package is only released if the changelog is not empty. iftrue
is passed as value the package is always released
Fix failing job¶
If the releasing job fails on the Gitlab CI, some packages are already published to NPM but the new version is not committed to the release/master branch. First you must manually change these versions (check log to know which packages were already released). Commit these changes and merge them. Last you must move for each package the git tag. You must execute:
git checkout ${COMMIT_ID}
git tag ${PACKAGE_NAME}/${RELEASE_TAG} -f
git push origin ${PACKAGE_NAME}/${RELEASE_TAG} -f
COMMIT_ID
: commit id on the release/master branch of your created commit on GitlabPACKAGE_NAME
: npm package name which always has the tocco prefix (e.g.tocco-admin
)RELEASE_TAG
: npm release tag name (e.g.nice34
). On the master the tag is named like the upcoming version
Cherry-pick job¶
To explain this rater complex job.
Naming¶
source branch: branch on which the CI job is running (e.g.
nice-releases/227
)target branch: branch to which the commits should be cherry-picked (next release branch or master; e.g.
nice-releases/228
)auto merge tag: each release branch has a git tag like
auto-merge/${VERSION}
. All commits before this tag (including the commit itself) are already cherry-picked (if necessary)
Workflow¶
setup variables (auto merge tag, source branch, nice version)
configure git and clone repo (only source & target branch and all tags of these branches)
get commit id which belongs to the auto merge tag of the source branch (e.g. tag
auto-merge/227
)checkout source branch
retrieve commits which should be cherry-picked. Such a commit has a
Cherry-pick:Up
in the commit message body and are after the auto merge tag in the history (=not already cherry-picked)stop if nothing should be cherry-picked
checkout target branch
cherry pick commit by commit
create and push new branch
cherry-picking/_TARGET-BRANCH_-auto-xxxxxxxx
move auto merge tag to head of source branch
Fix failing cherry-pick job¶
For example there could be an auto merge conflict. In such a case there is log message like this:
CONFLICT (content): Merge conflict in packages/entity-browser/src/routes/list/containers/ListViewContainer.js
Workflow for fixing (in the example the source branch is nice-releases/227
and the target branch nice-releases/228
) :
Check log which commits are cherry-picked (see line 2 und 3):
c2b6b34cdca570fd36bdef75682b702a9702938b is commit id of tag auto-merge/227
00f9bed1a0969efceaaed43e10c952f1e0fd57f0 has 'Cherry-pick: Up' in commit message
952f1e0fd57f000f9bed1a0969efceaaed43e10c has 'Cherry-pick: Up' in commit message
Switched to branch 'nice-releases/228'
Checkout the target branch
Cherry pick the commits with
git cherry-pick ${COMMIT_ID} -x
and fix conflictsCreate a PR
Optional: if the PR is approved release all necessary packages (all necessary packages will be released automatically every night)
Move the auto merge tag to the head of the source branch
git checkout ${HEAD_COMMIT_ID}
git tag -f auto-merge/${SOURCE_BRANCH_NICE_VERSION}
git push origin auto-merge/${SOURCE_BRANCH_NICE_VERSION} -f
Merge Request¶
There is one job that automates the creation of merge requests after a successful release of a package (or cherry picking), the branch containing the release (or cherry picked) commits must be rebased to its target branch using a merge request.
CI scripts on different versions¶
In version 2.27
many client CI scripts were introduced or adjusted such as auto-releasing or cherry-picking.
If a CI feature is added or modified this should be done ideally on version 2.27
if possible.
However between 2.27
and the latest version are some differences in the scripts which are documented here.
The command git diff origin/nice-releases/34 .gitlab-ci.yml scripts ':!scripts/nice2' ':!scripts/widgets'
(on branch nice-releases/35
) is used to extract the difference
(Folder nice2
and widgets
are excluded as the are only used by Cypress which runs on the master branch).
Changed in version 2.28¶
tagging.sh
exit 0
removed
Changed in version 2.29¶
helpers.sh
in method
setNiceVersion
command adjusted
release-package.sh
return statement adjusted
Changed in version 3.1¶
.gitlab-ci.yml
image from node 14 to node 16 updated
formatting job added
cypress job
CYPRESS_INSTALL_BINARY
version changedexecution command adjusted
artifacts storage added
sonarcube job added
msg-validation.js
code reformatted
Changed in version 3.2¶
.gitlab-ci.yml
yarn_setup job
packages/*/
replaced withpackages/*/*/
test job artifacts storage added
cypress job
expire_in
changedsonarqube job
before_script
added
helpers.sh
in method
getDevDependenciesGreps
command adjusted to get dev dependenciesmethod
setPackageDir
addedmethod
checkPackage
adjusted
release-message.js
script added
release-package.sh
adjust navigation to folder
add slack message for package release =>
release-message.js
release-packages.sh
adjust iterating over folders
tag-all-packages.sh
script added
Changed in version 3.3¶
.gitlab-ci.yml
- cypress job
upgrade cypress image
CYPRESS_INSTALL_BINARY
version changed
helpers.sh
setPackageDir
supports now bundle packages
tag-all-packages.sh
add registry to
npm dist-tag add ...
command
Changed in version 3.4¶
.gitlab-ci.yml
variables
NICE_VERSION
,DOCKER_NICE_VERSION
,NICE_TIMEOUT_SECS
,S3_SECRET_ACCESS_KEY
,CYPRESS_API_KEY
,CYPRESS_PASSWORD
,CYPRESS_USER
,CYPRESS_USER_API_KEY_HASH
,CYPRESS_USER_PASSWORD_HASH
added.nice-as-service
addedyarn_setup job runs now also on
schedules
cypress job
extends
.nice-as-service
CYPRESS_INSTALL_BINARY
version changedvariables
HIBERNATE_MAIN_SERVERNAME
,HIBERNATE_MAIN_DATABASENAME
,POSTGRES_USER
,CYPRESS_BASE_URL
,BACKEND_URL
replace scripts
msg-validation.js
add command
remove unused code
Changed in version 3.5¶
.gitlab-ci.yml
variable
NICE_VERSION
removedvariable
POSTGRES_VERSION
added and Postgres updated to v15test job
artifacts are no longer saved
command
yarn test:ci
instead ofyarn test --ci
cypress job
CYPRESS_INSTALL_BINARY
version changedadd command
yarn db:create-template
run cypress with
--browser chrome
sonarqube job no longer depends on test job
helpers.sh
in method
getDevDependenciesGreps
command adjusted to extract dev dependencies after updating lerna
Changed in version 3.6¶
.gitlab-ci.yml
image from node 16 to node 18 updated
cypress job
CYPRESS_INSTALL_BINARY
version changed
Changed in version 3.8¶
.gitlab-ci.yml
removed tagging script
adjust command for releasing
else
rewrite of releasing script
added files
helpers/*
,releasePackages.js
,releasePackagesHelpers.js
,releasePackagesHelpers.spec.js
andvariables.json
removed files
auto-release.sh
,helpers.sh
,release-package.sh
,release-packages.sh
andtagging.sh.sh