Because I am responsible for the team’s shared CI matters, I often need to configure CI pipelines or help others solve problems, so I put together the CI scripts I use most often for easy reference.
1. Structure of .gitlab-ci.yml
Below is the structure of a GitLab CI configuration file.
.gitlab-ci.yml file
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| # 一些前置脚本，完成激活环境等操作
before_script:
- source /data/runner/node/bin/activate
- which node && node --version
- which npm && npm --version
- LANG="zh_CN.utf8"
- export LC_ALL=zh_CN.UTF-8
# 编排需要执行的 stage
stages:
- build
- deploy
# 定义 job。job 属于某一个 stage，比如这里的 build 、deploy。GitLab CI 会按照 stages 配置的先后，顺序执行每一个 stage。
|
2. Building a Webpack Project and Committing It to the GitLab Repository
The conventions here are:
- The frontend project is in the webpack directory at the repository root
- The frontend project’s compiled output goes to the static/dist directory at the repository root
- The frontend build command is
npm run build
GitLab CI’s built-in variables are documented here. You can also define your own variables with the variables keyword. The directory paths above, for instance, could be configured as variables. But every project has its own peculiarities, so I have not written them in a generic form here, to avoid misleading anyone.
Usually I configure account information on the settings/ci_cd page and check these variables as protected. Once protected, only protected branches can obtain these variables — that is, the GIT_USERNAME and GIT_PASSWORD parts below.
It is strongly recommended here that you do not commit Git directly in the current code directory; you should clone a fresh copy of the code locally instead, so that the environment is not polluted.
.gitlab-ci.yml file,
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
| stages:
- build
build-webpack:
stage: build
variables:
CI_REPOSITORY_URL:
http://$GIT_USERNAME:$GIT_PASSWORD@gitlab.yourdomain.com/$CI_PROJECT_PATH.git
cache:
untracked: true
paths:
- webpack/node_modules
script:
- echo "start build and commit"
- cd webpack
# 可通过关键字优化
- npm install
# 可通过关键字优化
- npm run build
- cd ..
- rm -rf git-dir
- git clone -b $CI_COMMIT_REF_NAME http://$GIT_USERNAME:$GIT_PASSWORD@gitlab.yourdomain.com/$CI_PROJECT_PATH.git git-dir
- cd git-dir && rm -rf ./static/dist
- mv ../static/dist static/
- git config --global user.name $GITLAB_USER_NAME
- git config --global user.email $GITLAB_USER_EMAIL
- git add static/dist
# 避免循环构建
- git commit -m "auto commit [ci skip]`git log -1 --pretty=%B`" || exit 0
- git push $CI_REPOSITORY_URL $CI_COMMIT_REF_NAME >/dev/null 2>&1 || exit 0
- echo "end build and commit"
tags:
# 指定 runner
- linux
- shell
only:
# 指定分支
- master
|
3. Pushing Code from GitLab to an SVN Repository
Because our release system uses an SVN repository, but Git is better suited to multi-person collaboration, we use a GitLab repository during development.
To automatically push the GitLab repository to the SVN repository, you can use the following job:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| push-to-svn:
stage: deploy
variables:
CI_REPOSITORY_URL:
http://$GIT_USERNAME:$GIT_PASSWORD@gitlab.yourdomain.com/$CI_PROJECT_PATH.git
script:
- echo "start push to svn"
- echo "step 1/3: git clone"
- git clone -b $CI_COMMIT_REF_NAME $CI_REPOSITORY_URL git-dir
- echo "finished"
- echo "setp 2/3: svn checkout"
- echo 't' | svn checkout $SVN_PATH svn-dir --username $SVN_USERNAME --password $SVN_PASSWORD --no-auth-cache
- echo "finshed"
- echo "step 3/3: push git master to svn trunk"
# 确保删除文件操作有效，可优化
- cd svn-dir && svn delete *
- rsync -avq git-dir/ svn-dir/
- cd ./svn-dir
- svn add * --force
- echo 't' | svn commit -m "`git log -1 --pretty=%B`" --username $SVN_USERNAME --password $SVN_PASSWORD --no-auth-cache
- echo "end push to svn"
|
4. Controlling Script Execution with Keywords
GitLab CI provides some built-in keywords for controlling CI behavior. Adding [ci skip] or [skip ci] to the commit message, for example, skips the CI build triggered by that commit.
Inspired by this idea, we can also match certain keywords in the commit message inside our scripts, to control the execution logic of the script.
- Using whether the keyword
[install] is present, decide whether to install dependencies
1
| if [[ $(git log -1 --pretty=%B) = *"["*"install"*"]"* ]]; then npm install; else echo "not npm install"; fi;
|
- Using whether the keyword
[build] is present, decide whether to bundle the frontend
1
| if [[ $(git log -1 --pretty=%B) = *"["*"build"*"]"* ]]; then npm run build; else echo "not npm install"; fi;
|
- Using whether the keyword
[delete] is present, decide whether files need to be deleted when pushing to SVN. This is mainly because the svn add * command cannot commit deleted files; deleting a file requires svn delete. If you do not use the svn delete command, files deleted in GitLab will in fact not be deleted in SVN.
1
| if [[ $(git log -1 --pretty=%B) = *"["*"delete"*"]"*]]; then cd svn-dir && svn delete * && cd ..; else echo "not svn delete"; fi;
|
5. Reading Variables from a yml
yml file format:
1
2
3
4
| code: test
name: 测试
author: admin
version: 1.2.3
|
Taking the yml format above as an example, we need to read the relevant field values.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
| parse_yaml() {
local prefix=$2
local s='[[:space:]]*' w='[a-zA-Z0-9_]*' fs=$(echo @|tr @ '\034')
sed -ne "s|^\($s\)\($w\)$s:$s\"\(.*\)\"$s\$|\1$fs\2$fs\3|p" \
-e "s|^\($s\)\($w\)$s:$s\(.*\)$s\$|\1$fs\2$fs\3|p" $1 |
awk -F$fs '{
indent = length($1)/2;
vname[indent] = $2;
for (i in vname) {if (i > indent) {delete vname[i]}}
if (length($3) > 0) {
vn=""; for (i=0; i<indent; i++) {vn=(vn)(vname[i])("_")}
printf("%s%s%s=\"%s\"\n", "'$prefix'",vn, $2, $3);
}
}'
}
eval $(parse_yaml test.yml "config_")
|
Running the script above gives you the field values from test.yml.
1
2
3
| echo ${config_code}
echo ${config_name}
...
|
6. Writing an Environment Variable Value into a yml File
yml file format:
1
2
3
4
| code: test
name: 测试
author: admin
version: 1.2.3
|
Taking the yml format above as an example, read the VER value from the environment variables and write it after version:.
1
| sed -i "/version:/s/[0-9].*$/${VER}/g" test.yml
|
