1. Problem Description

When a Webhook was configured to automatically trigger a Jenkins pipeline, the following error occurred:

1
2
3

hudson.plugins.git.GitException: Command "git checkout -f 23b446ea" returned status code 128:
stdout:
stderr: fatal: reference is not a tree: 23b446ea

2. How Git Manages Versions

Git is a content-addressable filesystem. Git maintains a sha tree, and any historical node can be traced back to via its sha value. Let’s look at the commit records first:

Run the command:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

git log

commit e5562ec75fedf0bd66fe752e686db108cf20cf9b (HEAD -> fix_create_pipeline, origin/fix_create_pipeline)
Date:   Thu May 14 15:31:50 2020 +0800

    fix create pipeline error

commit 1bd660c370aca1113ae0b4fa2e814e9ea337e4d7
Date:   Thu May 14 13:22:30 2020 +0800

    fix proxy bug (#2070)

commit 1464ca197d161b2ed693c7e9a053fcc97e0fdbaa (origin/master, origin/HEAD, fix_apiserver_devops)
Merge: 1d48ca34 250dd4b0
Date:   Wed May 13 17:24:31 2020 +0800

    Merge pull request #2057 from

    fix bug of pvc api

You can see that every time you commit, git generates a commitId, which is a sha. The parentheses immediately after the commitId explain the changes in the branch that the commit content belongs to.

So what do these (origin/master, origin/HEAD, fix_apiserver_devops) mean?

In .git/refs/, Git stores pointers to the commit objects that point to data (branches).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

tree .git/refs/

.git/refs/
├── heads
│   ├── feature_add_pipeline_api
│   ├── fix_apiserver_devops
│   ├── fix_create_devops
│   ├── fix_create_pipeline
│   ├── master
│   └── release-2.1
├── remotes
│   ├── origin
│   │   ├── HEAD
│   │   ├── feature_add_pipeline_api
│   │   ├── fix_create_devops
│   │   ├── fix_create_pipeline
│   │   ├── fix_devops
│   │   └── master
│   └── upstream
│       ├── master
│       └── release-2.1
└── tags

1
2
3

cat .git/refs/heads/feature_add_pipeline_api

f0749ac442ef683a2de6ed3fdc8e3e3b44b71076

To Git, branches and tags are just pointers to a certain sha in the sha tree.

3. Cause Analysis

The chance of Jenkins or a related Git plugin being at fault is small; of course, you can also check the Git version. Using the controlled-variable method, first run the failing command outside of Jenkins.

1

git checkout -f 23b446ea

After understanding how Git works, the cause can probably be analyzed: people use branchName and tagName, while Git uses sha. reference not a tree means that the sha tree changed between the trigger stage and the execution stage. At trigger time, the sha corresponding to branchName did not exist at the execution stage. Below is some possible analysis based on scenarios others have encountered:

• Branch name and tag name are the same

The difference between a branch and a tag is that a branch can continue to receive commits, while a tag cannot be changed further. When the two have the same name, some odd behavior occurs, for example branchName disappearing.

• Caused by rebase

rebase is usually used to merge code from upstream, and also to merge multiple linear commits into one. For example, the command below merges the commits of the first 3 versions into 1:

1

git rebase -i HEAD~3

This causes commits to be lost, and the lost commit may have been exactly what triggered some action in Jenkins.

• After a PR is merged, the branch is deleted

Based on the idea of feature development, trunk integration, branch release, in the continuous integration process the best practice is to delete the branch immediately after the code is merged; GitHub has this flow wired up.

After a feature branch triggers a Jenkins pipeline, the branch is deleted, and when execution reaches pulling the code, an error is also reported.

• Subumodule

When the commit records of a sub-repository change, operations such as git update may also report errors.

• Not the latest code

Cached code was used, and the latest commit cannot be found on the old code.

Because usage and scenarios differ in Jenkins practice, unless the error is reproducible every time, troubleshooting needs to center on the crux of the problem and screen it out step by step with controlled variables.

4. References

• https://github.com/kubernetes/kubernetes/issues/27462