Let's Learn Go -- (3) Go Modules

1. Go’s Package Management Mechanisms

1.1 GOPATH

GOPATH pulls code with the go get command and places it in the GOPATH directory.

The problems with GOPATH are:

It cannot manage package versions
It uses a global repository, so it cannot isolate projects effectively

1.2 Vendor

Starting with version 1.5, Go added the Vendor mechanism. Vendor solved some of GOPATH’s problems.

The Vendor mechanism manages dependency packages by adding a vendor folder under the project directory.

The problems with Vendor are:

It cannot resolve nested dependencies
vendor is only valid under a GOPATH path

1.3 Go modules

Go modules allows project code to live in any directory, with dependency packages stored uniformly under $GOPATH/pkg/mod, avoiding the duplicated code of the vendor approach.

2. Features of Go modules

Go controls whether the Go modules feature is enabled through the GO111MODULE switch variable, which has three possible values: auto/on/off. It was introduced in version 1.11, with a default of auto, and enabled by default in version 1.13.

auto means that the feature is enabled when there is a go.mod in the current directory. In auto mode:

When compiling inside a GOPATH directory, vendor and GOPATH are used for package management by default
When compiling outside a GOPATH directory, the settings in go.mod are used for package management by default.

While using Go modules, two files are generated automatically: go.sum and go.mod. Usually both files are also committed to the code repository.

go.mod

go.mod records the version information and operation commands for dependency packages. go.mod provides four commands: module, require, replace, and exclude.

1
2
3
4
cat go.mod
module github.com/project/hello
go 1.12
require rsc.io/quote v3.1.0+incompatible

go.sum

go.sum provides version checksums.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
cat go.sum
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:qgOY6WgZOaTkIIMiVjBQcw
93ERBE4m30iBm00nkL0i8=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3
fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
rsc.io/quote v3.1.0+incompatible h1:5v8TkzZ3hgTuFV/P47Ib17+Lc9DHpLRa+HPYYhT/X9
o=
rsc.io/quote v3.1.0+incompatible/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPX
sUe+TKr0=
rsc.io/sampler v1.3.0 h1:7uVkIFmeBqHfdjD+gZwtXXI+RODJ2Wc4O7MPEh/QiW4=
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9JXDnKaTXpA=

3. Configuring a Proxy

When installing Go dependency packages, data is requested from the public internet. Some packages are hosted on github.com, and others are hosted in repositories such as golang.org and k8s.gcr.io. Since Google-related URLs are blocked, you often run into network access problems.

Version 1.11 added a new environment variable, GOPROXY, which can be used to configure a mirror proxy for code repositories.

Taking the configuration of jfrog’s GoCenter as an example, run the following in the runtime environment:

1
export GOPROXY=https://gocenter.io

That is all it takes to use the proxy provided by jfrog. In addition, GoCenter also offers a package search feature.

Of course, aliyun also provides a GOPROXY:

1
export GOPROXY=https://mirrors.aliyun.com/goproxy/

4. Basic Go modules Commands

The Go modules help documentation already describes this in great detail.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
go help mod
Go mod provides access to operations on modules.

Note that support for modules is built into all the go commands,
not just 'go mod'. For example, day-to-day adding, removing, upgrading,
and downgrading of dependencies should be done using 'go get'.
See 'go help modules' for an overview of module functionality.

Usage:

	go mod <command> [arguments]

The commands are:

	download    download modules to local cache(下载依赖包到本地)
	edit        edit go.mod from tools or scripts(编辑 go.mod 文件)
	graph       print module requirement graph(打印模块依赖图)
	init        initialize new module in current directory(将当前目录初始化为新模块)
	tidy        add missing and remove unused modules(拉取缺失的模块，移除没有使用到的模块)
	vendor      make vendored copy of dependencies(将依赖包复制到 vendor)
	verify      verify dependencies have expected content(验证依赖包)
	why         explain why packages or modules are needed(解释为什么需要依赖)

Use "go help mod <command>" for more information about a command.

Create a new directory hello, and add a file hello.go inside it

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
cat hello.go
package main

import (
    "fmt"
    "rsc.io/quote"
)

func main() {
    fmt.Println(quote.Hello())
}

Initialize the package with the repository address

1
go mod init github.com/project/hello

Compile and produce an executable file

1
go build

Archive the dependency packages into the project’s vendor directory

1
2
3
4
5
6
7
8
go mod vendor -v
# golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
golang.org/x/text/language
golang.org/x/text/internal/tag
# rsc.io/quote v3.1.0+incompatible
rsc.io/quote
# rsc.io/sampler v1.3.0
rsc.io/sampler

Verify the dependencies

1
2
go mod verify
all modules verified

Check the final directory structure

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
tree -L 2
.
├── go.mod
├── go.sum
├── hello
├── hello.go
└── vendor
    ├── golang.org
    ├── modules.txt
    └── rsc.io

5. References

https://blog.golang.org/using-go-modules