Switching to Go Modules

Package management has not been part of core Golang before. Each year, there has been a different de facto standard. In recent years, it was “official [core team] experiment” dep. Finally, we have Go Modules.

In this blog post I’ll share the quick guide starting a new project with Go Modules and the solution to replace packages with your own forked branches.

What am I using?

Your project was using dep if it had Gopkg.toml and Gopkg.lock files.

You’ll now see projects everywhere switching to go.mod and go.sum as the two metadata files for Go Modules.

Learn more about Go Modules

After reading this blog post, read https://blog.golang.org/using-go-modules and https://blog.golang.org/modules2019 to learn more about Go Modules.

Getting Started

To install Go Modules you need to do nothing beyond upgrading to Go 1.11 or 1.12 (1.12.5 is latest at time of writing). You will now have a go mod family of commands.

Let’s create a go mod project with some dependencies, and aliased forks of dependencies.

mkdir -p ~/workspace/my-go-projectcd ~/workspace/my-go-project

Historically, we needed to create Go projects within $GOPATH. No more. We can now create Go projects anywhere we like.

To initialize the project for Go Modules, we run go mod init [module path]. Typically a module path is a url such as the github.com url for the project.

go mod init github.com/starkandwayne/my-go-project

This creates a go.mod :

module github.com/starkandwayne/my-go-project
go 1.12

Create a main.go that uses a color library to print Hello World in red.

package main
import (
	"github.com/fatih/color"
)
func main() {
	color.Red("Hello Red Planet")
}

Without explicitly importing github.com/fatih/color dependency we can immediately run our application:

$ go run main.go
Hello Red Planet

Trust me, it’ll be in red. I could have picked any example module but I go and use a module that I cannot visually demonstrate on this blog. Idiot.

During go run the Go Module subsystem automatically detected the new dependency and added it to go.mod:

module github.com/starkandwayne/my-go-project
go 1.12
require (
	github.com/fatih/color v1.7.0 // indirect
	github.com/mattn/go-colorable v0.1.2 // indirect
)

It also documented the complete set of nested dependencies in go.sum:

github.com/fatih/color v1.7.0 h1:DkWD4oS2D8LGGgTQ6IvwJJXSL5Vp2ffcQg58nFV38Ys=
github.com/fatih/color v1.7.0/go.mod h1:Zm6kSWBoL9eyXnKyktHP6abPY2pDugNf5KwzbycvMj4=
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

Congratulations, we’re successfully using Go Modules.

I implore you to try this yourself just to see the color red.

Using forks and branches

Let’s say we want to fork github.com/fatih/color to add a bug fix or new feature. We need Go Modules to give us a way to leave our main.go application untouched, but in place of github.com/fatih/color we want to use our own github.com/drnic/fatih-color fork, and perhaps an alternate branch or tag.

That is, I want to fork/change this project, but I want main.go to stay with github.com/fatih/color.

package main
import (
	"github.com/fatih/color"
)
func main() {
	color.Red("Hello Red Planet")
}

To use my fork, https://github.com/drnic/fatih-color, and a branch silly-changes, I need to update go.mod to use the replace helper:

module github.com/starkandwayne/my-go-project
go 1.12
require (
	github.com/fatih/color v1.7.0 // indirect
	github.com/mattn/go-colorable v0.1.2 // indirect
)
replace github.com/fatih/color => github.com/drnic/fatih-color silly-change

When we run our application Go Modules will automatically switch out the github/fatih/color usage for my own fork/branch.

$ go run main.go
go: finding github.com/drnic/fatih-color silly-change
go: downloading github.com/drnic/fatih-color v1.7.1-0.20181010231311-3f9d52f7176a
go: extracting github.com/drnic/fatih-color v1.7.1-0.20181010231311-3f9d52f7176a
Hello Red Planet

If we look at go.mod again, we’ll see that it replaced my branch name silly-change with a specific reference that is part version number v1.7.1-0., and part git sha 3f9d52f7176a.

module github.com/starkandwayne/my-go-project
...
replace github.com/fatih/color => github.com/drnic/fatih-color v1.7.1-0.20181010231311-3f9d52f7176a

It will no longer fetch any new changes to the silly-change branch.

If I make new changes to my silly-change branch and want to try them out inside my application, I need to edit my replace again to the branch name, run my code or tests, and Go Modules will do the rest. The go.mod file will again be updated to replace silly-change to an explicit reference.

$ cat go.mod
...
replace github.com/fatih/color => github.com/drnic/fatih-color silly-change
$ go run main.go
go: finding github.com/drnic/fatih-color silly-change
go: downloading github.com/drnic/fatih-color v1.7.1-0.20190530040759-06d1dfbe5aa7
go: extracting github.com/drnic/fatih-color v1.7.1-0.20190530040759-06d1dfbe5aa7
Hello Red Planet
$ cat go.mod
...
replace github.com/fatih/color => github.com/drnic/fatih-color v1.7.1-0.20190530040759-06d1dfbe5aa7

Initial Thoughts on Go Modules

I like that my Go projects – applications or libraries – can now we placed anywhere I want on my computer, rather than curate a large $GOPATH forest of my projects and their dependencies.

I like that Go Modules functionality “just works” without explicit commands to install dependencies.

I am glad I was pointed to the replace command. I do fork, fix, change other people’s libraries a lot and definitely need a way to temporarily/long-term bring in my upstream changes into my downstream projects.

I did discover that if my project vendors its dependencies, then I needed to explicitly run go run vendor to update the vendor/ folder after any changes to go.mod. I’m not sure how I feel about this extra step that I might forget to do. Or that you and others might not know you need to do and so your go.mod and go.sum become out of sync with the contents of vendor.

Thanks

Thanks to Stephen Levine who first explained what these new go.mod and go.sum files were that had started appearing in Cloud Foundry projects, and then helped answered my bonus questions about Go Modules, pointed me to replace, and more.