Why?

When setting up a Golang working environment, broadly speaking, there are two ways: using a package manager/installer or installing from source. Most of the time it’s much easier to use a package manager or installer and by executing one or two commands, the entire working environment is ready to use. Well, this time, I just would like to see how manual installation works.

Let me first list all the steps that I can think of for the installation:

  1. Get the Golang source code from somewhere, maybe a Github repo
  2. Build the Golang source into executable binaries
  3. 😱 It seems that I need to have a Go compiler ready before Step 2 (bootstrap)
  4. Install the executable binaries, there might be scripts doing this or there are standard places to put these binaries
  5. A working Go installation depends on several environment varialbes, I may need to set up those variables
  6. Verify the installation

Let me think through the process carefully.

The result of the process

Of course, after running through the process I should have a working Go environment available on my computer, but since I’m building from source, I’ll have to git clone two copies of the Golang source code, one for the the compiler and the other for the actual source. Again, since I will have two copies of Go, I need to tell the system which one to use for the go command.

The compiler

The Go toolchain is written in Go. To build it, I need a Go compiler installed (ref).

This is the compiler that is used for “bootstrapping” the installation. This compiler can be a fully working Go toolchain already setup on my computer or a designated version of Go commands setup just for building and installing of Golang from source.

  • 😱 A working version of Golang means in order to build Go from source, I need to have a working version of Golang on my system.
  • 👻 A designated version of Go setup just for building and installing Golang requires the use of an environment variable named GOROOT_BOOTSTRAP, which directs the bootstrap process to the designated Go compiler.
    • In this case, $GOROOT_BOOTSTRAP/bin/go should be the go command to use.

The source

The source code of Golang is obtained with Git, which means I need to have Git installed locally. I need to find a good place to put the source code, and as suggested by the doc (ref), I can use goroot as the folder name. I need to run a build script to build this copy of source code and the Go tool chain will be installed in <parent dir>/goroot/bin.

Use environment variables to tell the difference

The Go environment uses environment vairables (link) to help the OS to locate Go distributions:

  • GOROOT: the directory containing the Golang distribution (binary files bin, archive files pkg, source code src, and others)
    • The Go distribution’s executables are in GOROOT/bin, e.g., the go command
  • GOPATH: the directorying containing projects outside the Go distribution (three subdirectories, i.e., bin, pkg, and src)
  • GOBIN: the directory containing executables outside the Go distribution, by default, GOBIN is set to $GOPATH/bin

Note:

When using the official Go installer (link) to do the installation, the Go distribution will be installed in /usr/local/go (ref) and its structure is something like this:

/
├─ usr/
│  ├─ local/
│  │  ├─ go/
│  │  │  ├─ bin/
│  │  │  ├─ pkg/
│  │  │  ├─ src/
|  |  |  ├─ <other dirs or files>

  • This directory, /usr/local/go, is the GOPATH and it has three subdirectories containing the source files (src), archive files (pkg), and binary files (bin).

  • The /usr/local/go/bin (the value for GOROOT/bin if GOROOT is set) directory is also added to the PATH environment variable so that all go binaries are accessible as executables from the terminal.

Fetch the source (Step 1 and 3)

I need to have a version of Go toolchain just for bootstrapping the building process. I can use Go 1.4, “the last Go release with a compiler written in C”. This version is available in the official Golang Github repo.

# clone the go source repo into a folder, go1.4 in this case
git clone git@github.com:golang/go.git go1.4

# navigate to the source folder
cd go1.4

# checkout the release-branch.go1.4 branch
git checkout release-branch.go1.4

cd into the src subdirectory, and set CGO_ENABLED=0 in the envionment, and run make.bash

# navigate into the src directory
cd src

# set environment variable CGO_ENABLED to 0
CGO_ENABLED=0

# run make.bash
./make.bash

Once the Golang v1.4 has been unpacked, I need to set the GOROOT_BOOTSTRAP environment variable to its directory, so that $GOROOT_BOOTSTRAP/bin/go is the go command to use for building the Golang source code.

  • ❗️ Do not attempt to reuse this git clone in the later steps, this is the bootstrapping compiler’s repo.
  • ❗️ The go1.4 bootstrap toolchain must be able to properly traverse the go1.4 sources that it assumes are present under this repository root

Fetch the source (Step 1 again 😓)

This time I need to get the copy of Golang source code that will be built.

# clone the source code into a folder, in this case, a folder named goroot
git clone https://go.googlesource.com/go goroot
# or
git clone git@github.com:golang/go.git goroot

# navigate into goroot
cd goroot

# get all the tags (so that I can select a particular version)
git fetch --all --tags 

# checkout a particular tag, e.g., go1.16.3
git checkout go1.16.3
# or create a new branch from a tag
git checkout -b go1.16.3 go1.16.3

Go will be installed in the directory where it is checkout out. If Go is checked out in $HOME/goroot, executables will then be installed in $HOME/goroot/bin.

The directory containing the Golang source code may have any name, but if that folder is $HOME/go, it will conflict with the default location of $GOPATH.

Build and install (Step 2)

To build the Go distribution, just run the commands below, and once the commands finish executing, the building part is done and the Go executables should be in $HOME/goroot/bin.

# navigate into the src subfolder
cd src

# run all.bash
./all.bash

The “installation” is done by setting up environment variables (ref):

  • GOROOT: The root of the Go distribution, often $HOME/go1.X.
    • Its value is built into the tree when it is compiled, and defaults to the parent of the directory where all.bash was run. There is no need to set this unless I want to switch between multiple local copies of the repository.
    • The value should be the directory where the second copy of Go source code is check out into
  • GOPATH: The directory where Go projects outside the Go distribution are typically checked out.
    • The value to this variable should be the parent directory of the bin, pkg, and src directory, where the go install result, library archives, and source dependencies are placed respectively.
  • GOBIN: The directory where executables outside the Go distribution are installed using the go command.
    • Most of the time it’s just $GOPATH/bin

Because the Go toolchain (commands) should be available in terminals, $GOROOT/bin and $GOBIN should be added to the PATH environment variable.

Test the installation

Once the environment variables are set, I can test the installation.

Check the current go version by executing the command below in a termianl:

go version

Check whether the installed Golang is able to build a “Hello, world!” project (ref):

  • Create a main.go file
  • Do something simple with Golang, e.g., printing “Hello, world!” to the terminal (shown below).
package main

import "fmt"

func main() {
	fmt.Println("Hello, world!")
}
  • In a terminal, navigate to the directory containing the main.go file and execute:
go run main.go

When I see “Hello, world!” gets printed in the terminal, I know the installation is good.

Update

How do I update the current installed version to a different one or how do I experiment with my local changes to the Golang source code?

The answer is: just rebuild the updated source.

The code in the $GOROOT/src folder reflects the version of Golang source code I want to build, otherwise git checkout the correct version. Then run

# navigate into the src subfolder
cd $GOROOT/src

# run all.bash
./all.bash

Tail

Now that I have a working Go environment, I can build a 3rd copy of Go source and play with it or I can have multiple Go versions installed at the same time (ref).