What's coming in Go 1.15

Go 1.15, the 16th major version of the Go programming language, is due out on August 1. It will be a release with fewer changes than usual, but many of the major changes are behind-the-scenes or in the tooling: for example, there is a new linker, which will speed up build times and reduce the size of binaries. In addition, there are performance improvements to the language's runtime, changes to the architectures supported, and some updates to the standard library. Overall, it should be a solid upgrade for the language.

Since the release of Go 1.0, the Go team has consistently shipped improvements to the tooling and the standard library with each version, but has always been conservative about language changes. Many other languages ship significant language features every release, but Go has only shipped a few minor ones in the versions since 1.0.

This is a conscious design choice: since the 1.0 release, the emphasis from the team has been stability and simplicity. The Go 1 compatibility promise guarantees that all programs written for Go 1.0 will continue to run correctly, unchanged, for all 1.x versions. Go programmers usually see this as a good thing — their programs continue to "just work", but generally get consistently faster.

In the upcoming 1.15 version, changes to the language specification are basically non-existent as expected; the improvements are in the tooling, the performance of the compiler, and in the standard library. As tech lead Russ Cox noted, the core developers are planning to be extra-conservative in 1.15 given the pandemic:

On May 1, Go 1.15 entered feature freeze, and the Go team plans to make the final release on August 1, keeping to the regular sixth-month release cycle.

The Go development model is rather different than that of most open-source languages. The language was designed at Google and most of the core developers work there (so ongoing development is effectively sponsored by Google). The language has a permissive, BSD-style license, and development is done in the open, with general discussion on the golang-dev mailing list. Changes or new features are proposed and discussed in the GitHub repository's issues, and code review is done via comments on the Gerrit code changes (called "changelists" or "CLs").

A new linker

One of the largest tooling changes in 1.15 is the completely rewritten linker. The new linker design document, authored by Go core contributor Austin Clements in September 2019, details the motivation for the rewrite and the improvements it will bring. There are three major structural changes in the new linker:

• Moving work from the linker to the compiler: this enables parallelization, as compiles are done in parallel across multiple CPUs (or machines), but the link step almost always has to be done in serial at the end of the build. Additionally, the results of the compiler are cached by the Go tooling.
• Improving key data structures, primarily by avoiding strings. The current linker uses a big symbol table indexed by string; the new design avoids strings as much as possible by using a symbol-numbering technique.
• Avoiding loading all input object files into memory at once: this makes the new linker use less memory for large programs, and allocate less memory overall (the current linker spends over 20% of its time in the garbage collector).

Now that Ken Thompson, author of the original linker, has retired, there's also the matter of maintainability. As Clements put it:

Given the sweeping long-term changes, this work is being done on a branch (dev.link) that is merged into master only at stable points. Than McIntosh, who is working on the new linker, described what has already been done for 1.15: most of the structural improvements in the design document have been completed, including the new object file format and tighter symbol representation. Builds are already faster and use less memory than in 1.14, but some features (for example, using the DWARF 5 debugging format) will have to wait for 1.16.

Clements added more detail on the parallelization efforts, as well as the gradual way the work is being phased in:

For now, the linker still converts the output back to the old in-memory representation for the last part of the linking. Presumably in a future version of Go these last steps will be moved into the new linker and the conversion phase will be removed entirely, reducing link time and memory usage further.

Smaller binaries

Related are several improvements that reduce the size of executables built with Go 1.15. As Brad Fitzpatrick showed, the new linker eliminates a lot more unused code, bringing Fitzpatrick's (rather artificial) test program down from 8.2MB in Go 1.14 to 3.9MB in 1.15. For more realistic programs, binary sizes go down by 3.5% or as much as 22%. A web server program that I run went down from 21.4MB to 20.3MB, a reduction of 4.9%.

The biggest contributors to this are the unused code elimination in the new linker, along with several targeted improvements, such as Clements's CL 230544, which reduces the number of stack and register maps included in the executable. These maps are used by Go's garbage collector (GC) to determine what objects are alive, but are now only needed at call sites, instead of for every instruction. This change reduces the size of the go binary by 5.7%; it also speeds up compiles and links by a significant amount.

Due to Go's ability to inspect types at runtime (using the reflect package), Go binaries contain a significant amount of type information. CL 231397 by Cherry Zhang only includes a symbol's type information in the output if it's converted to an interface (only values converted to an interface can be used with reflection). This change reduces the size of a hello-world program by 7.2%.

There are a few other minor improvements to binary size, such as Brad Fitzpatrick's CL 228111, which avoids including both the TLS client and server code in the output if only one of them is used, reducing the size of a TLS dial hello world program by 3.2%.

Performance improvements

Go 1.15 introduces many minor performance improvements, but two of the more notable ones are from prolific non-Google contributor Josh Bleecher Snyder. CL 216401 avoids allocating memory when converting small integers to an interface value, giving a 2% improvement in compile-to-assembly times. Converting to interface values is like "boxing" in other languages; the optimization is similar in spirit to Python's small integer caching, though it happens in Go far less often due to static typing.

The second of Snyder's changes is CL 226367 in the internals of the compiler and runtime, which allows the compiler to use more x86 registers for the garbage collector's write-barrier calls. Go uses a write barrier (kind of like a lock) to maintain data integrity on the heap when the GC is running concurrently with user code (this detailed analysis of Go's GC has more information). This results in slightly smaller binaries and a 1% improvement in compile times.

Michael Knysze significantly increased throughput of memory allocation for large blocks by redesigning the memory allocator's "mcentral" data structure to reduce lock contention. The new allocation code is more than twice as fast for blocks of 12KB or larger.

Tooling and ports

The Go "modules" feature (Go's dependency management system) was first introduced in Go 1.11, and support for a module mirror or "proxy" was added in 1.13. Version 1.15 adds support for a fallback proxy, allowing the go tool to fall back to a secondary host if the first one fails when downloading module source code. Fallbacks are specified using the GOMODCACHE environment variable's new "|" separator.

Go 1.15 removes two older ports: darwin/386 and darwin/arm, which provided 32-bit binaries on macOS and other Apple operating systems. Fitzpatrick notes that macOS Catalina doesn't support running 32-bit apps, so removing those ports will help free up macOS build machines as well as shrinking the compiler slightly. These ports were announced as deprecated in the Go 1.14 release, and will be removed in Go 1.15.

On the other hand, the linux/arm64 port was upgraded to a "first class port", which means that broken builds for linux/arm64 will block releases; official binaries as well as install documentation are provided by the Go team. As Fitzpatrick noted, Linux 64-bit Arm is now at least as important as 32-bit Arm, which is already a first-class port.

On Windows, Go 1.15 now generates executables that use address-space layout randomization (ASLR) by default. ASLR uses position-independent code to randomize the addresses of various data areas on startup, making it harder for attackers to predict target addresses and create memory-corruption exploits.

Standard library additions

Go's standard library is large and fairly stable; in Go 1.15 only relatively minor features were added.

The standard library's testing package is quite minimalist — the Go philosophy is to avoid domain-specific languages for writing tests and assertions, and instead to just write plain Go, which the developer already knows. But the core developers found creating a temporary directory useful enough to approve adding a TempDir() method that lazily creates a temporary directory for the current test and deletes it automatically when the test is finished.

The net/url package adds a new URL.Redacted() method that returns the URL as a string, but with the password redacted (replaced by xxxxx). URLs with passwords such as https://username:password@example.com/ are not usually used in browsers anymore, but are still surprisingly common in scripts and tools. Redacted() can be used to log URLs more securely, in line with RFC 3986's guidelines to not render the part after the : as clear text.

A new time/tzdata package was added to allow embedding a static copy of the time zone database in executables. Because it adds about 800KB to the executable, it's opt-in: either by importing the time/tzdata package, or by compiling with the timetzdata build tag. The embedded database can make time zone database access more consistent and reliable on some systems (particularly Windows), and it may also be useful in virtualized environments like Docker containers and the Go playground.

Parting thoughts

Go uses GitHub issues to track all bugs and feature requests, so you can scan the list of closed issues in the Go 1.15 milestone for further exploration of what's in the release. The 1.15 final release is still over 2 months away, but you can easily test your own code against the latest version using the gotip tool, or wait for the binary beta release — scheduled for June 1. Bugs found now will almost certainly be fixed before the 1.15 final release.