Write packages, not programs
How to climb a mountain from the top
All design decisions start and end with the package.
—Bill Kennedy, “Design Philosophy On Packaging”
This is the first of a two-part tutorial on designing Go packages, guided by tests:
- Write packages, not programs
- From packages to commands
This is fine
What’s wrong with this program?
package main
import (
"fmt"
)
func main() {
.Println("Hello, world")
fmt}
If you looked in vain, unable to spot the bug, I don’t blame you. The fact is, there’s nothing wrong with this program as such. It works, to the extent that it does what the author intended: print a message to the terminal and exit.
But there are some limitations on what we can do with the program. The most serious of these is that it’s not an importable package. Let’s talk about why this is a big deal.
The earliest computers were single-purpose machines: they could only execute the specific computation they were designed for. To compute something different meant physically re-wiring the machine.
It took a leap of insight to realise that a much more useful kind of computer would be one that could execute any computation, without being re-wired. That is to say, it would be programmable.
Packages are a force multiplier
A further significant advance was the idea that we don’t need to write every program from scratch every time. Instead, we could create re-usable “routines” For example, the code to calculate square roots only has to be written once, and we can then copy and use it in any program that needs to take a square root.
Nowadays, we would call such independent chunks of software components, or packages, and they’re fundamental to all modern programming.
Without packages we would always have to instruct the computer about every detail of what we want to do. The packages in the Go standard library, for example, would be very hard to manage without. Every time we wanted to write an HTTP server, for example, we’d have to implement the HTTP protocol ourselves, which is far from a trivial task (try it).
Packages, in other words, are an immensely powerful force multiplier. When we’re programming in a language like Go that has a rich ecosystem of importable packages, we never have to reinvent the wheel. If we can figure out how to break down our unsolved problems into a bunch of mini-problems that have already been solved by existing packages, we’re 90% done.
The universal Go library is huge
Go’s standard library, in particular, is an insanely great idea. The Go language is pretty small, in the sense that there aren’t many keywords and there’s not a vast amount of syntax to learn. And that’s great news for those of us learning it.
But since this little language ships with a big standard library, full of all sorts of useful and well-designed packages, we can use it to construct some really powerful programs right away. If we listen carefully enough, we can hear Go sending us a message: Packages are awesome. Let’s write more of them.
Fortunately, lots of Gophers have heard this message and acted on it, contributing their own packages to what we might call the universal library. These packages aren’t “built in” to Go in the way that the standard library packages are, but they’re nonetheless available to us. Providing their licence allows it, we can simply import them and use the functionality they provide in any way we want.
A quick search on GitHub reveals something close to half a million packages in this universal library (and there are more published in other places). If you can imagine it, in other words, there’s probably a Go package that provides it. So many, indeed, that simply finding the package you want can be a challenge. The pkg.go.dev site lets you search and browse the whole universal library, but a good place to look first is awesome-go, a carefully-curated list of a couple of thousand or so of the very best Go packages, by subject area.
This is one of the many reasons that Go is such a popular choice for developing software nowadays. In many cases, all we need to do to create a particular program is to figure out the right way to connect up the various packages that we need. We can then make our program a package that other people can use, and the process continues as a chain reaction.
Sharing our code benefits us all
No program is an island, in other words. But that’s what’s wrong with
our “hello, world” program: it is isolated, breaking the chain
of importability. The syntax rules of Go mean that all code has to be in
some package, and it happens that ours is in package
main
. But there’s something special about the
main
package in Go: it can’t be imported.
That means no one else can benefit from our wonderful code. We’re taking from the universal package ecosystem, but not giving anything back. That’s just rude. If our program is worth writing, it’s worth sharing with the millions of other Gophers who also benefit from the universal library.
Good programmers, then, are always thinking in terms of writing importable packages, not mere dead-end programs. And since packages are components, this would still be a good design idea even if we can’t contribute them to the universal library for some reason.
So what do we do differently when we’re writing packages, not programs?
Writing packages, not programs
What Bill Kennedy has aptly called package-oriented design represents a fundamental shift of mindset:
All design decisions start and end with the package. The purpose of a package is to provide a solution to a specific problem domain. To be purposeful, packages must provide, not contain. The more focused each package’s purpose is, the more clear it should be what the package provides.
—Bill Kennedy, “Design Philosophy On Packaging”
In other words, we start with some problem that we need to solve. Instead of jumping straight to a program that solves the problem, we first of all design a well-focused package that solves the problem, and then we can use it in a program.
The biggest shift in our thinking, then, is from solving our very specific and parochial problem, to solving a general class of problems that includes ours. For example, instead of writing code to calculate the square root of 2, we write a package that calculates any square root, and then we apply it to the number 2.
Both approaches produce the square root of 2, but the package approach is much more valuable because it also solves the square root problem for all developers, for all time. We can then contribute our package back to the universal library, so that everybody else can benefit from our work in the same way that we benefit every day from theirs.
Command-line tools
We can write as many packages as we want, of course, but nothing will
actually happen until we run some executable binary. That means
there must be a main
package, but we’ve already said that
the substantive code in our program should be in some
importable package, which means it can’t be in
main
.
It follows, then, that package main
should do almost
nothing except import our package and call some entrypoint function to
start the real program. We don’t really want to write any non-trivial
logic in the main
package, since it can’t be
(directly) tested. And whether it’s correct or not, it can’t be imported
and used in other programs, so it’s a dead end as far as the open-source
community is concerned.
Let’s see what would be left if we extracted all the substantive code
out of main
, then:
package main
import (
"hello"
)
func main() {
.Print()
hello}
This won’t work yet, because we haven’t written the
hello
package, but we can see how it would work.
We import hello
, so that we can use its machinery. To do
that, we call some exported function hello.Print
, which
presumably does the actual printing of “Hello, world”.
Zen mountaineering
How can we call a function that doesn’t exist? Well, it’s an exercise
in imagination. As we’re writing main
, we can ask ourselves
“What kind of function would we like to call here?” What name
would make sense for it? Would it need to take any arguments? If so,
what? Would it return any results? How many? What type? And so on.
In fact, this is a good way to design such a function—and, by extension, the whole public API of our package: by using it.
There’s a Zen saying that applies here:
If you want to climb a mountain, begin at the top.
In other words, if we want to design a package, we should begin by pretending it already exists, and writing code that uses it to solve our problem. When this code looks clear, simple, and readable, we probably have a nice design. The hard part is over: all we need to do now is actually implement that design.
We’ve done a little design work already on the hello
package, even though we haven’t written a single line of code in that
package. For example, we know there will be a Print
function that takes no parameters, returns nothing, and whose behaviour
is to print a message to the terminal.
Guided by tests
We also want a test for this behaviour. In fact, let’s try to build this package guided by tests. A good way to do this is often to write the test first, before implementing the behaviour it tests.
If that sounds strange, it shouldn’t. This is actually the way we test most things, including software engineers. We decide the problem for the coding challenge in advance, and then ask the candidates to write a program that solves it.
The other way round would be crazy: imagine asking people to submit random programs, then choosing a problem and hoping one of the programs happens to solve it. That probably wouldn’t be a helpful way of identifying good programmers.
Similarly, if you want to make good software components, the right way to do that is to decide in advance what you want the behaviour to be, express that requirement in code, and then write the component.
This not only avoids wasting time on solving the wrong problem, but it also helps us keep things focused, and avoid writing more code than we actually need. When the test starts passing, you know you’ve got it right.
Building a hello
package
Let’s try this idea with our hello
-printing example.
We’ll write a test and then see if we can come up with the right code to
pass it.
All Go tests need to be in a file whose name ends with
_test.go
, so we’ll create a new file named
hello_test.go
, and start there.
Here’s a first attempt:
package hello_test
import (
"hello"
"testing"
)
func TestPrintPrintsHelloMessageToTerminal(t *testing.T) {
.Parallel()
t.Print()
hello}
Let’s break this down, line by line. Every Go file must begin with a
package
clause defining what package its code belongs to.
Since this code will be about testing the hello
package,
let’s put it in package hello_test
.
We need to import the standard library testing
package
for tests, and we will also need our hello
package.
The naming of tests
The test itself has a name, which can convey useful information if we
choose it wisely. It’s a good idea to name each test function after the behaviour it tests. You don’t
need to use words like Should
, or Must
; these
are implicit. Just say what the function does when it’s working
correctly. In this case, it prints a hello message to the terminal.
To help us think about what our test names are saying about the
behaviour of the system, we can use the gotestdox
tool. It simply rewrites the test names as space-separated words that we
can read as a sentence:
✔ Print prints hello message to terminal
As you probably know, every Go test function takes a single
parameter, conventionally named t
, which is a pointer to a
testing.T
struct. This t
value contains the
state of the test during its execution, and we use its methods to
control the outcome of the test (for example, calling
t.Error
to fail the test).
The structure of tests
If you’re not already familiar with writing tests in Go, I recommend you read For the Love of Go, which will give you some helpful background on what follows. (If you are pretty familiar with testing in Go, try The Power of Go: Tests for a more in-depth treatment.)
The call to t.Parallel()
signals that the test should be
run concurrently with other tests, and is a standard prelude to any
test. Now here comes the substantive part, where we actually call the
function under test:
.Print() hello
It doesn’t seem, from the way we’ve used it in our modified
main
package, that the Print
function needs to
take any parameters, and at the moment there don’t seem to be any useful
results it could return.
And that’s the end of the test. But there’s a problem: when would this test fail?
I’ll let you think about this a bit, and we’ll talk about the solution in Part 2.