Mar 10

Mar 10 Generics in Go

I went to a general store, but they wouldn’t let me buy anything specific.
—Steven Wright

This is the first in a four-part series of tutorials on generics in Go.

One of the newest and most exciting features of the Go language is support for generics. This tutorial series, extracted from my book Know Go, explains what that means, why it’s useful, where we’d use it, and how it changes the way we write Golang programs. Let’s dive right in!

Types

Go generics is about types, as I’m sure you know. So we’ll start with a quick review of how types work in Go.

Specific programming

I’m sure you know that Go has data types: numbers, strings, and so on. Every variable and value in Go has some type, whether it’s a built-in type such as int, or a user-defined type such as a struct.

Go keeps track of these types, and you’ll be well aware that it won’t let you get away with any type mismatches, such as trying to assign an int value to a float64 variable.

And you’ve probably written functions in Go that take some specific type of parameter, such as a string. If we tried to pass a value of a different type, Go would complain.

So that’s specific programming, if you like: writing functions that take parameters of some specific type. And that’s the kind of programming you’re probably used to doing in Go.

Generic programming

What would generic programming be, then? It would have to be writing functions that can take either any type of parameter, or, more usefully, a set of possible types.

The generic equivalent of our PrintString function, for example, might be able to print not just a string, but any type of value. What would that look like? What kind of parameter type would we declare?

It’s tricky, because we have to put something in the function’s parameter list, and we simply don’t know what to write there yet. We will learn how generics solves this problem in a moment, but first let’s look at some other ways we could have achieved the same goal.

Interface types

Go has always had a limited kind of support for functions that can take an argument of more than one specific type, using interfaces.

You might have encountered interface types like io.Writer, for example. Here’s a function that declares a parameter w of type io.Writer:

func PrintTo(w io.Writer, msg string) {
    fmt.Fprintln(w, msg)
}

Here we don’t know what the precise type of the argument w will be at run time (its dynamic type, we say), but we (and Go) can at least say something about it. We can say that it must implement the interface io.Writer.

What does it mean to implement an interface? Well, we can look at the interface definition for clues:

type Writer interface {
    Write(p []byte) (n int, err error)
}

What this is saying is that to be an io.Writer—to implement io.Writer—is to have a particular set of methods. In this case, just one method, Write, with a particular signature (it must take a []byte parameter and return int and error).

This means that more than one type can implement io.Writer. In fact, any type that has a suitable Write method implements it automatically.

You don’t even need to explicitly declare that your type implements a certain interface. If you have the right set of methods, you implicitly implement any interface that specifies those methods.

For example, we can define some struct type of our own, and give it a Write method that does nothing at all:

type MyWriter struct {}

func (MyWriter) Write([]byte) (int, error) {
    return 0, nil
}

We now know that the presence of this Write method implicitly makes our struct type an io.Writer. So we could pass an instance of MyWriter to any function that expects an io.Writer parameter, for example.

Interface parameters

The MyWriter type may not be very useful in practice, since it doesn’t do anything. Nonetheless, any value of type MyWriter is a valid io.Writer, because it has the required Write method.

It can have other methods, too, but Go doesn’t care about that when it’s deciding whether or not a MyWriter is an io.Writer. It just needs to see a Write method with the correct signature.

This means that we can pass an instance of MyWriter to PrintTo, for example:

PrintTo(MyWriter{}, "Hello, world!")

If we tried to pass a value of some other type that doesn’t satisfy the interface, we feel like it shouldn’t work:

type BogusWriter struct{}

PrintTo(BogusWriter{}, "This won't compile!")

And indeed, we get this error:

cannot use BogusWriter{} (type BogusWriter) as type io.Writer
in argument to PrintTo:
    BogusWriter does not implement io.Writer
    (missing Write method)

That’s fair enough. A function wouldn’t declare a parameter of type io.Writer unless it knew it needed to call Write on that value. By accepting that interface type, it’s saying something about what it plans to do with the parameter: write to it!

Go can tell in advance that this won’t work with a BogusWriter, because it doesn’t have any such method. So it won’t let us pass a BogusWriter where an io.Writer is expected.

Polymorphism

What’s the point of all this, though? Why not just define the PrintTo function to take a MyWriter parameter, for example? That is to say, some concrete (non-interface) type?

Interfaces make code flexible

Well, you already know the answer to that: because more than one concrete type can be an io.Writer. There are many such types in the standard library: for example, *os.File or *bytes.Buffer. A function that takes a io.Writer can work with any of these.

Now we can see why interfaces are so useful: they let us write very flexible functions. We don’t have to write multiple versions of the function, like PrintToFile, PrintToBuffer, PrintToBuilder, and so on.

Instead, we can write one function that takes an interface parameter, io.Writer, and it’ll work with any type that implements this interface. Indeed, it works with types that don’t even exist yet! As long as it has a Write method, it’ll be acceptable to our function.

The fancy computer science term for this is polymorphism (“many forms”). But it just means we can take “many types” of value as a parameter, providing they implement some interface (that is, some set of methods) that we specify.

Constraining parameters with interfaces

Interfaces in Go are a neat way of introducing some degree of polymorphism into our programs. When we don’t care what type our parameter is, so long as we can call certain methods on it, we can use an interface to express that requirement.

It doesn’t have to be a standard library interface, such as io.Writer; we can define any interface we want.

For example, suppose we’re writing some function that takes a value and turns it into a string, by calling a String method on it. What sort of interface parameter could we take?

Well, we know we’ll be calling String on the value, so it must have at least a String method. How can we express that requirement as an interface? Like this:

type Stringer interface {
    String() string
}

In other words, any type can be a Stringer so long as it has a String method. Then we can define our Stringify function to take a parameter of this interface type:

func Stringify(s Stringer) string {
    return s.String()
}

In fact, this interface already exists in the standard library (it’s called fmt.Stringer), but you get the point. By declaring a function parameter of interface type, we can use the same code to handle multiple dynamic types.

Note that all we can require about a method using an interface is its name and signature (that is, what types it takes and returns). We can’t specify anything about what that method actually does.

Indeed, it might do nothing at all, as we saw with the MyWriter type, and that’s okay: it still implements the interface.

Limitations of method sets

This “method set” approach to constraining parameters is useful, but fairly limited. Suppose we want to write a function that adds two numbers. We might write something like this:

func AddNumbers(x, y int) int {
    return x + y
}

That’s great for int values, but what about float64? Well, we’d have to write essentially the same function again, but this time with a different parameter and result type:

func AddFloats(x, y float64) float64 {
    return x + y
}

The actual logic (x + y) is exactly the same in both cases, so the type system is hurting us more than it’s helping us here.

Indeed, we’d also have to write AddInt64s, AddInt32s, AddUints, and so on, and they’d all consist of the same code. This is boring, and it’s not the kind of thing that we became programmers to do.

So we need to think of something else. Maybe interfaces can come to our rescue?

Let’s try. Suppose we change AddNumbers to take a pair of parameters of some interface type, instead of a concrete type like int or float64.

What interface could we use? In other words, what methods would we need to specify that the parameter type must implement?

Well, here’s where we run into a limitation of interfaces defined by method sets. Actually, int has no methods, and nor do any of the other built-in types! So there’s no method set we can specify that would be implemented by int, float64, and friends.

We could still define some interface: for example, we could require an Add method, and then we could define struct types with such a method, and pass them to AddNumber. Great. But it wouldn’t allow us to use any of Go’s built-in number types, and that would be a most inconvenient limitation.

The empty interface: `any`

Here’s another idea. What about the empty interface, named any? That specifies no methods at all, so literally every concrete type implements it.

Could we use any as the type of our parameters? Since that would allow us to pass arguments of any type at all, we might rename our function AddAnything:

// invalid
func AddAnything(x, y any) any {
    return x + y
}

Unfortunately, this doesn’t compile:

invalid operation: x + y (operator + not defined on interface)

The problem here is what we tried to do with the parameters: that is, add them. To do that, we used the + operator, and that’s not allowed here.

Why not? Because we said, in effect, that x and y can be any type, and not every type works with the + operator.

If x and y were instances of some kind of struct, for example, what would it even mean to add them together? There’s no way to know, so Go plays it safe by disallowing the + operation altogether.

And there’s another, subtler problem here, too. We presumably need x and y to be the same concrete type, whatever it is. But because they’re both declared as any, we can call this function with different concrete types for x and y, which almost certainly wouldn’t make sense.

Type assertions and switches

You probably know that we can write a type assertion in Go, to detect what concrete type an interface value contains. And there’s a more elaborate construct called a type switch, which lets us detect a whole set of possible types, like this:

switch v := x.(type) {
case int:
    return v + y
case float64:
    return v + y
case ...

Using a switch statement like this, we can list all the concrete types that we know do support the + operator, and use it with each of them.

This seems promising at first, but really we’re just right back where we started! We wanted to avoid writing a separate, essentially identical version of the Add function for each concrete type, but here we are doing basically just that. So an interface is no use here.

In practice, we don’t often need to write functions like AddAnything, which is just as well. But this is an awkward limitation of Go, or so it would seem: it makes it difficult for us to write general-purpose packages, among other things.

Look at the math package in the standard library, for example. It provides lots of useful utility functions such as Pow, Abs, and Max… but only on float64 values.

If you want to use those functions with some other type, you’ll have to explicitly convert it to float64 on the way in, and back to your preferred type on the way out. That’s just lame.

Go, meet generics

This isn’t full generic programming, then, in the way that we now understand the term. We can’t write the equivalent of an AddAnything function using just method-based interfaces, even the empty interface.

Or rather, we can write functions that take values of any type: we just can’t do anything useful with those values, like add them together.

How it started

At least, that was true until recently, but now there is a way to do this kind of thing. We can use Go generics!

We’ll see more in the next post what that actually involves, but first let’s take a very brief trip through the history of Go to see how we got where we are today.

Go was released on November 10, 2009. Less than 24 hours later we saw the first comment about generics.
—Ian Lance Taylor, “Why Generics?”

Go was deliberately designed to be a very simple language, and also to make it easy to compile and build Go programs very fast, without using a lot of resources. That means it doesn’t have everything, and generics is one of the features Go didn’t have at launch.

Why didn’t the designers just add generics later, then? Well, there are a couple of compelling reasons.

How it’s going

Go was intended to be quick to learn, without a lot of syntax and keywords to master before you can be productive with the language. Every new thing you add to it is something else that beginners will have to learn.

The Go team also puts a great value on backwards compatibility: that is to say, no breaking changes can be introduced to the language. So if you introduce some new syntax, it has to be done in a way that doesn’t conflict with any possible existing programs. That’s hard!

Various proposals for generics in Go have been made over the years, in fact, but most of them fell at one or another of these hurdles. Some involved an unacceptable hit to compiler performance, or to runtime performance; others introduced too much complexity or weren’t backwards compatible with existing code.

That’s why it took about ten years for generics to finally land in Go. What we have, after all that thinking and arguing, is actually a very nice design. Like Go itself, it does a lot with a little.

With the absolute minimum of new syntax, the Go team have opened up a whole new world of programming, and enabled us to write new kinds of programs in Go. It’ll take a while for the consequences of all this to feed through into the mainstream, and for most people, even for experienced Gophers, generics are something very new.

That’s why I wrote a whole book on the subject. Know Go is a complete guide to generics in Go and how to use generic functions and types to build more powerful and flexible packages.

In the next post, we’ll talk about exactly what it is that’s been added to Go, and start writing some generic programs of our own.

Next: Type parameters