summaryrefslogtreecommitdiff
path: root/_posts/2015-07-15-go-http.md
blob: 334d68a56a895a71428261d9a9fbc8b8822bc252 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
---
title: Go's http package by example
---

Go's [http](http://golang.org/pkg/net/http/) package has turned into one of my
favorite things about the Go programming language. Initially it appears to be
somewhat complex, but in reality it can be broken down into a couple of simple
components that are extremely flexible in how they can be used. This guide will
cover the basic ideas behind the http package, as well as examples in using,
testing, and composing apps built with it.

This guide assumes you have some basic knowledge of what an interface in Go is,
and some idea of how HTTP works and what it can do.

## Handler

The building block of the entire http package is the `http.Handler` interface,
which is defined as follows:

```go
type Handler interface {
	ServeHTTP(ResponseWriter, *Request)
}
```

Once implemented the `http.Handler` can be passed to `http.ListenAndServe`,
which will call the `ServeHTTP` method on every incoming request.

`http.Request` contains all relevant information about an incoming http request
which is being served by your `http.Handler`.

The `http.ResponseWriter` is the interface through which you can respond to the
request. It implements the `io.Writer` interface, so you can use methods like
`fmt.Fprintf` to write a formatted string as the response body, or ones like
`io.Copy` to write out the contents of a file (or any other `io.Reader`). The
response code can be set before you begin writing data using the `WriteHeader`
method.

Here's an example of an extremely simple http server:

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

type helloHandler struct{}

func (h helloHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "hello, you've hit %s\n", r.URL.Path)
}

func main() {
	err := http.ListenAndServe(":9999", helloHandler{})
	log.Fatal(err)
}
```

`http.ListenAndServe` serves requests using the handler, listening on the given
address:port. It will block unless it encounters an error listening, in which
case we `log.Fatal`.

Here's an example of using this handler with curl:

```
 ~ $ curl localhost:9999/foo/bar
 hello, you've hit /foo/bar
```


## HandlerFunc

Often defining a full type to implement the `http.Handler` interface is a bit
overkill, especially for extremely simple `ServeHTTP` functions like the one
above. The `http` package provides a helper function, `http.HandlerFunc`, which
wraps a function which has the signature
`func(w http.ResponseWriter, r *http.Request)`, returning an `http.Handler`
which will call it in all cases.

The following behaves exactly like the previous example, but uses
`http.HandlerFunc` instead of defining a new type.

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "hello, you've hit %s\n", r.URL.Path)
	})

	err := http.ListenAndServe(":9999", h)
	log.Fatal(err)
}
```

## ServeMux

On their own, the previous examples don't seem all that useful. If we wanted to
have different behavior for different endpoints we would end up with having to
parse path strings as well as numerous `if` or `switch` statements.  Luckily
we're provided with `http.ServeMux`, which does all of that for us.  Here's an
example of it being used:

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	h := http.NewServeMux()

	h.HandleFunc("/foo", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintln(w, "Hello, you hit foo!")
	})

	h.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintln(w, "Hello, you hit bar!")
	})

	h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "You're lost, go home")
	})

	err := http.ListenAndServe(":9999", h)
	log.Fatal(err)
}
```

The `http.ServeMux` is itself an `http.Handler`, so it can be passed into
`http.ListenAndServe`. When it receives a request it will check if the request's
path is prefixed by any of its known paths, choosing the longest prefix match it
can find. We use the `/` endpoint as a catch-all to catch any requests to
unknown endpoints. Here's some examples of it being used:

```
 ~ $ curl localhost:9999/foo
Hello, you hit foo!

 ~ $ curl localhost:9999/bar
Hello, you hit bar!

 ~ $ curl localhost:9999/baz
You're lost, go home
```

`http.ServeMux` has both `Handle` and `HandleFunc` methods. These do the same
thing, except that `Handle` takes in an `http.Handler` while `HandleFunc` merely
takes in a function, implicitly wrapping it just as `http.HandlerFunc` does.

### Other muxes

There are numerous replacements for `http.ServeMux` like
[gorilla/mux](http://www.gorillatoolkit.org/pkg/mux) which give you things like
automatically pulling variables out of paths, easily asserting what http methods
are allowed on an endpoint, and more. Most of these replacements will implement
`http.Handler` like `http.ServeMux` does, and accept `http.Handler`s as
arguments, and so are easy to use in conjunction with the rest of the things
I'm going to talk about in this post.

## Composability

When I say that the `http` package is composable I mean that it is very easy to
create re-usable pieces of code and glue them together into a new working
application. The `http.Handler` interface is the way all pieces communicate with
each other. Here's an example of where we use the same `http.Handler` to handle
multiple endpoints, each slightly differently:

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

type numberDumper int

func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Here's your number: %d\n", n)
}

func main() {
	h := http.NewServeMux()

	h.Handle("/one", numberDumper(1))
	h.Handle("/two", numberDumper(2))
	h.Handle("/three", numberDumper(3))
	h.Handle("/four", numberDumper(4))
	h.Handle("/five", numberDumper(5))

	h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "That's not a supported number!")
	})

	err := http.ListenAndServe(":9999", h)
	log.Fatal(err)
}
```

`numberDumper` implements `http.Handler`, and can be passed into the
`http.ServeMux` multiple times to serve multiple endpoints. Here's it in action:

```
 ~ $ curl localhost:9999/one
Here's your number: 1
 ~ $ curl localhost:9999/five
Here's your number: 5
 ~ $ curl localhost:9999/bazillion
That's not a supported number!
```

## Testing

Testing http endpoints is extremely easy in Go, and doesn't even require you to
actually listen on any ports! The `httptest` package provides a few handy
utilities, including `NewRecorder` which implements `http.ResponseWriter` and
allows you to effectively make an http request by calling `ServeHTTP` directly.
Here's an example of a test for our previously implemented `numberDumper`,
commented with what exactly is happening:

```go
package main

import (
	"fmt"
	"net/http"
	"net/http/httptest"
	. "testing"
)

func TestNumberDumper(t *T) {
	// We first create the http.Handler we wish to test
	n := numberDumper(1)

	// We create an http.Request object to test with. The http.Request is
	// totally customizable in every way that a real-life http request is, so
	// even the most intricate behavior can be tested
	r, _ := http.NewRequest("GET", "/one", nil)

	// httptest.Recorder implements the http.ResponseWriter interface, and as
	// such can be passed into ServeHTTP to receive the response. It will act as
	// if all data being given to it is being sent to a real client, when in
	// reality it's being buffered for later observation
	w := httptest.NewRecorder()

	// Pass in our httptest.Recorder and http.Request to our numberDumper. At
	// this point the numberDumper will act just as if it was responding to a
	// real request
	n.ServeHTTP(w, r)

	// httptest.Recorder gives a number of fields and methods which can be used
	// to observe the response made to our request. Here we check the response
	// code
	if w.Code != 200 {
		t.Fatalf("wrong code returned: %d", w.Code)
	}

	// We can also get the full body out of the httptest.Recorder, and check
	// that its contents are what we expect
	body := w.Body.String()
	if body != fmt.Sprintf("Here's your number: 1\n") {
		t.Fatalf("wrong body returned: %s", body)
	}

}
```

In this way it's easy to create tests for your individual components that you
are using to build your application, keeping the tests near to the functionality
they're testing.

Note: if you ever do need to spin up a test server in your tests, `httptest`
also provides a way to create a server listening on a random open port for use
in tests as well.

## Middleware

Serving endpoints is nice, but often there's functionality you need to run for
*every* request before the actual endpoint's handler is run. For example, access
logging. A middleware component is one which implements `http.Handler`, but will
actually pass the request off to another `http.Handler` after doing some set of
actions. The `http.ServeMux` we looked at earlier is actually an example of
middleware, since it passes the request off to another `http.Handler` for actual
processing. Here's an example of our previous example with some logging
middleware:

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

type numberDumper int

func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Here's your number: %d\n", n)
}

func logger(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		log.Printf("%s requested %s", r.RemoteAddr, r.URL)
		h.ServeHTTP(w, r)
	})
}

func main() {
	h := http.NewServeMux()

	h.Handle("/one", numberDumper(1))
	h.Handle("/two", numberDumper(2))
	h.Handle("/three", numberDumper(3))
	h.Handle("/four", numberDumper(4))
	h.Handle("/five", numberDumper(5))

	h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "That's not a supported number!")
	})

	hl := logger(h)

	err := http.ListenAndServe(":9999", hl)
	log.Fatal(err)
}
```

`logger` is a function which takes in an `http.Handler` called `h`, and returns
a new `http.Handler` which, when called, will log the request it was called with
and then pass off its arguments to `h`. To use it we pass in our
`http.ServeMux`, so all incoming requests will first be handled by the logging
middleware before being passed to the `http.ServeMux`.

Here's an example log entry which is output when the `/five` endpoint is hit:

```
2015/06/30 20:15:41 [::1]:34688 requested /five
```

## Middleware chaining

Being able to chain middleware together is an incredibly useful ability which we
get almost for free, as long as we use the signature
`func(http.Handler) http.Handler`. A middleware component returns the same type
which is passed into it, so simply passing the output of one middleware
component into the other is sufficient.

However, more complex behavior with middleware can be tricky. For instance, what
if you want a piece of middleware which takes in a parameter upon creation?
Here's an example of just that, with a piece of middleware which will set a
header and its value for all requests:

```go
package main

import (
	"fmt"
	"log"
	"net/http"
)

type numberDumper int

func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Here's your number: %d\n", n)
}

func logger(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		log.Printf("%s requested %s", r.RemoteAddr, r.URL)
		h.ServeHTTP(w, r)
	})
}

type headerSetter struct {
	key, val string
	handler  http.Handler
}

func (hs headerSetter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	w.Header().Set(hs.key, hs.val)
	hs.handler.ServeHTTP(w, r)
}

func newHeaderSetter(key, val string) func(http.Handler) http.Handler {
	return func(h http.Handler) http.Handler {
		return headerSetter{key, val, h}
	}
}

func main() {
	h := http.NewServeMux()

	h.Handle("/one", numberDumper(1))
	h.Handle("/two", numberDumper(2))
	h.Handle("/three", numberDumper(3))
	h.Handle("/four", numberDumper(4))
	h.Handle("/five", numberDumper(5))

	h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "That's not a supported number!")
	})

	hl := logger(h)
	hhs := newHeaderSetter("X-FOO", "BAR")(hl)

	err := http.ListenAndServe(":9999", hhs)
	log.Fatal(err)
}
```

And here's the curl output:

```
 ~ $ curl -i localhost:9999/three
 HTTP/1.1 200 OK
 X-Foo: BAR
 Date: Wed, 01 Jul 2015 00:39:48 GMT
 Content-Length: 22
 Content-Type: text/plain; charset=utf-8

 Here's your number: 3

```

`newHeaderSetter` returns a function which accepts and returns an
`http.Handler`. Calling that returned function with an `http.Handler` then gets
you an `http.Handler` which will set the header given to `newHeaderSetter`
before continuing on to the given `http.Handler`.

This may seem like a strange way of organizing this; for this example the
signature for `newHeaderSetter` could very well have looked like this:

```
func newHeaderSetter(key, val string, h http.Handler) http.Handler
```

And that implementation would have worked fine. But it would have been more
difficult to compose going forward. In the next section I'll show what I mean.

## Composing middleware with alice

[Alice](https://github.com/justinas/alice) is a very simple and convenient
helper for working with middleware using the function signature we've been using
thusfar. Alice is used to create and use chains of middleware. Chains can even
be appended to each other, giving even further flexibility. Here's our previous
example with a couple more headers being set, but also using alice to manage the
added complexity.

```go
package main

import (
	"fmt"
	"log"
	"net/http"

	"github.com/justinas/alice"
)

type numberDumper int

func (n numberDumper) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Here's your number: %d\n", n)
}

func logger(h http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		log.Printf("%s requested %s", r.RemoteAddr, r.URL)
		h.ServeHTTP(w, r)
	})
}

type headerSetter struct {
	key, val string
	handler  http.Handler
}

func (hs headerSetter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	w.Header().Set(hs.key, hs.val)
	hs.handler.ServeHTTP(w, r)
}

func newHeaderSetter(key, val string) func(http.Handler) http.Handler {
	return func(h http.Handler) http.Handler {
		return headerSetter{key, val, h}
	}
}

func main() {
	h := http.NewServeMux()

	h.Handle("/one", numberDumper(1))
	h.Handle("/two", numberDumper(2))
	h.Handle("/three", numberDumper(3))
	h.Handle("/four", numberDumper(4))

	fiveHS := newHeaderSetter("X-FIVE", "the best number")
	h.Handle("/five", fiveHS(numberDumper(5)))

	h.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(404)
		fmt.Fprintln(w, "That's not a supported number!")
	})

	chain := alice.New(
		newHeaderSetter("X-FOO", "BAR"),
		newHeaderSetter("X-BAZ", "BUZ"),
		logger,
	).Then(h)

	err := http.ListenAndServe(":9999", chain)
	log.Fatal(err)
}
```

In this example all requests will have the headers `X-FOO` and `X-BAZ` set, but
the `/five` endpoint will *also* have the `X-FIVE` header set.

## Fin

Starting with a simple idea of an interface, the `http` package allows us to
create for ourselves an incredibly useful and flexible (yet still rather simple)
ecosystem for building web apps with re-usable components, all without breaking
our static checks.