diff options
| -rw-r--r-- | _drafts/program-structure-and-composability.md | 161 |
1 files changed, 150 insertions, 11 deletions
diff --git a/_drafts/program-structure-and-composability.md b/_drafts/program-structure-and-composability.md index 96baee8..7968405 100644 --- a/_drafts/program-structure-and-composability.md +++ b/_drafts/program-structure-and-composability.md @@ -6,6 +6,9 @@ description: >- complex structures, and a pattern which helps in solving those problems. --- +TODO: +* Double check if I'm using "I" or "We" everywhere (probably should use "I") + ## Part 0: Introduction This post is focused on a concept I call "program structure", which I will try @@ -113,9 +116,9 @@ looks something like: ```go // A mapping of connection names to redis connections. -var globalConns = map[string]redisConnection +var globalConns = map[string]*RedisConn{} -func Get(name string) redisConnection { +func Get(name string) *RedisConn { if globalConns[name] == nil { globalConns[name] = makeConnection(name) } @@ -155,7 +158,7 @@ breaking compartmentalization. The person/team responsible for the central library often finds themselves as the maintainers of the shared resource as well, rather than the team actually using it. -### Program Structure +### Component Structure So what does proper program structure look like? In my mind the structure of a program is a hierarchy of components, or, in other words, a tree. The leaf nodes @@ -179,19 +182,19 @@ TODO diagram: http ``` -This structure contains the addition of the `debug` component. Clearly the -`http` and `redis` components are reusable in different contexts, but for this -example the `debug` endpoint is as well. It creates a separate http server which -can be queried to perform runtime debugging of the program, and can be tacked -onto virtually any program. The `rest-api` component is specific to this program -and therefore not reusable. Let's dive into it a bit to see how it might be -implemented: +This component structure contains the addition of the `debug` component. Clearly +the `http` and `redis` components are reusable in different contexts, but for +this example the `debug` endpoint is as well. It creates a separate http server +which can be queried to perform runtime debugging of the program, and can be +tacked onto virtually any program. The `rest-api` component is specific to this +program and therefore not reusable. Let's dive into it a bit to see how it might +be implemented: ```go // RestAPI is very much not thread-safe, hopefully it doesn't have to handle // more than one request at once. type RestAPI struct { - redisConn *redis.Conn + redisConn *redis.RedisConn httpSrv *http.Server // Statistics exported for other components to see @@ -265,4 +268,140 @@ discussed in the next section. ## Part 2: Context, Configuration, and Runtime +The key to the configuration problem is to recognize that, even if there are two +of the same component in a program, they can't occupy the same place in the +program's structure. In the above example there are two `http` components, one +under `rest-api` and the other under `debug`. Since the structure is represented +as a tree of components, the "path" of any node in the tree uniquely represents +it in the structure. For example, the two `http` components in the previous +example have these paths: + +``` +root -> rest-api -> http +root -> debug -> http +``` + +If each component were to know its place in the component tree, then it would +easily be able to ensure that its configuration and initialization didn't +conflict with other components of the same type. If the `http` component sets up +a command-line parameter to know what address to listen on, the two `http` +components in that program would set up: + +``` +--rest-api-listen-addr +--debug-listen-addr +``` + +So how can we enable each component to know its path in the component structure? +To answer this we'll have to take a detour through go's `Context` type. + +### Context and Configuration + +As I mentioned in the Introduction, my example language in this post is Go, but +there's nothing about the concepts I'm presenting which are specific to Go. To +put it simply, Go's builtin `context` package implements a type called +`context.Context` which is, for all intents and purposes, an immutable key/value +store. This means that when you set a key to a value on a Context (using the +`context.WithValue` function) a new Context is returned. The new Context +contains all of the original's key/values, plus the one just set. The original +remains untouched. + +(Go's Context also has some behavior built into it surrounding deadlines and +process cancellation, but those aren't relevant for this discussion.) + +Context makes sense to use for carrying information about the program's +structure to it's different components; it is informing each of what _context_ +it exists in within the larger structure. To use Context effectively, however, +it is necessary to implement some helper functions. Here are their function +signatures: + +```go +// NewChild creates and returns a new Context based off of the parent one. The +// child will have a path which is the parent's path appended with the given +// name. +func NewChild(parent context.Context, name string) context.Context + +// Path returns the sequence of names which were used to produce this Context +// via calls to the NewChild function. +func Path(ctx context.Context) []string +``` + +`NewChild` is used to create a new Context, corresponding to a new child node in +the component structure, and `Path` is used retrieve the path of any Context +within that structure. For the sake of keeping the examples simple let's pretend +these functions have been implemented in a package called `mctx`. Here's an +example of how `mctx` might be used in the `redis` component's code: + +```go +func NewRedis(ctx context.Context, defaultAddr string) *RedisConn { + ctx = mctx.NewChild(ctx, "redis") + ctxPath := mctx.Path(ctx) + paramPrefix := strings.Join(ctxPath, "-") + + addrParam := flag.String(paramPrefix+"-addr", defaultAddr, "Address of redis instance to connect to") + // finish setup + + return redisConn +} +``` + +In our above example, the two `redis` components' parameters would be: + +``` +// This first parameter is for stats redis, whose parent is the root and +// therefore doesn't have a prefix. Perhaps stats should be broken into its own +// component in order to fix this. +--redis-addr +--rest-api-redis-addr +``` + +The prefix joining stuff will probably get annoying after a while though, so +let's invent a new package, `mcfg`, which acts like `flag` but is aware of +`mctx`. Then `NewRedis` is reduced to: + +```go +func NewRedis(ctx context.Context, defaultAddr string) *RedisConn { + ctx = mctx.NewChild(ctx, "redis") + addrParam := flag.String(ctx, "-addr", defaultAddr, "Address of redis instance to connect to") + // finish setup + + return redisConn +} +``` + +Sharp-eyed gophers will notice that there's a key piece missing: When is +`mcfg.Parse` called? When does `addrParam` actually get populated? Because you +can't create the redis connection until that happens, but that can't happen +inside `NewRedis` because there might be other things after `NewRedis` which +want to set up parameters. To illustrate the problem, let's look at a simple +program which wants to set up two `redis` components: + +```go +func main() { + // Create the root context, and empty Context. + ctx := context.Background() + + // Create the Contexts for two sub-components, foo and bar. + ctxFoo := mctx.NewChild(ctx, "foo") + ctxBar := mctx.NewChild(ctx, "bar") + + // Now we want to try to create a redis instances for each component. But... + + // This will set up the parameter "--foo-redis-addr", but bar hasn't had a + // chance to set up its corresponding parameter, so the command-line can't + // be parsed yet. + fooRedis := redis.NewRedis(ctxFoo, "127.0.0.1:6379") + + // This will set up the parameter "--bar-redis-addr", but, as mentioned + // before, NewRedis can't parse command-line. + barRedis := redis.NewRedis(ctxBar, "127.0.0.1:6379") + + // If the command-line is parsed here, then how can fooRedis and barRedis + // have been created yet? Creating the redis connection depends on the addr + // parameters having already been parsed and filled. +} +``` + +We will solve this problem in the next section. +## Init vs. Start |