program structure: edits from kelly, thanks kelly!

1 files changed, 141 insertions, 138 deletions

diff --git a/_posts/2019-08-02-program-structure-and-composability.md b/_posts/2019-08-02-program-structure-and-composability.md
index 54b0b76..bf13645 100644
--- a/_posts/2019-08-02-program-structure-and-composability.md
+++ b/_posts/2019-08-02-program-structure-and-composability.md
@@ -3,20 +3,20 @@ title: >-
     Program Structure and Composability
 description: >-
     Discussing the nature of program structure, the problems presented by
-    complex structures, and a pattern which helps in solving those problems.
+    complex structures, and a pattern that helps in solving those problems.
 hide: true
 ---
 
 ## Part 0: Introduction
 
-This post is focused on a concept I call "program structure", which I will try
-to shed some light on before discussing complex program structures, then
-discussing why complex structures can be problematic to deal with, and finally
-discussing a pattern for dealing with those problems.
+This post is focused on a concept I call “program structure,” which I will try
+to shed some light on before discussing complex program structures. I will then
+discuss why complex structures can be problematic to deal with, and will finally
+discuss a pattern for dealing with those problems.
 
 My background is as a backend engineer working on large projects that have had
-many moving parts; most had multiple programs interacting with each other, using
-many different databases in various contexts, and facing large amounts of load
+many moving parts; most had multiple programs interacting with each other, used
+many different databases in various contexts, and faced large amounts of load
 from millions of users. Most of this post will be framed from my perspective,
 and will present problems in the way I have experienced them. I believe,
 however, that the concepts and problems I discuss here are applicable to many
@@ -24,11 +24,11 @@ other domains, and I hope those with a foot in both backend systems and a second
 domain can help to translate the ideas between the two.
 
 Also note that I will be using Go as my example language, but none of the
-concepts discussed here are specific to Go. To that end, I've decided to favor
-readable code over "correct" code, and so have elided things that most gophers
+concepts discussed here are specific to Go. To that end, I’ve decided to favor
+readable code over “correct” code, and so have elided things that most gophers
 hold near-and-dear, such as error checking and proper documentation, in order to
 make the code as accessible as possible to non-gophers as well. As with before,
-I trust someone with a foot in Go and another language can translate help me
+I trust that someone with a foot in Go and another language can help me
 translate between the two.
 
 ## Part 1: Program Structure
@@ -40,8 +40,8 @@ think about program structure.
 
 ### Directory Structure
 
-For a long time I thought about program structure in terms of the hierarchy
-present in the filesystem. In my mind, a program's structure looked like this:
+For a long time, I thought about program structure in terms of the hierarchy
+present in the filesystem. In my mind, a program’s structure looked like this:
 
 ```
 // The directory structure of a project called gobdns.
@@ -56,11 +56,11 @@ src/
     main.go
 ```
 
-What I grew to learn was that this conflation of "program structure" with
-"directory structure" is ultimately unhelpful. While can't be denied that every
-program has a directory structure (and if not, it ought to), this does not mean
-that the way the program looks in a filesystem in any way corresponds to how it
-looks in our mind's eye.
+What I grew to learn was that this conflation of “program structure” with
+“directory structure” is ultimately unhelpful. While it can’t be denied that
+every program has a directory structure (and if not, it ought to), this does not
+mean that the way the program looks in a filesystem in any way corresponds to
+how it looks in our mind’s eye.
 
 The most notable way to show this is to consider a library package. Here is the
 structure of a simple web-app which uses redis (my favorite database) as a
@@ -73,13 +73,13 @@ src/
     main.go
 ```
 
-If I were to ask you, based on that directory strucure, what the program does,
-in the most abstract terms, you might say something like: "The program
-establishes an http server which listens for requests. It also establishes a
+If I were to ask you, based on that directory structure, what the program does
+in the most abstract terms, you might say something like: “The program
+establishes an http server that listens for requests. It also establishes a
 connection to the redis server. The program then interacts with redis in
-different ways, based on the http requests which are received on the server."
+different ways based on the http requests that are received on the server.”
 
-And that would be a good guess. Here's a diagram which depicts the program
+And that would be a good guess. Here’s a diagram that depicts the program
 structure, wherein the root node, `main.go`, takes in requests from `http` and
 processes them using `redis`.
 
@@ -88,15 +88,15 @@ processes them using `redis`.
     descr="Example 1"
     %}
 
-This is certainly a viable guess for how a program with that directory structure
-operates, but consider another answer: "A component of the program called
-`server` establishes an http server which listens for requests. `server` also
-establishes a connection to a redis server. `server` then interacts with that
-redis connection in different ways, based on the http requests which are
+This is certainly a viable guess for how a program with that directory
+structure operates, but consider another answer: “A component of the program
+called `server` establishes an http server that listens for requests. `server`
+also establishes a connection to a redis server. `server` then interacts with
+that redis connection in different ways based on the http requests that are
 received on the http server. Additionally, `server` tracks statistics about
 these interactions and makes them available to other components. The root
 component of the program establishes a connection to a second redis server, and
-stores those statistics in that redis server." Here's another diagram to depict
+stores those statistics in that redis server.” Here’s another diagram to depict
 _that_ program.
 
 {% include image.html
@@ -105,19 +105,20 @@ _that_ program.
     %}
 
 The directory structure could apply to either description; `redis` is just a
-library which allows for interacting with a redis server, but it doesn't specify
-_which_ server, or _how many_. And those are extremely important factors which
-are definitely reflected in our concept of the program's structure, and yet not
-in the directory structure. **What the directory structure reflects are the
-different _kinds_ of components available to use, but it does not reflect how a
-program will use those components.**
+library which allows for interaction with a redis server, but it doesn’t
+specify _which_ or _how many_ servers. However, those are extremely important
+factors that are definitely reflected in our concept of the program’s
+structure, and not in the directory structure. **What the directory structure
+reflects are the different _kinds_ of components available to use, but it does
+not reflect how a program will use those components.**
+
 
 ### Global State vs Compartmentalization
 
 The directory-centric view of structure often leads to the use of global
 singletons to manage access to external resources like RPC servers and
 databases. In examples 1 and 2 the `redis` library might contain code which
-looks something like:
+looks something like this:
 
 ```go
 // A mapping of connection names to redis connections.
@@ -132,45 +133,45 @@ func Get(name string) *RedisConn {
 ```
 
 Even though this pattern would work, it breaks with our conception of the
-program structure in more complexes cases like example 2. Rather than the
-`redis` component being owned by the `server` component, which actually uses it,
-it would be practically owned by _all_ components, since all are able to use it.
+program structure in more complex cases like example 2. Rather than the `redis`
+component being owned by the `server` component, which actually uses it, it
+would be practically owned by _all_ components, since all are able to use it.
 Compartmentalization has been broken, and can only be held together through
 sheer human discipline.
 
-**This is the problem with all global state. It's shareable amongst all components
-of a program, and so is accountable to none of them.** One must look at an
-entire codebase to understand how a globally held component is used, which might
-not even be possible for a large codebase. And so the maintainers of these
-shared components rely entirely on the discipline of their fellow coders when
-making changes, usually discovering where that discipline broke down once the
-changes have been pushed live.
+**This is the problem with all global state. It is shareable among all
+components of a program, and so is accountable to none of them.** One must look
+at an entire codebase to understand how a globally held component is used,
+which might not even be possible for a large codebase. Therefore, the
+maintainers of these shared components rely entirely on the discipline of their
+fellow coders when making changes, usually discovering where that discipline
+broke down once the changes have been pushed live.
 
 Global state also makes it easier for disparate programs/components to share
 datastores for completely unrelated tasks. In example 2, rather than creating a
-new redis instance for the root component's statistics storage, the coder might
-have instead said "well, there's already a redis instance available, I'll just
-use that." And so compartmentalization would have been broken further. Perhaps
-the two instances _could_ be coalesced into the same one, for the sake of
+new redis instance for the root component’s statistics storage, the coder might
+have instead said, “well, there’s already a redis instance available, I’ll just
+use that.” And so, compartmentalization would have been broken further. Perhaps
+the two instances _could_ be coalesced into the same instance for the sake of
 resource efficiency, but that decision would be better made at runtime via the
 configuration of the program, rather than being hardcoded into the code.
 
 From the perspective of team management, global state-based patterns do nothing
 except slow teams down. The person/team responsible for maintaining the central
-library in which shared components live (`redis`, in the above examples) becomes
-the bottleneck for creating new instances for new components, which will further
-lead to re-using existing instances rather than creating new ones, further
-breaking compartmentalization. The person/team responsible for the central
-library often finds themselves as the maintainers of the shared resource as
-well, rather than the team actually using it.
+library in which shared components live (`redis`, in the above examples)
+becomes the bottleneck for creating new instances for new components, which
+will further lead to re-using existing instances rather than creating new ones,
+further breaking compartmentalization. Additionally the person/team responsible
+for the central library, rather than the team using it, often finds themselves
+as the maintainers of the shared resource.
 
 ### Component Structure
 
 So what does proper program structure look like? In my mind the structure of a
-program is a hierarchy of components, or, in other words, a tree. The leaf nodes
-of the tree are almost _always_ IO related components, e.g. database
-connections, RPC server frameworks or clients, message queue consumers, etc...
-The non-leaf nodes will _generally_ be components which bring together the
+program is a hierarchy of components, or, in other words, a tree. The leaf
+nodes of the tree are almost _always_ IO related components, e.g., database
+connections, RPC server frameworks or clients, message queue consumers, etc.
+The non-leaf nodes will _generally_ be components that bring together the
 functionalities of their children in some useful way, though they may also have
 some IO functionality of their own.
 
@@ -182,13 +183,13 @@ Let's look at an even more complex structure, still only using the `redis` and
     descr="Example 3"
     %}
 
-This component structure contains the addition of the `debug` component. Clearly
-the `http` and `redis` components are reusable in different contexts, but for
-this example the `debug` endpoint is as well. It creates a separate http server
-which can be queried to perform runtime debugging of the program, and can be
-tacked onto virtually any program. The `rest-api` component is specific to this
-program and therefore not reusable. Let's dive into it a bit to see how it might
-be implemented:
+This component structure contains the addition of the `debug` component.
+Clearly the `http` and `redis` components are reusable in different contexts,
+but for this example the `debug` endpoint is as well. It creates a separate
+http server that can be queried to perform runtime debugging of the program,
+and can be tacked onto virtually any program. The `rest-api` component is
+specific to this program and is therefore not reusable. Let’s dive into it a
+bit to see how it might be implemented:
 
 ```go
 // RestAPI is very much not thread-safe, hopefully it doesn't have to handle
@@ -232,8 +233,9 @@ func (r *RestAPI) barHandler(rw http.ResponseWriter, r *http.Request) {
 }
 ```
 
-As can be seen, `rest-api` coalesces `http` and `redis` into a simple REST-like
-api, using pre-made library components. `main.go`, the root component, does much
+
+In that snippet `rest-api` coalesced `http` and `redis` into a simple REST-like
+api using pre-made library components. `main.go`, the root component, does much
 the same:
 
 ```go
@@ -255,26 +257,26 @@ func main() {
 }
 ```
 
-One thing which is clearly missing in this program is proper configuration,
-whether from command-line, environment variables, etc.... As it stands, all
-configuration parameters, such as the redis addresses and http listen addresses,
-are hardcoded. Proper configuration actually ends up being somewhat difficult,
-as the ideal case would be for each component to set up its own configuration
-variables, without its parent needing to be aware. For example, `redis` could
-set up `addr` and `pool-size` parameters. The problem is that there are two
-`redis` components in the program, and their parameters would therefore conflict
-with each other. An elegant solution to this problem is discussed in the next
-section.
+One thing that is clearly missing in this program is proper configuration,
+whether from command-line or environment variables, etc. As it stands, all
+configuration parameters, such as the redis addresses and http listen
+addresses, are hardcoded. Proper configuration actually ends up being somewhat
+difficult, as the ideal case would be for each component to set up its own
+configuration variables without its parent needing to be aware. For example,
+`redis` could set up `addr` and `pool-size` parameters. The problem is that there
+are two `redis` components in the program, and their parameters would therefore
+conflict with each other. An elegant solution to this problem is discussed in
+the next section.
 
 ## Part 2: Components, Configuration, and Runtime
 
-The key to the configuration problem is to recognize that, even if there are two
-of the same component in a program, they can't occupy the same place in the
-program's structure. In the above example there are two `http` components, one
-under `rest-api` and the other under `debug`. Since the structure is represented
-as a tree of components, the "path" of any node in the tree uniquely represents
-it in the structure. For example, the two `http` components in the previous
-example have these paths:
+The key to the configuration problem is to recognize that, even if there are
+two of the same component in a program, they can’t occupy the same place in the
+program’s structure. In the above example, there are two `http` components: one
+under `rest-api` and the other under `debug`. Because the structure is
+represented as a tree of components, the “path” of any node in the tree
+uniquely represents it in the structure. For example, the two `http` components
+in the previous example have these paths:
 
 ```
 root -> rest-api -> http
@@ -282,9 +284,9 @@ root -> debug -> http
 ```
 
 If each component were to know its place in the component tree, then it would
-easily be able to ensure that its configuration and initialization didn't
-conflict with other components of the same type. If the `http` component sets up
-a command-line parameter to know what address to listen on, the two `http`
+easily be able to ensure that its configuration and initialization didn’t
+conflict with other components of the same type. If the `http` component sets
+up a command-line parameter to know what address to listen on, the two `http`
 components in that program would set up:
 
 ```
@@ -293,13 +295,13 @@ components in that program would set up:
 ```
 
 So how can we enable each component to know its path in the component structure?
-To answer this we'll have to take a detour through a type, called `Component`.
+To answer this, we’ll have to take a detour through a type, called `Component`.
 
 ### Component and Configuration
 
-The `Component` type is a made up type (though you'll be able to find an
+The `Component` type is a made-up type (though you’ll be able to find an
 implementation of it at the end of this post). It has a single primary purpose,
-and that is to convey the program's structure to new components.
+and that is to convey the program’s structure to new components.
 
 To see how this is done, let's look at a couple of `Component`'s methods:
 
@@ -318,12 +320,11 @@ func (*Component) Child(name string) *Component
 func (*Component) Path() []string
 ```
 
-
 `Child` is used to create a new `Component`, corresponding to a new child node
 in the component structure, and `Path` is used retrieve the path of any
-`Component` within that structure. For the sake of keeping the examples simple
-let's pretend these functions have been implemented in a package called `mcmp`.
-Here's an example of how `Component` might be used in the `redis` component's
+`Component` within that structure. For the sake of keeping the examples simple,
+let’s pretend these functions have been implemented in a package called `mcmp`.
+Here’s an example of how `Component` might be used in the `redis` component’s
 code:
 
 ```go
@@ -354,9 +355,9 @@ In our above example, the two `redis` components' parameters would be:
 in our program, since it allows them to know their place in the component
 structure.
 
-Having to construct the prefix for the parameters ourselves is pretty annoying
-though, so let's introduce a new package, `mcfg`, which acts like `flag` but is
-aware of `Component`. Then `redis.NewConn` is reduced to:
+Having to construct the prefix for the parameters ourselves is pretty annoying,
+so let’s introduce a new package, `mcfg`, which acts like `flag` but is aware
+of `Component`. Then `redis.NewConn` is reduced to:
 
 ```go
 // Package redis
@@ -370,16 +371,16 @@ func NewConn(cmp *mcmp.Component, defaultAddr string) *RedisConn {
 }
 ```
 
-Easy-peazy.
+Easy-peasy.
 
 #### But What About Parse?
 
-Sharp-eyed gophers will notice that there's a key piece missing: When is
+Sharp-eyed gophers will notice that there is a key piece missing: When is
 `flag.Parse`, or its `mcfg` counterpart, called? When does `addrParam` actually
-get populated? You can't use the redis connection until that happens, but that
-can't happen inside `redis.NewConn` because there might be other components
-after `redis.NewConn` which want to set up parameters. To illustrate the
-problem, let's look at a simple program which wants to set up two `redis`
+get populated? You can’t use the redis connection until that happens, but that
+can’t happen inside `redis.NewConn` because there might be other components
+after `redis.NewConn` that want to set up parameters. To illustrate the
+problem, let’s look at a simple program that wants to set up two `redis`
 components:
 
 ```go
@@ -413,31 +414,33 @@ We will solve this problem in the next section.
 
 ### Instantiation vs Initialization
 
-Let's break down `redis.NewConn` into two phases: instantiation and initialization.
-Instantiation refers to creating the component on the component structure and
-having it declare what it needs in order to initialize (e.g. configuration
-parameters). During instantiation nothing external to the program is performed;
-no IO, no reading of the command-line, no logging, etc... All that's happened is
-that the empty template of a `redis` component has been created.
+Let’s break down `redis.NewConn` into two phases: instantiation and
+initialization. Instantiation refers to creating the component on the component
+structure and having it declare what it needs in order to initialize (e.g.,
+configuration parameters). During instantiation, nothing external to the
+program is performed; no IO, no reading of the command-line, no logging, etc.
+All that’s happened is that the empty template of a `redis` component has been
+created.
 
-Initialization is the phase when that template is filled in. Configuration
-parameters are read, startup actions like the creation of database connections
-are performed, and logging is output for informational and debugging purposes.
+Initialization is the phase during which the template is filled in.
+Configuration parameters are read, startup actions like the creation of database
+connections are performed, and logging is output for informational and debugging
+purposes.
 
-The key to making effective use of this dichotemy is to allow _all_ components
+The key to making effective use of this dichotomy is to allow _all_ components
 to instantiate themselves before they initialize themselves. By doing this we
-can ensure that, for example, all components have had the chance to declare
+can ensure, for example, that all components have had the chance to declare
 their configuration parameters before configuration parsing is done.
 
-So let's modify `redis.NewConn` so that it follows this dichotemy. It makes
-sense to leave instantiation related code where it is, but we need a mechanism
+So let’s modify `redis.NewConn` so that it follows this dichotomy. It makes
+sense to leave instantiation-related code where it is, but we need a mechanism
 by which we can declare initialization code before actually calling it. For
-this, I will introduce the idea of a "hook".
+this, I will introduce the idea of a “hook.”
 
 #### But First: Augment Component
 
 In order to support hooks, however, `Component` will need to be augmented with
-a few new methods. Right now it can only carry with it information about the
+a few new methods. Right now, it can only carry with it information about the
 component structure, but here we will add the ability to carry arbitrary
 key/value information as well:
 
@@ -457,13 +460,13 @@ func (*Component) Children() []*Component
 ```
 
 The final method allows us to, starting at the root `Component`, traverse the
-component structure, interacting with each `Component`'s key/value store. This
+component structure and interact with each `Component`’s key/value store. This
 will be useful for implementing hooks.
 
 #### Hooks
 
-A hook is, simply a function which will run later. We will declare a new
-package, calling it `mrun`, and say that it has two new functions:
+A hook is simply a function that will run later. We will declare a new package,
+calling it `mrun`, and say that it has two new functions:
 
 ```go
 // Package mrun
@@ -476,12 +479,12 @@ func InitHook(cmp *mcmp.Component, hook func())
 func Init(cmp *mcmp.Component)
 ```
 
-With these two functions we are able to defer the initialization phase of
-startup by using the same `Component`s we were passing around for the purpose of
-denoting component structure.
+With these two functions, we are able to defer the initialization phase of
+startup by using the same `Components` we were passing around for the purpose
+of denoting component structure.
 
-Now, with these few extra pieces of functionality in place, let's reconsider the
-most recent example, and make a program which creates two redis components which
+Now, with these few extra pieces of functionality in place, let’s reconsider the
+most recent example, and make a program that creates two redis components which
 exist independently of each other:
 
 ```go
@@ -549,19 +552,19 @@ func main() {
 ## Conclusion
 
 While the examples given here are fairly simplistic, the pattern itself is quite
-powerful. Codebases naturally accumulate small, domain specific behaviors and
+powerful. Codebases naturally accumulate small, domain-specific behaviors and
 optimizations over time, especially around the IO components of the program.
 Databases are used with specific options that an organization finds useful,
 logging is performed in particular places, metrics are counted around certain
-pieces of code, etc...
+pieces of code, etc.
 
-By programming with component structure in mind we are able to keep these
+By programming with component structure in mind, we are able to keep these
 optimizations while also keeping the clarity and compartmentalization of the
-code in-tact. We are able to keep our code flexible and configurable, while also
-re-usable and testable. And the simplicity of the tools involved means it can be
-extended and retrofitted for nearly any situation or use-case.
+code intact. We can keep our code flexible and configurable, while also
+re-usable and testable. Also, the simplicity of the tools involved means they
+can be extended and retrofitted for nearly any situation or use-case.
 
-Overall, it's a powerful pattern that I've found myself unable to do without
+Overall, this is a powerful pattern that I’ve found myself unable to do without
 once I began using it.
 
 ### Implementation
@@ -573,10 +576,10 @@ described in this post here:
 * [mcfg](https://godoc.org/github.com/mediocregopher/mediocre-go-lib/mcfg)
 * [mrun](https://godoc.org/github.com/mediocregopher/mediocre-go-lib/mrun)
 
-The packages are not stable and are likely to change frequently. You'll also
+The packages are not stable and are likely to change frequently. You’ll also
 find that they have been extended quite a bit from the simple descriptions found
-here, based on what I've found useful as I've implemented programs using
+here, based on what I’ve found useful as I’ve implemented programs using
 component structures. With these two points in mind, I would encourage you to
-look in and take whatever functionality you find useful for yourself, and not
-use the packages directly. The core pieces are not different from what has been
+look and take whatever functionality you find useful for yourself, and not use
+the packages directly. The core pieces are not different from what has been
 described in this post.