component oriented programming post

1 files changed, 568 insertions, 0 deletions

diff --git a/_posts/2020-11-16-component-oriented-programming.md b/_posts/2020-11-16-component-oriented-programming.md
new file mode 100644
index 0000000..c17b78d
--- /dev/null
+++ b/_posts/2020-11-16-component-oriented-programming.md
@@ -0,0 +1,568 @@
+---
+title: >-
+    Component Oriented Programming
+description: >-
+    A concise description of.
+---
+
+[A previous post in this
+blog](2019-08-02-program-structure-and-composability.html) focused on a
+framework developed to make designing component-based programs easier. In
+retrospect pattern/framework proposed was over-engineered; this post attempts to
+present the same ideas but in a more distilled form, as a simple programming
+pattern and without the unnecessary framework.
+
+Nothing in this post will be revelatory; it's surely all been said before. But
+hopefully the form it takes here will be useful to someone, as it would have
+been useful to myself when I first learned to program.
+
+## Axioms
+
+For the sake of brevity let's assume the following: within the context of
+single-process (_not_ the same as single-threaded), non-graphical programs the
+following may be said:
+
+1. A program may be thought of as a black-box with certain input and output
+   methods. It is the programmer's task to construct a program such that
+   specific inputs yield specific desired outputs.
+
+2. A program is not complete without sufficient testing to prove it's complete.
+
+3. Global state and global impure functions makes testing more difficult. This
+   can include singletons and system calls.
+
+Any of these may be argued, but that will be left for other posts. Any of these
+may be said of other types of programs as well, but that can also be left for
+other posts.
+
+## Components
+
+Properties of components include:
+
+1. *Creatable*: An instance of a component, given some defined set of
+   parameters, can be created independently of any other instance of that or any
+   other component.
+
+2. *Composable*: A component may be used as a parameter of another component's
+   instantiation. This would make it a child component of the one being
+   instantiated (i.e. the parent).
+
+3. *Abstract*: A component is an interface consisting of one or more methods.
+   Being an interface, a component may have one or more implementations, but
+   generally will have a primary implementation, which is used during a
+   program's runtime, and secondary "mock" implementations, which are only used
+   when testing other components.
+
+4. *Isolated*: A component may not use mutable global variables (i.e.
+   singletons) or impure global functions (e.g. system calls). It may only use
+   constants and variables/components given to it during instantiation.
+
+5. *Ephemeral*: A component may have a specific method used to clean up all
+   resources that it's holding (e.g. network connections, file handles,
+   language-specific lightweight threads, etc).
+
+   5a. This cleanup method should _not_ clean up any child components given as
+   instantiation parameters.
+
+   5b. This cleanup method should not return until the component's cleanup is
+   complete.
+
+Components are composed together to create programs. This is done by passing
+components as parameters to other components during instantiation. The `main`
+process of the program is responsible for instantiating and composing most, if
+not all, components in the program.
+
+A component oriented program is one which primarily, if not entirely, uses
+components for its functionality. Components generally have the quality of being
+able to interact with code written in other patterns without any toes being
+stepped on.
+
+## Example
+
+Let's start with an example: suppose a program is desired which accepts a string
+over stdin, hashes it, then writes the string to a file whose name is the hash.
+
+A naive implementation of this program in go might look like:
+
+```go
+package main
+
+import (
+	"crypto/sha1"
+	"encoding/hex"
+	"io"
+	"io/ioutil"
+	"os"
+)
+
+func hashFileWriter() error {
+	h := sha1.New()
+	r := io.TeeReader(os.Stdin, h)
+	body, _ := ioutil.ReadAll(r)
+	fileName := hex.EncodeToString(h.Sum(nil))
+
+	if err := ioutil.WriteFile(fileName, body, 0644); err != nil {
+		return err
+	}
+
+	return nil
+}
+
+func main() {
+	if err := hashFileWriter(); err != nil {
+		panic(err) // consider the error handled
+	}
+}
+```
+
+Notice that there's not a clear separation here between different components;
+`hashFileWriter` _might_ be considered a one method component, except that it
+breaks component property 4, which says that a component may not use mutable
+global variables (`os.Stdin`) or impure global functions (`ioutil.WriteFile`).
+
+Notice also that testing the program would require integration tests, and could
+not be unit tested (because there are no units, i.e. components). For a trivial
+program like this one writing unit and integration tests would be redundant, but
+for larger programs it may not be. Unit tests are important because they are
+fast to run, (usually) easy to formulate, and yield consistent results.
+
+This program could instead be written as being composed of three components:
+
+* `stdin`, a construct given by the runtime which outputs a stream of bytes.
+
+* `disk`, accepts a file name and file contents as input, writes the file
+  contents to a file of the given name, and potentially returns an error back.
+
+* `hashFileWriter`, reads a stream of bytes off a `stdin`, collects the stream
+  into a string, hashes that string to generate a file name, and uses `disk` to
+  create a corresponding file with the string as its contents. If `disk` returns
+  an error then `hashFileWriter` returns that error.
+
+Sprucing up our previous example to use these more clearly defined components
+might look like:
+
+```go
+package main
+
+import (
+	"crypto/sha1"
+	"encoding/hex"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+)
+
+// Disk defines the methods of the disk component.
+type Disk interface {
+	WriteFile(fileName string, fileContents []byte) error
+}
+
+// disk is the primary implementation of Disk. It implements the methods of
+// Disk (WriteFile) by performing actual system calls.
+type disk struct{}
+
+func NewDisk() Disk { return disk{} }
+
+func (disk) WriteFile(fileName string, fileContents []byte) error {
+	return ioutil.WriteFile(fileName, fileContents, 0644)
+}
+
+func hashFileWriter(stdin io.Reader, disk Disk) error {
+	h := sha1.New()
+	r := io.TeeReader(stdin, h)
+	body, err := ioutil.ReadAll(r)
+	if err != nil {
+		return fmt.Errorf("reading input: %w", err)
+	}
+
+	fileName := hex.EncodeToString(h.Sum(nil))
+
+	if err := disk.WriteFile(fileName, body); err != nil {
+		return fmt.Errorf("writing to file %q: %w", fileName, err)
+	}
+	return nil
+}
+
+func main() {
+	if err := hashFileWriter(os.Stdin, NewDisk()); err != nil {
+		panic(err) // consider the error handled
+	}
+}
+```
+
+`hashFileWriter` no longer directly uses `os.Stdin` and `ioutil.WriteFile`, but
+instead takes in components wrapping them; `io.Reader` is a built-in interface
+which `os.Stdin` inherently implements, and `Disk` is a simple interface defined
+just for this program.
+
+At first glance this would seem to have doubled the line-count for very little
+gain. This is because we have not yet written tests.
+
+## Testing
+
+As has already been firmly established, testing is important.
+
+In the second form of the program we can test the core-functionality of the
+`hashFileWriter` component without resorting to using the actual `stdin` and
+`disk` components. Instead we use mocks of those components. A mock component
+implements the same input/outputs that the "real" component does, but in a way
+which makes testing a particular component possible without reaching outside the
+process. These are unit tests.
+
+Tests for the latest form of the program might look like this:
+
+```go
+package main
+
+import (
+	"strings"
+	"testing"
+)
+
+// mockDisk implements the Disk interface. When WriteFile is called mockDisk
+// will pretend to write the file, but instead will simply store what arguments
+// WriteFile was called with.
+type mockDisk struct {
+	fileName     string
+	fileContents []byte
+}
+
+func (d *mockDisk) WriteFile(fileName string, fileContents []byte) error {
+	d.fileName = fileName
+	d.fileContents = fileContents
+	return nil
+}
+
+func TestHashFileWriter(t *testing.T) {
+	type test struct {
+		in          string
+		expFileName string
+		// expFileContents can be inferred from in
+	}
+
+	tests := []test{
+		{
+			in:          "",
+			expFileName: "da39a3ee5e6b4b0d3255bfef95601890afd80709",
+		},
+		{
+			in:          "hello",
+			expFileName: "aaf4c61ddcc5e8a2dabede0f3b482cd9aea9434d",
+		},
+		{
+			in:          "hello\nworld", // make sure newlines don't break things
+			expFileName: "7db827c10afc1719863502cf95397731b23b8bae",
+		},
+	}
+
+	for _, test := range tests {
+		// stdin is mocked via a strings.Reader, which outputs the string it was
+		// initialized with as a stream of bytes.
+		in := strings.NewReader(test.in)
+
+		// Disk is mocked by mockDisk, go figure.
+		disk := new(mockDisk)
+
+		if err := hashFileWriter(in, disk); err != nil {
+			t.Errorf("in:%q got err:%v", test.in, err)
+		} else if string(disk.fileContents) != test.in {
+			t.Errorf("in:%q got contents:%q", test.in, disk.fileContents)
+		} else if string(disk.fileName) != test.expFileName {
+			t.Errorf("in:%q got fileName:%q", test.in, disk.fileName)
+		}
+	}
+}
+```
+
+Notice that these tests do not _completely_ cover the desired functionality of
+the program: if `disk` returns an error that error should be returned from
+`hashFileWriter`. Whether or not this must be tested as well, and indeed the
+pedantry level of tests overall, is a matter of taste. I believe these to be
+sufficient.
+
+## Configuration
+
+Practically all programs require some level of runtime configuration. This may
+take the form of command-line arguments, environment variables, configuration
+files, etc. Almost all configuration methods will require some system call, and
+so any component accessing configuration directly would likely break component
+property 4.
+
+Instead each component should take in whatever configuration parameters it needs
+during instantiation, and let `main` handle collecting all configuration from
+outside of the process and instantiating the components appropriately.
+
+Let's take our previous program, but add in two new desired behaviors: first,
+there should be a command-line parameter which allows for specifying the string
+on the command-line, rather than reading from stdin, and second, there should be
+a command-line parameter declaring which directory to write files into. The new
+implementation looks like:
+
+```
+package main
+
+import (
+	"crypto/sha1"
+	"encoding/hex"
+	"flag"
+	"fmt"
+	"io"
+	"io/ioutil"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+// Disk defines the methods of the disk component.
+type Disk interface {
+	WriteFile(fileName string, fileContents []byte) error
+}
+
+// disk is the concrete implementation of Disk. It implements the methods of
+// Disk (WriteFile) by performing actual OS calls.
+type disk struct {
+	dir string
+}
+
+func NewDisk(dir string) Disk { return disk{dir: dir} }
+
+func (d disk) WriteFile(fileName string, fileContents []byte) error {
+	fileName = filepath.Join(d.dir, fileName)
+	return ioutil.WriteFile(fileName, fileContents, 0644)
+}
+
+func hashFileWriter(in io.Reader, disk Disk) error {
+	h := sha1.New()
+	r := io.TeeReader(in, h)
+	body, err := ioutil.ReadAll(r)
+	if err != nil {
+		return fmt.Errorf("reading input: %w", err)
+	}
+
+	fileName := hex.EncodeToString(h.Sum(nil))
+
+	if err := disk.WriteFile(fileName, body); err != nil {
+		return fmt.Errorf("writing to file %q: %w", fileName, err)
+	}
+	return nil
+}
+
+func main() {
+	str := flag.String("str", "", "If set, hash and write this string instead of stdin")
+	dir := flag.String("dir", ".", "Directory which files should be written to")
+	flag.Parse()
+
+	var in io.Reader
+	if *str == "" {
+		in = os.Stdin
+	} else {
+		in = strings.NewReader(*str)
+	}
+
+	disk := NewDisk(*dir)
+
+	if err := hashFileWriter(in, disk); err != nil {
+		panic(err) // consider the error handled
+	}
+}
+```
+
+Very little has changed, and in fact `hashFileWriter` was not touched at all,
+meaning all unit tests remained valid. 
+
+## Setup/Runtime/Cleanup
+
+A program can be split into three stages: setup, runtime, and cleanup. Setup
+is the stage during which internal state is assembled in order to make runtime
+possible. Runtime is the stage during which a program's actual function is being
+performed. Cleanup is the stage during which runtime stop and internal state is
+disassembled.
+
+A graceful (i.e. reliably correct) setup is quite natural to accomplish, but
+unfortunately a graceful cleanup is not a programmer's first concern, and
+frequently is not a concern at all. However, when building reliable and correct
+programs, a graceful cleanup is as important as a graceful setup and runtime. A
+program is still running while it is being cleaned up, and it's possibly even
+acting on the outside world still. Shouldn't it behave correctly during that
+time?
+
+Achieving a graceful setup and cleanup with components is quite simple:
+
+During setup a single-threaded process (usually `main`) will construct the
+"leaf" components (those which have no child components of their own) first,
+then the components which take those leaves as parameters, then the components
+which take _those_ as parameters, and so on, until all are constructed. The
+components end up assembled into a directed acyclic graph.
+
+At this point the program will begin runtime.
+
+Once runtime is over and it is time for the program to exit it's only necessary
+to call each component's cleanup method(s) in the reverse of the order the
+components were instantiated in. A component's cleanup method should not be
+called until all of its parent components have been cleaned up.
+
+Inherent to the pattern is the fact that each component will certainly be
+cleaned up before any of its child components, since its child components must
+have been instantiated first and a component will not clean up child components
+given as parameters (as-per component property 5a).
+
+With go this pattern can be achieved easily using `defer`, but writing it out
+manually is not so hard, as in this toy example:
+
+```
+package main
+
+import (
+	"fmt"
+	"time"
+)
+
+// sleeper is a component which prints its children and sleeps when it's time to
+// cleanup.
+type sleeper struct {
+	children []*sleeper
+	toSleep  time.Duration
+
+	// The builtin time.Sleep is an impure global function, a component can't
+	// use it, so the component must be instantiated with it as a parameter.
+	sleep func(time.Duration)
+
+	// likewise os.Stdout is a global singleton, and so must also be a
+	parameter.
+	stdout io.Writer
+}
+
+func (s *sleeper) print() {
+	fmt.Fprintf(s.stdout, "I will sleep for %v\n", s.toSleep)
+	for _, child := range s.children {
+		child.print()
+	}
+}
+
+func (s *sleeper) cleanup() {
+	s.sleep(s.toSleep)
+	fmt.Fprintf(s.stdout, "I slept for %v\n", s.toSleep)
+}
+
+func main() {
+
+	// Within main we make a helper function to easily construct sleepers. for a
+	// toy like this it's not worth the effort of giving sleeper a real
+	// initialization function.
+	newSleeper := func(toSleep time.Duration, children ...*sleeper) *sleeper {
+		return &sleeper{
+			children: children,
+			toSleep:  toSleep,
+			sleep:    time.Sleep,
+			stdout:   os.Stdout,
+		}
+	}
+
+	aa := newSleeper(250 * time.Millisecond)
+	defer aa.cleanup()
+
+	ab := newSleeper(250 * time.Millisecond)
+	defer ab.cleanup()
+
+	// A's children are AA and AB
+	a := newSleeper(500*time.Millisecond, aa, ab)
+	defer a.cleanup()
+
+	b := newSleeper(750 * time.Millisecond)
+	defer b.cleanup()
+
+	// root's children are A and B
+	root := newSleeper(1*time.Second, a, b)
+	defer root.cleanup()
+
+	// All components are now instantiated and runtime begins.
+	root.print()
+    // ... and just like that, runtime ends.
+	fmt.Println("--- Alright, fun is over, time for bed ---")
+
+	// Now to clean up, cleanup methods are called in the reverse order of the
+	// component's instantiation.
+	root.cleanup()
+	b.cleanup()
+	a.cleanup()
+	ab.cleanup()
+	aa.cleanup()
+
+	// Expected output is:
+	//
+	// I will sleep for 1s
+	// I will sleep for 500ms
+	// I will sleep for 250ms
+	// I will sleep for 250ms
+	// I will sleep for 750ms
+	// --- Alright, fun is over, time for bed ---
+	// I slept for 1s
+	// I slept for 750ms
+	// I slept for 500ms
+	// I slept for 250ms
+	// I slept for 250ms
+}
+```
+
+## Criticisms
+
+In lieu of a FAQ I will attempt to premeditate criticisms of the component
+oriented pattern laid out in this post:
+
+*This seems like a lot of extra work.*
+
+Building reliable programs is a lot of work, just as building reliable-anything
+is a lot of work. Many of us work in an industry which likes to balance
+reliability (sometimes referred to by the more specious "quality") with
+maleability and deliverability, which naturally leads to skepticism of any
+suggestions which require more time spent on reliability. This is not
+necessarily a bad thing, it's just how the industry functions.
+
+All that said, a pattern need not be followed perfectly to be worthwhile, and
+the amount of extra work incurred by it can be decided based on practical
+considerations. I merely maintain that when it comes time to revisit some
+existing code, either to fix or augment it, that the job will be notably easier
+if the code _mostly_ follows this pattern.
+
+*My language makes this difficult.*
+
+I don't know of any language which makes this pattern particularly easy, so
+unfortunately we're all in the same boat to some extent (though I recognize that
+some languages, or their ecosystems, make it more difficult than others). It
+seems to me that this pattern shouldn't be unbearably difficult for anyone to
+implement in any language either, however, as the only language feature needed
+is abstract typing.
+
+It would be nice to one day see a language which explicitly supported this
+pattern by baking the component properties in as compiler checked rules.
+
+*This will result in over-abstraction.*
+
+Abstraction is a necessary tool in a programmer's toolkit, there is simply no
+way around it. The only questions are "how much?" and "where?".
+
+The use of this pattern does not effect how those questions are answered, but
+instead aims to more clearly delineate the relationships and interactions
+between the different abstracted types once they've been established using other
+methods. Over-abstraction is the fault of the programmer, not the language or
+pattern or framework.
+
+*The acronymn is CoP.*
+
+Why do you think I've just been ackwardly using "this pattern" instead of the
+acronymn for the whole post? Better names are welcome.
+
+## Conclusion
+
+The component oriented pattern helps make our code more reliable with only a
+small amount of extra effort incurred. In fact most of the pattern has to do
+establishing sensible abstractions around global functionality and remembering
+certain idioms for how those abstractions should be composed together, something
+most of us do to some extent already anyway.
+
+While beneficial in many ways, component oriented programming is merely a tool
+which can be applied in many cases. It is certain that there are cases where it
+is not the right tool for the job. I've found these cases to be
+few-and-far-between, however. It's a solid pattern that I've gotten good use out
+of, and hopefully you'll find it, or some parts of it, to be useful as well.