summaryrefslogtreecommitdiff
path: root/static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md
diff options
context:
space:
mode:
Diffstat (limited to 'static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md')
-rw-r--r--static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md209
1 files changed, 0 insertions, 209 deletions
diff --git a/static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md b/static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md
deleted file mode 100644
index 749bd02..0000000
--- a/static/src/_posts/2021-08-04-selfhosting-a-blog-mailing-list.md
+++ /dev/null
@@ -1,209 +0,0 @@
----
-title: >-
- Self-Hosting a Blog Mailing List
-description: >-
- For fun and no profit.
-tags: tech
----
-
-As of this week the Mediocre Blog has a new follow mechanism: email! [Sign up
-on the **Follow** page][follow] and you'll get an email everytime a new post
-is published to the blog. It's like RSS, except there's a slight chance you
-might actually use it.
-
-This post will detail my relatively simple setup for this, linking to points
-within my blog's server code which are relevant. While I didn't deliberately
-package my code up into a nice public package, if you know have some cursory
-knowledge of Go you could probably rip my code and make it work for you. Don't
-worry, it has a [permissive license](/assets/wtfpl.txt).
-
-[follow]: /follow.html
-
-## Email Server
-
-Self-hosting email is the hardest and most foreign part of this whole
-thing for most devs. The long and the short of it is that it's very unlikely you
-can do this without renting a VPS somewhere. Luckily there are VPSs out there
-which are cheap and which allow SMTP traffic, so it's really just a matter of
-biting the cost bullet and letting your definition of "self-hosted" be a bit
-flexible. At least you still control the code!
-
-I highly recommend [maddy][maddy] as an email server which has everything you
-need out of the box, no docker requirements, and a flexible-yet-simple
-configuration language. I've discussed [in previous posts][maddypost] the
-general steps I've used to set up maddy on a remote VPS, and so I won't
-re-iterate here. Just know that I have a VPS on my private [nebula][nebula] VPN,
-with a maddy server listening for outgoing mail on port 587, with
-username/password authentication on that port.
-
-[maddy]: https://maddy.email
-[maddypost]: {% post_url 2021-07-06-maddy-vps %}
-[nebula]: https://github.com/slackhq/nebula
-
-## General API Design
-
-The rest of the system lies within the Go server which hosts my blog. There is
-only a single instance of the server, and it runs in my living room. With these
-as the baseline environmental requirements, the rest of the design follows
-easily:
-
-* The Go server provides [three REST API endpoints][restendpoints]:
-
- - `POST /api/mailinglist/subscribe`: Accepts a POST form argument `email`, sends a
- verification email to that email address.
-
- - `POST /api/mailinglist/finalize`: Accepts a POST form argument `subToken`,
- which is a random token sent to the user when they subscribe. Only by
- finalizing their subscription can a user be considered actually
- subscribed.
-
- - `POST /api/mailinglist/unsubscribe`: Accepts a POST form argument
- `unsubToken`, which is sent with each blog post notification to the user.
-
-* The static frontend code has [two pages][staticpages] related to the mailing
- list:
-
- - `/mailinglist/finalize.html`: The verification email which is sent to the
- user links to this page, with the `subToken` as a GET argument. This page
- then submits the `subToken` to the `POST /api/mailinglist/finalize`
- endpoint.
-
- - `/mailinglist/unsubscribe.html`: Each blog post notification email sent to
- users contains a link to this page, with an `unsubToken` as a GET
- argument. This page then submits the `unsubToken` to the `POST
- /api/mailinglist/unsubscribe` endpoint.
-
-It's a pretty small API, but it covers all the important things, namely
-verification (because I don't want people signed up against their will, nor do I
-want to be sending emails to fake email addresses), and unsubscribing.
-
-[restendpoints]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/5ca7dadd02fb49dd62ad448d12021359e41beec1/srv/cmd/mediocre-blog/main.go#L169
-[staticpages]: https://github.com/mediocregopher/blog.mediocregopher.com/tree/9c3ea8dd803d6f0df768e3ae37f8c4ab2efbcc5c/static/src/mailinglist
-
-## Proof-of-work
-
-It was important to me that someone couldn't just sit and submit random emails
-to the `POST /api/mailinglist/subscribe` endpoint in a loop, causing my email
-server to eventually get blacklisted. To prevent this I've implemented a simple
-proof-of-work (PoW) system, whereby the client must first obtain a PoW
-challenge, generate a solution for that challenge (which involves a lot of CPU
-time), and then submit that solution as part of the subscribe endpoint call.
-
-Both the [server-side][powserver] and [client-side][powclient] code can be found
-in the blog's git repo. You could theoretically view the Go documentation for
-the server code on pkg.go.dev, but apparently their bot doesn't like my WTFPL.
-
-When providing a challenge to the client, the server sends back two values: the
-seed and the target.
-
-The target is simply a number whose purpose will become apparent in a second.
-
-The seed is a byte-string which encodes:
-
-* Some random bytes.
-
-* An expiration timestamp.
-
-* A target (matching the one returned to the client alongside the seed).
-
-* An HMAC-MD5 which signs all of the above.
-
-When the client submits a valid solution the server checks the HMAC to ensure
-that the seed was generated by the server, it checks the expiration to make sure
-the client didn't take too long to solve it, and it checks in an [internal
-storage][powserverstore] whether that seed hasn't already been solved. Because
-the expiration is built into the seed the server doesn't have to store each
-solved seed forever, only until the seed has expired.
-
-To generate a solution to the challenge the client does the following:
-
-* Concatenate up to `len(seed)` random bytes onto the original seed given by the
- server.
-
-* Calculate the SHA512 of that.
-
-* Parse the first 4 bytes of the resulting hash as a big-endian uint32.
-
-* If that uint32 is less than the target then the random bytes generated in the
- first step are a valid solution. Otherwise the client loops back to the first
- step.
-
-Finally, a new endpoint was added: `GET /api/pow/challenge`, which returns a PoW
-seed and target for the client to solve. Since seeds don't require storage in a
-database until _after_ they are solved there are essentially no consequences to
-someone spamming this in a loop.
-
-With all of that in place, the `POST /api/mailinglist/subscribe` endpoint
-described before now also requires a `powSeed` and a `powSolution` argument. The
-[Follow][follow] page, prior to submitting a subscribe request, first retrieves
-a PoW challenge, generates a solution, and only _then_ will it submit the
-subscribe request.
-
-[powserver]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/9c3ea8dd803d6f0df768e3ae37f8c4ab2efbcc5c/srv/pow/pow.go
-[powserverstore]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/5ca7dadd02fb49dd62ad448d12021359e41beec1/srv/pow/store.go
-[powclient]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/9c3ea8dd803d6f0df768e3ae37f8c4ab2efbcc5c/static/src/assets/solvePow.js
-
-## Storage
-
-Storage of emails is fairly straightforward: since I'm not running this server
-on multiple hosts, I can just use [SQLite][sqlite]. My code for storage in
-SQLite can all be found [here][sqlitecode].
-
-My SQLite table has a single table:
-
-```
-CREATE TABLE emails (
- id TEXT PRIMARY KEY,
- email TEXT NOT NULL,
- sub_token TEXT NOT NULL,
- created_at INTEGER NOT NULL,
-
- unsub_token TEXT,
- verified_at INTEGER
-)
-```
-
-It will probably one day need an index on `sub_token` and `unsub_token`, but I'm
-not quite there yet.
-
-The `id` field is generated by first lowercasing the email (because emails are
-case-insensitive) and then hashing it. This way I can be sure to identify
-duplicates easily. It's still possible for someone to do the `+` trick to get
-their email in multiple times, but as long as they verify each one I don't
-really care.
-
-[sqlite]: https://sqlite.org/index.html
-[sqlitecode]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/5ca7dadd02fb49dd62ad448d12021359e41beec1/srv/mailinglist/store.go
-
-## Publishing
-
-Publishing is quite easy: my [MailingList interface][mailinglistinterface] has a
-`Publish` method on it, which loops through all records in the SQLite table,
-discards those which aren't verified, and sends an email to the rest containing:
-
-* A pleasant greeting.
-
-* The new post's title and URL.
-
-* An unsubscribe link.
-
-I will then use a command-line interface to call this `Publish` method. I
-haven't actually made that interface yet, but no one is subscribed yet so it
-doesn't matter.
-
-[mailinglistinterface]: https://github.com/mediocregopher/blog.mediocregopher.com/blob/5ca7dadd02fb49dd62ad448d12021359e41beec1/srv/mailinglist/mailinglist.go#L23
-
-## Easy-Peasy
-
-The hardest part of the whole thing was probably getting maddy set up, with a
-close second being trying to decode a hex string to a byte string in javascript
-(I tried Crypto-JS, but it wasn't working without dragging in webpack or a bunch
-of other nonsense, and vanilla JS doesn't have any way to do it!).
-
-Hopefully reading this will make you consider self-hosting your own blog's
-mailing list as well. If we let these big companies keep taking over all
-internet functionality then eventually they'll finagle the standards so that
-no one can self-host anything, and we'll have to start all over.
-
-And really, do you _need_ tracking code on the emails you send out for your
-recipe blog? Just let your users ignore you in peace and quiet.