Remove old code related to static, it's not needed anymore

1 files changed, 0 insertions, 193 deletions

diff --git a/static/src/_posts/2014-10-29-erlang-pitfalls.md b/static/src/_posts/2014-10-29-erlang-pitfalls.md
deleted file mode 100644
index 7358430..0000000
--- a/static/src/_posts/2014-10-29-erlang-pitfalls.md
+++ /dev/null
@@ -1,193 +0,0 @@
----
-title: Erlang Pitfalls
-description: >-
-    Common pitfalls that people may run into when designing and writing
-    large-scale erlang applications.
-tags: tech
----
-
-I've been involved with a large-ish scale erlang project at Grooveshark since
-sometime around 2011. I started this project knowing absolutely nothing about
-erlang, but now I feel I have accumulated enough knowlege over time that I could
-conceivably give some back. Specifically, common pitfalls that people may run
-into when designing and writing a large-scale erlang application. Some of these
-may show up when searching for them, but some of them you may not even know you
-need to search for.
-
-## now() vs timestamp()
-
-The cononical way of getting the current timestamp in erlang is to use
-`erlang:now()`. This works great at small loads, but if you find your
-application slowing down greatly at highly parallel loads and you're calling
-`erlang:now()` a lot, it may be the culprit.
-
-A property of this method you may not realize is that it is monotonically
-increasing, meaning even if two processes call it at the *exact* same time they
-will both receive different output. This is done through some locking on the
-low-level, as well as a bit of math to balance out the time getting out of sync
-in the scenario.
-
-There are situations where fetching always unique timestamps is useful, such as
-seeding RNGs and generating unique identifiers for things, but usually when
-people fetch a timestamp they just want a timestamp. For these cases,
-`os:timestamp()` can be used. It is not blocked by any locks, it simply returns
-the time.
-
-## The rpc module is slow
-
-The built-in `rpc` module is slower than you'd think. This mostly stems from it
-doing a lot of extra work for every `call` and `cast` that you do, ensuring that
-certain conditions are accounted for. If, however, it's sufficient for the
-calling side to know that a call timed-out on them and not worry about it any
-further you may benefit from simply writing your own rpc module. Alternatively,
-use [one which already exists](https://github.com/cloudant/rexi).
-
-## Don't send anonymous functions between nodes
-
-One of erlang's niceties is transparent message sending between two phsyical
-erlang nodes. Once nodes are connected, a process on one can send any message to
-a process on the other exactly as if they existed on the same node. This is fine
-for many data-types, but for anonymous functions it should be avoided.
-
-For example:
-
-```erlang
-RemotePid ! {fn, fun(I) -> I + 1 end}.
-```
-
-Would be better written as
-
-```erlang
-incr(I) ->
-    I + 1.
-
-RemotePid ! {fn, ?MODULE, incr}.
-```
-
-and then using an `apply` on the RemotePid to actually execute the function.
-
-This is because hot-swapping code messes with anonymous functions quite a bit.
-Erlang isn't actually sending a function definition across the wire; it's simply
-sending a reference to a function. If you've changed the code within the
-anonymous function on a node, that reference changes. The sending node is
-sending a reference to a function which may not exist anymore on the receiving
-node, and you'll get a weird error which Google doesn't return many results for.
-
-Alternatively, if you simply send atoms across the wire and use `apply` on the
-other side, only atoms are sent and the two nodes involved can have totally
-different ideas of what the function itself does without any problems.
-
-## Hot-swapping code is a convenience, not a crutch
-
-Hot swapping code is the bees-knees. It lets you not have to worry about
-rolling-restarts for trivial code changes, and so adds stability to your
-cluster. My warning is that you should not rely on it. If your cluster can't
-survive a node being restarted for a code change, then it can't survive if that
-node fails completely, or fails and comes back up. Design your system pretending
-that hot-swapping does not exist, and only once you've done that allow yourself
-to use it.
-
-## GC sometimes needs a boost
-
-Erlang garbage collection (GC) acts on a per-erlang-process basis, meaning that
-each process decides on its own to garbage collect itself. This is nice because
-it means stop-the-world isn't a problem, but it does have some interesting
-effects.
-
-We had a problem with our node memory graphs looking like an upwards facing
-line, instead of a nice sinusoid relative to the number of connections during
-the day. We couldn't find a memory leak *anywhere*, and so started profiling. We
-found that the memory seemed to be comprised of mostly binary data in process
-heaps. On a hunch my coworker Mike Cugini (who gets all the credit for this) ran
-the following on a node:
-
-```erlang
-lists:foreach(erlang:garbage_collect/1, erlang:processes()).
-```
-
-and saw memory drop in a huge way. We made that code run every 10 minutes or so
-and suddenly our memory problem went away.
-
-The problem is that we had a lot of processes which individually didn't have
-much heap data, but all-together were crushing the box. Each didn't think it had
-enough to garbage collect very often, so memory just kept going up. Calling the
-above forces all processes to garbage collect, and thus throw away all those
-little binary bits they were hoarding.
-
-## These aren't the solutions you are looking for
-
-The `erl` process has tons of command-line options which allow you to tweak all
-kinds of knobs. We've had tons of performance problems with our application, as
-of yet not a single one has been solved with turning one of these knobs. They've
-all been design issues or just run-of-the-mill bugs. I'm not saying the knobs
-are *never* useful, but I haven't seen it yet.
-
-## Erlang processes are great, except when they're not
-
-The erlang model of allowing processes to manage global state works really well
-in many cases. Possibly even most cases. There are, however, times when it
-becomes a performance problem. This became apparent in the project I was working
-on for Grooveshark, which was, at its heart, a pubsub server.
-
-The architecture was very simple: each channel was managed by a process, client
-connection processes subscribed to that channel and received publishes from it.
-Easy right? The problem was that extremely high volume channels were simply not
-able to keep up with the load. The channel process could do certain things very
-fast, but there were some operations which simply took time and slowed
-everything down. For example, channels could have arbitrary properties set on
-them by their owners. Retrieving an arbitrary property from a channel was a
-fairly fast operation: client `call`s the channel process, channel process
-immediately responds with the property value. No blocking involved.
-
-But as soon as there was any kind of call which required the channel process to
-talk to yet *another* process (unfortunately necessary), things got hairy. On
-high volume channels publishes/gets/set operations would get massively backed up
-in the message queue while the process was blocked on another process. We tried
-many things, but ultimately gave up on the process-per-channel approach.
-
-We instead decided on keeping *all* channel state in a transactional database.
-When client processes "called" operations on a channel, they really are just
-acting on the database data inline, no message passing involved. This means that
-read-only operations are super-fast because there is minimal blocking, and if
-some random other process is being slow it only affects the one client making
-the call which is causing it to be slow, and not holding up a whole host of
-other clients.
-
-## Mnesia might not be what you want
-
-This one is probably a bit controversial, and definitely subject to use-cases.
-Do your own testing and profiling, find out what's right for you.
-
-Mnesia is erlang's solution for global state. It's an in-memory transactional
-database which can scale to N nodes and persist to disk. It is hosted
-directly in the erlang processes memory so you interact with it in erlang
-directly in your code; no calling out to database drivers and such. Sounds great
-right?
-
-Unfortunately mnesia is not a very full-featured database. It is essentially a
-key-value store which can hold arbitrary erlang data-types, albeit in a set
-schema which you lay out for it during startup. This means that more complex
-types like sorted sets and hash maps (although this was addressed with the
-introduction of the map data-type in R17) are difficult to work with within
-mnesia. Additionally, erlang's data model of immutability, while awesome
-usually, can bite you here because it's difficult (impossible?) to pull out
-chunks of data within a record without accessing the whole record.
-
-For example, when retrieving the list of processes subscribed to a channel our
-application doesn't simply pull the full list and iterate over it. This is too
-slow, and in some cases the subscriber list was so large it wasn't actually
-feasible. The channel process wasn't cleaning up its heap fast enough, so
-multiple publishes would end up with multiple copies of the giant list in
-memory. This became a problem. Instead we chain spawned processes, each of which
-pull a set chunk of the subsciber list, and iterate over that. This is very
-difficult to implement in mnesia without pulling the full subscriber list into
-the process' memory at some point in the process.
-
-It is, however, fairly trivial to implement in redis using sorted sets. For this
-case, and many other cases after, the motto for performance improvements became
-"stick it in redis". The application is at the point where *all* state which
-isn't directly tied to a specific connection is kept in redis, encoded using
-`term_to_binary`. The performance hit of going to an outside process for data
-was actually much less than we'd originally thought, and ended up being a plus
-since we had much more freedom to do interesting hacks to speedup up our
-accesses.