1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
|
---
title: >-
The Case for Open Infrastructure
description: >-
Without using the word which starts with "W" and ends in "3".
tags: tech crypto
---
Today I saw an image which made feel deeply sad. This one:
In May of 2020 Keybase, one of my all-time favorite web services, was bought by
Zoom, the international collaboration sensation. Almost precisely since that
moment, Keybase has effectively been put on ice. [The final post on the Keybase
Blog][zoom] is refreshingly up-front about this:
[zoom]: https://keybase.io/blog/keybase-joins-zoom
> Initially, our single top priority is helping to make Zoom even more secure. There are no specific plans for the Keybase app yet. Ultimately Keybase's future is in Zoom's hands, and we'll see where that takes us. Of course, if anything changes about Keybase’s availability, our users will get plenty of notice.
To be fair, there are still a few devs still handling Github issues and
committing code. The servers have not been shut down, user data is not lost. The
source code, for both client and server, is still available. The lights are
still on! Nevertheless, I've made every effort to move my data off of Keybase.
This hasn't been done lightly. I was a long-time Keybase user, frequent
advocate, and even friend to some of the team. Keybase has a lot of cool
features built on top of its rock-solid private storage, and I used them to
build the foundation of my digital presence over many years. Clearly, I don't
_want_ to leave Keybase. It feels like saying goodbye to an old friend.
But I've seen this movie before, and I know how it ends. At least doing it now
means I can find replacements for each of Keybase's features at my leisure,
rather than being forced into doing so in a scramble just before the hammer
falls.
## Open-Source is not Enough
The source code for Keybase is, right now, fully open source on
[their Github][kb-src]. Why couldn't someone spin up their own copy for
themselves and their friends, give it a stupid name, and keep the party rolling?
[kb-src]: https://github.com/keybase
The reason is that Keybase wasn't designed to be hosted by anyone except the
Keybase team. There's no documentation on how to run the backend, and only a
9-page long string of Github projects to go on. Even if someone were to figure
it out, the system is probably designed to be a globally scalable service, not a
small clone being hosted for the benefit of a few. The design and expense of two
such systems are vastly different, and not always compatible.
While the Keybase _source code_ may be open, the _infrastructure_ is closed.
Infrastructure refers to the computer hardware which hosts data and runs
processes on behalf of a service's users, as opposed to users doing so
themselves. Users generally have very small resources, think a phone or laptop,
which are not always online. If you want to host 22 terabytes of family photos,
you won't be doing so on your laptop.
You might instead upload them to Keybase, in which case Keybase's servers
will hold onto them for you, and make them available to you whenever you want to
view them. You are now a user of the Keybase service, which is hosted on the
Keybase company's infrastructure.
This arrangement, dubbed a "cloud service", is exceedingly common in the tech
industry, and its weakest point is the company part. Even if the code and the
infrastructure are perfect, and users are completely hooked, and the service
could change the world... if there's no money there's no company.
And yet, even if there is no money, the company must still posture as if there
is going to be, in order to attract investors. These investors then pay for the
infrastructure, in exchange for a cut of any future money.
This posturing excludes anyone who's not downstream of the company from
participating in hosting the infrastructure, as they might then deserve some of
the money too. The investors will likely make more by selling the whole company
to a bigger fish than if they dilute their share of the profits. This is
reflected in the design of the infrastructure itself. It's locked down to anyone
not employed, directly or indirectly, by the company.
In the end the services we rely on rank profitability and exclusivity over
usefulness and endurance, and the internet is worse off for it. We can do
better.
## Open Infrastructure
Open infrastructure is the idea that anyone can help host the infrastructure of
a service they care about, without any barriers to them doing so beyond their
ability to manage the hardware.
The developers of an open infrastructure service don't have to actually manage
the infrastructure themselves, a demanding and time-consuming task, so the
service can be built by volunteers or a small company that isn't directly making
money from the service. And because usefulness now ranks above profitability,
the actual needs of the users of this service can be directly addressed.
None of these ideas around open infrastructure are new, though the wording might
be, and there's already quite a bit of progress made in this direction. To get a
feel for this progress it's worth perusing some of the existing projects and
communities in this space. As we do so I'm going to break the open
infrastructure space up into three notable, though frequently overlapping,
parts: self-hosted, federated, and distributed.
### Self-Hosted
Self-hosted services are those designed to be hosted by individuals or
households for their own personal use. They may be hosted on a home-server,
Raspberry Pi, or other similar device. One installation of a self-hosted service
often never interacts with another, as the user's own infrastructure is enough
to accomplish the service's goal.
Examples of self-hosting use-cases include:
* A personal website or blog (like this one)
* Media library platforms (Jellyfin, Ultrasonic)
* Photo and video sync/storage (Piwigo)
* Document archival and search (Paperless)
* Entire "cloud" replacement suites (NextCloud)
Self-hosting is popular among enthusiasts, but is not popular outside that realm
due to the requirement of managing your own infrastructure.
### Federated
Federated services are designed to be hosted by a small to medium sized entity,
and to serve many more users than are involved in the hosting. The entity may be
a company servicing its employees, members of a community hosting services
for their peers, or a paid service hosting for its customers. One installation
of a federated service will often communicate with another by design, so that
users of the one installation may interact with those on another.
Email is the classic example of a federated service. Each user has an account
with someone running an email server, and they use those servers to exchange
messages with users of other email servers. In the past it wasn't uncommon for
an ISP to host a mail server for its customers, and to this day many companies
still manage email servers for their employees.
The line between federated and self-hosted can sometimes be quite blury, as it's
frequently possible for an individual to set up a federated service just for
themselves.
Examples of federation use-cases, besides email, include:
* Chat servers (IRC, Matrix)
* Micro-blogs, aka Twitter replacements (Mastadon, MissKey)
* Code hosting (Gitea)
* Social link aggregators, aka Reddit replacements (Lemmy)
* Video streaming platforms (PeerTube)
### Distributed
Where self-hosted services are hosted by-and-for individuals or very small
groups, and federated services are hosted by small groups for larger groups,
distributed services are hosted by both individuals and groups for _everyone_.
The key differentiator between a federated and a distributed service is that in
a federated service the user interacts with a specific server, or set of
servers, that they have a relationship with. In a distributed service the user
has no relationship with those who run the infrastructure, and so users interact
with whoever they can get ahold of.
Examples of distributed service use-cases include:
* Data distribution (IPFS, BitTorrent magnet links)
* Distributed transaction ledgers (Blockchains, love 'em or hate 'em!)
* Onion routing (Tor)
There aren't many examples of truly distributed services that have stood the
test of time, as the protocols and algorithms used to manage these global
datasets are quite tricky to get right. It's possible that the recent advent of
blockchains will change this, though blockchains usually assume some sort of
direct compensation from users to "miners", a barrier not usually found in
federated services.
-----
I don't delineate these categories in order to direct you to one or the other,
or to exhaustively list _every_ ongoing project in this space. Rather, I hope
this gives you a starting point if you're wondering what sorts of problems are
able to be solved using open infrastructure, and how people are approaching
them. Perhaps there's a problem you care about, and there's a way to solve it
using an open infrastructure based solution.
## It's on us
The ideas presented here aren't some pie-in-the-sky, utopian fantasy; this is
happening, and it's happening for good reasons. Some may argue that
infrastructure can't survive without a profit model to pay for it, or that
for-profit services are of higher quality and so free versions will never take
off, or that if services are free to host then no one will make them. People
said much the same about open source software.
The open source movement has shown that participation, not profitability, is the
primary driver of successful software. Why should it not be the case for our
service infrastructure?
-----
**Footnotes**
* I use a lot of words here whose definitions are by no means agreed upon. I've
done my best to stick to my best understanding of word meanings, and to be
consistent in them. If you disagree with my usage of a word, I'm sorry. It
doesn't mean either of us is wrong, we just speak different.
* There is an existing project, under the OpenStack project, called Open
Infrastructure. It has to do with some kind of standardization around the
infrastructure used for cloud services. Unfortunately, the word
"infrastructure" has a much broader meaning than what they are using it for,
and I don't think there's a better word for what I'm trying to describe.
|
