Choose boring technology
This commit is contained in:
David Eisinger
@@ -31,6 +31,10 @@ references:
|
|||||||
url: https://techxplore.com/news/2023-06-problems.html
|
url: https://techxplore.com/news/2023-06-problems.html
|
||||||
date: 2023-07-03T21:19:28Z
|
date: 2023-07-03T21:19:28Z
|
||||||
file: techxplore-com-8o7hmu.txt
|
file: techxplore-com-8o7hmu.txt
|
||||||
|
- title: "Dan McKinley :: Choose Boring Technology"
|
||||||
|
url: https://mcfunley.com/choose-boring-technology
|
||||||
|
date: 2023-07-04T03:40:17Z
|
||||||
|
file: mcfunley-com-5myzcq.txt
|
||||||
---
|
---
|
||||||
|
|
||||||
### Thoughts on priorities in software development
|
### Thoughts on priorities in software development
|
||||||
@@ -51,9 +55,11 @@ references:
|
|||||||
* [Imaginary Problems Are the Root of Bad Software][5]
|
* [Imaginary Problems Are the Root of Bad Software][5]
|
||||||
* [When to Build Millennia Sewers][6]
|
* [When to Build Millennia Sewers][6]
|
||||||
* [We are wasting up to 20% of our time on computer problems, says study][7]
|
* [We are wasting up to 20% of our time on computer problems, says study][7]
|
||||||
|
* [Choose Boring Technology][8]
|
||||||
|
|
||||||
[3]: https://grugbrain.dev/
|
[3]: https://grugbrain.dev/
|
||||||
[4]: https://world.hey.com/dhh/even-amazon-can-t-make-sense-of-serverless-or-microservices-59625580
|
[4]: https://world.hey.com/dhh/even-amazon-can-t-make-sense-of-serverless-or-microservices-59625580
|
||||||
[5]: https://cerebralab.com/Imaginary_Problems_Are_the_Root_of_Bad_Software
|
[5]: https://cerebralab.com/Imaginary_Problems_Are_the_Root_of_Bad_Software
|
||||||
[6]: https://taylor.town/millennium-sewer
|
[6]: https://taylor.town/millennium-sewer
|
||||||
[7]: https://techxplore.com/news/2023-06-problems.html
|
[7]: https://techxplore.com/news/2023-06-problems.html
|
||||||
|
[8]: https://mcfunley.com/choose-boring-technology
|
||||||
|
|||||||
260
static/archive/mcfunley-com-5myzcq.txt
Normal file
260
static/archive/mcfunley-com-5myzcq.txt
Normal file
@@ -0,0 +1,260 @@
|
|||||||
|
#[1]alternate
|
||||||
|
|
||||||
|
[2]
|
||||||
|
|
||||||
|
Dan McKinley
|
||||||
|
Math, Programming, and Minority Reports
|
||||||
|
|
||||||
|
[3]Tweet [4]Follow @mcfunley
|
||||||
|
|
||||||
|
[5]Choose Boring Technology
|
||||||
|
March 30th, 2015
|
||||||
|
|
||||||
|
Probably the single best thing to happen to me in my career was having
|
||||||
|
had [6]Kellan placed in charge of me. I stuck around long enough to see
|
||||||
|
Kellanâs technical decisionmaking start to bear fruit. I learned a
|
||||||
|
great deal from this, but I also learned a great deal as a result of
|
||||||
|
this. I would not have been free to become the engineer that wrote
|
||||||
|
[7]Data Driven Products Now! if Kellan had not been there to so
|
||||||
|
thoroughly stick the landing on technology choices.
|
||||||
|
[FRQKLCy.jpg] Being inspirational as always.
|
||||||
|
|
||||||
|
In the year since leaving Etsy, Iâve resurrected my ability to care
|
||||||
|
about technology. And my thoughts have crystallized to the point where
|
||||||
|
I can write them down coherently. What follows is a distillation of the
|
||||||
|
Kellan gestalt, which will hopefully serve to horrify him only
|
||||||
|
slightly.
|
||||||
|
|
||||||
|
Embrace Boredom.
|
||||||
|
|
||||||
|
Letâs say every company gets about three innovation tokens. You can
|
||||||
|
spend these however you want, but the supply is fixed for a long while.
|
||||||
|
You might get a few more after you achieve a [8]certain level of
|
||||||
|
stability and maturity, but the general tendency is to overestimate the
|
||||||
|
contents of your wallet. Clearly this model is approximate, but I think
|
||||||
|
it helps.
|
||||||
|
|
||||||
|
If you choose to write your website in NodeJS, you just spent one of
|
||||||
|
your innovation tokens. If you choose to use [9]MongoDB, you just spent
|
||||||
|
one of your innovation tokens. If you choose to use [10]service
|
||||||
|
discovery tech thatâs existed for a year or less, you just spent one of
|
||||||
|
your innovation tokens. If you choose to write your own database, oh
|
||||||
|
god, youâre in trouble.
|
||||||
|
|
||||||
|
Any of those choices might be sensible if youâre a javascript
|
||||||
|
consultancy, or a database company. But youâre probably not. Youâre
|
||||||
|
probably working for a company that is at least ostensibly
|
||||||
|
[11]rethinking global commerce or [12]reinventing payments on the web
|
||||||
|
or pursuing some other suitably epic mission. In that context, devoting
|
||||||
|
any of your limited attention to innovating ssh is an excellent way to
|
||||||
|
fail. Or at best, delay success [13][1].
|
||||||
|
|
||||||
|
What counts as boring? Thatâs a little tricky. âBoringâ should not be
|
||||||
|
conflated with âbad.â There is technology out there that is both boring
|
||||||
|
and bad [14][2]. You should not use any of that. But there are many
|
||||||
|
choices of technology that are boring and good, or at least good
|
||||||
|
enough. MySQL is boring. Postgres is boring. PHP is boring. Python is
|
||||||
|
boring. Memcached is boring. Squid is boring. Cron is boring.
|
||||||
|
|
||||||
|
The nice thing about boringness (so constrained) is that the
|
||||||
|
capabilities of these things are well understood. But more importantly,
|
||||||
|
their failure modes are well understood. Anyone who knows me well will
|
||||||
|
understand that itâs only with a overwhelming sense of malaise that I
|
||||||
|
now invoke the spectre of Don Rumsfeld, but I must.
|
||||||
|
[n8ElWr3.jpg] To be clear, fuck this guy.
|
||||||
|
|
||||||
|
When choosing technology, you have both known unknowns and unknown
|
||||||
|
unknowns [15][3].
|
||||||
|
* A known unknown is something like: we donât know what happens when
|
||||||
|
this database hits 100% CPU.
|
||||||
|
* An unknown unknown is something like: geez it didnât even occur to
|
||||||
|
us that [16]writing stats would cause GC pauses.
|
||||||
|
|
||||||
|
Both sets are typically non-empty, even for tech thatâs existed for
|
||||||
|
decades. But for shiny new technology the magnitude of unknown unknowns
|
||||||
|
is significantly larger, and this is important.
|
||||||
|
|
||||||
|
Optimize Globally.
|
||||||
|
|
||||||
|
I unapologetically think a bias in favor of boring technology is a good
|
||||||
|
thing, but itâs not the only factor that needs to be considered.
|
||||||
|
Technology choices donât happen in isolation. They have a scope that
|
||||||
|
touches your entire team, organization, and the system that emerges
|
||||||
|
from the sum total of your choices.
|
||||||
|
|
||||||
|
Adding technology to your company comes with a cost. As an abstract
|
||||||
|
statement this is obvious: if weâre already using Ruby, adding Python
|
||||||
|
to the mix doesnât feel sensible because the resulting complexity would
|
||||||
|
outweigh Pythonâs marginal utility. But somehow when weâre talking
|
||||||
|
about Python and Scala or MySQL and Redis people [17]lose their minds,
|
||||||
|
discard all constraints, and start raving about using the best tool for
|
||||||
|
the job.
|
||||||
|
|
||||||
|
[18]Your function in a nutshell is to map business problems onto a
|
||||||
|
solution space that involves choices of software. If the choices of
|
||||||
|
software were truly without baggage, you could indeed pick a whole mess
|
||||||
|
of locally-the-best tools for your assortment of problems.
|
||||||
|
Created with Sketch. Problems Technical Solutions The way you might
|
||||||
|
choose technology in a world where choices are cheap: "pick the right
|
||||||
|
tool for the job."
|
||||||
|
|
||||||
|
But of course, the baggage exists. We call the baggage âoperationsâ and
|
||||||
|
to a lesser extent âcognitive overhead.â You have to monitor the thing.
|
||||||
|
You have to figure out unit tests. You need to know the first thing
|
||||||
|
about it to hack on it. You need an init script. I could go on for days
|
||||||
|
here, and all of this adds up fast.
|
||||||
|
Created with Sketch. Problems Technical Solutions The way you choose
|
||||||
|
technology in the world where operations are a serious concern (i.e.,
|
||||||
|
"reality").
|
||||||
|
|
||||||
|
The problem with âbest tool for the jobâ thinking is that it takes a
|
||||||
|
myopic view of the words âbestâ and âjob.â Your job is keeping the
|
||||||
|
company in business, god damn it. And the âbestâ tool is the one that
|
||||||
|
occupies the âleast worstâ position for as many of your problems as
|
||||||
|
possible.
|
||||||
|
|
||||||
|
It is basically always the case that the long-term costs of keeping a
|
||||||
|
system working reliably vastly exceed any inconveniences you encounter
|
||||||
|
while building it. Mature and productive developers understand this.
|
||||||
|
|
||||||
|
Choose New Technology, Sometimes.
|
||||||
|
|
||||||
|
Taking this reasoning to its reductio ad absurdum would mean picking
|
||||||
|
Java, and then trying to implement a website without using anything
|
||||||
|
else at all. And that would be crazy. You need some means to add things
|
||||||
|
to your toolbox.
|
||||||
|
|
||||||
|
An important first step is to acknowledge that this is a process, and a
|
||||||
|
conversation. New tech eventually has company-wide effects, so adding
|
||||||
|
tech is a decision that requires company-wide visibility. Your
|
||||||
|
organizational specifics may force the conversation, or [19]they may
|
||||||
|
facilitate developers adding new databases and queues without talking
|
||||||
|
to anyone. One way or another you have to set cultural expectations
|
||||||
|
that this is something we all talk about.
|
||||||
|
|
||||||
|
One of the most worthwhile exercises I recommend here is to consider
|
||||||
|
how you would solve your immediate problem without adding anything new.
|
||||||
|
First, posing this question should detect the situation where the
|
||||||
|
âproblemâ is that someone really wants to use the technology. If that
|
||||||
|
is the case, you should immediately abort.
|
||||||
|
[rmdSx.gif] I just watched a webinar about this graph database, we
|
||||||
|
should try it out.
|
||||||
|
|
||||||
|
It can be amazing how far a small set of technology choices can go. The
|
||||||
|
answer to this question in practice is almost never âwe canât do it,â
|
||||||
|
itâs usually just somewhere on the spectrum of âwell, we could do it,
|
||||||
|
but it would be too hardâ [20][4]. If you think you canât accomplish
|
||||||
|
your goals with what youâve got now, you are probably just not thinking
|
||||||
|
creatively enough.
|
||||||
|
|
||||||
|
Itâs helpful to write down exactly what it is about the current stack
|
||||||
|
that makes solving the problem prohibitively expensive and difficult.
|
||||||
|
This is related to the previous exercise, but itâs subtly different.
|
||||||
|
|
||||||
|
New technology choices might be purely additive (for example: âwe donât
|
||||||
|
have caching yet, so letâs add memcachedâ). But they might also overlap
|
||||||
|
or replace things you are already using. If thatâs the case, you should
|
||||||
|
set clear expectations about migrating old functionality to the new
|
||||||
|
system. The policy should typically be âweâre committed to migrating,â
|
||||||
|
with a proposed timeline. The intention of this step is to keep
|
||||||
|
wreckage at manageable levels, and to avoid proliferating
|
||||||
|
locally-optimal solutions.
|
||||||
|
|
||||||
|
This process is not daunting, and itâs not much of a hassle. Itâs a
|
||||||
|
handful of questions to fill out as homework, followed by a meeting to
|
||||||
|
talk about it. I think that if a new technology (or a new service to be
|
||||||
|
created on your infrastructure) can pass through this gauntlet
|
||||||
|
unscathed, adding it is fine.
|
||||||
|
|
||||||
|
Just Ship.
|
||||||
|
|
||||||
|
Polyglot programming is sold with the promise that letting developers
|
||||||
|
choose their own tools with complete freedom will make them more
|
||||||
|
effective at solving problems. This is a naive definition of the
|
||||||
|
problems at best, and motivated reasoning at worst. The weight of
|
||||||
|
day-to-day operational [21]toil this creates crushes you to death.
|
||||||
|
|
||||||
|
Mindful choice of technology gives engineering minds real freedom: the
|
||||||
|
freedom to [22]contemplate bigger questions. Technology for its own
|
||||||
|
sake is snake oil.
|
||||||
|
|
||||||
|
Update, July 27th 2015: I wrote a talk based on this article. You can
|
||||||
|
see it [23]here.
|
||||||
|
__________________________________________________________________
|
||||||
|
|
||||||
|
1. Etsy in its early years suffered from this pretty badly. We hired a
|
||||||
|
bunch of Python programmers and decided that we needed to find
|
||||||
|
something for them to do in Python, and the only thing that came to
|
||||||
|
mind was creating a pointless middle layer that [24]required years
|
||||||
|
of effort to amputate. Meanwhile, the 90th percentile search
|
||||||
|
latency was about two minutes. [25]Etsy didn't fail, but it went
|
||||||
|
several years without shipping anything at all. So it took longer
|
||||||
|
to succeed than it needed to.
|
||||||
|
2. We often casually refer to the boring/bad intersection of doom as
|
||||||
|
“enterprise software,” but that terminology may be imprecise.
|
||||||
|
3. In saying this Rumsfeld was either intentionally or unintentionally
|
||||||
|
alluding to [26]the Socratic Paradox. Socrates was by all accounts
|
||||||
|
a thoughtful individual in a number of ways that Rumsfeld is not.
|
||||||
|
4. A good example of this from my experience is [27]Etsy’s activity
|
||||||
|
feeds. When we built this feature, we were working pretty hard to
|
||||||
|
consolidate most of Etsy onto PHP, MySQL, Memcached, and Gearman (a
|
||||||
|
PHP job server). It was much more complicated to implement the
|
||||||
|
feature on that stack than it might have been with something like
|
||||||
|
Redis (or [28]maybe not). But it is absolutely possible to build
|
||||||
|
activity feeds on that stack.
|
||||||
|
An amazing thing happened with that project: our attention turned
|
||||||
|
elsewhere for several years. During that time, activity feeds
|
||||||
|
scaled up 20x while nobody was watching it at all. We made no
|
||||||
|
changes whatsoever specifically targeted at activity feeds, but
|
||||||
|
everything worked out fine as usage exploded because we were using
|
||||||
|
a shared platform. This is the long-term benefit of restraint in
|
||||||
|
technology choices in a nutshell.
|
||||||
|
This isn’t an absolutist position--while activity feeds stored in
|
||||||
|
memcached was judged to be practical, implementing full text search
|
||||||
|
with faceting in raw PHP wasn't. So Etsy used Solr.
|
||||||
|
|
||||||
|
[29]Back home
|
||||||
|
|
||||||
|
[30]Tweet [31]Follow @mcfunley
|
||||||
|
|
||||||
|
* [32]GitHub
|
||||||
|
* [33]LinkedIn
|
||||||
|
|
||||||
|
[34]Feed | Copyright © 2004-2023 Dan McKinley.
|
||||||
|
|
||||||
|
References
|
||||||
|
|
||||||
|
1. https://mcfunley.com/feed.xml
|
||||||
|
2. file:///
|
||||||
|
3. https://twitter.com/intent/tweet
|
||||||
|
4. https://twitter.com/mcfunley
|
||||||
|
5. file:///choose-boring-technology
|
||||||
|
6. http://laughingmeme.org/
|
||||||
|
7. file:///data-driven-products-lean-startup-2014
|
||||||
|
8. http://rc3.org/2015/03/24/the-pleasure-of-building-big-things/
|
||||||
|
9. file:///why-mongodb-never-worked-out-at-etsy
|
||||||
|
10. https://consul.io/
|
||||||
|
11. https://www.etsy.com/
|
||||||
|
12. https://stripe.com/
|
||||||
|
13. file:///var/folders/q9/qlz2w5251kzdfgn0np7z2s4c0000gn/T/L21912-441TMP.html#f1
|
||||||
|
14. file:///var/folders/q9/qlz2w5251kzdfgn0np7z2s4c0000gn/T/L21912-441TMP.html#f2
|
||||||
|
15. file:///var/folders/q9/qlz2w5251kzdfgn0np7z2s4c0000gn/T/L21912-441TMP.html#f3
|
||||||
|
16. http://www.evanjones.ca/jvm-mmap-pause.html
|
||||||
|
17. http://martinfowler.com/bliki/PolyglotPersistence.html
|
||||||
|
18. https://twitter.com/coda/status/580531932393504768
|
||||||
|
19. https://twitter.com/mcfunley/status/578603932949164032
|
||||||
|
20. file:///var/folders/q9/qlz2w5251kzdfgn0np7z2s4c0000gn/T/L21912-441TMP.html#f4
|
||||||
|
21. https://twitter.com/handler
|
||||||
|
22. file:///effective-web-experimentation-as-a-homo-narrans
|
||||||
|
23. http://boringtechnology.club/
|
||||||
|
24. https://www.youtube.com/watch?v=eenrfm50mXw
|
||||||
|
25. http://www.sec.gov/Archives/edgar/data/1370637/000119312515077045/d806992ds1.htm
|
||||||
|
26. http://en.wikipedia.org/wiki/I_know_that_I_know_nothing
|
||||||
|
27. https://speakerdeck.com/mcfunley/etsy-activity-feed-architecture
|
||||||
|
28. https://aphyr.com/posts/283-call-me-maybe-redis
|
||||||
|
29. file:///
|
||||||
|
30. https://twitter.com/intent/tweet
|
||||||
|
31. https://twitter.com/mcfunley
|
||||||
|
32. https://github.com/mcfunley
|
||||||
|
33. https://www.linkedin.com/in/mcfunley
|
||||||
|
34. https://mcfunley.com/feed.xml
|
||||||
Reference in New Issue
Block a user