292 lines
17 KiB
Plaintext
292 lines
17 KiB
Plaintext
A Manifesto for Preserving Content on the Web
|
||
|
||
This Page is Designed to Last
|
||
|
||
By [1]Jeff Huang, published 2019-12-19, updated 2021-08-24
|
||
|
||
The end of the year is an opportunity to clean up and reset for the
|
||
upcoming new semester. I found myself clearing out old bookmarks—yes,
|
||
bookmarks: that formerly beloved browser feature that seems to have
|
||
lost the battle to 'address bar autocomplete'. But this nostalgic act
|
||
of tidying led me to despair.
|
||
|
||
Bookmark after bookmark led to dead link after dead link. What's
|
||
vanished: unique pieces of writing on kuro5hin about tech culture; a
|
||
collection of mathematical puzzles and their associated discussion by
|
||
academics that my father introduced me to; Woodman's Reverse
|
||
Engineering tutorials from my high school years, where I first tasted
|
||
the feeling of control over software; even my most recent bookmark, a
|
||
series of posts on Google+ exposing usb-c chargers' non-compliance with
|
||
the specification, all disappeared.
|
||
|
||
This is more than just link rot, it's the increasing complexity of
|
||
keeping alive indie content on the web, leading to a reliance on
|
||
platforms and time-sorted publication formats (blogs, feeds, tweets).
|
||
|
||
Of course, I have also contributed to the problem. A paper I published
|
||
7 years ago has an abstract that includes a demo link, which has been
|
||
taken over by a spammy page with a pumpkin picture on it. Part of that
|
||
lapse was laziness to avoid having to renew and keep a functioning web
|
||
application up year after year.
|
||
|
||
I've recommended my students to push websites to Heroku, and publish
|
||
portfolios on Wix. Yet every platform with irreplaceable content dies
|
||
off some day. Geocities, LiveJournal, what.cd, now Yahoo Groups. One
|
||
day, Medium, Twitter, and even hosting services like GitHub Pages will
|
||
be plundered then discarded when they can no longer grow or cannot find
|
||
a working business model.
|
||
|
||
The problem is multi-faceted. First, content takes effort to maintain.
|
||
The content may need updating to remain relevant, and will eventually
|
||
have to be rehosted. A lot of content, what used to be the vast
|
||
majority of content, was put up by individuals. But individuals (maybe
|
||
you?) lose interest, so one day maybe you just don't want to deal with
|
||
migrating a website to a new hosting provider.
|
||
|
||
Second, a growing set of libraries and frameworks are making the web
|
||
more sophisticated but also more complex. First came jquery, then
|
||
bootstrap, npm, angular, grunt, webpack, and more. If you are a web
|
||
developer who is keeping up with the latest, then that's not a problem.
|
||
|
||
But if not, maybe you are an embedded systems programmer or startup CTO
|
||
or enterprise Java developer or chemistry PhD student, sure you could
|
||
probably figure out how to set up some web server and toolchain, but
|
||
will you keep this up year after year, decade after decade? Probably
|
||
not, and when the next year when you encounter a package dependency
|
||
problem or figure out how to regenerate your html files, you might just
|
||
throw your hands up and zip up the files to deal with "later". Even
|
||
simple technology stacks like static site generators (e.g., Jekyll)
|
||
require a workflow and will stop working at some point. You fall into
|
||
npm dependency hell, and forget the command to package a release. And
|
||
having a website with multiple html pages is complex; how would you
|
||
know how each page links to each other? index.html.old, Copy of
|
||
about.html, index.html (1), nav.html?
|
||
|
||
Third, and this has been touted by others already (and even
|
||
[2]rebutted), the disappearance of the public web in favor of mobile
|
||
and web apps, walled gardens (Facebook pages), just-in-time WebSockets
|
||
loading, and AMP decreases the proportion of the web on the world wide
|
||
web, which now seems more like a continental web than a "world wide
|
||
web".
|
||
|
||
So for these problems, what can we do about it? It's not such a simple
|
||
problem that can be solved in this one article. The Wayback Machine and
|
||
archive.org helps keep some content around for longer. And sometimes an
|
||
altruistic individual rehosts the content elsewhere.
|
||
|
||
But the solution needs to be multi-pronged. How do we make web content
|
||
that can last and be maintained for at least 10 years? As someone
|
||
studying human-computer interaction, I naturally think of the
|
||
stakeholders we aren't supporting. Right now putting up web content is
|
||
optimized for either the professional web developer (who use the latest
|
||
frameworks and workflows) or the non-tech savvy user (who use a
|
||
platform).
|
||
|
||
But I think we should consider both 1) the casual web content
|
||
"maintainer", someone who doesn't constantly stay up to date with the
|
||
latest web technologies, which means the website needs to have low
|
||
maintenance needs; 2) and the crawlers who preserve the content and
|
||
[3]personal archivers, the "archiver", which means the website should
|
||
be easy to save and interpret.
|
||
|
||
So my proposal is seven unconventional guidelines in how we handle
|
||
websites designed to be informative, to make them easy to maintain and
|
||
preserve. The guiding intention is that the maintainer will try to keep
|
||
the website up for at least 10 years, maybe even 20 or 30 years. These
|
||
are not controversial views necessarily, but are aspirations that are
|
||
not mainstream—a manifesto for a long-lasting website.
|
||
1. Return to vanilla HTML/CSS – I think we've reached the point where
|
||
html/css is more powerful, and nicer to use than ever before.
|
||
Instead of starting with a giant template filled with .js includes,
|
||
it's now okay to just write plain HTML from scratch again. CSS
|
||
Flexbox and Grid, canvas, Selectors, box-shadow, the video element,
|
||
filter, etc. eliminate a lot of the need for JavaScript libraries.
|
||
We can avoid jquery and bootstrap when they're not needed. The more
|
||
libraries incorporated into the website, the more fragile it
|
||
becomes. Skip the polyfills and CSS prefixes, and stick with the
|
||
CSS attributes that work across all browsers. And frequently
|
||
validate your HTML; it could save you a headache in the future when
|
||
you encounter a bug.
|
||
2. Don't minimize that HTML – minimizing (compressing) your HTML and
|
||
associated CSS/JS seems like it saves precious bandwidth and all
|
||
the big companies are doing it. But why not? Well, you don't save
|
||
much because your web pages should be gzipped before being sent
|
||
over the network, so preemptively shrinking your content probably
|
||
doesn't do much to save bandwidth if anything at all. But even if
|
||
it did save a few bytes (it's just text in the end), you now need
|
||
to have a build process and to add this to your workflow, so
|
||
updating a website just became more complex. If there's a bug or
|
||
future incompatibility in the html, the minimized form is harder to
|
||
debug. And it's unfriendly to your users; so many people got their
|
||
start with HTML by smashing that View Source button, and minimizing
|
||
your HTML prevents this ideal of learning by seeing what they did.
|
||
Minimizing HTML does not preserve its educational quality, and what
|
||
gets archived is only the resulting codejunk.
|
||
3. Prefer one page over several – several pages are hard to maintain.
|
||
You can lose track of which pages link to what, and it also leads
|
||
to some system of page templates to reduce redundancy. How many
|
||
pages can one person really maintain? Having one file, probably
|
||
just an index.html, is simple and unforgettable. Make use of that
|
||
infinite vertical scroll. You never have to dig around your files
|
||
or grep to see where some content lies. And how should your version
|
||
control that file? Should you use git? Shove them in an 'old/'
|
||
folder? Well I like the simple approach of naming old files with
|
||
the date they are retired, like index.20191213.html. Using the ISO
|
||
format of the date makes it so that it sorts easily, and there's no
|
||
confusion between American and European date formats. If I have
|
||
multiple versions in one day, I would use a style similar to that
|
||
which is customary in log files, of index.20191213.1.html. A nice
|
||
side effect is then you can access an older version of the file if
|
||
you remember the date, without logging into the web host.
|
||
4. End all forms of hotlinking – this cautionary word seems to have
|
||
disappeared from internet vocabulary, but it's one of the reasons
|
||
I've seen a perfectly good website fall apart for no reason. Stop
|
||
directly including images from other websites, stop "borrowing"
|
||
stylesheets by just linking to them, and especially stop linking to
|
||
JavaScript files, even the ones hosted by the original developers.
|
||
Hotlinking is [4]usually considered rude since your visitors use
|
||
someone else's bandwidth, it makes the user experience slower, you
|
||
let another website track your users, and worse of all if the
|
||
location you're linking to changes their folder structure or just
|
||
goes offline, then the failure cascades to your website as well.
|
||
Google Analytics is unnecessary; store your own server logs and set
|
||
up [5]GoAccess or cut them up however you like, giving you more
|
||
detailed statistics. Don't give away your logs to Google for free.
|
||
5. Stick with native fonts – we're focusing on content first, so
|
||
decorative and unusual typefaces are completely unnecessary. Stick
|
||
with either the 13 web-safe fonts or a [6]system font stack that
|
||
matches the default font to the operating system of your visitor.
|
||
Using the system font stack might look a bit different between
|
||
operating systems, but your layout shouldn't be so brittle that an
|
||
extra word wrap will ruin it. Then you don't have to worry about
|
||
the flashing font problem either. Your focus should be about
|
||
delivering the content to the user effectively and making the
|
||
choice of font be invisible, rather than getting noticed to stroke
|
||
your design ego.
|
||
6. Obsessively compress your images – faster for your users, less
|
||
space to archive, and easier to maintain when you don't have to
|
||
back up a humongous folder. Your images can have the same high
|
||
quality, but be smaller. [7]Minify your SVGs, losslessly compress
|
||
your PNGs, generate JPEGs to exactly fit the width of the image.
|
||
It's worth spending some time figuring out the most optimal way to
|
||
compress and [8]reduce the size of your images without losing
|
||
quality. And once [9]WebP gains support on Safari, switch over to
|
||
that format. Ruthlessly minimize the total size of your website and
|
||
keep it as small as possible. Every MB can cost someone real money,
|
||
and in fact, my mobile carrier (Google Fi) charges a cent per MB,
|
||
so a 25 MB website which is fairly common nowadays, costs a quarter
|
||
itself, about as much as a newspaper when I was a child.
|
||
7. Eliminate the broken URL risk – there are [10]monitoring services
|
||
that will tell you when your URL is down, preventing you from
|
||
realizing one day that your homepage hasn't been loading for a
|
||
month and the search engines have deindexed it. Because 10 years is
|
||
longer than most hard drives or operating systems are meant to
|
||
last. But to eliminate the risk of a URL breaking completely, set
|
||
up a second monitoring service. Because if the first one stops for
|
||
any reason (they move to a pay model, they shut down, you forget to
|
||
renew something, etc.) you will still get one notification when
|
||
your URL is down, then realize the other monitoring service is down
|
||
because you didn't get the second notification. Remember that we're
|
||
trying to keep something up for over 10 years (ideally way longer,
|
||
even 30 years), and a lot of services will shut down during this
|
||
period, so two monitoring services is safer.
|
||
|
||
After doing these things, go ahead and place a bit of text in the
|
||
footer, "The page was designed to last", linking to this page
|
||
explaining what that means. The words promise that the maintainer will
|
||
do their best to follow the ideas in this manifesto.
|
||
|
||
Before you protest, this is obviously not for web applications. If you
|
||
are making an application, then make your web or mobile app with the
|
||
workflow you need. I don't even know any web applications that have
|
||
remained similarly functioning over 10 years so it seems like a lost
|
||
cause anyway (except Philip Guo's python tutor, due to his
|
||
[11]minimalist strategy for maintaining it). It's also not for websites
|
||
maintained by an organization like Wikipedia or Twitter. The salaries
|
||
for an IT team is probably enough to keep a website alive for a while.
|
||
|
||
In fact, it's not even that important you strictly follow the 7
|
||
"rules", as they're more of a provocation than strict rules.
|
||
|
||
But let's say some small part of the web starts designing websites to
|
||
last for content that is meant to last. What happens then? Well, people
|
||
may prefer to link to them since they have a promise of working in the
|
||
future. People more generally may be more mindful of making their pages
|
||
more permanent. And users and archivers both save bandwidth when
|
||
visiting and storing these pages.
|
||
|
||
The effects are long term, but the achievements are incremental and can
|
||
be implemented by website owners without being dependent on anyone else
|
||
or waiting for a network effect. You can do this now for your website,
|
||
and that already would be a positive outcome. Like using a recycled
|
||
shopping bag instead of a taking a plastic one, it's a small individual
|
||
action.
|
||
|
||
This article is meant to provoke and lead to individual action, not
|
||
propose a complete solution to the decaying web. It's a small simple
|
||
step for a complex sociotechnical system. So I'd love to see this
|
||
happen. I intend to keep this page up for at least 10 years.
|
||
|
||
If you are interested in receiving updates to [12]irchiver, our project
|
||
for a personal archive of the web pages you visit, please [13]subscribe
|
||
here.
|
||
|
||
Thanks to my Ph.D. students Shaun Wallace, Nediyana Daskalova, Talie
|
||
Massachi, Alexandra Papoutsaki, my colleagues James Tompkin, Stephen
|
||
Bach, my teaching assistant Kathleen Chai, and my research assistant
|
||
Yusuf Karim for feedback on earlier drafts.
|
||
|
||
See discussions on [14]Hacker News and [15]reddit /r/programming
|
||
|
||
Also in this series
|
||
|
||
[16]Behind the scenes: the struggle for each paper to get published
|
||
|
||
[17]Illustrative notes for obsessing over publishing aesthetics
|
||
|
||
Other articles I've written
|
||
|
||
[18]My productivity app is a never-ending .txt file
|
||
|
||
[19]The Coronavirus pandemic has changed our sleep behavior
|
||
|
||
[20]Extracting data from tracking devices by going to the cloud
|
||
|
||
[21]CS Faculty Composition and Hiring Trends
|
||
|
||
[22]Bias in Computer Science Rankings
|
||
|
||
[23]Who Wins CS Best Paper Awards?
|
||
|
||
[24]Verified Computer Science Ph.D. Stipends
|
||
|
||
This page is [25]designed to last.
|
||
|
||
References
|
||
|
||
1. https://jeffhuang.com/
|
||
2. https://gomakethings.com/the-web-is-not-dying/
|
||
3. https://archivebox.io/
|
||
4. https://webmasters.stackexchange.com/questions/25315/hotlinking-what-is-it-and-why-shouldnt-people-do-it
|
||
5. https://goaccess.io/
|
||
6. https://systemfontstack.com/
|
||
7. https://victorzhou.com/blog/minify-svgs/
|
||
8. https://evilmartians.com/chronicles/images-done-right-web-graphics-good-to-the-last-byte-optimization-techniques
|
||
9. https://caniuse.com/#feat=webp
|
||
10. https://uptimerobot.com/
|
||
11. https://pg.ucsd.edu/publications/Python-Tutor-scalable-sustainable-research-software_UIST-2021.pdf
|
||
12. https://irchiver.com/
|
||
13. https://docs.google.com/forms/d/e/1FAIpQLSeTCgnwF1gjrc1O8mfJ_5TmT_TLowFQ2DUhsollmqPG84pAFQ/viewform?usp=pp_url&entry.1299571007=Irchiver:+browser+history+search&entry.1760653896=designed_to_last
|
||
14. https://news.ycombinator.com/item?id=21840140
|
||
15. https://www.reddit.com/r/programming/comments/ed88ra/this_page_is_designed_to_last_a_manifesto_for/
|
||
16. https://jeffhuang.com/struggle_for_each_paper/
|
||
17. https://jeffhuang.com/illustrative-notes-for-publishing-aesthetics/
|
||
18. https://jeffhuang.com/productivity_text_file/
|
||
19. https://jeffhuang.com/covid_sleep/
|
||
20. https://jeffhuang.com/extracting_data_from_tracking_devices/
|
||
21. https://jeffhuang.com/computer-science-open-data/#cs-faculty-composition-and-hiring-trends
|
||
22. https://jeffhuang.com/computer-science-open-data/#bias-in-computer-science-rankings
|
||
23. https://jeffhuang.com/computer-science-open-data/#who-wins-cs-best-paper-awards
|
||
24. https://jeffhuang.com/computer-science-open-data/#verified-computer-science-phd-stipends
|
||
25. http://jeffhuang.com/designed_to_last/
|