p-migdal-pl-4of5up.txt (12716B)
1 [1]Piotr Migdał[2]Blog[3]Projects[4]Publications[5]Resume 2 3 If it is worth keeping, save it in Markdown 4 5 17 Feb 2025 | by Piotr Migdał 6 7 • [6]r/DataHoarder thread 8 • [7]r/ObisdianMD thread 9 • [8]Hacker News front page 10 11 One of Stanisław Lem's stories, [9]The Memoirs Found in a Bathtub, begins with 12 a strange phenomenon that turns all written materials into dust. While this is 13 science fiction, something similar happens in our digital world. 14 15 [10]Digital memento mori 16 17 If you publish something online, sooner or later, it will vanish.^[11]1 18 19 In the best-case scenario, a link changes during website restructuring. More 20 commonly, the content is lost. The only hope is that someone saved it from 21 oblivion in the [12]Internet Archive Wayback Machine. 22 23 Walled gardens requiring login are even worse - when they go down, everything 24 within them vanishes forever. If you haven't saved it yourself, it's gone. 25 Moreover, any service (free or paid) may restrict access to content at any time 26 - either completely or practically, by making it impossible to find what you're 27 looking for. The same content you posted on Twitter a few years ago, now is on 28 X, and in a few years might be available after login, paid subscription, or - 29 not at all . 30 31 Even self-hosting isn't foolproof - your content can vanish when you forget to 32 pay for hosting or after a server crash. And even if your data survives, 33 accessing it can be tricky: WordPress blogs store posts in databases that 34 server updates can break. I learned this lesson when my PHP photo gallery went 35 down - thankfully, I had kept all photos as simple JPGs organized by date. 36 37 The only reliable solution is to store content in formats that can be opened 38 without specialized software - formats that will remain accessible for decades 39 to come. 40 41 [galadriel-] 42 Galadriel in "the Lord of the Rings" opening scene ([13]video, [14]transcript). 43 44 [15]Why things are worth saving 45 46 There are many motivations for preserving content, ranging from a digital "non 47 omnis moriar" through practical arguments, to archiving as a goal in itself^ 48 [16]2. 49 50 For me, the key reasons are: 51 52 • I want to keep and own things I wrote - they are parts of me, my history, 53 my lived experience 54 • I want to have everything in one place and easily searchable 55 • I want to use it with AI tools (looking for similar notes, summarizing, 56 using as context) 57 • I want to be able to reuse or share things however I want (email, blog 58 post, ebook, anything) 59 60 [17]Plaintext 61 62 As a data scientist, [18]I turn things into vectors. 63 As an unabashed archivist, I turn things into Markdown. 64 65 The most durable solution would be carving things in stone - it would last for 66 millennia. But that's hardly practical, and it wouldn't make things easily 67 searchable or shareable. 68 69 The second best option is plaintext files with UTF-8 encoding and Markdown 70 formatting^[19]3. As long as computers exist, we'll be able to read plaintext 71 files with ease. 72 73 Markdown files are essentially plaintext with some extra syntax for common 74 elements like sections, bullet points, and links. The format deliberately 75 avoids precise control over display details like font selection^[20]4. 76 Following [21]the rule of least power, I consider this limitation a feature. 77 For contrast, consider PDF - a format so powerful that [22]it can run Doom. 78 79 For personal notes, I use [23]Obsidian, a note-taking app I love and use daily. 80 While it's a powerful tool with great plugins, what keeps me loyal is its 81 simplicity - it stores everything in plain files. The lack of a proprietary 82 format moat is precisely what makes it so compelling. 83 84 For blogging, most [24]static site generators embrace Markdown. This very blog 85 post is written in Markdown^[25]5. Using the same markup for note-taking and 86 publishing makes sharing smooth. 87 88 [26]How I do it 89 90 I dream of automatically converting everything I write or encounter into 91 Markdown. The reality is messier - there's a constant tension between my 92 autistic urge to archive everything and my ADHD that makes maintaining such 93 systems challenging. 94 95 So I take a pragmatic approach - when I find content worth keeping, I copy it 96 to a markdown file, adding frontmatter with its publication date, source, and 97 relevant tags: 98 99 [sauna-post] 100 101 I particularly save things I post that might be useful later. Conference talk 102 abstracts, sauna event descriptions, technical explanations - in the future, 103 they're much easier to find and reuse. 104 105 When I catch myself searching for old content (like a Facebook post I want to 106 share or reread), I save it immediately. If I discover a blog post has 107 vanished, I retrieve it from the Wayback Machine and preserve it. When 108 forwarding an email with a detailed explanation - you guessed it, I save it. 109 110 Content worth searching for once is content worth preserving forever. 111 112 Worried about saving too much? Well, disk storage is cheap - and for text 113 files, it's practically free. 114 115 [27]Tools that help 116 117 Sometimes manual copying suffices. For trickier formatting, AI tools are 118 invaluable - being trained on Markdown, they excel at processing and extracting 119 content. You can use them to convert online text or parse PDFs (like slides), 120 as shown in [28]Ingesting Millions of PDFs and why Gemini 2.0 Changes 121 Everything. 122 123 For some sources, I've created semi-automated solutions. For instance, I wrote 124 a [29]Python script to convert my Kindle highlights and notes into Markdown. 125 126 Many tools exist to help with format conversion. The most versatile is [30] 127 pandoc, which can convert between dozens of formats - from Word documents to 128 LaTeX, and everything in between. 129 130 The community has also created specialized tools for specific platforms. You 131 can find tools for converting [31]Medium posts to Markdown (either from export 132 or [32]directly by URL), [33]archiving Reddit threads, and many other use 133 cases. 134 135 Since we're dealing with lightweight text files, there are many for backing it 136 up. Git is particularly well-suited for version-controlling and syncing this 137 content. 138 139 Additionally, in each service I own, I periodically download my data. Even if 140 it's a mesh of JSON, XML, HTML, CSV and other formats, I have it. Even if at a 141 given moment I have no time to process it into Markdown, at least the data is 142 there. 143 144 [34]Next steps 145 146 I would love to have a comprehensive tool for exporting everything - especially 147 from social media. Both the posts that resonated with many people and those 148 that hold personal significance deserve preservation. 149 150 While Facebook offers limited data export capabilities, they're incomplete. 151 Most notably, there's no way to preserve entire discussion threads - often the 152 most valuable part of a post. 153 154 And you - what content do you find yourself searching for? What have you 155 archived, and what do you wish you had saved? 156 157 Discuss this post on [35]Hacker News, [36]Mastodon, [37]Reddit, or [38]LinkedIn 158 . 159 160 [39]Footnotes 161 162 1. [40]Link rot can be addressed using services like [41]Perma.cc - though 163 they too could eventually disappear. Studies show that for legal documents, 164 half of links die within 5 years. My focus here is on preserving and 165 searching personal content. [42]↩ 166 2. But for practical reasons, and hoarding for its own sake, I gathered over 167 14k links in [43]Pinboard. Yes, downloaded data in JSON. [44]↩ 168 3. I don't claim Markdown is the only solution. There are valid reasons to use 169 other formats. My focus is on plaintext in UTF-8. If you prefer other 170 markup languages (like reStructuredText, AsciiDoc, Org-Mode) or just plain 171 text without formatting - the principles still apply. In some cases 172 original format works - e.g. if it is JSON or code. [45]↩ 173 4. Consider HTML (Hypertext Markup Language) as a counterexample. It was meant 174 to enrich text with semantics, but now serves primarily as a tool for 175 building UIs. While this evolution brought many benefits, typical end-user 176 HTML is no longer suitable for pure content storage. At the same time, if 177 you can use simple HTML with actual semantic <strong> and <em> tags, go for 178 it. But it's often a slippery slope - from "just add a few colors," through 179 "add tables," to creating a full-fledged app. [46]↩ 180 5. This blog uses [47]Nuxt 3 Content (source: [48]github.com/stared/ 181 stared.github.io). It follows my previous versions in [49]Jekyll and [50] 182 Gridsome. Thanks to Markdown, migration between platforms has been seamless 183 - see [51]New blog - moving from Medium to Gridsome. For the latest 184 migration from Gridsome to Nuxt 3 Content, [52]Cursor IDE was a great help. 185 [53]Astro is another static site generator gaining significant traction. 186 [54]↩ 187 188 See also cosine-similar posts 189 190 • 0.617[55]New blog - moving from Medium to Gridsome 191 • 0.604[56]How I learned to stop worrying and love the types & tests 192 • 0.598[57]AI won’t make artists redundant - thanks to information theory 193 • 0.591[58]ADHD tech stack: auto time tracking 194 • 0.589[59]The first post: why Jekyll? 195 196 By [60]Piotr Migdał, a curious being, doctor of sorcery. See [61]my other blog 197 posts. 198 199 Keep in the loop with the [62]RSS feed or join the [63]newsletter. 200 201 202 References: 203 204 [1] https://p.migdal.pl/ 205 [2] https://p.migdal.pl/blog 206 [3] https://p.migdal.pl/projects 207 [4] https://p.migdal.pl/publications 208 [5] https://p.migdal.pl/resume 209 [6] https://www.reddit.com/r/DataHoarder/comments/1is1wbn/if_it_is_worth_keeping_save_it_in_markdown/ 210 [7] https://www.reddit.com/r/ObsidianMD/comments/1is1snu/if_it_is_worth_keeping_save_it_in_markdown/ 211 [8] https://news.ycombinator.com/item?id=43137616 212 [9] https://en.wikipedia.org/wiki/Memoirs_Found_in_a_Bathtub 213 [10] https://p.migdal.pl/blog/2025/02/markdown-saves/#digital-memento-mori 214 [11] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fn-link-rot 215 [12] https://web.archive.org/ 216 [13] https://www.youtube.com/watch?v=qj139dE7tFI 217 [14] https://www.tk421.net/lotr/film/fotr/01.html 218 [15] https://p.migdal.pl/blog/2025/02/markdown-saves/#why-things-are-worth-saving 219 [16] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fn-pinboard 220 [17] https://p.migdal.pl/blog/2025/02/markdown-saves/#plaintext 221 [18] https://p.migdal.pl/blog/2025/01/dont-use-cosine-similarity 222 [19] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fn-plaintext 223 [20] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fn-html 224 [21] https://en.wikipedia.org/wiki/Rule_of_least_power 225 [22] https://www.reddit.com/r/itrunsdoom/comments/1i02c6b/doom_in_a_pdf_file/ 226 [23] https://obsidian.md/ 227 [24] https://jamstack.org/generators/ 228 [25] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fn-blog 229 [26] https://p.migdal.pl/blog/2025/02/markdown-saves/#how-i-do-it 230 [27] https://p.migdal.pl/blog/2025/02/markdown-saves/#tools-that-help 231 [28] https://www.sergey.fyi/articles/gemini-flash-2 232 [29] https://gist.github.com/stared/ce732ef27d97d559b34d7e294481f1b0 233 [30] https://github.com/jgm/pandoc 234 [31] https://github.com/gautamdhameja/medium-2-md 235 [32] https://medium2md.nabilmansour.com/ 236 [33] https://farnots.github.io/RedditToMarkdown/ 237 [34] https://p.migdal.pl/blog/2025/02/markdown-saves/#next-steps 238 [35] https://news.ycombinator.com/item?id=43137616 239 [36] https://mathstodon.xyz/@pmigdal/114021315189570737 240 [37] https://www.reddit.com/r/DataHoarder/comments/1is1wbn/if_it_is_worth_keeping_save_it_in_markdown/ 241 [38] https://www.linkedin.com/posts/piotrmigdal_if-it-is-worth-keeping-save-it-in-markdown-activity-7299139148634841089-_Xe3 242 [39] https://p.migdal.pl/blog/2025/02/markdown-saves/#footnote-label 243 [40] https://en.wikipedia.org/wiki/Link_rot 244 [41] https://perma.cc/ 245 [42] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fnref-link-rot 246 [43] https://pinboard.in/ 247 [44] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fnref-pinboard 248 [45] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fnref-plaintext 249 [46] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fnref-html 250 [47] https://content.nuxt.com/ 251 [48] https://github.com/stared/stared.github.io 252 [49] https://jekyllrb.com/ 253 [50] https://gridsome.org/ 254 [51] https://p.migdal.pl/blog/2022/12/medium-to-markdown 255 [52] https://www.cursor.com/ 256 [53] https://astro.build/ 257 [54] https://p.migdal.pl/blog/2025/02/markdown-saves#user-content-fnref-blog 258 [55] https://p.migdal.pl/blog/2022/12/medium-to-markdown 259 [56] https://p.migdal.pl/blog/2020/03/types-tests-typescript 260 [57] https://p.migdal.pl/blog/2023/02/ai-artists-information-theory 261 [58] https://p.migdal.pl/blog/2020/05/adhd-tech-stack-auto-time-tracking 262 [59] https://p.migdal.pl/blog/2015/12/first-post 263 [60] https://p.migdal.pl/ 264 [61] https://p.migdal.pl/blog 265 [62] https://p.migdal.pl/feed.xml 266 [63] https://eepurl.com/bVJlgL