www-wrecka-ge-rdi1xr.txt (23459B)
1 [1]wreckage/salvage 2 [2][ ] 3 4 • [3]about 5 • [4]posts 6 7 • [5]sign up (free or paid) 8 • [6]sign in 9 • Search 10 11 Bad shape 12 21 days ago by [7]Erin Kissane — 11 min read 13 14 Bad shape 15 16 The idea I keep coming back to is that the big platforms, like Dickens' Marley, 17 were dead to begin with, and are now something particularly bad, which is dead 18 on their feet. 19 20 In the game of Go, bad shape is the term for configurations of stones on the 21 game board that are inefficient in achieving their offensive goal (territory 22 capture) and unlikely to achieve their defensive goal (the state of "[8]life"). 23 You can extend a bad shape in a fruitless attempt to make it better, but you'll 24 generally be wasting your time. 25 26 The idea I keep coming back to is that the big platforms, [9]like Marley, were 27 dead to begin with, and are now something particularly bad, which is dead on 28 their feet. Not because they’re been abandoned by users (yet) but because 29 they’re structurally incapable of governing the systems they made, and most of 30 the things they try to do about it introduce more and weirder problems. 31 32 While they were still gobbling hundreds of millions of new users each year—and 33 while the old political machines were still catching up—platforms could outrun 34 the problem. Now, though, the number of remaining uncaptured humans dwindles, 35 the politicians and propagandists have adapted to exploit the mass-scale 36 machinery the platforms provide, and the positions platform companies have 37 contorted themselves into trying to shoehorn governance into ungovernable 38 structures are increasingly hard to maintain. 39 40 Facebook especially is likely to zombie-shamble along for some time, held 41 upright by its deep weave into the coordination of offline life and by [10]the 42 communities for whom it serves as a sole accessible connection to the internet, 43 but the whole apparatus looks increasingly precarious. 44 45 (These are very simple points, but it remains a wince-inducing faux pas to say 46 them in a lot of tech-thinking spaces, so I will keep pushing on the obvious.) 47 48 The evidence of the past decade and a half argues strongly that platform 49 corporations are structurally incapable of good governance, primarily because 50 most of their central aims (continuous growth, market dominance, profit via 51 extraction) conflict with many basic human and societal needs. 52 53 As entities, large social platforms continuously undergo rapid mutations in 54 service of their need to maximize profit and expansion while minimizing the 55 kinds of societal and individual harm that can plausibly cause them regulatory 56 trouble or user disengagement. (The set of things that can cause trouble is 57 also always shifting, as political and cultural spheres influence and are 58 influenced by the platforms.) But platform mutations emerge only within a 59 narrow range of possibilities delineated by the set of decisions considered 60 valid in, roughly speaking, [11]Milton Friedman's model of corporate purpose. 61 62 Within this circumscribed mutation zone, certain goals are able to be named and 63 earnestly pursued ("stop spam" or "eliminate the distribution of CSAM"), even 64 if they're never achieved. Other goals (anything to do with hate speech, 65 incitement to violence, or misinformation, for example) can be named and 66 pursued, but only in ways that don't hinder the workings of the 67 profit-extraction machinery—which mostly means that they come in on the margins 68 and after the fact, as in "[12]after the fact of a genocide that Facebook had 69 years of explicit advance warnings about." Working on the margins and after the 70 fact still matters—less damage is better than more damage—but it means "trust 71 and safety" is kept well clear of the core. 72 73 Again, this is all simple and obvious. A tractor structurally can't spare a 74 thought for the lives of the fieldmice; shouting at the tractor when it 75 destroys their nests is a category error. Business does business. The 76 production line doesn't stop just because a few people lose fingers or lives. 77 And what is a modern corporation but a legal spell for turning reasoning beings 78 into temporarily vacant machines? We know this, which is why we have OSHA and 79 the FAA and the FTC, for now. 80 81 It's no surprise that when prodded by entities with cultural or regulatory 82 power, platforms build more semi-effective AI classifiers, hire more underpaid 83 contract moderators, and temporarily stiffen their unevenly enforced community 84 rules, but then immediately slump back toward their natural form, which appears 85 to be a cartoonishly overgrown early-2000s web forum crammed with soft targets 86 and overrun by trolling, spam, and worse. 87 88 It’s possible to make the argument that sufficiently strong leadership could 89 make even a tech corporation appear to be capable of holding an ethical line, 90 and maybe even capable of accepting slightly smaller profits in service of 91 socially beneficial goals—and that, conversely, the awful people in charge are 92 the main source of the problems. It’s not a very good argument, though, even 93 when I make it myself. 94 95 Yes, X is currently controlled by a bizarrely gibbering billionaire with 96 obvious symptoms of late-stage Mad King disease. Yes, Facebook and 97 Instagram—which control vastly more territory than X—are controlled by a 98 feckless, Tulip-craze-mainlining billionaire with a long history of grudgingly 99 up-regulating governance efforts when under public or governmental pressure and 100 then immediately axeing them when the spotlight moves on. But would these 101 platforms inflict less damage if they were led by people who valued the well 102 being of others? Probably yes, to a degree. Twitter/X has offered a lurid 103 natural experiment, and the changes in X after it moved from Jack Dorsey’s 104 spacey techno-libertarian leadership to Elon Musk’s desperately needy 105 quasi-fascist circus act have been obviously bad. A version of Meta founded and 106 led by someone with a reasonably sharp ethical grounding clearly wouldn’t look 107 much like the real Meta at all. 108 109 On the other hand, TikTok’s social function is reasonably close to Meta and 110 X’s, and the fact that [13]its CEO, Shou Zi Chew, seems like a relatively 111 normal person, doesn’t seem to have correlated with dramatically better 112 performance in eliminating [14]Nazi organizing, [15]genocidal and 113 violence-inciting content, [16]CSAM distribution ([17]archive link), or the 114 kind of [18]semi-pro disinformation that makes it harder for people 115 experiencing natural disasters to understand what’s happening. 116 117 Crucially, more reasonable CEO behavior doesn’t seem to prevent the lower-level 118 and potentially even more destructive social effects of platforms that [19] 119 Henry Farrell persuasively explains from a social theory perspective, or that 120 Renée DiResta memorably calls a “Cambrian explosion of subjective, bespoke 121 realities” in [20]Invisible Rulers.* (I'll do a separate post collecting 122 thoughts on this angle, because it's too important to breeze by.) 123 124 The realities of our moment also work against arguments for the potential of 125 heroic leadership: even apparently level-headed tech executives now appear to 126 understand that the next Trump Administration intends to rule unreasonably and 127 vengefully, and that failure to [21]perform obeisance and [22]make tribute may 128 result in federal interference that could plausibly unmake their companies. 129 130 Those are not risks any global corporation can take, but our oddball lineup of 131 big platform companies is in a special bind. No matter how desperately they 132 want to be seen as neutral utilities, they have functioned, for good and ill, 133 like social and political wrecking balls—and real or feigned misapprehensions 134 about algorithms and censorship notwithstanding, real-world governments 135 understand this. The second coming of Trump makes the situation especially 136 stark, but the underlying dynamics are neither new nor temporary. 137 138 Given that every large platform posing as a public square has put itself into 139 the genuinely untenable situation of acting as a global corporate arbiter of 140 politically hot speech, they will all always be in the gunsights of the world’s 141 least reasonable governments. This was bad enough for the platforms when the 142 least reasonable governments were Putin’s or Erdoğan’s or Modi’s—a truly 143 unreasonable government in control of their home jurisdiction is an existential 144 threat. 145 146 And again, in reality, the corporations are configured to try to address the 147 least political kinds of abuse—CSAM, spam, scams, and a few other forms of 148 inauthentic behavior—and very little else. As a result, they can’t govern more 149 subtle or politicized speech for much longer than I can roll a quarter down a 150 piece of string. 151 152 So what would it take for a corporation to become capable of good governance of 153 things like political speech, incitement to violence and genocide, hate speech, 154 most forms of inauthentic behavior, and platform manipulation? Two things, at 155 least: 156 157 • The ability and willingness to take and hold ethical stances that will be 158 sharply unpopular with large swathes of the people mostly likely to 159 effectively target them with legislation and abuses of power, and 160 • the ability and willingness to devote something approaching the majority of 161 their company’s time, money, and attention to building and running [23] 162 devolved or [24]federated systems for doing high-performance high-context 163 local governance according to those unpopular ethical stances. 164 165 Can you bring yourself to imagine—concretely and in detail—these conditions 166 occurring in the leadership of a global corporation? 167 168 And again, achieving a mode of governance that can appropriately handle those 169 most obvious elements—the hate speech, the network abuse, the inauthentic 170 behavior, all of it—is necessary but not sufficient for reaching something like 171 a healthy equilibrium. The elements of big social platforms that make them 172 attractive and fun and profitable are the same elements that, as currently 173 implemented, turn low-level human behavior patterns around status, belief, 174 conformity, and predation into a high-speed mass-scale mess of fractured 175 publics and realities. 176 177 Two points of clarification: First, I’m not saying “Can’t fix people problems 178 with technology,” which is exactly as true and useful as “Guns don’t kill 179 people, people kill people.” (I used the former in what I thought was a very 180 obviously sarcastic way, but apparently the intent was insufficiently clear.) 181 If a technological system makes human problems worse, you have to fix the 182 system or break it and build a better one. 183 184 Second, none of what I’m trying to get at here is about the intent of people 185 who work on big platforms. Corporate platform trust and safety staff routinely 186 work themselves to the brink of individual illness or collapse to handle what 187 they’re permitted and resourced to handle—which is itself a tiny fraction of 188 what would be necessary to handle to make platforms good. Corporate platform 189 governance by technology companies whose success requires growth and 190 attention-extraction, though, is a bankrupt idea. 191 192 If we briefly isolate the reality of our technological present, it’s hard to 193 find it anything but absurd to expect a corporation to govern global or even 194 local speech for any humanist value of “well.” And no one chose it, exactly, it 195 just happened when the fantasies of the internet as an Apollonian zone of 196 libertarian splendor met the reality of globally connected primate brains under 197 late capitalism. I explicitly blame the connected-computer dream of 198 technologically mediated liberation as cartoonishly exemplified in [25]JP 199 Barlow’s Declaration, which centered on keeping the bad old world of human 200 governance, which it equated with censorship, out of the internet: 201 202 You claim there are problems among us that you need to solve. You use this 203 claim as an excuse to invade our precincts. Many of these problems don't 204 exist. Where there are real conflicts, where there are wrongs, we will 205 identify them and address them by our means. We are forming our own Social 206 Contract. This governance will arise according to the conditions of our 207 world, not yours. Our world is different. 208 209 The governance arose, all right, once the money got real. 210 211 The computer dream’s rapidly evaporating and over-salinated shallows are still 212 keeping the tech industry’s dumbest boats afloat, but the platforms have been 213 scraping bottom for years while their owners slap on layers and layers of 214 patches and bilge-pumps and bucket brigades manned by people from former 215 colonies. The problem isn't (just) turning fact-checking on or off or 216 deactivating a swarm of halfassed AI classifers or ceasing to pretend to act on 217 most reports of misconduct, it's bad shape. 218 219 All of which is to say that yes, Zuckerberg is a terrible chump and Musk is a 220 grotesque quasi-Rasputin, and that does matter, but the boards they stand on 221 have been rotten the whole time. Centralized corporate governance of global 222 mega-platforms was always a goofy idea, and we should have given up on it years 223 ago. 224 225 This is where I get into awkward situations with lovely people, including 226 several I count as friends, because they’re determined "not to let platforms 227 off the hook.” I feel this, deeply, along with things like send the Sacklers to 228 the guillotine. But keeping the fucked-up mutant fish on the hook will not 229 magically transform it into an entity capable of governing. 230 231 — 232 233 Earlier this week at Platformer, [26]Casey Newton reported some insider views 234 on what Meta’s most recent roll-back of content moderation and fact-checking 235 means. The post is worth reading, and after the Myanmar research I did in 2023, 236 and for what it’s worth, I don’t think Casey’s sources overstate the dangers 237 inherent in what Meta’s doing: more real human beings are going to suffer and 238 lose children and be killed because of this. But I want to look at something in 239 the cursory background section of the newsletter, about the work that Meta put 240 in after 2016, when Facebook got criticized for hosting election-interference 241 ops in the US: 242 243 Chastened by the criticism, Meta set out to shore up its defenses. It hired 244 40,000 content moderators around the world, invested heavily in building 245 new technology to analyze content for potential harms and flag it for 246 review, and became the world’s leading funder of third-party fact-checking 247 organizations. It spent $280 million to create an independent Oversight 248 Board to adjudicate the most difficult questions about online speech. It 249 disrupted dozens of networks of state-sponsored trolls who sought to use 250 Facebook, Instagram, and WhatsApp to spread propaganda and attack 251 dissenters. 252 253 [27]According to Financial Times reporting ([28]archive link), Meta currently 254 employs or has contracted with about 40,000 people to work on “safety and 255 security,” of which just 15,000 are content moderators, for a user base of 256 roughly four billion users, which works out to more than a quarter of a million 257 users per moderator. This chimes with [29]New York Times reporting ([30]archive 258 link) suggesting that in 2021, Accenture was billing Facebook for about 5,800 259 full-time contract moderators. (For what it’s worth, in 2017, Meta promised to 260 add all of [31]3,000 trust and safety staff.) Nor are Meta’s moderation 261 resources allocated evenly: About 90% of Facebook users are outside the US and 262 Canada; that overwhelming majority gets [32]approximately 13% of the company’s 263 moderation time ([33]archive link). 264 265 And while we’re here, in 2020—the year Oversight Board started hiring—Meta 266 cleared about $91 billion in profit. The Oversight Board trust got $280 million 267 from Meta, or just over 0.3% of the company’s annual profits. The Oversight 268 Board itself, though inclined to deliver thoughtful if glacially slow 269 recommendations, appears to have accomplished [34]remarkably little. 270 271 Again: The work tens of thousands of people around the world put in to try to 272 make platforms less terrible is real and essential work, and it’s often done at 273 a terrible cost. It’s also the barest gesture at serious governance, and much 274 of it is pure Potemkin Village. 275 276 That’s only a couple of things pulled from one paragraph that happened to hit 277 my inbox while I was drafting this post, but I did [35]a whole lot of that kind 278 of close reading in 2023, and came out believing that platform intensifications 279 of governance in response to periodic governmental pressure are best understood 280 as a little bit of real (though deeply inadequate) change and a whole lot of 281 [36]flopping. Then, when the pressure comes off, the platforms re-orient like 282 compass needles tossed into in an MRI machine. 283 284 — 285 286 [37]Barlow’s Declaration—which is excruciating and which I’ve been making 287 myself reread annually for years as penance for participating in tech 288 culture—ends like this: 289 290 We will create a civilization of the Mind in Cyberspace. May it be more 291 humane and fair than the world your governments have made before. 292 293 What we got instead was a handful of global-scale company towns that continue 294 to prove their comprehensive unfitness to govern and their absolute 295 vulnerability to the offline governments the free internet was meant to work 296 around. 297 298 So sure: [38]Protocols over platforms. Then we have to actually do the 299 inelegant, un-heroic, expensive work of rebuilding the essential structures of 300 human civilization on top of the protocols, because it turns out we just have 301 the one world, online or off, no way out. 302 303 Thank you 304 305 This post, like the others on this site, exists [39]because people have signed 306 on to support the work. If you find it useful, and your situation allows for it 307 with ease, please consider signing up! Enormous thanks to those of you who 308 have. And a note to members: I've wrestled down the Ghost commenting problem 309 and the first real discussion post for project members goes up tomorrow, so if 310 you've signed up for a paid membership, look for that in your inbox soon. 311 312 Notes 313 314 * Renée’s book is very good and I recommend it for its lucid explanations and 315 commitment to drawing on previous eras of mass communication without doing too 316 deeply into either theoretical or historical rabbitholes (which I love, but 317 which don’t make for popular reading). I don’t 100% agree with her conclusions, 318 but they’re clearly stated and cleanly argued, which allows for productive 319 disagreement—and I value that more than full alignment. 320 321 A common response to the things I've been posting is "Okay, but what will work, 322 then?" I think there are hints at answers in the very chunky [40]fediverse 323 governance research I worked on last year, in online and offline cooperatives, 324 in [41]Rudy Fraser's Blacksky, and in the kinds of projects Nathan Schneider 325 assesses in [42]Governable Spaces. I'll continue to explore what I think might 326 be good shapes for governance here in ways that—I hope—will be more pragmatic 327 than quixotic. 328 329 The featured diagram for this post is International Marine Engineering's 1912 330 depiction of the profile and deck of the Titanic ([43]v. 17, p. 199). 331 332 [44] [45] [46] The link has been copied! 333 [47][https://www.wrecka.g] 334 [48] How we'll do discussions here 335 Newer post 336 337 [49]How we'll do discussions here 338 339 Older post 340 341 [50]What people in the global majority need from networks 342 343 [51] What people in the global majority need from networks 344 345 Subscribe to new posts. 346 347 [52][ ] Subscribe 348 Processing your application Great! Check your inbox and confirm your 349 subscription There was an error sending the email 350 choose your type: 351 352 • [54]( ) fancy (default) 353 • [55]( ) simpler 354 355 [56]wreckage/salvage 356 357 Making and mending networks for humans. 358 359 wreckage/salvage © Erin Kissane 2025 360 You’ve successfully subscribed to wreckage/salvage 361 Welcome back! You’ve successfully signed in. 362 Great! You’ve successfully signed up. 363 Success! Your email is updated. 364 Your link has expired 365 Success! Check your email for magic link to sign-in. 366 367 References: 368 369 [1] https://www.wrecka.ge/ 370 [3] https://www.wrecka.ge/about/ 371 [4] https://www.wrecka.ge/tag/posts/ 372 [5] https://www.wrecka.ge/#/portal 373 [6] https://www.wrecka.ge/signin/ 374 [7] https://www.wrecka.ge/author/erin/ 375 [8] https://senseis.xmp.net/?LifeAndDeath 376 [9] https://www.gutenberg.org/files/46/46-h/46-h.htm 377 [10] https://www.theengineroom.org/library/new-report-exploring-a-transition-to-alternative-social-media-platforms-for-social-justice-organizations-in-the-majority-world/ 378 [11] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctrine-the-social-responsibility-of-business-is-to.html 379 [12] https://erinkissane.com/meta-in-myanmar-part-i-the-setup 380 [13] https://www.wired.com/story/shou-zi-chew-tik-tok-big-interview/ 381 [14] https://www.isdglobal.org/digital_dispatches/naztok-an-organized-neo-nazi-tiktok-network-is-getting-millions-of-views/ 382 [15] https://www.isdglobal.org/wp-content/uploads/2021/08/HateScape_v5.pdf 383 [16] https://www.ft.com/content/d5549d48-8f02-464c-9c7d-17404a5b6d02 384 [17] https://archive.ph/tvn8K 385 [18] https://newrepublic.com/article/186928/misinformation-new-normal-disaster-response 386 [19] https://www.programmablemutter.com/p/were-getting-the-social-media-crisis 387 [20] https://www.hachettebookgroup.com/titles/renee-diresta/invisible-rulers/9781541703377/ 388 [21] https://www.newsweek.com/tech-ceos-donations-donald-trump-joe-biden-inaugurations-compared-2010457 389 [22] https://qz.com/google-youtube-trump-inauguration-meta-amazon-apple-1851736124 390 [23] https://en.wikipedia.org/wiki/Devolution 391 [24] https://en.wikipedia.org/wiki/Federalism 392 [25] https://www.eff.org/cyberspace-independence 393 [26] https://www.platformer.news/meta-fact-checking-free-speech-surrender/?ref=platformer-newsletter 394 [27] https://www.ft.com/content/afeb56f2-9ba5-4103-890d-91291aea4caa 395 [28] https://archive.ph/UxgFy 396 [29] https://www.nytimes.com/2021/08/31/technology/facebook-accenture-content-moderation.html 397 [30] https://archive.ph/SEZpr 398 [31] https://www.nytimes.com/2017/05/03/technology/facebook-moderators-q1-earnings.html 399 [32] https://www.wsj.com/articles/facebook-drug-cartels-human-traffickers-response-is-weak-documents-11631812953 400 [33] https://archive.ph/https://www.wsj.com/articles/facebook-drug-cartels-human-traffickers-response-is-weak-documents-11631812953 401 [34] https://www.lawfaremedia.org/article/meta-s-oversight-board-and-the-need-for-a-new-theory-of-online-speech 402 [35] https://erinkissane.com/meta-in-myanmar-part-iii-the-inside-view 403 [36] https://www.youtube.com/watch?v=Ml8kf3UIpN0 404 [37] https://www.eff.org/cyberspace-independence 405 [38] https://knightcolumbia.org/content/protocols-not-platforms-a-technological-approach-to-free-speech 406 [39] https://www.wrecka.ge/#/portal 407 [40] https://fediverse-governance.github.io/ 408 [41] https://www.wired.com/story/blacksky-is-nothing-like-black-twitter/ 409 [42] https://www.ucpress.edu/books/governable-spaces/epub-pdf 410 [43] https://archive.org/details/internationalma171912newy/page/198/mode/2up 411 [44] https://tootpick.org/#text=%22Bad%20shape%22%20https://www.wrecka.ge/bad-shape/ 412 [45] https://bsky.app/intent/compose?text=%22Bad%20shape%22+https://www.wrecka.ge/bad-shape/ 413 [46] javascript: 414 [48] https://www.wrecka.ge/how-we-do-it-here/ 415 [49] https://www.wrecka.ge/how-we-do-it-here/ 416 [50] https://www.wrecka.ge/what-people-in-the-global-majority-need-from-networks/ 417 [51] https://www.wrecka.ge/what-people-in-the-global-majority-need-from-networks/ 418 [56] https://www.wrecka.ge/