www-oneusefulthing-org-kop2ys.txt (20249B)
1 [1][https] 2 3 [2]One Useful Thing 4 5 SubscribeSign in 6 Share this post 7 [https] 8 9 Thinking Like an AI 10 11 www.oneusefulthing.org 12 Copy link 13 Facebook 14 Email 15 Note 16 Other 17 18 Thinking Like an AI 19 20 A little intuition can help 21 22 [13][https] 23 [14]Ethan Mollick 24 Oct 20, 2024 25 528 26 Share this post 27 [https] 28 29 Thinking Like an AI 30 31 www.oneusefulthing.org 32 Copy link 33 Facebook 34 Email 35 Note 36 Other 37 [21] 38 60 39 41 40 [22] 41 Share 42 43 This is my 100th post on this Substack, which got me thinking about how I could 44 summarize the many things I have written about how to use AI. I came to the 45 conclusion that [23]the advice in my book is still the advice I would give: 46 just use AI to do stuff that you do for work or fun, for about 10 hours, and 47 you will figure out a remarkable amount. 48 49 However, I do think having a little bit of intuition about the way Large 50 Language Models work can be helpful for understanding how to use it best. I 51 would ask my technical readers for their forgiveness, because I will simplify 52 here, but here are some clues for getting into the “mind” of an AI: 53 54 LLMs do next token prediction 55 56 Large Language Models are, ultimately, incredibly sophisticated autocomplete 57 systems. They use a vast model of human language to predict the next token in a 58 sentence. For models working with text, tokens are words or parts of words. 59 Many common words are single tokens, or tokens containing spaces, but other 60 words are broken into multiple tokens. For example, one tokenizer takes the 10 61 word sentence, “This breaks up words (even phantasmagorically long words) into 62 tokens” into 20 tokens. 63 64 [25] 65 [https] 66 67 When you give an AI a prompt, you are effectively asking it to predict the next 68 token that would come after the prompt. The AI then takes everything that has 69 been written before, runs it through a mathematical model of language, and 70 generates the probability of which token is likely to come next in the 71 sequence. For example, if I write “The best type of pet is a” the LLM predicts 72 that the most likely tokens to come next, based on its model of human language, 73 are either “dog”, “personal,” “subjective,” or “cat.” The most likely is 74 actually dog, but LLMs are generally set to include some randomness, which is 75 what makes LLM answers interesting, so it does not always pick the most likely 76 token (in most cases, even attempts to eliminate this randomness cannot remove 77 it entirely). Thus, I will often get “dog,” but I may get a different word 78 instead. 79 80 [26] 81 [https] 82 These are the actual probabilities from GPT-3.5, as are the other examples in 83 this post. 84 85 But these predictions take into account everything in the memory of the LLM 86 (more on memory in a bit), and even tiny changes can radically alter the 87 predictions of what token comes next. I created three examples with minor 88 changes on the original sentence. If I choose not to capitalize the first word, 89 the model now says that “dog” and “cat” are much more likely answers than they 90 were originally, and “fish” joins the top three. If I change the word “type” to 91 “kind” in the sentence, the probabilities of all the top tokens drop and I am 92 much more likely to get an exotic answer like “calm” or “bunny.” If I add an 93 extra space after the word “pet,” then “dog” isn’t even in the top three 94 predicted tokens! 95 96 [27] 97 [https] 98 99 But the LLM does not just produce one token, instead, after each token, it now 100 looks at the entire original sentence plus the new token (“The best type of pet 101 is a dog”) and predicts the next token after that, and then uses that whole 102 sentence plus the next to make a prediction, and so on. It chains one token to 103 another like cars on a train. Current LLMs can’t go back and change a token 104 that came before, they have to soldier on, adding word after word. This results 105 in a butterfly effect. If the first predicted token was the word “dog” than the 106 rest of the sentence will follow on like that, if it is “subjective” then you 107 will get an entirely different sentence. Any difference between the tokens in 108 two different answers will result in radically diverging responses. 109 110 [28] 111 [https] 112 113 The intuition: This helps explain why you may get very different answers than 114 someone else using the same AI, even if you ask exactly the same question. Tiny 115 differences in probabilities result in very different answers. It also gives 116 you a sense about why one of the biases that people worry about with AI is that 117 it may respond differently to people depending on their writing style, as the 118 probabilities for the next token may lead on the path to worse answers. Indeed, 119 [29]some of the early LLMs gave less accurate answers if you wrote in a less 120 educated way. 121 122 You can also see some of why hallucinations happen, and why they are so 123 pernicious. The AI is not pulling from a database, it is guessing the next word 124 based on statistical patterns in its training data. That means that what it 125 produces is not necessarily true (in fact, one of many surprises about LLMs are 126 how often they are right, given this), but, even when it provides false 127 information, it likely sounds plausible. That makes it hard to tell when it is 128 making things up. 129 130 It is also helpful to think about tokens to understand why AIs get stubborn 131 about a topic. If the first prediction is “dog” the AI is much more likely to 132 keep producing text about how great dogs are because those tokens are more 133 likely. However, if it is “subjective” it is less likely to give you an 134 opinion, even when you push it. Additionally, once the AI has written 135 something, it cannot go back, so it needs to justify (or explain or lie about) 136 that statement in the future. I like this example that [30]Rohit Krishnan [31] 137 shared, where you can see the AI makes an error, but then attempts to justify 138 the results. 139 140 [32] 141 [https] 142 143 The caveat: Saying “AI is just next-token prediction” is a bit of a joke 144 online, because it doesn’t really help us understand why AI can produce such 145 seemingly creative, novel, and interesting results. If you have been reading my 146 posts for any length of time, you will realize that AI accomplishes impressive 147 outcomes that, intuitively, we would not expect from an autocomplete system. 148 149 [33] 150 [https] 151 Claude makes themed Excel formulas on demand and explains them in delightful 152 ways. Next token prediction is capable of lots of unexpected results. 153 154 LLMs make predictions based on their training data 155 156 Where does an LLM get the material on which it builds a model of language? From 157 the data it was trained on. Modern LLMs are trained over an incredibly vast set 158 of data, incorporating large amounts of the web and every free book or archive 159 possible (plus some archives that almost certainly contain copyrighted work). 160 The AI companies largely did not ask permission before using this information, 161 but leaving aside the legal and ethical concerns, it can be helpful to 162 conceptualize the training data. 163 164 The original [35]Pile dataset, which most of the major AI companies used for 165 training, is about 1/3 based on the internet, 1/3 on scientific papers, and the 166 rest divided up between books, coding, chats, and more. So, your intuition is 167 often a good guide - if you expect something was on the internet or in the 168 public domain, it is likely in the training data. But we can get a little more 169 granular. For example, [36]thanks to this study, we have a rough idea of which 170 fiction books appear most often in the training data for GPT-4, which largely 171 tracks the books most commonly found on the web (many of the top 20 are out of 172 copyright, with a couple notable exceptions of books that are much pirated). 173 174 [37] 175 [https] 176 177 Remember that LLMs use a statistical model of language, they do not pull from a 178 database. So the more common a piece of work is in the training data, the more 179 likely the AI is to “recall” that data accurately when prompted. You can see 180 this at work when I give it a sentence from the most fiction common book in its 181 training data - Alice in Wonderland. It gets the next sentence exactly right, 182 and you can see that almost every possible next token would continue along the 183 lines of the original passage. 184 185 [38] 186 [https] 187 188 Let’s try something different, a passage from a fairly obscure mid-century 189 science fiction author, [39]Cordwainer Smith, with an unusual writing style in 190 part shaped by his time in China (he was Sun Yat-sen’s godson) and his 191 knowledge of multiple languages. One of his stories starts: Go back to An-fang, 192 the Peace Square at An-fang, the Beginning Place at An-fang, where all things 193 start. It then continues: Bright it was. Red square, dead square, clear square, 194 under a yellow sun. If I give the AI the first section, looking at the 195 probabilities, there is almost no chance that it will produce the correct next 196 word “Bright.” Instead, perhaps primed by the mythic language and the fact that 197 An-fang registers as potentially Chinese (it is actually a play on the German 198 word for beginning), it creates a passage about a religious journey. 199 200 [40] 201 [https] 202 203 The intuition: The fact that the LLM does not directly recall text would be 204 frustrating if you were trying to use an LLM like Google, but LLMs are not like 205 Google. They are capable of producing original material, and, even when they 206 attempt to give you Alice in Wonderland word-for-word, small differences will 207 randomly appear and eventually the stories will diverge. However, knowing what 208 is in the training data can help you in a number of ways. 209 210 First, it can help you understand what the AI is good at. Any document or 211 writing style that is common in its training data is likely something the AI is 212 very good at producing. But, more interestingly, it can help you think about 213 how to get more original work from the AI. By pushing it through your prompts 214 to a more unusual section of its probability space, you will get very different 215 answers than other people. Asking AI to write a memo in the style of [41]Walter 216 Pater will give you more interesting answers (and overwrought ones) than asking 217 for a professional memo, of which there are millions in the training data. 218 219 [42] 220 [https] 221 222 The caveat: Contrary to some people's beliefs, the AI is rarely producing 223 substantial text from its training data verbatim. The sentences the AI provides 224 are usually entirely novel, extrapolated from the language patterns it learned. 225 Occasionally, the model might reproduce a specific fact or phrase it memorized 226 from its training data, but more often, it's generalizing from learned patterns 227 to produce new content. 228 229 Outside of training, carefully crafted prompts can guide the model to produce 230 more original or task-specific content, demonstrating a capability known as 231 “in-context learning.” This allows LLMs to appear to learn new tasks within a 232 conversation, even though they're not actually updating their underlying model, 233 as you will see. 234 235 LLMs have a limited memory 236 237 Given how much we have discussed training, it may be surprising to learn that 238 AIs are not generally learning anything permanent from their conversations with 239 you. Training is usually a discrete event, not something that happens all the 240 time. If you have privacy features turned on, your chats are not being fed into 241 the training data at all, but, even if your data will be used for training, the 242 training process is not continuous. Instead, chats happen within what's called 243 a 'context window'. This context window is like the AI's short-term memory - 244 it's the amount of previous text the AI can consider when generating its next 245 response. As long as you stay in a single chat session and the conversation 246 fits inside the context window, the AI will keep track of what is happening, 247 but as soon as you start a new chat, the memories from the last one generally 248 do not carry over. You are starting fresh. The only exception is the limited 249 “memory” feature of ChatGPT, which notes down scattered facts about you in a 250 memory file and inserts those into the context window of every conversation. 251 Otherwise, the AI is not learning about you between chats. 252 253 Even as I write this, I know I will be getting comments from some people 254 arguing that I am wrong, along with descriptions of insights from the AI that 255 seem to violate this rule. People are often fooled because the AI is a very 256 good guesser, w[44]hich Simon Willison explains at length in his excellent post 257 on the topic of asking the AI for insights into yourself. It is worth reading. 258 259 The intuition: It can help to think about what the AI knows and doesn’t know 260 about you. Do not expect deep insights based on information that the AI does 261 not have but do expect it to make up insightful-sounding things if you push it. 262 Knowing how memory works, you can also see why it can help to start a new chat 263 when the AI gets stuck, or you don’t like where things are heading in a 264 conversation. Also, if you use ChatGPT, you may want to check out and[45] clean 265 up your memories every once in a while. 266 267 The caveat: The context windows of AIs are growing very long (Google’s Gemini 268 can hold 2 million tokens in memory), and AI companies want the experience of 269 working with their models to feel personal. I expect we will see more tricks to 270 get AIs to remember things about you across conversations being implemented 271 soon. 272 273 All of this is only sort of helpful 274 275 We still do not have a solid answer about how these basic principles of how 276 LLMs work have come together to make a system that is [47]seemingly more 277 creative than most humans, that we enjoy speaking with, and which does a 278 surprisingly good job at tasks ranging from corporate strategy to medicine. 279 There is no manual that lists what AI does well or where it might mess up, and 280 we can only tell so much from the underlying technology itself. 281 282 Understanding token prediction, training data, and memory constraints gives us 283 a peek behind the curtain, but it doesn't fully explain the magic happening on 284 stage. That said, this knowledge can help you push AI in more interesting 285 directions. Want more original outputs? Try prompts that veer into less common 286 territory in the training data. Stuck in a conversational rut? Remember the 287 context window and start fresh. 288 289 But the real way to understand AI is to use it. A lot. For about 10 hours, just 290 do stuff with AI that you do for work or fun. Poke it, prod it, ask it weird 291 questions. See where it shines and where it stumbles. Your hands-on experience 292 will teach you more than any article ever could (even this long one). You'll 293 figure out a remarkable amount about how to use AI effectively, and you might 294 even surprise yourself with what you discover. 295 296 [56][ ] 297 Subscribe 298 [58]Share 299 300 528 301 Share this post 302 [https] 303 304 Thinking Like an AI 305 306 www.oneusefulthing.org 307 Copy link 308 Facebook 309 Email 310 Note 311 Other 312 [65] 313 60 314 41 315 [66] 316 Share 317 PreviousNext 318 319 Discussion about this post 320 321 Comments 322 Restacks 323 [https] 324 [ ] 325 [73] 326 Mickey Schafer 327 [74]Oct 20 328 329 Perfect timing! This will be the first post students read next semester 330 for a one-credit class called Prompting Curiosities 😊. I'm struggling 331 to find those 10 hours so embedding it into a class seemed like a fun 332 [72] way to get it done. Just me, 15 students, and the university's AI 333 [https] system which has most of the LLMs in 3-4 versions. We will start with 334 simple prompts across different LLMs, then as each finds their 335 favorite, they'll choose one thing as their final project and work on 336 it. All in all, it should produce at least 20 per person which will 337 help me understand these much better moving forward! 338 339 Expand full comment 340 Reply 341 Share 342 343 [76]2 replies 344 345 [78] 346 Clarke Pitts 347 [79]Oct 21Liked by Ethan Mollick 348 349 [77] An excellent essay, interesting and intelligible. Very little 350 [https] explanation about AI and LLM is as lucid. 351 352 Expand full comment 353 Reply 354 Share 355 356 [81]58 more comments... 357 Top 358 Latest 359 Discussions 360 361 No posts 362 363 Ready for more? 364 365 [94][ ] 366 Subscribe 367 © 2024 Ethan Mollick 368 [96]Privacy ∙ [97]Terms ∙ [98]Collection notice 369 [99] Start Writing[100]Get the app 370 [101]Substack is the home for great culture 371 Share 372 Copy link 373 Facebook 374 Email 375 Note 376 Other 377 This site requires JavaScript to run correctly. Please [108]turn on JavaScript 378 or unblock scripts 379 380 References: 381 382 [1] https://www.oneusefulthing.org/ 383 [2] https://www.oneusefulthing.org/ 384 [13] https://substack.com/profile/846835-ethan-mollick 385 [14] https://substack.com/@oneusefulthing 386 [21] https://www.oneusefulthing.org/p/thinking-like-an-ai/comments 387 [22] javascript:void(0) 388 [23] https://a.co/d/9onRd33 389 [25] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F805116f4-c2dc-4804-b277-253d14b2139d_1292x105.png 390 [26] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcfb74661-2025-4694-b0db-a96d2166865e_1098x711.png 391 [27] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F623e802b-c122-4ef0-a667-6e429b09cc54_1992x504.png 392 [28] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fff7f2a21-1252-474d-896d-d307dc88eea7_1255x837.png 393 [29] https://arxiv.org/pdf/2212.09251 394 [30] https://www.strangeloopcanon.com/ 395 [31] https://x.com/krishnanrohit/status/1802747007838384382 396 [32] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ffc187f7b-6341-4ac9-b2e4-0c97d1eddef9_924x502.jpeg 397 [33] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd959adb9-d728-4e2f-b0f1-840b125ac9e0_1900x1126.png 398 [35] https://arxiv.org/abs/2101.00027 399 [36] https://arxiv.org/abs/2305.00118 400 [37] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fedb0dd91-9b8e-468e-8c37-cdda8bd3db5c_1290x864.jpeg 401 [38] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3dc09899-6a1a-47b3-90b9-c23be78835f8_1504x429.png 402 [39] https://en.wikipedia.org/wiki/Cordwainer_Smith 403 [40] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc63b6bec-2dc7-48e4-8e71-ec056768ac96_1494x430.png 404 [41] https://en.wikipedia.org/wiki/Walter_Pater 405 [42] https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F59a0b6a1-37ca-4447-8777-b94593809c4f_2025x1324.png 406 [44] https://simonwillison.net/2024/Oct/15/chatgpt-horoscopes/ 407 [45] https://openai.com/index/memory-and-new-controls-for-chatgpt/ 408 [47] https://docs.iza.org/dp17302.pdf 409 [58] https://www.oneusefulthing.org/p/thinking-like-an-ai?utm_source=substack&utm_medium=email&utm_content=share&action=share 410 [65] https://www.oneusefulthing.org/p/thinking-like-an-ai/comments 411 [66] javascript:void(0) 412 [72] https://substack.com/profile/244712-mickey-schafer 413 [73] https://substack.com/profile/244712-mickey-schafer 414 [74] https://www.oneusefulthing.org/p/thinking-like-an-ai/comment/73352564 415 [76] https://www.oneusefulthing.org/p/thinking-like-an-ai/comment/73352564 416 [77] https://substack.com/profile/14800577-clarke-pitts 417 [78] https://substack.com/profile/14800577-clarke-pitts 418 [79] https://www.oneusefulthing.org/p/thinking-like-an-ai/comment/73452831 419 [81] https://www.oneusefulthing.org/p/thinking-like-an-ai/comments 420 [96] https://substack.com/privacy 421 [97] https://substack.com/tos 422 [98] https://substack.com/ccpa#personal-data-collected 423 [99] https://substack.com/signup?utm_source=substack&utm_medium=web&utm_content=footer 424 [100] https://substack.com/app/app-store-redirect?utm_campaign=app-marketing&utm_content=web-footer-button 425 [101] https://substack.com/ 426 [108] https://enable-javascript.com/