interconnected-org-zhcj46.txt (17836B)
1 2 3 [1]Interconnected 4 5 A blog by Matt Webb 6 7 • [2]About 8 • [3]Archive 9 • [4]Work 10 11 Subscribe for $0 12 13 • [5]Email 14 • [6]RSS feed 15 • [7](What is a feed?) 16 17 Unoffice Hours 18 19 • [8]Book a call 20 • [9](What is this?) 21 22 a.k.a. genmon 23 24 • [10]Bluesky 25 • [11]X/Twitter 26 • [12]Insta 27 • [13]Mastodon 28 • [14]LinkedIn 29 30 Building the AI clock 31 32 • [15]Check out Poem/1 33 34 Resident: vibe coding firmware (our new sandbox library for ESP32 devices) 35 36 14.14, Wednesday 20 May 2026 [16]Link to this post 37 38 We’re open sourcing Resident, our library for running AI-authored code on 39 microcontrollers – with no compile step and no firmware flashing. 40 41 It’s our twist on vibe coding firmware, built for instantly loading new device 42 functionality coded by end users. It’s aimed at device developers, like us. We 43 use Resident in all our work. 44 45 (We = Inanimate.) 46 47 [17]Resident gives you a code sandbox on ESP32 devices and a driver API to 48 provide hardware control and events. So an end user can push an app over Wi-Fi 49 that instantly turns their clock into an interactive pill timer (for example) 50 but the app can’t run probes on the home Wi-Fi network. 51 52 It comes bundled with a set of Claude skills. I told Claude about the 53 capabilities of a compatible dev kit and asked it to push a simple app. Here it 54 is: 55 56 [hello-resi] 57 58 At Inanimate, we believe that on-device sandboxes are an essential low-level 59 primitive for AI agents in the real world. 60 61 The announcement is [18]over on Lab Notes but I want to take a moment to 62 connect this to some previous themes… 63 64 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 65 66 Look, I want to bring software into my room. 67 68 Like: if I’m working with Claude Code and I step away from my keyboard, it 69 should be able to give me updates on what it’s working on by taking over my 70 desk clock, and ping its permission requests to the gumstick device I have in 71 my pocket for me to accept/decline while I’m making tea. Untether me from my 72 desk! (btw I built this, it was awesome.) 73 74 Or - more prosaically - why can’t I yell at my stove as I’m leaving the house 75 in the morning oh I forgot I’m roasting a chicken that needs to be ready for 76 7pm and have it look up timings and push me a notification when it’s 77 pre-heating, and show a custom basting timer app on its 14-seg LED display. 78 79 `Why can’t I point at a lamp and say ‘on’ and the light come on?' – [19]I was 80 asking in 2020. 81 82 So half of the solution here is AI: LLMs are super good at translating intent 83 into action. I use regular language and the computer will [20]Do What I Mean 84 (2025). 85 86 But the other half is a problem: how does this actually work? Where does the 87 software run? 88 89 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 90 91 Every device needs to be able to run user code, that’s the answer. 92 93 I’ve been bouncing off this since 2023 [21]when I put an LLM in charge of a 94 smart home (using tech that we now call agents, but then was my implementation 95 of the ReAct pattern from a paper out of Princeton and Google Research). 96 97 In a nutshell, let’s say you press a button on your desk clock and your AI 98 agent has to decide in the moment what to do with that. It won’t work, it’s too 99 slow. Cognitively, an interface has to respond inside 150ms or it is no longer 100 instant – but the AI is a network hop away. 101 102 Ok so let’s remove the network hop. Let’s pretend we have edge AI: a 103 GPT-4-equiv AI as a component in every physical thing; ubiquitous intelligence 104 is coming and `a Feynmann-level light switch could guess your intentions pretty 105 well,' [22]that was was my guess in 2023. 106 107 It’s not an outrageous extrapolation! [23]Taalas is baking LLMs into silicon 108 and delivers `17k tokens per second per user on Llama 3.1 8B' ([24]try it here, 109 e.g. 4,000 words on Hamlet as a space opera, it’s wild it’s so instant). So 110 GPT-4-equiv is a matter of time. 111 112 Even with that speed, it turns that inference is not enough. To Do What I Mean, 113 you still need to import user context for personalisation and grounding. But 114 memory, Gmail, Wikipedia and the rest are still mostly in the cloud. That 115 network hop again. 116 117 So AI can’t be inside the event loop, not if you want really great on-device 118 interactions. 119 120 Instead – let the AI write device code. That’s the approach we’ve found. 121 122 Take that toy example of a roast chicken basting timer on the display of my 123 stove. My AI agent should be able to dynamically write code for that 124 interactive app, and have that app code executed on the stove itself. 125 126 I’m not saying that the AI needs to vibe-code firmware. 127 128 Firmware is the code that runs on device microcontrollers: you author it and 129 compile it and you flash it and from that point on, it never changes. AI agents 130 can vibe firmware (very happily). But I’m not sure I want to load code onto my 131 stove that has boundless control over the heating element and the network stack 132 and whatever other low-level capabilities are managed by the firmware itself. 133 We’ve already had [25]smart fridges sending spam email (BBC News, 2014), no 134 more thanks. 135 136 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 137 138 Instead of firmware, the AI can run code in sandboxes. 139 140 Cloudflare made the case for sandboxes in March this year [26]when they 141 introduced dynamic workers: 142 143 Last September we introduced Code Mode, the idea that agents should perform 144 tasks not by making tool calls, but instead by writing code … 145 146 You can’t just eval() AI-generated code directly in your app: a malicious 147 user could trivially prompt the AI to inject vulnerabilities. 148 149 You need a sandbox: a place to execute code that is isolated from your 150 application and from the rest of the world, except for the specific 151 capabilities the code is meant to access. 152 153 Sandboxing is a hot topic in the AI industry … 154 155 We asked ourselves: 156 157 What if, when we allow an AI agent to spread its arms and stretch its legs in a 158 room, it could inhabit devices by writing code that runs in an on-device 159 sandbox, a sandbox that gives access to buttons and screens but not the network 160 stack, and the app code could load and run instantly? 161 162 That’s Resident. It provides a code sandbox for ESP32 devices, and a toolchain 163 for AI agents to write apps that target that sandbox. 164 165 At Inanimate, we use Resident for all our product prototyping. 166 167 And it will be at the heart of our future products. 168 169 Look, it may seem wildly disproportionate to write code to turn on a lamp. 170 171 But what are computers for? They do the hard work to make it easy for us. So 172 this approach scales well from basic on/off control to… well, it turns out 173 that, now we have this sandbox, we can compose all kinds of useful experiences 174 that take over an entire room – and also weird and wonderful interactive ones 175 that still work when you pull the network cable. 176 177 (An app arrives over the network but then the network is no longer required. 178 Resident is built on our messaging library [27]Courier which includes UDP 179 multicast for local inter-device messaging even when internet connectivity 180 drops.) 181 182 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 183 184 We’re opening Resident today as an alpha (v0.5.0) and open sourcing it under 185 the highly permissive MIT license. Just include our copyright notice with any 186 modifications. 187 188 Technically, we’re adding a Lua runtime to your ESP32 device. (Lua is a 189 language designed to be embedded.) 190 191 We love Espressif’s ESP32 microcontroller family because it has a unique span 192 in the ecosystem: it is used by individual makers, new hardware startups, and 193 in production at real scale. It has built-in Wi-Fi and supports its native 194 framework esp-idf and Arduino too, which is great for quick prototyping. 195 196 Resident gives you an API to add extensions to that Lua runtime: hardware 197 drivers. Those managed capabilities are what makes it a sandbox. So the apps in 198 the sandbox can respond to events that your button driver injects, and they can 199 write to the display via a module added by your display driver. 200 201 The apps can be hot loaded at runtime. The way we have it wired up, you push 202 app code down a websocket to the device and it run immediately in the sandbox. 203 204 Adding Resident is straightforward during development: bring up your new device 205 as normal, then point your coding agent at docs/start-building.md. It’ll walk 206 you through adding the sandbox and writing the drivers. 207 208 Resident comes bundled with: 209 210 • Connectivity: websockets, JSON messaging, and easy Wi-Fi config 211 • A default back-end server at resident.inanimate.tech so you can easily push 212 new apps and events – use the example code to build out your own back-end 213 when you’re ready 214 • A collection of Claude skills to create, validate and push new apps to your 215 devices (and even write device documentation) 216 • Example projects. 217 218 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 219 220 Want to try Resident now? 221 222 Ahead of developing your new device, pick up an [28]M5StickS3. These are made 223 by M5, I talked about them [29]when we announced Courier. 224 225 The M5Stick is a dev kit that has an ESP32 with a screen, battery, couple of 226 buttons, buzzer and IMU. You can bring it up using Arduino pretty simply. Then 227 add Resident and start writing apps. [30]We have an example project. 228 229 Want to get going even faster? 230 231 You don’t even need hardware… 232 233 The [31]Try it now section of the Resident homepage has a M5Stick simulator, 234 running the Resident sandbox in-browser. 235 236 Drag and drop an app onto the simulator to see it run. 237 238 Or even: install the Claude skills, tap the button on the webpage to make the 239 simulator live, and create apps from your local Claude Code session. The 240 in-browser simulator will update live and run your app. 241 242 [resident-s] 243 244 Hey, deep cut reference alert: 245 246 Back in 2021 I went on a dive into files: [32]Golems, smart objects, and the 247 file metaphor. 248 249 What would it mean to drag and drop a file onto a lightbulb? I asked. `Do I 250 literally mean that the lightbulb needs a little slot like the golem’s mouth, 251 into which you insert your instructions stamped on microfiche?' 252 253 This is my answer haha 254 255 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 256 257 Resident is what we’ve been using to prototype products and use cases at 258 Inanimate. 259 260 We believe that one day all products will work this way. 261 262 For more info, the GitHub, and to try it now: [33]Resident. 263 264 For updates: [34]subscribe to Lab Notes. 265 266 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 267 268 More posts tagged: 269 270 • [35]inanimate (6 posts) 271 272 If you enjoyed this post, please consider sharing it by email or on social 273 media. [36]Here’s the link. Thanks, —Matt. 274 275 276 277 Most recent posts 278 279 • [37]How global logistics got me over my fear of personal agents 30 May 2026 280 • Resident: vibe coding firmware (our new sandbox library for ESP32 devices) 281 20 May 2026 (This post) 282 • [38]Filtered for bad addresses and good emotions 15 May 2026 283 • [39]I want my MTV 8 May 2026 284 • [40]We need RSS for sharing abundant vibe-coded apps 29 Apr 2026 285 • [41]The Wind in the Willows and reading out loud 24 Apr 2026 286 • [42]Courier: real-time messaging for ESP32 with batteries included (new 287 library) 21 Apr 2026 288 • [43]Headless everything for personal AI 18 Apr 2026 289 • [44]mist is now open source and looking for interop 10 Apr 2026 290 • [45]A sleep aid 3 Apr 2026 291 • [46]An appreciation for (technical) architecture 28 Mar 2026 292 • [47]Filtered for home security 20 Mar 2026 293 294 Continue reading: [48]All in 2026 295 296 streak New posts for 322 consecutive weeks (see: [49]blogging tips) 297 298 New? Start here: [50]Best of 2025 (also [51]2024, [52]2023, [53]2022, [54]2021, 299 [55]2020) 300 Or explore the archives: [56]On this day 301 302 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 303 304 Archive 305 306 • [57]2026 23 posts 307 • [58]2025 61 posts 308 • [59]2024 60 posts 309 • [60]2023 68 posts 310 • [61]2022 96 posts 311 • [62]2021 128 posts 312 • [63]2020 118 posts 313 • [64]2019 23 posts 314 • [65]2018 47 posts 315 • [66]2017 22 posts 316 • [67]2016 48 posts 317 • [68]2015 88 posts 318 • [69]2014 30 posts 319 • [70]2013 6 posts 320 • [71]2012 27 posts 321 • [72]2011 76 posts 322 • [73]2010 2 posts 323 • [74]2009 2 posts 324 • [75]2008 59 posts 325 • [76]2007 20 posts 326 327 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 328 [77][ ] Search 329 Since February 2000. Copyright © 2026 Matt Webb. 330 331 p.s. here’s [85]my blogroll and the [86]colophon. 332 333 Quiet mode[87][ ] 334 335 References: 336 337 [1] https://interconnected.org/home/ 338 [2] https://interconnected.org/ 339 [3] https://interconnected.org/home/2026/05/20/resident#archive 340 [4] https://www.actsnotfacts.com/ 341 [5] https://buttondown.com/genmon 342 [6] https://interconnected.org/home/feed 343 [7] https://aboutfeeds.com/ 344 [8] https://calendly.com/mwie/30min 345 [9] https://interconnected.org/home/2020/09/24/unoffice_hours 346 [10] https://bsky.app/profile/genmon.org 347 [11] https://x.com/genmon 348 [12] https://www.instagram.com/genmon/ 349 [13] https://mastodon.social/@genmon 350 [14] https://www.linkedin.com/in/genmon/ 351 [15] https://poem.town/ 352 [16] https://interconnected.org/home/2026/05/20/resident 353 [17] https://resident.inanimate.tech/ 354 [18] https://news.inanimate.tech/p/lab-notes-drum-loops-prototyped-as-oranges 355 [19] https://interconnected.org/home/2020/05/26/voice 356 [20] https://interconnected.org/home/2025/08/29/dwim 357 [21] https://interconnected.org/home/2023/04/26/lares 358 [22] https://interconnected.org/home/2023/10/06/ubigpt 359 [23] https://taalas.com/products/ 360 [24] https://chatjimmy.ai/ 361 [25] https://www.bbc.co.uk/news/technology-25780908 362 [26] https://blog.cloudflare.com/dynamic-workers/ 363 [27] https://github.com/inanimate-tech/courier 364 [28] https://docs.m5stack.com/en/core/StickS3 365 [29] https://interconnected.org/home/2026/04/21/courier 366 [30] https://github.com/inanimate-tech/resident/tree/main/examples/m5stick-demo 367 [31] https://resident.inanimate.tech/#try-it-now 368 [32] https://interconnected.org/home/2021/02/01/golems 369 [33] https://resident.inanimate.tech/ 370 [34] https://news.inanimate.tech/ 371 [35] https://interconnected.org/home/tagged/inanimate 372 [36] https://interconnected.org/home/2026/05/20/resident 373 [37] https://interconnected.org/home/2026/05/30/fedex 374 [38] https://interconnected.org/home/2026/05/15/filtered 375 [39] https://interconnected.org/home/2026/05/08/mtv 376 [40] https://interconnected.org/home/2026/04/29/syndicating-vibes 377 [41] https://interconnected.org/home/2026/04/24/willows 378 [42] https://interconnected.org/home/2026/04/21/courier 379 [43] https://interconnected.org/home/2026/04/18/headless 380 [44] https://interconnected.org/home/2026/04/10/open-mist 381 [45] https://interconnected.org/home/2026/04/03/sleep 382 [46] https://interconnected.org/home/2026/03/28/architecture 383 [47] https://interconnected.org/home/2026/03/20/filtered 384 [48] https://interconnected.org/home/2026 385 [49] https://interconnected.org/home/2020/09/10/streak 386 [50] https://interconnected.org/home/2026/01/03/top-posts 387 [51] https://interconnected.org/home/2024/12/30/top-posts 388 [52] https://interconnected.org/home/2023/12/22/top-posts 389 [53] https://interconnected.org/home/2022/12/21/top_posts 390 [54] https://interconnected.org/home/2021/12/23/top_posts 391 [55] https://interconnected.org/home/2020/12/17/top_posts 392 [56] https://interconnected.org/home/on-this-day 393 [57] https://interconnected.org/home/2026 394 [58] https://interconnected.org/home/2025 395 [59] https://interconnected.org/home/2024 396 [60] https://interconnected.org/home/2023 397 [61] https://interconnected.org/home/2022 398 [62] https://interconnected.org/home/2021 399 [63] https://interconnected.org/home/2020 400 [64] https://interconnected.org/home/2019 401 [65] https://interconnected.org/home/2018 402 [66] https://interconnected.org/home/2017 403 [67] https://interconnected.org/home/2016 404 [68] https://interconnected.org/home/2015 405 [69] https://interconnected.org/home/2014 406 [70] https://interconnected.org/home/2013 407 [71] https://interconnected.org/home/2012 408 [72] https://interconnected.org/home/2011 409 [73] https://interconnected.org/home/2010 410 [74] https://interconnected.org/home/2009 411 [75] https://interconnected.org/home/2008 412 [76] https://interconnected.org/home/2007 413 [85] https://interconnected.org/home/blogroll 414 [86] https://interconnected.org/home/2024/10/28/colophon