304 lines
11 KiB
Plaintext
304 lines
11 KiB
Plaintext
[1]
|
||
|
||
Our interfaces have
|
||
lost their senses
|
||
|
||
Think about how you experience the world—
|
||
|
||
you touch, you hear, you move.
|
||
|
||
[dance1] [dance1] [dance1] [dance1]
|
||
[dance-grou]
|
||
|
||
But our digital world has been getting flatter, more muted.
|
||
|
||
Reduced to text under glass screens.
|
||
|
||
This shift made interfaces simpler.
|
||
But was that really the goal?
|
||
|
||
An interface is the bridge between
|
||
the human
|
||
&
|
||
the machine.
|
||
[human]
|
||
[human] [machine]
|
||
It's how we tell computers what we want,
|
||
[arrow-righ]
|
||
and it's how computers communicate back to us.
|
||
[arrow-left]
|
||
The shape should fit how we work,
|
||
for ergonomics and ease of use
|
||
and it should fit how the computer works.
|
||
for simplicity and a good mental model
|
||
Recently, we've been too focused on fitting to the computer's shape, and not
|
||
enough to our own bodies.
|
||
[machine]
|
||
|
||
The Great Flattening
|
||
|
||
Computers used to be physical beasts.
|
||
|
||
We programmed them by punching cards, plugging in wires, and flipping switches.
|
||
Programmers walked among banks of switches and cables, physically
|
||
choreographing their logic. Being on a computer used to be a full-body
|
||
experience.
|
||
|
||
[tech0]
|
||
[tech1]
|
||
[transition]
|
||
|
||
Then came terminals and command lines. Physical knobs turned into typed
|
||
commands—more powerful, but our digital world became less embodied. Then came
|
||
terminals and command lines. Physical knobs turned into typed commands—more
|
||
powerful, but our digital world became less embodied. Then came terminals and
|
||
command lines. Physical knobs turned into typed commands—more powerful, but our
|
||
digital world became less embodied. Then came terminals and command lines.
|
||
Physical knobs turned into typed commands—more powerful, but our digital world
|
||
became less embodied. Then came terminals and command lines. Physical knobs
|
||
turned into typed commands—more powerful, but our digital world became less
|
||
embodied. Then came terminals and command lines. Physical knobs turned into
|
||
typed commands—more powerful, but our digital world became less embodied.
|
||
|
||
[tech2]
|
||
[transition]
|
||
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
We brought back some of the tactile controls with GUIs—graphical user
|
||
interfaces. We skeumorphed the heck out of our screens, with digital switches,
|
||
flat sliders, and folder icons. But we kept some of the the functionality in
|
||
the physical world, with slots to stick disks into and big ol' power buttons.
|
||
|
||
[tech3]
|
||
[transition]
|
||
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [2]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [3]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [4]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [5]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [6]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
Then came touchscreens.
|
||
What a beautiful thing! We get to [7]poke things directly!
|
||
But now we live in an flat land, with everything behind a glass display case.
|
||
|
||
[tech4]
|
||
[transition]
|
||
|
||
With increasing amounts of AI chatbots, we're losing even more: texture, color,
|
||
shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text. With increasing amounts of AI
|
||
chatbots, we're losing even more: texture, color, shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text. With increasing amounts of AI
|
||
chatbots, we're losing even more: texture, color, shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text. With increasing amounts of AI
|
||
chatbots, we're losing even more: texture, color, shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text. With increasing amounts of AI
|
||
chatbots, we're losing even more: texture, color, shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text. With increasing amounts of AI
|
||
chatbots, we're losing even more: texture, color, shape.
|
||
Instead of interactive controls, we have a text input.
|
||
Want to edit an image? Type a command.
|
||
Adjust a setting? Type into a text box.
|
||
Learn something? Read another block of text.
|
||
|
||
[tech5]
|
||
[tech6]
|
||
|
||
The Joy of Doing
|
||
|
||
We've been successfully removing all friction from our apps — think about how
|
||
effortless it is to scroll through a social feed. But is that what we want?
|
||
Compare the feeling of doomscrolling to kneading dough, playing an instrument,
|
||
sketching... these take effort, but they're also deeply satisfying. When you
|
||
strip away too much friction, meaning and satisfaction go with it.
|
||
|
||
Think about how you use physical tools. Drawing isn't just moving your
|
||
hand—it's the feel of the pencil against paper, the tiny adjustments of
|
||
pressure, the sound of graphite scratching. You shift your body to reach the
|
||
other side of the canvas. You erase with your other hand. You step back to see
|
||
the whole picture.
|
||
|
||
We made painting feel like typing,
|
||
|
||
[typing]
|
||
[art-transi]
|
||
|
||
but we should have made typing feel like painting.
|
||
|
||
[artist]
|
||
|
||
Putting the you back in UI
|
||
|
||
So how might our interfaces look if we shaped them to fit us?
|
||
|
||
We think in movement, [movement]
|
||
in space, [space]
|
||
in sound,
|
||
[sound]
|
||
in patterns.
|
||
[patterns]
|
||
|
||
We use our hands to sculpt, our eyes to scan, our ears to catch patterns.
|
||
|
||
Our computers can communicate to us in many different formats, each with their
|
||
own strengths:
|
||
|
||
Text
|
||
Great for depth, detail, and precision.
|
||
[images]
|
||
But it doesn't always have to be in full paragraphs. How about showing key
|
||
points first, then letting users expand?
|
||
Visualizations
|
||
Ideal for spatial relationships, trends, and quick insights.
|
||
[vision]
|
||
Can we show more content spatially? Or encode it in charts or colors?
|
||
Sound
|
||
Perfect for alerts and background awareness. Also, patterns.
|
||
[hearing]
|
||
Why are most web UIs silent? Can we use subtle chimes or sonification to
|
||
highlight patterns?
|
||
Haptics
|
||
Provides passive feedback (vibrations, force).
|
||
[touch]
|
||
Here's one I always forget about! We can vibrate phones to alert or convey
|
||
patterns.
|
||
|
||
And what about the reverse! We can communicate to our computers in many
|
||
different ways, each with their own strengths:
|
||
|
||
Typing
|
||
Precise, detailed, and familiar
|
||
[typing2]
|
||
Good for composing long-form thoughts, keyboard shortcuts, and rough direction.
|
||
Clicking & Dragging
|
||
Direct, fine-grained control.
|
||
[clicking]
|
||
Great for spatial tasks (design, organization) and pointing at
|
||
things-on-a-screen.
|
||
Tapping, Swiping, Pinching
|
||
Intuitive for direct manipulation.
|
||
[tapping]
|
||
Great for mobile, but do we have to limit guestures to mimicking mouse
|
||
interactions?
|
||
Gesturing
|
||
Hands-free, fluid, and expressive.
|
||
[guesturing]
|
||
Could be powerful for accessibility, quick actions, and complex fine
|
||
control—reliable detection feels very possible at this time.
|
||
Speaking
|
||
Easy for loose thoughts.
|
||
[speaking]
|
||
LLMs have made speech more viable—can we let users think out loud or navigate
|
||
roughly with their voice?
|
||
|
||
And the real magic happens when we combine different modalities. You can't read
|
||
and listen and speak at the same time—try reading this excerpt while talking
|
||
about your day:
|
||
|
||
If it had not rained on a certain May morning Valancy Stirling’s whole life
|
||
would have been entirely different. She would have gone, with the rest of her
|
||
clan, to Aunt Wellington’s engagement picnic and Dr. Trent would have gone to
|
||
Montreal. But it did rain and you shall hear what happened to her because of
|
||
it.
|
||
|
||
~ [8]The Blue Castle
|
||
But you can talk while clicking,
|
||
[click]
|
||
listen while reading,
|
||
[listen]
|
||
look at an image while spinning a knob,
|
||
[look]
|
||
guesture while talking.
|
||
[guesture]
|
||
|
||
Let's build interfaces that let us multitask across senses.
|
||
|
||
Rebuilding the bridge
|
||
|
||
So, what might a richer interface look like? I have strong conviction that our
|
||
future interfaces should:
|
||
|
||
• let us collaborate on tangible artifacts, not just ephemeral chat logs.
|
||
• support multiple concurrent modalities—voice, gestures, visuals, spatial
|
||
components.
|
||
• respond to ambient signals—detecting context, organizing information,
|
||
helping us think better.
|
||
|
||
Last year, I did a rough exploration of what this could look like for a thought
|
||
organizing tool. One that listened as you talked or typed, and organized your
|
||
rambling thoughts into cards.
|
||
|
||
This interface is very rough, but felt like a different way of working with
|
||
technology. Especially how it let me bumble through rough ideas one second,
|
||
then responded to commands like "re-group my cards" or "add 3 cards about this"
|
||
the next.
|
||
|
||
I would love to see more explorations like this!
|
||
|
||
Our interfaces have lost their senses
|
||
|
||
All day, we poke, swipe, and scroll through flat, silent screens. But we're
|
||
more than just eyes and a pointer finger. We think with our hands, our ears,
|
||
our bodies.
|
||
|
||
The future of computing is being designed right now. Can we build something
|
||
richer—something that moves with us, speaks our language, and molds to our
|
||
bodies?
|
||
|
||
[footer]
|
||
|
||
References:
|
||
|
||
[1] https://wattenberger.com/
|
||
[2] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[3] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[4] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[5] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[6] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[7] https://www.youtube.com/watch?v=RyBEUyEtxQo
|
||
[8] https://www.gutenberg.org/ebooks/67979
|