The biggest design challenge isn't making AR glasses capable of showing information—it's deciding what not to show.
People have a surprisingly low tolerance for visual interruptions. Your brain is already processing the world at high bandwidth. Any digital overlay competes with reality itself.
A useful mental model is to think of AR as a second layer of perception, not a floating smartphone screen.
Here are some principles that could make contextual AI genuinely helpful.
1. Information should "arrive just in time"
Instead of persistent labels everywhere:
- Walking toward a train station → departure time fades in.
- Looking at a restaurant → today's hours and rating appear for two seconds.
- Looking at a product → price comparison only after a brief pause.
Nothing stays on screen unless it's currently useful.
2. Confidence should control visibility
The AI should know when it isn't sure.
For example:
- Face recognition: 99% confidence → "Sarah"
- 70% confidence → "Possibly Sarah?"
- 40% confidence → show nothing.
Likewise:
- Translation confidence
- Navigation certainty
- Object recognition
Showing fewer overlays is often better than showing wrong ones.
3. Use the edges of vision
The center of your vision is for the real world.
The AI should reserve that area for only the most important alerts.
Imagine:
Center:
Peripheral:
- Navigation arrows.
- Incoming message indicator.
- Battery.
- Meeting reminder.
The interface should feel more like peripheral awareness than a dashboard.
4. Attention is a scarce resource
The AI should estimate:
"How busy is the user?"
If you're:
- crossing a street
- riding a bike
- talking to someone
- carrying groceries
the system should aggressively reduce visual output.
If you're sitting in an airport waiting:
It can become much richer.
5. Social situations deserve special handling
This is where contextual AI could be amazing—or creepy.
Helpful:
- "You met Alex at SIGGRAPH."
- "Their daughter's name is Emma."
- "Last conversation: renewable energy."
Not helpful:
Floating labels above everyone's heads.
A better interaction might be:
You intentionally glance at someone for a second.
Then a small card appears off to the side.
Intent matters.
6. Navigation should blend into the environment
Instead of:
⬅ Turn left in 300 feet.
Imagine:
- a subtle glowing path on the sidewalk
- highlighted doorway
- illuminated staircase
- bus you need gently outlined
The world itself becomes the map.
7. Translation should replace—not duplicate
If someone speaks Japanese:
Instead of subtitles floating across your view,
you almost perceive them speaking English.
The translation appears near the speaker naturally, then fades away.
The goal is reducing cognitive load, not adding more text.
8. Make curiosity pull-based
Don't tell me everything.
Let me ask silently.
Looking at a building:
Nothing.
A second longer:
Its name appears.
Another second:
Construction date.
Another:
Historical facts.
Your gaze becomes the query language.
9. Give the AI a "social volume knob"
Three modes might work well:
- Minimal: emergencies, navigation, translations.
- Balanced: contextual facts when relevant.
- Explorer: identify plants, architecture, stars, artwork, products, historical context.
Most people would likely stay in Minimal or Balanced most of the time.
10. Respect ambiguity
Reality is messy.
If someone looks sad:
Don't display:
"User is sad."
Instead:
"They may be tired."
Or better:
Show nothing.
Inferring emotions, intentions, or relationships carries a high risk of error and could affect interactions in unintended ways. In many cases, restraint is the better design choice.
A broader vision: an "attention operating system"
The most compelling AR system may not be the one that displays the most information, but the one that understands when to stay silent.
Imagine the hierarchy:
-
Safety
- Hazard warnings
- Navigation
- Accessibility assistance
-
Task support
- Translation
- Instructions
- Memory aids
- Scheduling
-
Knowledge
- Object identification
- Historical facts
- Product information
-
Curiosity
- Deep dives
- Recommendations
- Trivia
Higher-priority information can temporarily suppress lower-priority overlays. If you're crossing a busy intersection, a restaurant review shouldn't compete with a warning about an approaching cyclist.
In that sense, the ideal AR experience is less like wearing a computer on your face and more like having an attentive companion: it notices what's relevant, speaks only when it adds value, and otherwise lets you experience the world directly.
