RouteMAX's "Optymize" button could plan a full day of delivery routes in seconds, work that took expert planners hours. The algorithm was right. The engineering was solid. Only 18% of users ever tried it, and most never came back.
The catch: our 80% operational-efficiency promise to clients only held if people actually used the feature. Demos and tutorials hadn't moved the number.
The original launch popup: inputs, a button, and "This action cannot be reversed"
Sole researcher and designer. I:
- pitched research when the default plan was more training
- defined trust in measurable terms before a single interview
- designed and ran a 3-phase, 5-week study across two user segments
- redesigned the flow, then turned the findings into a framework for every automation feature after it
It wasn't an awareness problem
The PM wanted more training. Engineering wanted attribution data. Both answered the wrong question. The question wasn't "why won't they use it?" It was "what would it take for planners to trust it with the expertise they'd spent years building?"
Defined before measured: reliance, transparency, comprehension.
Dummy data measures novelty, not use. Output depends on live route complexity.
Joint debriefs, so the team heard unexplained outputs first-hand.
- Why do non-users never try Optymize, and why do bounce users quit after one or two runs?
- Where does trust break: reliance, transparency, or comprehension?
- What would planners need to see and control before handing routing to an algorithm?
- Segment: usage data split users into non-users (never tried it) and bounce users (tried it, stopped).
- Cognitive walkthrough: surfaced the mental-model assumptions planners brought to the feature.
- 1-week diary study: planners used Optymize on their own live routes, under real daily time pressure.
- Joint debrief + ranking: walked through diary entries with users and engineering together, then had users rank why they stopped.
Planners weren't unaware. The button asked them to invert years of expertise (group shipments, apply constraints from memory) into "enter constraints, then wait." Nobody designed the bridge.
- Non-users couldn't tell what it replaced. The popup never named the old workflow or explained dynamic routing.
- One unexplained output ended the relationship. A single "extra route" read as broken, and bounce users never came back.
- Awareness ranked near the bottom. Confusing interface and unexplained outputs ranked first.
Five trust principles I built here, and still apply to AI
- Calibrate trust, don't maximize it
- Planners learned when to rely and when to override.
AI now: too little trust means bypass, too much means over-reliance. - Show the process, not just the output
- A step tracker that mirrored their workflow.
AI now: show what the model considered and left out. - Design for reversibility
- Checkpoints replaced "cannot be reversed."
AI now: undo, rollback, and audit trails for agents. - Expose the system's state
- Preferences, capacity, and anchor points made visible.
AI now: confidence, sources, flagged partial results. - Preserve user agency
- Per-route preferences and confirmation before changes.
AI now: suggest before acting; confirm high-impact actions.
PrincipleAutomation isn't a feature you ship. It's a relationship you build.
- 18% one-time → 52% daily users within 2 months of the redesign
- North star hit: led to an industry press release and client expansion
- Became the standard: the mental-model migration framework was applied to 3+ later automation modules
- Beyond scope: surfaced a client data-quality issue, sparking a client-led data correction that improved algorithm outputs
The redesign: a step tracker, explicit trailer selection, per-route preferences instead of locks, and no scary warnings
"The software is intuitive and most planners were able to understand the majority of features with limited training." — Patrick Sugar, VP Linehaul & Engineering, Saia
- Reframing before solving
- Moved the org from "awareness" to trust, with evidence
- Operationalizing a fuzzy construct
- Defined trust as reliance, transparency, and comprehension before measuring it
- Method fit to the question
- A diary study on live data, because novelty isn't use
- Research → design → metric
- Owned all three as a team of one: 18% → 52% daily use
- Human-AI trust, early
- Principles that map directly onto today's copilots and agents