Find My Genie / 2026Product design × engineering

When the agent
gets stuck, who takes over?

I designed and built a multi-sided AI concierge that could coordinate 20+ vendors at once—while keeping consequential states traceable and humans in control.

Role: Founding Design Engineer
Timeline: Jan—Mar 2026
Scope: Product + frontend
Surfaces: Customer, vendor, admin, HITL

White paths moving across a green landscape toward a distant pavilion

Problem

AI coordination became opaque when vendors, prices, and exceptions changed asynchronously.

I owned

Product design, interaction model, frontend, and HITL operator surfaces.

Key decision

Expose operational states and make human takeover deliberate and reversible.

Shipped

Four connected customer, vendor, admin, and operator experiences.

Outcome

A four-step booking flow with explicit escalation and recovery paths.

01 / The system

A concierge is easy—until it touches the real world.

A customer can ask for a tow in one sentence. Behind that sentence sit changing availability, parallel vendor calls, pricing, location, payment, and exceptions. The product had to feel calm to the customer without hiding the operational uncertainty underneath.

4Connected product surfaces designed and coded

20+Vendors contacted simultaneously

HITLIntervention designed as core behavior

01CustomerStates the intent

02AI agentClarifies and coordinates

03Vendor networkReturns price and availability

04Human operatorInspects, verifies, takes over

Find My Genie customer command center set within a meadow service gateway — FMG / Figure 01 · Concept visualization Customer command center — one request, many operational actors

02 / Customer journey

Operational progress needs explicit states.

The customer experience translated the agent’s asynchronous work into understandable decisions: what was happening, which vendors responded, why an option was recommended, and what would happen next.

Decision 01

Visible stages

Replace indefinite loading with specific operational progress.

Decision 02

Comparable options

Present trade-offs in price, rating, distance, and arrival time.

Decision 03

One status language

Keep booking states consistent across customer and operator views.

Find My Genie conversational assistant embedded in a vertical service marker — FMG / Figure 02 · Concept visualization Conversational clarification

Find My Genie search state branching toward multiple service providers — FMG / Figure 03 · Concept visualization Parallel vendor outreach

Find My Genie vendor comparison interface at the end of converging service paths — FMG / Figure 04 · Concept visualization Vendor comparison

Find My Genie service tracking interface with illuminated progress markers — FMG / Figure 05 · Concept visualization Service tracking

03 / AI safety

Human takeover is a designed product state.

The operator surface prioritized calls by urgency and made escalation reasons explicit: a time threshold was exceeded, the agent reached its capability limit, or the vendor requested a human. Operators could inspect the full context before acting.

Find My Genie human-in-the-loop operations dashboard illuminated at night — FMG / Figure 06 · Concept visualization Operations dashboard — urgency, ownership, duration, and escalation in one scan

Escalated Find My Genie call with transcript and human takeover control — FMG / Figure 07 · Concept visualization Escalation detail — inspect the transcript before intervention

Deliberate control

Hold for 1.5 seconds to take over.

A consequential action should not happen through an accidental tap. The circular progress state makes the commitment visible, remains cancellable on release, and confirms when the operator is live.

FMG / Motion 01 Deliberate 1.5-second takeover and confirmed human-control state

Unmodified Find My Genie human-in-the-loop dashboard product capture — Shipped interface · Unmodified capture Human-in-the-loop operations dashboard

04 / Design engineering

The takeover control runs in production code.

The React and TypeScript implementation connects a visible progress ring to a cancellable hold gesture. Mouse and touch paths share the same state model, active operators receive a confirmed state, and the control exposes an accessible name.

TakeOverButton.tsxReact / TypeScript / Framer Motion

const startHold = useCallback(() => {
  if (isActive) return
  setIsHolding(true)

  animate(progress, 1, {
    duration: 1.5,
    ease: 'linear',
    onComplete: () => {
      setIsHolding(false)
      onTakeOver()
    },
  })
}, [isActive, progress, onTakeOver])

const cancelHold = useCallback(() => {
  if (isActive) return
  setIsHolding(false)
  animate(progress, 0, { duration: 0.3 })
}, [isActive, progress])

API contracts

Validated onboarding

Authenticated routes validate required fields, conditional phone and insurance data, and return explicit error codes.

Data integrity

Transactional writes

Identity, vendor configuration, compliance documents, and onboarding progress update inside a database transaction.

Async states

Failures stay actionable

Loading, permission, validation, empty, escalation, and completion states are represented in the interface.

Accessibility

Semantics included

Tab roles, alerts, input labels, focus states, reduced-motion handling, and control names are present in the implementation.

05 / Outcome

From open-ended request to a clear four-step booking flow.

The final experience made parallel agent work understandable to customers and actionable to operators. It reduced the booking journey to four steps while preserving intervention, verification, and recovery paths for the moments automation could not safely resolve alone.

~35%Faster booking flow

4 stepsFinal customer journey

0→1Product design through production frontend

What I would test next: operator response time by escalation reason, false-positive escalation rate, takeover completion and cancellation rates, and whether customers understand when control moves from agent to human.

Next case study

Claussal.ai
Every answer, traceable.