Did the AI Assistant Understand the Request or Actually Complete It?

Trace an appointment request through intent, missing details, confirmation and the calendar’s final receipt.

A desk speaker sits beside a calendar with movable appointment markers.
A proposed appointment and a saved calendar event need different evidence.

“Move my Tuesday appointment to the same time next week” sounds complete until the calendar shows two Tuesday appointments. A conversational AI model can infer the requested action and keep track of details across turns. A successful exchange depends on resolving which event the person meant and checking the calendar after the action.

What did the person specify?

The Schema-Guided Dialogue researchers represented services with intents and slots, such as an action, date or place. Their benchmark includes services unseen during training. It lets researchers check whether a dialogue system carried the correct values through a conversation. A benchmark state can be right even when a real calendar service is unavailable. The Schema-Guided Dialogue study Schema-Guided Dialogue task definitions

Violet and blue rectangular modules glow against a dark background.
The request contains a proposed action and details the assistant must resolve.

Carry a correction all the way through

The user may correct “Tuesday” to “Thursday,” or mean the clinic visit rather than the planning call. Update the chosen event and its dates, preserve the time zone and ask for the missing distinction. Show the proposed change in a form the person can verify before committing it. An assistant can often ask a shorter question by naming the two matching events.

A person appears amid translucent images.
A correction later in the conversation should replace the earlier value.

What would establish completion?

The 2024 τ-bench study evaluated simulated service conversations by comparing the final database state against the intended state and testing success across repeated runs. A persuasive confirmation message cannot substitute for that check. The τ-bench tool-use evaluation

A tool can reject an update because the event changed since it was read. A useful completion message contains the event identifier, the accepted time and the service’s result. For a test, score correctly interpreted requests, necessary clarification, wrong edits and confirmed completions separately. Voice activation has its own earlier success criterion. When Does a Voice Device Think It Heard Its Wake Word?

A person looks at a large display beside a window.
The event service supplies the receipt for a completed change.

Try it with the evidence

Reconstruct an appointment change

Source notes. Fictional calendar. On Tuesday, June 3, event A is “Planning,” 10:00, and B is “Clinic,” 14:00. User says “move Tuesday’s appointment a week later,” then “the clinic one.” The tool returns “B conflict: calendar changed; no write.”

Prompt. Write the confirmed intent, selected event and proposed date and time. Draft an accurate one-sentence status to the user and identify the next read-only check.

Check your reading

The intent is to move B to Tuesday, June 10 at 14:00 in the original calendar time zone. The write failed. A truthful status says the clinic appointment has not yet been moved because the calendar changed. Refresh B and inspect the current event before proposing another update.

Illustrated field notes

Understanding the person and context
Several people appear in an illustration with speech bubbles.
A reference to something on screen can change what a spoken request means.
A watch with a red strap rests on a pale surface.
A reminder arriving on time does not confirm that an event was rescheduled.
A speaker, phone and other devices sit together.
A request made on one device should refer to the same event on another.
An illustrated figure studies panels on a wall.
Two similarly named calendar events may belong to different people.
Permission, execution and confirmation
Colored rings trace paths on a dark surface.
Account access and user consent define which calendar the service can change.
Connected multicolored circles form a diagram.
An unfamiliar service may use different names for a familiar action.
A glowing outline of a face appears against a dark background.
The final message should distinguish a proposed change from a confirmed write.