Skip to content

fix(hotel_receptionist): move book_room's sequencing and pricing rules into the tools - #6849

Open
u9g wants to merge 20 commits into
mainfrom
jason/book-room-tool-gated-state-machine
Open

fix(hotel_receptionist): move book_room's sequencing and pricing rules into the tools#6849
u9g wants to merge 20 commits into
mainfrom
jason/book-room-tool-gated-state-machine

Conversation

@u9g

@u9g u9g commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

_status()'s if-ladder was already a state machine β€” rendered as English, appended to every tool return, and obeyed only if the model remembered to. This points the same ladder at the tools themselves, and applies the same idea to the numbers the flow speaks.

Sequencing

Every tool stays listed to the model. Each step declares what it accepts, and a tool called outside its step returns the refusal instead of running:

choose_room did NOT run - nothing was recorded, no booking was made, and no
confirmation code exists. Available right now: give_up, set_stay. offer these
room types to the caller and ask which one they want - choose_room stays
closed until they've answered
step accepts
NEED_STAY set_stay
OFFERING set_stay
NEED_ROOM choose_room, set_stay
NEED_EXTRAS the recorders
CAPTURE the recorders + the dialogs for details still missing
READ_BACK the recorders
AWAIT_AGREEMENT the recorders + confirm_booking

The step is derived from the captured values on every read, so a correction lands the flow back on the right step with no rollback bookkeeping. OFFERING and READ_BACK are discharged by the caller replying, in on_user_turn_completed β€” the model can't offer the room options and act on them in one breath, or book before the read-back is out.

That makes structural what four instruction paragraphs used to request: a captured detail's dialog stops accepting calls so it can't be re-asked, choose_room is closed before a stay exists, and confirm_booking is closed until the read-back has been answered. The refusal names the outcome rather than only the next step β€” a bare "do X next" reads as progress and gets relayed as though the call had gone through, which for confirm_booking means speaking a confirmation code for a reservation that was never written.

Pricing

The flow quoted numbers before the things that determine them were settled.

The view moves the price. Rate is a per-room attribute: a city king is $240/night, an ocean king $260. Options were listed one line per type, so the price was spoken before the view was picked and then changed. _SQL_AVAILABILITY also selected a bare r.nightly_rate under GROUP BY r.type, which SQLite fills from an arbitrary row in the group β€” so the quoted rate wasn't even a defined property of the type, and ORDER BY sorted on that same arbitrary value. Now it groups by type and view with MIN(nightly_rate), and RoomTypeAvailability becomes RoomOption β€” one bookable pairing:

- queen 2beds, city view: 220 dollars/night
- king, city view: 240 dollars/night
- king, ocean view: 260 dollars/night
- suite, ocean view: 480 dollars/night

The extras move the price. choose_room took them as an argument and quoted a total in the same call, so a model that never asked passed an empty list and spoke a total that changed the moment breakfast came up. choose_room no longer takes extras and produces no total; set_extras records the answer and computes it, so no total exists until the question is answered. It takes one boolean per extra rather than a list β€” an extra never raised with the caller is otherwise indistinguishable from one they declined, since both are just absent from the array.

The room picked is now the cheapest match. _SQL_FREE_ROOM resolved by id, so a caller could be handed RM_301 at $280 while RM_202 sat free at $260. Ordering by nightly_rate first makes the quoted minimum the rate actually charged. The :prefer clause still wins outright, so a modification keeps the guest in their room.

Together: the rate on the line the caller picks, times the nights, plus extras, plus tax, equals what confirm_booking charges.

calc

"Quote ONLY this total - never compute your own" only appeared in _status()'s last branch, once name, email, phone and card were all captured. The window where a caller asks "what's that for three nights?" is right after set_stay, which hands over per-night rates and no total β€” so the ban was absent exactly where the model was most likely to multiply, and a figure worked out in its head is short by the 12% tax at best.

The ban is gone. In its place a calc tool evaluates + - * / and parentheses over numbers through an AST walk; names, calls, attributes and ** all raise. It's registered on the AgentSession rather than on one agent, so AgentActivity.tools hands it to every agent and every AgentTask in the call, the name / email / phone dialogs included.

Relationship to #6804

This supersedes that PR's option-line reshaping: a view a type doesn't have no longer has a line to be offered from, so the view-binding fix and its accompanying prose are unnecessary. Its computed weekday names and _not_booked() are independent and still worth landing.

Testing

  • make lint / make format-check
  • the refactor commit's _status() is byte-identical to main over all 128 combinations of the fields the ladder reads
  • each refusal exercised against the seeded DB β€” confirm_booking before anything, choose_room before a stay, choose_room at OFFERING, re-asking a captured detail, confirm_booking before the read-back and before extras β€” each records nothing
  • quoted rate Γ— nights + tax equals peek_stay_total for every (type, view) pairing, and every combination of the four extras
  • shifting the dates after extras are chosen reprices rather than leaving a stale figure: 2 nights with breakfast at $638.40 becomes $957.60 at 3
  • full path books with quoted_total == booking.total; modify_booking's view redirect and check_room_availability re-exercised against the new shape
  • calc rejects __import__('os').system('ls'), open('x'), 2 ** 10, a * 2, [1,2], 1/0

The example isn't covered by pytest --unit β€” nothing under examples/ is in the suite's scope β€” so verification is by driving the code directly.

No agents-js counterpart: its hotel example books through a single flat bookRoom tool, not a multi-step AgentTask.

u9g added 2 commits August 13, 2026 16:05
…tep enum

_status()'s if-ladder was already a state machine rendered as English. Name
the steps and dispatch off them, so the same ladder can drive something other
than a string. _missing() carries each uncaptured detail's dialog tool
alongside its directive.

No behavior change: _status() returns byte-identical strings, verified over
all 128 combinations of the fields the ladder reads.
…at is open

The flow's sequencing rules were prompt text the model had to remember to
follow. Each step now declares which tools it accepts; every tool stays listed
to the model, and one called outside its step returns the refusal instead of
running:

  choose_room did NOT run - nothing was recorded, no booking was made, and no
  confirmation code exists. Available right now: give_up, set_stay. <directive>

So a captured detail's dialog stops accepting calls and cannot re-ask it,
choose_room is closed until the options have been offered and the caller has
picked, and confirm_booking is closed until the read-back has been spoken and
answered - the caller's reply is what discharges both, via
on_user_turn_completed. Re-recording the stay or the room re-arms the
read-back.

The refusal names the outcome rather than only the next step: a bare "do X
next" reads as progress and gets relayed as though the call had gone through,
which for confirm_booking means speaking a confirmation code for a reservation
that was never written.

Instructions drop from 300 to 160 words: the clauses about ordering,
re-asking, and when confirm_booking may fire are now answered by the tools
themselves.
@u9g
u9g requested a review from a team as a code owner August 13, 2026 20:56

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

View 2 additional findings in Devin Review.

Open in Devin Review

Comment on lines +63 to +64
_Step.READ_BACK: _RECORDERS,
_Step.AWAIT_AGREEMENT: _RECORDERS | {"confirm_booking"},

@devin-ai-integration devin-ai-integration Bot Aug 13, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 Caller can no longer correct their name, email, phone or card once given, and the booking is written with the wrong detail

The dialogs that collect name, email, phone and card are refused once the detail has been captured (_ALLOWED at examples/hotel_receptionist/book_room.py:63-64 lists no dialog for the read-back and agreement steps), so a caller who says "wait, that email is wrong" at the read-back has no way to fix it and the flow is told to finish the booking instead.
Impact: A mistyped email, phone or card is permanently baked into the reservation - the confirmation goes to the wrong address - and the only escape is abandoning the whole booking.

Why the correction path is closed: step-gated dialogs plus a read-back status that points straight at confirm_booking

_allowed() (examples/hotel_receptionist/book_room.py:163-167) only adds dialog tools for the CAPTURE step, and only for details still uncaptured (_missing() at examples/hotel_receptionist/book_room.py:119-146). Once all four details exist, _step() returns READ_BACK then AWAIT_AGREEMENT, whose allowed sets are the recorders (set_stay / choose_room / set_extras) plus confirm_booking and give_up.

So a re-open of open_email_dialog hits _closed() (examples/hotel_receptionist/book_room.py:188-203), which returns "open_email_dialog did NOT run ..." followed by _status(), which at that point reads "the read-back is done - call confirm_booking() the moment the caller agrees" (examples/hotel_receptionist/book_room.py:238-239). The model is thus actively steered to commit the booking with the wrong value. Before this PR the dialogs were always callable and only English guidance discouraged re-asking, so corrections were possible.

A narrower gate would keep the anti-re-ask property while allowing corrections, e.g. permitting the dialogs at READ_BACK / AWAIT_AGREEMENT (where the caller has just heard the details) or refusing a dialog only when it is called immediately after that same detail was captured.

Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a problem

Comment thread examples/hotel_receptionist/calc.py Outdated

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

View 2 additional findings in Devin Review.

Open in Devin Review

Comment on lines +253 to +258
if step is _Step.NEED_EXTRAS:
return (
"room captured, no total yet - offer the extras and ask which the caller "
"wants, then call set_extras (empty list if they want none). Each extra "
"moves the total, so the total only exists once this is answered"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 The booking flow tells the model to record extras as an empty list, but the tool only accepts four yes/no answers

The status text handed back to the model asks for "set_extras (empty list if they want none)" (examples/hotel_receptionist/book_room.py:253-258) even though that tool now takes one boolean per extra, so the model is steered toward an argument shape that does not exist.
Impact: The step that records add-ons can stall or fail on rejected arguments, leaving the call stuck before any total can be quoted.

Stale wording left over from the list-valued signature

set_extras is declared as set_extras(breakfast: bool, valet: bool, late_checkout: bool, pets: bool) (examples/hotel_receptionist/book_room.py:384-386) and its own docstring says "Every extra takes an explicit true or false". The NEED_EXTRAS branch of _status() still describes the removed list argument ("empty list if they want none"), and that string is appended to choose_room's return and to every refusal produced by _closed(), so it is in front of the model at exactly the moment it has to call set_extras.

Suggested change
if step is _Step.NEED_EXTRAS:
return (
"room captured, no total yet - offer the extras and ask which the caller "
"wants, then call set_extras (empty list if they want none). Each extra "
"moves the total, so the total only exists once this is answered"
)
if step is _Step.NEED_EXTRAS:
return (
"room captured, no total yet - offer the extras and ask which the caller "
"wants, then call set_extras (true/false for each one; all four false if "
"they want none). Each extra moves the total, so the total only exists "
"once this is answered"
)
Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, it's now a function with 4 booleans

Comment on lines +79 to +86
self.session.generate_reply(
instructions=(
"Get the user's email address. First scan the conversation - if an email "
"address was already given (e.g. the user volunteered it before the task "
"started), use it via update_email_address rather than re-asking. Only ask "
"fresh when no email address is in the conversation yet."
)
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 Contact-detail collection steps can now reuse a value from earlier in the conversation without the user being asked

The collection steps for email, phone and address are now told to reuse whatever value appears earlier in the conversation instead of asking (generate_reply at livekit-agents/livekit/agents/beta/workflows/email_address.py:79-86), which contradicts the option callers use to force an explicit ask.
Impact: A value the user never gave for this purpose - possibly someone else's - can be accepted silently, and steps configured to always ask may now say nothing at all.

Conflict with require_explicit_ask and with the sibling name workflow

The same instruction was added to livekit-agents/livekit/agents/beta/workflows/address.py:78-85 and livekit-agents/livekit/agents/beta/workflows/phone_number.py:114-121.

  1. require_explicit_ask=True exists specifically to prevent this: it flags the update tool IGNORE_ON_ENTER so "the model can't silent-fill from chat_ctx during on_enter - it must produce an asking utterance first" (see the comment at livekit-agents/livekit/agents/beta/workflows/name.py:182-186, mirrored in each of the three edited files). _on_enter_ignored_tools drops the tool from the on-enter reply (livekit-agents/livekit/agents/voice/agent_activity.py:2984-2998), so the new instruction directs the model at a tool it cannot call and tells it not to ask β€” a plausible outcome is an empty/confused turn.

  2. When confirmation is not required (the default for the text modality, _confirmation_required), _update_email_impl / _update_phone_number_impl complete the task immediately, so a context-scraped value finishes the step with no user interaction at all.

  3. GetNameTask handles the same situation differently and safely: it instructs "ask a short confirmation question rather than asking from scratch" (livekit-agents/livekit/agents/beta/workflows/name.py:166-178) rather than silently filling.

Prompt for agents
The new on_enter instructions in beta/workflows/email_address.py, phone_number.py and address.py tell the model to lift a previously mentioned value out of the conversation and submit it via the update_* tool instead of asking. Two problems: (1) when require_explicit_ask=True the update tool is flagged IGNORE_ON_ENTER precisely so the model cannot silent-fill from chat_ctx and must ask first, so the instruction should be conditional on that flag; (2) with confirmation disabled (the default for text input) submitting a scraped value completes the task with zero user interaction, accepting a value the user never gave for this purpose. Align with GetNameTask (name.py), which instructs the model to ask a short confirmation question about the previously given value rather than silently reusing it, and skip the reuse instruction entirely when require_explicit_ask is set.
Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's the goal

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 3 additional findings in Devin Review.

Open in Devin Review

- both rooms arrive Friday, July 10th
- your room (room one) is for you and your wife, two adults, checking out Monday July 13th - you want a king with an ocean view
- the kids' room (room two) is two of them, checking out Sunday July 12th (they leave a day earlier) - a double-queen is fine, you don't care about the view
- the kids' room (room two) is two of them, checking out Sunday July 12th (they leave a day earlier) - a two-queen room is fine, you don't care about the view

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 Evaluation scenario for two family rooms can no longer be satisfied after the room types were merged

The two-room family scenario now says the second room's view does not matter (scenarios.yaml:3035) while its expected end state still demands an ocean-view room, so a correct run is graded as a failure.
Impact: That scenario reports a false failure on every otherwise-correct run.

Why the expected state became unreachable

double_queen was folded into queen_2beds (examples/hotel_receptionist/fake_data/seed.py:36,40). Previously every double_queen room was ocean-view, so booking that type without stating a view necessarily produced double_queen/ocean, matching the expected RM_206. Now queen_2beds spans city (RM_204 at 220, RM_303 at 240), garden (RM_205) and ocean (RM_206, RM_304); with no view requested, _SQL_FREE_ROOM β€” newly ordered by nightly_rate (examples/hotel_receptionist/hotel_db.py:1992) β€” hands out the cheapest, i.e. a city-view room. The grader resolves room_id to type || '/' || room_view (examples/hotel_receptionist/benchmark.py:71-79), so queen_2beds/city will never match the expected RM_206 (queen_2beds/ocean) at scenarios.yaml:3072. Either the scenario should ask for an ocean view for the kids' room, or the expected room should become a city-view queen.

Prompt for agents
In scenarios.yaml, the "Two rooms for the family, different checkout days" scenario states the kids' room view doesn't matter, but its expected_state pins RM_206 (queen_2beds/ocean). After the double_queen -> queen_2beds merge, queen_2beds now spans city/garden/ocean and _SQL_FREE_ROOM picks the cheapest free room, which is a city-view queen (RM_204). Because benchmark.py resolves room_id to type/room_view, the diff will always report a mismatch. Fix by aligning the two: either change the expected room to the cheapest queen_2beds that will actually be picked for July 10-12, or make the scenario state an ocean view for the second room.
Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is a problem

u9g added 18 commits August 14, 2026 17:15
_SQL_FREE_ROOM resolved the concrete room by id, so which room a caller got -
and therefore what they paid - depended on the seed's numbering. Rooms of one
type and view carry different rates (an ocean king is RM_202 at 260/night or
RM_301 at 280), so a caller could be handed the pricier one while a cheaper
match sat free.

Order by nightly_rate before id. The :prefer clause still wins outright, so a
modification keeps the guest in their existing room. book_room, update_booking
and peek_stay_total all resolve through this query, so the quote and the charge
stay the same room.
The rate is a property of a room, and the view decides which room the caller
gets: a city king is 240/night, an ocean king 260. Listing one line per type
meant the price was spoken before the thing that determines it was settled, so
the caller heard 240 and was charged 268 a night once they said "ocean".

_SQL_AVAILABILITY also selected a bare r.nightly_rate under GROUP BY r.type,
which SQLite fills from an arbitrary row in the group, so the quoted rate
wasn't even a defined property of the type - and ORDER BY sorted on that same
arbitrary value, making "what's the cheapest?" unanswerable.

Group by type and view with MIN(nightly_rate). RoomTypeAvailability becomes
RoomOption - one bookable type+view pairing and the cheapest free room in it -
and since _SQL_FREE_ROOM now hands out the cheapest match, the quoted rate is
the rate that gets charged. Verified across every pairing: quoted rate x nights
+ tax equals peek_stay_total.

Rendering moves to describe_room_options, shared by set_stay,
check_room_availability, and the view-unavailable errors in both booking flows.
A pairing that doesn't exist has no line to be offered from, so a garden-view
double queen can't be constructed by binding a neighbouring type's view.
Extras move the total the same way the view does - breakfast is 25/night, valet
35, and the pet fee is 50 flat - but choose_room took them as an argument and
quoted a total in the same call. A model that never asked passed an empty list,
spoke the total that came back, and the number changed as soon as the caller
mentioned breakfast.

choose_room no longer takes extras and no longer produces a total. It records
the room, returns that pairing's nightly rate and the extras priced for the
actual nights, and leaves the flow on NEED_EXTRAS. set_extras records the
answer and computes the total, so no total exists until the question has been
answered - an empty list is a real answer, tracked separately from the value
because "declined" and "never asked" are otherwise the same empty list.

_requote() is now the single place the total is computed, so shifting the dates
after extras are chosen reprices instead of leaving a stale figure: a 2-night
stay with breakfast at 638.40 becomes 957.60 at 3 nights, and the booking is
charged what was quoted.
set_extras(extras: list[RoomExtra]) let the model answer for whichever extras it
happened to think of - an extra never raised with the caller is indistinguishable
from one they declined, since both are simply absent from the array.

Four required booleans have no such gap: every extra is answered explicitly, and
"we never discussed valet" has no way to be expressed. All four false is still a
real answer, so declining everything remains one call.
The refusal already says it did nothing and lists what is available, at the
moment the model reads it. Restating that up front in the instructions teaches
a rule the tool return carries on its own.
… carry

Each had a home closer to where it applies:

- "collect in whatever order" - CAPTURE accepts every dialog for a detail
  still missing, so order-freedom is the tool set, not a request
- "set_stay's options are for YOU to offer, not to act on" - set_stay's own
  docstring says exactly this, and OFFERING closes choose_room until the
  caller has answered
- "follow the directive, don't narrate what the tool just did" - the directive
  is appended to every return as an imperative, and COMMON_INSTRUCTIONS bans
  narration in stronger terms than this restatement did
- "if the room sells out, just pick another - everything else stays captured" -
  confirm_booking's Unavailable path returns that sentence when it happens

_BOOK_ROOM_INSTRUCTIONS is 41 words, from 300 on main. What is left is the one
thing no tool return can say: mine the conversation for details already given
before asking for them.
…the audio hook

on_user_turn_completed runs only from AgentActivity.on_end_of_turn, the audio
end-of-turn path. A caller who types reaches the session through
generate_reply(user_input=...), which never calls the hook, so neither
obligation it cleared - the options have been offered, the read-back has been
spoken - was ever discharged. The flow armed OFFERING or READ_BACK, served the
tool-less step, and stayed on that step for the rest of the call: choose_room
and confirm_booking never reopened, so no booking could be written.

Both paths do append the caller's message to the history. _Owed replaces the two
booleans and clears itself off the caller's turn count instead - it stamps the
count the first time the flow reads the obligation as pending and disarms once
the count moves past it - so the gate behaves the same whether the caller speaks
or types.

The stamp is taken on that first read rather than in arm() because set_stay arms
the read-back while the name, email, phone and card are still being captured,
and the caller turns spent giving those are not answers to a read-back that has
not been spoken yet. _step() consults it only once _missing() is empty, which is
the moment the read-back is actually owed.
Both types named the same bed layout - two queens, sleeping four - so the
agent offered callers a choice between a thing and itself, with no
attribute it could explain the difference with. What actually separated
them was view and rate, and both are already columns: queen_2beds was the
city/garden rooms, double_queen the ocean ones.

Rooms 206 and 304 become queen_2beds. The spoken option list keeps its
shape, since availability groups by (type, view) and prices each pairing:
"double queen, ocean view: 260" now reads "queen 2beds, ocean view: 260".

The free-upgrade scenario is unaffected. Re-accommodation filters on
nightly_rate >= the rate already paid and never on type, so the cheaper
city/garden queens stay out of reach despite fitting four, and the suite
is still the cheapest room the resolver can reach.
scenarios.yaml writes every date as a literal against a pinned seed date of
2026-06-08, but nothing ever set HOTEL_TODAY, so hotel_db.TODAY fell through to
date.today() and the scenarios rotted as the wall clock drifted past them.

The pin sits above the first import that reaches hotel_db (benchmark), since
TODAY is a module-level constant frozen at import. Setting it on the parent's
environment is enough: job subprocesses inherit it under both spawn and fork.
The wrong-binding example named a garden-view double queen, a type that no
longer exists and a pairing that now does: RM_205 is the garden room and it
is a queen_2beds. Garden exists for no other type, and king is its
neighbour in the cheapest-first list, so a garden-view king is the binding
the split row actually prevents.
…urn order

The pass condition was a speech act in a fixed position - clarify before
listing options - which the room flow cannot satisfy: set_stay returns the
room types and holds choose_room until the caller picks one, so the options
list is where the ambiguity gets resolved. Grade the commitment instead: no
bed layout the caller never stated, and one room with two beds at the end.
The read-back directive listed dates, room, extras, total and card, so the
guest count was never confirmed aloud - a party size the caller corrected
could be booked without them ever hearing it back. The count decides which
rooms fit, so it belongs in the sentence they agree to.
…-asking

GetNameTask and the three credit-card sub-tasks scan the conversation on
enter before asking. GetEmailTask, GetPhoneNumberTask and GetAddressTask
were left with a bare "ask the user to provide ...", so a caller who
volunteers the value up front is asked for it a second time - the task sees
it in chat_ctx and its update tool is callable on enter, but nothing tells
it to look. Same wording as the tasks that already do this.
…idate

set_stay re-armed _must_offer on every call, so a caller correcting one date
at the read-back landed the flow back on OFFERING with only set_stay open -
confirm_booking closed and the model told to re-offer room types it had
already picked. Arm it only when the picked type+view pairing didn't survive
the new dates.
A caller can abandon anywhere, and the flows own the toolset while they do:
gemma called record_followup mid-booking when a caller had to leave, got
"Unknown function: record_followup", and promised a callback that was never
written. record_followup touches nothing but ctx.userdata.db, so it lifts out
of the mixin to sit beside calc in the session tools - reachable from the
flows and their dialogs, where the caller actually says it.

give_up now says leaving the flow reopens the other tools, the way
modify_booking's already does.
`feature` is near-unique per scenario, so there was no way to select the
scenarios that exercise book_room_task as a set. The 16 tagged here are the
ones whose caller is trying to book a room; the 11 with a deterministic end
state already assert a hotel_bookings row, and the other 5 are graded on
agent_expectations because their outcome is not deterministic.
2146a64 narrowed the _must_offer arm to a pairing that didn't survive the new
dates, but the flow's first set_stay has no pairing at all, so the arm still
fired on every booking - closing choose_room for a turn, in the very turn
_BOOK_ROOM_INSTRUCTIONS tells the model to record the room already discussed.

In SRJ_cZWP7jsuDPin the model spent that turn taking the card in conversation
instead. choose_room landed four turns later, but _card_last4 was never set, so
the flow sat in CAPTURE refusing confirm_booking; the model read that as a
system fault and ended the call with a callback and no booking. Arm only when
this call invalidated a room the flow had.
@u9g
u9g force-pushed the jason/book-room-tool-gated-state-machine branch from 7815bf1 to d4cf631 Compare August 14, 2026 21:20

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 4 additional findings in Devin Review.

Open in Devin Review

Comment on lines +1061 to 1062
old_activity._inline_task = None
self.__inactive_ev.set()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟑 A failed or interrupted hand-back permanently blocks every later sub-dialog in the call

The marker that says a sub-dialog is running is cleared only as the last statement after several hand-back steps (old_activity._inline_task = None at livekit-agents/livekit/agents/voice/agent.py:1061), so if any of those steps fails or the caller's turn is cancelled the marker stays set and every later sub-dialog in that call is refused.
Impact: A call can reach a state where the assistant can no longer collect any detail (name, email, card) and keeps saying the step cannot start.

Clear is outside a finally, after awaits that can raise

AgentTask.__await_impl sets old_activity._inline_task = self at livekit-agents/livekit/agents/voice/agent.py:912 and the new guard at :906 refuses any further AgentTask on that activity with a ToolError while it is set. The reset at :1061 sits at the end of the outer finally block, after await asyncio.shield(pending_on_enter_task) and await session._update_activity(old_agent, new_activity="resume", ...). If the resume raises (e.g. _update_activity raising "cannot resume agent: no existing active activity to resume" when the paused activity was torn down) or the awaiting task is cancelled again while unwinding, line 1061 never runs. Unlike the also-skipped __inactive_ev.set(), the stale _inline_task keeps working code broken: the resumed activity is still live and serving turns, but every subsequent await SomeAgentTask() from a tool hits the guard and returns the "already running for this turn" refusal. Wrapping the hand-back block in an inner try/finally that resets _inline_task (and sets __inactive_ev) would make the release unconditional.

Prompt for agents
In AgentTask.__await_impl (livekit-agents/livekit/agents/voice/agent.py), the inline-task guard flag `old_activity._inline_task` is set before the activity is paused (~line 912) and cleared as the final statement of the outer `finally` (~line 1061), after `await asyncio.shield(pending_on_enter_task)` and `await session._update_activity(old_agent, new_activity="resume", ...)`. If either of those awaits raises (resume failure, or re-delivered cancellation while unwinding), the flag is never cleared, and the new guard at ~line 906 then permanently rejects every subsequent AgentTask awaited on that still-live activity with a ToolError, so no further detail-capture dialog can ever run in that session. Restructure so the flag release (and the `__inactive_ev.set()` that shares its position) happens unconditionally, e.g. by wrapping the hand-back/resume portion of the finally in its own try/finally.
Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant