Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
139 changes: 138 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ Runs **Monday to Friday at 10:00 Melbourne** over a 72h window (~70 tickets) —

The window is on `updated>`, not `created>`, so a ticket the requester adds detail to days after opening it is fetched again — a created-window would never see it. 72h rather than the 24h between runs so a failed run doesn't drop a day and Monday still reaches back past the weekend. Neither the overlap nor the wider net duplicates posts, because of the dedup state above.

[Zendesk Resolve Positive Reviews](#zendesk-resolve-positive-reviews) runs first, as the unit's first `ExecStart`. Order matters: the triage query is `status<solved`, so a review the resolver solves leaves the window — running second would re-count reviews just closed. Its failure does not stop the digest, because the resolver is an optimisation for it rather than a precondition; the failure is still reported, so a resolver broken for weeks cannot pass for one with nothing to do.
[Zendesk Resolve Positive Reviews](#zendesk-resolve-positive-reviews) runs first, as the unit's first `ExecStart`. Order matters: the triage query is `status<pending`, so a review the resolver solves leaves the window — running second would re-count reviews just closed. Its failure does not stop the digest, because the resolver is an optimisation for it rather than a precondition; the failure is still reported, so a resolver broken for weeks cannot pass for one with nothing to do.

Run it by hand with `sudo systemctl start zendesk-digest.service`, which does exactly what the timer does. For anything narrower, invoke the scripts directly — `--window-hours`, `--max-tickets`, `--query`, and `--no-discord` to exercise the job without posting (that run records nothing, so the next one still reports the tickets it saw). Failures are reported by `OnFailure=zendesk-alert@%n.service` on the unit itself, which cannot be silently unsubscribed by a rename the way matching on a workflow's name could.

Expand Down Expand Up @@ -491,6 +491,143 @@ Zendesk is unreachable, a draft that survives the round trip byte for byte, an
English reply that reaches the customer as typed, a re-run that cannot write twice,
and an attachment URL that never reaches a channel-visible message.

## Zendesk Reply from a Private Note

The Discord path answers one ticket from the digest. This one answers a ticket from
inside Zendesk, where the queue is actually worked: an agent writes a private note
saying what the answer is, Claude writes it properly in the requester's language, and
the agent sends it with a second note.

```
claude: draft - attachments are only kept on the server for 14 days. A second
device that was offline for longer cannot fetch them.
```

Claude replies with a private note carrying the drafted reply, a back-translation,
and the brief it was written from. A draft usually offers two or three genuinely
different approaches, numbered, so the agent reads:

```
claude: reply 2
```

which publishes that option **verbatim** and moves the ticket to `pending`. A bare
`claude: reply` sends the only option when there is one, and refuses to guess when
there are several.

### The commands

| Command | What it does | Touches the customer |
| --- | --- | --- |
| `claude: draft - <brief>` | Compose the reply from the brief, in the requester's language. A second `draft` amends the one already there rather than starting over | no |
| `claude: reply [n]` | Publish the chosen option verbatim, status -> `pending` | **yes** |
| `claude: english` | Post the conversation, both sides, in English. Says so and writes nothing when the ticket is already English | no |
| `claude: explain` | Post what support usually replied to this kind of ticket, what was actually done about it, and the caveats | no |
| `claude: solve [reason]` | Solve without writing to the customer, for tickets that need no reply. The note records who decided and why | no comment, but **solving fires the CSAT automation** |

### Why a draft is always reviewed

[reply.py](zendesk_triage/reply.py) sends an English ticket immediately, because the
agent typed the exact words and there is nothing to check. Here Claude *composes* the
reply from a brief, so nobody has read that wording yet — every draft is reviewed,
English included. `reply` never re-composes: what was reviewed is what goes out, or
the review means nothing. To change a draft, write a new brief.

### What stops it drafting against itself

Claude's own draft note names both commands in its instructions. If those parsed as
commands, every draft would trigger another one, forever. Two independent guards:

- `COMMAND` only matches at the **start of a line**, and the instructions in a draft
note are written mid-line on purpose. `test_a_generated_draft_note_is_not_a_command`
asserts it.
- The command search skips notes authored by the API user, and the Zendesk trigger
should exclude that same user so a draft never reaches the webhook at all.

Give the automation its own Zendesk user rather than reusing an account a human signs
into — otherwise excluding it in the trigger also excludes that person's notes, and
the tool silently stops working for them.

### Who may command it

Only private comments count, so a customer typing `claude:` into a public reply is
ignored. The author must be an agent or admin — the set of people who can write a
private note at all. `ZENDESK_NOTE_AUTHORS` narrows that to named user ids; the role
check still applies, so an id on the list that is not an agent is still refused.

An unauthorised author **stops** the search rather than falling through to an older
command. Their note is the most recent instruction on the ticket, and quietly acting
on a previous one instead would be a surprising thing to do.

### What the model may write

The brief is the only source of facts. The system prompt forbids adding a version
number, a date, a retention period, a link or a timeline the brief does not contain —
and forbids claiming an action was taken unless the brief says it was. That second
rule is the important one: 183 solved tickets in this account tell a reporter their
Account ID "has been banned from communities we operate", and a reply asserting
something nobody did is the worst thing this can produce. Both rules are asserted by
`test_the_prompt_forbids_inventing_facts_and_actions`, so a prompt edit cannot
quietly drop them.

### Idempotency

Zendesk retries a webhook that does not answer cleanly, and the reply is written
before the run finishes — so without a guard, a slow run emails the customer twice.
Every outcome note carries `[claude:done:<comment id>]`, keyed on the commanding
comment rather than the ticket, because two briefs on one ticket are two commands and
the second must not be swallowed by the first one's marker. Refusals carry it too: a
command that cannot be satisfied is still a command that was answered.

### Tags

| Tag | Set by | Cleared by |
| --- | --- | --- |
| `claude-queued` | the Zendesk trigger, when the note lands | a successful run |
| `claude-drafted` | a draft being posted | the reply going out |
| `claude-sent` | the reply going out | — |
| `claude-error` | a refusal, with the reason in the note | the next successful run |

The tag is the durable queue and the webhook is only a latency optimisation. A relay
that is down leaves `claude-queued` on the ticket, so `tags:claude-queued` older than
a few minutes is the list of dropped jobs — a webhook-only design would lose them
silently. Two views are worth making: `tags:claude-queued` for what did not run, and
`tags:claude-drafted` for what is waiting on a human.

### Zendesk setup

A trigger, and a webhook it calls:

- **Webhook** — POST to `https://<host>/zendesk/notes`, JSON body `{"ticket_id":
"{{ticket.id}}"}`, signed. Put the signing secret in `ZENDESK_WEBHOOK_SECRET`;
without it the route refuses everything, because a URL that writes to customers
must not default to open.
- **Trigger** — conditions: *Ticket is Updated*, *Comment is Private*, *Comment text
contains `claude:`*, and *Current user is not* the automation user. Actions: notify
the webhook, and add the tag `claude-queued`.

### Required Secrets

| Secret | Description |
| --- | --- |
| `ZENDESK_WEBHOOK_SECRET` | Shared secret Zendesk signs the webhook with. Unset refuses every request |
| `ZENDESK_NOTE_AUTHORS` | *(optional)* Comma-separated Zendesk user ids allowed to command it. Unset means any agent or admin |
| `ZENDESK_NOTE_MODEL` | *(optional)* Overrides the model |

The Zendesk credentials and Claude authentication are the ones the digest already
uses. `RELAY_DRY_RUN` covers this path too: the whole run happens and nothing is
written.

### Local Testing

```
# what the webhook does, against a real ticket, writing nothing
python zendesk_triage/note_reply.py --ticket 27603 --dry-run
```

A ticket with no command note prints `no command note to act on` and stops, so this
is safe to point at anything.

## Workflow Failure Notificaiton

If a workflow fails and is in the list of workflows monitored by the failure notificaiton workflow, the failure notificaiton workflow will send a message to a discord webhook.
Expand Down
12 changes: 11 additions & 1 deletion deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,15 @@ DISCORD_GUILD_ID=
ALLOWED_USER_IDS=
ALLOWED_ROLE_IDS=

# Verifies Zendesk's webhook signatures, for the `claude:` private-note route.
# Empty refuses every note webhook: a URL that writes public comments must not
# default to open. Same value as the secret on the Zendesk webhook itself.
ZENDESK_WEBHOOK_SECRET=
# Optional. Comma-separated Zendesk user ids allowed to drive `claude:` notes.
# Empty means any agent or admin, which is already everybody who can write a
# private note. The agent/admin role check applies either way.
#ZENDESK_NOTE_AUTHORS=

# Optional, and read by both units. The numeric id of a multi-line text ticket
# field in Zendesk; the digest renders every non-English ticket it is about to post
# into English there — both sides of the conversation, timestamped — and the compose
Expand All @@ -143,7 +152,8 @@ ALLOWED_ROLE_IDS=
# -> Multi-line text, not customer-visible, then read the id off the field's URL.
#ZENDESK_ENGLISH_FIELD_ID=

# Uncomment to run the whole path and write nothing to Zendesk.
# Uncomment to run the whole path and write nothing to Zendesk. Covers both the
# Discord reply flow and the `claude:` note flow.
#RELAY_DRY_RUN=1
```

Expand Down
15 changes: 15 additions & 0 deletions deploy/nginx-webhooks.conf
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,21 @@ server {
proxy_connect_timeout 5s;
}

# Zendesk's turn: a `claude:` private note on a ticket. Same shape as above and
# for the same reasons — exact path, nothing that touches the body, since the
# HMAC signature covers the raw bytes too.
location = /zendesk/notes {
proxy_pass http://127.0.0.1:8080;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# The relay answers straight away and does the work in the background, so
# this waits on an acknowledgement rather than on Claude.
proxy_read_timeout 10s;
proxy_connect_timeout 5s;
}

location = /healthz {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
Expand Down
Loading