Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/crisp-triage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ jobs:
set +e
git config user.name "tg-autopilot"
git config user.email "tg-autopilot@users.noreply.github.com"
git add state/cursor.json state/active-notified.json state/escalated.json state/investigated.json
git add state/cursor.json state/active-notified.json state/escalated.json state/investigated.json state/resolved-seen.json
git diff --cached --quiet || git commit -m "Advance Crisp triage state [skip ci]"
for attempt in 1 2 3 4 5; do
git push && exit 0
Expand Down
20 changes: 0 additions & 20 deletions .github/workflows/diagnose-message-shape.yml

This file was deleted.

15 changes: 8 additions & 7 deletions PHASE2-SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,20 +60,21 @@ Phase 1's PAT has **Contents: Read-only**. Stage 1 needs to commit the advanced
- `themegrill`'s org default repo permission happens to be **write** for all members, so once `tg-autopilot` is a member there, it already has write access to every repo — no per-repo collaborator setup was needed, just the org invite and this token.
- The workflow picks whichever token matches a conversation's target repo's org: `startsWith(matrix.repo, 'themegrill/') && secrets.BOT_TOKEN_THEMEGRILL || secrets.BOT_TOKEN`. Onboarding a third org later means the same pattern: a new PAT, a new secret, one more branch in that expression (or, once there are more than two, worth switching to a lookup table instead).

## 4c. Escalating a still-open conversation, without waiting for it to resolve
## 4c. What actually triggers a full investigation

Normally, full investigation only ever runs on a *resolved* conversation — see the reasoning in `prompts/crisp-triage-agent.md`. Two exceptions, both handled in `crisp-classify.mjs`, not the lightweight active-dedupe script:
**Policy: a resolved conversation is trusted as fully handled by support.** Resolving it — first time or the tenth time — never triggers investigation on its own; the resolved-conversation loop in `crisp-classify.mjs` only records "session X seen resolved at time T" into `state/resolved-seen.json`, nothing more. Three things can still trigger a full investigation:

- **Manual**: a support agent adds a private note containing `!tg-autopilot investigate`. Skips the cheap classifier entirely — a human already made the call — and goes straight to full investigation. Works any number of times; each *new* note re-triggers it (tracked by counting matching notes per conversation, not by trying to identify "which" note, since two notes can have identical text). Detection has two layers: the cheap path only fetches full messages for conversations already touched since `cursor.last_checked` (adding a note is itself an update); separately, `searchConversationsForManualTrigger()` searches Crisp directly for the trigger phrase and appends anything found there too, regardless of `fetchActiveConversations`' page cap. The second layer exists because the first one alone isn't reliable on a high-volume account — confirmed for real on `THEMEGRILL`: a manually-noted conversation didn't rank in the top 200 most-recently-updated active conversations because 200+ *other* conversations were touched in the same window, so it never got its message history fetched at all. A manual trigger is an explicit human action and shouldn't ever silently fail to reach the pipeline just because the account is busy.
- **Automatic**: a conversation open longer than **12 hours** (measured from `active.last`, not `created_at` -- a conversation that resolved and later reopened after a long gap shouldn't look artificially old) gets checked by the same cheap classifier used for resolved conversations. If it agrees this looks like a real bug/feature, it's escalated the same way. This fires **at most once per conversation** automatically, and never for a conversation older than **30 days** (`AUTO_ESCALATE_MAX_HOURS` in `crisp-classify.mjs`) -- a backlog that's sat untouched that long is treated as intentionally left open, not a scan miss. If something changes later and it genuinely needs another look, that's what the manual note is for.
- **Manual**: a support agent adds a private note containing `!tg-autopilot investigate`. Skips the cheap classifier entirely — a human already made the call — and goes straight to full investigation, on the *full* transcript regardless of any of the below. Works any number of times; each *new* note re-triggers it (tracked by counting matching notes per conversation, not by trying to identify "which" note, since two notes can have identical text). Detection has two layers: the cheap path only fetches full messages for conversations already touched since `cursor.last_checked` (adding a note is itself an update); separately, `searchConversationsForManualTrigger()` searches Crisp directly for the trigger phrase and appends anything found there too, regardless of `fetchActiveConversations`' page cap. The second layer exists because the first one alone isn't reliable on a high-volume account — confirmed for real on `THEMEGRILL`: a manually-noted conversation didn't rank in the top 200 most-recently-updated active conversations because 200+ *other* conversations were touched in the same window, so it never got its message history fetched at all. A manual trigger is an explicit human action and shouldn't ever silently fail to reach the pipeline just because the account is busy.
- **Reopen after resolve**: a conversation `state/resolved-seen.json` already has a record for shows up in the *active* list again (the customer replied after it was marked resolved). Investigated using only the messages timestamped after that recorded resolve time — whatever was there before is assumed already covered by support's original resolve. If it gets resolved again before a `crisp-triage` run ever catches it in this in-between "active again" state, it's never investigated for that reopen at all — the resolved-conversation loop just records the newer resolve time on its next pass, and there's nothing left to catch. This is intentional, not a bug: by the same policy, a second resolve is trusted as handled too.
- **Automatic, stale-and-never-resolved**: a conversation open longer than **12 hours** (measured from `active.last`, not `created_at`) that has *never* been resolved at all gets checked by the same cheap classifier. If it agrees this looks like a real bug/feature, it's escalated the same way. This fires **at most once per conversation** automatically, and never for a conversation older than **30 days** (`AUTO_ESCALATE_MAX_HOURS` in `crisp-classify.mjs`) -- a backlog that's sat untouched that long is treated as intentionally left open, not a scan miss. This path is unrelated to the reopen path above — it exists for tickets that were *never* marked resolved, which the "trust the resolve" policy says nothing about.

**Onboarding a new account: run `Seed escalated state for a new Crisp account` (`.github/workflows/seed-escalated.yml`, `workflow_dispatch`, input = the account key) once, before that account's first scheduled `crisp-triage` run.** `escalated.json` has no history for a brand-new account, so every one of its currently-active conversations already past 12 hours old looks auto-escalation-eligible on day one -- confirmed for real onboarding User Registration: ~83% of its ~400-conversation active backlog auto-escalated within the first few minutes of a single run. The seed workflow marks the current backlog as already-escalated so none of it fires; only conversations that go stale *after* seeding will auto-escalate from then on. It doesn't touch the manual-note path -- `!tg-autopilot investigate` still works on any of those backlog conversations if someone wants one looked at anyway.

Both paths write to `state/escalated.json` (also committed) so neither one repeats itself needlessly.
All three paths write to `state/escalated.json` and/or `state/resolved-seen.json` (both committed) so none of them repeats itself needlessly.

**Separately**, `state/investigated.json` (also committed) tracks every session_id that has ever completed a full Stage 2 investigation, from *any* path (resolved, auto-escalation, or manual note). The resolved-conversation loop and the auto-escalation branch both skip a session already in this set — otherwise a conversation that resolves, gets reopened by the customer, and resolves again looks like a brand-new resolved conversation every cycle, with no memory that it was already fully investigated (confirmed for real: one conversation was reinvestigated 6 times over 3 days, posting a fresh "not a real defect" note each time — `crisp-post-note.mjs`'s dedup only catches a repeated note that references the *same GitHub issue URL*, so a no-issue conclusion is never deduped). The manual-note path deliberately does **not** check this set — a human explicitly asking to (re-)investigate should always go through.
**Separately**, `state/investigated.json` (also committed) tracks every session_id that has ever completed a full Stage 2 investigation, from *any* path. The reopen and stale-auto-escalation branches both skip a session already in this set for the *same* unresolved content — a conversation reopening with a genuinely new, unrelated problem after already being investigated once still gets picked up, since the reopen path is driven by `resolved-seen.json`'s timestamp, not this set. The manual-note path deliberately does **not** check this set — a human explicitly asking to (re-)investigate should always go through.

This does not retroactively clean up notes already posted before the fix, and it does mean a conversation that reopens with a genuinely new, unrelated problem after already being investigated once won't get a second automatic look — only a fresh manual `!tg-autopilot investigate` note reaches it at that point.
**Rollout note**: `state/resolved-seen.json` starts empty when this policy first ships. A conversation that was resolved *before* that point and reopens shortly after has no prior "seen resolved" record yet, so that first reopen looks like a first-time resolve and gets skipped rather than recognized as a reopen — it self-corrects from its next resolve onward. Accepted as a one-time gap rather than backfilling the whole history.

## 4d. Investigate-job concurrency and OpenAI rate limits

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ We use a machine user instead of a GitHub App because it keeps one identity acro
|---|---|---|
| **PR build-zip comment** — builds a plugin/theme zip on every ready-for-review PR, uploads it, and posts/updates one comment with a direct download link | `.github/workflows/pr-build-zip.yml` + `.caller.yml` | [SETUP.md](SETUP.md) |
| **PR review automation** — requests Copilot as a reviewer when a PR opens, and lets team members trigger a Copilot code review by commenting a trigger phrase | `.github/workflows/copilot-review-on-comment.yml` + `.caller.yml` | inline comments in the workflow file |
| **Crisp → AI → GitHub issue** — hourly pipeline that reads resolved (and some still-open) Crisp support conversations, classifies whether they describe a real bug/feature, and either files a GitHub issue, comments on an existing one, or leaves a note back in the Crisp conversation | `.github/workflows/crisp-triage.yml`, `scripts/`, `prompts/`, `config/`, `state/` | [PHASE2-SETUP.md](PHASE2-SETUP.md) |
| **Crisp → AI → GitHub issue** — scheduled pipeline over Crisp support conversations. A resolved conversation is trusted as fully handled by support and is never classified on its own; investigation only happens on a manual `!tg-autopilot investigate` note, a conversation reopening after being resolved (just the new part), or a stale conversation that was never resolved at all. See § 4c for the full policy | `.github/workflows/crisp-triage.yml`, `scripts/`, `prompts/`, `config/`, `state/` | [PHASE2-SETUP.md § 4c](PHASE2-SETUP.md#4c-what-actually-triggers-a-full-investigation) |

## Maintainer skills

Expand Down
Loading