From f064b934d3353a5ac02c89b84345ee8627c1b72a Mon Sep 17 00:00:00 2001 From: eric-wang-1990 Date: Wed, 12 Aug 2026 19:27:07 -0700 Subject: [PATCH 1/7] feat(learning): add daily retrospective flow + bump engine to 654a31d0 Turn on the engineer-bot learning loop for this repo, mirroring databricks-sql-python (the reference). Three parts: - engineer-bot-learning.yml (new): daily schedule (17:23 UTC) + workflow_dispatch (string since / window-hours recovery inputs). Own job sharing ./.github/actions/ bot-prelude for Python + pinned engine install, PAT-free. No setup-jfrog step: install-bot-engine self-mints its pip credential via OIDC and never reads PIP_INDEX_URL from env, so a setup-jfrog step would be redundant (the engine install is the only fetch; the learning flow runs no npm build) and would leak a token-bearing PIP_INDEX_URL to the job env. actions:read added for Track B (lists engineer-bot author runs + downloads their logs via the App token). - .bot/config.yaml: add the retrospective block (log_path, branch_prefix, pr_label, plus the repo's existing prompts/retrospective_system.md as additive guidance) and close the loop with author.knowledge_log matching retrospective.log_path (.claude/knowledge/learning-log.md), so the author phase reads back what the retrospective learns. - bot-prelude: bump engine-ref d05dcb11 to 654a31d0 (engine main) so the pinned engine has the daily-cron retrospective + per-bot models. SDK/CLI (0.2.102 / 2.1.61) unchanged at that SHA, so this is a SHA-only bump. The schedule registers only once this lands on the default branch; validate before the first cron via a manual workflow_dispatch with a recent since. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 --- .bot/config.yaml | 20 ++++ .github/actions/bot-prelude/action.yml | 2 +- .github/workflows/engineer-bot-learning.yml | 124 ++++++++++++++++++++ 3 files changed, 145 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/engineer-bot-learning.yml diff --git a/.bot/config.yaml b/.bot/config.yaml index 3a081f26..ace955eb 100644 --- a/.bot/config.yaml +++ b/.bot/config.yaml @@ -96,6 +96,11 @@ author: issue_url: ISSUE_URL context_files: - issue_body.txt # {{issue_body}} + # Close the loop: the author phase reads this log back so fixes benefit from + # what the retrospective has learned. MUST match `retrospective.log_path` below + # — the retrospective WRITES that path; the author only READS it if the two + # agree. (Set together, they connect learning → application.) + knowledge_log: .claude/knowledge/learning-log.md # Engine orchestration for the author phase. `bug-fix` runs the plan → # author_tests → fix pipeline (write a failing test → fix the code → re-run to @@ -104,3 +109,18 @@ author: # testing specifics (commands, layout, live e2e requirement) live in # prompts/engineer/system.md, not here. flow: bug-fix + +# Daily learning extraction (retrospective flow). Run by engineer-bot-learning.yml +# via `python -m databricks_bot_engine.engineer_bot.retrospective`: over an adaptive +# look-back window it sweeps every merged PR (diff + review comments, which the +# engine gathers ITSELF per PR) plus engineer-bot author runs, and if the model +# finds a durable, reusable learning, opens/updates a single rolling PR appending a +# dated section to log_path. Human-gated — never commits the canonical log directly. +# Omitting this block makes the retrospective a no-op. The system prompt is the +# engine's built-in base; prompts/retrospective_system.md (present in this repo) is +# appended as additive guidance. +retrospective: + system_prompt: prompts/retrospective_system.md + log_path: .claude/knowledge/learning-log.md # MUST match author.knowledge_log above + branch_prefix: ai/learning-pr- + pr_label: engineer-bot-learning diff --git a/.github/actions/bot-prelude/action.yml b/.github/actions/bot-prelude/action.yml index c248a949..d7d0db90 100644 --- a/.github/actions/bot-prelude/action.yml +++ b/.github/actions/bot-prelude/action.yml @@ -30,7 +30,7 @@ inputs: # value to move every bot to a new engine commit; never @main. description: 'Engine commit SHA (full 40-char) to install.' required: false - default: 'd05dcb113332401b4aee8d6aa05c7107399ad44f' + default: '654a31d0d26d2b77bcb58e84f1e4abe018f9fd5a' engine-repo: description: 'owner/name of the engine repo.' required: false diff --git a/.github/workflows/engineer-bot-learning.yml b/.github/workflows/engineer-bot-learning.yml new file mode 100644 index 00000000..9caa9450 --- /dev/null +++ b/.github/workflows/engineer-bot-learning.yml @@ -0,0 +1,124 @@ +# Engineer Bot — learning (retrospective) extraction — DAILY CRON. +# +# Over an adaptive look-back window the engine gathers merged PRs (diff + review +# comments) AND recent engineer-bot author-run console logs ITSELF via the GitHub +# API — no in-workflow context gathering, no per-PR trigger — and if the model +# finds durable, reusable learnings, opens ONE ROLLING PR on a stable branch +# (`ai/learning-pr`), appending a dated section per day until a human merges it. +# Human-gated by design: it NEVER commits the canonical log directly. +# +# Own job (NOT `uses: databricks/databricks-bot-engine/...`): an external repo +# can't resolve the internal engine's reusable workflows ("not found"). It shares +# the SAME prelude the other bots use — ./.github/actions/bot-prelude (tokens + +# Node + pinned engine install) — so the engine pin stays single-sourced in +# bot-prelude's `engine-ref` default (no second SHA to drift). +# +# Opt-in is purely via the `retrospective:` block in .bot/config.yaml + this +# workflow; absent that block the engine phase is a clean no-op. +name: Engineer Bot — Learning + +on: + schedule: + # 17:23 UTC daily — off-peak, off-:00 minute (GitHub delays/drops on-the-hour crons). + - cron: '23 17 * * *' + workflow_dispatch: + inputs: + since: + description: 'ISO lower bound to shorten the window and recover a wedged flow. Empty = adaptive cursor.' + type: string + default: '' + window-hours: + # STRING, not number: a `type: number` workflow_dispatch input fails the + # whole run at startup ("workflow file issue") when combined with the + # `schedule` trigger. argparse coerces it to int downstream. + description: 'Fallback look-back window (hours) used only when there is no prior successful run.' + type: string + default: '24' + +permissions: + contents: write # push the learning branch / open the learning PR + pull-requests: write + # Track B lists engineer-bot author runs + downloads their logs via the App + # token; the engineer-bot App installation must ALSO carry actions:read (a + # missing scope surfaces as a 403 that fails the whole run — list_author_runs + # raises, no escape hatch). + actions: read + id-token: write # JFrog OIDC exchange for the engine/SDK/CLI install + +concurrency: + # One learning run at a time; a queued run waits rather than racing the rolling + # PR's branch. Not keyed on a PR number (this is a cron, no PR event). + group: engineer-bot-learning-cron + cancel-in-progress: false + +jobs: + learning: + environment: azure-prod # DATABRICKS_HOST / DATABRICKS_TOKEN live here + runs-on: + group: databricks-protected-runner-group + labels: linux-ubuntu-latest + timeout-minutes: 20 + steps: + # Checkout the default branch (the learning PR is cut from it) FIRST, so the + # local `./` composites below resolve. persist-credentials:false — the + # retrospective sets its own authenticated push remote (see the run step), + # so no token is left in .git/config. + - name: Checkout default branch (learning PR is cut from it) + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 + with: + fetch-depth: 0 + persist-credentials: false + + # No separate `setup-jfrog` step. install-bot-engine (via bot-prelude) does + # its own keyless OIDC→JFrog mint and passes the credential through job-local + # files + per-command --index-url flags; it never reads PIP_INDEX_URL / + # JFROG_ACCESS_TOKEN from the environment. A preceding setup-jfrog would be + # redundant (the engine install is the only fetch — the learning flow runs no + # `npm` build) AND harmful: setup-jfrog exports a token-bearing PIP_INDEX_URL + # to $GITHUB_ENV, exposing it to every later step including the model run. + # Mirrors the read-only sibling reviewer-bot.yml. + - name: Setup Python + uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0 + with: + python-version: '3.11' + + # Shared prelude: mint the engineer-bot token (opens the learning PR) + the + # engine-scoped token, set up Node, install the pinned engine (PAT-free). The + # engine pin comes from bot-prelude's `engine-ref` default — the SINGLE source + # of truth for every bot; there is no second SHA in this file to drift. + - name: Bot prelude (tokens + Node + engine install) + id: prelude + uses: ./.github/actions/bot-prelude + with: + app-id: ${{ secrets.ENGINEER_BOT_APP_ID }} + private-key: ${{ secrets.ENGINEER_BOT_APP_PRIVATE_KEY }} + + # NOTE: no git-identity step — the engine's retrospective configures the git + # user AND DCO sign-off itself from .bot/config.yaml `bot_login_prefix`. + # NOTE: no context-gather step — the daily-cron engine enumerates merged PRs + # + author runs itself over the adaptive window. + - name: Extract learnings + open rolling PR + env: + GH_TOKEN: ${{ steps.prelude.outputs.token }} + GITHUB_REPOSITORY: ${{ github.repository }} + # Only the `/serving-endpoints/` prefix matters: + # sdk_agent.translate_endpoint rewrites this to `.../serving-endpoints/anthropic` + # and discards the model path segment. The effective model comes from + # .bot/config.yaml `retrospective.model` (or the engine default). + MODEL_ENDPOINT: https://${{ secrets.DATABRICKS_HOST }}/serving-endpoints/anthropic/invocations + DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }} + RUNNER_TEMP: ${{ runner.temp }} + SINCE: ${{ inputs.since }} + WINDOW_HOURS: ${{ inputs.window-hours }} + # The retrospective pushes the learning branch with a plain `git push + # origin`, and the checkout ran persist-credentials:false — so set an + # authenticated push remote from the minted App token first, mirroring + # engineer-bot.yml's publish step. --since / --window-hours are passed only + # when provided via workflow_dispatch (the schedule trigger leaves them + # empty → the adaptive cursor drives the window). + run: | + git remote set-url origin "https://x-access-token:${GH_TOKEN}@github.com/${GITHUB_REPOSITORY}.git" + args=(--repo-dir "$GITHUB_WORKSPACE") + [ -n "$SINCE" ] && args+=(--since "$SINCE") + [ -n "$WINDOW_HOURS" ] && args+=(--window-hours "$WINDOW_HOURS") + python -m databricks_bot_engine.engineer_bot.retrospective "${args[@]}" From d16b566747fcd614825d60e2bcc123ad5a87b5dc Mon Sep 17 00:00:00 2001 From: eric-wang-1990 Date: Wed, 12 Aug 2026 20:15:40 -0700 Subject: [PATCH 2/7] fix(learning): drop retrospective.system_prompt (no prompt file in this repo) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The retrospective block pointed system_prompt at prompts/retrospective_system.md, which does not exist in this repo (only databricks-sql-kernel ships one; the sql-python reference correctly omits the key). The engine treats a set-but-missing system_prompt as a HARD ERROR, so the daily cron would fail every run. Drop the key so the flow uses the engine's built-in base prompt (engineer_prompts.RETRO_SYSTEM_PROMPT) — matching databricks-sql-python and the odbc learning PR. Also corrects the config comment, which wrongly claimed the file was "present in this repo." Caught by peco-review-bot on the sibling PRs. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 --- .bot/config.yaml | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/.bot/config.yaml b/.bot/config.yaml index ace955eb..a39af00d 100644 --- a/.bot/config.yaml +++ b/.bot/config.yaml @@ -116,11 +116,14 @@ flow: bug-fix # engine gathers ITSELF per PR) plus engineer-bot author runs, and if the model # finds a durable, reusable learning, opens/updates a single rolling PR appending a # dated section to log_path. Human-gated — never commits the canonical log directly. -# Omitting this block makes the retrospective a no-op. The system prompt is the -# engine's built-in base; prompts/retrospective_system.md (present in this repo) is -# appended as additive guidance. +# Omitting this block makes the retrospective a no-op. +# +# `system_prompt` is intentionally OMITTED: this repo ships no +# prompts/retrospective_system.md, so the engine's built-in base prompt is used. +# (A set-but-missing system_prompt file would be a hard error; an UNSET key uses +# the base. Add the key later only if you write a repo-specific additive prompt.) +# The learning log is created on first write — no seed file needed. retrospective: - system_prompt: prompts/retrospective_system.md log_path: .claude/knowledge/learning-log.md # MUST match author.knowledge_log above branch_prefix: ai/learning-pr- pr_label: engineer-bot-learning From c45c23710224c3deff9727b190eec8ae15175fb7 Mon Sep 17 00:00:00 2001 From: "peco-engineer-bot[bot]" Date: Thu, 13 Aug 2026 03:20:14 +0000 Subject: [PATCH 3/7] ai: apply changes for #499 (1 review thread) Addresses: - #3771964358 at .bot/config.yaml:103 Signed-off-by: peco-engineer-bot[bot] --- .claude/knowledge/learning-log.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 .claude/knowledge/learning-log.md diff --git a/.claude/knowledge/learning-log.md b/.claude/knowledge/learning-log.md new file mode 100644 index 00000000..b65b6b34 --- /dev/null +++ b/.claude/knowledge/learning-log.md @@ -0,0 +1,12 @@ +# Engineer-bot learning log + +This file is the shared knowledge log for the databricks-sql-nodejs engineer-bot. + +- The **retrospective** flow (see `.bot/config.yaml` → `retrospective.log_path`) + appends a dated section here whenever it distills a durable, reusable learning + from merged PRs and author runs. +- The **author** flow (see `.bot/config.yaml` → `author.knowledge_log`) reads this + file back so fixes benefit from what the retrospective has learned. + +No learnings have been recorded yet — the first retrospective PR will append the +initial dated section below. From 55149960b68b77bcef24c9ed26fbef338a3a9dc3 Mon Sep 17 00:00:00 2001 From: "peco-engineer-bot[bot]" Date: Thu, 13 Aug 2026 03:21:32 +0000 Subject: [PATCH 4/7] ai: apply changes for #499 (1 review thread) Addresses: - #3772141334 at .github/workflows/engineer-bot-learning.yml:101 Signed-off-by: peco-engineer-bot[bot] --- .bot/config.yaml | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.bot/config.yaml b/.bot/config.yaml index a39af00d..0ddcafb4 100644 --- a/.bot/config.yaml +++ b/.bot/config.yaml @@ -124,6 +124,14 @@ flow: bug-fix # the base. Add the key later only if you write a repo-specific additive prompt.) # The learning log is created on first write — no seed file needed. retrospective: + # Pin the model explicitly for parity with the sibling bots (reviewer-bot.yml / + # engineer-bot.yml both route to databricks-claude-opus-4-8). Learning + # extraction is the task that most benefits from the strongest model, so we do + # NOT leave it to the engine default — an unpinned key would silently track + # whatever that default is and could drift quality/cost out from under us. + # (translate_endpoint discards the model path segment, so this key is the only + # place the effective retrospective model is chosen.) + model: databricks-claude-opus-4-8 log_path: .claude/knowledge/learning-log.md # MUST match author.knowledge_log above branch_prefix: ai/learning-pr- pr_label: engineer-bot-learning From f7056bf71c8c056c20bb7c3d22faf03ac75eb379 Mon Sep 17 00:00:00 2001 From: "peco-engineer-bot[bot]" Date: Thu, 13 Aug 2026 03:24:19 +0000 Subject: [PATCH 5/7] ai: apply changes for #499 (1 review thread) Addresses: - #3771964358 at .bot/config.yaml:103 Signed-off-by: peco-engineer-bot[bot] --- .bot/config.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.bot/config.yaml b/.bot/config.yaml index 0ddcafb4..d4a21845 100644 --- a/.bot/config.yaml +++ b/.bot/config.yaml @@ -122,7 +122,9 @@ flow: bug-fix # prompts/retrospective_system.md, so the engine's built-in base prompt is used. # (A set-but-missing system_prompt file would be a hard error; an UNSET key uses # the base. Add the key later only if you write a repo-specific additive prompt.) -# The learning log is created on first write — no seed file needed. +# The log is seeded at `log_path` (see `.claude/knowledge/learning-log.md`) so the +# author phase has something to read before the first retrospective PR lands; the +# retrospective appends dated sections to that same file thereafter. retrospective: # Pin the model explicitly for parity with the sibling bots (reviewer-bot.yml / # engineer-bot.yml both route to databricks-claude-opus-4-8). Learning From 6c4a3e79a3839ef454d62d2c0810b86aef894d35 Mon Sep 17 00:00:00 2001 From: eric-wang-1990 Date: Wed, 12 Aug 2026 22:39:45 -0700 Subject: [PATCH 6/7] fix(learning): correct MODEL_ENDPOINT to the concrete serving-endpoint form MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The learning workflow set MODEL_ENDPOINT to `.../serving-endpoints/anthropic/invocations`. That is NOT translated the way the old comment claimed: sdk_agent.translate_endpoint has an early-return for URLs already containing `/serving-endpoints/anthropic`, which keeps the trailing `/invocations` — so the CLI appends `/v1/messages` and calls `.../serving-endpoints/anthropic/invocations/v1/messages`, which the gateway rejects with `400 Unsupported native API path`. Proof: databricks-sql-python's learning cron (same endpoint form) has failed every scheduled run with exactly this 400. The reviewer-bot flagged it on the sibling PRs (Low), and it is the real cause. Use the concrete `.../serving-endpoints/databricks-claude-opus-4-8/invocations` form that reviewer-bot.yml / engineer-bot.yml already use successfully: translate_endpoint strips `/invocations` to the `.../serving-endpoints/ anthropic` base the CLI needs. Comment corrected to explain the trap. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 --- .github/workflows/engineer-bot-learning.yml | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/.github/workflows/engineer-bot-learning.yml b/.github/workflows/engineer-bot-learning.yml index 9caa9450..4d3ebd94 100644 --- a/.github/workflows/engineer-bot-learning.yml +++ b/.github/workflows/engineer-bot-learning.yml @@ -101,11 +101,15 @@ jobs: env: GH_TOKEN: ${{ steps.prelude.outputs.token }} GITHUB_REPOSITORY: ${{ github.repository }} - # Only the `/serving-endpoints/` prefix matters: - # sdk_agent.translate_endpoint rewrites this to `.../serving-endpoints/anthropic` - # and discards the model path segment. The effective model comes from - # .bot/config.yaml `retrospective.model` (or the engine default). - MODEL_ENDPOINT: https://${{ secrets.DATABRICKS_HOST }}/serving-endpoints/anthropic/invocations + # Use the concrete `.../serving-endpoints//invocations` form (same + # as reviewer-bot.yml / engineer-bot.yml). sdk_agent.translate_endpoint + # strips it to the `.../serving-endpoints/anthropic` base the CLI needs. + # Do NOT use `.../serving-endpoints/anthropic/invocations` here: that hits + # translate_endpoint's already-v2 early-return, which keeps the trailing + # `invocations`, so the CLI appends `/v1/messages` → + # `.../anthropic/invocations/v1/messages` → HTTP 400 (unsupported path). + # The effective model is set by the engine default (no retrospective.model). + MODEL_ENDPOINT: https://${{ secrets.DATABRICKS_HOST }}/serving-endpoints/databricks-claude-opus-4-8/invocations DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }} RUNNER_TEMP: ${{ runner.temp }} SINCE: ${{ inputs.since }} From 664deeee9bd6e8d595e4a19de5b1dde85124059b Mon Sep 17 00:00:00 2001 From: eric-wang-1990 Date: Wed, 12 Aug 2026 22:40:42 -0700 Subject: [PATCH 7/7] fix(learning): drop retired retrospective.model key MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The model key was added earlier but is RETIRED in the engine (retrospective.py warns "retrospective.model is no longer honored") — the learning run always uses the engine-owned engineer-bot model. Keeping it only emits a per-run ::warning::. Remove it to match the other driver repos. Co-authored-by: Isaac Signed-off-by: eric-wang-1990 --- .bot/config.yaml | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/.bot/config.yaml b/.bot/config.yaml index d4a21845..dac7eaa3 100644 --- a/.bot/config.yaml +++ b/.bot/config.yaml @@ -126,14 +126,11 @@ flow: bug-fix # author phase has something to read before the first retrospective PR lands; the # retrospective appends dated sections to that same file thereafter. retrospective: - # Pin the model explicitly for parity with the sibling bots (reviewer-bot.yml / - # engineer-bot.yml both route to databricks-claude-opus-4-8). Learning - # extraction is the task that most benefits from the strongest model, so we do - # NOT leave it to the engine default — an unpinned key would silently track - # whatever that default is and could drift quality/cost out from under us. - # (translate_endpoint discards the model path segment, so this key is the only - # place the effective retrospective model is chosen.) - model: databricks-claude-opus-4-8 + # No `model:` key: it is RETIRED in the engine (retrospective.py warns + # "`retrospective.model` is no longer honored") — the learning run always uses + # the engine-owned engineer-bot model (databricks-claude-opus-4-8[1m], override + # via the ENGINEER_BOT_MODEL_OVERRIDE workflow env). Setting it here only emits a + # per-run ::warning::. Matches the other driver repos' retrospective blocks. log_path: .claude/knowledge/learning-log.md # MUST match author.knowledge_log above branch_prefix: ai/learning-pr- pr_label: engineer-bot-learning