Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/skills/stackql-provider-development/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The detail lives in `references/`; `scripts/` holds two generic helpers:
## Ground rules

- **Read before inventing.** Clone the reference repos ([references/reference-repos.md](references/reference-repos.md)) beside the provider repo and use `find_extension_examples.sh` to see a primitive shipped for real before authoring it. Sibling build repos carry NOTES.md findings - reuse them, do not re-derive.
- **Latest toolchain, always.** `npm view @stackql/provider-utils version`, `npm view @stackql/pgwire-lite version`, `npm view @docusaurus/core version` before starting; pin `^latest` in package.json. Node >= 20, `"type": "module"`. The lockfile pins what CI installs, so generation stays byte-for-byte reproducible; a toolchain bump is an explicit commit that regenerates the artifacts.
- **Latest toolchain, always.** `npm view @stackql/provider-utils version`, `npm view @stackql/pgwire-lite version`, `npm view @apidevtools/swagger-parser version`, `npm view @docusaurus/core version` before starting; pin `^latest` in package.json. Node >= 22.19 (Node 20 reached end of life in April 2026; swagger-parser 13 needs 22.19), `"type": "module"`. `js-yaml` stays on 4.x: provider-utils emits with js-yaml 4, and v5 drops the default export, removes the `quotingType` dump option and changes scalar quoting, so that bump is a coordinated change with provider-utils. The lockfile pins what CI installs, so generation stays byte-for-byte reproducible; a toolchain bump is an explicit commit that regenerates the artifacts.
- **Deterministic pipeline.** Every step is a script that validates and fails without writing. Manual decisions are rules in scripts (mapping tables, skip codes, fix classes), never hand-edits to generated YAML or CSV. A regeneration reproduces the committed artifacts byte-for-byte (CI checks this).
- **`all_services.csv` is the contract.** Committed and diffed on every regeneration; a method moving resource or a resource renamed is a breaking change to review, not noise.
- **Environment.** Linux, macOS or WSL (on Windows prefer WSL and the latest `stackql` release on its PATH). GNU make + bash. `stackql` resolves as `$STACKQL`, `./stackql`, then PATH.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,4 +18,4 @@ The example section has one shape on every provider site, so the table of conten

## CI

`build-and-test.yml`: on push/PR - `npm ci`, `stackql/setup-stackql@v2`, `make fetch-spec` (warn on drift, build from the pin), build steps, **fail on uncommitted generation drift** (`git add -N . && git diff --quiet -- provider-dev/openapi provider-dev/config provider-dev/source`), `make test-offline test-integration test-meta`, `make docs`. A `smoke` job gated on secrets (skipped with a notice otherwise; never the expensive lifecycle). A weekly `spec-drift` job that fetches, compares with the pin, and opens a labelled issue with the diff stat. Web deploy workflows build the site from `main`.
`build-and-test.yml`: on push/PR - `actions/setup-node@v7` on Node 22, `npm ci`, `stackql/setup-stackql@v2`, `make fetch-spec` (warn on drift, build from the pin), build steps, **fail on uncommitted generation drift** (`git add -N . && git diff --quiet -- provider-dev/openapi provider-dev/config provider-dev/source`), `make test-offline test-integration test-meta`, `make docs`. A `smoke` job gated on secrets (skipped with a notice otherwise; never the expensive lifecycle). A weekly `spec-drift` job that fetches, compares with the pin, and opens a labelled issue with the diff stat. Web deploy workflows build the site from `main` with `yarn install --frozen-lockfile`, cached on `website/yarn.lock`; the first `make website` writes that lockfile, so commit it before enabling them.
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
- **NOTES.md** - numbered findings with evidence (what was measured, against what, what was decided and why), a Blockers list of what only a live run can establish, and the testing requirements. It is the memory of the build; a future refresh reads it before touching a rule.
- **README.md** - the numbered build guide (steps 0-8: pin, inventory, split, mappings, normalize, generate, test, publish, docs) with counts that match the committed artifacts, the Makefile targets, credentials, and the CI description.
- **.env.example** - every live-test variable with a one-line comment; never a real value.
- **SECURITY.md** and **LICENSE** - carried from the template, placeholders rewritten by `bin/init-provider.sh`: the repository's private advisory form, the scope split between the provider and the engine, MIT. Not a build artifact; update only when the policy changes.

## Hand-over

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ tests/
smoke_test.py
website/ # Docusaurus microsite
.github/workflows/ # build-and-test.yml, prod-web-deploy.yml, test-web-deploy.yml
CLAUDE.md NOTES.md README.md .env.example
CLAUDE.md NOTES.md README.md SECURITY.md LICENSE .env.example
```

Makefile targets (bash shell, `.DEFAULT_GOAL := help`, `##` help comments): `deps`, `fetch-spec` (verify pin), `refresh-spec` (`--update`), `inventory`, `split`, `mappings` (rm CSV + analyze + map), `pre-normalize`, `normalize`, `generate` (rm output + generate + post-process), `post-process`, `build` (the chain), `test-offline`, `test-integration`, `test-meta` (start server; run; stop; preserve exit status), `test`, `venv`, `smoke`, `smoke-live`, `smoke-read-only`, `smoke-<gated-lifecycle>`, `smoke-cleanup`, `docs`, `website`, `website-start`, `clean`, `all` = `deps build test docs website`. `make all` never needs credentials and never bills. The smoke targets source `.env` when present (`set -a; source <(tr -d '\r' < .env); set +a`).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Uplifting an existing provider

1. Bump `@stackql/provider-utils` / `@stackql/pgwire-lite` / Docusaurus to latest; `npm install`.
1. Bump `@stackql/provider-utils` / `@stackql/pgwire-lite` / `@apidevtools/swagger-parser` / Docusaurus to latest (`js-yaml` stays on 4.x - see the SKILL.md ground rules); Node >= 22.19 in `engines`, the Makefile header, the README prerequisites and every `setup-node` step; `npm install`. Add `SECURITY.md` and `LICENSE` from the template when absent.
2. `make fetch-spec`; if drift, review the diff, extend fix classes, `make refresh-spec`, record in NOTES.md.
3. Re-run the pipeline; diff `all_services.csv` - every moved method or renamed resource is a decision to make explicitly (keep the old name via a rule unless the rename is worth the break).
4. Add what is missing against this list: env-var scoping, naive bodies, snake surface, pagination, pushdown, transforms, objectKeys, lifecycle EXECs attached to their resources, views, GraphQL merge, skip codes for what is not mappable.
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:

- uses: actions/setup-node@v7
with:
node-version: 20
node-version: 22
cache: npm

- name: Install dependencies
Expand Down Expand Up @@ -115,7 +115,7 @@ jobs:

- uses: actions/setup-node@v7
with:
node-version: 20
node-version: 22
cache: npm

- uses: actions/setup-python@v7
Expand Down Expand Up @@ -147,7 +147,7 @@ jobs:

- uses: actions/setup-node@v7
with:
node-version: 20
node-version: 22
cache: npm

- name: Install dependencies
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/prod-web-deploy.yml.disabled
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,11 @@ jobs:
with:
fetch-depth: 0

# website/yarn.lock is written by the first `make website`; commit it
# before enabling this workflow (the install below is --frozen-lockfile).
- uses: actions/setup-node@v7
with:
node-version: 20
node-version: 22
cache: yarn
cache-dependency-path: website/yarn.lock

Expand Down Expand Up @@ -54,5 +56,3 @@ jobs:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v5
with:
working-directory: website/build # Ensures the correct directory is used for deployment
4 changes: 3 additions & 1 deletion .github/workflows/test-web-deploy.yml.disabled
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,11 @@ jobs:
with:
fetch-depth: 0

# website/yarn.lock is written by the first `make website`; commit it
# before enabling this workflow (the install below is --frozen-lockfile).
- uses: actions/setup-node@v7
with:
node-version: 20
node-version: 22
cache: yarn
cache-dependency-path: website/yarn.lock

Expand Down
4 changes: 2 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ TODO(template): one bullet each, as they are decided:

## Toolchain rules

- Latest `@stackql/provider-utils` and `@stackql/pgwire-lite` (check npm before starting: `npm view @stackql/provider-utils version`); Docusaurus `^3.10.x`. Node >= 20, `type: module`. `package-lock.json` pins what CI installs; a toolchain bump is an explicit commit that regenerates the artifacts.
- Latest `@stackql/provider-utils`, `@stackql/pgwire-lite` and `@apidevtools/swagger-parser` (check npm before starting: `npm view @stackql/provider-utils version`); Docusaurus `^3.10.x`. Node >= 22.19 (Node 20 reached end of life in April 2026; swagger-parser 13 needs 22.19), `type: module`. `js-yaml` stays on 4.x: provider-utils emits with js-yaml 4, and v5 drops the default export, removes the `quotingType` dump option and changes scalar quoting, so that bump is a coordinated change with provider-utils. `package-lock.json` pins what CI installs; a toolchain bump is an explicit commit that regenerates the artifacts.
- Linux, macOS or WSL: GNU make + bash, a `stackql` binary (`$STACKQL`, `./stackql`, then PATH; `bin/start-server.sh` downloads one if none is found), Python 3 and yarn. The Makefile is the operator surface (`make help`).
- The two provider-utils CLI entry points are npm scripts invoked through `node` (never `.bin` shims); flags go after `--`.

Expand Down Expand Up @@ -62,7 +62,7 @@ tests/
website/ # Docusaurus microsite (shared stackql/docusaurus-config vendored at build)
.github/workflows/ # build-and-test.yml (pin check, build, drift check, 3 test layers, gated smoke,
# weekly spec-drift); web deploys (.disabled until GitHub Pages is configured)
CLAUDE.md NOTES.md README.md .env.example
CLAUDE.md NOTES.md README.md SECURITY.md LICENSE .env.example
```

## Build pipeline
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# `make all` never needs credentials and never bills; the live smoke targets
# (`make smoke*`) are separate and source .env when present.
#
# Requirements: Node >= 20, GNU make + bash, a stackql binary ($STACKQL,
# Requirements: Node >= 22.19, GNU make + bash, a stackql binary ($STACKQL,
# ./stackql or on PATH - bin/start-server.sh downloads one if none is found),
# Python 3 (a venv with pystackql is created on demand for the smoke suite),
# yarn for the website. Runs on Linux, macOS or WSL.
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ What the template is not: a publisher. Publishing to the public registry is a se

## Prerequisites

- Node.js >= 20
- Node.js >= 22.19
- A `stackql` binary (`$STACKQL`, `./stackql`, or on `PATH`; `bin/start-server.sh` downloads one if none is found)
- GNU make and bash (Linux, macOS or WSL); Python 3 for the smoke suite; yarn for the website
- For live smoke tests: a dedicated dev account for the provider (never a production account) and its credentials in `.env` (see `.env.example`)
Expand Down Expand Up @@ -172,6 +172,10 @@ A user can override at runtime with `stackql --auth='{"<provider>": {...}}'`.

Contributions are welcome - rules in scripts, `make build && make test`, then a pull request.

## Security

Report vulnerabilities privately through the repository's advisory form, never in a public issue - see [SECURITY.md](SECURITY.md).

## License

MIT
MIT - see [LICENSE](LICENSE).
46 changes: 46 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Security Policy

The StackQL team takes the security of the `myprovider` StackQL provider seriously. We appreciate everyone who reports vulnerabilities responsibly, and we will do our best to acknowledge and address valid reports promptly.

## Reporting a Vulnerability

Please do not open a public GitHub issue for security reports. A public issue discloses the problem before a fix is available and puts users at risk.

Instead, report privately using GitHub private vulnerability reporting:

1. Open the [new advisory form](https://github.com/stackql-registry/stackql-provider-myprovider/security/advisories/new) (Security tab -> Advisories -> "Report a vulnerability").
2. Provide as much detail as you can: the provider version (`SHOW PROVIDERS` in a `stackql shell`, or the version directory under `provider-dev/openapi/src/myprovider/`), the resource and method involved, reproduction steps (the SQL statement and the request it produced, with credentials redacted), impact, and any suggested remediation.
3. Submit. This opens a private advisory thread visible only to you and the maintainers.

If you cannot use GitHub private vulnerability reporting, email us at [info@stackql.io](mailto:info@stackql.io). Please put "SECURITY" in the subject line and treat the contents as confidential.

## Supported Versions

Security fixes are applied to the latest version of the provider published to the [StackQL provider registry](https://github.com/stackql/stackql-provider-registry); `REGISTRY PULL myprovider` installs it. Earlier published versions are supported on a best-effort basis only; we recommend pulling the latest version.

## Response Expectations

These are targets, not contractual guarantees:

- We aim to acknowledge a report within a few business days.
- We will keep you updated as we investigate and work on a fix.
- We will coordinate disclosure timing with you once a fix or mitigation is ready.

## Disclosure Policy

We follow coordinated (responsible) disclosure. Please give us a reasonable opportunity to release a fix before any public disclosure. We are happy to credit reporters in the advisory and release notes, unless you prefer to remain anonymous.

## Scope

This policy covers what this repository produces and runs: the generated provider definition under `provider-dev/openapi/`, the build scripts under `provider-dev/scripts/` and `bin/`, the test harnesses under `tests/` (including the mock API server), the documentation site under `website/`, and the CI workflows. Reports in scope include a credential or token leaking into a generated document, a log, a docs page or a test fixture; a server template or request transform that sends a credential to a host other than the My Provider API; and a test harness that reaches a live account when it should not.

Issues in the following belong elsewhere:

- The `stackql` engine (SQL parsing, authentication handling, the HTTP client): [stackql/stackql](https://github.com/stackql/stackql/security/policy).
- The request-execution library: [stackql/any-sdk](https://github.com/stackql/any-sdk).
- The provider build tooling: [stackql-registry/stackql-provider-utils](https://github.com/stackql-registry/stackql-provider-utils).
- The My Provider API itself: the vendor's own security program. This provider describes that API; it does not operate it.

## Credentials

The provider reads credentials from environment variables at query time and does not store them. The live smoke suite reads them from `.env`, which is excluded from version control; never commit a real credential. If a credential is committed or published by mistake, rotate it first, then report the exposure as above.
2 changes: 1 addition & 1 deletion bin/init-provider.sh
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ REPO_ROOT="$( cd "$( dirname "${BASH_SOURCE[0]}" )/.." && pwd )"
cd "$REPO_ROOT"

FILES=(
Makefile package.json package-lock.json README.md CLAUDE.md NOTES.md .env.example
Makefile package.json package-lock.json README.md CLAUDE.md NOTES.md SECURITY.md .env.example
bin/fetch-spec.sh
provider-dev/scripts/lib/spec_helpers.mjs
provider-dev/scripts/record_spec_pin.mjs
Expand Down
Loading
Loading