Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .changeset/27687-code-datasource-env-credential.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
'@objectstack/service-datasource': patch
'@objectstack/cli': patch
'@objectstack/spec': patch
---

feat(service-datasource): a datasource defined in code reads its credential at connect from `OS_DATASOURCE_<NAME>_PASSWORD`

Clause-②: yes (widening) — `@objectstack/service-datasource` exports `datasourceEnvCredentialVar`, `resolveDatasourceEnvCredentialsSwitch`, `DATASOURCE_ENV_CREDENTIALS_SWITCH` and the `DatasourceEnvCredentialSource` type, and `DatasourceConnectionServiceConfig` gains the optional `envCredentials`; `@objectstack/spec/data` exports `CODE_DEFINED_CREDENTIAL_ROUTE`. Nothing an author writes is added or refused.
Metadata-protocol: none

A datasource declared in code (`defineDatasource`, or a package's `*.datasource.ts`) had no way to get its password: the driver contracts refuse `config.password` and a `user:password@` URL, the Setup → Datasources form cannot edit a code datasource, and nothing mints a `sys_secret:` handle for one. Now the credential stays out of the declaration and is read at connect from a variable derived from the datasource name: `warehouse` reads `OS_DATASOURCE_WAREHOUSE_PASSWORD` (upper-cased; every character outside `A-Z0-9` becomes `_`). The value is the connection's secret: the password for postgres, mysql and mongodb, the `authToken` for turso.

- **A host switch.** `OS_DATASOURCE_ENV_CREDENTIALS` takes `on` or `off`; any other value refuses the boot with a sentence. `os serve` (and so `os start` and `os dev`) turns it on unless the operator set it. Every other composition leaves it off, so a host that loads several tenants' artifacts into one process never reads the variable.
- **Code datasources only, by provenance.** The variable is read on the boot's declared connect, for a name in the host's code-datasource set: the booted artifact's datasources and those of a package installed locally. A datasource created through Setup or the API under the same name never reads it, and neither does the `default` datasource, which keeps `OS_DATABASE_URL`. A record's own `origin` is never consulted.
- **A declared credential wins.** With `external.credentialsRef` or `external.connector` declared the variable is not read, and a reference that fails to resolve stays fail-closed.
- **Unset.** The connect goes ahead without a secret as before; when the database then refuses, the reason names the variable to set.
- **Two names deriving one variable** are refused at their connect.
- **Who can read it.** Any code datasource the host registers under a name reads that name's variable, including one in an installed package. Install only packages you trust with the credentials in the process environment.

The credential refusals (`config.password`, a URL userinfo password, a credential query parameter, the mongo `options` passthrough) and the two `password` guidance entries on `DatasourceSchema` / `ExternalDatasourceSettingsSchema` end with one shared sentence naming this route, after the Setup and `external.credentialsRef` routes they already name. `planCredentialMigration`'s remedy for a code-defined row names the row's variable.
30 changes: 17 additions & 13 deletions content/docs/data-modeling/drivers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -944,11 +944,13 @@ export default defineStack({
```

Each `url` must be credential-free when the stack is published: a
`user:password@` URL is refused, like an inline `config.password`. Bind the
password through `external.credentialsRef` instead, keeping the user in the URL
(`mongodb://svc@host/db`): the bound secret is injected beside it at connect. The
[read-only example](#read-only-grant-it-at-the-database-not-in-metadata) below
shows the binding.
`user:password@` URL is refused, like an inline `config.password`. Keep the user
in the URL (`mongodb://svc@host/db`) and supply the password at runtime: a
datasource declared in code reads it at connect from `OS_DATASOURCE_<NAME>_PASSWORD`
(`OS_DATASOURCE_ANALYTICS_PASSWORD` here), which `os serve` / `os start` honour by
default, and the value is injected beside the user. A datasource created through
Setup binds it through `external.credentialsRef` instead. See
[Credentials](/docs/data-modeling/external-datasources#4-credentials).

Then in your object definition:

Expand Down Expand Up @@ -990,19 +992,21 @@ export const Analytics = defineDatasource({
driver: 'postgres',
// The connection itself cannot write. Nothing in the app can talk past it.
config: { host: 'db.internal', database: 'analytics', username: 'analytics_ro' },
// The account's password, bound through the secret store (see below).
external: { credentialsRef: 'sys_secret:9f2c…' },
// The account's password is read at connect from OS_DATASOURCE_ANALYTICS_PASSWORD.
active: true,
});
```

The read-only account's password never goes in `config`: the driver contract
refuses `config.password`, and a `user:password@` URL, at publish. Bind it
instead. The Setup → Datasources connection form's secret field encrypts it into
`sys_secret` and stores the opaque handle at `external.credentialsRef`, or you
write a handle the secret store minted there yourself. `credentialsRef` is the
one `external` key a `managed` datasource may carry; every federation key is
still refused there. See
refuses `config.password`, and a `user:password@` URL, at publish. Because this
datasource is defined in code, the runtime reads the password at connect from
`OS_DATASOURCE_ANALYTICS_PASSWORD`, derived from its name. The CLI host
(`os serve` / `os start`) reads it by default; elsewhere the host turns it on
with `OS_DATASOURCE_ENV_CREDENTIALS=on`. The same datasource created through
Setup → Datasources binds the password instead: the connection form's secret
field encrypts it into `sys_secret` and stores the opaque handle at
`external.credentialsRef`. `credentialsRef` is the one `external` key a `managed`
datasource may carry; every federation key is still refused there. See
[Credentials](/docs/data-modeling/external-datasources#4-credentials).

Note that a read-only account also refuses **DDL**, so such a datasource cannot
Expand Down
75 changes: 74 additions & 1 deletion content/docs/data-modeling/external-datasources.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,9 @@ const policy: DatasourceConnectPolicy = {
Never inline a password. Put a reference in `external.credentialsRef` and store the
secret in the secret store (the same `SecretBinder` / `ICryptoProvider` the
runtime-admin "Add Datasource" wizard uses). The credential is resolved to
cleartext **at connect, before the driver is built**.
cleartext **at connect, before the driver is built**. A datasource defined in code
reads an environment variable instead; see
[A datasource defined in code](#a-datasource-defined-in-code-the-environment-variable).

The shipped binder encrypts the cleartext into a `sys_secret` row and mints an
**opaque** handle — `sys_secret:<id>` — as the `credentialsRef`; that is the only
Expand All @@ -274,6 +276,77 @@ for the datasources listed under [When auto-connect fails](#when-auto-connect-fa
and degrades with a warning otherwise. (Credential-less drivers such as SQLite
simply have no `credentialsRef`.)

### A datasource defined in code: the environment variable

The handle above is minted by the Setup → Datasources form, which cannot edit a
datasource defined in code (`defineDatasource`, or a `*.datasource.ts` file in a
package). A code author leaves the credential out of the declaration entirely,
and the runtime reads it **at connect** from an environment variable whose name
the platform derives from the datasource name:

| Datasource name | Variable read at connect |
| :--- | :--- |
| `warehouse` | `OS_DATASOURCE_WAREHOUSE_PASSWORD` |
| `crm_replica` | `OS_DATASOURCE_CRM_REPLICA_PASSWORD` |

The name is upper-cased and every character outside `A-Z0-9` becomes `_`. The
value is handed to the driver as the connection's secret: the password for
`postgres`, `mysql` and `mongodb`, the `authToken` for `turso`. It is never
logged, stored or answered.

```typescript
import { defineDatasource } from '@objectstack/spec/data';

export const Warehouse = defineDatasource({
name: 'warehouse',
driver: 'postgres',
// No password here: it is read from OS_DATASOURCE_WAREHOUSE_PASSWORD at connect.
config: { host: 'db.internal', database: 'warehouse', username: 'svc_app' },
});
```

```bash
OS_DATASOURCE_WAREHOUSE_PASSWORD='…' os start
```

What the slot does and does not do:

- **It is a host switch, `OS_DATASOURCE_ENV_CREDENTIALS`, which takes `on` or
`off`.** `os serve`, `os start` and `os dev` turn it on unless you set it
yourself. Every other composition (a host that builds its own kernel, a cloud
runtime) leaves it off, so the variable is never read there. Any other value
is refused at boot.
- **It only serves datasources the host registers from code.** That means the
ones in the booted artifact and in packages installed locally. A datasource
created through Setup or the API under the same name never reads the
variable. Nor does the `default` datasource, which keeps `OS_DATABASE_URL`.
- **A declared credential wins.** With `external.credentialsRef` or
`external.connector` declared, the variable is not consulted. If that
reference fails to resolve, the datasource stays unconnected (fail-closed, as
above) and does not fall back to the environment.
- **An unset variable leaves the connect as it was.** The driver connects
without a secret; if the database refuses, the datasource is unconnected and
its reason names the variable to set.
- **Two datasources that derive one variable are refused.** Valid datasource
names are lower-case, so this cannot happen between two declarations that
pass the schema; the guard covers a host that hands `connect()` other names.
- **A plugin driver that resolves its credential per request** (the OData
driver resolves `credentialsRef` that way) does not read this slot: it feeds
only the secret a driver receives when its connection is built.

<Callout type="warn">
**Who can read the variable.** The derived name is the only key, so any code
datasource the host registers under that name reads it. Under `os serve` that
includes every package installed into the environment: **a package can declare
a datasource of any free name and so read the matching
`OS_DATASOURCE_<NAME>_PASSWORD`**. Install only packages you trust with the
credentials in that process's environment. On a host where several tenants'
artifacts share one process, a process-wide variable would be readable by every
tenant artifact that declares the name. That is why the switch is off outside the
CLI host; turn it on there only when one operator owns every artifact the process
loads.
</Callout>

### Borrowing a connector instance's credential

An **API-backed** datasource, one served by a plugin driver such as the
Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mongo.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ MongoDB Connection Configuration
| **host** | `string` | optional (default: `"localhost"`) | Host address |
| **port** | `integer` | optional (default: `27017`) | Port number |
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config`. A datasource defined in code (`defineDatasource`) leaves the credential out of the declaration: it is read at connect time from the runtime environment variable derived from the datasource name (`OS_DATASOURCE_WAREHOUSE_PASSWORD` for a datasource named `warehouse`), which `os serve` / `os start` honour by default (`OS_DATASOURCE_ENV_CREDENTIALS=on`). |
| **authSource** | `string` | optional | Authentication database |
| **options** | `Record<string, any>` | optional | Extra MongoClient options (replicaSet, tls, timeouts, …). Credential-spelled keys (`password`, `authToken`, and their former aliases) are refused inline at any depth — bind the secret via the connection form / external.credentialsRef. `proxyPassword`, `tlsCertificateKeyFilePassword`, `key`, and `passphrase` are accepted and stored at rest in cleartext; they're redacted only when the datasource is read back, not refused at write. |

Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-mysql.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ MySQL / MariaDB connection configuration
| **port** | `integer` | optional (default: `3306`) | Port number |
| **database** | `string` | optional | Database name |
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config`. A datasource defined in code (`defineDatasource`) leaves the credential out of the declaration: it is read at connect time from the runtime environment variable derived from the datasource name (`OS_DATASOURCE_WAREHOUSE_PASSWORD` for a datasource named `warehouse`), which `os serve` / `os start` honour by default (`OS_DATASOURCE_ENV_CREDENTIALS=on`). |
| **ssl** | `boolean` | optional | Enable TLS. Certificates go in the datasource-level `ssl` block. |
| **autoMigrate** | `Enum<'off' \| 'safe'>` | optional | Dev-only non-destructive schema self-heal |

Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-postgres.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ PostgreSQL connection configuration
| **port** | `integer` | optional (default: `5432`) | Port number |
| **database** | `string` | optional | Database name |
| **username** | `string` | optional | Authentication user |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **password** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config`. A datasource defined in code (`defineDatasource`) leaves the credential out of the declaration: it is read at connect time from the runtime environment variable derived from the datasource name (`OS_DATASOURCE_WAREHOUSE_PASSWORD` for a datasource named `warehouse`), which `os serve` / `os start` honour by default (`OS_DATASOURCE_ENV_CREDENTIALS=on`). |
| **ssl** | `boolean` | optional | Enable TLS. Certificates go in the datasource-level `ssl` block. |
| **schema** | `string` | optional (default: `"public"`) | Default schema (knex searchPath) |
| **applicationName** | `string` | optional | Postgres application_name |
Expand Down
2 changes: 1 addition & 1 deletion content/docs/references/data/driver-turso.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Turso / libSQL Connection Configuration
| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **url** | `string` | ✅ | libSQL endpoint or local file: a remote libsql/https Turso URL, a local file written as a file: URL (never a bare path), or :memory: |
| **authToken** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config` |
| **authToken** | `never` | optional | Set through the connection form's secret field or `external.credentialsRef` — encrypted into `sys_secret`, never stored in `config`. A datasource defined in code (`defineDatasource`) leaves the credential out of the declaration: it is read at connect time from the runtime environment variable derived from the datasource name (`OS_DATASOURCE_WAREHOUSE_PASSWORD` for a datasource named `warehouse`), which `os serve` / `os start` honour by default (`OS_DATASOURCE_ENV_CREDENTIALS=on`). |
| **encryptionKey** | `never` | optional | [REMOVED] `turso config.encryptionKey` was removed in @objectstack/spec 18 and @objectstack/driver-turso 18 (ADR-0049) — it never encrypted anything, in any mode: a local or embedded-replica file is read and written through a SQLite engine that has no encryption codec, and the driver never handed the key to a remote database, whose storage encryption the server configures. Delete the key. A local file that must be encrypted at rest belongs on an encrypted disk or volume. Run `os migrate meta --from 17` to list the mechanical edits for the unset and remote-url cases; `--write` applies the ones it can prove, and you delete the key from a local or replica datasource by hand. |
| **concurrency** | `integer` | optional | Maximum concurrent requests to the remote database |
| **syncUrl** | `string` | optional | Remote sync URL for embedded-replica mode: a libsql or https Turso endpoint |
Expand Down
9 changes: 9 additions & 0 deletions packages/cli/src/commands/serve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import fs from 'fs';
import net from 'net';
import chalk, { chalkStderr } from 'chalk';
import { bundleRequire } from 'bundle-require';
import { stampDatasourceEnvCredentialsDefault } from '../utils/datasource-env-credentials-default.js';
import { loadConfig, BUNDLE_REQUIRE_EXTERNALS } from '../utils/config.js';
import { mergeBootConfig } from '../utils/merge-boot-config.js';
import { provisionTelemetryDatasource, standaloneTelemetryPrimary } from '../utils/telemetry-datasource.js';
Expand Down Expand Up @@ -5279,6 +5280,14 @@ export default class Serve extends Command {
registerDatasourceAdminRoutes,
} = adminMod;

// [#27687] This host boots one operator's artifact, so a code-defined
// datasource may read its credential from OS_DATASOURCE_{NAME}_PASSWORD
// at connect. Stamped before the admin plugin's `init()` reads the
// switch; an explicit operator value (`off` included) wins.
if (typeof adminMod.DATASOURCE_ENV_CREDENTIALS_SWITCH === 'string') {
stampDatasourceEnvCredentialsDefault(process.env, adminMod.DATASOURCE_ENV_CREDENTIALS_SWITCH);
}

if (
DatasourceAdminServicePlugin &&
!hasPluginMatching(['com.objectstack.service-datasource-admin', 'DatasourceAdminServicePlugin'])
Expand Down
53 changes: 53 additions & 0 deletions packages/cli/src/utils/datasource-env-credentials-default.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #27687 — the CLI host turns the code-datasource env-credential switch on by
* default; every other composition leaves it off. Two halves: the stamp keeps
* an operator's explicit value, and `os serve` (which `os start` and `os dev`
* run) stamps it before the datasource admin plugin is built, whose `init()`
* reads the switch.
*/

import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { describe, expect, it } from 'vitest';
import { stampDatasourceEnvCredentialsDefault } from './datasource-env-credentials-default.js';

const SWITCH = 'OS_DATASOURCE_ENV_CREDENTIALS';

describe('stampDatasourceEnvCredentialsDefault', () => {
it('stamps `on` when the switch is unset or empty', () => {
for (const env of [{}, { [SWITCH]: '' }, { [SWITCH]: ' ' }] as Record<string, string | undefined>[]) {
expect(stampDatasourceEnvCredentialsDefault(env, SWITCH)).toBe(true);
expect(env[SWITCH]).toBe('on');
}
});

it("keeps an operator's explicit value — `off`, `on`, and an invalid one the boot then refuses", () => {
for (const value of ['off', 'OFF', 'on', 'true']) {
const env: Record<string, string | undefined> = { [SWITCH]: value };
expect(stampDatasourceEnvCredentialsDefault(env, SWITCH)).toBe(false);
expect(env[SWITCH]).toBe(value);
}
});
});

describe('`os serve` stamps the default before the datasource admin plugin is built', () => {
const source = readFileSync(join(dirname(fileURLToPath(import.meta.url)), '..', 'commands', 'serve.ts'), 'utf8');

it('stamps once, into the process environment, under the name the admin module exports', () => {
const calls = source.match(/stampDatasourceEnvCredentialsDefault\(/g) ?? [];
expect(calls).toHaveLength(1);
expect(source).toContain(
'stampDatasourceEnvCredentialsDefault(process.env, adminMod.DATASOURCE_ENV_CREDENTIALS_SWITCH);',
);
});

it('the stamp precedes the plugin construction', () => {
const stamp = source.indexOf('stampDatasourceEnvCredentialsDefault(process.env');
const construct = source.indexOf('new DatasourceAdminServicePlugin(');
expect(stamp).toBeGreaterThan(0);
expect(construct).toBeGreaterThan(stamp);
});
});
Loading
Loading