From 5e9da21ff37f5b3a0b8ca96a3f763c9572923985 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 15:34:12 +0100 Subject: [PATCH 01/54] Document the missing OIDC configuration options Compared the `acl.oidc.*` keys in EntPropertyKey against the OIDC configuration page. Five options were missing and one was documented under a name the server does not recognise. Added: - `acl.oidc.state.required` - `acl.oidc.device.authorization.endpoint` - `acl.oidc.public.keys.expiry` - `acl.oidc.response.buffer.size` - `acl.oidc.string.pool.capacity` Corrected: - `acl.oidc.pkce.enabled` does not exist, the property is `acl.oidc.pkce.required` - `acl.oidc.groups.claim` has no default, it is mandatory when OIDC is enabled - setting both `acl.oidc.host` and `acl.oidc.configuration.url` fails server startup, they are mutually exclusive Also moved `acl.oidc.cache.ttl` out of the claims section into a new "Caching and buffers" section, next to the other caching and buffer settings. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/configuration/oidc.md | 78 +++++++++++++++++++++++------ 2 files changed, 65 insertions(+), 14 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index ca4f230c94..c044d9cfd3 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -48,6 +48,7 @@ This page tracks significant updates to the QuestDB documentation. - Added the [`cairo.sql.parquet.cache.memory.size`](/docs/configuration/cairo-engine/) configuration property (256 MB default), deprecating the slot-based `cairo.sql.parquet.frame.cache.capacity` - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` +- Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.device.authorization.endpoint`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default ### Updated diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index e0c5932bdc..481322a568 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -55,8 +55,9 @@ configuration options must also be set. - **Default**: none - **Reloadable**: no -OIDC provider hostname. Required when OIDC is enabled, unless the OIDC -configuration URL is set. +OIDC provider hostname. Required when OIDC is enabled, unless +`acl.oidc.configuration.url` is set. The two are mutually exclusive, setting +both of them fails server startup. ### acl.oidc.http.timeout @@ -100,7 +101,7 @@ When enabled, the PGWire endpoint supports OIDC authentication. The OAuth2 token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option. -### acl.oidc.pkce.enabled +### acl.oidc.pkce.required - **Default**: `true` - **Reloadable**: no @@ -116,6 +117,16 @@ be enabled in production. The Web Console is not fully secure without it. Enables or disables the Resource Owner Password Credentials flow. When enabled, this flow must also be configured in the OIDC Provider. +### acl.oidc.state.required + +- **Default**: `false` +- **Reloadable**: no + +Requires the `state` parameter in the Authorization Code Flow. The client +generates a random state value, which the OIDC Provider returns unchanged +together with the authorization code. Checking it protects against CSRF +attacks. Enable it if the Identity Provider supports the `state` parameter. + ## Endpoints ### acl.oidc.authorization.endpoint @@ -126,6 +137,17 @@ enabled, this flow must also be configured in the OIDC Provider. OIDC Authorization Endpoint. The default value should work for the Ping Identity Platform. +### acl.oidc.device.authorization.endpoint + +- **Default**: none +- **Reloadable**: no + +OIDC Device Authorization Endpoint, used by clients which authenticate with +the Device Code Flow. Unlike the other endpoints it has no default value. It +is resolved automatically if `acl.oidc.configuration.url` is set and the +OIDC Provider advertises a `device_authorization_endpoint`. The Device Code +Flow is unavailable if the endpoint is neither configured nor discovered. + ### acl.oidc.public.keys.endpoint - **Default**: `/pf/JWKS` @@ -196,22 +218,13 @@ which it connects. ## User and group claims -### acl.oidc.cache.ttl - -- **Default**: `30000` -- **Reloadable**: no - -User info cache entry TTL in milliseconds. QuestDB caches user info responses -for each valid access token. This setting controls how often the access token -is validated and user info refreshed. - ### acl.oidc.groups.claim -- **Default**: `groups` +- **Default**: none - **Reloadable**: no The name of the custom claim in the user information that contains the -group memberships of the user. +group memberships of the user. Required when OIDC is enabled. ### acl.oidc.groups.encoded.in.token @@ -230,3 +243,40 @@ group memberships directly into the token. The name of the claim in the user information that contains the user's name. Could be a username, full name, or email. Displayed in the Web Console and logged for audit purposes. + +## Caching and buffers + +### acl.oidc.cache.ttl + +- **Default**: `30000` +- **Reloadable**: no + +User info cache entry TTL in milliseconds. QuestDB caches user info responses +for each valid access token. This setting controls how often the access token +is validated and user info refreshed. + +### acl.oidc.public.keys.expiry + +- **Default**: `120000` +- **Reloadable**: no + +Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. QuestDB caches +the public keys used to validate tokens issued by the OIDC Provider, and +reloads them from the public keys endpoint when the cache expires. + +### acl.oidc.response.buffer.size + +- **Default**: `1M` +- **Reloadable**: no + +Size of the buffer used to receive HTTP responses from the OIDC Provider. +Increase it if the provider sends large responses, such as user info +containing a long list of group memberships. + +### acl.oidc.string.pool.capacity + +- **Default**: `128` +- **Reloadable**: no + +Initial capacity of the string pool used when parsing JSON responses received +from the OIDC Provider. From dc7276c753619b6bee7555acabd17bec9334bff8 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 17:10:17 +0100 Subject: [PATCH 02/54] Note the state parameter requirement in the OIDC guide Some Identity Providers require the `state` parameter in the authorization request. Documented this in the Authentication and Authorization Flow walkthrough, next to the authorization code request, pointing at `acl.oidc.state.required`. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/security/oidc.mdx | 15 +++++++++++++++ 2 files changed, 16 insertions(+) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index c044d9cfd3..e5167b7e55 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -49,6 +49,7 @@ This page tracks significant updates to the QuestDB documentation. - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` - Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.device.authorization.endpoint`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default +- [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers ### Updated diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index ecd71b2953..0beb72944b 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -124,6 +124,21 @@ been authenticated already: https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.host%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 ``` +:::note + +Some Identity Providers require the `state` parameter in the authorization +request. It is another random value generated by the client, which the OIDC +Provider returns unchanged together with the authorization code. Checking it +protects against CSRF attacks. + +If your provider requires it, set +[`acl.oidc.state.required`](/docs/configuration/oidc/#acloidcstaterequired) to +`true`. The [Web Console](/docs/getting-started/web-console/overview/) then +generates the `state` parameter, sends it in the authorization request, and +validates the value returned by the provider. + +::: + ### 2. Prove identity Next, the user must prove its identity. From 8724eceff978cb869c2d5e46e6a101aed46bc982 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:45:18 +0100 Subject: [PATCH 03/54] Document the OIDC constraints which fail server startup The audit of `acl.oidc.*` covered defaults but not the validation which runs alongside them in EntPropServerConfiguration. Four constraints abort startup and none of them were documented: - OIDC and the Basic Auth Realm cannot both be enabled - the keystore path and password must be set together, or neither - `acl.oidc.tls.enabled` must match the scheme of every provider URL, including the endpoints discovered from the configuration document - `acl.oidc.enabled` needs `acl.oidc.client.id`, `acl.oidc.groups.claim` and one of `acl.oidc.host` / `acl.oidc.configuration.url` The last one replaces "several other configuration options must also be set", which left the reader to reconstruct the mandatory set by reading all 29 entries. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/iam.md | 4 ++++ documentation/configuration/oidc.md | 17 +++++++++++++---- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/documentation/configuration/iam.md b/documentation/configuration/iam.md index 7829c04982..e3f3b22354 100644 --- a/documentation/configuration/iam.md +++ b/documentation/configuration/iam.md @@ -47,6 +47,10 @@ Enables or disables the built-in admin user. When enabled, the browser's basic auth popup window is used instead of the Web Console's login screen. Present for backwards compatibility only. +Cannot be enabled together with +[`acl.oidc.enabled`](/docs/configuration/oidc/#acloidcenabled). Setting both +to `true` fails server startup. + ### acl.enabled - **Default**: `true` diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 481322a568..de73855353 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -47,8 +47,13 @@ JSON format. Should always end with `/.well-known/openid-configuration`. - **Default**: `false` - **Reloadable**: no -Enables or disables OIDC authentication. When enabled, several other -configuration options must also be set. +Enables or disables OIDC authentication. When enabled, `acl.oidc.client.id` +and `acl.oidc.groups.claim` must also be set, along with either +`acl.oidc.host` or `acl.oidc.configuration.url`. + +OIDC cannot be enabled together with +[`acl.basic.auth.realm.enabled`](/docs/configuration/iam/#aclbasicauthrealmenabled). +Setting both to `true` fails server startup. ### acl.oidc.host @@ -189,13 +194,17 @@ Whether the OIDC provider requires a secure connection. If the OpenID Provider endpoints do not require TLS, this can be set to `false`. This is unlikely in production. +This setting must match the scheme of every OIDC Provider URL QuestDB uses, +including `acl.oidc.configuration.url` and each endpoint discovered from it. +A URL whose scheme does not match fails server startup. + ### acl.oidc.tls.keystore.password - **Default**: none - **Reloadable**: no -Keystore password. Required if a keystore file is configured and is password -protected. +Keystore password. Must be set whenever `acl.oidc.tls.keystore.path` is set. +Setting either one without the other fails server startup. ### acl.oidc.tls.keystore.path From 5d31797869133eaa162cb85867f4608bf480ca03 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:45:33 +0100 Subject: [PATCH 04/54] Note that endpoint discovery supersedes the endpoint settings Setting `acl.oidc.configuration.url` makes the whole Endpoints section inert. PropOidcConfiguration only builds endpoints from the individual properties when no configuration URL is set, and init() then overwrites all five from the discovered document. `acl.oidc.port` goes the same way, as host and port come from the discovered URLs. The page implied the opposite, and only in the device authorization entry: "resolved automatically if acl.oidc.configuration.url is set" reads as though setting the property is the alternative to discovery. It is not, and the reader who tries it gets no log line saying so. State the rule under `acl.oidc.configuration.url`, where a reader meets it first, and again at the top of the Endpoints section. Also fixes the comma splice in the `acl.oidc.host` entry. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index de73855353..6ce8f5da6a 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -42,6 +42,14 @@ enabled. URL where the OpenID Provider's configuration information can be loaded in JSON format. Should always end with `/.well-known/openid-configuration`. +QuestDB downloads the document at startup and takes every endpoint from it, +so the settings under [Endpoints](#endpoints) and `acl.oidc.port` are not +used. The server does not start if the document cannot be downloaded or +parsed, or if it is missing the authorization, token, user info or JWKS +endpoint. + +Mutually exclusive with `acl.oidc.host`: setting both fails server startup. + ### acl.oidc.enabled - **Default**: `false` @@ -61,8 +69,8 @@ Setting both to `true` fails server startup. - **Reloadable**: no OIDC provider hostname. Required when OIDC is enabled, unless -`acl.oidc.configuration.url` is set. The two are mutually exclusive, setting -both of them fails server startup. +`acl.oidc.configuration.url` is set. The two are mutually exclusive: setting +both fails server startup. ### acl.oidc.http.timeout @@ -76,7 +84,8 @@ OIDC provider HTTP request timeout in milliseconds. - **Default**: `443` - **Reloadable**: no -OIDC provider port number. +OIDC provider port number. Not used when `acl.oidc.configuration.url` is +set, because the port is taken from the discovered endpoint URLs. ### acl.oidc.redirect.uri @@ -134,6 +143,10 @@ attacks. Enable it if the Identity Provider supports the `state` parameter. ## Endpoints +These settings apply only when the OIDC Provider is configured by host. When +`acl.oidc.configuration.url` is set, QuestDB takes every endpoint from the +provider's configuration document and the settings below are not used. + ### acl.oidc.authorization.endpoint - **Default**: `/as/authorization.oauth2` From 438f84eebe993a997231239a8d2878921846777e Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:46:21 +0100 Subject: [PATCH 05/54] Document the Device Code Flow in the OIDC guide `acl.oidc.device.authorization.endpoint` named a flow that appeared nowhere else in the docs. A reader who needed it found the knob and nothing else: no definition, no link, no explanation of who consumes the endpoint. The gap sat exactly where the flow belongs. The guide points CLI and standalone clients at the Resource Owner Password Credentials flow, which the same page flags as "legacy, and should be used as a last resort", and the client pages tell developers to acquire a token out-of-band without saying how. Describe the flow, state that QuestDB advertises the endpoint rather than running the flow, and cover both ways the endpoint is resolved. Add a tip in the CLI section so the reader is not steered to ROPC by default. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/security/oidc.mdx | 34 +++++++++++++++++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index e5167b7e55..40e51f97be 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -50,6 +50,7 @@ This page tracks significant updates to the QuestDB documentation. - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` - Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.device.authorization.endpoint`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [OpenID Connect (OIDC)](/docs/security/oidc/#device-code-flow) - Documented the Device Code Flow for clients without a browser of their own, how QuestDB advertises the Device Authorization Endpoint to them, and why it is preferable to the Resource Owner Password Credentials flow ### Updated diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 0beb72944b..a47eb034e9 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -533,6 +533,40 @@ testldap=> testldap=> ``` +:::tip + +The Resource Owner Password Credentials flow requires the client to handle the +user's password. A client which is able to display a URL and poll for the +result should use the [Device Code Flow](#device-code-flow) instead. + +::: + +### Device Code Flow + +The [Device Code Flow](https://datatracker.ietf.org/doc/html/rfc8628) is the +standard way for a client with no browser of its own to obtain tokens without +ever handling the user's password. The client asks the OIDC Provider for a +device code, displays a short user code together with a verification URL, and +polls the Token endpoint while the user completes the login in a browser on any +device. + +QuestDB does not run the flow itself. It publishes the OIDC Provider's Device +Authorization Endpoint to clients through its settings endpoint, so a client +knows where to start: + +- When `acl.oidc.configuration.url` is set, QuestDB takes the endpoint from the + provider's configuration document, provided the provider advertises a + `device_authorization_endpoint`. +- When the provider is configured by host, set + [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) + explicitly. It is the only endpoint with no default value. + +If QuestDB does not advertise the endpoint, the client has to be given it +directly, or resolve it from the OIDC Provider itself. + +Once the client holds an access token, it presents it to QuestDB the same way +as any other token. + ## Non-interactive clients Non-interactive clients are usually jobs or standalone applications, such as a From a890f52d3b02fa84f8476f3bf8a5e609e9544cb0 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:46:35 +0100 Subject: [PATCH 06/54] Correct what the device authorization endpoint setting controls Two claims in the entry did not hold. "The Device Code Flow is unavailable if the endpoint is neither configured nor discovered" describes a gate the server does not have. QuestDB has no Device Code Flow implementation; the endpoint's only use is exportConfiguration, which publishes it to clients. A client that knows its provider's device endpoint runs the flow either way, and QuestDB validates the resulting token as usual. "It is resolved automatically if acl.oidc.configuration.url is set" presented discovery as specific to this endpoint and implied the property is the fallback when a provider does not advertise one. Discovery covers all five endpoints, and in that mode this property is not read at all. The general rule now lives under `acl.oidc.configuration.url`, so the entry no longer needs to half-state it. Also links the flow to its new section in the guide, and adds the comma after the fronted adverbial. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 6ce8f5da6a..aab1c7091b 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -161,10 +161,13 @@ Identity Platform. - **Reloadable**: no OIDC Device Authorization Endpoint, used by clients which authenticate with -the Device Code Flow. Unlike the other endpoints it has no default value. It -is resolved automatically if `acl.oidc.configuration.url` is set and the -OIDC Provider advertises a `device_authorization_endpoint`. The Device Code -Flow is unavailable if the endpoint is neither configured nor discovered. +the [Device Code Flow](/docs/security/oidc/#device-code-flow). Unlike the +other endpoints, it has no default value. + +QuestDB does not run the Device Code Flow itself. It publishes this endpoint +to clients through the settings endpoint, so that a client knows where to +start the flow. A client which does not receive it has to be given the +endpoint directly, or resolve it from the OIDC Provider itself. ### acl.oidc.public.keys.endpoint From 6da5c5b9c92028c9556264b62f21fbf6773a471f Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:46:48 +0100 Subject: [PATCH 07/54] Distinguish the millisecond settings which accept a duration `acl.oidc.public.keys.expiry` is read with getMillis, so `2m` and `120s` work. `acl.oidc.cache.ttl` and `acl.oidc.http.timeout` are read with getInt, which rejects anything but digits and turns it into a ServerConfigurationException. All three said only "in milliseconds". Grouping cache.ttl and public.keys.expiry under the new "Caching and buffers" heading put the two adjacent, which makes the identical wording worse: `acl.oidc.public.keys.expiry=30s` starts, `acl.oidc.cache.ttl=30s` does not. While in the cache.ttl entry, document that `0` disables the user info cache. isOidcCacheEnabled() tests `cacheTtl > 0` and swaps in a no-op cache. That is the knob for making a revoked account take effect immediately, which the guide discusses without naming a value. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index aab1c7091b..debd7eec0d 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -77,7 +77,8 @@ both fails server startup. - **Default**: `30000` - **Reloadable**: no -OIDC provider HTTP request timeout in milliseconds. +OIDC provider HTTP request timeout in milliseconds. Accepts a plain integer +only. ### acl.oidc.port @@ -276,18 +277,25 @@ logged for audit purposes. - **Default**: `30000` - **Reloadable**: no -User info cache entry TTL in milliseconds. QuestDB caches user info responses -for each valid access token. This setting controls how often the access token -is validated and user info refreshed. +User info cache entry TTL in milliseconds, as a plain integer only. QuestDB +caches user info responses for each valid access token. This setting controls +how often the access token is validated and user info refreshed. + +Set it to `0` to disable the cache, so that every request is validated against +the OIDC Provider. ### acl.oidc.public.keys.expiry - **Default**: `120000` - **Reloadable**: no -Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. QuestDB caches -the public keys used to validate tokens issued by the OIDC Provider, and -reloads them from the public keys endpoint when the cache expires. +Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. Also accepts a +duration, such as `2m` or `120s`. + +QuestDB caches the public keys used to validate tokens issued by the OIDC +Provider, and reloads them from the public keys endpoint when the cache +expires. Lower it if the OIDC Provider rotates its signing keys frequently, at +the cost of more requests to the endpoint. ### acl.oidc.response.buffer.size From 97559b4e66db15b89d4cd70d844e9a7f587a56cf Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:47:01 +0100 Subject: [PATCH 08/54] Give one rule for enabling the state parameter The two pages disagreed on when to set `acl.oidc.state.required`. The reference page said to enable it if the provider "supports" the `state` parameter; the guide said to enable it if the provider "requires" it. Effectively every provider supports `state`, so the reference page's rule resolved to "always enable it", which its own default of `false` contradicts. Both rules are half the answer: enable it when the provider requires it, and enabling it is a reasonable defence in depth otherwise. Say that once, and drop the mechanism description the guide already carries. The entry also read as though the server enforces something. It does not - isStateRequired() has one consumer, exportConfiguration - so name the Web Console as the component which generates and checks the value, and link to the step in the guide that walks through it. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index debd7eec0d..c8ff2bb925 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -137,10 +137,16 @@ enabled, this flow must also be configured in the OIDC Provider. - **Default**: `false` - **Reloadable**: no -Requires the `state` parameter in the Authorization Code Flow. The client -generates a random state value, which the OIDC Provider returns unchanged -together with the authorization code. Checking it protects against CSRF -attacks. Enable it if the Identity Provider supports the `state` parameter. +Requires the `state` parameter in the Authorization Code Flow, as a defence +against CSRF attacks. QuestDB does not see the value itself; it publishes the +setting to clients through the settings endpoint, the same way it publishes +`acl.oidc.pkce.required`. + +Enable it if the OIDC Provider requires the `state` parameter, or to add CSRF +protection on top of PKCE. The +[Web Console](/docs/security/oidc/#1-secret-generation) generates the value, +sends it in the authorization request, and checks that the provider returns it +unchanged. ## Endpoints From 88ee167637248d1deffce63831a53a63020211c6 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:47:12 +0100 Subject: [PATCH 09/54] Correct what happens when the groups claim is missing The guide said the user "is authenticated, they have no permissions at all". There is no such state. OidcClientImpl.verifyTokenSlow returns false when the sub claim is empty or the groups list is empty, which propagates to a bare 401. The empty sub claim was not mentioned at all. The distinction matters when reading logs: a 401 sends an operator looking at tokens, clocks and JWKS, while the actual cause is a group claim that stopped populating on the provider side. The passage also said only that the claim name "is configurable in QuestDB" without naming the property. Now that `acl.oidc.groups.claim` is documented as mandatory with no default, name it and link it, so the guide and the reference page agree. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index a47eb034e9..7c5a58248a 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -707,12 +707,14 @@ QuestDB works the list of external groups out from the User Info response message. If we take the example used earlier, we will see that the message contains a -claim called `groups`. This name is configurable in QuestDB. - -If the groups claim is missing or it is an empty list, the user cannot access -the database. - -Although the user is authenticated, they have no permissions at all. +claim called `groups`. Its name is set with +[`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which +has no default and must be set whenever OIDC is enabled. + +If the groups claim is missing or it is an empty list, authentication fails and +QuestDB replies with `401 Unauthorized`. The same happens when the claim named +by [`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing +or empty. The user has to have at least the `HTTP` permission to be able to successfully login via the [Web Console](/docs/getting-started/web-console/overview/). From 93dc7f7360f8172bff575ad843ecda21c5177694 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:47:55 +0100 Subject: [PATCH 10/54] Add the missing QuestDB configuration to the PingFederate guide The PingFederate walkthrough contains no `acl.oidc.*` settings at all. It ends with "To test, head to http://localhost:9000 and login. If all has been wired up well, then login will succeed", having never told the reader to enable OIDC or point QuestDB at the provider. It worked as a narrative because the reference page's endpoint defaults are Ping-shaped and the page also claimed `acl.oidc.groups.claim` defaulted to `groups` - the exact claim name this walkthrough builds. The property has had no default since the OIDC client was introduced, so the implied recipe never started a server. Correcting the default made the omission explicit: the reference page now says the property is mandatory and the walkthrough still does not mention it. Give the section the same `server.conf` block the Entra ID section has, and note that the endpoint defaults cover PingFederate. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/security/oidc.mdx | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 40e51f97be..f9bee8cdfb 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -50,6 +50,7 @@ This page tracks significant updates to the QuestDB documentation. - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` - Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.device.authorization.endpoint`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [PingFederate SSO](/docs/security/oidc/#pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - [OpenID Connect (OIDC)](/docs/security/oidc/#device-code-flow) - Documented the Device Code Flow for clients without a browser of their own, how QuestDB advertises the Device Authorization Endpoint to them, and why it is preferable to the Resource Owner Password Credentials flow ### Updated diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 7c5a58248a..3f121bce06 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1101,6 +1101,37 @@ to the existing PCV. Then select the `username` attribute of the PCV as `USER_KEY`. +#### QuestDB configuration + +The below should be set in QuestDB's `server.conf`: + +```shell +# enable OIDC +acl.oidc.enabled=true + +# hostname of the PingFederate server +acl.oidc.host=pingfederate.host + +# the client id picked when setting up the client above +acl.oidc.client.id=questdb + +# the claim which contains the user's group memberships +acl.oidc.groups.claim=groups + +# enable ROPC flow +# optional, required only if ROPC is enabled in PingFederate +acl.oidc.ropc.flow.enabled=true +``` + +The endpoint defaults listed under +[Endpoints](/docs/configuration/oidc/#endpoints) already match the PingFederate +paths, so they do not have to be set. + +Alternatively, set `acl.oidc.configuration.url` instead of `acl.oidc.host` and +let QuestDB discover the endpoints from PingFederate. The two are mutually +exclusive, and the endpoint settings are not used when the configuration URL is +set. + #### Confirm QuestDB mappings and login QuestDB requires a mapping, as laid out in the From eec6f6d2a24c1f4dead2e652ce6e107cc3814b15 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 18:50:44 +0100 Subject: [PATCH 11/54] Use the repo's spelling convention in the state parameter entry "as a defence against" was the only instance of British "defence" in the docs, which otherwise favor American spelling. Reword rather than respell, which also tightens the sentence. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index c8ff2bb925..e2b73d505a 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -137,7 +137,7 @@ enabled, this flow must also be configured in the OIDC Provider. - **Default**: `false` - **Reloadable**: no -Requires the `state` parameter in the Authorization Code Flow, as a defence +Requires the `state` parameter in the Authorization Code Flow, which protects against CSRF attacks. QuestDB does not see the value itself; it publishes the setting to clients through the settings endpoint, the same way it publishes `acl.oidc.pkce.required`. From 6dab13728d6e538525d72cf0f5e707484dab0197 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 19:50:17 +0100 Subject: [PATCH 12/54] Name both QuestDB configuration sections after their provider Two headings called "QuestDB configuration" left the slug of the older Entra ID one to the newly added PingFederate section, silently sending every existing link to /docs/security/oidc/#questdb-configuration to the wrong provider. Pin the Entra ID slug so it keeps the anchor it has always had. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 3f121bce06..526eded08e 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1101,7 +1101,7 @@ to the existing PCV. Then select the `username` attribute of the PCV as `USER_KEY`. -#### QuestDB configuration +#### QuestDB configuration for PingFederate The below should be set in QuestDB's `server.conf`: @@ -1326,7 +1326,7 @@ With this we have finished setting up the QuestDB client application in Entra ID, and now we can wire QuestDB and Entra ID together by adding OIDC configuration to QuestDB. -#### QuestDB configuration +#### QuestDB configuration for Entra ID {#questdb-configuration} The below should be set in QuestDB's `server.conf`: From e76c0f9f30c11076ab27326ae8781b25f9507c3f Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 19:50:17 +0100 Subject: [PATCH 13/54] Document the settings endpoint clients read the OIDC configuration from The settings endpoint was named three times without being described anywhere, so a client author had no path, no response shape and no field names to work with. The only entry which mentioned it, http.context.settings, presented it as a Web Console detail. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/configuration/http-server.md | 6 ++- documentation/configuration/oidc.md | 22 +++++++--- documentation/security/oidc.mdx | 50 ++++++++++++++++++++++ 4 files changed, 70 insertions(+), 9 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index f9bee8cdfb..5ca80bffb1 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -31,6 +31,7 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another +- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token and device authorization endpoints, and whether PKCE and the `state` parameter are required ### Reference diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index 65b14ef09f..17bee5659b 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -493,8 +493,10 @@ Context path for the file import service. - **Default**: `/settings` - **Reloadable**: no -Context path for the service which provides server-side settings to the Web -Console. +Context path for the service which provides server-side settings to clients. The +[Web Console](/docs/getting-started/web-console/overview/) reads it, and so does +any OIDC client which discovers the provider's endpoints from the +[settings endpoint](/docs/security/oidc/#settings-endpoint). ### http.context.table.status diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index e2b73d505a..1bb5bf3b2c 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -138,9 +138,11 @@ enabled, this flow must also be configured in the OIDC Provider. - **Reloadable**: no Requires the `state` parameter in the Authorization Code Flow, which protects -against CSRF attacks. QuestDB does not see the value itself; it publishes the -setting to clients through the settings endpoint, the same way it publishes -`acl.oidc.pkce.required`. +against CSRF attacks. QuestDB does not see the value itself. It publishes the +setting to clients through the +[settings endpoint](/docs/security/oidc/#settings-endpoint), the same way it +publishes `acl.oidc.pkce.required`, and the client is what generates and checks +the value. Enable it if the OIDC Provider requires the `state` parameter, or to add CSRF protection on top of PKCE. The @@ -171,10 +173,16 @@ OIDC Device Authorization Endpoint, used by clients which authenticate with the [Device Code Flow](/docs/security/oidc/#device-code-flow). Unlike the other endpoints, it has no default value. -QuestDB does not run the Device Code Flow itself. It publishes this endpoint -to clients through the settings endpoint, so that a client knows where to -start the flow. A client which does not receive it has to be given the -endpoint directly, or resolve it from the OIDC Provider itself. +QuestDB does not run the Device Code Flow itself. It publishes this endpoint to +clients through the +[settings endpoint](/docs/security/oidc/#settings-endpoint), so that a client +knows where to start the flow. A client which does not receive it has to be +given the endpoint directly, or resolve it from the OIDC Provider itself. + +When `acl.oidc.configuration.url` is set, QuestDB takes this endpoint from the +provider's configuration document instead, if the provider advertises a +`device_authorization_endpoint`. It is the only endpoint discovery is allowed to +omit. ### acl.oidc.public.keys.endpoint diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 526eded08e..d0580ed376 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -318,6 +318,56 @@ and then sends the results back: } ``` +## Settings endpoint + +QuestDB publishes the parts of its OIDC configuration that a client needs in +order to start an authentication flow. Any client can read them from the +settings endpoint, which requires no authentication: + +```bash title="Settings request example" +curl https://questdb.host:9000/settings +``` + +The OIDC-related entries of the response look like this, alongside other server +settings: + +```json title="Settings response example" +{ + "config": { + "acl.oidc.enabled": true, + "acl.oidc.pkce.required": true, + "acl.oidc.state.required": false, + "acl.oidc.groups.encoded.in.token": false, + "acl.oidc.client.id": "questdb", + "acl.oidc.redirect.uri": "https://questdb.host:9000", + "acl.oidc.scope": "openid", + "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", + "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", + "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" + }, + "preferences.version": 0, + "preferences": {} +} +``` + +Each key is the name of the +[configuration option](/docs/configuration/oidc/) it carries. The +[Web Console](/docs/getting-started/web-console/overview/) reads them to build +its authorization request, and any other client can do the same instead of +hard coding the provider's details. + +The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the +[endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's +configuration document when `acl.oidc.configuration.url` is set. +`acl.oidc.device.authorization.endpoint` is the only one which may be absent, +because it is the only one QuestDB can run without, and the only one with no +default: the value above is whatever the operator configured or discovery +returned. + +The path of the endpoint is set by +[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings), +and defaults to `/settings`. + ## Interactive clients Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an From ad08e2ee4dbd1698b339c03d0b946637738c4dcc Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 19:50:17 +0100 Subject: [PATCH 14/54] Show how to run the Device Code Flow The section described the flow but gave nothing to run, and never said where the client gets the Token endpoint it polls. Add the three requests, state that there is no setting to switch the flow on, and note that the ID token is the one to take when the groups are encoded in it. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 80 ++++++++++++++++++++++++++++++--- 1 file changed, 74 insertions(+), 6 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index d0580ed376..e64573b039 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -600,22 +600,90 @@ device code, displays a short user code together with a verification URL, and polls the Token endpoint while the user completes the login in a browser on any device. -QuestDB does not run the flow itself. It publishes the OIDC Provider's Device -Authorization Endpoint to clients through its settings endpoint, so a client -knows where to start: +QuestDB does not run the flow itself, and there is no setting to switch it on. +The client talks to the OIDC Provider directly. QuestDB only advertises the +provider's Device Authorization Endpoint through its +[settings endpoint](#settings-endpoint), so that a client knows where to start: - When `acl.oidc.configuration.url` is set, QuestDB takes the endpoint from the provider's configuration document, provided the provider advertises a `device_authorization_endpoint`. - When the provider is configured by host, set [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) - explicitly. It is the only endpoint with no default value. + explicitly. It is the only endpoint with no default value, so take the path + from the OIDC Provider's documentation. If QuestDB does not advertise the endpoint, the client has to be given it directly, or resolve it from the OIDC Provider itself. -Once the client holds an access token, it presents it to QuestDB the same way -as any other token. +The OIDC Provider has to support the flow too, with the device code grant +enabled for the client. + +#### Run the flow + +First read the endpoints from QuestDB. The +[settings endpoint](#settings-endpoint) carries both the Device Authorization +Endpoint the flow starts at, and the Token endpoint it polls: + +```python title="Discover the endpoints" +import requests +import time + +settings = requests.get("https://questdb.host:9000/settings").json()["config"] +device_endpoint = settings["acl.oidc.device.authorization.endpoint"] +token_endpoint = settings["acl.oidc.token.endpoint"] +client_id = settings["acl.oidc.client.id"] +scope = settings["acl.oidc.scope"] +``` + +Then ask the OIDC Provider for a device code, and tell the user where to log in. +The fields of the response are the ones defined by +[RFC 8628](https://datatracker.ietf.org/doc/html/rfc8628#section-3.2): + +```python title="Request a device code" +device = requests.post(device_endpoint, + data={"client_id": client_id, + "scope": scope}, + headers={"Content-Type": "application/x-www-form-urlencoded"}).json() + +print(f"Open {device['verification_uri']} and enter the code {device['user_code']}") +``` + +Finally, poll the Token endpoint until the user completes the login. The error +`authorization_pending` means the user has not finished yet, `slow_down` asks +for a longer interval, and any other error ends the flow: + +```python title="Poll for the access token" +interval = device.get("interval", 5) +while True: + time.sleep(interval) + response = requests.post(token_endpoint, + data={"grant_type": "urn:ietf:params:oauth:grant-type:device_code", + "device_code": device["device_code"], + "client_id": client_id}, + headers={"Content-Type": "application/x-www-form-urlencoded"}).json() + if "access_token" in response: + access_token = response["access_token"] + break + error = response.get("error") + if error == "slow_down": + interval += 5 + elif error != "authorization_pending": + raise RuntimeError(response) +``` + +The client now holds a token, and presents it to QuestDB the same way as any +other token, as shown in the [7th step](#7-database-access) of the Authorization +Code Flow. + +:::note + +Take the ID token from the response instead of the access token when +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is set to `true`. QuestDB then reads the group memberships from the token +itself, and only the ID token carries them. + +::: ## Non-interactive clients From 8e96bb8fb3b208761d0d4a34abc90aa99e98896e Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 19:50:17 +0100 Subject: [PATCH 15/54] Correct what the public keys expiry controls Key rotation does not depend on the expiry, because a token signed with an uncached key triggers an immediate reload. The setting governs revocation. Both JWKS settings are also inert unless the groups are encoded in the token. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 1bb5bf3b2c..75f5a1dc12 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -193,6 +193,11 @@ JSON Web Key Set (JWKS) Endpoint. Provides the list of public keys used to decode and validate ID tokens issued by the OIDC Provider. The default value should work for the Ping Identity Platform. +The keys are only read when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`. +With the default user info flow QuestDB validates tokens by calling the user +info endpoint instead. + ### acl.oidc.token.endpoint - **Default**: `/as/token.oauth2` @@ -307,9 +312,16 @@ Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. Also accepts a duration, such as `2m` or `120s`. QuestDB caches the public keys used to validate tokens issued by the OIDC -Provider, and reloads them from the public keys endpoint when the cache -expires. Lower it if the OIDC Provider rotates its signing keys frequently, at -the cost of more requests to the endpoint. +Provider, and reloads them from the public keys endpoint when the cache expires. + +Key rotation does not depend on this setting: a token signed with a key QuestDB +has not cached triggers an immediate reload. The expiry governs how long a key +the provider has already withdrawn stays usable, so lower it if signing keys are +revoked, at the cost of more requests to the endpoint. + +Only used when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, +which is the only case in which QuestDB validates token signatures itself. ### acl.oidc.response.buffer.size From 58c7dfda304dc56806f653edd9d9e1a5597564e5 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 19:50:17 +0100 Subject: [PATCH 16/54] Name the response each protocol gives when the claims are missing Authentication does fail, but only the HTTP endpoint answers with 401. PGWire returns an authentication error instead. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index e64573b039..2183e8ae24 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -829,10 +829,12 @@ claim called `groups`. Its name is set with [`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which has no default and must be set whenever OIDC is enabled. -If the groups claim is missing or it is an empty list, authentication fails and -QuestDB replies with `401 Unauthorized`. The same happens when the claim named -by [`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing -or empty. +If the groups claim is missing or it is an empty list, authentication fails. The +same happens when the claim named by +[`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing or +empty. Over HTTP QuestDB replies with `401 Unauthorized`, while on the PGWire +endpoint the client receives an authentication error. Either way the reason is +only visible in the server log. The user has to have at least the `HTTP` permission to be able to successfully login via the [Web Console](/docs/getting-started/web-console/overview/). From fb1a3c22200729757bb77a83162740bf25ce2933 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:03:10 +0100 Subject: [PATCH 17/54] Remove the Device Code Flow documentation Server-side support for the Device Code Flow is not finished, the clients are still work in progress, and there are open PRs which will change the server behaviour further. Documenting it now would describe an interface which is going to move. Removes the `acl.oidc.device.authorization.endpoint` configuration entry, the Device Code Flow section of the OIDC guide with its worked example, the tip which pointed the CLI section at it, and the endpoint from the settings endpoint example. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 5 +- documentation/configuration/oidc.md | 20 ----- documentation/security/oidc.mdx | 109 +--------------------------- 3 files changed, 3 insertions(+), 131 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 5ca80bffb1..6b8e897c43 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -31,7 +31,7 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another -- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token and device authorization endpoints, and whether PKCE and the `state` parameter are required +- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization and token endpoints, and whether PKCE and the `state` parameter are required ### Reference @@ -49,10 +49,9 @@ This page tracks significant updates to the QuestDB documentation. - Added the [`cairo.sql.parquet.cache.memory.size`](/docs/configuration/cairo-engine/) configuration property (256 MB default), deprecating the slot-based `cairo.sql.parquet.frame.cache.capacity` - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` -- Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.device.authorization.endpoint`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default +- Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers - [PingFederate SSO](/docs/security/oidc/#pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none -- [OpenID Connect (OIDC)](/docs/security/oidc/#device-code-flow) - Documented the Device Code Flow for clients without a browser of their own, how QuestDB advertises the Device Authorization Endpoint to them, and why it is preferable to the Resource Owner Password Credentials flow ### Updated diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 75f5a1dc12..fe1829e518 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -164,26 +164,6 @@ provider's configuration document and the settings below are not used. OIDC Authorization Endpoint. The default value should work for the Ping Identity Platform. -### acl.oidc.device.authorization.endpoint - -- **Default**: none -- **Reloadable**: no - -OIDC Device Authorization Endpoint, used by clients which authenticate with -the [Device Code Flow](/docs/security/oidc/#device-code-flow). Unlike the -other endpoints, it has no default value. - -QuestDB does not run the Device Code Flow itself. It publishes this endpoint to -clients through the -[settings endpoint](/docs/security/oidc/#settings-endpoint), so that a client -knows where to start the flow. A client which does not receive it has to be -given the endpoint directly, or resolve it from the OIDC Provider itself. - -When `acl.oidc.configuration.url` is set, QuestDB takes this endpoint from the -provider's configuration document instead, if the provider advertises a -`device_authorization_endpoint`. It is the only endpoint discovery is allowed to -omit. - ### acl.oidc.public.keys.endpoint - **Default**: `/pf/JWKS` diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 2183e8ae24..1b4db78936 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -342,8 +342,7 @@ settings: "acl.oidc.redirect.uri": "https://questdb.host:9000", "acl.oidc.scope": "openid", "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", - "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", - "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" + "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2" }, "preferences.version": 0, "preferences": {} @@ -359,10 +358,6 @@ hard coding the provider's details. The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the [endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's configuration document when `acl.oidc.configuration.url` is set. -`acl.oidc.device.authorization.endpoint` is the only one which may be absent, -because it is the only one QuestDB can run without, and the only one with no -default: the value above is whatever the operator configured or discovery -returned. The path of the endpoint is set by [`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings), @@ -583,108 +578,6 @@ testldap=> testldap=> ``` -:::tip - -The Resource Owner Password Credentials flow requires the client to handle the -user's password. A client which is able to display a URL and poll for the -result should use the [Device Code Flow](#device-code-flow) instead. - -::: - -### Device Code Flow - -The [Device Code Flow](https://datatracker.ietf.org/doc/html/rfc8628) is the -standard way for a client with no browser of its own to obtain tokens without -ever handling the user's password. The client asks the OIDC Provider for a -device code, displays a short user code together with a verification URL, and -polls the Token endpoint while the user completes the login in a browser on any -device. - -QuestDB does not run the flow itself, and there is no setting to switch it on. -The client talks to the OIDC Provider directly. QuestDB only advertises the -provider's Device Authorization Endpoint through its -[settings endpoint](#settings-endpoint), so that a client knows where to start: - -- When `acl.oidc.configuration.url` is set, QuestDB takes the endpoint from the - provider's configuration document, provided the provider advertises a - `device_authorization_endpoint`. -- When the provider is configured by host, set - [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) - explicitly. It is the only endpoint with no default value, so take the path - from the OIDC Provider's documentation. - -If QuestDB does not advertise the endpoint, the client has to be given it -directly, or resolve it from the OIDC Provider itself. - -The OIDC Provider has to support the flow too, with the device code grant -enabled for the client. - -#### Run the flow - -First read the endpoints from QuestDB. The -[settings endpoint](#settings-endpoint) carries both the Device Authorization -Endpoint the flow starts at, and the Token endpoint it polls: - -```python title="Discover the endpoints" -import requests -import time - -settings = requests.get("https://questdb.host:9000/settings").json()["config"] -device_endpoint = settings["acl.oidc.device.authorization.endpoint"] -token_endpoint = settings["acl.oidc.token.endpoint"] -client_id = settings["acl.oidc.client.id"] -scope = settings["acl.oidc.scope"] -``` - -Then ask the OIDC Provider for a device code, and tell the user where to log in. -The fields of the response are the ones defined by -[RFC 8628](https://datatracker.ietf.org/doc/html/rfc8628#section-3.2): - -```python title="Request a device code" -device = requests.post(device_endpoint, - data={"client_id": client_id, - "scope": scope}, - headers={"Content-Type": "application/x-www-form-urlencoded"}).json() - -print(f"Open {device['verification_uri']} and enter the code {device['user_code']}") -``` - -Finally, poll the Token endpoint until the user completes the login. The error -`authorization_pending` means the user has not finished yet, `slow_down` asks -for a longer interval, and any other error ends the flow: - -```python title="Poll for the access token" -interval = device.get("interval", 5) -while True: - time.sleep(interval) - response = requests.post(token_endpoint, - data={"grant_type": "urn:ietf:params:oauth:grant-type:device_code", - "device_code": device["device_code"], - "client_id": client_id}, - headers={"Content-Type": "application/x-www-form-urlencoded"}).json() - if "access_token" in response: - access_token = response["access_token"] - break - error = response.get("error") - if error == "slow_down": - interval += 5 - elif error != "authorization_pending": - raise RuntimeError(response) -``` - -The client now holds a token, and presents it to QuestDB the same way as any -other token, as shown in the [7th step](#7-database-access) of the Authorization -Code Flow. - -:::note - -Take the ID token from the response instead of the access token when -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -is set to `true`. QuestDB then reads the group memberships from the token -itself, and only the ID token carries them. - -::: - ## Non-interactive clients Non-interactive clients are usually jobs or standalone applications, such as a From 8733605be664aa4a5fe7ce7a8ee563785be67072 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:03:20 +0100 Subject: [PATCH 18/54] Correct when QuestDB reads the OIDC public keys The page said the keys are only read when the group memberships are encoded in the token. QuestDB builds the public keys store whenever OIDC is enabled, and the store downloads the keys from its constructor, so the endpoint is contacted at startup in the user info flow too. Only the use of the keys is conditional: token signatures are verified in the encoded-in-token flow. Names what the gate actually controls, and records that a failed download is logged rather than fatal, so an operator seeing that error in the log knows it is not a startup failure. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index fe1829e518..79e04f6f5d 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -173,11 +173,15 @@ JSON Web Key Set (JWKS) Endpoint. Provides the list of public keys used to decode and validate ID tokens issued by the OIDC Provider. The default value should work for the Ping Identity Platform. -The keys are only read when +The keys are only used to validate tokens when [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`. With the default user info flow QuestDB validates tokens by calling the user info endpoint instead. +QuestDB downloads the keys from this endpoint at startup either way, so that the +cache is never empty. A failure to download them is logged, and does not stop +the server. + ### acl.oidc.token.endpoint - **Default**: `/as/token.oauth2` From d827870b14364d69911a452f6a7f16c04e65b698 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:03:28 +0100 Subject: [PATCH 19/54] Say what disabling the user info cache does in each flow Setting the TTL to zero was documented as validating every request against the OIDC Provider. That holds only in the user info flow. When the group memberships are encoded in the token, revalidation verifies the token signature locally and never calls the provider, so disabling the cache buys no round trip and gives no revocation. The distinction matters because the Entra ID walkthrough turns that setting on, and an operator zeroing the TTL there would expect revocation. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 79e04f6f5d..472c58847b 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -284,8 +284,10 @@ User info cache entry TTL in milliseconds, as a plain integer only. QuestDB caches user info responses for each valid access token. This setting controls how often the access token is validated and user info refreshed. -Set it to `0` to disable the cache, so that every request is validated against -the OIDC Provider. +Set it to `0` to disable the cache, so that every request is revalidated. In the +default user info flow that means a call to the OIDC Provider on every request. +When [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is +`true` the token is revalidated locally, and the provider is not contacted. ### acl.oidc.public.keys.expiry From 952e8d5d28112e44176db4cf9b82a117f7529549 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:03:35 +0100 Subject: [PATCH 20/54] Describe the whole OIDC guide, not just Web Console SSO The description ended at "QuestDB Enterprise Web Console", while the page also covers PGWire token authentication, group mapping, and the settings endpoint a client reads its OIDC configuration from. The build copies title and description verbatim into llms.txt, so that sentence is the whole of what a retrieval index sees for this page, and it pointed away from most of the content. Avoids a colon in the value, which breaks the YAML front matter. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 1b4db78936..b074d2ad04 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1,6 +1,6 @@ --- title: OpenID Connect (OIDC) Integration -description: Configure OpenID Connect (OIDC) integration with external Identity Providers for SSO authentication in QuestDB Enterprise Web Console. +description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, PGWire token authentication, group mapping, and the settings endpoint. --- import Screenshot from "@theme/Screenshot"; From 8792eac4997bc74ea3094f5d5ebb31e4fdf65741 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:40:19 +0100 Subject: [PATCH 21/54] Say what the OIDC settings actually require and enforce Add a Minimum configuration section carrying the four settings a working setup needs, and all eight conditions which make the server refuse to start. acl.oidc.enabled listed four of them and read as complete, so it now points at the section instead. The section also names acl.enabled, which OIDC needs and which neither page mentioned. Correct four entries which claimed more than the server does: - acl.oidc.pkce.required read as an enforcement switch. The server only publishes it, the same as acl.oidc.state.required, and enforces neither. Both entries now say so through a shared section intro, which also lets the state entry drop the mechanism it repeated. - acl.oidc.tls.keystore.password said the pair fails startup. The check runs only when OIDC is enabled. - acl.oidc.sub.claim read as cosmetic. An empty sub claim fails authentication, the same way a missing groups claim does. - acl.oidc.response.buffer.size advised raising it for large user info responses. The size which rejects a response is the request buffer, not this one, so the advice does not hold. It now states the accepted format, which takes K and M but no G, and what a response that does not fit looks like. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 81 +++++++++++++++++++++++------ 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 472c58847b..94401110dd 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -16,6 +16,40 @@ Provider (IdP). For detailed information about OIDC, see the [OpenID Connect (OIDC) integration guide](/docs/security/oidc). +## Minimum configuration + +OIDC requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be `true`, +which is the default. With access control disabled the OIDC settings are +ignored, and no OIDC authentication takes place. + +A working setup against a Ping Identity provider needs four settings. Every +other setting has a usable default: + +```shell +acl.oidc.enabled=true +acl.oidc.host=oidc.provider +acl.oidc.client.id=questdb +acl.oidc.groups.claim=groups +``` + +QuestDB refuses to start when the OIDC configuration is inconsistent. With +`acl.oidc.enabled=true`: + +- [`acl.oidc.client.id`](#acloidcclientid) and + [`acl.oidc.groups.claim`](#acloidcgroupsclaim) must be set. +- Exactly one of [`acl.oidc.host`](#acloidchost) and + [`acl.oidc.configuration.url`](#acloidcconfigurationurl) must be set. +- [`acl.basic.auth.realm.enabled`](/docs/configuration/iam/#aclbasicauthrealmenabled) + must be `false`. +- [`acl.oidc.tls.keystore.path`](#acloidctlskeystorepath) and + [`acl.oidc.tls.keystore.password`](#acloidctlskeystorepassword) must both be + set, or neither. +- [`acl.oidc.tls.enabled`](#acloidctlsenabled) must match the scheme of every + OIDC Provider URL. +- When [`acl.oidc.configuration.url`](#acloidcconfigurationurl) is set, the + document must be downloadable and parseable, and must name the authorization, + token, user info and JWKS endpoints. + ## General ### acl.oidc.audience @@ -57,7 +91,9 @@ Mutually exclusive with `acl.oidc.host`: setting both fails server startup. Enables or disables OIDC authentication. When enabled, `acl.oidc.client.id` and `acl.oidc.groups.claim` must also be set, along with either -`acl.oidc.host` or `acl.oidc.configuration.url`. +`acl.oidc.host` or `acl.oidc.configuration.url`. See +[Minimum configuration](#minimum-configuration) for the full set of startup +requirements. OIDC cannot be enabled together with [`acl.basic.auth.realm.enabled`](/docs/configuration/iam/#aclbasicauthrealmenabled). @@ -107,6 +143,12 @@ scope `openid` is mandatory and must always be included. ## Authentication flows +QuestDB publishes [`acl.oidc.pkce.required`](#acloidcpkcerequired) and +[`acl.oidc.state.required`](#acloidcstaterequired) to clients through the +[settings endpoint](/docs/security/oidc/#settings-endpoint), and enforces +neither. The client generates the code verifier and the `state` value, and +checks them. + ### acl.oidc.pg.token.as.password.enabled - **Default**: `false` @@ -121,8 +163,9 @@ contain the string `_sso`, or left empty if that is an option. - **Default**: `true` - **Reloadable**: no -Enables or disables PKCE for the Authorization Code Flow. This should always -be enabled in production. The Web Console is not fully secure without it. +Tells clients that PKCE is required for the Authorization Code Flow. This +should always be enabled in production. The Web Console is not fully secure +without it. ### acl.oidc.ropc.flow.enabled @@ -137,18 +180,14 @@ enabled, this flow must also be configured in the OIDC Provider. - **Default**: `false` - **Reloadable**: no -Requires the `state` parameter in the Authorization Code Flow, which protects -against CSRF attacks. QuestDB does not see the value itself. It publishes the -setting to clients through the -[settings endpoint](/docs/security/oidc/#settings-endpoint), the same way it -publishes `acl.oidc.pkce.required`, and the client is what generates and checks -the value. +Tells clients that the `state` parameter is required in the Authorization Code +Flow, which protects against CSRF attacks. Enable it if the OIDC Provider +requires the `state` parameter, or to add CSRF protection on top of PKCE. -Enable it if the OIDC Provider requires the `state` parameter, or to add CSRF -protection on top of PKCE. The -[Web Console](/docs/security/oidc/#1-secret-generation) generates the value, -sends it in the authorization request, and checks that the provider returns it -unchanged. +The [Web Console](/docs/getting-started/web-console/overview/) generates the +value, sends it in the authorization request, and checks that the provider +returns it unchanged. See +[Secret generation](/docs/security/oidc/#1-secret-generation). ## Endpoints @@ -224,7 +263,8 @@ A URL whose scheme does not match fails server startup. - **Reloadable**: no Keystore password. Must be set whenever `acl.oidc.tls.keystore.path` is set. -Setting either one without the other fails server startup. +When OIDC is enabled, setting either one without the other fails server +startup. ### acl.oidc.tls.keystore.path @@ -273,6 +313,10 @@ The name of the claim in the user information that contains the user's name. Could be a username, full name, or email. Displayed in the Web Console and logged for audit purposes. +The claim must be present and non-empty, otherwise authentication fails, the +same way it does when the groups claim is missing. See +[Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). + ## Caching and buffers ### acl.oidc.cache.ttl @@ -315,8 +359,11 @@ which is the only case in which QuestDB validates token signatures itself. - **Reloadable**: no Size of the buffer used to receive HTTP responses from the OIDC Provider. -Increase it if the provider sends large responses, such as user info -containing a long list of group memberships. +Accepts a plain byte count, or a value with a `K` or `M` suffix, such as +`512K`. There is no `G` suffix. + +If a response from the OIDC Provider does not fit, authentication fails and the +reason is logged by the server. ### acl.oidc.string.pool.capacity From b875d2d7404d9de954a6b581ad449a3b1707c5fd Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 21 Aug 2026 22:40:27 +0100 Subject: [PATCH 22/54] Document what the settings endpoint guarantees to a client The section told any client to read the endpoints from /settings instead of hard coding them, without saying what the response guarantees. Add a table giving each key its type and when it is present, and note that a provider advertising a device authorization endpoint adds a tenth key. acl.oidc.client.id and acl.oidc.redirect.uri are published as null when unset, so the redirect URI now carries the fallback a client needs. Say to read acl.oidc.enabled first. The acl.oidc.* entries are published whether or not OIDC is enabled, and the endpoint URLs are then built from the defaults, so a client which skips the check sends users to a URL which looks real and is not. The same note names acl.enabled, which can leave acl.oidc.enabled reporting true while nothing authenticates. Correct http.context.settings, which said the Web Console reads it. The Web Console requests /settings and ignores the setting, so moving the path stops it from loading. Both pages now say to leave it alone. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/http-server.md | 9 ++++-- documentation/security/oidc.mdx | 35 ++++++++++++++++++++-- 2 files changed, 39 insertions(+), 5 deletions(-) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index 17bee5659b..1470f4c820 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -493,11 +493,14 @@ Context path for the file import service. - **Default**: `/settings` - **Reloadable**: no -Context path for the service which provides server-side settings to clients. The -[Web Console](/docs/getting-started/web-console/overview/) reads it, and so does -any OIDC client which discovers the provider's endpoints from the +Context path for the service that serves server-side settings to clients, such +as an OIDC client discovering the provider's endpoints from the [settings endpoint](/docs/security/oidc/#settings-endpoint). +Leave this at the default. The +[Web Console](/docs/getting-started/web-console/overview/) requests `/settings` +and does not read this setting, so changing the path stops it from loading. + ### http.context.table.status - **Default**: `/chk` diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index b074d2ad04..b29fdc9829 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -355,13 +355,44 @@ Each key is the name of the its authorization request, and any other client can do the same instead of hard coding the provider's details. +| Key | Type | Present | +| --- | --- | --- | +| `acl.oidc.enabled` | boolean | always | +| `acl.oidc.pkce.required` | boolean | always | +| `acl.oidc.state.required` | boolean | always | +| `acl.oidc.groups.encoded.in.token` | boolean | always | +| `acl.oidc.client.id` | string or `null` | always | +| `acl.oidc.redirect.uri` | string or `null` | always | +| `acl.oidc.scope` | string | always | +| `acl.oidc.authorization.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.token.endpoint` | string | when the endpoint is resolved | + The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the [endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's -configuration document when `acl.oidc.configuration.url` is set. +configuration document when `acl.oidc.configuration.url` is set. A provider +which advertises a device authorization endpoint adds +`acl.oidc.device.authorization.endpoint` to the response as well. + +`acl.oidc.redirect.uri` is `null` unless +[`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. +A client should then fall back to its own location, as the Web Console does. + +:::note + +Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries +are published whether or not OIDC is enabled, and with OIDC disabled the +endpoint URLs are built from the defaults rather than from a real provider. + +OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be +`true`. When access control is disabled, `acl.oidc.enabled` still reports the +configured value while no OIDC authentication takes place. + +::: The path of the endpoint is set by [`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings), -and defaults to `/settings`. +and defaults to `/settings`. Leave it at the default: the Web Console requests +`/settings` and does not read this setting. ## Interactive clients From afb0e2a17e16ba87b26370e19f3f74a10ad721ab Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 22 Aug 2026 00:18:57 +0100 Subject: [PATCH 23/54] Correct what overriding the settings context path does Both pages said to leave `http.context.settings` alone because changing it stops the Web Console from loading. It does not. After reading the override, PropServerConfiguration re-adds the default `http.context.web.console` + `/settings` unconditionally, under a comment saying it exists so that customization does not break the Web Console, and HttpServer binds the settings processor to every path in the set. Overriding the setting adds a path; the default keeps serving and the console keeps working. The `http.context.web.console` entry twenty-five lines below already documented this correctly, so the page contradicted itself. Keep the half that is true - the Web Console does request `/settings` and does not read the setting - and note that the property takes a comma-separated list, which neither page mentioned. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/http-server.md | 7 ++++--- documentation/security/oidc.mdx | 4 ++-- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index 1470f4c820..be97adbf46 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -497,9 +497,10 @@ Context path for the service that serves server-side settings to clients, such as an OIDC client discovering the provider's endpoints from the [settings endpoint](/docs/security/oidc/#settings-endpoint). -Leave this at the default. The -[Web Console](/docs/getting-started/web-console/overview/) requests `/settings` -and does not read this setting, so changing the path stops it from loading. +Accepts a comma-separated list of paths. Setting it adds paths rather than +moving the service: QuestDB keeps serving the default path as well, so the +[Web Console](/docs/getting-started/web-console/overview/), which requests +`/settings` and does not read this setting, keeps working. ### http.context.table.status diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index b29fdc9829..f5e2c29abd 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -391,8 +391,8 @@ configured value while no OIDC authentication takes place. The path of the endpoint is set by [`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings), -and defaults to `/settings`. Leave it at the default: the Web Console requests -`/settings` and does not read this setting. +and defaults to `/settings`. Setting it adds paths rather than moving the +endpoint, so `/settings` keeps working whatever else is configured. ## Interactive clients From 7ac66b92a4351ae90635191f2af5c5adcb43388d Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 22 Aug 2026 00:39:41 +0100 Subject: [PATCH 24/54] Stop overstating what the OIDC settings guarantee Four claims on the configuration page promised more than the server delivers. `acl.oidc.cache.ttl` said that with the groups encoded in the token the token is "revalidated locally, and the provider is not contacted". Both halves were too strong. The local check covers the signature and the audience; jwt.rs sets validate_exp to false, so the token's expiry is never tested. And the JWKS store still calls the provider when the cached keys expire or the key id is unknown. `acl.oidc.sub.claim` said a missing claim fails authentication "the same way it does when the groups claim is missing" - a rule this page did not contain. `acl.oidc.groups.claim` documented only the startup requirement, never the runtime failure, and unlike its sibling carried no link to the guide. State the runtime failure on both, and let sub.claim point at groups.claim. `acl.oidc.response.buffer.size` claimed that a response which does not fit fails authentication with a logged reason. The setting sizes the HTTP client's response parser buffer; no code path ties an overflow to that outcome. Keep the generic behaviour, which does hold. The PKCE and state paragraph said the client checks both values. It generates both, but the provider checks the verifier at the token endpoint; the client checks only the state value it gets back. Also widen the page description. It stopped at "configuration settings" while the page covers startup rules, endpoints, TLS, claims, caching and buffers, and both OIDC pages share a title - in llms.txt the description is the only thing telling them apart. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/configuration/oidc.md | 35 ++++++++++++++++++----------- 1 file changed, 22 insertions(+), 13 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 94401110dd..0444d69d9e 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -1,6 +1,6 @@ --- title: OpenID Connect (OIDC) -description: Configuration settings for OpenID Connect integration in QuestDB Enterprise. +description: "QuestDB Enterprise acl.oidc.* settings reference: minimum configuration and startup rules, endpoints, TLS, user and group claims, caching and buffers." --- :::note @@ -146,8 +146,9 @@ scope `openid` is mandatory and must always be included. QuestDB publishes [`acl.oidc.pkce.required`](#acloidcpkcerequired) and [`acl.oidc.state.required`](#acloidcstaterequired) to clients through the [settings endpoint](/docs/security/oidc/#settings-endpoint), and enforces -neither. The client generates the code verifier and the `state` value, and -checks them. +neither. The client generates the code verifier and the `state` value; the +provider checks the verifier, and the client checks the `state` value it gets +back. ### acl.oidc.pg.token.as.password.enabled @@ -295,6 +296,10 @@ which it connects. The name of the custom claim in the user information that contains the group memberships of the user. Required when OIDC is enabled. +If the claim is missing from the user information, or it is an empty list, +authentication fails. See +[Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). + ### acl.oidc.groups.encoded.in.token - **Default**: `false` @@ -313,8 +318,9 @@ The name of the claim in the user information that contains the user's name. Could be a username, full name, or email. Displayed in the Web Console and logged for audit purposes. -The claim must be present and non-empty, otherwise authentication fails, the -same way it does when the groups claim is missing. See +If the claim is missing from the user information, or empty, authentication +fails. The same applies to the claim named by +[`acl.oidc.groups.claim`](#acloidcgroupsclaim). See [Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). ## Caching and buffers @@ -328,10 +334,13 @@ User info cache entry TTL in milliseconds, as a plain integer only. QuestDB caches user info responses for each valid access token. This setting controls how often the access token is validated and user info refreshed. -Set it to `0` to disable the cache, so that every request is revalidated. In the -default user info flow that means a call to the OIDC Provider on every request. -When [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is -`true` the token is revalidated locally, and the provider is not contacted. +Set it to `0` to disable the cache, so that every request is checked again. In +the default user info flow that means a call to the OIDC Provider on every +request. When +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true` +QuestDB checks the token's signature and audience locally instead, and contacts +the provider only when the public keys have to be reloaded. The local check +does not test the token's expiry. ### acl.oidc.public.keys.expiry @@ -358,11 +367,11 @@ which is the only case in which QuestDB validates token signatures itself. - **Default**: `1M` - **Reloadable**: no -Size of the buffer used to receive HTTP responses from the OIDC Provider. -Accepts a plain byte count, or a value with a `K` or `M` suffix, such as -`512K`. There is no `G` suffix. +Size of the buffer used to receive and parse HTTP responses from the OIDC +Provider. Accepts a plain byte count, or a value with a `K` or `M` suffix, such +as `512K`. There is no `G` suffix. -If a response from the OIDC Provider does not fit, authentication fails and the +When a request to the OIDC Provider fails, authentication fails with it and the reason is logged by the server. ### acl.oidc.string.pool.capacity From 76237322193bc42c23b2a829d575069d135d199f Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 22 Aug 2026 00:39:49 +0100 Subject: [PATCH 25/54] Complete what the settings endpoint section tells a client The section enumerated the response but left out three things a client needs. `acl.oidc.device.authorization.endpoint` was described in prose below the table and missing from the table itself, and the prose attributed it solely to a provider advertising the endpoint. In host mode the property drives it - the project's own SettingsEndpointTest is host-based and its payload carries the key. Say configured or discovered, and give it a row. `acl.enabled` is published in the same response, immediately before the acl.oidc entries. The note warned that acl.oidc.enabled can read true while access control is off, and then withheld the key that settles it. Add it to the example and the table, and tell the client to read both. That QuestDB advertises the PKCE and state flags and enforces neither was stated only on the configuration page. A client author reading this section alone would conclude the server requires PKCE. Say it here too, in the same words. Also link the settings endpoint from the two sections whose readers are its audience. Browser-based clients builds an authorization request, and non-interactive clients posts to a token endpoint it currently hard codes. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index f5e2c29abd..2a17b0e9ee 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -334,6 +334,7 @@ settings: ```json title="Settings response example" { "config": { + "acl.enabled": true, "acl.oidc.enabled": true, "acl.oidc.pkce.required": true, "acl.oidc.state.required": false, @@ -357,6 +358,7 @@ hard coding the provider's details. | Key | Type | Present | | --- | --- | --- | +| `acl.enabled` | boolean | always | | `acl.oidc.enabled` | boolean | always | | `acl.oidc.pkce.required` | boolean | always | | `acl.oidc.state.required` | boolean | always | @@ -366,17 +368,23 @@ hard coding the provider's details. | `acl.oidc.scope` | string | always | | `acl.oidc.authorization.endpoint` | string | when the endpoint is resolved | | `acl.oidc.token.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.device.authorization.endpoint` | string | when the endpoint is configured or discovered | The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the [endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's -configuration document when `acl.oidc.configuration.url` is set. A provider -which advertises a device authorization endpoint adds -`acl.oidc.device.authorization.endpoint` to the response as well. +configuration document when `acl.oidc.configuration.url` is set. +`acl.oidc.device.authorization.endpoint` is present when a device authorization +endpoint has been configured, or when the provider advertises one. `acl.oidc.redirect.uri` is `null` unless [`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. A client should then fall back to its own location, as the Web Console does. +QuestDB advertises `acl.oidc.pkce.required` and `acl.oidc.state.required`, and +enforces neither. The client generates the code verifier and the `state` value; +the provider checks the verifier, and the client checks the `state` value it +gets back. + :::note Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries @@ -384,8 +392,9 @@ are published whether or not OIDC is enabled, and with OIDC disabled the endpoint URLs are built from the defaults rather than from a real provider. OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be -`true`. When access control is disabled, `acl.oidc.enabled` still reports the -configured value while no OIDC authentication takes place. +`true`, which the same response carries. When access control is disabled, +`acl.oidc.enabled` still reports the configured value while no OIDC +authentication takes place, so read both keys. ::: @@ -419,6 +428,10 @@ possible flows to request an access token.: 2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) flow +The client can read the authorization and token endpoints, the client id and +the scopes from the [settings endpoint](#settings-endpoint) instead of hard +coding them. + The Web Console implements the [Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a special version of the Authorization Code flow designed for mobile apps and @@ -616,7 +629,9 @@ client for ingesting data. It is practical to manage their credentials via an OAuth2 provider too. As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion: +themselves and then use it to authorise data ingestion. The token endpoint they +post to is published on the [settings endpoint](#settings-endpoint), so it does +not have to be hard coded: ```python import json From 32a17399f238b78b8e744992bd7b4b5522f5d13c Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 22 Aug 2026 00:39:54 +0100 Subject: [PATCH 26/54] Point the client pages at the OIDC settings endpoint Every client page tells the reader to acquire a token out of band from their IdP, and links to the top of the OIDC guide, which drops them at the architecture diagram. None of them mentioned that QuestDB publishes the provider's authorization and token endpoints on its settings endpoint, so a client can discover them instead of hard coding them. These pages are the settings endpoint's audience, and it had no inbound link from any of them. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/connect/clients/c-and-cpp.md | 2 +- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 3 +++ documentation/connect/clients/python.md | 2 +- documentation/connect/wire-protocols/qwp-ingress-websocket.md | 4 +++- 5 files changed, 9 insertions(+), 4 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index 24857c2724..3f091a5a9d 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -332,7 +332,7 @@ following are **not** supported: | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the pool when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...`, and rebuild the pool when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. See the connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, close the pool and build a fresh one with the new token. | diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index ca4dc261f4..d597deba9f 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -223,7 +223,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 78cb68217f..982b340406 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -362,6 +362,9 @@ the ingress and egress WebSocket upgrades. It is a **static credential**: the client sends exactly the string you pass and never refreshes or renews it. Acquire it out of band — QuestDB Enterprise issues bearer tokens through its [OpenID Connect flow](/docs/security/oidc/) — and manage its lifetime yourself. +QuestDB publishes the provider's authorization and token endpoints on its +[settings endpoint](/docs/security/oidc/#settings-endpoint), so a client can +discover them instead of hard coding them. When the token expires or is rotated, construct a new handle with the new token. An expired or rejected token surfaces as an authentication failure (see [Connection-level errors](#connection-level-errors)). It is mutually exclusive diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 50de24dba0..0b275a0fcd 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -171,7 +171,7 @@ following are **not** supported: | Path | Status | Workaround | | --- | --- | --- | -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the handle when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...`, and rebuild the handle when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. | | Token rotation mid-session | Not supported. The handle keeps the credentials it was built with and presents them on every connection it opens — including reconnects and failover, so an expired token also breaks mid-session reconnection. | On token expiry, close the handle and build a fresh one with the new token. | diff --git a/documentation/connect/wire-protocols/qwp-ingress-websocket.md b/documentation/connect/wire-protocols/qwp-ingress-websocket.md index e27c7ea5eb..2711a7feb4 100644 --- a/documentation/connect/wire-protocols/qwp-ingress-websocket.md +++ b/documentation/connect/wire-protocols/qwp-ingress-websocket.md @@ -143,7 +143,9 @@ Supported methods: [HTTP basic authentication](/docs/connect/compatibility/rest-api/#http-basic-authentication). - **Token-based auth** (Enterprise only): see [Authentication via token in QuestDB Enterprise](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise). -- **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). +- **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). The + [settings endpoint](/docs/security/oidc/#settings-endpoint) publishes the + provider's authorization and token endpoints. A failed authentication results in a `401` or `403` HTTP response before the WebSocket connection is established. No QWP-level auth handshake exists. From 6703c21edd248a7ce4935d94c6893d3f917c3083 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 22 Aug 2026 00:40:00 +0100 Subject: [PATCH 27/54] Move the OIDC guide entries out of the reference section The docs-changelog criteria put config properties and syntax additions to reference pages under Reference, and rewritten sections and new examples under Updated. A note added to a walkthrough step and a new server.conf example in the PingFederate walkthrough are both guide changes, not reference ones. The PingFederate entry also linked to the top of a walkthrough some 330 lines long, rather than the configuration block it describes. The entry for the four missing configuration options stays under Reference, where it belongs. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 6b8e897c43..356e13a881 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -50,11 +50,11 @@ This page tracks significant updates to the QuestDB documentation. - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` - Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default -- [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers -- [PingFederate SSO](/docs/security/oidc/#pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none ### Updated +- [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) - [Web Console](/docs/getting-started/web-console/overview/) - Documented query sharing by link and tab import/export in the [code editor](/docs/getting-started/web-console/code-editor/), custom AI providers and per-provider permission levels in [QuestDB AI](/docs/getting-started/web-console/questdb-ai/), automatic column sizing in the [result grid](/docs/getting-started/web-console/result-grid/), and the storage policy section in [table details](/docs/getting-started/web-console/table-details/) - [AI coding agents](/docs/getting-started/ai-coding-agents/) - Repositioned around the agent skill and the Web Console MCP bridge together From 5e815f7d74078cc3632826be2b539e5ad2f6bc93 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 23 Aug 2026 22:35:06 +0100 Subject: [PATCH 28/54] Tell clients which token to send, and correct three claims Review of the OIDC pages turned up one gap and three statements that do not hold. Each is verified against the Enterprise source. Which token to send. The settings endpoint publishes acl.oidc.groups.encoded.in.token to every client and never said what to do with it, yet it decides which token belongs in the Authorization header: the ID token when the groups are encoded in it, the access token otherwise. Sending the wrong one is a 401 with the reason only in the log. The Web Console already picks the token this way. Documented in a new section, in the option's own entry, and in the ingestion example. The step 7 aside claiming the token "is rather opaque and does not contain user details" is only true of the access token, so it is now scoped: the ID token is a JWT whose payload is readable base64. The settings path. Setting http.context.settings adds paths rather than moving the service, but the path that survives is the Web Console context path plus /settings, not a literal /settings. Both pages said /settings keeps working whatever else is configured, which stops being true as soon as http.context.web.console is changed, as the neighbouring entry on the same page already noted. The Web Console also requests a relative "settings" rather than an absolute path. The non-interactive example. The prose said the token endpoint does not have to be hard coded and the example below it hard coded the endpoint and the client id. It now reads them from the settings endpoint, which is the only worked demonstration of the feature on the page. acl.enabled. Disabling access control stops OIDC authentication, but it does not stop the OIDC settings from being validated: the startup rules still fire and the provider's configuration document is still downloaded, so a broken OIDC configuration still prevents the server from starting. Also documented acl.oidc.device.authorization.endpoint, which the settings table already named while the configuration page did not. QuestDB never calls it and does not implement the Device Authorization Flow, it resolves the endpoint and republishes it for clients which do, so the option is documented without documenting the flow. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 3 +- documentation/configuration/http-server.md | 11 +++- documentation/configuration/oidc.md | 28 +++++++++- documentation/security/oidc.mdx | 64 ++++++++++++++++------ 4 files changed, 85 insertions(+), 21 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 356e13a881..66d2b25747 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -49,11 +49,12 @@ This page tracks significant updates to the QuestDB documentation. - Added the [`cairo.sql.parquet.cache.memory.size`](/docs/configuration/cairo-engine/) configuration property (256 MB default), deprecating the slot-based `cairo.sql.parquet.frame.cache.capacity` - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` -- Documented the OIDC configuration options that were missing from the [OIDC page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, and `acl.oidc.string.pool.capacity`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default +- Documented the OIDC configuration options that were missing from the [OIDC configuration page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, `acl.oidc.string.pool.capacity`, and `acl.oidc.device.authorization.endpoint`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default ### Updated - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [Which token to send](/docs/security/oidc/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication - [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) - [Web Console](/docs/getting-started/web-console/overview/) - Documented query sharing by link and tab import/export in the [code editor](/docs/getting-started/web-console/code-editor/), custom AI providers and per-provider permission levels in [QuestDB AI](/docs/getting-started/web-console/questdb-ai/), automatic column sizing in the [result grid](/docs/getting-started/web-console/result-grid/), and the storage policy section in [table details](/docs/getting-started/web-console/table-details/) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index be97adbf46..6a7f4dfc1a 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -499,8 +499,15 @@ as an OIDC client discovering the provider's endpoints from the Accepts a comma-separated list of paths. Setting it adds paths rather than moving the service: QuestDB keeps serving the default path as well, so the -[Web Console](/docs/getting-started/web-console/overview/), which requests -`/settings` and does not read this setting, keeps working. +[Web Console](/docs/getting-started/web-console/overview/), which does not read +this setting, keeps working. + +The default path follows +[`http.context.web.console`](#httpcontextwebconsole), so it is `/settings` only +while that setting is at its default. Setting +`http.context.web.console=/console` makes the preserved path +`/console/settings`, and bare `/settings` is then served only if it is listed +here explicitly. ### http.context.table.status diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 0444d69d9e..bdc7fc943d 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -19,8 +19,12 @@ For detailed information about OIDC, see the ## Minimum configuration OIDC requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be `true`, -which is the default. With access control disabled the OIDC settings are -ignored, and no OIDC authentication takes place. +which is the default. With access control disabled no OIDC authentication takes +place, but the OIDC settings are still validated at startup: with +`acl.oidc.enabled=true` the server enforces every rule below and downloads the +provider's configuration document, so an inconsistent OIDC configuration still +prevents it from starting. Set `acl.oidc.enabled=false` to take the settings out +of play entirely. A working setup against a Ping Identity provider needs four settings. Every other setting has a usable default: @@ -204,6 +208,20 @@ provider's configuration document and the settings below are not used. OIDC Authorization Endpoint. The default value should work for the Ping Identity Platform. +### acl.oidc.device.authorization.endpoint + +- **Default**: none +- **Reloadable**: no + +OIDC Device Authorization Endpoint. Unlike the other endpoint settings this one +has no default, and QuestDB never calls it. QuestDB resolves the endpoint and +publishes it on the +[settings endpoint](/docs/security/oidc/#settings-endpoint), for clients which +implement the Device Authorization Flow themselves. + +Left unset, and absent from the provider's configuration document, the endpoint +stays unresolved and the key is omitted from the settings response. + ### acl.oidc.public.keys.endpoint - **Default**: `/pf/JWKS` @@ -309,6 +327,12 @@ When `true`, QuestDB looks for group memberships in the ID token instead of calling the User Info endpoint. Set to `true` if the OIDC Provider encodes group memberships directly into the token. +This also changes which token the client has to send: the ID token when the +setting is `true`, the access token when it is `false`. QuestDB publishes the +setting on the +[settings endpoint](/docs/security/oidc/#settings-endpoint) so that clients can +[pick the right one](/docs/security/oidc/#which-token-to-send). + ### acl.oidc.sub.claim - **Default**: `sub` diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 2a17b0e9ee..2b36565a4c 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -240,10 +240,14 @@ The validity of the tokens are configurable inside the OIDC Provider. With the tokens, the Web Console can interact with the database. -The access token is in the header of every request sent to QuestDB. +A token is in the header of every request sent to QuestDB. Which of the two goes +there depends on where QuestDB reads the group memberships from, see +[Which token to send](#which-token-to-send). -> **Worried about exposing the token?** It is rather opaque and does not contain -> user details. +> **Worried about exposing the token?** The access token is rather opaque and +> does not contain user details. The ID token does: it is a JWT whose payload is +> base64 encoded, not encrypted, so treat it as you would the user's directory +> record. To carry out permission checks, the database has to know more about the user. @@ -373,8 +377,10 @@ hard coding the provider's details. The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the [endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's configuration document when `acl.oidc.configuration.url` is set. -`acl.oidc.device.authorization.endpoint` is present when a device authorization -endpoint has been configured, or when the provider advertises one. +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) +is present when a device authorization endpoint has been configured, or when the +provider advertises one. QuestDB publishes it without using it, for clients +which implement the Device Authorization Flow themselves. `acl.oidc.redirect.uri` is `null` unless [`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. @@ -385,6 +391,24 @@ enforces neither. The client generates the code verifier and the `state` value; the provider checks the verifier, and the client checks the `state` value it gets back. +### Which token to send + +`acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned +by the provider belongs in the `Authorization: Bearer` header: + +| Value | Token to send | How QuestDB validates it | +| --- | --- | --- | +| `false`, the default | the access token | by calling the user info endpoint | +| `true` | the ID token | locally, by checking its signature and audience | + +Sending the wrong one fails authentication with `401 Unauthorized` and the +reason in the server log. With +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +enabled the token has to be a JWT, because QuestDB reads the group memberships +straight out of its payload. The +[Web Console](/docs/getting-started/web-console/overview/) picks the token this +way, and so should any other client. + :::note Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries @@ -399,9 +423,13 @@ authentication takes place, so read both keys. ::: The path of the endpoint is set by -[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings), -and defaults to `/settings`. Setting it adds paths rather than moving the -endpoint, so `/settings` keeps working whatever else is configured. +[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). +Setting it adds paths rather than moving the endpoint, so the default path keeps +working too. That default is `/settings` only while +[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) +is at its default, as the settings path follows the Web Console context path. +A client which cannot assume both are at their defaults should make the path +configurable. ## Interactive clients @@ -630,8 +658,10 @@ OAuth2 provider too. As seen in the Jupyter notebook examples, the clients can request a token themselves and then use it to authorise data ingestion. The token endpoint they -post to is published on the [settings endpoint](#settings-endpoint), so it does -not have to be hard coded: +post to, the client id and the scopes are all published on the +[settings endpoint](#settings-endpoint), so none of them have to be hard coded. +The same response also tells the client +[which token to send](#which-token-to-send): ```python import json @@ -645,20 +675,22 @@ load_dotenv() user = os.environ.get("username") pwd = os.environ.get("password") -token_endpoint = "https://oidc.provider:443/as/token.oauth2" -response = requests.post(token_endpoint, +settings = requests.get("http://localhost:9000/settings").json()["config"] + +response = requests.post(settings["acl.oidc.token.endpoint"], data={"grant_type": "password", - "client_id": "testclient", + "client_id": settings["acl.oidc.client.id"], "username": user, "password": pwd, - "scope": "openid"}, + "scope": settings["acl.oidc.scope"]}, headers={"Content-Type": "application/x-www-form-urlencoded"}) response_body = response.content.decode("utf-8") tokens = json.loads(response_body) -access_token = tokens["access_token"] +groups_in_token = settings["acl.oidc.groups.encoded.in.token"] +token = tokens["id_token"] if groups_in_token else tokens["access_token"] -conf = f"http::addr=localhost:9000;token={access_token};" +conf = f"http::addr=localhost:9000;token={token};" with Sender.from_conf(conf) as sender: df = pd.read_csv("data.csv") df["ts"] = pd.to_datetime(df["ts"]) From 390408ce9e9b5ecb0c766cc0da3424b962031f01 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 23 Aug 2026 22:45:46 +0100 Subject: [PATCH 29/54] Say what QuestDB checks when it validates a token itself With acl.oidc.groups.encoded.in.token enabled QuestDB stops calling the provider and validates the ID token on its own. That it does not check the expiry was documented, but as the closing clause of acl.oidc.cache.ttl, a caching setting nobody opens unless they are tuning the cache. The consequence belongs where the mode is switched on. The option's own entry now carries the full picture: the signature, the audience and the presence of the claims are checked, exp, nbf and iss are not, and neither acl.oidc.cache.ttl nor a shorter token lifetime in the provider shortens how long an issued token is accepted. Once issued it stays good for as long as the signing key is in QuestDB's cache. The Entra ID walkthrough turns this mode on, because Entra ID cannot serve the group memberships from its User Info endpoint, so it now warns about it as well. acl.oidc.cache.ttl keeps a short note, because someone lowering the TTL to tighten revocation needs to know it will not help here, and points at the option for the detail. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/configuration/oidc.md | 35 ++++++++++++++++++++++++++--- documentation/security/oidc.mdx | 13 +++++++++++ 3 files changed, 46 insertions(+), 3 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 66d2b25747..c8ccdce603 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -55,6 +55,7 @@ This page tracks significant updates to the QuestDB documentation. - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers - [Which token to send](/docs/security/oidc/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication +- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the audience and the presence of the claims, but not `exp`, `nbf` or `iss`, so an expired token is still accepted - [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) - [Web Console](/docs/getting-started/web-console/overview/) - Documented query sharing by link and tab import/export in the [code editor](/docs/getting-started/web-console/code-editor/), custom AI providers and per-provider permission levels in [QuestDB AI](/docs/getting-started/web-console/questdb-ai/), automatic column sizing in the [result grid](/docs/getting-started/web-console/result-grid/), and the storage policy section in [table details](/docs/getting-started/web-console/table-details/) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index bdc7fc943d..6bad18d102 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -333,6 +333,31 @@ setting on the [settings endpoint](/docs/security/oidc/#settings-endpoint) so that clients can [pick the right one](/docs/security/oidc/#which-token-to-send). +It changes how tokens are validated too. In the default user info flow QuestDB +hands the token to the OIDC Provider on every cache miss, so the provider +decides whether it is still valid. With this setting enabled QuestDB validates +the token itself and never asks the provider about it: + +| Checked | Not checked | +| --- | --- | +| the signature, against the public key named by the token's `kid` | `exp`, the expiry | +| `aud`, against [`acl.oidc.audience`](#acloidcaudience) | `nbf`, the not-before time | +| that `sub` and the group memberships are present | `iss`, the issuer | + +:::caution + +An expired token is therefore still accepted. Once issued, a token stays valid +for as long as the key that signed it is in QuestDB's cache, which is governed +by [`acl.oidc.public.keys.expiry`](#acloidcpublickeysexpiry) and by how long the +provider publishes the key. Neither +[`acl.oidc.cache.ttl`](#acloidccachettl) nor a shorter token lifetime in the +provider shortens it. + +Do not enable this setting where you rely on being able to revoke a token, or on +the provider's token lifetimes being enforced. + +::: + ### acl.oidc.sub.claim - **Default**: `sub` @@ -362,9 +387,13 @@ Set it to `0` to disable the cache, so that every request is checked again. In the default user info flow that means a call to the OIDC Provider on every request. When [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true` -QuestDB checks the token's signature and audience locally instead, and contacts -the provider only when the public keys have to be reloaded. The local check -does not test the token's expiry. +QuestDB checks the token locally instead, and contacts the provider only when +the public keys have to be reloaded. + +That local check does not test the token's expiry, so shortening this TTL does +not shorten how long an issued token is accepted. See +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) for what is +and is not validated. ### acl.oidc.public.keys.expiry diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 2b36565a4c..cccb664148 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1465,6 +1465,19 @@ title="Application overview" width={600} /> +:::caution + +This setup relies on +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), +because Entra ID cannot serve the group memberships from its User Info +endpoint. QuestDB then validates the ID token itself and never asks Entra ID +about it, which means it does not check the token's expiry: an issued token is +accepted for as long as the key that signed it stays in QuestDB's cache. Read +that option before going to production, and note that clients must send the ID +token here, not the access token. + +::: + #### Map groups and grant permissions Now we can start QuestDB, and login with the built-in admin to create From 4ddda15294398ce8f4e2c7e787ceedf548d31e28 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 23 Aug 2026 22:50:57 +0100 Subject: [PATCH 30/54] Scope three OIDC options to what the server actually does The same class of imprecision the rest of this branch fixed, left behind in three entries which were not otherwise being touched. acl.oidc.audience reads as though QuestDB always checks the audience. It only does when acl.oidc.groups.encoded.in.token is true, the one case in which it validates tokens itself. In the user info flow the provider decides, and the audience is never looked at. This is the qualifier already carried by acl.oidc.public.keys.expiry and acl.oidc.public.keys.endpoint. acl.oidc.scope said the openid scope "is mandatory and must always be included", on a page which now lists what makes the server refuse to start. It is the provider that enforces it: QuestDB passes the value on without inspecting it, so omitting openid fails at the provider. acl.oidc.ropc.flow.enabled said only that it enables the flow, which undersells it. QuestDB runs the flow itself, exchanging HTTP basic and PGWire credentials for a token at the provider, which is what lets psql and similar clients log in with SSO credentials. Local users are matched first, so a local account shadows an Identity Provider one of the same name. The setting is also absent from the settings endpoint, so a client cannot discover whether the flow is available. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/changelog.mdx | 1 + documentation/configuration/oidc.md | 32 ++++++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index c8ccdce603..9421bec728 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -56,6 +56,7 @@ This page tracks significant updates to the QuestDB documentation. - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers - [Which token to send](/docs/security/oidc/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication - [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the audience and the presence of the claims, but not `exp`, `nbf` or `iss`, so an expired token is still accepted +- Scoped three [OIDC options](/docs/configuration/oidc/) to what the server actually does: `acl.oidc.audience` is only checked when the groups are encoded in the token, the mandatory `openid` in `acl.oidc.scope` is enforced by the provider rather than at startup, and `acl.oidc.ropc.flow.enabled` makes QuestDB itself exchange HTTP basic and PGWire credentials for a token at the provider - [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) - [Web Console](/docs/getting-started/web-console/overview/) - Documented query sharing by link and tab import/export in the [code editor](/docs/getting-started/web-console/code-editor/), custom AI providers and per-provider permission levels in [QuestDB AI](/docs/getting-started/web-console/questdb-ai/), automatic column sizing in the [result grid](/docs/getting-started/web-console/result-grid/), and the storage policy section in [table details](/docs/getting-started/web-console/table-details/) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 6bad18d102..0fb8daa107 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -64,6 +64,12 @@ QuestDB refuses to start when the OIDC configuration is inconsistent. With OAuth2 audience as set on the tokens issued by the OIDC Provider. Defaults to the client ID if not set. +Only used when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, +which is the only case in which QuestDB validates tokens itself. In the default +user info flow the OIDC Provider decides whether the token is valid, and +QuestDB does not check the audience at all. + ### acl.oidc.client.id - **Default**: none @@ -143,7 +149,15 @@ location where it was loaded from (`window.location.href`). - **Reloadable**: no The OIDC server asks consent for the scopes listed in this property. The -scope `openid` is mandatory and must always be included. +scope `openid` is mandatory and must always be included. That is an OIDC +protocol requirement enforced by the provider, not a QuestDB startup check: +QuestDB passes the value on without inspecting it, so leaving `openid` out +fails at the provider rather than at startup. + +QuestDB uses the scopes in the requests it makes itself, in the +[ROPC flow](#acloidcropcflowenabled), and publishes them on the +[settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which +run the flow themselves. ## Authentication flows @@ -180,6 +194,22 @@ without it. Enables or disables the Resource Owner Password Credentials flow. When enabled, this flow must also be configured in the OIDC Provider. +With it enabled QuestDB runs the flow itself: a username and password arriving +over HTTP basic authentication or PGWire that match no local user are sent on to +the OIDC Provider's token endpoint as a password grant, and the user is logged +in if the provider issues a token. This lets clients which cannot follow a +browser redirect, such as `psql`, authenticate with their SSO credentials. + +Local users are matched first, so a QuestDB user whose name also exists in the +Identity Provider is authenticated against its local password, without involving +the provider. + +Unlike [`acl.oidc.pkce.required`](#acloidcpkcerequired) and +[`acl.oidc.state.required`](#acloidcstaterequired), this setting is not +published on the +[settings endpoint](/docs/security/oidc/#settings-endpoint), so a client cannot +discover whether the flow is available. + ### acl.oidc.state.required - **Default**: `false` From 1b30683b0b90b0c078da21eb6d99e97a2dc020a0 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Mon, 24 Aug 2026 00:20:47 +0100 Subject: [PATCH 31/54] Stop the client pages naming the wrong token The three client pages which list OIDC token acquisition as unsupported tell the reader to acquire an access token out of band. That is the wrong token whenever acl.oidc.groups.encoded.in.token is set, which is what the Entra ID walkthrough requires, because Entra ID cannot serve the group memberships from its User Info endpoint. Sending the access token there fails with a 401 and the reason only in the server log. Pointing these pages at the settings endpoint gave them a link to the response which carries the answer, but not to the section which explains it, and no client page linked "Which token to send" at all. Each now links it next to the settings endpoint and asks for "a token" rather than "an access token". The Go page and the QWP ingress page had the same missing link without the wrong noun, so they get it too. Also fixes a referent: "discovering its authorization and token endpoints from QuestDB's settings endpoint" attached "its" to the IdP, while the endpoints are the ones QuestDB publishes. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/connect/clients/c-and-cpp.md | 2 +- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 4 +++- documentation/connect/clients/python.md | 2 +- documentation/connect/wire-protocols/qwp-ingress-websocket.md | 3 ++- 5 files changed, 8 insertions(+), 5 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index 3f091a5a9d..5b65faf780 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -332,7 +332,7 @@ following are **not** supported: | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...`, and rebuild the pool when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...`, and rebuild the pool when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. See the connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, close the pool and build a fresh one with the new token. | diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index d597deba9f..24286e1fa1 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -223,7 +223,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 982b340406..135aa4afc4 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -364,7 +364,9 @@ Acquire it out of band — QuestDB Enterprise issues bearer tokens through its [OpenID Connect flow](/docs/security/oidc/) — and manage its lifetime yourself. QuestDB publishes the provider's authorization and token endpoints on its [settings endpoint](/docs/security/oidc/#settings-endpoint), so a client can -discover them instead of hard coding them. +discover them instead of hard coding them. The same response tells the client +[which token to send](/docs/security/oidc/#which-token-to-send): the access +token, or the ID token when QuestDB reads group memberships from the token. When the token expires or is rotated, construct a new handle with the new token. An expired or rejected token surfaces as an authentication failure (see [Connection-level errors](#connection-level-errors)). It is mutually exclusive diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 0b275a0fcd..cfd3b1c404 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -171,7 +171,7 @@ following are **not** supported: | Path | Status | Workaround | | --- | --- | --- | -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, discovering its authorization and token endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), pass it via `token=...`, and rebuild the handle when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...`, and rebuild the handle when the token nears expiry. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. | | Token rotation mid-session | Not supported. The handle keeps the credentials it was built with and presents them on every connection it opens — including reconnects and failover, so an expired token also breaks mid-session reconnection. | On token expiry, close the handle and build a fresh one with the new token. | diff --git a/documentation/connect/wire-protocols/qwp-ingress-websocket.md b/documentation/connect/wire-protocols/qwp-ingress-websocket.md index 2711a7feb4..0d64562109 100644 --- a/documentation/connect/wire-protocols/qwp-ingress-websocket.md +++ b/documentation/connect/wire-protocols/qwp-ingress-websocket.md @@ -145,7 +145,8 @@ Supported methods: [Authentication via token in QuestDB Enterprise](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise). - **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). The [settings endpoint](/docs/security/oidc/#settings-endpoint) publishes the - provider's authorization and token endpoints. + provider's authorization and token endpoints, and tells the client + [which token to send](/docs/security/oidc/#which-token-to-send). A failed authentication results in a `401` or `403` HTTP response before the WebSocket connection is established. No QWP-level auth handshake exists. From e7bd1848c7b890f716edd91a5489049e1d191176 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Mon, 24 Aug 2026 00:20:56 +0100 Subject: [PATCH 32/54] Give the ingestion example its own client id, and guard its lookups Reading the client id and the scopes from the settings endpoint went a step too far. acl.oidc.client.id and acl.oidc.scope describe QuestDB's own registration with the provider, so the example had every ingestion job present the Web Console's OAuth2 client, against the page's own advice that each application which integrates via OIDC should be given a different Client Id. The borrowed scope is whatever QuestDB asks consent for, which under the Entra ID walkthrough is "openid profile offline_access", so the job requested a refresh token it never uses. The token endpoint stays discovered, being a server fact. Two lookups could also fail on a bare KeyError. acl.oidc.token.endpoint is absent when acl.oidc.configuration.url is set while OIDC is disabled, which is what the presence table means by "when the endpoint is resolved" and what the note above the example means by reading acl.oidc.enabled first. The example did neither, and now gates on that flag. The password grant returns an id_token only if the provider issues one for it and openid is among the requested scopes, which QuestDB does not enforce, so the example says so and fails with a diagnostic rather than a KeyError. Also replaces the manual json.loads of response.content with response.json(), and adds raise_for_status so a refused grant does not surface three lines later as a missing token. Co-Authored-By: Claude Opus 5 (1M context) --- documentation/security/oidc.mdx | 49 +++++++++++++++++++++++++-------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index cccb664148..e89ce6af7f 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -657,14 +657,19 @@ client for ingesting data. It is practical to manage their credentials via an OAuth2 provider too. As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion. The token endpoint they -post to, the client id and the scopes are all published on the -[settings endpoint](#settings-endpoint), so none of them have to be hard coded. -The same response also tells the client -[which token to send](#which-token-to-send): +themselves and then use it to authorise data ingestion. QuestDB publishes the +provider's token endpoint on the [settings endpoint](#settings-endpoint), so it +does not have to be hard coded, and the same response tells the client +[which token to send](#which-token-to-send). + +The client id and the scopes stay the client's own. `acl.oidc.client.id` and +`acl.oidc.scope` describe QuestDB's registration with the provider, not the +job's. As noted in [Architecture overview](#architecture-overview), each +application which integrates via OIDC should be given a different Client Id, so +that the two can be told apart in the provider's audit log and given different +policies. ```python -import json import os import requests import pandas as pd @@ -675,20 +680,30 @@ load_dotenv() user = os.environ.get("username") pwd = os.environ.get("password") +# this job's own registration in the OIDC Provider, not QuestDB's; +# the openid scope is what makes the provider issue an ID token +client_id = os.environ.get("client_id") +scope = "openid" + settings = requests.get("http://localhost:9000/settings").json()["config"] +if not settings.get("acl.oidc.enabled"): + raise SystemExit("OIDC is not enabled on this QuestDB instance") response = requests.post(settings["acl.oidc.token.endpoint"], data={"grant_type": "password", - "client_id": settings["acl.oidc.client.id"], + "client_id": client_id, "username": user, "password": pwd, - "scope": settings["acl.oidc.scope"]}, + "scope": scope}, headers={"Content-Type": "application/x-www-form-urlencoded"}) +response.raise_for_status() +tokens = response.json() -response_body = response.content.decode("utf-8") -tokens = json.loads(response_body) -groups_in_token = settings["acl.oidc.groups.encoded.in.token"] -token = tokens["id_token"] if groups_in_token else tokens["access_token"] +# QuestDB reads the group memberships from the ID token when this is enabled +token_key = "id_token" if settings["acl.oidc.groups.encoded.in.token"] else "access_token" +if token_key not in tokens: + raise SystemExit(f"the provider did not issue an {token_key} for the password grant") +token = tokens[token_key] conf = f"http::addr=localhost:9000;token={token};" with Sender.from_conf(conf) as sender: @@ -697,6 +712,16 @@ with Sender.from_conf(conf) as sender: sender.dataframe(df, table_name="foo", at="ts") ``` +:::note + +The Resource Owner Password Credentials flow returns an ID token only if the +provider issues one for the password grant, and only when `openid` is among the +requested scopes. Check that the provider does before relying on +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +with a non-interactive client. + +::: + Alternatively, a user may rely on QuestDB to authenticate them via the OAuth2 provider when the Resource Owner Password Credentials flow is enabled on the server side: From 1b79708f6f9a66b476cc3c9757c6777fd70ddc71 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Wed, 2 Sep 2026 16:05:01 +0100 Subject: [PATCH 33/54] Document OIDC device flow authentication --- documentation/changelog.mdx | 3 +- documentation/configuration/oidc.md | 10 +- documentation/connect/clients/c-and-cpp.md | 111 +++++++- documentation/connect/clients/java.md | 40 +++ documentation/connect/clients/python.md | 54 +++- documentation/connect/clients/rust.md | 50 ++++ documentation/security/oidc.mdx | 298 ++++++++++++++++++++- 7 files changed, 533 insertions(+), 33 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 2e0d2b86b0..433d854b70 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -42,7 +42,8 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another -- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization and token endpoints, and whether PKCE and the `state` parameter are required +- [OIDC Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients +- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 0fb8daa107..849dc66212 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -159,6 +159,11 @@ QuestDB uses the scopes in the requests it makes itself, in the [settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which run the flow themselves. +For the [Device Authorization Flow](/docs/security/oidc/#device-authorization-flow), +add `offline_access` when the provider requires that scope before issuing a +refresh token. Without a refresh token, a client must ask the user to sign in +again after the current token expires. + ## Authentication flows QuestDB publishes [`acl.oidc.pkce.required`](#acloidcpkcerequired) and @@ -247,7 +252,10 @@ OIDC Device Authorization Endpoint. Unlike the other endpoint settings this one has no default, and QuestDB never calls it. QuestDB resolves the endpoint and publishes it on the [settings endpoint](/docs/security/oidc/#settings-endpoint), for clients which -implement the Device Authorization Flow themselves. +implement the +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) +themselves. The official Java, Python, Rust, C, and C++ clients can discover and +use the published endpoint. Left unset, and absent from the provider's configuration document, the endpoint stays unresolved and the key is omitted from the settings response. diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index 5b65faf780..f2830ece6d 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -325,16 +325,111 @@ Because the pool connects lazily, a bad credential surfaces as Handle it there, not at `connect` (see [Which errors mean what](#which-errors-mean-what)). -### Unsupported auth paths +### OIDC device flow (Enterprise) -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: +The C and C++ APIs sign in an interactive user with the +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +They discover the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public `/settings` endpoint. Attach the resulting auth +object to the pool so sender and reader connections share its rotating token: -| Path | Status | Workaround | -|---|---|---| -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...`, and rebuild the pool when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. See the connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). | -| Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, close the pool and build a fresh one with the new token. | + + + +```cpp +#include +#include +#include + +int main() { + auto auth = questdb::oidc::builder::from_questdb( + "https://questdb.example.com:9000") + .event_handler([](const questdb::oidc::event_view& event) { + if (event.kind() == questdb::oidc::event_kind::prompt) + std::cerr << "Open " << event.verification_uri() + << " and enter " << event.user_code() << '\n'; + }) + .build(); + + auth.sign_in(); // the only call which may prompt or open a browser + questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; +} +``` + + + + +```c +#include +#include +#include +#include + +static void show_prompt(void *data, const questdb_oidc_event *event) { + (void)data; + if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) + fprintf(stderr, "Open %.*s and enter %.*s\n", + (int)event->verification_uri_len, event->verification_uri, + (int)event->user_code_len, event->user_code); +} + +int main(void) { + questdb_error *error = NULL; + questdb_oidc_builder *builder = NULL; + questdb_oidc_auth *auth = NULL; + questdb_db *db = NULL; + int status = 1; + const char *url = "https://questdb.example.com:9000"; + builder = questdb_oidc_builder_from_questdb(url, strlen(url), &error); + if (!builder || !questdb_oidc_builder_event_handler( + builder, show_prompt, NULL, NULL, &error)) + goto done; + + auth = questdb_oidc_builder_build(builder, &error); + if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) + goto done; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.oidc_auth = auth; + const char *conf = "wss::addr=questdb.example.com:9000;"; + db = questdb_db_connect_ex(conf, strlen(conf), &options, &error); + if (db) + status = 0; + +done: + questdb_db_close(db); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + questdb_error_free(error); + return status; +} +``` + + + + +The pool retains the auth state and gets a cached or silently refreshed token +for every connection and reconnect. Those transport operations never prompt; +when another user approval is needed, call `sign_in()` / +`questdb_oidc_auth_sign_in()` explicitly on the main or UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish its +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); +otherwise set the builder's issuer or configure the client explicitly. +Tokens stay in memory by default; see the [complete client +examples](/docs/security/oidc/#official-client-examples) for error handling, +endpoint pinning, and opt-in persistence across process restarts. + +### Other authentication limitations + +- Mutual TLS client certificates are not supported because the QuestDB server + does not negotiate them. Use bearer-token authentication over `wss`; see the + connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). +- A token supplied directly through `token=...` remains fixed. Attach an OIDC + auth object for device-flow token refresh, or rebuild the pool when an + externally acquired token rotates. ## Headers diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index 1df7d8239f..5dac1f40e2 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -397,6 +397,46 @@ The token is sent as an `Authorization: Bearer YOUR_BEARER_TOKEN` header on both the ingress and egress WebSocket upgrades. It is mutually exclusive with `username`/`password`. +### OIDC device flow (Enterprise) + +`OidcDeviceAuth` signs in an interactive user with the +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +It discovers the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public `/settings` endpoint. The user can approve the +sign-in from a browser on any device, so this also works from a container or +remote notebook kernel: + +```java +import io.questdb.client.QuestDB; +import io.questdb.client.cutlass.auth.OidcDeviceAuth; + +try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000")) { + auth.signIn(); // the only call which may prompt or open a browser + + try (QuestDB db = QuestDB.connect( + "wss::addr=questdb.example.com:9000;", + auth::getToken)) { + // Ingestion and query connections share the rotating token provider. + } +} +``` + +Pass `auth::getToken` as a provider instead of putting the current token in the +connect string. Every new connection and reconnect then receives the cached or +silently refreshed token. A transport call never starts an interactive flow; +if user approval is needed again, call `signIn()` explicitly on the main or UI +thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish its +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); +otherwise pin the provider with `DiscoveryOptions.issuer(...)` or configure the +client explicitly. +Tokens stay in memory by default; see the [OIDC guide's client +examples](/docs/security/oidc/#official-client-examples) for endpoint pinning, +explicit configuration, and opt-in persistence across process restarts. + ### HTTP basic auth ```java diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index d4487e1a0c..ea20c9aead 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -142,6 +142,45 @@ token = questdb.connect( ) ``` +### OIDC device flow (Enterprise) + +`OidcDeviceAuth` signs in an interactive user with the +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +It discovers the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public `/settings` endpoint. The default prompt works +in terminals and remote Jupyter kernels: + +```python +import questdb +from questdb.auth import OidcDeviceAuth + +with OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000") as auth: + auth.sign_in() # the only call which may prompt or open a browser + + with questdb.connect( + "wss::addr=questdb.example.com:9000;", + oidc_auth=auth) as db: + # sender(), dataframe(), and query() share the rotating token provider. + pass +``` + +`oidc_auth=auth` keeps shared ownership of the provider and obtains the cached +or silently refreshed token for every connection and reconnect. It is mutually +exclusive with a fixed `token=` setting. Transport operations never prompt; if +they raise `OidcInteractionRequired`, call `auth.sign_in()` explicitly on the +main or UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish its +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); +otherwise pass `issuer=...` or configure the client explicitly. +Tokens stay in memory by default. `FileTokenStore.at_default_location()` enables +opt-in persistence across restarts, but writes the refresh token as plaintext +protected by filesystem permissions. See the [OIDC client +examples](/docs/security/oidc/#official-client-examples) for the full lifecycle +and security considerations. + Authentication happens during the WebSocket upgrade, before any data frames are exchanged. Bad credentials raise `QuestDBErrorCode.AuthError` from the first operation that needs the connection, not from `connect()`. Queries and @@ -164,16 +203,13 @@ operating-system certificate store. Override it with configuration keys: See the [connect string reference](/docs/connect/clients/connect-string/) for the full grammar. -### Unsupported auth paths +### Other authentication limitations -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: - -| Path | Status | Workaround | -| --- | --- | --- | -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...`, and rebuild the handle when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. | -| Token rotation mid-session | Not supported. The handle keeps the credentials it was built with and presents them on every connection it opens — including reconnects and failover, so an expired token also breaks mid-session reconnection. | On token expiry, close the handle and build a fresh one with the new token. | +- Mutual TLS client certificates are not supported because the QuestDB server + does not negotiate them. Use bearer-token authentication over `wss`. +- A token supplied directly through `token=...` remains fixed. Use + `oidc_auth=...` for device-flow token refresh, or close the handle and build a + new one when an externally acquired token rotates. ## The pool diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 40ec9e4cc3..c226ded4fa 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -152,6 +152,56 @@ let token = QuestDb::connect( )?; ``` +### OIDC device flow (Enterprise) + +Enable the `oidc` feature to sign in an interactive user with the +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow): + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7", features = ["oidc"] } +``` + +`OidcDeviceAuth::from_questdb` discovers the provider endpoints, client ID, +scope, and the token QuestDB expects from the public `/settings` endpoint: + +```rust +use std::sync::Arc; +use questdb::{oidc::OidcDeviceAuth, QuestDb}; + +fn main() -> questdb::Result<()> { + let auth = Arc::new( + OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .build()?, + ); + auth.sign_in()?; // the only call which may prompt or open a browser + + let db = QuestDb::connect_with_token_provider( + "wss::addr=questdb.example.com:9000;", + { + let auth = Arc::clone(&auth); + move || auth.token() + }, + )?; + // Sender and reader connections share the rotating token provider. + Ok(()) +} +``` + +Pass the closure as a provider instead of putting the current token in the +connect string. Every new connection and reconnect then receives the cached or +silently refreshed token. A transport call never starts an interactive flow; +if it returns `OidcErrorKind::InteractionRequired`, call `sign_in()` explicitly +on the main or UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish its +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); +otherwise add `.issuer(...)` to the builder or configure the client explicitly. +Tokens stay in memory by default; see the [OIDC client +examples](/docs/security/oidc/#official-client-examples) for endpoint pinning +and opt-in persistence across process restarts. + With the default crate features, the TLS root set is `webpki_roots`. Other choices have feature requirements: diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 8021114f3e..175238bd9f 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -1,9 +1,11 @@ --- title: OpenID Connect (OIDC) Integration -description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, PGWire token authentication, group mapping, and the settings endpoint. +description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, device-flow client authentication, PGWire token authentication, and group mapping. --- import Screenshot from "@theme/Screenshot"; +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; import { EnterpriseNote } from "@site/src/components/EnterpriseNote" @@ -74,7 +76,9 @@ endpoints. The database, in OAuth2/OIDC terms the _protected resource_ or _resource server_. -Only processes requests which contain a valid access token. +Only processes requests which contain the bearer token selected by its OIDC +configuration: normally the access token, or the ID token when group memberships +are encoded in it. ## Authentication and Authorization Flow @@ -347,7 +351,8 @@ settings: "acl.oidc.redirect.uri": "https://questdb.host:9000", "acl.oidc.scope": "openid", "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", - "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2" + "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", + "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" }, "preferences.version": 0, "preferences": {} @@ -380,7 +385,8 @@ configuration document when `acl.oidc.configuration.url` is set. [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) is present when a device authorization endpoint has been configured, or when the provider advertises one. QuestDB publishes it without using it, for clients -which implement the Device Authorization Flow themselves. +which implement the [Device Authorization Flow](#device-authorization-flow) +themselves. `acl.oidc.redirect.uri` is `null` unless [`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. @@ -431,6 +437,256 @@ is at its default, as the settings path follows the Web Console context path. A client which cannot assume both are at their defaults should make the path configurable. +## Device Authorization Flow + +The [OAuth 2.0 Device Authorization Grant](https://www.rfc-editor.org/rfc/rfc8628) +signs in a person without redirecting a browser back to the client. It is a good +fit for command-line applications, containers, and remote notebook kernels: the +client displays a verification URL and short code, and the user can authorize +from a browser on any laptop or phone. + +QuestDB does not run the device flow. The client communicates directly with the +Identity Provider, then presents the resulting bearer token to QuestDB: + +```mermaid +sequenceDiagram + participant Client as QuestDB client + participant QDB as QuestDB + participant IdP as Identity Provider + participant User + Client->>QDB: GET /settings + QDB-->>Client: client ID, scope, endpoints, token mode + Client->>IdP: Request device and user codes + IdP-->>Client: device_code, user_code, verification URI + Client-->>User: Display URL and code + User->>IdP: Sign in and approve in a browser + loop Until approved or expired + Client->>IdP: Poll token endpoint + IdP-->>Client: authorization_pending / tokens + end + Client->>QDB: Authorization: Bearer selected token +``` + +The client must respect the polling interval, `slow_down` responses, and the +device code's expiry. The official clients handle those protocol details, +choose the access or ID token according to +[`acl.oidc.groups.encoded.in.token`](#which-token-to-send), cache it in memory, +and refresh it silently when the provider issues a refresh token. They never +send the device code or the user's Identity Provider password to QuestDB. + +### Configure the provider and QuestDB + +Before using the flow: + +1. Enable the device authorization grant for the public client registered with + the Identity Provider. The discovery examples below use the client ID in + `acl.oidc.client.id`; configure the client explicitly when an application + has its own registration. +2. Make the device authorization endpoint available to clients. For the + zero-configuration examples below, `acl.oidc.configuration.url` must resolve + a provider document containing `device_authorization_endpoint`, or the + host-based configuration must set + [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +3. Keep `openid` in `acl.oidc.scope`. Add `offline_access` if your provider + requires that scope before issuing a refresh token to a device-flow client. + +For example, a host-based configuration can add: + +```ini title="server.conf" +acl.oidc.scope=openid offline_access +acl.oidc.device.authorization.endpoint=/as/device_authz.oauth2 +``` + +The exact endpoint path and refresh-token scope are provider-specific. With +configuration-document discovery, do not set the endpoint property: all +individual endpoint settings are ignored in that mode. + +If QuestDB does not publish the device authorization endpoint, pin the provider +with the client's `issuer` option so the client can fetch the provider's +`/.well-known/openid-configuration` document, or configure the client ID and +both endpoints explicitly. + +### Official client examples + +Each example discovers the OIDC client ID, scope, token-selection mode, and +provider endpoints from QuestDB's [settings endpoint](#settings-endpoint), then +signs in before opening the database connection. Use an `https` QuestDB URL for +discovery so an attacker cannot replace the advertised Identity Provider +endpoints. + + + + +```java +import io.questdb.client.QuestDB; +import io.questdb.client.cutlass.auth.OidcDeviceAuth; + +try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000")) { + auth.signIn(); // the only call which may prompt or open a browser + + try (QuestDB db = QuestDB.connect( + "wss::addr=questdb.example.com:9000;", + auth::getToken)) { + // db.borrowSender() and db.borrowQuery() use the rotating token. + } +} +``` + + + + +```python +import questdb +from questdb.auth import OidcDeviceAuth + +with OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000") as auth: + auth.sign_in() # the only call which may prompt or open a browser + + with questdb.connect( + "wss::addr=questdb.example.com:9000;", + oidc_auth=auth) as db: + # db.sender(), db.dataframe(), and db.query() use the rotating token. + pass +``` + + + + +Enable the `oidc` crate feature: + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7", features = ["oidc"] } +``` + +```rust +use std::sync::Arc; +use questdb::{oidc::OidcDeviceAuth, QuestDb}; + +fn main() -> questdb::Result<()> { + let auth = Arc::new( + OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .build()?, + ); + auth.sign_in()?; // the only call which may prompt or open a browser + + let db = QuestDb::connect_with_token_provider( + "wss::addr=questdb.example.com:9000;", + { + let auth = Arc::clone(&auth); + move || auth.token() + }, + )?; + // db.borrow_sender() and db.borrow_reader() use the rotating token. + Ok(()) +} +``` + + + + +```cpp +#include +#include +#include + +int main() { + auto auth = questdb::oidc::builder::from_questdb( + "https://questdb.example.com:9000") + .event_handler([](const questdb::oidc::event_view& event) { + if (event.kind() == questdb::oidc::event_kind::prompt) + std::cerr << "Open " << event.verification_uri() + << " and enter " << event.user_code() << '\n'; + }) + .build(); + + auth.sign_in(); // the only call which may prompt or open a browser + questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; + // pool.borrow_sender() and pool.borrow_reader() use the rotating token. +} +``` + + + + +```c +#include +#include +#include +#include + +static void show_prompt(void *data, const questdb_oidc_event *event) { + (void)data; + if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) + fprintf(stderr, "Open %.*s and enter %.*s\n", + (int)event->verification_uri_len, event->verification_uri, + (int)event->user_code_len, event->user_code); +} + +int main(void) { + questdb_error *error = NULL; + questdb_oidc_auth *auth = NULL; + const char *url = "https://questdb.example.com:9000"; + questdb_oidc_builder *builder = + questdb_oidc_builder_from_questdb(url, strlen(url), &error); + if (!builder || !questdb_oidc_builder_event_handler( + builder, show_prompt, NULL, NULL, &error)) + goto fail; + + auth = questdb_oidc_builder_build(builder, &error); + if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) + goto fail; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.oidc_auth = auth; + + const char *conf = "wss::addr=questdb.example.com:9000;"; + questdb_db *db = questdb_db_connect_ex( + conf, strlen(conf), &options, &error); + if (!db) + goto fail; + + /* Sender and reader borrows use the rotating token. */ + questdb_db_close(db); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + return 0; + +fail: + if (error) { + size_t len = 0; + const char *message = questdb_error_msg(error, &len); + fprintf(stderr, "OIDC sign-in failed: %.*s\n", (int)len, message); + } + questdb_error_free(error); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + return 1; +} +``` + + + + +The authentication object owns the token state. Pass it as a rotating provider, +as shown above, instead of copying the current token into the connection string; +otherwise a reconnect after token expiry will keep sending the stale value. +Interactive sign-in belongs on the main or UI thread. Transport operations may +refresh silently, but return an interaction-required error when a new user +authorization is necessary. + +Tokens remain in memory by default. Java, Python, Rust, C, and C++ also offer an +opt-in file token store for avoiding a new prompt after process restarts. The +store contains a long-lived refresh token as plaintext protected by filesystem +permissions; enable it only when that tradeoff is acceptable. + +Device flow always requires a person to approve the sign-in. For unattended +services, scheduled jobs, or CI, use a service-account token or a provider flow +intended for machine identities instead. + ## Interactive clients Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an @@ -466,8 +722,9 @@ special version of the Authorization Code flow designed for mobile apps and single page applications. Regardless of which flow is used by the web or mobile application, the requested -access token can be used for authentication and authorization when communicating -with QuestDB as explained in the [above 7th step](#7-database-access). +token can be used for authentication and authorization when communicating with +QuestDB. Select the access or ID token as described in +[Which token to send](#which-token-to-send). ### Jupyter notebook @@ -478,11 +735,14 @@ The OAuthenticator documentation also contains [examples](https://oauthenticator.readthedocs.io/en/latest/tutorials/provider-specific-setup/index.html) using different identity providers. -If Jupyter notebooks are used without JupyterHub, one option for OAuth2 -integration is to use the -Resource Owner Password Credentials (ROPC) flow. -It is likely that enabling this flow in your OAuth2 provider will require -additional setup. +If Jupyter notebooks are used without JupyterHub, use the official Python +client's [Device Authorization Flow](#device-authorization-flow). It works when +the browser and notebook kernel are on different machines and does not put the +user's password in the notebook. + +For tooling without device-flow support, a last-resort option is the +Resource Owner Password Credentials (ROPC) flow. Enabling it usually +requires additional provider configuration. :::caution @@ -490,7 +750,13 @@ The Resource Owner Password Credentials flow is legacy, and should be used as a ::: -We can use the code below to acquire an access token in our notebook: +The ROPC snippets below assume +`acl.oidc.groups.encoded.in.token=false` and therefore send the access token. If +the setting is `true`, the client must send the ID token instead, and the +provider must issue one for the password grant. The official device-flow client +selects the correct token automatically. + +As a fallback, the code below acquires an access token with ROPC: ```python from urllib import request, parse @@ -634,8 +900,12 @@ with pg.connect(conn_str, autocommit=True) as connection: ### CLI, standalone applications -When using CLI tools, such as `psql`, or standalone applications like Microsoft -Access, the best option may be the Resource Owner Password Credentials flow. +For a CLI or standalone application built with an official Java, Rust, C, or +C++ client, use the [Device Authorization Flow](#device-authorization-flow). +Tools such as `psql` and Microsoft Access cannot run the flow themselves; for +those tools, acquire a token separately and use +[PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the +legacy Resource Owner Password Credentials flow. The user logs in with their SSO credentials, and the server validates the details with the OAuth2 provider: From f2963720c71483704b5a239010d9084579124e70 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 00:39:58 +0100 Subject: [PATCH 34/54] Fix three security gaps in the OIDC guide The ROPC example discovered the provider's token endpoint over plaintext http and then posted the user's SSO password to whatever URL came back, contradicting the rule the same page states for the device flow. Both snippets now use https against questdb.example.com, matching the device-flow examples, and a caution explains why the discovery request and the connection carrying the token both need it. "Which token to send" is the section every client page links to, but the consequence of acl.oidc.groups.encoded.in.token lived only on the configuration page and in the Entra ID walkthrough. It now carries its own caution: exp, nbf and iss go unchecked, so an expired token is still accepted and provider token lifetimes are not enforced. The validation column also named two checks where the server performs three. https for discovery was written as advice; the clients enforce it. Say so, and name the opt-out for each language. Note that Rust, C, C++ and Python accept loopback http for local development while Java rejects it, since that difference decides whether http://localhost:9000 works. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0111nSmXn2CmqkYMGobjvxgY --- documentation/security/oidc.mdx | 43 +++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 175238bd9f..0ddab5a8ca 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -405,7 +405,7 @@ by the provider belongs in the `Authorization: Bearer` header: | Value | Token to send | How QuestDB validates it | | --- | --- | --- | | `false`, the default | the access token | by calling the user info endpoint | -| `true` | the ID token | locally, by checking its signature and audience | +| `true` | the ID token | locally, by checking its signature, audience, and claims | Sending the wrong one fails authentication with `401 Unauthorized` and the reason in the server log. With @@ -415,6 +415,18 @@ straight out of its payload. The [Web Console](/docs/getting-started/web-console/overview/) picks the token this way, and so should any other client. +:::caution + +With `acl.oidc.groups.encoded.in.token` enabled, QuestDB checks the token's +signature, its `aud` claim, and that `sub` and the group memberships are +present. It does not check `exp`, `nbf` or `iss`, so an expired token is still +accepted, and the provider's token lifetimes are not enforced. Do not enable the +setting where you rely on being able to revoke a token. See +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +for the full list of what is and is not validated. + +::: + :::note Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries @@ -510,9 +522,17 @@ both endpoints explicitly. Each example discovers the OIDC client ID, scope, token-selection mode, and provider endpoints from QuestDB's [settings endpoint](#settings-endpoint), then -signs in before opening the database connection. Use an `https` QuestDB URL for -discovery so an attacker cannot replace the advertised Identity Provider -endpoints. +signs in before opening the database connection. + +The discovery URL must use `https`: reaching QuestDB over plaintext would let an +attacker replace the advertised Identity Provider endpoints and redirect the +device code and the refresh token, so the clients reject it rather than warn. +The Rust, C, C++ and Python clients accept loopback `http` for local +development; the Java client does not. To use a plaintext non-loopback host +anyway, opt in with `allowInsecureTransport(true)` (Java), +`.allow_insecure_transport(true)` (Rust), +`questdb_oidc_builder_allow_insecure_transport()` (C), +`allow_insecure_transport()` (C++) or `insecure=True` (Python). @@ -939,6 +959,15 @@ application which integrates via OIDC should be given a different Client Id, so that the two can be told apart in the provider's audit log and given different policies. +:::caution + +Both requests must go over `https`. The settings response names the URL this +code posts the user's password to, so anyone able to tamper with a plaintext +response can redirect the password grant to a host of their choosing. The same +applies to the connection below, which carries the resulting token. + +::: + ```python import os import requests @@ -955,7 +984,7 @@ pwd = os.environ.get("password") client_id = os.environ.get("client_id") scope = "openid" -settings = requests.get("http://localhost:9000/settings").json()["config"] +settings = requests.get("https://questdb.example.com:9000/settings").json()["config"] if not settings.get("acl.oidc.enabled"): raise SystemExit("OIDC is not enabled on this QuestDB instance") @@ -975,7 +1004,7 @@ if token_key not in tokens: raise SystemExit(f"the provider did not issue an {token_key} for the password grant") token = tokens[token_key] -conf = f"http::addr=localhost:9000;token={token};" +conf = f"https::addr=questdb.example.com:9000;token={token};" with Sender.from_conf(conf) as sender: df = pd.read_csv("data.csv") df["ts"] = pd.to_datetime(df["ts"]) @@ -1006,7 +1035,7 @@ load_dotenv() user = os.environ.get("username") pwd = os.environ.get("password") -conf = f"http::addr=localhost:9000;username={user};password={pwd};" +conf = f"https::addr=questdb.example.com:9000;username={user};password={pwd};" with Sender.from_conf(conf) as sender: df = pd.read_csv("data.csv") df["ts"] = pd.to_datetime(df["ts"]) From 0ad71781f8f01e7de0f0ccd02b46a92ef84d9061 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 00:49:18 +0100 Subject: [PATCH 35/54] Name the OIDC device-flow APIs the docs tell you to use The four client pages sent readers to #official-client-examples for endpoint pinning, explicit configuration, persistence and error handling. That section held five copies of the same happy path and none of those things, so the pointers dead-ended. It now carries four subsections covering them, and the pointers name what is actually there. Each subsection names the API per language rather than describing it: - The sign-in prompt. The five clients behave differently and the tabs implied they did not: Java prints to System.out and performs no terminal check, so redirecting output hides the code; Rust, C and C++ refuse without a TTY; Python defers to the IPython kernel. Records the custom-prompt and force-interactive setters, and which URL is vetted for opening. - Explicit configuration. "Configure the client explicitly" appeared on five pages and named nothing. Lists what each setting is for and where the setters live, including that Java splits them between DiscoveryOptions and builder(). Notes that acl.oidc.audience is never published, so discovery cannot supply it and a provider that needs an audience requires an explicit one. - Token persistence. FileTokenStore was named once, in prose, with no wiring. Gives the per-language constructor and attachment, the default directory and its override, and the 0600/0700 permissions, so the tradeoff the page asks the reader to accept can actually be judged. - Interaction-required errors. Java has no distinct type and Rust flattens the kind to ErrorCode::AuthError, reachable only through err.oidc_error(), so the blanket claim that transports "return an interaction-required error" was wrong for two of the five clients. Also corrects DiscoveryOptions.issuer(...) on the Java page, which reads as a static call on a top-level type and is neither, and notes on all four pages that silent refresh depends on offline_access being in acl.oidc.scope. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0111nSmXn2CmqkYMGobjvxgY --- documentation/connect/clients/c-and-cpp.md | 28 ++++-- documentation/connect/clients/java.md | 29 ++++-- documentation/connect/clients/python.md | 28 +++--- documentation/connect/clients/rust.md | 28 ++++-- documentation/security/oidc.mdx | 103 +++++++++++++++++++-- 5 files changed, 169 insertions(+), 47 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index f2830ece6d..fc53123385 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -410,17 +410,27 @@ done: The pool retains the auth state and gets a cached or silently refreshed token -for every connection and reconnect. Those transport operations never prompt; -when another user approval is needed, call `sign_in()` / -`questdb_oidc_auth_sign_in()` explicitly on the main or UI thread. +for every connection and reconnect. Silent refresh needs a refresh token, which +most providers issue only when `offline_access` is among the scopes in +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's +own `scope` override. Those transport operations never prompt; they fail with +`QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` in C, or +`questdb::error_kind::interaction_required` in C++, and the application calls +`sign_in()` / `questdb_oidc_auth_sign_in()` explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an -override, QuestDB must publish its -[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); -otherwise set the builder's issuer or configure the client explicitly. -Tokens stay in memory by default; see the [complete client -examples](/docs/security/oidc/#official-client-examples) for error handling, -endpoint pinning, and opt-in persistence across process restarts. +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise set the builder's `issuer`, or set `client_id`, `scope`, `audience`, +`token_endpoint` and `device_authorization_endpoint` yourself; in C the same +names are prefixed with `questdb_oidc_builder_`. + +Tokens stay in memory until `.default_file_token_store()` in C++, or +`questdb_oidc_builder_default_file_token_store()` in C, writes a long-lived +refresh token to disk as plaintext. See the [complete client +examples](/docs/security/oidc/#official-client-examples) for error handling, the +sign-in prompt, and the store's location and permissions. ### Other authentication limitations diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index 5dac1f40e2..cc963c1dd2 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -424,18 +424,27 @@ try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( Pass `auth::getToken` as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. A transport call never starts an interactive flow; -if user approval is needed again, call `signIn()` explicitly on the main or UI -thread. +silently refreshed token. Silent refresh needs a refresh token, which most +providers issue only when `offline_access` is among the scopes in +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's +own `scope` override. A transport call never starts an interactive flow; it +throws `OidcAuthException`, which carries no distinct interaction-required type, +so call `signIn()` again on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an -override, QuestDB must publish its -[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); -otherwise pin the provider with `DiscoveryOptions.issuer(...)` or configure the -client explicitly. -Tokens stay in memory by default; see the [OIDC guide's client -examples](/docs/security/oidc/#official-client-examples) for endpoint pinning, -explicit configuration, and opt-in persistence across process restarts. +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise pin the provider by passing +`new OidcDeviceAuth.DiscoveryOptions().issuer("https://idp")` as the second +argument to `fromQuestDB`, or set the client ID, scope and endpoints yourself +with `OidcDeviceAuth.builder()`. + +Tokens stay in memory until a store is attached with +`DiscoveryOptions.tokenStore(FileTokenStore.atDefaultLocation())`, which writes +a long-lived refresh token to disk as plaintext. See the [OIDC guide's client +examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +explicit configuration, and the token store's location and permissions. ### HTTP basic auth diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index ea20c9aead..3b78289a15 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -167,19 +167,25 @@ with OidcDeviceAuth.from_questdb( `oidc_auth=auth` keeps shared ownership of the provider and obtains the cached or silently refreshed token for every connection and reconnect. It is mutually -exclusive with a fixed `token=` setting. Transport operations never prompt; if -they raise `OidcInteractionRequired`, call `auth.sign_in()` explicitly on the -main or UI thread. +exclusive with a fixed `token=` setting. Silent refresh needs a refresh token, +which most providers issue only when `offline_access` is among the scopes in +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's +own `scope` override. Transport operations never prompt; if they raise +`OidcInteractionRequired`, importable from `questdb.auth`, call `auth.sign_in()` +explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an -override, QuestDB must publish its -[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); -otherwise pass `issuer=...` or configure the client explicitly. -Tokens stay in memory by default. `FileTokenStore.at_default_location()` enables -opt-in persistence across restarts, but writes the refresh token as plaintext -protected by filesystem permissions. See the [OIDC client -examples](/docs/security/oidc/#official-client-examples) for the full lifecycle -and security considerations. +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise pass `issuer=...`, or set `client_id=`, `scope=`, `audience=`, +`token_endpoint=` and `device_authorization_endpoint=` on `from_questdb`. + +Tokens stay in memory until a store is passed as +`token_store=FileTokenStore.at_default_location()`, which writes a long-lived +refresh token to disk as plaintext. See the [OIDC client +examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +explicit configuration, and the store's location and permissions. Authentication happens during the WebSocket upgrade, before any data frames are exchanged. Bad credentials raise `QuestDBErrorCode.AuthError` from the diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index c226ded4fa..63823bf02e 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -190,17 +190,27 @@ fn main() -> questdb::Result<()> { Pass the closure as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. A transport call never starts an interactive flow; -if it returns `OidcErrorKind::InteractionRequired`, call `sign_in()` explicitly -on the main or UI thread. +silently refreshed token. Silent refresh needs a refresh token, which most +providers issue only when `offline_access` is among the scopes in +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's +own `scope` override. A transport call never starts an interactive flow; it +returns a `questdb::Error` with `ErrorCode::AuthError`, whose `err.oidc_error()` +reports `OidcErrorKind::InteractionRequired` when a new sign-in is needed. Call +`sign_in()` explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an -override, QuestDB must publish its -[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint); -otherwise add `.issuer(...)` to the builder or configure the client explicitly. -Tokens stay in memory by default; see the [OIDC client -examples](/docs/security/oidc/#official-client-examples) for endpoint pinning -and opt-in persistence across process restarts. +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise add `.issuer(...)` to the builder, or set `.client_id()`, `.scope()`, +`.audience()`, `.token_endpoint()` and `.device_authorization_endpoint()` +yourself. + +Tokens stay in memory until +`.token_store(FileTokenStore::at_default_location()?)` is added, which writes a +long-lived refresh token to disk as plaintext. See the [OIDC client +examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +explicit configuration, and the store's location and permissions. With the default crate features, the TLS root set is `webpki_roots`. Other choices have feature requirements: diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 0ddab5a8ca..168f459ab1 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -694,19 +694,106 @@ fail: The authentication object owns the token state. Pass it as a rotating provider, as shown above, instead of copying the current token into the connection string; otherwise a reconnect after token expiry will keep sending the stale value. -Interactive sign-in belongs on the main or UI thread. Transport operations may -refresh silently, but return an interaction-required error when a new user -authorization is necessary. - -Tokens remain in memory by default. Java, Python, Rust, C, and C++ also offer an -opt-in file token store for avoiding a new prompt after process restarts. The -store contains a long-lived refresh token as plaintext protected by filesystem -permissions; enable it only when that tradeoff is acceptable. Device flow always requires a person to approve the sign-in. For unattended services, scheduled jobs, or CI, use a service-account token or a provider flow intended for machine identities instead. +#### The sign-in prompt + +Signing in renders the verification URL and the user code, then blocks until the +user approves or the device code expires. Where that text goes, and whether the +call runs at all without a terminal, differs by client: + +| Client | Default prompt | Without a terminal | +| --- | --- | --- | +| Java | prints to `System.out`, then tries to open a browser | runs anyway; the client performs no terminal check | +| Python | prints to stderr, or to the frontend inside an IPython kernel | fails, unless the kernel accepts stdin | +| Rust, C, C++ | prints to stderr | fails with an interaction-required error | + +Redirecting output therefore hides the code in Java, while a container with no +TTY fails outright in the other four. Override the default to render the +challenge elsewhere, or to declare the process interactive anyway: + +| Client | Custom prompt | Force interactive | +| --- | --- | --- | +| Java | `prompt(...)`, or `DeviceCodePrompt.SYSTEM_OUT` to skip the browser launch | not applicable | +| Python | `renderer=` | `interactive=True` | +| Rust | `.renderer(...)` | `.interactive(true)` | +| C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | +| C++ | `.event_handler(...)` | `.interactive(true)` | + +The prompt's text fields are display-safe. Where a client opens the URL or makes +it clickable, use the separately vetted browser target instead: `browser_target` +on the C event struct, `event_view::browser_target()` in C++. + +#### Explicit configuration and endpoint pinning + +Discovery supplies the client ID, scope, audience, token-selection mode, and +endpoints. Set any of them explicitly when the application has its own +registration with the provider, or when QuestDB does not publish what the client +needs: + +| Setting | Set it when | +| --- | --- | +| `client_id` | the application has its own registration, which is the recommended setup | +| `scope` | the provider requires `offline_access` before it issues a refresh token | +| `audience` | the provider requires one, or QuestDB validates `aud` against a non-default [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | +| `issuer` | QuestDB publishes no device authorization endpoint, so the client reads the provider's `/.well-known/openid-configuration` instead | +| `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | +| `groups_in_token` | overriding which of the two tokens the client selects | + +Java exposes `issuer`, `prompt`, `tokenStore`, `tlsConfig` and +`allowInsecureTransport` on `OidcDeviceAuth.DiscoveryOptions`, and the full set +on `OidcDeviceAuth.builder()`, in camel case. Python passes them as keyword +arguments to `from_questdb`. Rust and C++ are builder methods in snake case, and +C prefixes the same names with `questdb_oidc_builder_`. + +#### Token persistence + +Tokens stay in memory unless a file token store is enabled. The store writes the +access, ID and long-lived refresh tokens as unencrypted JSON, so turn it on only +where that at-rest exposure is acceptable: + +| Client | Default location | A directory you choose | +| --- | --- | --- | +| Java | `FileTokenStore.atDefaultLocation()` | `FileTokenStore.at(path)` | +| Python | `FileTokenStore.at_default_location()` | `FileTokenStore.at(path)` | +| Rust | `FileTokenStore::at_default_location()?` | `FileTokenStore::at(path)` | +| C | `questdb_oidc_builder_default_file_token_store()` | `questdb_oidc_builder_file_token_store()` | +| C++ | `.default_file_token_store()` | `.file_token_store(directory)` | + +Java, Python and Rust attach the store with `tokenStore(...)`, `token_store=` +and `.token_store(...)` respectively; the C and C++ calls above are builder +methods and attach it themselves. + +The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the +`questdb.client.oidc.token.store.dir` environment variable. Java reads that same +name as a JVM system property rather than an environment variable, and falls +back to `${user.home}/.questdb/oidc-tokens/`. On Unix the clients create token +files with mode `0600` and the store directory with `0700`; on other platforms +the directory's existing ACL governs access, so restrict it before enabling the +store. + +#### Interaction-required errors + +A transport call never prompts. When the cached token cannot be refreshed +silently, the call fails instead, and the application has to sign in again on +the main or UI thread: + +| Client | How it surfaces | +| --- | --- | +| Python | `OidcInteractionRequired`, importable from `questdb.auth` | +| Rust | a `questdb::Error` carrying `ErrorCode::AuthError`; read `err.oidc_error()` and match on `OidcErrorKind::InteractionRequired` | +| C | error kind `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` | +| C++ | `questdb::error_kind::interaction_required` | +| Java | `OidcAuthException`, with no distinct interaction-required type to match on | + +In C++ a failure raised through an attached sender arrives as +`questdb::ingress::line_sender_error` carrying `oidc_diagnostic()`, not as +`questdb::oidc::error`, so catch the common base `const questdb::error&` to +handle both. + ## Interactive clients Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an From c366b4bec52d8522b324d9c4ac10049fb5f09358 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 01:02:00 +0100 Subject: [PATCH 36/54] Reconcile OIDC across the pages that contradict it Six places where the docs disagreed with themselves or with the server. Auth enumerations that predate OIDC now include it. security/rbac.md counted three authentication methods and connect/compatibility/rest-api.md counted two; both are canonical pages that other pages defer to, and both omitted the credential this PR documents. rest-api.md also now says an OIDC bearer token is a different credential from the REST API token it describes below, since the two share the Authorization: Bearer header and are easy to confuse. connect/clients/connect-string.md is where the client pages send readers for the full credential grammar, and it listed only username, password and token. It now records that device-flow credentials are not connect-string keys: the auth object rides alongside, and rotates, where a static token= does not. high-availability/failover.md told operators to send "an OIDC access token". Under acl.oidc.groups.encoded.in.token, which the Entra ID walkthrough in this same PR sets to true, that token is rejected with a 401. It now names both tokens and links the rule. The Client Id rule had three answers. The architecture overview and the non-interactive section both say each application should have its own; the device-flow examples reuse QuestDB's without saying so. The examples now say what discovery costs, and step 1 states the rule once and points at the explicit-configuration section instead of gesturing at it. Go and .NET readers could reach the settings endpoint and no further: the sections they landed on pointed back at each other, and neither page linked the one section describing a flow they could implement. Both now do, and name a library that implements the device grant. Finally, the token-flow validator deserializes the payload into a fixed struct with sub and groups, so it requires those literal names and rejects a token without them whatever acl.oidc.sub.claim and acl.oidc.groups.claim say. Custom names only ever applied to the user info flow. Both option entries now say so. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0111nSmXn2CmqkYMGobjvxgY --- documentation/configuration/oidc.md | 12 +++++++++++ .../connect/clients/connect-string.md | 6 ++++++ documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 6 ++++++ .../connect/compatibility/rest-api.md | 7 ++++++- documentation/high-availability/failover.md | 4 +++- documentation/security/oidc.mdx | 15 +++++++++---- documentation/security/rbac.md | 21 ++++++++++++------- 8 files changed, 59 insertions(+), 14 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 849dc66212..c0b6e30085 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -356,6 +356,10 @@ If the claim is missing from the user information, or it is an empty list, authentication fails. See [Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). +A custom name applies to the user info flow. Set this to `groups` when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, +because the token validator requires that name. + ### acl.oidc.groups.encoded.in.token - **Default**: `false` @@ -382,6 +386,14 @@ the token itself and never asks the provider about it: | `aud`, against [`acl.oidc.audience`](#acloidcaudience) | `nbf`, the not-before time | | that `sub` and the group memberships are present | `iss`, the issuer | +The validator reads those two claims under the names `sub` and `groups` +literally, and rejects a token which does not carry both, whatever +[`acl.oidc.sub.claim`](#acloidcsubclaim) and +[`acl.oidc.groups.claim`](#acloidcgroupsclaim) are set to. `groups` must be an +array of strings. Custom claim names apply to the user info flow, so with this +setting enabled leave `acl.oidc.sub.claim` at its default and set +`acl.oidc.groups.claim=groups`. + :::caution An expired token is therefore still accepted. Once issued, a token stays valid diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index c027700dd5..4283cee237 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -220,6 +220,12 @@ WebSocket upgrade request. exclusive with `username` / `password`. Token auth avoids the per-request overhead of basic auth and is the recommended path for Enterprise deployments. +- Device-flow OIDC credentials are **not** connect-string keys. The Java, + Python, Rust, C and C++ clients take an auth object alongside the connect + string, which rotates the token on every reconnect; see + [OIDC device flow](/docs/security/oidc/#device-authorization-flow). A + static `token=` does not rotate, so a reconnect after expiry keeps sending + the stale value. - `auth_timeout_ms` — per-host upper bound on the upgrade response read. Does not cover TLS handshake or post-upgrade frame reads, which use OS or hard-coded defaults. Default: `15000` (15 s). diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index cc07628164..c36dada8bf 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 2c552d8bee..086781b576 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -367,6 +367,12 @@ QuestDB publishes the provider's authorization and token endpoints on its discover them instead of hard coding them. The same response tells the client [which token to send](/docs/security/oidc/#which-token-to-send): the access token, or the ID token when QuestDB reads group memberships from the token. +This client does not run an OIDC flow, so acquire the token with an OAuth2 +library: `golang.org/x/oauth2` implements the device grant, and QuestDB +publishes the device authorization endpoint on the same settings response. The +[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) +describes the protocol, including the polling interval, `slow_down`, and the +device code's expiry. When the token expires or is rotated, construct a new handle with the new token. An expired or rejected token surfaces as an authentication failure (see [Connection-level errors](#connection-level-errors)). It is mutually exclusive diff --git a/documentation/connect/compatibility/rest-api.md b/documentation/connect/compatibility/rest-api.md index 033060ff2c..c1e6a1b573 100644 --- a/documentation/connect/compatibility/rest-api.md +++ b/documentation/connect/compatibility/rest-api.md @@ -794,11 +794,16 @@ A HTTP status code of `400` is returned with the following response body: ## Authentication -The REST API supports two authentication types: +The REST API supports three authentication types: - **HTTP basic authentication**, available in QuestDB Open Source and QuestDB Enterprise. - **Token-based authentication**, available in QuestDB Enterprise only. +- **OIDC bearer tokens**, available in QuestDB Enterprise only. These also + travel in the `Authorization: Bearer` header but are issued by an external + Identity Provider rather than by QuestDB, so they are a different credential + from the REST API token below. See [OpenID Connect](/docs/security/oidc/) and + [which token to send](/docs/security/oidc/#which-token-to-send). :::note diff --git a/documentation/high-availability/failover.md b/documentation/high-availability/failover.md index 93f58855d3..67345bc76d 100644 --- a/documentation/high-availability/failover.md +++ b/documentation/high-availability/failover.md @@ -284,7 +284,9 @@ The [minimal HTTP server](/docs/operations/logging-metrics/#minimal-http-server) on port 9003 exposes the same switch to external coordinators. It accepts the same credentials as the main HTTP server: HTTP basic authentication, a [REST token](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise) -as `Authorization: Bearer`, or an OIDC access token. The principal needs the +as `Authorization: Bearer`, or an OIDC token — [the access token or the ID +token](/docs/security/oidc/#which-token-to-send) depending on +`acl.oidc.groups.encoded.in.token`. The principal needs the `HTTP` endpoint permission and, for the switch, `SWITCH ROLE`. With access control disabled no credentials are needed. TLS for this port is configured with the `http.min.tls.*` settings on the [TLS](/docs/configuration/tls/) page. diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 168f459ab1..085b88423a 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -490,10 +490,12 @@ send the device code or the user's Identity Provider password to QuestDB. Before using the flow: -1. Enable the device authorization grant for the public client registered with - the Identity Provider. The discovery examples below use the client ID in - `acl.oidc.client.id`; configure the client explicitly when an application - has its own registration. +1. Register the application as a public client with the Identity Provider and + enable the device authorization grant on it. Each application which + integrates via OIDC should have its own Client Id, so that it can be told + apart from QuestDB in the provider's audit log and given its own policies. + Pass that Client Id to the client explicitly, as described under + [Explicit configuration and endpoint pinning](#explicit-configuration-and-endpoint-pinning). 2. Make the device authorization endpoint available to clients. For the zero-configuration examples below, `acl.oidc.configuration.url` must resolve a provider document containing `device_authorization_endpoint`, or the @@ -524,6 +526,11 @@ Each example discovers the OIDC client ID, scope, token-selection mode, and provider endpoints from QuestDB's [settings endpoint](#settings-endpoint), then signs in before opening the database connection. +Discovering the client ID means reusing QuestDB's own registration, which keeps +the examples short but gives the application no separate identity at the +provider. That is fine for evaluation; in production register the application +and set its Client Id explicitly, as described below. + The discovery URL must use `https`: reaching QuestDB over plaintext would let an attacker replace the advertised Identity Provider endpoints and redirect the device code and the refresh token, so the clients reject it rather than warn. diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md index 5b10f3c266..77c14eaaf7 100644 --- a/documentation/security/rbac.md +++ b/documentation/security/rbac.md @@ -314,13 +314,20 @@ dropped, all members lose the permissions they inherited from that group. width={745} /> -QuestDB supports three authentication methods: - -| Method | Use case | Endpoints | -| ------------------ | ------------------------ | ------------------------- | -| **Password** | Interactive users | REST API, PostgreSQL Wire | -| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | -| **REST API Token** | Programmatic REST access | REST API | +QuestDB supports four authentication methods: + +| Method | Use case | Endpoints | +| --------------------- | --------------------------- | ------------------------------ | +| **Password** | Interactive users | REST API, PostgreSQL Wire | +| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | +| **REST API Token** | Programmatic REST access | REST API | +| **OIDC bearer token** | SSO users, external clients | REST API, PostgreSQL Wire, QWP | + +The first three are QuestDB's own credentials, created with the statements +below. An OIDC bearer token is issued by an external Identity Provider instead, +and the user's group memberships come from the token or the provider's user +info endpoint; see [OpenID Connect](/docs/security/oidc/) and +[which token to send](/docs/security/oidc/#which-token-to-send). Users can have multiple authentication methods enabled simultaneously: From 81131f9d213f40c7fae151796617d77e5c8f7746 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 01:11:27 +0100 Subject: [PATCH 37/54] Give the OIDC device flow its own page and single-source its examples The device flow was 350 lines with no sidebar path: reachable only by landing on the OIDC guide and scrolling to entry 16 of 32. It is now security/oidc-device-flow.mdx with its own sidebar entry, and the guide keeps a short section at #device-authorization-flow so external links still resolve. Every internal link points at the new page directly rather than through that stub. Extracting it also settles the ordering: the flow no longer sits 250 lines above the "Interactive clients" taxonomy that motivates it, and the Jupyter and CLI sections simply link out. The five examples move to partials under documentation/partials/ and are imported by both the new page and the client page for that language. They were duplicated before, and the C copy had already drifted: the guide used goto fail and printed the error, the client page used goto done and discarded it. One copy now, so they cannot diverge again. Each example also does some work. All five stopped at connect(), so the rotating token was never seen carrying a request; they now write a row to trades and flush, using the same schema and idiom as each page's quick start. The C++ example gains the qwp_reader.hpp include. Its comment advertised pool.borrow_reader(), which does not compile against the sender header alone because the return type is incomplete there. Which token to send is promoted to a top-level section. Nine pages link to it as the canonical answer for the Authorization header, and it sat as an h3 under Settings endpoint, where a reader arriving from a client page landed mid-way through a JSON response contract. The two paragraphs that really do belong to the settings endpoint - the note on reading acl.oidc.enabled first, and the http.context.settings path - move back above it. Finally, the OIDC subsection on the Python and Rust client pages was placed before the parent section's own content, so it swallowed the general TLS root-store tables and, in Python, the WebSocket-upgrade auth prose. Both now sit where the C and C++ page already put theirs. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0111nSmXn2CmqkYMGobjvxgY --- documentation/changelog.mdx | 2 +- documentation/configuration/oidc.md | 4 +- documentation/connect/clients/c-and-cpp.md | 73 +--- .../connect/clients/connect-string.md | 2 +- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 2 +- documentation/connect/clients/java.md | 21 +- documentation/connect/clients/python.md | 65 ++- documentation/connect/clients/rust.md | 51 +-- .../partials/_oidc.device-flow.c.partial.mdx | 70 +++ .../_oidc.device-flow.cpp.partial.mdx | 33 ++ .../_oidc.device-flow.java.partial.mdx | 24 ++ .../_oidc.device-flow.python.partial.mdx | 22 + .../_oidc.device-flow.rust.partial.mdx | 37 ++ documentation/security/oidc-device-flow.mdx | 246 +++++++++++ documentation/security/oidc.mdx | 408 ++---------------- documentation/sidebars.js | 5 + 17 files changed, 530 insertions(+), 537 deletions(-) create mode 100644 documentation/partials/_oidc.device-flow.c.partial.mdx create mode 100644 documentation/partials/_oidc.device-flow.cpp.partial.mdx create mode 100644 documentation/partials/_oidc.device-flow.java.partial.mdx create mode 100644 documentation/partials/_oidc.device-flow.python.partial.mdx create mode 100644 documentation/partials/_oidc.device-flow.rust.partial.mdx create mode 100644 documentation/security/oidc-device-flow.mdx diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 433d854b70..4502d0dc19 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -42,7 +42,7 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another -- [OIDC Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients +- [OIDC Device Authorization Flow](/docs/security/oidc-device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients - [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index c0b6e30085..08cc01049d 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -159,7 +159,7 @@ QuestDB uses the scopes in the requests it makes itself, in the [settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which run the flow themselves. -For the [Device Authorization Flow](/docs/security/oidc/#device-authorization-flow), +For the [Device Authorization Flow](/docs/security/oidc-device-flow/), add `offline_access` when the provider requires that scope before issuing a refresh token. Without a refresh token, a client must ask the user to sign in again after the current token expires. @@ -253,7 +253,7 @@ has no default, and QuestDB never calls it. QuestDB resolves the endpoint and publishes it on the [settings endpoint](/docs/security/oidc/#settings-endpoint), for clients which implement the -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) +[Device Authorization Flow](/docs/security/oidc-device-flow/) themselves. The official Java, Python, Rust, C, and C++ clients can discover and use the published endpoint. diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index fc53123385..ea98fcd735 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -7,6 +7,8 @@ description: "QuestDB C and C++ client: the questdb_db / questdb::pool connectio import Tabs from "@theme/Tabs" import TabItem from "@theme/TabItem" +import OidcDeviceFlowCpp from "../../partials/_oidc.device-flow.cpp.partial.mdx"; +import OidcDeviceFlowC from "../../partials/_oidc.device-flow.c.partial.mdx"; The C and C++ clients ingest and query over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/), a columnar binary @@ -328,7 +330,7 @@ Handle it there, not at `connect` (see ### OIDC device flow (Enterprise) The C and C++ APIs sign in an interactive user with the -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +[Device Authorization Flow](/docs/security/oidc-device-flow/). They discover the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. Attach the resulting auth object to the pool so sender and reader connections share its rotating token: @@ -336,75 +338,12 @@ object to the pool so sender and reader connections share its rotating token: -```cpp -#include -#include -#include - -int main() { - auto auth = questdb::oidc::builder::from_questdb( - "https://questdb.example.com:9000") - .event_handler([](const questdb::oidc::event_view& event) { - if (event.kind() == questdb::oidc::event_kind::prompt) - std::cerr << "Open " << event.verification_uri() - << " and enter " << event.user_code() << '\n'; - }) - .build(); - - auth.sign_in(); // the only call which may prompt or open a browser - questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; -} -``` + -```c -#include -#include -#include -#include - -static void show_prompt(void *data, const questdb_oidc_event *event) { - (void)data; - if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) - fprintf(stderr, "Open %.*s and enter %.*s\n", - (int)event->verification_uri_len, event->verification_uri, - (int)event->user_code_len, event->user_code); -} - -int main(void) { - questdb_error *error = NULL; - questdb_oidc_builder *builder = NULL; - questdb_oidc_auth *auth = NULL; - questdb_db *db = NULL; - int status = 1; - const char *url = "https://questdb.example.com:9000"; - builder = questdb_oidc_builder_from_questdb(url, strlen(url), &error); - if (!builder || !questdb_oidc_builder_event_handler( - builder, show_prompt, NULL, NULL, &error)) - goto done; - - auth = questdb_oidc_builder_build(builder, &error); - if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) - goto done; - - questdb_db_connect_options options; - questdb_db_connect_options_init(&options, sizeof options); - options.oidc_auth = auth; - const char *conf = "wss::addr=questdb.example.com:9000;"; - db = questdb_db_connect_ex(conf, strlen(conf), &options, &error); - if (db) - status = 0; - -done: - questdb_db_close(db); - questdb_oidc_auth_free(auth); - questdb_oidc_builder_free(builder); - questdb_error_free(error); - return status; -} -``` + @@ -429,7 +368,7 @@ names are prefixed with `questdb_oidc_builder_`. Tokens stay in memory until `.default_file_token_store()` in C++, or `questdb_oidc_builder_default_file_token_store()` in C, writes a long-lived refresh token to disk as plaintext. See the [complete client -examples](/docs/security/oidc/#official-client-examples) for error handling, the +examples](/docs/security/oidc-device-flow/#official-client-examples) for error handling, the sign-in prompt, and the store's location and permissions. ### Other authentication limitations diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index 4283cee237..9e0d5babe6 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -223,7 +223,7 @@ WebSocket upgrade request. - Device-flow OIDC credentials are **not** connect-string keys. The Java, Python, Rust, C and C++ clients take an auth object alongside the connect string, which rotates the token on every reconnect; see - [OIDC device flow](/docs/security/oidc/#device-authorization-flow). A + [OIDC device flow](/docs/security/oidc-device-flow/). A static `token=` does not rotate, so a reconnect after expiry keeps sending the stale value. - `auth_timeout_ms` — per-host upper bound on the upgrade response read. diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index c36dada8bf..76df3b4781 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [Device Authorization Flow](/docs/security/oidc-device-flow/) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 086781b576..b66b2748c5 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -370,7 +370,7 @@ token, or the ID token when QuestDB reads group memberships from the token. This client does not run an OIDC flow, so acquire the token with an OAuth2 library: `golang.org/x/oauth2` implements the device grant, and QuestDB publishes the device authorization endpoint on the same settings response. The -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow) +[Device Authorization Flow](/docs/security/oidc-device-flow/) describes the protocol, including the polling interval, `slow_down`, and the device code's expiry. When the token expires or is rotated, construct a new handle with the new token. diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index cc963c1dd2..dfb9932c92 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -12,6 +12,7 @@ import TabItem from "@theme/TabItem" import SfDedupWarning from "../../partials/_sf-dedup-warning.partial.mdx" import CodeBlock from "@theme/CodeBlock" +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.java.partial.mdx"; :::note @@ -400,27 +401,13 @@ both the ingress and egress WebSocket upgrades. It is mutually exclusive with ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +[Device Authorization Flow](/docs/security/oidc-device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. The user can approve the sign-in from a browser on any device, so this also works from a container or remote notebook kernel: -```java -import io.questdb.client.QuestDB; -import io.questdb.client.cutlass.auth.OidcDeviceAuth; - -try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( - "https://questdb.example.com:9000")) { - auth.signIn(); // the only call which may prompt or open a browser - - try (QuestDB db = QuestDB.connect( - "wss::addr=questdb.example.com:9000;", - auth::getToken)) { - // Ingestion and query connections share the rotating token provider. - } -} -``` + Pass `auth::getToken` as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or @@ -443,7 +430,7 @@ with `OidcDeviceAuth.builder()`. Tokens stay in memory until a store is attached with `DiscoveryOptions.tokenStore(FileTokenStore.atDefaultLocation())`, which writes a long-lived refresh token to disk as plaintext. See the [OIDC guide's client -examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, explicit configuration, and the token store's location and permissions. ### HTTP basic auth diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 3b78289a15..e5b0358fe5 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -5,6 +5,8 @@ sidebar_label: Python description: "Use the QuestDB Python client's QuestDB pool for row, DataFrame, and Arrow ingestion plus SQL queries over QWP." --- +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.python.partial.mdx"; + The QuestDB Python client uses one `QuestDB` handle for ingestion and SQL queries over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/). Lease a short-lived sender for each unit of row-building work, bulk-load DataFrames through the handle, and run SQL with `query()`. @@ -142,28 +144,37 @@ token = questdb.connect( ) ``` +Authentication happens during the WebSocket upgrade, before any data frames +are exchanged. Bad credentials raise `QuestDBErrorCode.AuthError` from the +first operation that needs the connection, not from `connect()`. Queries and +`dataframe()` calls raise it directly. Row senders connect in the +background, so a fire-and-forget `flush()` can return before the upgrade +fails; the error then surfaces from `flush(wait=True)`, `wait()`, or the +next call on the lease — and immediately through the +[connection listener](#failover-and-errors) as an `AuthFailed` event. + +With `wss`, the default combines the bundled `webpki` root store with the +operating-system certificate store. Override it with configuration keys: + +| Setting | Meaning | +| --- | --- | +| `tls_ca=os_roots` | Use only the operating-system certificate store. | +| `tls_ca=webpki_roots` | Use only the bundled `webpki` root store. | +| `tls_roots=/path/to/ca.pem` | Use a private CA bundle. | +| `tls_verify=unsafe_off` | Rejected: certificate verification cannot be disabled in released wheels. | + +See the [connect string reference](/docs/connect/clients/connect-string/) for +the full grammar. + ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow). +[Device Authorization Flow](/docs/security/oidc-device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. The default prompt works in terminals and remote Jupyter kernels: -```python -import questdb -from questdb.auth import OidcDeviceAuth - -with OidcDeviceAuth.from_questdb( - "https://questdb.example.com:9000") as auth: - auth.sign_in() # the only call which may prompt or open a browser - - with questdb.connect( - "wss::addr=questdb.example.com:9000;", - oidc_auth=auth) as db: - # sender(), dataframe(), and query() share the rotating token provider. - pass -``` + `oidc_auth=auth` keeps shared ownership of the provider and obtains the cached or silently refreshed token for every connection and reconnect. It is mutually @@ -184,31 +195,9 @@ Otherwise pass `issuer=...`, or set `client_id=`, `scope=`, `audience=`, Tokens stay in memory until a store is passed as `token_store=FileTokenStore.at_default_location()`, which writes a long-lived refresh token to disk as plaintext. See the [OIDC client -examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, explicit configuration, and the store's location and permissions. -Authentication happens during the WebSocket upgrade, before any data frames -are exchanged. Bad credentials raise `QuestDBErrorCode.AuthError` from the -first operation that needs the connection, not from `connect()`. Queries and -`dataframe()` calls raise it directly. Row senders connect in the -background, so a fire-and-forget `flush()` can return before the upgrade -fails; the error then surfaces from `flush(wait=True)`, `wait()`, or the -next call on the lease — and immediately through the -[connection listener](#failover-and-errors) as an `AuthFailed` event. - -With `wss`, the default combines the bundled `webpki` root store with the -operating-system certificate store. Override it with configuration keys: - -| Setting | Meaning | -| --- | --- | -| `tls_ca=os_roots` | Use only the operating-system certificate store. | -| `tls_ca=webpki_roots` | Use only the bundled `webpki` root store. | -| `tls_roots=/path/to/ca.pem` | Use a private CA bundle. | -| `tls_verify=unsafe_off` | Rejected: certificate verification cannot be disabled in released wheels. | - -See the [connect string reference](/docs/connect/clients/connect-string/) for -the full grammar. - ### Other authentication limitations - Mutual TLS client certificates are not supported because the QuestDB server diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 63823bf02e..fa467b4267 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -5,6 +5,8 @@ sidebar_label: Rust description: "Use the QuestDB Rust connection pool for Buffer, Chunk, Arrow, and Polars ingestion plus streaming SQL queries over QWP." --- +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.rust.partial.mdx"; + The QuestDB Rust client uses a thread-safe `QuestDb` pool for ingestion and SQL queries over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/). Borrow a short-lived writer or reader for each unit of work, then let `Drop` return its @@ -152,10 +154,21 @@ let token = QuestDb::connect( )?; ``` +With the default crate features, the TLS root set is `webpki_roots`. Other +choices have feature requirements: + +| Setting | Requirement | +| --- | --- | +| `tls_ca=os_roots` | Enable `tls-native-certs`. | +| `tls_ca=webpki_and_os_roots` | Enable both `tls-webpki-certs` and `tls-native-certs`. | +| `tls_roots=/path/to/roots.pem` | Uses the supplied PEM bundle and implies `tls_ca=pem_file`. | +| `tls_roots_password=...` | Unlocks a JKS or PKCS#12 store named by `tls_roots`. | +| `tls_verify=unsafe_off` | Enable `insecure-skip-verify`; use only in controlled tests. | + ### OIDC device flow (Enterprise) Enable the `oidc` feature to sign in an interactive user with the -[Device Authorization Flow](/docs/security/oidc/#device-authorization-flow): +[Device Authorization Flow](/docs/security/oidc-device-flow/): ```toml title="Cargo.toml" [dependencies] @@ -165,28 +178,7 @@ questdb-rs = { version = "7", features = ["oidc"] } `OidcDeviceAuth::from_questdb` discovers the provider endpoints, client ID, scope, and the token QuestDB expects from the public `/settings` endpoint: -```rust -use std::sync::Arc; -use questdb::{oidc::OidcDeviceAuth, QuestDb}; - -fn main() -> questdb::Result<()> { - let auth = Arc::new( - OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") - .build()?, - ); - auth.sign_in()?; // the only call which may prompt or open a browser - - let db = QuestDb::connect_with_token_provider( - "wss::addr=questdb.example.com:9000;", - { - let auth = Arc::clone(&auth); - move || auth.token() - }, - )?; - // Sender and reader connections share the rotating token provider. - Ok(()) -} -``` + Pass the closure as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or @@ -209,20 +201,9 @@ yourself. Tokens stay in memory until `.token_store(FileTokenStore::at_default_location()?)` is added, which writes a long-lived refresh token to disk as plaintext. See the [OIDC client -examples](/docs/security/oidc/#official-client-examples) for the sign-in prompt, +examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, explicit configuration, and the store's location and permissions. -With the default crate features, the TLS root set is `webpki_roots`. Other -choices have feature requirements: - -| Setting | Requirement | -| --- | --- | -| `tls_ca=os_roots` | Enable `tls-native-certs`. | -| `tls_ca=webpki_and_os_roots` | Enable both `tls-webpki-certs` and `tls-native-certs`. | -| `tls_roots=/path/to/roots.pem` | Uses the supplied PEM bundle and implies `tls_ca=pem_file`. | -| `tls_roots_password=...` | Unlocks a JKS or PKCS#12 store named by `tls_roots`. | -| `tls_verify=unsafe_off` | Enable `insecure-skip-verify`; use only in controlled tests. | - ## The pool `QuestDb` owns reusable QWP/WebSocket connections. Create one pool per diff --git a/documentation/partials/_oidc.device-flow.c.partial.mdx b/documentation/partials/_oidc.device-flow.c.partial.mdx new file mode 100644 index 0000000000..de8dfd2381 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.c.partial.mdx @@ -0,0 +1,70 @@ +```c +#include +#include // questdb_db_borrow_sender + line_sender_buffer +#include +#include +#include + +static void show_prompt(void *data, const questdb_oidc_event *event) { + (void)data; + if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) + fprintf(stderr, "Open %.*s and enter %.*s\n", + (int)event->verification_uri_len, event->verification_uri, + (int)event->user_code_len, event->user_code); +} + +int main(void) { + questdb_error *error = NULL; + questdb_oidc_builder *builder = NULL; + questdb_oidc_auth *auth = NULL; + questdb_db *db = NULL; + qwp_sender *sender = NULL; + line_sender_buffer *buffer = NULL; + int status = 1; + + const char *url = "https://questdb.example.com:9000"; + builder = questdb_oidc_builder_from_questdb(url, strlen(url), &error); + if (!builder || !questdb_oidc_builder_event_handler( + builder, show_prompt, NULL, NULL, &error)) + goto done; + + auth = questdb_oidc_builder_build(builder, &error); + if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) + goto done; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.oidc_auth = auth; + + const char *conf = "wss::addr=questdb.example.com:9000;"; + db = questdb_db_connect_ex(conf, strlen(conf), &options, &error); + if (!db) goto done; + + /* Sender and reader borrows use the rotating token. */ + sender = questdb_db_borrow_sender(db, &error); + if (!sender) goto done; + buffer = questdb_db_new_buffer(db, &error); + if (!buffer) goto done; + if (!line_sender_buffer_table(buffer, QDB_TABLE_NAME_LITERAL("trades"), &error)) goto done; + if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("symbol"), + QDB_UTF8_LITERAL("ETH-USD"), &error)) goto done; + if (!line_sender_buffer_column_f64(buffer, QDB_COLUMN_NAME_LITERAL("price"), 2615.54, &error)) goto done; + if (!line_sender_buffer_at_nanos(buffer, line_sender_now_nanos(), &error)) goto done; + if (!qwp_sender_flush_buffer_and_wait(sender, buffer, qwpws_ack_level_ok, &error)) goto done; + status = 0; + +done: + if (error) { + size_t len = 0; + const char *message = questdb_error_msg(error, &len); + fprintf(stderr, "OIDC device flow failed: %.*s\n", (int)len, message); + questdb_error_free(error); + } + line_sender_buffer_free(buffer); + if (sender) questdb_db_return_sender(db, sender); + questdb_db_close(db); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + return status; +} +``` diff --git a/documentation/partials/_oidc.device-flow.cpp.partial.mdx b/documentation/partials/_oidc.device-flow.cpp.partial.mdx new file mode 100644 index 0000000000..cdc84f0d69 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.cpp.partial.mdx @@ -0,0 +1,33 @@ +```cpp +#include // pool::borrow_sender +#include // pool::borrow_reader +#include +#include + +using namespace questdb::ingress::literals; + +int main() { + auto auth = questdb::oidc::builder::from_questdb( + "https://questdb.example.com:9000") + .event_handler([](const questdb::oidc::event_view& event) { + if (event.kind() == questdb::oidc::event_kind::prompt) + std::cerr << "Open " << event.verification_uri() + << " and enter " << event.user_code() << '\n'; + }) + .build(); + + auth.sign_in(); // the only call which may prompt or open a browser + questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; + + auto sender = pool.borrow_sender(); + auto buffer = sender.new_buffer(); + buffer.table("trades"_tn) + .symbol("symbol"_cn, "ETH-USD"_utf8) + .symbol("side"_cn, "sell"_utf8) + .column("price"_cn, 2615.54) + .column("amount"_cn, 0.00044) + .at(questdb::ingress::timestamp_nanos::now()); + sender.flush_and_wait(buffer); + // pool.borrow_reader() uses the same rotating token. +} +``` diff --git a/documentation/partials/_oidc.device-flow.java.partial.mdx b/documentation/partials/_oidc.device-flow.java.partial.mdx new file mode 100644 index 0000000000..361ad24f43 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.java.partial.mdx @@ -0,0 +1,24 @@ +```java +import io.questdb.client.QuestDB; +import io.questdb.client.Sender; +import io.questdb.client.cutlass.auth.OidcDeviceAuth; + +try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000")) { + auth.signIn(); // the only call which may prompt or open a browser + + try (QuestDB db = QuestDB.connect( + "wss::addr=questdb.example.com:9000;", + auth::getToken)) { + try (Sender sender = db.borrowSender()) { + sender.table("trades") + .symbol("symbol", "ETH-USD") + .symbol("side", "sell") + .doubleColumn("price", 2615.54) + .doubleColumn("amount", 0.00044) + .atNow(); + } + // db.borrowQuery() uses the same rotating token. + } +} +``` diff --git a/documentation/partials/_oidc.device-flow.python.partial.mdx b/documentation/partials/_oidc.device-flow.python.partial.mdx new file mode 100644 index 0000000000..dc06df19bb --- /dev/null +++ b/documentation/partials/_oidc.device-flow.python.partial.mdx @@ -0,0 +1,22 @@ +```python +import questdb +from questdb import TimestampNanos +from questdb.auth import OidcDeviceAuth + +with OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000") as auth: + auth.sign_in() # the only call which may prompt or open a browser + + with questdb.connect( + "wss::addr=questdb.example.com:9000;", + oidc_auth=auth) as db: + with db.sender() as sender: + sender.row( + "trades", + symbols={"symbol": "ETH-USD", "side": "sell"}, + columns={"price": 2615.54, "amount": 0.00044}, + at=TimestampNanos.now(), + ) + sender.flush(wait=True) + # db.dataframe() and db.query() use the same rotating token. +``` diff --git a/documentation/partials/_oidc.device-flow.rust.partial.mdx b/documentation/partials/_oidc.device-flow.rust.partial.mdx new file mode 100644 index 0000000000..b333dfc372 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.rust.partial.mdx @@ -0,0 +1,37 @@ +```rust +use std::sync::Arc; +use questdb::{ + ingress::{AckLevel, TimestampNanos}, + oidc::OidcDeviceAuth, + QuestDb, +}; + +fn main() -> questdb::Result<()> { + let auth = Arc::new( + OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .build()?, + ); + auth.sign_in()?; // the only call which may prompt or open a browser + + let db = QuestDb::connect_with_token_provider( + "wss::addr=questdb.example.com:9000;", + { + let auth = Arc::clone(&auth); + move || auth.token() + }, + )?; + + let mut sender = db.borrow_sender()?; + let mut buffer = sender.new_buffer(); + buffer + .table("trades")? + .symbol("symbol", "ETH-USD")? + .symbol("side", "sell")? + .column_f64("price", 2615.54)? + .column_f64("amount", 0.00044)? + .at(TimestampNanos::now())?; + sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok)?; + // db.borrow_reader() uses the same rotating token. + Ok(()) +} +``` diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx new file mode 100644 index 0000000000..5b1b4d25d1 --- /dev/null +++ b/documentation/security/oidc-device-flow.mdx @@ -0,0 +1,246 @@ +--- +title: OIDC Device Authorization Flow +description: Sign a user in to QuestDB Enterprise from a command-line application, container, or remote notebook kernel with the OAuth 2.0 device authorization grant, with worked client examples. +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; +import JavaExample from "../partials/_oidc.device-flow.java.partial.mdx"; +import PythonExample from "../partials/_oidc.device-flow.python.partial.mdx"; +import RustExample from "../partials/_oidc.device-flow.rust.partial.mdx"; +import CppExample from "../partials/_oidc.device-flow.cpp.partial.mdx"; +import CExample from "../partials/_oidc.device-flow.c.partial.mdx"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + The Device Authorization Flow signs in an interactive user from a client that + cannot receive a browser redirect. + + +The [OAuth 2.0 Device Authorization Grant](https://www.rfc-editor.org/rfc/rfc8628) +signs in a person without redirecting a browser back to the client. It is a good +fit for command-line applications, containers, and remote notebook kernels: the +client displays a verification URL and short code, and the user can authorize +from a browser on any laptop or phone. + +QuestDB does not run the device flow. The client communicates directly with the +Identity Provider, then presents the resulting bearer token to QuestDB: + +```mermaid +sequenceDiagram + participant Client as QuestDB client + participant QDB as QuestDB + participant IdP as Identity Provider + participant User + Client->>QDB: GET /settings + QDB-->>Client: client ID, scope, endpoints, token mode + Client->>IdP: Request device and user codes + IdP-->>Client: device_code, user_code, verification URI + Client-->>User: Display URL and code + User->>IdP: Sign in and approve in a browser + loop Until approved or expired + Client->>IdP: Poll token endpoint + IdP-->>Client: authorization_pending / tokens + end + Client->>QDB: Authorization: Bearer selected token +``` + +The client must respect the polling interval, `slow_down` responses, and the +device code's expiry. The official clients handle those protocol details, +choose the access or ID token according to +[`acl.oidc.groups.encoded.in.token`](/docs/security/oidc/#which-token-to-send), cache it in memory, +and refresh it silently when the provider issues a refresh token. They never +send the device code or the user's Identity Provider password to QuestDB. + +## Configure the provider and QuestDB + +Before using the flow: + +1. Register the application as a public client with the Identity Provider and + enable the device authorization grant on it. Each application which + integrates via OIDC should have its own Client Id, so that it can be told + apart from QuestDB in the provider's audit log and given its own policies. + Pass that Client Id to the client explicitly, as described under + [Explicit configuration and endpoint pinning](#explicit-configuration-and-endpoint-pinning). +2. Make the device authorization endpoint available to clients. For the + zero-configuration examples below, `acl.oidc.configuration.url` must resolve + a provider document containing `device_authorization_endpoint`, or the + host-based configuration must set + [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +3. Keep `openid` in `acl.oidc.scope`. Add `offline_access` if your provider + requires that scope before issuing a refresh token to a device-flow client. + +For example, a host-based configuration can add: + +```ini title="server.conf" +acl.oidc.scope=openid offline_access +acl.oidc.device.authorization.endpoint=/as/device_authz.oauth2 +``` + +The exact endpoint path and refresh-token scope are provider-specific. With +configuration-document discovery, do not set the endpoint property: all +individual endpoint settings are ignored in that mode. + +If QuestDB does not publish the device authorization endpoint, pin the provider +with the client's `issuer` option so the client can fetch the provider's +`/.well-known/openid-configuration` document, or configure the client ID and +both endpoints explicitly. + +## Official client examples + +Each example discovers the OIDC client ID, scope, token-selection mode, and +provider endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), then +signs in before opening the database connection. + +Discovering the client ID means reusing QuestDB's own registration, which keeps +the examples short but gives the application no separate identity at the +provider. That is fine for evaluation; in production register the application +and set its Client Id explicitly, as described below. + +The discovery URL must use `https`: reaching QuestDB over plaintext would let an +attacker replace the advertised Identity Provider endpoints and redirect the +device code and the refresh token, so the clients reject it rather than warn. +The Rust, C, C++ and Python clients accept loopback `http` for local +development; the Java client does not. To use a plaintext non-loopback host +anyway, opt in with `allowInsecureTransport(true)` (Java), +`.allow_insecure_transport(true)` (Rust), +`questdb_oidc_builder_allow_insecure_transport()` (C), +`allow_insecure_transport()` (C++) or `insecure=True` (Python). + + + + + + + + + + + + + + +Enable the `oidc` crate feature: + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7", features = ["oidc"] } +``` + + + + + + + + + + + + + + + + +The authentication object owns the token state. Pass it as a rotating provider, +as shown above, instead of copying the current token into the connection string; +otherwise a reconnect after token expiry will keep sending the stale value. + +Device flow always requires a person to approve the sign-in. For unattended +services, scheduled jobs, or CI, use a service-account token or a provider flow +intended for machine identities instead. + +### The sign-in prompt + +Signing in renders the verification URL and the user code, then blocks until the +user approves or the device code expires. Where that text goes, and whether the +call runs at all without a terminal, differs by client: + +| Client | Default prompt | Without a terminal | +| --- | --- | --- | +| Java | prints to `System.out`, then tries to open a browser | runs anyway; the client performs no terminal check | +| Python | prints to stderr, or to the frontend inside an IPython kernel | fails, unless the kernel accepts stdin | +| Rust, C, C++ | prints to stderr | fails with an interaction-required error | + +Redirecting output therefore hides the code in Java, while a container with no +TTY fails outright in the other four. Override the default to render the +challenge elsewhere, or to declare the process interactive anyway: + +| Client | Custom prompt | Force interactive | +| --- | --- | --- | +| Java | `prompt(...)`, or `DeviceCodePrompt.SYSTEM_OUT` to skip the browser launch | not applicable | +| Python | `renderer=` | `interactive=True` | +| Rust | `.renderer(...)` | `.interactive(true)` | +| C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | +| C++ | `.event_handler(...)` | `.interactive(true)` | + +The prompt's text fields are display-safe. Where a client opens the URL or makes +it clickable, use the separately vetted browser target instead: `browser_target` +on the C event struct, `event_view::browser_target()` in C++. + +### Explicit configuration and endpoint pinning + +Discovery supplies the client ID, scope, audience, token-selection mode, and +endpoints. Set any of them explicitly when the application has its own +registration with the provider, or when QuestDB does not publish what the client +needs: + +| Setting | Set it when | +| --- | --- | +| `client_id` | the application has its own registration, which is the recommended setup | +| `scope` | the provider requires `offline_access` before it issues a refresh token | +| `audience` | the provider requires one, or QuestDB validates `aud` against a non-default [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | +| `issuer` | QuestDB publishes no device authorization endpoint, so the client reads the provider's `/.well-known/openid-configuration` instead | +| `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | +| `groups_in_token` | overriding which of the two tokens the client selects | + +Java exposes `issuer`, `prompt`, `tokenStore`, `tlsConfig` and +`allowInsecureTransport` on `OidcDeviceAuth.DiscoveryOptions`, and the full set +on `OidcDeviceAuth.builder()`, in camel case. Python passes them as keyword +arguments to `from_questdb`. Rust and C++ are builder methods in snake case, and +C prefixes the same names with `questdb_oidc_builder_`. + +### Token persistence + +Tokens stay in memory unless a file token store is enabled. The store writes the +access, ID and long-lived refresh tokens as unencrypted JSON, so turn it on only +where that at-rest exposure is acceptable: + +| Client | Default location | A directory you choose | +| --- | --- | --- | +| Java | `FileTokenStore.atDefaultLocation()` | `FileTokenStore.at(path)` | +| Python | `FileTokenStore.at_default_location()` | `FileTokenStore.at(path)` | +| Rust | `FileTokenStore::at_default_location()?` | `FileTokenStore::at(path)` | +| C | `questdb_oidc_builder_default_file_token_store()` | `questdb_oidc_builder_file_token_store()` | +| C++ | `.default_file_token_store()` | `.file_token_store(directory)` | + +Java, Python and Rust attach the store with `tokenStore(...)`, `token_store=` +and `.token_store(...)` respectively; the C and C++ calls above are builder +methods and attach it themselves. + +The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the +`questdb.client.oidc.token.store.dir` environment variable. Java reads that same +name as a JVM system property rather than an environment variable, and falls +back to `${user.home}/.questdb/oidc-tokens/`. On Unix the clients create token +files with mode `0600` and the store directory with `0700`; on other platforms +the directory's existing ACL governs access, so restrict it before enabling the +store. + +### Interaction-required errors + +A transport call never prompts. When the cached token cannot be refreshed +silently, the call fails instead, and the application has to sign in again on +the main or UI thread: + +| Client | How it surfaces | +| --- | --- | +| Python | `OidcInteractionRequired`, importable from `questdb.auth` | +| Rust | a `questdb::Error` carrying `ErrorCode::AuthError`; read `err.oidc_error()` and match on `OidcErrorKind::InteractionRequired` | +| C | error kind `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` | +| C++ | `questdb::error_kind::interaction_required` | +| Java | `OidcAuthException`, with no distinct interaction-required type to match on | + +In C++ a failure raised through an attached sender arrives as +`questdb::ingress::line_sender_error` carrying `oidc_diagnostic()`, not as +`questdb::oidc::error`, so catch the common base `const questdb::error&` to +handle both. diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 085b88423a..719947d1c2 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -385,7 +385,7 @@ configuration document when `acl.oidc.configuration.url` is set. [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) is present when a device authorization endpoint has been configured, or when the provider advertises one. QuestDB publishes it without using it, for clients -which implement the [Device Authorization Flow](#device-authorization-flow) +which implement the [Device Authorization Flow](/docs/security/oidc-device-flow/) themselves. `acl.oidc.redirect.uri` is `null` unless @@ -397,7 +397,29 @@ enforces neither. The client generates the code verifier and the `state` value; the provider checks the verifier, and the client checks the `state` value it gets back. -### Which token to send +:::note + +Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries +are published whether or not OIDC is enabled, and with OIDC disabled the +endpoint URLs are built from the defaults rather than from a real provider. + +OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be +`true`, which the same response carries. When access control is disabled, +`acl.oidc.enabled` still reports the configured value while no OIDC +authentication takes place, so read both keys. + +::: + +The path of the endpoint is set by +[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). +Setting it adds paths rather than moving the endpoint, so the default path keeps +working too. That default is `/settings` only while +[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) +is at its default, as the settings path follows the Web Console context path. +A client which cannot assume both are at their defaults should make the path +configurable. + +## Which token to send `acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned by the provider belongs in the `Authorization: Bearer` header: @@ -427,379 +449,17 @@ for the full list of what is and is not validated. ::: -:::note - -Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries -are published whether or not OIDC is enabled, and with OIDC disabled the -endpoint URLs are built from the defaults rather than from a real provider. - -OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be -`true`, which the same response carries. When access control is disabled, -`acl.oidc.enabled` still reports the configured value while no OIDC -authentication takes place, so read both keys. - -::: - -The path of the endpoint is set by -[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). -Setting it adds paths rather than moving the endpoint, so the default path keeps -working too. That default is `/settings` only while -[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) -is at its default, as the settings path follows the Web Console context path. -A client which cannot assume both are at their defaults should make the path -configurable. - ## Device Authorization Flow -The [OAuth 2.0 Device Authorization Grant](https://www.rfc-editor.org/rfc/rfc8628) -signs in a person without redirecting a browser back to the client. It is a good -fit for command-line applications, containers, and remote notebook kernels: the -client displays a verification URL and short code, and the user can authorize -from a browser on any laptop or phone. - -QuestDB does not run the device flow. The client communicates directly with the -Identity Provider, then presents the resulting bearer token to QuestDB: - -```mermaid -sequenceDiagram - participant Client as QuestDB client - participant QDB as QuestDB - participant IdP as Identity Provider - participant User - Client->>QDB: GET /settings - QDB-->>Client: client ID, scope, endpoints, token mode - Client->>IdP: Request device and user codes - IdP-->>Client: device_code, user_code, verification URI - Client-->>User: Display URL and code - User->>IdP: Sign in and approve in a browser - loop Until approved or expired - Client->>IdP: Poll token endpoint - IdP-->>Client: authorization_pending / tokens - end - Client->>QDB: Authorization: Bearer selected token -``` - -The client must respect the polling interval, `slow_down` responses, and the -device code's expiry. The official clients handle those protocol details, -choose the access or ID token according to -[`acl.oidc.groups.encoded.in.token`](#which-token-to-send), cache it in memory, -and refresh it silently when the provider issues a refresh token. They never -send the device code or the user's Identity Provider password to QuestDB. - -### Configure the provider and QuestDB - -Before using the flow: - -1. Register the application as a public client with the Identity Provider and - enable the device authorization grant on it. Each application which - integrates via OIDC should have its own Client Id, so that it can be told - apart from QuestDB in the provider's audit log and given its own policies. - Pass that Client Id to the client explicitly, as described under - [Explicit configuration and endpoint pinning](#explicit-configuration-and-endpoint-pinning). -2. Make the device authorization endpoint available to clients. For the - zero-configuration examples below, `acl.oidc.configuration.url` must resolve - a provider document containing `device_authorization_endpoint`, or the - host-based configuration must set - [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). -3. Keep `openid` in `acl.oidc.scope`. Add `offline_access` if your provider - requires that scope before issuing a refresh token to a device-flow client. - -For example, a host-based configuration can add: - -```ini title="server.conf" -acl.oidc.scope=openid offline_access -acl.oidc.device.authorization.endpoint=/as/device_authz.oauth2 -``` - -The exact endpoint path and refresh-token scope are provider-specific. With -configuration-document discovery, do not set the endpoint property: all -individual endpoint settings are ignored in that mode. - -If QuestDB does not publish the device authorization endpoint, pin the provider -with the client's `issuer` option so the client can fetch the provider's -`/.well-known/openid-configuration` document, or configure the client ID and -both endpoints explicitly. - -### Official client examples - -Each example discovers the OIDC client ID, scope, token-selection mode, and -provider endpoints from QuestDB's [settings endpoint](#settings-endpoint), then -signs in before opening the database connection. - -Discovering the client ID means reusing QuestDB's own registration, which keeps -the examples short but gives the application no separate identity at the -provider. That is fine for evaluation; in production register the application -and set its Client Id explicitly, as described below. - -The discovery URL must use `https`: reaching QuestDB over plaintext would let an -attacker replace the advertised Identity Provider endpoints and redirect the -device code and the refresh token, so the clients reject it rather than warn. -The Rust, C, C++ and Python clients accept loopback `http` for local -development; the Java client does not. To use a plaintext non-loopback host -anyway, opt in with `allowInsecureTransport(true)` (Java), -`.allow_insecure_transport(true)` (Rust), -`questdb_oidc_builder_allow_insecure_transport()` (C), -`allow_insecure_transport()` (C++) or `insecure=True` (Python). - - - - -```java -import io.questdb.client.QuestDB; -import io.questdb.client.cutlass.auth.OidcDeviceAuth; - -try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( - "https://questdb.example.com:9000")) { - auth.signIn(); // the only call which may prompt or open a browser - - try (QuestDB db = QuestDB.connect( - "wss::addr=questdb.example.com:9000;", - auth::getToken)) { - // db.borrowSender() and db.borrowQuery() use the rotating token. - } -} -``` - - - - -```python -import questdb -from questdb.auth import OidcDeviceAuth - -with OidcDeviceAuth.from_questdb( - "https://questdb.example.com:9000") as auth: - auth.sign_in() # the only call which may prompt or open a browser - - with questdb.connect( - "wss::addr=questdb.example.com:9000;", - oidc_auth=auth) as db: - # db.sender(), db.dataframe(), and db.query() use the rotating token. - pass -``` - - - - -Enable the `oidc` crate feature: - -```toml title="Cargo.toml" -[dependencies] -questdb-rs = { version = "7", features = ["oidc"] } -``` - -```rust -use std::sync::Arc; -use questdb::{oidc::OidcDeviceAuth, QuestDb}; - -fn main() -> questdb::Result<()> { - let auth = Arc::new( - OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") - .build()?, - ); - auth.sign_in()?; // the only call which may prompt or open a browser - - let db = QuestDb::connect_with_token_provider( - "wss::addr=questdb.example.com:9000;", - { - let auth = Arc::clone(&auth); - move || auth.token() - }, - )?; - // db.borrow_sender() and db.borrow_reader() use the rotating token. - Ok(()) -} -``` - - - - -```cpp -#include -#include -#include - -int main() { - auto auth = questdb::oidc::builder::from_questdb( - "https://questdb.example.com:9000") - .event_handler([](const questdb::oidc::event_view& event) { - if (event.kind() == questdb::oidc::event_kind::prompt) - std::cerr << "Open " << event.verification_uri() - << " and enter " << event.user_code() << '\n'; - }) - .build(); - - auth.sign_in(); // the only call which may prompt or open a browser - questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; - // pool.borrow_sender() and pool.borrow_reader() use the rotating token. -} -``` - - - - -```c -#include -#include -#include -#include - -static void show_prompt(void *data, const questdb_oidc_event *event) { - (void)data; - if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) - fprintf(stderr, "Open %.*s and enter %.*s\n", - (int)event->verification_uri_len, event->verification_uri, - (int)event->user_code_len, event->user_code); -} - -int main(void) { - questdb_error *error = NULL; - questdb_oidc_auth *auth = NULL; - const char *url = "https://questdb.example.com:9000"; - questdb_oidc_builder *builder = - questdb_oidc_builder_from_questdb(url, strlen(url), &error); - if (!builder || !questdb_oidc_builder_event_handler( - builder, show_prompt, NULL, NULL, &error)) - goto fail; - - auth = questdb_oidc_builder_build(builder, &error); - if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) - goto fail; - - questdb_db_connect_options options; - questdb_db_connect_options_init(&options, sizeof options); - options.oidc_auth = auth; - - const char *conf = "wss::addr=questdb.example.com:9000;"; - questdb_db *db = questdb_db_connect_ex( - conf, strlen(conf), &options, &error); - if (!db) - goto fail; - - /* Sender and reader borrows use the rotating token. */ - questdb_db_close(db); - questdb_oidc_auth_free(auth); - questdb_oidc_builder_free(builder); - return 0; - -fail: - if (error) { - size_t len = 0; - const char *message = questdb_error_msg(error, &len); - fprintf(stderr, "OIDC sign-in failed: %.*s\n", (int)len, message); - } - questdb_error_free(error); - questdb_oidc_auth_free(auth); - questdb_oidc_builder_free(builder); - return 1; -} -``` - - - - -The authentication object owns the token state. Pass it as a rotating provider, -as shown above, instead of copying the current token into the connection string; -otherwise a reconnect after token expiry will keep sending the stale value. - -Device flow always requires a person to approve the sign-in. For unattended -services, scheduled jobs, or CI, use a service-account token or a provider flow -intended for machine identities instead. - -#### The sign-in prompt - -Signing in renders the verification URL and the user code, then blocks until the -user approves or the device code expires. Where that text goes, and whether the -call runs at all without a terminal, differs by client: +For command-line applications, containers, and remote notebook kernels, where +no browser redirect can reach the client, QuestDB Enterprise supports the +OAuth 2.0 Device Authorization Grant. QuestDB does not run the flow: the client +talks to the Identity Provider directly and presents the resulting bearer +token. -| Client | Default prompt | Without a terminal | -| --- | --- | --- | -| Java | prints to `System.out`, then tries to open a browser | runs anyway; the client performs no terminal check | -| Python | prints to stderr, or to the frontend inside an IPython kernel | fails, unless the kernel accepts stdin | -| Rust, C, C++ | prints to stderr | fails with an interaction-required error | - -Redirecting output therefore hides the code in Java, while a container with no -TTY fails outright in the other four. Override the default to render the -challenge elsewhere, or to declare the process interactive anyway: - -| Client | Custom prompt | Force interactive | -| --- | --- | --- | -| Java | `prompt(...)`, or `DeviceCodePrompt.SYSTEM_OUT` to skip the browser launch | not applicable | -| Python | `renderer=` | `interactive=True` | -| Rust | `.renderer(...)` | `.interactive(true)` | -| C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | -| C++ | `.event_handler(...)` | `.interactive(true)` | - -The prompt's text fields are display-safe. Where a client opens the URL or makes -it clickable, use the separately vetted browser target instead: `browser_target` -on the C event struct, `event_view::browser_target()` in C++. - -#### Explicit configuration and endpoint pinning - -Discovery supplies the client ID, scope, audience, token-selection mode, and -endpoints. Set any of them explicitly when the application has its own -registration with the provider, or when QuestDB does not publish what the client -needs: - -| Setting | Set it when | -| --- | --- | -| `client_id` | the application has its own registration, which is the recommended setup | -| `scope` | the provider requires `offline_access` before it issues a refresh token | -| `audience` | the provider requires one, or QuestDB validates `aud` against a non-default [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | -| `issuer` | QuestDB publishes no device authorization endpoint, so the client reads the provider's `/.well-known/openid-configuration` instead | -| `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | -| `groups_in_token` | overriding which of the two tokens the client selects | - -Java exposes `issuer`, `prompt`, `tokenStore`, `tlsConfig` and -`allowInsecureTransport` on `OidcDeviceAuth.DiscoveryOptions`, and the full set -on `OidcDeviceAuth.builder()`, in camel case. Python passes them as keyword -arguments to `from_questdb`. Rust and C++ are builder methods in snake case, and -C prefixes the same names with `questdb_oidc_builder_`. - -#### Token persistence - -Tokens stay in memory unless a file token store is enabled. The store writes the -access, ID and long-lived refresh tokens as unencrypted JSON, so turn it on only -where that at-rest exposure is acceptable: - -| Client | Default location | A directory you choose | -| --- | --- | --- | -| Java | `FileTokenStore.atDefaultLocation()` | `FileTokenStore.at(path)` | -| Python | `FileTokenStore.at_default_location()` | `FileTokenStore.at(path)` | -| Rust | `FileTokenStore::at_default_location()?` | `FileTokenStore::at(path)` | -| C | `questdb_oidc_builder_default_file_token_store()` | `questdb_oidc_builder_file_token_store()` | -| C++ | `.default_file_token_store()` | `.file_token_store(directory)` | - -Java, Python and Rust attach the store with `tokenStore(...)`, `token_store=` -and `.token_store(...)` respectively; the C and C++ calls above are builder -methods and attach it themselves. - -The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the -`questdb.client.oidc.token.store.dir` environment variable. Java reads that same -name as a JVM system property rather than an environment variable, and falls -back to `${user.home}/.questdb/oidc-tokens/`. On Unix the clients create token -files with mode `0600` and the store directory with `0700`; on other platforms -the directory's existing ACL governs access, so restrict it before enabling the -store. - -#### Interaction-required errors - -A transport call never prompts. When the cached token cannot be refreshed -silently, the call fails instead, and the application has to sign in again on -the main or UI thread: - -| Client | How it surfaces | -| --- | --- | -| Python | `OidcInteractionRequired`, importable from `questdb.auth` | -| Rust | a `questdb::Error` carrying `ErrorCode::AuthError`; read `err.oidc_error()` and match on `OidcErrorKind::InteractionRequired` | -| C | error kind `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` | -| C++ | `questdb::error_kind::interaction_required` | -| Java | `OidcAuthException`, with no distinct interaction-required type to match on | - -In C++ a failure raised through an attached sender arrives as -`questdb::ingress::line_sender_error` carrying `oidc_diagnostic()`, not as -`questdb::oidc::error`, so catch the common base `const questdb::error&` to -handle both. +See [OIDC Device Authorization Flow](/docs/security/oidc-device-flow/) for the +protocol walkthrough, what to configure first, and worked examples for the +Java, Python, Rust, C and C++ clients. ## Interactive clients @@ -850,7 +510,7 @@ The OAuthenticator documentation also contains using different identity providers. If Jupyter notebooks are used without JupyterHub, use the official Python -client's [Device Authorization Flow](#device-authorization-flow). It works when +client's [Device Authorization Flow](/docs/security/oidc-device-flow/). It works when the browser and notebook kernel are on different machines and does not put the user's password in the notebook. @@ -1015,7 +675,7 @@ with pg.connect(conn_str, autocommit=True) as connection: ### CLI, standalone applications For a CLI or standalone application built with an official Java, Rust, C, or -C++ client, use the [Device Authorization Flow](#device-authorization-flow). +C++ client, use the [Device Authorization Flow](/docs/security/oidc-device-flow/). Tools such as `psql` and Microsoft Access cannot run the flow themselves; for those tools, acquire a token separately and use [PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the diff --git a/documentation/sidebars.js b/documentation/sidebars.js index e0a4b89577..57525fb832 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -723,6 +723,11 @@ module.exports = { type: "doc", label: "OpenID Connect (OIDC)", }, + { + id: "security/oidc-device-flow", + type: "doc", + label: "OIDC device flow", + }, { type: "doc", id: "security/tls", From f3cb07c1caf3e1a22110864631a92780fc6c0dc7 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 01:17:49 +0100 Subject: [PATCH 38/54] Correct the remaining OIDC inaccuracies and inconsistencies Factual corrections, each checked against the server or the Web Console: - acl.oidc.enabled=false does not take the settings out of play. The configuration URL is still parsed and its scheme still validated, so a malformed value fails startup with OIDC switched off. - With acl.oidc.configuration.url set and OIDC disabled, the endpoint keys are absent from the settings response rather than built from the defaults. Only the host-configured case defaults them. - The settings example and its key table omitted acl.basic.auth.realm.enabled, which the server emits between acl.enabled and acl.oidc.enabled. Both now match SettingsEndpointTest key for key and in order. - The Web Console falls back to the location it was loaded from without the query string or fragment, not window.location.href. - Step 7 of the walkthrough was generalised to either token, but the sentence about the user info cache still said "the access token". - The CLI section listed Java, Rust, C and C++ and omitted Python, which two other lists on the same page and the configuration reference include. Consistency: - The Rust crate-features table gained the oidc row it documents elsewhere. - http.context.settings said its default is /settings, four paragraphs above the prose explaining that this holds only while http.context.web.console is at its default. Qualified inline, where a config scraper will read it. - The two sidebar entries both rendered "OpenID Connect (OIDC)". They are now the guide, the device flow, and the settings reference. - nodejs.md was the one client page with no OIDC route, and its Enterprise auth link led to a page that does not mention OIDC. - The Implicit flow is marked deprecated: it cannot use PKCE, which the settings endpoint advertises as required by default. - Three same-page jumps were written as third-person cross-page references to "the OIDC operations document", which is the page the reader is on. - The two non-interactive examples used the legacy ILP Sender over http while the Python client page teaches questdb.connect over wss. - server.conf blocks were fenced as shell in three places and ini in another. - The QuestDB host placeholder was questdb.host in the older examples and questdb.example.com in the new ones; the query example also used port 9999. Nits left alone deliberately: "Device Authorization Flow" keeps its title case, matching the sibling "Authentication and Authorization Flow" and RFC 8628's own term, and renaming it would break the anchor the extraction preserved. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0111nSmXn2CmqkYMGobjvxgY --- documentation/configuration/http-server.md | 3 +- documentation/configuration/oidc.md | 12 ++- documentation/connect/clients/nodejs.md | 7 ++ documentation/connect/clients/rust.md | 1 + documentation/security/oidc-device-flow.mdx | 2 +- documentation/security/oidc.mdx | 83 ++++++++++++--------- documentation/sidebars.js | 2 +- 7 files changed, 66 insertions(+), 44 deletions(-) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index fec0b1848d..76e968d7a6 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -515,7 +515,8 @@ Context path for the file import service. ### http.context.settings -- **Default**: `/settings` +- **Default**: `/settings`, relative to + [`http.context.web.console`](#httpcontextwebconsole) - **Reloadable**: no Context path for the service that serves server-side settings to clients, such diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 08cc01049d..20820b5c1a 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -1,5 +1,6 @@ --- title: OpenID Connect (OIDC) +sidebar_label: OIDC settings description: "QuestDB Enterprise acl.oidc.* settings reference: minimum configuration and startup rules, endpoints, TLS, user and group claims, caching and buffers." --- @@ -23,13 +24,15 @@ which is the default. With access control disabled no OIDC authentication takes place, but the OIDC settings are still validated at startup: with `acl.oidc.enabled=true` the server enforces every rule below and downloads the provider's configuration document, so an inconsistent OIDC configuration still -prevents it from starting. Set `acl.oidc.enabled=false` to take the settings out -of play entirely. +prevents it from starting. Set `acl.oidc.enabled=false` to stop the rules below being +enforced. `acl.oidc.configuration.url` is still parsed even then, and its +scheme must still match [`acl.oidc.tls.enabled`](#acloidctlsenabled), so a +malformed value there fails startup with OIDC switched off. A working setup against a Ping Identity provider needs four settings. Every other setting has a usable default: -```shell +```ini title="server.conf" acl.oidc.enabled=true acl.oidc.host=oidc.provider acl.oidc.client.id=questdb @@ -141,7 +144,8 @@ set, because the port is taken from the discovered endpoint URLs. The redirect URI tells the OIDC server where to redirect the user after successful authentication. If not set, the Web Console defaults it to the -location where it was loaded from (`window.location.href`). +location it was loaded from, without the query string or fragment +(`window.location.origin + window.location.pathname`). ### acl.oidc.scope diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index de60fe5610..0df1db8f1a 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -91,6 +91,13 @@ When using QuestDB Enterprise, authentication can also be done via REST token. Please check the [RBAC docs](/docs/security/rbac/#authentication) for more info. +This client does not run an OIDC flow. QuestDB Enterprise also accepts an OIDC +bearer token, which you acquire out of band from your Identity Provider: +QuestDB publishes the provider's endpoints on its +[settings endpoint](/docs/security/oidc/#settings-endpoint), and the same +response tells you [which token to send](/docs/security/oidc/#which-token-to-send). +See [OpenID Connect](/docs/security/oidc/). + ## Basic insert Example: inserting executed trades for cryptocurrencies. diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index fa467b4267..f97eaa9ddc 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -948,6 +948,7 @@ integrations only when your application uses them: | `ndarray` | No | `Buffer::column_arr` from `ndarray` views. | | `rust_decimal` / `bigdecimal` | No | Row-buffer decimal values from those crates. Decimal strings need neither feature. | | `chrono-timestamp` | No | Timestamp values built from `chrono::DateTime`. | +| `oidc` | No | Interactive OIDC sign-in with the [device flow](/docs/security/oidc-device-flow/). Pulls in `sync-sender-http`, and needs a TLS root source for `https` discovery. | | `tls-native-certs` | No | TLS validation through the operating-system certificate store. | | `insecure-skip-verify` | No | `tls_verify=unsafe_off` for controlled testing only. | | `almost-all-features` | No | Client development and testing with most compatible features. It excludes Arrow and Polars. | diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 5b1b4d25d1..091acede13 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -36,7 +36,7 @@ sequenceDiagram QDB-->>Client: client ID, scope, endpoints, token mode Client->>IdP: Request device and user codes IdP-->>Client: device_code, user_code, verification URI - Client-->>User: Display URL and code + Client->>User: Display URL and code User->>IdP: Sign in and approve in a browser loop Until approved or expired Client->>IdP: Poll token endpoint diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 719947d1c2..1d76f1f6e5 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -125,7 +125,7 @@ been authenticated already: Identity Provider for authentication. ```bash title="Authorization code request example" -https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.host%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 +https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 ``` :::note @@ -191,7 +191,7 @@ The Authorization Server redirects the user back to the [Web Console](/docs/getting-started/web-console/overview/) with the _authorization code_: ```bash title="Authorization code response example" -https://questdb.host:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A +https://questdb.example.com:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A ``` ### 5. Credential request @@ -213,7 +213,7 @@ which requested the authorization code, and it was not stolen: ```bash title="Token request example" POST https://oidc.provider:443/as/token.oauth2 HTTP/1.1 Content-Type: application/x-www-form-urlencoded -grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.host%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF +grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF ``` ### 6. Credentials received @@ -248,20 +248,24 @@ A token is in the header of every request sent to QuestDB. Which of the two goes there depends on where QuestDB reads the group memberships from, see [Which token to send](#which-token-to-send). -> **Worried about exposing the token?** The access token is rather opaque and -> does not contain user details. The ID token does: it is a JWT whose payload is -> base64 encoded, not encrypted, so treat it as you would the user's directory -> record. +:::note + +Worried about exposing the token? An access token is usually opaque and carries +no user details, though some providers issue a JWT here too. An ID token always +does: it is a JWT whose payload is base64 encoded, not encrypted, so treat it as +you would the user's directory record. + +::: To carry out permission checks, the database has to know more about the user. For this, QuestDB has a User Info Cache. -If it finds a valid entry with the access token in the cache, steps 8 and 9 are +If it finds a valid entry for the token in the cache, steps 8 and 9 are skipped: ```bash title="Query request example" -https://questdb.host:9999/exec?query=select%20current_user() +https://questdb.example.com:9000/exec?query=select%20current_user() Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom ``` @@ -333,7 +337,7 @@ order to start an authentication flow. Any client can read them from the settings endpoint, which requires no authentication: ```bash title="Settings request example" -curl https://questdb.host:9000/settings +curl https://questdb.example.com:9000/settings ``` The OIDC-related entries of the response look like this, alongside other server @@ -343,12 +347,13 @@ settings: { "config": { "acl.enabled": true, + "acl.basic.auth.realm.enabled": false, "acl.oidc.enabled": true, "acl.oidc.pkce.required": true, "acl.oidc.state.required": false, "acl.oidc.groups.encoded.in.token": false, "acl.oidc.client.id": "questdb", - "acl.oidc.redirect.uri": "https://questdb.host:9000", + "acl.oidc.redirect.uri": "https://questdb.example.com:9000", "acl.oidc.scope": "openid", "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", @@ -368,6 +373,7 @@ hard coding the provider's details. | Key | Type | Present | | --- | --- | --- | | `acl.enabled` | boolean | always | +| `acl.basic.auth.realm.enabled` | boolean | always | | `acl.oidc.enabled` | boolean | always | | `acl.oidc.pkce.required` | boolean | always | | `acl.oidc.state.required` | boolean | always | @@ -400,8 +406,10 @@ gets back. :::note Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries -are published whether or not OIDC is enabled, and with OIDC disabled the -endpoint URLs are built from the defaults rather than from a real provider. +are published whether or not OIDC is enabled. With OIDC disabled and +`acl.oidc.host` in use, the endpoint URLs are built from the defaults rather +than from a real provider; with `acl.oidc.configuration.url` set they are not +resolved at all, and the endpoint keys are absent from the response. OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be `true`, which the same response carries. When access control is disabled, @@ -484,7 +492,9 @@ possible flows to request an access token.: flow **(Recommended, more secure)** 2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) - flow + flow **(deprecated)**. It cannot use PKCE, so it does not satisfy + [`acl.oidc.pkce.required`](/docs/configuration/oidc/#acloidcpkcerequired), + which is `true` by default. The client can read the authorization and token endpoints, the client id and the scopes from the [settings endpoint](#settings-endpoint) instead of hard @@ -674,8 +684,9 @@ with pg.connect(conn_str, autocommit=True) as connection: ### CLI, standalone applications -For a CLI or standalone application built with an official Java, Rust, C, or -C++ client, use the [Device Authorization Flow](/docs/security/oidc-device-flow/). +For a CLI or standalone application built with an official Java, Python, Rust, +C, or C++ client, use the +[Device Authorization Flow](/docs/security/oidc-device-flow/). Tools such as `psql` and Microsoft Access cannot run the flow themselves; for those tools, acquire a token separately and use [PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the @@ -726,8 +737,8 @@ applies to the connection below, which carries the resulting token. import os import requests import pandas as pd +import questdb from dotenv import load_dotenv -from questdb.ingress import Sender load_dotenv() user = os.environ.get("username") @@ -758,11 +769,11 @@ if token_key not in tokens: raise SystemExit(f"the provider did not issue an {token_key} for the password grant") token = tokens[token_key] -conf = f"https::addr=questdb.example.com:9000;token={token};" -with Sender.from_conf(conf) as sender: - df = pd.read_csv("data.csv") - df["ts"] = pd.to_datetime(df["ts"]) - sender.dataframe(df, table_name="foo", at="ts") +conf = f"wss::addr=questdb.example.com:9000;token={token};" +with questdb.connect(conf) as db: + df = pd.read_csv("trades.csv") + df["timestamp"] = pd.to_datetime(df["timestamp"]) + db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") ``` :::note @@ -782,18 +793,18 @@ server side: ```python import os import pandas as pd +import questdb from dotenv import load_dotenv -from questdb.ingress import Sender load_dotenv() user = os.environ.get("username") pwd = os.environ.get("password") -conf = f"https::addr=questdb.example.com:9000;username={user};password={pwd};" -with Sender.from_conf(conf) as sender: - df = pd.read_csv("data.csv") - df["ts"] = pd.to_datetime(df["ts"]) - sender.dataframe(df, table_name="foo", at="ts") +conf = f"wss::addr=questdb.example.com:9000;username={user};password={pwd};" +with questdb.connect(conf) as db: + df = pd.read_csv("trades.csv") + df["timestamp"] = pd.to_datetime(df["timestamp"]) + db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") ``` ## OIDC for the PGWire endpoint @@ -1255,9 +1266,8 @@ The value is `memberOf`. #### Enable Resource Owner Password Credentials (ROPC) flow -As described in the -[OIDC operations document](/docs/security/oidc/#enable-ropc) -tools - such as `psql` - can be integrated with the OIDC provider using the ROPC flow. +As described under [Enable ROPC](#enable-ropc), tools such as `psql` can be +integrated with the OIDC provider using the ROPC flow. When setting this flow up, enable the Resource Owner Password Credentials flow in the client settings. @@ -1274,7 +1284,7 @@ Then select the `username` attribute of the PCV as `USER_KEY`. The below should be set in QuestDB's `server.conf`: -```shell +```ini title="server.conf" # enable OIDC acl.oidc.enabled=true @@ -1303,8 +1313,8 @@ set. #### Confirm QuestDB mappings and login -QuestDB requires a mapping, as laid out in the -[OIDC operations document](/docs/security/oidc/#mapping-user-permissions). +QuestDB requires a mapping, as laid out under +[Mapping user permissions](#mapping-user-permissions). If a given user has the HTTP permission, they will be able to now login via the [Web Console](/docs/getting-started/web-console/overview/). @@ -1438,8 +1448,7 @@ validate the ID token, and take the group information from there. QuestDB authorization relies on receiving the group memberships of the user. Entra ID groups should be mapped to QuestDB groups, and permissions can be granted to the QuestDB groups. Detailed information about group mappings can -be found in the [OIDC integration](/docs/security/oidc/#user-permissions) -documentation. +be found under [User permissions](#user-permissions). Date: Thu, 3 Sep 2026 12:50:05 +0100 Subject: [PATCH 39/54] Correct the prompt sanitization claim and the C++ error kind The page told every reader the prompt's text fields were display-safe. That holds for Java, C and C++, which sanitize before the callback runs, but not for Rust and Python, whose built-in renderers sanitize at display time and whose custom renderers receive the provider's response verbatim. The page recommends a custom renderer for headless and notebook use, so the assurance was given exactly where it does not apply. State the split, and name the helpers that appeared nowhere in the docs: display_user_code() and display_verification_uri() in Rust, sanitize_display_text in questdb.auth. Extend the browser target sentence past C and C++, since Rust and Python expose one too. questdb::error_kind::interaction_required does not compile: the enum is declared in namespace questdb::oidc. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QUNkemSPGkrzov1yFrq6xt --- documentation/connect/clients/c-and-cpp.md | 2 +- documentation/security/oidc-device-flow.mdx | 24 +++++++++++++++++---- 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index ea98fcd735..eb86ad584d 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -354,7 +354,7 @@ most providers issue only when `offline_access` is among the scopes in [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's own `scope` override. Those transport operations never prompt; they fail with `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` in C, or -`questdb::error_kind::interaction_required` in C++, and the application calls +`questdb::oidc::error_kind::interaction_required` in C++, and the application calls `sign_in()` / `questdb_oidc_auth_sign_in()` explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 091acede13..888cdb245c 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -174,9 +174,25 @@ challenge elsewhere, or to declare the process interactive anyway: | C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | | C++ | `.event_handler(...)` | `.interactive(true)` | -The prompt's text fields are display-safe. Where a client opens the URL or makes -it clickable, use the separately vetted browser target instead: `browser_target` -on the C event struct, `event_view::browser_target()` in C++. +The Java, C and C++ clients strip terminal controls, bidi marks and zero-width +characters from the prompt's text fields before the callback sees them. The Rust +and Python clients sanitize inside their built-in renderers only, so a custom +renderer receives the Identity Provider's response verbatim. + +:::caution + +A custom Rust or Python renderer must sanitize the user code and verification +URL itself before writing them to a terminal or a DOM. Rendering them raw +reopens the spoofing surface the built-in renderers close. Rust exposes +`display_user_code()` and `display_verification_uri()` on the challenge; Python +exports `sanitize_display_text` from `questdb.auth`. + +::: + +The text fields are for display only. Where a client opens the URL or makes it +clickable, use the separately vetted browser target instead: `browser_target` on +the C event struct, `event_view::browser_target()` in C++, `browser_target()` on +the Rust challenge, and the `browser_target` key on the Python prompt event. ### Explicit configuration and endpoint pinning @@ -237,7 +253,7 @@ the main or UI thread: | Python | `OidcInteractionRequired`, importable from `questdb.auth` | | Rust | a `questdb::Error` carrying `ErrorCode::AuthError`; read `err.oidc_error()` and match on `OidcErrorKind::InteractionRequired` | | C | error kind `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` | -| C++ | `questdb::error_kind::interaction_required` | +| C++ | `questdb::oidc::error_kind::interaction_required` | | Java | `OidcAuthException`, with no distinct interaction-required type to match on | In C++ a failure raised through an attached sender arrives as From a4368d89fa16789ad6615232f3b274252db1a1fc Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 12:56:05 +0100 Subject: [PATCH 40/54] Fix the OIDC device flow's misleading client guidance Ten corrections, each checked against the client and server sources. The browser table said only Java opens a browser. All five do by default, and four expose an opt-out the page never named, so a headless host got a process nobody can see and no documented switch. The plaintext opt-out was described as a single flag. A client over plaintext also refuses settings-sourced credential endpoints, so the flag needs an issuer pin and explicit values alongside it. Say so, and say that the opt-out never relaxes the Identity Provider legs. issuer was framed as plumbing for a missing endpoint. /settings needs no authentication, so pinning the issuer is what stops a tampered response steering the sign-in, which is why it belongs against any untrusted server. Java splits the settings across DiscoveryOptions and builder(), and the builder demands the client ID and both endpoints together, so Java alone cannot combine its own registration with discovered endpoints. The page claimed parity. acl.oidc.groups.claim: the configured claim names apply in both flows, not just the user info flow. The validator additionally requires literal sub and groups in the payload. Conflating the two produced an instruction to reset acl.oidc.sub.claim which contradicted the Entra ID walkthrough on the next page. Also: rbac.md implied OIDC works on PGWire untouched, when it needs an opt-in setting; steps 8 and 9 of the walkthrough described the user info flow as if it were the only one; the PGWire section never said which token to send; the token store, endpoint pinning and interaction-required recovery had no code in any language; and the page linked to no client page at all, to no provider walkthrough, and to no TLS setting despite requiring https. Examples: ETH-USD to ETH-USDT per the demo dataset, the C partial gains the side and amount columns the other four write, and Java assigns the timestamp client-side like its siblings. Retitle the configuration page to OIDC settings, so it no longer collides with the guide in search results, and update the overview row to match. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QUNkemSPGkrzov1yFrq6xt --- documentation/configuration/oidc.md | 28 ++- documentation/configuration/overview.md | 2 +- documentation/connect/clients/java.md | 14 +- .../partials/_oidc.device-flow.c.partial.mdx | 5 +- .../_oidc.device-flow.cpp.partial.mdx | 2 +- .../_oidc.device-flow.java.partial.mdx | 6 +- .../_oidc.device-flow.python.partial.mdx | 2 +- .../_oidc.device-flow.rust.partial.mdx | 2 +- documentation/security/oidc-device-flow.mdx | 224 ++++++++++++++++-- documentation/security/oidc.mdx | 12 +- documentation/security/rbac.md | 8 +- 11 files changed, 259 insertions(+), 46 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 20820b5c1a..9f8878c2ea 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -1,6 +1,5 @@ --- -title: OpenID Connect (OIDC) -sidebar_label: OIDC settings +title: OIDC settings description: "QuestDB Enterprise acl.oidc.* settings reference: minimum configuration and startup rules, endpoints, TLS, user and group claims, caching and buffers." --- @@ -360,9 +359,9 @@ If the claim is missing from the user information, or it is an empty list, authentication fails. See [Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). -A custom name applies to the user info flow. Set this to `groups` when +The name applies to both flows. Set it to `groups` when [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, -because the token validator requires that name. +because the token validator accepts no other name for the group memberships. ### acl.oidc.groups.encoded.in.token @@ -390,13 +389,22 @@ the token itself and never asks the provider about it: | `aud`, against [`acl.oidc.audience`](#acloidcaudience) | `nbf`, the not-before time | | that `sub` and the group memberships are present | `iss`, the issuer | -The validator reads those two claims under the names `sub` and `groups` -literally, and rejects a token which does not carry both, whatever +Two things have to line up here, because two different components read the +token. The signature validator deserializes the payload into a fixed structure, +so the JWT must carry claims named `sub` and `groups` literally, and `groups` +must be an array of strings. A token without both is rejected whatever the claim +settings say. QuestDB then reads the principal and the group memberships out of +that same payload using [`acl.oidc.sub.claim`](#acloidcsubclaim) and -[`acl.oidc.groups.claim`](#acloidcgroupsclaim) are set to. `groups` must be an -array of strings. Custom claim names apply to the user info flow, so with this -setting enabled leave `acl.oidc.sub.claim` at its default and set -`acl.oidc.groups.claim=groups`. +[`acl.oidc.groups.claim`](#acloidcgroupsclaim), exactly as it does in the user +info flow. + +So set `acl.oidc.groups.claim=groups`, since that is the only name the validator +accepts for the group memberships. `acl.oidc.sub.claim` may name any claim the +token carries: a provider which issues both `sub` and a friendlier claim lets +you keep the readable one as the principal. The Entra ID walkthrough does this, +setting `acl.oidc.sub.claim=name` while the token still carries `sub` for the +validator. :::caution diff --git a/documentation/configuration/overview.md b/documentation/configuration/overview.md index 45dbc1c738..a6dc02454e 100644 --- a/documentation/configuration/overview.md +++ b/documentation/configuration/overview.md @@ -536,7 +536,7 @@ http.net.connection.sndbuf=2m | [Logging & Metrics](/docs/configuration/logging-metrics/) | Log levels and metrics | | | [Materialized views](/docs/configuration/materialized-views/) | Materialized view refresh settings | | | [Minimal HTTP server](/docs/configuration/http-min-server/) | Health check and metrics endpoint | | -| [OpenID Connect (OIDC)](/docs/configuration/oidc/) | OIDC integration | ✓ | +| [OIDC settings](/docs/configuration/oidc/) | `acl.oidc.*` reference for OIDC authentication | ✓ | | [Parallel SQL execution](/docs/configuration/parallel-sql-execution/) | Query parallelism settings | | | [Postgres wire protocol](/docs/configuration/postgres-wire-protocol/) | PostgreSQL wire protocol connections | | | [QuestDB Wire Protocol (QWP)](/docs/configuration/qwp/) | QWP protocol limits and UDP receiver | | diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index dfb9932c92..7dd708ba0a 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -427,11 +427,17 @@ Otherwise pin the provider by passing argument to `fromQuestDB`, or set the client ID, scope and endpoints yourself with `OidcDeviceAuth.builder()`. +`DiscoveryOptions` carries `issuer`, `prompt`, `tokenStore`, `tlsConfig` and +`allowInsecureTransport` only. The client ID, scope, audience and +token-selection mode live on `OidcDeviceAuth.builder()`, which requires the +client ID and both endpoints together, so registering this application under its +own Client Id means pinning the endpoints rather than discovering them. + Tokens stay in memory until a store is attached with -`DiscoveryOptions.tokenStore(FileTokenStore.atDefaultLocation())`, which writes -a long-lived refresh token to disk as plaintext. See the [OIDC guide's client -examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, -explicit configuration, and the token store's location and permissions. +`new OidcDeviceAuth.DiscoveryOptions().tokenStore(FileTokenStore.atDefaultLocation())`, +which writes a long-lived refresh token to disk as plaintext. See the [OIDC device flow examples](/docs/security/oidc-device-flow/#official-client-examples) +for the sign-in prompt, explicit configuration, and the token store's location +and permissions. ### HTTP basic auth diff --git a/documentation/partials/_oidc.device-flow.c.partial.mdx b/documentation/partials/_oidc.device-flow.c.partial.mdx index de8dfd2381..057f39cddd 100644 --- a/documentation/partials/_oidc.device-flow.c.partial.mdx +++ b/documentation/partials/_oidc.device-flow.c.partial.mdx @@ -47,8 +47,11 @@ int main(void) { if (!buffer) goto done; if (!line_sender_buffer_table(buffer, QDB_TABLE_NAME_LITERAL("trades"), &error)) goto done; if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("symbol"), - QDB_UTF8_LITERAL("ETH-USD"), &error)) goto done; + QDB_UTF8_LITERAL("ETH-USDT"), &error)) goto done; + if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("side"), + QDB_UTF8_LITERAL("sell"), &error)) goto done; if (!line_sender_buffer_column_f64(buffer, QDB_COLUMN_NAME_LITERAL("price"), 2615.54, &error)) goto done; + if (!line_sender_buffer_column_f64(buffer, QDB_COLUMN_NAME_LITERAL("amount"), 0.00044, &error)) goto done; if (!line_sender_buffer_at_nanos(buffer, line_sender_now_nanos(), &error)) goto done; if (!qwp_sender_flush_buffer_and_wait(sender, buffer, qwpws_ack_level_ok, &error)) goto done; status = 0; diff --git a/documentation/partials/_oidc.device-flow.cpp.partial.mdx b/documentation/partials/_oidc.device-flow.cpp.partial.mdx index cdc84f0d69..8f2caa6ae8 100644 --- a/documentation/partials/_oidc.device-flow.cpp.partial.mdx +++ b/documentation/partials/_oidc.device-flow.cpp.partial.mdx @@ -22,7 +22,7 @@ int main() { auto sender = pool.borrow_sender(); auto buffer = sender.new_buffer(); buffer.table("trades"_tn) - .symbol("symbol"_cn, "ETH-USD"_utf8) + .symbol("symbol"_cn, "ETH-USDT"_utf8) .symbol("side"_cn, "sell"_utf8) .column("price"_cn, 2615.54) .column("amount"_cn, 0.00044) diff --git a/documentation/partials/_oidc.device-flow.java.partial.mdx b/documentation/partials/_oidc.device-flow.java.partial.mdx index 361ad24f43..2db37f1717 100644 --- a/documentation/partials/_oidc.device-flow.java.partial.mdx +++ b/documentation/partials/_oidc.device-flow.java.partial.mdx @@ -3,6 +3,8 @@ import io.questdb.client.QuestDB; import io.questdb.client.Sender; import io.questdb.client.cutlass.auth.OidcDeviceAuth; +import java.time.Instant; + try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( "https://questdb.example.com:9000")) { auth.signIn(); // the only call which may prompt or open a browser @@ -12,11 +14,11 @@ try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( auth::getToken)) { try (Sender sender = db.borrowSender()) { sender.table("trades") - .symbol("symbol", "ETH-USD") + .symbol("symbol", "ETH-USDT") .symbol("side", "sell") .doubleColumn("price", 2615.54) .doubleColumn("amount", 0.00044) - .atNow(); + .at(Instant.now()); } // db.borrowQuery() uses the same rotating token. } diff --git a/documentation/partials/_oidc.device-flow.python.partial.mdx b/documentation/partials/_oidc.device-flow.python.partial.mdx index dc06df19bb..ccaf6334b3 100644 --- a/documentation/partials/_oidc.device-flow.python.partial.mdx +++ b/documentation/partials/_oidc.device-flow.python.partial.mdx @@ -13,7 +13,7 @@ with OidcDeviceAuth.from_questdb( with db.sender() as sender: sender.row( "trades", - symbols={"symbol": "ETH-USD", "side": "sell"}, + symbols={"symbol": "ETH-USDT", "side": "sell"}, columns={"price": 2615.54, "amount": 0.00044}, at=TimestampNanos.now(), ) diff --git a/documentation/partials/_oidc.device-flow.rust.partial.mdx b/documentation/partials/_oidc.device-flow.rust.partial.mdx index b333dfc372..f5559813f3 100644 --- a/documentation/partials/_oidc.device-flow.rust.partial.mdx +++ b/documentation/partials/_oidc.device-flow.rust.partial.mdx @@ -25,7 +25,7 @@ fn main() -> questdb::Result<()> { let mut buffer = sender.new_buffer(); buffer .table("trades")? - .symbol("symbol", "ETH-USD")? + .symbol("symbol", "ETH-USDT")? .symbol("side", "sell")? .column_f64("price", 2615.54)? .column_f64("amount", 0.00044)? diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 888cdb245c..cde0c25c93 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -1,6 +1,6 @@ --- title: OIDC Device Authorization Flow -description: Sign a user in to QuestDB Enterprise from a command-line application, container, or remote notebook kernel with the OAuth 2.0 device authorization grant, with worked client examples. +description: "OIDC device flow for QuestDB Enterprise: headless SSO sign-in from a CLI, container, or notebook, with Java, Python, Rust, C and C++ examples." --- import Tabs from "@theme/Tabs"; @@ -86,6 +86,13 @@ with the client's `issuer` option so the client can fetch the provider's `/.well-known/openid-configuration` document, or configure the client ID and both endpoints explicitly. +Clients reach QuestDB over `https` for discovery, so the HTTP endpoint needs TLS +enabled with [`http.tls.enabled`](/docs/configuration/tls/#httptlsenabled) and a +certificate. For setting OIDC up against a specific provider, the OIDC guide has +walkthroughs for +[Microsoft Entra ID](/docs/security/oidc/#microsoft-entraid) and +[PingFederate](/docs/security/oidc/#pingfederate). + ## Official client examples Each example discovers the OIDC client ID, scope, token-selection mode, and @@ -101,11 +108,24 @@ The discovery URL must use `https`: reaching QuestDB over plaintext would let an attacker replace the advertised Identity Provider endpoints and redirect the device code and the refresh token, so the clients reject it rather than warn. The Rust, C, C++ and Python clients accept loopback `http` for local -development; the Java client does not. To use a plaintext non-loopback host -anyway, opt in with `allowInsecureTransport(true)` (Java), -`.allow_insecure_transport(true)` (Rust), -`questdb_oidc_builder_allow_insecure_transport()` (C), -`allow_insecure_transport()` (C++) or `insecure=True` (Python). +development; the Java client requires the opt-out below even for `localhost`. + +The opt-out covers the QuestDB discovery request only. Every client holds the +Identity Provider itself to `https` or loopback, and no flag relaxes that: + +| Client | Opt out of `https` for discovery | +| --- | --- | +| Java | `allowInsecureTransport(true)` | +| Python | `insecure=True` | +| Rust | `.allow_insecure_transport(true)` | +| C | `questdb_oidc_builder_allow_insecure_transport()` | +| C++ | `.allow_insecure_transport(true)` | + +The flag alone is not enough. Over plaintext a client also refuses to take the +credential endpoints from a `/settings` response that could have been tampered +with in transit, so pin the provider with `issuer` as well, and pass explicitly +any client ID, scope, audience, or token-selection mode you rely on. An issuer +pin covers the endpoints, not those four values. @@ -142,9 +162,12 @@ questdb-rs = { version = "7", features = ["oidc"] } -The authentication object owns the token state. Pass it as a rotating provider, -as shown above, instead of copying the current token into the connection string; -otherwise a reconnect after token expiry will keep sending the stale value. +The authentication object owns the token state, and the connection asks it for a +token on every connect and reconnect. Java and Rust take a token provider +(`auth::getToken`, a closure over `auth.token()`); Python, C and C++ take the +authentication object itself. Either way, hand the connection the object rather +than copying the current token into the connection string, otherwise a reconnect +after expiry keeps sending the stale value. Device flow always requires a person to approve the sign-in. For unattended services, scheduled jobs, or CI, use a service-account token or a provider flow @@ -158,18 +181,34 @@ call runs at all without a terminal, differs by client: | Client | Default prompt | Without a terminal | | --- | --- | --- | -| Java | prints to `System.out`, then tries to open a browser | runs anyway; the client performs no terminal check | +| Java | prints to `System.out` | runs anyway; the client performs no terminal check | | Python | prints to stderr, or to the frontend inside an IPython kernel | fails, unless the kernel accepts stdin | | Rust, C, C++ | prints to stderr | fails with an interaction-required error | Redirecting output therefore hides the code in Java, while a container with no -TTY fails outright in the other four. Override the default to render the -challenge elsewhere, or to declare the process interactive anyway: +TTY fails outright in the other four. + +All five clients also try to open the verification URL in a local browser by +default. That is useful on a workstation and pointless on a remote or headless +host, where the browser opens on the wrong machine or a process starts that +nobody can see. Python is the exception: it suppresses the launch by itself +inside a Jupyter kernel. Turn it off explicitly elsewhere: + +| Client | Suppress the browser launch | +| --- | --- | +| Java | `prompt(DeviceCodePrompt.SYSTEM_OUT)`, or the `questdb.client.oidc.open.browser` system property set to `false` | +| Python | `open_browser=False` | +| Rust | `.open_browser(false)` | +| C | `questdb_oidc_builder_open_browser(builder, false, &error)` | +| C++ | `.open_browser(false)` | + +Override the default prompt to render the challenge elsewhere, or declare the +process interactive anyway: | Client | Custom prompt | Force interactive | | --- | --- | --- | -| Java | `prompt(...)`, or `DeviceCodePrompt.SYSTEM_OUT` to skip the browser launch | not applicable | -| Python | `renderer=` | `interactive=True` | +| Java | `prompt(...)` | not applicable | +| Python | `renderer=`, or `qr=True` for a QR code | `interactive=True`, and `interactive=False` to fail rather than prompt | | Rust | `.renderer(...)` | `.interactive(true)` | | C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | | C++ | `.event_handler(...)` | `.interactive(true)` | @@ -206,15 +245,76 @@ needs: | `client_id` | the application has its own registration, which is the recommended setup | | `scope` | the provider requires `offline_access` before it issues a refresh token | | `audience` | the provider requires one, or QuestDB validates `aud` against a non-default [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | -| `issuer` | QuestDB publishes no device authorization endpoint, so the client reads the provider's `/.well-known/openid-configuration` instead | +| `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads the provider's `/.well-known/openid-configuration` instead | | `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | | `groups_in_token` | overriding which of the two tokens the client selects | -Java exposes `issuer`, `prompt`, `tokenStore`, `tlsConfig` and -`allowInsecureTransport` on `OidcDeviceAuth.DiscoveryOptions`, and the full set -on `OidcDeviceAuth.builder()`, in camel case. Python passes them as keyword -arguments to `from_questdb`. Rust and C++ are builder methods in snake case, and -C prefixes the same names with `questdb_oidc_builder_`. +Python passes these as keyword arguments to `from_questdb`. Rust and C++ are +builder methods in snake case, and C prefixes the same names with +`questdb_oidc_builder_`. + +Java splits them. `OidcDeviceAuth.DiscoveryOptions` carries only `issuer`, +`prompt`, `tokenStore`, `tlsConfig` and `allowInsecureTransport`; `client_id`, +`scope`, `audience` and `groups_in_token` live on `OidcDeviceAuth.builder()`, +which requires the client ID and both endpoints together. So Java cannot combine +its own client ID with endpoints discovered from `/settings`: use the builder +and pin the endpoints too. The other four clients accept a `client_id` override +on top of discovery. + +:::caution + +`/settings` needs no authentication, so it designates where your users type +their credentials and where the refresh token goes. Pinning `issuer` binds the +advertised endpoints to that issuer's origin, which is why it is worth setting +against any server you do not fully trust, not only when an endpoint is missing. + +::: + +Pinning a provider and using an application's own registration: + + + + +```java +OidcDeviceAuth auth = OidcDeviceAuth.builder() + .clientId("my-cli") + .audience("questdb-api") + .deviceAuthorizationEndpoint("https://login.example.com/oauth2/v2.0/devicecode") + .tokenEndpoint("https://login.example.com/oauth2/v2.0/token") + .build(); +``` + + + + +```python +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + issuer="https://login.example.com", + client_id="my-cli", + audience="questdb-api", +) +``` + + + + +```rust +let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .issuer("https://login.example.com") + .client_id("my-cli") + .audience("questdb-api") + .build()?; +``` + + + + +An endpoint QuestDB advertises must sit under the pinned issuer's origin, or the +client rejects it. An endpoint the client reads from the provider's own +`/.well-known/openid-configuration` is exempt, which is what makes the pin work +with providers that host their endpoints outside the issuer, such as Microsoft +Entra ID and Google. ### Token persistence @@ -232,7 +332,40 @@ where that at-rest exposure is acceptable: Java, Python and Rust attach the store with `tokenStore(...)`, `token_store=` and `.token_store(...)` respectively; the C and C++ calls above are builder -methods and attach it themselves. +methods and attach it themselves. Attach it before signing in, so the first +`sign_in()` can reuse a refresh token an earlier run left behind: + + + + +```java +OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000", + new OidcDeviceAuth.DiscoveryOptions() + .tokenStore(FileTokenStore.at(Paths.get("/var/lib/myapp/oidc")))); +``` + + + + +```python +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + token_store=FileTokenStore.at("/var/lib/myapp/oidc"), +) +``` + + + + +```rust +let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .token_store(FileTokenStore::at("/var/lib/myapp/oidc")) + .build()?; +``` + + + The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the `questdb.client.oidc.token.store.dir` environment variable. Java reads that same @@ -260,3 +393,52 @@ In C++ a failure raised through an attached sender arrives as `questdb::ingress::line_sender_error` carrying `oidc_diagnostic()`, not as `questdb::oidc::error`, so catch the common base `const questdb::error&` to handle both. + +Catching it and signing in again: + + + + +```python +try: + sender.flush(wait=True) +except OidcInteractionRequired: + auth.sign_in() # on the main or UI thread + sender.flush(wait=True) +``` + + + + +```rust +fn needs_sign_in(err: &questdb::Error) -> bool { + err.code() == ErrorCode::AuthError + && matches!( + err.oidc_error().map(|e| e.kind()), + Some(OidcErrorKind::InteractionRequired) + ) +} + +if let Err(err) = sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok) { + if needs_sign_in(&err) { + auth.sign_in()?; // on the main or UI thread + sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok)?; + } else { + return Err(err); + } +} +``` + + + + +Java has no distinct type to match on, so an application that must recover +automatically has to treat any `OidcAuthException` from a transport call as a +prompt to call `signIn()` again. + +For the per-language API surface, the connection options, and the rest of each +client's behaviour, see the client pages: +[Java](/docs/connect/clients/java/#oidc-device-flow-enterprise), +[Python](/docs/connect/clients/python/#oidc-device-flow-enterprise), +[Rust](/docs/connect/clients/rust/#oidc-device-flow-enterprise), and +[C and C++](/docs/connect/clients/c-and-cpp/#oidc-device-flow-enterprise). diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 1d76f1f6e5..2da7d4c02e 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -261,8 +261,12 @@ To carry out permission checks, the database has to know more about the user. For this, QuestDB has a User Info Cache. -If it finds a valid entry for the token in the cache, steps 8 and 9 are -skipped: +Steps 8 and 9 describe the default flow, where QuestDB validates the access +token by asking the OIDC Provider about it. They are skipped when the cache +already holds a valid entry for the token, and they do not happen at all when +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is `true`, in which case QuestDB validates the ID token itself and never calls +the provider. See [Which token to send](#which-token-to-send). ```bash title="Query request example" https://questdb.example.com:9000/exec?query=select%20current_user() @@ -816,7 +820,9 @@ This method works wherever a Postgres client library is available, including jup Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. -The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option: +The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option. Send the token +selected by [Which token to send](#which-token-to-send): the access token by +default, or the ID token when group memberships are encoded in it. ```python import psycopg as pg diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md index 77c14eaaf7..27707e4496 100644 --- a/documentation/security/rbac.md +++ b/documentation/security/rbac.md @@ -321,7 +321,7 @@ QuestDB supports four authentication methods: | **Password** | Interactive users | REST API, PostgreSQL Wire | | **JWK Token** | ILP ingestion | InfluxDB Line Protocol | | **REST API Token** | Programmatic REST access | REST API | -| **OIDC bearer token** | SSO users, external clients | REST API, PostgreSQL Wire, QWP | +| **OIDC bearer token** | SSO users, external clients | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire once enabled | The first three are QuestDB's own credentials, created with the statements below. An OIDC bearer token is issued by an external Identity Provider instead, @@ -329,6 +329,12 @@ and the user's group memberships come from the token or the provider's user info endpoint; see [OpenID Connect](/docs/security/oidc/) and [which token to send](/docs/security/oidc/#which-token-to-send). +OIDC on the PostgreSQL Wire endpoint is off by default. Enable it with +[`acl.oidc.pg.token.as.password.enabled`](/docs/configuration/oidc/#acloidcpgtokenaspasswordenabled), +which accepts the token in the password field, or with +[`acl.oidc.ropc.flow.enabled`](/docs/configuration/oidc/#acloidcropcflowenabled), +which has QuestDB exchange the user's SSO credentials for a token itself. + Users can have multiple authentication methods enabled simultaneously: ```questdb-sql From 13912fb3b23570da6e77f2fa3ac2f45be1e36ad1 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 13:06:39 +0100 Subject: [PATCH 41/54] Settle the OIDC device flow's naming, fences and remaining snags The feature had five names across the surfaces a reader meets it on: the page title, the sidebar, the guide's heading, the changelog and eleven link texts. Settle on "OIDC device flow", which is what people search for and what the sidebar already said, and pin the guide heading's old anchor so any external link to it still resolves. The sequence diagram drew discovery through QuestDB as the only path, though the page documents pinning an issuer instead, and stopped at the first token though the rest of the page is about what happens after it. Add the branch, run the user's approval in parallel with the polling loop as RFC 8628 does, and note the refresh. Verified with mermaid 11.9.0, the version the site ships. The four client pages sent readers to #official-client-examples while promising the sign-in prompt, the token store and explicit configuration, so the link landed on the code the reader had just finished and left the promised material several sections below. Point each at the section it names. FileTokenStore::at_default_location() returns a std::io::Result, so the `?` shown against a questdb::Result signature does not compile. Say what it returns, and why it can fail at all. Also: the token store's override is a dotted name, which most shells will not export; the ROPC example indexed a settings key the same page documents as conditional; a .env block was fenced as python and a server.conf block was not fenced as anything, in a section that is now a link target; oidc.mdx imported Tabs and TabItem without using either; and failover.md and dotnet.md carried em-dashes. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QUNkemSPGkrzov1yFrq6xt --- documentation/changelog.mdx | 2 +- documentation/configuration/oidc.md | 4 +-- documentation/connect/clients/c-and-cpp.md | 8 ++--- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 2 +- documentation/connect/clients/java.md | 8 ++--- documentation/connect/clients/python.md | 8 ++--- documentation/connect/clients/rust.md | 11 +++--- documentation/high-availability/failover.md | 6 ++-- documentation/security/oidc-device-flow.mdx | 38 ++++++++++++++------- documentation/security/oidc.mdx | 25 ++++++++------ 11 files changed, 66 insertions(+), 48 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 4502d0dc19..47928db6ec 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -42,7 +42,7 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another -- [OIDC Device Authorization Flow](/docs/security/oidc-device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients +- [OIDC device flow](/docs/security/oidc-device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients - [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 9f8878c2ea..199fb66886 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -162,7 +162,7 @@ QuestDB uses the scopes in the requests it makes itself, in the [settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which run the flow themselves. -For the [Device Authorization Flow](/docs/security/oidc-device-flow/), +For the [OIDC device flow](/docs/security/oidc-device-flow/), add `offline_access` when the provider requires that scope before issuing a refresh token. Without a refresh token, a client must ask the user to sign in again after the current token expires. @@ -256,7 +256,7 @@ has no default, and QuestDB never calls it. QuestDB resolves the endpoint and publishes it on the [settings endpoint](/docs/security/oidc/#settings-endpoint), for clients which implement the -[Device Authorization Flow](/docs/security/oidc-device-flow/) +[OIDC device flow](/docs/security/oidc-device-flow/) themselves. The official Java, Python, Rust, C, and C++ clients can discover and use the published endpoint. diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index eb86ad584d..c8ef5d8895 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -330,7 +330,7 @@ Handle it there, not at `connect` (see ### OIDC device flow (Enterprise) The C and C++ APIs sign in an interactive user with the -[Device Authorization Flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc-device-flow/). They discover the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. Attach the resulting auth object to the pool so sender and reader connections share its rotating token: @@ -367,9 +367,9 @@ names are prefixed with `questdb_oidc_builder_`. Tokens stay in memory until `.default_file_token_store()` in C++, or `questdb_oidc_builder_default_file_token_store()` in C, writes a long-lived -refresh token to disk as plaintext. See the [complete client -examples](/docs/security/oidc-device-flow/#official-client-examples) for error handling, the -sign-in prompt, and the store's location and permissions. +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). ### Other authentication limitations diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index 76df3b4781..171bd320f9 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [Device Authorization Flow](/docs/security/oidc-device-flow/) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index b66b2748c5..809fdc29c3 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -370,7 +370,7 @@ token, or the ID token when QuestDB reads group memberships from the token. This client does not run an OIDC flow, so acquire the token with an OAuth2 library: `golang.org/x/oauth2` implements the device grant, and QuestDB publishes the device authorization endpoint on the same settings response. The -[Device Authorization Flow](/docs/security/oidc-device-flow/) +[OIDC device flow](/docs/security/oidc-device-flow/) describes the protocol, including the polling interval, `slow_down`, and the device code's expiry. When the token expires or is rotated, construct a new handle with the new token. diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index 7dd708ba0a..e32cc2c8a3 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -401,7 +401,7 @@ both the ingress and egress WebSocket upgrades. It is mutually exclusive with ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[Device Authorization Flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc-device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. The user can approve the sign-in from a browser on any device, so this also works from a container or @@ -435,9 +435,9 @@ own Client Id means pinning the endpoints rather than discovering them. Tokens stay in memory until a store is attached with `new OidcDeviceAuth.DiscoveryOptions().tokenStore(FileTokenStore.atDefaultLocation())`, -which writes a long-lived refresh token to disk as plaintext. See the [OIDC device flow examples](/docs/security/oidc-device-flow/#official-client-examples) -for the sign-in prompt, explicit configuration, and the token store's location -and permissions. +which writes a long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). ### HTTP basic auth diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index e5b0358fe5..ccc483596d 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -169,7 +169,7 @@ the full grammar. ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[Device Authorization Flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc-device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB expects from the server's public `/settings` endpoint. The default prompt works in terminals and remote Jupyter kernels: @@ -194,9 +194,9 @@ Otherwise pass `issuer=...`, or set `client_id=`, `scope=`, `audience=`, Tokens stay in memory until a store is passed as `token_store=FileTokenStore.at_default_location()`, which writes a long-lived -refresh token to disk as plaintext. See the [OIDC client -examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, -explicit configuration, and the store's location and permissions. +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). ### Other authentication limitations diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index f97eaa9ddc..5b4ed6e983 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -168,7 +168,7 @@ choices have feature requirements: ### OIDC device flow (Enterprise) Enable the `oidc` feature to sign in an interactive user with the -[Device Authorization Flow](/docs/security/oidc-device-flow/): +[OIDC device flow](/docs/security/oidc-device-flow/): ```toml title="Cargo.toml" [dependencies] @@ -199,10 +199,11 @@ Otherwise add `.issuer(...)` to the builder, or set `.client_id()`, `.scope()`, yourself. Tokens stay in memory until -`.token_store(FileTokenStore::at_default_location()?)` is added, which writes a -long-lived refresh token to disk as plaintext. See the [OIDC client -examples](/docs/security/oidc-device-flow/#official-client-examples) for the sign-in prompt, -explicit configuration, and the store's location and permissions. +`.token_store(FileTokenStore::at_default_location()?)` is added (it returns a +`std::io::Result`, so map the error into yours), which writes a +long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). ## The pool diff --git a/documentation/high-availability/failover.md b/documentation/high-availability/failover.md index 67345bc76d..4b6379e718 100644 --- a/documentation/high-availability/failover.md +++ b/documentation/high-availability/failover.md @@ -284,9 +284,9 @@ The [minimal HTTP server](/docs/operations/logging-metrics/#minimal-http-server) on port 9003 exposes the same switch to external coordinators. It accepts the same credentials as the main HTTP server: HTTP basic authentication, a [REST token](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise) -as `Authorization: Bearer`, or an OIDC token — [the access token or the ID -token](/docs/security/oidc/#which-token-to-send) depending on -`acl.oidc.groups.encoded.in.token`. The principal needs the +as `Authorization: Bearer`, or an OIDC token. Which of the two OIDC tokens to +send is decided by `acl.oidc.groups.encoded.in.token`; see +[which token to send](/docs/security/oidc/#which-token-to-send). The principal needs the `HTTP` endpoint permission and, for the switch, `SWITCH ROLE`. With access control disabled no credentials are needed. TLS for this port is configured with the `http.min.tls.*` settings on the [TLS](/docs/configuration/tls/) page. diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index cde0c25c93..86cb1850d5 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -1,5 +1,5 @@ --- -title: OIDC Device Authorization Flow +title: OIDC device flow description: "OIDC device flow for QuestDB Enterprise: headless SSO sign-in from a CLI, container, or notebook, with Java, Python, Rust, C and C++ examples." --- @@ -13,7 +13,7 @@ import CExample from "../partials/_oidc.device-flow.c.partial.mdx"; import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - The Device Authorization Flow signs in an interactive user from a client that + The OIDC device flow signs in an interactive user from a client that cannot receive a browser redirect. @@ -32,17 +32,26 @@ sequenceDiagram participant QDB as QuestDB participant IdP as Identity Provider participant User - Client->>QDB: GET /settings - QDB-->>Client: client ID, scope, endpoints, token mode + alt Discovery through QuestDB + Client->>QDB: GET /settings + QDB-->>Client: client ID, scope, endpoints, token mode + else issuer pinned on the client + Client->>IdP: GET /.well-known/openid-configuration + IdP-->>Client: endpoints + end Client->>IdP: Request device and user codes IdP-->>Client: device_code, user_code, verification URI Client->>User: Display URL and code - User->>IdP: Sign in and approve in a browser - loop Until approved or expired - Client->>IdP: Poll token endpoint - IdP-->>Client: authorization_pending / tokens + par User authorizes + User->>IdP: Sign in and approve in a browser + and Client polls + loop Until approved or expired + Client->>IdP: Poll token endpoint + IdP-->>Client: authorization_pending / tokens + end end Client->>QDB: Authorization: Bearer selected token + Note over Client,IdP: Later: silent refresh, or an interaction-required error ``` The client must respect the polling interval, `slow_down` responses, and the @@ -244,7 +253,7 @@ needs: | --- | --- | | `client_id` | the application has its own registration, which is the recommended setup | | `scope` | the provider requires `offline_access` before it issues a refresh token | -| `audience` | the provider requires one, or QuestDB validates `aud` against a non-default [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | +| `audience` | the provider requires one, or QuestDB checks `aud` against [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | | `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads the provider's `/.well-known/openid-configuration` instead | | `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | | `groups_in_token` | overriding which of the two tokens the client selects | @@ -326,7 +335,7 @@ where that at-rest exposure is acceptable: | --- | --- | --- | | Java | `FileTokenStore.atDefaultLocation()` | `FileTokenStore.at(path)` | | Python | `FileTokenStore.at_default_location()` | `FileTokenStore.at(path)` | -| Rust | `FileTokenStore::at_default_location()?` | `FileTokenStore::at(path)` | +| Rust | `FileTokenStore::at_default_location()` | `FileTokenStore::at(path)` | | C | `questdb_oidc_builder_default_file_token_store()` | `questdb_oidc_builder_file_token_store()` | | C++ | `.default_file_token_store()` | `.file_token_store(directory)` | @@ -368,9 +377,14 @@ let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the -`questdb.client.oidc.token.store.dir` environment variable. Java reads that same +`questdb.client.oidc.token.store.dir` environment variable. The name contains +dots, so most shells will not `export` it directly; set it through the container +or service manager, or pass an explicit directory instead. Java reads that same name as a JVM system property rather than an environment variable, and falls -back to `${user.home}/.questdb/oidc-tokens/`. On Unix the clients create token +back to `${user.home}/.questdb/oidc-tokens/`. Resolving the default location can +fail where there is no home directory, as in a distroless container, so Rust +returns a `std::io::Result` from `at_default_location()` and an explicit +directory avoids the question. On Unix the clients create token files with mode `0600` and the store directory with `0700`; on other platforms the directory's existing ACL governs access, so restrict it before enabling the store. diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 2da7d4c02e..0346b80ca8 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -4,8 +4,6 @@ description: Integrate QuestDB Enterprise with an external OIDC Identity Provide --- import Screenshot from "@theme/Screenshot"; -import Tabs from "@theme/Tabs"; -import TabItem from "@theme/TabItem"; import { EnterpriseNote } from "@site/src/components/EnterpriseNote" @@ -395,7 +393,7 @@ configuration document when `acl.oidc.configuration.url` is set. [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) is present when a device authorization endpoint has been configured, or when the provider advertises one. QuestDB publishes it without using it, for clients -which implement the [Device Authorization Flow](/docs/security/oidc-device-flow/) +which implement the [OIDC device flow](/docs/security/oidc-device-flow/) themselves. `acl.oidc.redirect.uri` is `null` unless @@ -461,7 +459,7 @@ for the full list of what is and is not validated. ::: -## Device Authorization Flow +## OIDC device flow {#device-authorization-flow} For command-line applications, containers, and remote notebook kernels, where no browser redirect can reach the client, QuestDB Enterprise supports the @@ -469,7 +467,7 @@ OAuth 2.0 Device Authorization Grant. QuestDB does not run the flow: the client talks to the Identity Provider directly and presents the resulting bearer token. -See [OIDC Device Authorization Flow](/docs/security/oidc-device-flow/) for the +See [OIDC device flow](/docs/security/oidc-device-flow/) for the protocol walkthrough, what to configure first, and worked examples for the Java, Python, Rust, C and C++ clients. @@ -524,7 +522,7 @@ The OAuthenticator documentation also contains using different identity providers. If Jupyter notebooks are used without JupyterHub, use the official Python -client's [Device Authorization Flow](/docs/security/oidc-device-flow/). It works when +client's [OIDC device flow](/docs/security/oidc-device-flow/). It works when the browser and notebook kernel are on different machines and does not put the user's password in the notebook. @@ -590,7 +588,7 @@ Here is an example using the `dotenv` library. First we need to create a file named `.env` with the settings: -```python +```ini title=".env" username=testuser password=testpwd ``` @@ -628,8 +626,8 @@ with request.urlopen(req) as f: The Resource Owner Password Credentials flow can be enabled in QuestDB within `server.conf`: -``` -acl.oidc.ropc.flow.enabled = true +```ini title="server.conf" +acl.oidc.ropc.flow.enabled=true ``` > Note that the flow also has to be configured in the OAuth2/OIDC provider! @@ -690,7 +688,7 @@ with pg.connect(conn_str, autocommit=True) as connection: For a CLI or standalone application built with an official Java, Python, Rust, C, or C++ client, use the -[Device Authorization Flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc-device-flow/). Tools such as `psql` and Microsoft Access cannot run the flow themselves; for those tools, acquire a token separately and use [PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the @@ -757,7 +755,12 @@ settings = requests.get("https://questdb.example.com:9000/settings").json()["con if not settings.get("acl.oidc.enabled"): raise SystemExit("OIDC is not enabled on this QuestDB instance") -response = requests.post(settings["acl.oidc.token.endpoint"], +# the endpoint keys are absent until QuestDB resolves them +token_endpoint = settings.get("acl.oidc.token.endpoint") +if not token_endpoint: + raise SystemExit("QuestDB has not resolved the provider's token endpoint") + +response = requests.post(token_endpoint, data={"grant_type": "password", "client_id": client_id, "username": user, From afd838e22dca73834fc2242bf7ae7aefcb81413d Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 13:28:20 +0100 Subject: [PATCH 42/54] Give the device flow's reference sections and the DIY path their own homes Four reference sections sat as H3s under "Official client examples", so the client pages linking to them landed the reader on the code they had just read with the promised material several sections below. Promote them to H2. Their anchors are derived from the heading text, not its level, so every inbound link still resolves. Go, Node.js and .NET readers were sent here for the polling interval, slow_down and the device code's expiry, which the page only named in passing before saying the official clients handle them. Add a section that actually sets out the two requests, the four token-endpoint error codes and what each one means, and the refresh token, for anyone driving the grant themselves. offline_access was hedged three ways: "if your provider requires" on the hub, "most providers issue only when" on four client pages, each with its own wording. State it once under acl.oidc.scope, where the setting lives, name Entra ID as a provider that needs it, and say that adding it is harmless where it is not needed. The hub and the client pages now defer to that. Move the pointer to the client pages to the end of the page; it is a page-level "where next" and had been sitting under Interaction-required errors. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QUNkemSPGkrzov1yFrq6xt --- documentation/configuration/oidc.md | 12 +++-- documentation/connect/clients/c-and-cpp.md | 11 ++-- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 11 ++-- documentation/connect/clients/java.md | 12 ++--- documentation/connect/clients/nodejs.md | 4 +- documentation/connect/clients/python.md | 9 ++-- documentation/connect/clients/rust.md | 12 ++--- documentation/security/oidc-device-flow.mdx | 57 ++++++++++++++++++--- 9 files changed, 90 insertions(+), 40 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 199fb66886..fd6c32999b 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -162,10 +162,14 @@ QuestDB uses the scopes in the requests it makes itself, in the [settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which run the flow themselves. -For the [OIDC device flow](/docs/security/oidc-device-flow/), -add `offline_access` when the provider requires that scope before issuing a -refresh token. Without a refresh token, a client must ask the user to sign in -again after the current token expires. +For the [OIDC device flow](/docs/security/oidc-device-flow/), add +`offline_access`. Most providers, Microsoft Entra ID among them, issue a +refresh token only when that scope was requested, and without one the client +has to make the user sign in again as soon as the current token expires. +Providers which issue refresh tokens regardless ignore the extra scope, so +adding it is the safe default. A client can also request it through its own +`scope` override, which leaves this server-wide value, and every other flow +using it, untouched. ## Authentication flows diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index c8ef5d8895..b78af78969 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -350,12 +350,13 @@ object to the pool so sender and reader connections share its rotating token: The pool retains the auth state and gets a cached or silently refreshed token for every connection and reconnect. Silent refresh needs a refresh token, which -most providers issue only when `offline_access` is among the scopes in -[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's -own `scope` override. Those transport operations never prompt; they fail with +the provider issues only when `offline_access` was requested; see +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Those transport +operations never prompt; they fail with `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` in C, or -`questdb::oidc::error_kind::interaction_required` in C++, and the application calls -`sign_in()` / `questdb_oidc_auth_sign_in()` explicitly on the main or UI thread. +`questdb::oidc::error_kind::interaction_required` in C++, and the application +calls `sign_in()` / `questdb_oidc_auth_sign_in()` explicitly on the main or UI +thread. The Identity Provider must enable the device grant. For discovery without an override, QuestDB must publish a device authorization endpoint, either from the diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index 171bd320f9..40ffed6253 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 809fdc29c3..b8f25cef61 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -360,8 +360,9 @@ db, err := qdb.Connect(ctx, The token is sent as an `Authorization: Bearer YOUR_BEARER_TOKEN` header on both the ingress and egress WebSocket upgrades. It is a **static credential**: the client sends exactly the string you pass and never refreshes or renews it. -Acquire it out of band — QuestDB Enterprise issues bearer tokens through its -[OpenID Connect flow](/docs/security/oidc/) — and manage its lifetime yourself. +Acquire it out of band, since QuestDB Enterprise issues bearer tokens through +its [OpenID Connect flow](/docs/security/oidc/), and manage its lifetime +yourself. QuestDB publishes the provider's authorization and token endpoints on its [settings endpoint](/docs/security/oidc/#settings-endpoint), so a client can discover them instead of hard coding them. The same response tells the client @@ -370,9 +371,9 @@ token, or the ID token when QuestDB reads group memberships from the token. This client does not run an OIDC flow, so acquire the token with an OAuth2 library: `golang.org/x/oauth2` implements the device grant, and QuestDB publishes the device authorization endpoint on the same settings response. The -[OIDC device flow](/docs/security/oidc-device-flow/) -describes the protocol, including the polling interval, `slow_down`, and the -device code's expiry. +[OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) +sets out the two requests, the polling interval, `slow_down`, and the device +code's expiry. When the token expires or is rotated, construct a new handle with the new token. An expired or rejected token surfaces as an authentication failure (see [Connection-level errors](#connection-level-errors)). It is mutually exclusive diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index e32cc2c8a3..debc149841 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -411,12 +411,12 @@ remote notebook kernel: Pass `auth::getToken` as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. Silent refresh needs a refresh token, which most -providers issue only when `offline_access` is among the scopes in -[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's -own `scope` override. A transport call never starts an interactive flow; it -throws `OidcAuthException`, which carries no distinct interaction-required type, -so call `signIn()` again on the main or UI thread. +silently refreshed token. Silent refresh needs a refresh token, which the +provider issues only when `offline_access` was requested; see +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call +never starts an interactive flow; it throws `OidcAuthException`, which carries +no distinct interaction-required type, so call `signIn()` again on the main or +UI thread. The Identity Provider must enable the device grant. For discovery without an override, QuestDB must publish a device authorization endpoint, either from the diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index 0df1db8f1a..82f636bc9b 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -96,7 +96,9 @@ bearer token, which you acquire out of band from your Identity Provider: QuestDB publishes the provider's endpoints on its [settings endpoint](/docs/security/oidc/#settings-endpoint), and the same response tells you [which token to send](/docs/security/oidc/#which-token-to-send). -See [OpenID Connect](/docs/security/oidc/). +See [OpenID Connect](/docs/security/oidc/), and +[implementing the flow yourself](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) for the device grant's +requests and polling rules. ## Basic insert diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index ccc483596d..3758b40b02 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -179,11 +179,10 @@ in terminals and remote Jupyter kernels: `oidc_auth=auth` keeps shared ownership of the provider and obtains the cached or silently refreshed token for every connection and reconnect. It is mutually exclusive with a fixed `token=` setting. Silent refresh needs a refresh token, -which most providers issue only when `offline_access` is among the scopes in -[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's -own `scope` override. Transport operations never prompt; if they raise -`OidcInteractionRequired`, importable from `questdb.auth`, call `auth.sign_in()` -explicitly on the main or UI thread. +which the provider issues only when `offline_access` was requested; see +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Transport operations +never prompt; if they raise `OidcInteractionRequired`, importable from +`questdb.auth`, call `auth.sign_in()` explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an override, QuestDB must publish a device authorization endpoint, either from the diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 5b4ed6e983..7e26898642 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -182,12 +182,12 @@ scope, and the token QuestDB expects from the public `/settings` endpoint: Pass the closure as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. Silent refresh needs a refresh token, which most -providers issue only when `offline_access` is among the scopes in -[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) or in the client's -own `scope` override. A transport call never starts an interactive flow; it -returns a `questdb::Error` with `ErrorCode::AuthError`, whose `err.oidc_error()` -reports `OidcErrorKind::InteractionRequired` when a new sign-in is needed. Call +silently refreshed token. Silent refresh needs a refresh token, which the +provider issues only when `offline_access` was requested; see +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call +never starts an interactive flow; it returns a `questdb::Error` with +`ErrorCode::AuthError`, whose `err.oidc_error()` reports +`OidcErrorKind::InteractionRequired` when a new sign-in is needed. Call `sign_in()` explicitly on the main or UI thread. The Identity Provider must enable the device grant. For discovery without an diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 86cb1850d5..9ee4048a43 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -76,8 +76,9 @@ Before using the flow: a provider document containing `device_authorization_endpoint`, or the host-based configuration must set [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). -3. Keep `openid` in `acl.oidc.scope`. Add `offline_access` if your provider - requires that scope before issuing a refresh token to a device-flow client. +3. Keep `openid` in [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope), + and add `offline_access` so the provider issues a refresh token. Without one + the client must make the user sign in again at every token expiry. For example, a host-based configuration can add: @@ -182,7 +183,7 @@ Device flow always requires a person to approve the sign-in. For unattended services, scheduled jobs, or CI, use a service-account token or a provider flow intended for machine identities instead. -### The sign-in prompt +## The sign-in prompt Signing in renders the verification URL and the user code, then blocks until the user approves or the device code expires. Where that text goes, and whether the @@ -242,7 +243,7 @@ clickable, use the separately vetted browser target instead: `browser_target` on the C event struct, `event_view::browser_target()` in C++, `browser_target()` on the Rust challenge, and the `browser_target` key on the Python prompt event. -### Explicit configuration and endpoint pinning +## Explicit configuration and endpoint pinning Discovery supplies the client ID, scope, audience, token-selection mode, and endpoints. Set any of them explicitly when the application has its own @@ -252,7 +253,7 @@ needs: | Setting | Set it when | | --- | --- | | `client_id` | the application has its own registration, which is the recommended setup | -| `scope` | the provider requires `offline_access` before it issues a refresh token | +| `scope` | requesting `offline_access` for this application alone, rather than server-wide | | `audience` | the provider requires one, or QuestDB checks `aud` against [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | | `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads the provider's `/.well-known/openid-configuration` instead | | `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | @@ -325,7 +326,7 @@ client rejects it. An endpoint the client reads from the provider's own with providers that host their endpoints outside the issuer, such as Microsoft Entra ID and Google. -### Token persistence +## Token persistence Tokens stay in memory unless a file token store is enabled. The store writes the access, ID and long-lived refresh tokens as unencrypted JSON, so turn it on only @@ -389,7 +390,7 @@ files with mode `0600` and the store directory with `0700`; on other platforms the directory's existing ACL governs access, so restrict it before enabling the store. -### Interaction-required errors +## Interaction-required errors A transport call never prompts. When the cached token cannot be refreshed silently, the call fails instead, and the application has to sign in again on @@ -450,6 +451,48 @@ Java has no distinct type to match on, so an application that must recover automatically has to treat any `OidcAuthException` from a transport call as a prompt to call `signIn()` again. +## Implementing the flow yourself + +The Go, Node.js and .NET clients do not run an OIDC flow, and neither does a +tool built directly on the REST API. Acquire the token out of band, with an +OAuth2 library that implements RFC 8628 or by making the two requests yourself, +then pass it as a static bearer token and replace it before it expires. + +Read the client ID, the scope, and the endpoints from the +[settings endpoint](/docs/security/oidc/#settings-endpoint), including +`acl.oidc.device.authorization.endpoint`, which QuestDB publishes without using. + +**Request a device code**, by posting `client_id` and `scope` as +`application/x-www-form-urlencoded` to the device authorization endpoint. The +response carries `device_code`, `user_code`, `verification_uri`, `expires_in`, +an optional `verification_uri_complete` with the code already embedded, and an +optional `interval` which defaults to 5 seconds. Show the URL and the user code, +sanitized, and treat them as untrusted text. + +**Poll the token endpoint** with +`grant_type=urn:ietf:params:oauth:grant-type:device_code`, the `device_code` and +the `client_id`, waiting `interval` seconds between attempts. Four error codes +decide what happens next: + +| `error` | What to do | +| --- | --- | +| `authorization_pending` | the user has not finished; poll again after `interval` | +| `slow_down` | polling too fast; add 5 seconds to `interval`, then poll again | +| `access_denied` | the user refused; stop | +| `expired_token` | `expires_in` elapsed; start a new device authorization request | + +Polling faster than `interval`, or ignoring `slow_down`, gets the client rate +limited or blocked by the provider. + +Send the token your configuration calls for, per +[which token to send](/docs/security/oidc/#which-token-to-send): the access +token, or the ID token where group memberships are encoded in it. Keep the +`refresh_token` if the provider issues one, and redeem it with +`grant_type=refresh_token` rather than making the user sign in again. + +[RFC 8628](https://www.rfc-editor.org/rfc/rfc8628) sections 3.1 to 3.5 give the +full request and response grammar. + For the per-language API surface, the connection options, and the rest of each client's behaviour, see the client pages: [Java](/docs/connect/clients/java/#oidc-device-flow-enterprise), From a97e79c432d8f8881c0264be18ebb7d8e1ed98b7 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Thu, 3 Sep 2026 14:00:04 +0100 Subject: [PATCH 43/54] Document the token expiry check and drop the JupyterHub detour The next Enterprise release checks exp when QuestDB validates a token itself, so the warnings built around an expired token still being accepted no longer hold. Move exp into the Checked column and rewrite the caution around the caveat that survives: QuestDB never asks the provider about the token, so a revoked token keeps working until it expires. The only lever is withdrawing the signing key, which is what acl.oidc.public.keys.expiry already documents. The acl.oidc.cache.ttl note, the Entra ID walkthrough and the changelog carried the old claim too, and now agree. nbf and iss are still unchecked. The Jupyter notebook section opened with JupyterHub and OAuthenticator, which decide who may log in to JupyterHub and hand the notebook nothing to send to QuestDB. Lead with the device flow, which is the answer to how a notebook authenticates against QuestDB, and keep the JupyterHub point as the disclaimer it is. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01QUNkemSPGkrzov1yFrq6xt --- documentation/changelog.mdx | 2 +- documentation/configuration/oidc.md | 29 +++++++++++----------- documentation/security/oidc.mdx | 37 +++++++++-------------------- 3 files changed, 27 insertions(+), 41 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 47928db6ec..903348ddd3 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -92,7 +92,7 @@ This page tracks significant updates to the QuestDB documentation. - [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers - [Which token to send](/docs/security/oidc/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication -- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the audience and the presence of the claims, but not `exp`, `nbf` or `iss`, so an expired token is still accepted +- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the expiry, the audience and the presence of the claims, but not `nbf` or `iss`. QuestDB never asks the provider about such a token, so a revoked token keeps working until it expires - Scoped three [OIDC options](/docs/configuration/oidc/) to what the server actually does: `acl.oidc.audience` is only checked when the groups are encoded in the token, the mandatory `openid` in `acl.oidc.scope` is enforced by the provider rather than at startup, and `acl.oidc.ropc.flow.enabled` makes QuestDB itself exchange HTTP basic and PGWire credentials for a token at the provider - [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - [Replication setup guide](/docs/high-availability/setup/#migration-procedures) - Planned primary migration now points at the in-place switch; the restart-based flow is kept for older versions and the emergency migration is marked as the lossy path diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index fd6c32999b..27eac9c212 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -389,9 +389,10 @@ the token itself and never asks the provider about it: | Checked | Not checked | | --- | --- | -| the signature, against the public key named by the token's `kid` | `exp`, the expiry | -| `aud`, against [`acl.oidc.audience`](#acloidcaudience) | `nbf`, the not-before time | -| that `sub` and the group memberships are present | `iss`, the issuer | +| the signature, against the public key named by the token's `kid` | `nbf`, the not-before time | +| `exp`, the expiry | `iss`, the issuer | +| `aud`, against [`acl.oidc.audience`](#acloidcaudience) | | +| that `sub` and the group memberships are present | | Two things have to line up here, because two different components read the token. The signature validator deserializes the payload into a fixed structure, @@ -412,15 +413,13 @@ validator. :::caution -An expired token is therefore still accepted. Once issued, a token stays valid -for as long as the key that signed it is in QuestDB's cache, which is governed -by [`acl.oidc.public.keys.expiry`](#acloidcpublickeysexpiry) and by how long the -provider publishes the key. Neither -[`acl.oidc.cache.ttl`](#acloidccachettl) nor a shorter token lifetime in the -provider shortens it. - -Do not enable this setting where you rely on being able to revoke a token, or on -the provider's token lifetimes being enforced. +Because QuestDB never asks the provider about the token, revoking a token does +not end the access it grants: the token keeps working until its `exp` passes, +and [`acl.oidc.cache.ttl`](#acloidccachettl) does not shorten that. The only +lever is withdrawing the signing key at the provider, which cuts short every +token signed with it and takes effect within +[`acl.oidc.public.keys.expiry`](#acloidcpublickeysexpiry). Keep token lifetimes +short in the provider if you need revocation to take effect quickly. ::: @@ -456,8 +455,10 @@ request. When QuestDB checks the token locally instead, and contacts the provider only when the public keys have to be reloaded. -That local check does not test the token's expiry, so shortening this TTL does -not shorten how long an issued token is accepted. See +That local check tests the token's expiry, so an issued token is accepted until +it expires whatever this TTL is. Shortening the TTL does not make QuestDB +consult the provider, so a token revoked before its expiry is not picked up any +sooner. See [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) for what is and is not validated. diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx index 0346b80ca8..5e0af978cd 100644 --- a/documentation/security/oidc.mdx +++ b/documentation/security/oidc.mdx @@ -447,18 +447,6 @@ straight out of its payload. The [Web Console](/docs/getting-started/web-console/overview/) picks the token this way, and so should any other client. -:::caution - -With `acl.oidc.groups.encoded.in.token` enabled, QuestDB checks the token's -signature, its `aud` claim, and that `sub` and the group memberships are -present. It does not check `exp`, `nbf` or `iss`, so an expired token is still -accepted, and the provider's token lifetimes are not enforced. Do not enable the -setting where you rely on being able to revoke a token. See -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -for the full list of what is and is not validated. - -::: - ## OIDC device flow {#device-authorization-flow} For command-line applications, containers, and remote notebook kernels, where @@ -514,17 +502,14 @@ QuestDB. Select the access or ID token as described in ### Jupyter notebook -JupyterHub can integrate with OAuth2 providers using OAuthenticator, as -described in its -[documentation](https://jupyterhub.readthedocs.io/en/stable/explanation/oauth.html). -The OAuthenticator documentation also contains -[examples](https://oauthenticator.readthedocs.io/en/latest/tutorials/provider-specific-setup/index.html) -using different identity providers. +Use the official Python client's +[OIDC device flow](/docs/security/oidc-device-flow/). It works when the browser +and the notebook kernel are on different machines, and does not put the user's +password in the notebook. -If Jupyter notebooks are used without JupyterHub, use the official Python -client's [OIDC device flow](/docs/security/oidc-device-flow/). It works when -the browser and notebook kernel are on different machines and does not put the -user's password in the notebook. +Note that JupyterHub's OAuth integration (OAuthenticator) is a separate concern: +it decides who may log in to JupyterHub, and gives the notebook no token to send +to QuestDB. For tooling without device-flow support, a last-resort option is the Resource Owner Password Credentials (ROPC) flow. Enabling it usually @@ -1567,10 +1552,10 @@ This setup relies on [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), because Entra ID cannot serve the group memberships from its User Info endpoint. QuestDB then validates the ID token itself and never asks Entra ID -about it, which means it does not check the token's expiry: an issued token is -accepted for as long as the key that signed it stays in QuestDB's cache. Read -that option before going to production, and note that clients must send the ID -token here, not the access token. +about it, so revoking a token in Entra ID does not end the access it grants: +the token keeps working until it expires. Read that option before going to +production, and note that clients must send the ID token here, not the access +token. ::: From 1d789cf16bdd9185acb299d6c38caa8cc3be805c Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 16:20:04 +0100 Subject: [PATCH 44/54] Split the OIDC page into a section The single oidc.mdx had grown to 1600 lines covering the architecture, the sign-in walkthrough, client discovery, integration patterns, group mapping and two provider setups. Each of those is now its own page under security/oidc/, with the overview at the section root and a "where each section moved" map for the old anchors. Inbound links across the configuration, client and deployment docs are repointed at the new anchors. Also corrects the group claim rules in the configuration reference. The token validator requires sub, aud and exp and never looks at the group memberships, so acl.oidc.groups.claim may name any claim the token carries rather than being pinned to "groups", and a bare string is accepted where a provider sends a lone group. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014Cz9Rp9SBqLYSUsFkGFbGN --- documentation/changelog.mdx | 8 +- documentation/configuration/http-server.md | 2 +- documentation/configuration/oidc.md | 47 +- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 4 +- documentation/connect/clients/nodejs.md | 4 +- .../connect/compatibility/rest-api.md | 2 +- .../wire-protocols/qwp-ingress-websocket.md | 4 +- documentation/deployment/azure.md | 2 +- documentation/high-availability/failover.md | 2 +- documentation/security/oidc-device-flow.mdx | 12 +- documentation/security/oidc.mdx | 1601 ----------------- .../security/oidc/client-discovery.mdx | 127 ++ .../security/oidc/client-integration.mdx | 378 ++++ documentation/security/oidc/entra-id.mdx | 286 +++ documentation/security/oidc/group-mapping.mdx | 119 ++ .../security/oidc/how-sign-in-works.mdx | 269 +++ documentation/security/oidc/index.mdx | 143 ++ documentation/security/oidc/pingfederate.mdx | 380 ++++ documentation/sidebars.js | 54 +- 20 files changed, 1792 insertions(+), 1654 deletions(-) delete mode 100644 documentation/security/oidc.mdx create mode 100644 documentation/security/oidc/client-discovery.mdx create mode 100644 documentation/security/oidc/client-integration.mdx create mode 100644 documentation/security/oidc/entra-id.mdx create mode 100644 documentation/security/oidc/group-mapping.mdx create mode 100644 documentation/security/oidc/how-sign-in-works.mdx create mode 100644 documentation/security/oidc/index.mdx create mode 100644 documentation/security/oidc/pingfederate.mdx diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 903348ddd3..d6ae54faa6 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -43,7 +43,7 @@ This page tracks significant updates to the QuestDB documentation. - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another - [OIDC device flow](/docs/security/oidc-device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients -- [OIDC settings endpoint](/docs/security/oidc/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required +- [OIDC settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` @@ -90,11 +90,11 @@ This page tracks significant updates to the QuestDB documentation. ### Updated -- [OpenID Connect (OIDC)](/docs/security/oidc/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers -- [Which token to send](/docs/security/oidc/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication +- [OpenID Connect (OIDC)](/docs/security/oidc/how-sign-in-works/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication - [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the expiry, the audience and the presence of the claims, but not `nbf` or `iss`. QuestDB never asks the provider about such a token, so a revoked token keeps working until it expires - Scoped three [OIDC options](/docs/configuration/oidc/) to what the server actually does: `acl.oidc.audience` is only checked when the groups are encoded in the token, the mandatory `openid` in `acl.oidc.scope` is enforced by the provider rather than at startup, and `acl.oidc.ropc.flow.enabled` makes QuestDB itself exchange HTTP basic and PGWire credentials for a token at the provider -- [PingFederate SSO](/docs/security/oidc/#questdb-configuration-for-pingfederate) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none +- [PingFederate SSO](/docs/security/oidc/pingfederate/#questdb-configuration) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - [Replication setup guide](/docs/high-availability/setup/#migration-procedures) - Planned primary migration now points at the in-place switch; the restart-based flow is kept for older versions and the emergency migration is marked as the lossy path - [Error codes](/docs/troubleshooting/error-codes/#er005) - ER005 now covers the refusal of an in-place promotion, and ER006 the restart of a demoted node with a stale `replication.role` - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index 76e968d7a6..01baf22037 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -521,7 +521,7 @@ Context path for the file import service. Context path for the service that serves server-side settings to clients, such as an OIDC client discovering the provider's endpoints from the -[settings endpoint](/docs/security/oidc/#settings-endpoint). +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). Accepts a comma-separated list of paths. Setting it adds paths rather than moving the service: QuestDB keeps serving the default path as well, so the diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 27eac9c212..e2e06b088d 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -159,7 +159,7 @@ fails at the provider rather than at startup. QuestDB uses the scopes in the requests it makes itself, in the [ROPC flow](#acloidcropcflowenabled), and publishes them on the -[settings endpoint](/docs/security/oidc/#settings-endpoint) for clients which +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) for clients which run the flow themselves. For the [OIDC device flow](/docs/security/oidc-device-flow/), add @@ -175,7 +175,7 @@ using it, untouched. QuestDB publishes [`acl.oidc.pkce.required`](#acloidcpkcerequired) and [`acl.oidc.state.required`](#acloidcstaterequired) to clients through the -[settings endpoint](/docs/security/oidc/#settings-endpoint), and enforces +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), and enforces neither. The client generates the code verifier and the `state` value; the provider checks the verifier, and the client checks the `state` value it gets back. @@ -219,7 +219,7 @@ the provider. Unlike [`acl.oidc.pkce.required`](#acloidcpkcerequired) and [`acl.oidc.state.required`](#acloidcstaterequired), this setting is not published on the -[settings endpoint](/docs/security/oidc/#settings-endpoint), so a client cannot +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so a client cannot discover whether the flow is available. ### acl.oidc.state.required @@ -234,7 +234,7 @@ requires the `state` parameter, or to add CSRF protection on top of PKCE. The [Web Console](/docs/getting-started/web-console/overview/) generates the value, sends it in the authorization request, and checks that the provider returns it unchanged. See -[Secret generation](/docs/security/oidc/#1-secret-generation). +[Secret generation](/docs/security/oidc/how-sign-in-works/#1-secret-generation). ## Endpoints @@ -258,7 +258,7 @@ Identity Platform. OIDC Device Authorization Endpoint. Unlike the other endpoint settings this one has no default, and QuestDB never calls it. QuestDB resolves the endpoint and publishes it on the -[settings endpoint](/docs/security/oidc/#settings-endpoint), for clients which +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), for clients which implement the [OIDC device flow](/docs/security/oidc-device-flow/) themselves. The official Java, Python, Rust, C, and C++ clients can discover and @@ -361,11 +361,12 @@ group memberships of the user. Required when OIDC is enabled. If the claim is missing from the user information, or it is an empty list, authentication fails. See -[Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). -The name applies to both flows. Set it to `groups` when -[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, -because the token validator accepts no other name for the group memberships. +The name applies to both flows, including when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`. +The claim's value is normally an array of group names. A single bare string is +accepted too, which is how some providers send a lone group. ### acl.oidc.groups.encoded.in.token @@ -379,8 +380,8 @@ group memberships directly into the token. This also changes which token the client has to send: the ID token when the setting is `true`, the access token when it is `false`. QuestDB publishes the setting on the -[settings endpoint](/docs/security/oidc/#settings-endpoint) so that clients can -[pick the right one](/docs/security/oidc/#which-token-to-send). +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) so that clients can +[pick the right one](/docs/security/oidc/client-discovery/#which-token-to-send). It changes how tokens are validated too. In the default user info flow QuestDB hands the token to the OIDC Provider on every cache miss, so the provider @@ -394,22 +395,20 @@ the token itself and never asks the provider about it: | `aud`, against [`acl.oidc.audience`](#acloidcaudience) | | | that `sub` and the group memberships are present | | -Two things have to line up here, because two different components read the -token. The signature validator deserializes the payload into a fixed structure, -so the JWT must carry claims named `sub` and `groups` literally, and `groups` -must be an array of strings. A token without both is rejected whatever the claim -settings say. QuestDB then reads the principal and the group memberships out of -that same payload using +Two components read the token. The signature validator requires the JWT to carry +`sub`, `aud` and `exp`, and checks the signature, the audience and the expiry +against them. It does not look at the group memberships. QuestDB then reads the +principal and the groups out of that same payload using [`acl.oidc.sub.claim`](#acloidcsubclaim) and [`acl.oidc.groups.claim`](#acloidcgroupsclaim), exactly as it does in the user info flow. -So set `acl.oidc.groups.claim=groups`, since that is the only name the validator -accepts for the group memberships. `acl.oidc.sub.claim` may name any claim the -token carries: a provider which issues both `sub` and a friendlier claim lets -you keep the readable one as the principal. The Entra ID walkthrough does this, -setting `acl.oidc.sub.claim=name` while the token still carries `sub` for the -validator. +So `acl.oidc.groups.claim` may name any claim the token carries, `roles` for +example. `acl.oidc.sub.claim` may name any claim too, though the token must +still carry `sub` itself for the validator: a provider which issues both `sub` +and a friendlier claim lets you keep the readable one as the principal. The +Entra ID walkthrough does this, setting `acl.oidc.sub.claim=name` while the +token still carries `sub` for the validator. :::caution @@ -435,7 +434,7 @@ logged for audit purposes. If the claim is missing from the user information, or empty, authentication fails. The same applies to the claim named by [`acl.oidc.groups.claim`](#acloidcgroupsclaim). See -[Mapping user permissions](/docs/security/oidc/#mapping-user-permissions). +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). ## Caching and buffers diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index 40ffed6253..fcc4a71fb2 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index b8f25cef61..3964a03968 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -364,9 +364,9 @@ Acquire it out of band, since QuestDB Enterprise issues bearer tokens through its [OpenID Connect flow](/docs/security/oidc/), and manage its lifetime yourself. QuestDB publishes the provider's authorization and token endpoints on its -[settings endpoint](/docs/security/oidc/#settings-endpoint), so a client can +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so a client can discover them instead of hard coding them. The same response tells the client -[which token to send](/docs/security/oidc/#which-token-to-send): the access +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access token, or the ID token when QuestDB reads group memberships from the token. This client does not run an OIDC flow, so acquire the token with an OAuth2 library: `golang.org/x/oauth2` implements the device grant, and QuestDB diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index 82f636bc9b..a8e38e26c1 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -94,8 +94,8 @@ info. This client does not run an OIDC flow. QuestDB Enterprise also accepts an OIDC bearer token, which you acquire out of band from your Identity Provider: QuestDB publishes the provider's endpoints on its -[settings endpoint](/docs/security/oidc/#settings-endpoint), and the same -response tells you [which token to send](/docs/security/oidc/#which-token-to-send). +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), and the same +response tells you [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). See [OpenID Connect](/docs/security/oidc/), and [implementing the flow yourself](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) for the device grant's requests and polling rules. diff --git a/documentation/connect/compatibility/rest-api.md b/documentation/connect/compatibility/rest-api.md index c1e6a1b573..2b3fb0884e 100644 --- a/documentation/connect/compatibility/rest-api.md +++ b/documentation/connect/compatibility/rest-api.md @@ -803,7 +803,7 @@ The REST API supports three authentication types: travel in the `Authorization: Bearer` header but are issued by an external Identity Provider rather than by QuestDB, so they are a different credential from the REST API token below. See [OpenID Connect](/docs/security/oidc/) and - [which token to send](/docs/security/oidc/#which-token-to-send). + [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). :::note diff --git a/documentation/connect/wire-protocols/qwp-ingress-websocket.md b/documentation/connect/wire-protocols/qwp-ingress-websocket.md index 0d64562109..f20ebf96fa 100644 --- a/documentation/connect/wire-protocols/qwp-ingress-websocket.md +++ b/documentation/connect/wire-protocols/qwp-ingress-websocket.md @@ -144,9 +144,9 @@ Supported methods: - **Token-based auth** (Enterprise only): see [Authentication via token in QuestDB Enterprise](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise). - **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). The - [settings endpoint](/docs/security/oidc/#settings-endpoint) publishes the + [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) publishes the provider's authorization and token endpoints, and tells the client - [which token to send](/docs/security/oidc/#which-token-to-send). + [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). A failed authentication results in a `401` or `403` HTTP response before the WebSocket connection is established. No QWP-level auth handshake exists. diff --git a/documentation/deployment/azure.md b/documentation/deployment/azure.md index 8351978eab..ebbb8ad5e6 100644 --- a/documentation/deployment/azure.md +++ b/documentation/deployment/azure.md @@ -345,6 +345,6 @@ QuestDB Enterprise adds production features for Azure: - **EntraID SSO** - Single sign-on with Microsoft Entra ID For EntraID integration, see the -[Microsoft EntraID OIDC guide](/docs/security/oidc/#microsoft-entraid). +[Microsoft EntraID OIDC guide](/docs/security/oidc/entra-id/). See [Enterprise Quick Start](/docs/getting-started/enterprise-quick-start/) for setup. diff --git a/documentation/high-availability/failover.md b/documentation/high-availability/failover.md index 4b6379e718..fdc3265be9 100644 --- a/documentation/high-availability/failover.md +++ b/documentation/high-availability/failover.md @@ -286,7 +286,7 @@ same credentials as the main HTTP server: HTTP basic authentication, a [REST token](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise) as `Authorization: Bearer`, or an OIDC token. Which of the two OIDC tokens to send is decided by `acl.oidc.groups.encoded.in.token`; see -[which token to send](/docs/security/oidc/#which-token-to-send). The principal needs the +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). The principal needs the `HTTP` endpoint permission and, for the switch, `SWITCH ROLE`. With access control disabled no credentials are needed. TLS for this port is configured with the `http.min.tls.*` settings on the [TLS](/docs/configuration/tls/) page. diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 9ee4048a43..6f0886da21 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -57,7 +57,7 @@ sequenceDiagram The client must respect the polling interval, `slow_down` responses, and the device code's expiry. The official clients handle those protocol details, choose the access or ID token according to -[`acl.oidc.groups.encoded.in.token`](/docs/security/oidc/#which-token-to-send), cache it in memory, +[`acl.oidc.groups.encoded.in.token`](/docs/security/oidc/client-discovery/#which-token-to-send), cache it in memory, and refresh it silently when the provider issues a refresh token. They never send the device code or the user's Identity Provider password to QuestDB. @@ -100,13 +100,13 @@ Clients reach QuestDB over `https` for discovery, so the HTTP endpoint needs TLS enabled with [`http.tls.enabled`](/docs/configuration/tls/#httptlsenabled) and a certificate. For setting OIDC up against a specific provider, the OIDC guide has walkthroughs for -[Microsoft Entra ID](/docs/security/oidc/#microsoft-entraid) and -[PingFederate](/docs/security/oidc/#pingfederate). +[Microsoft Entra ID](/docs/security/oidc/entra-id/) and +[PingFederate](/docs/security/oidc/pingfederate/). ## Official client examples Each example discovers the OIDC client ID, scope, token-selection mode, and -provider endpoints from QuestDB's [settings endpoint](/docs/security/oidc/#settings-endpoint), then +provider endpoints from QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), then signs in before opening the database connection. Discovering the client ID means reusing QuestDB's own registration, which keeps @@ -459,7 +459,7 @@ OAuth2 library that implements RFC 8628 or by making the two requests yourself, then pass it as a static bearer token and replace it before it expires. Read the client ID, the scope, and the endpoints from the -[settings endpoint](/docs/security/oidc/#settings-endpoint), including +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), including `acl.oidc.device.authorization.endpoint`, which QuestDB publishes without using. **Request a device code**, by posting `client_id` and `scope` as @@ -485,7 +485,7 @@ Polling faster than `interval`, or ignoring `slow_down`, gets the client rate limited or blocked by the provider. Send the token your configuration calls for, per -[which token to send](/docs/security/oidc/#which-token-to-send): the access +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access token, or the ID token where group memberships are encoded in it. Keep the `refresh_token` if the provider issues one, and redeem it with `grant_type=refresh_token` rather than making the user sign in again. diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx deleted file mode 100644 index 5e0af978cd..0000000000 --- a/documentation/security/oidc.mdx +++ /dev/null @@ -1,1601 +0,0 @@ ---- -title: OpenID Connect (OIDC) Integration -description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, device-flow client authentication, PGWire token authentication, and group mapping. ---- - -import Screenshot from "@theme/Screenshot"; -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - OpenID Connect (OIDC) enables SSO authentication with external Identity Providers. - - -OpenID Connect (OIDC) integrates with Identity Providers (IdP) external to -QuestDB. - -It is a convenient way to integrate QuestDB into your enterprise environment, -and it provides SSO (Single Sign-On) for the [Web Console](/docs/getting-started/web-console/overview/). - -Microsoft Active Directory and Azure AD, for example, can be turned into an -Identity Provider. - -Specific installation steps depend on the type of the provider. - -## Architecture overview - -Altogether, the architecture appears as such: - - - -We can break it down into core components. - -### Web Console - -QuestDB's interactive UI. Users must authenticate before accessing the database -via the interface. - -The [Web Console](/docs/getting-started/web-console/overview/) uses PKCE (Proof Key for Code Exchange) to -secure the authentication and authorization flow. - -In OAuth2/OIDC terms, the [Web Console](/docs/getting-started/web-console/overview/) is referred to as -the _client_, and it is assigned an identifier: the **Client Id**. - -Each application which integrates via OIDC should be given a different **Client -Id**. - -### OIDC Provider - -Typically consists of a number of modules. - -We are interested in two of them only. - -1. The _Identity Provider_ holds user identities and user information, capable - of authenticating users, and to issue an ID Token which uniquely identify - them. - -2. The _Authorization Server_ grants access to resources, such as a database, in - the form of access tokens. - -The OIDC Provider usually integrates with a number of applications which require -different access to a number of resources. - -These clients communicate with the OIDC Provider via its endpoints. - -It exposes a number of APIs, including the Authorization, Token and User Info -endpoints. - -### QuestDB - -The database, in OAuth2/OIDC terms the _protected resource_ or _resource -server_. - -Only processes requests which contain the bearer token selected by its OIDC -configuration: normally the access token, or the ID token when group memberships -are encoded in it. - -## Authentication and Authorization Flow - -The OAuth2/OIDC standard defines different ways of obtaining access and ID -tokens from the OIDC Provider, referred to as the "_flow_". - -The goal of this flow is to get the user, who is sitting in front of the Web -Console, authenticated. - -Then, it allows QuestDB to determine the user's permissions based on user -information provided by the Identity Providers. - -Specifically, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) uses the -`Authorization Code Flow with PKCE` option. - -It consists of ten steps... - -### 1. Secret generation - -First the [Web Console](/docs/getting-started/web-console/overview/) generates a cryptographically strong -random secret called the _code verifier_. - -The secret is hashed using the _SHA256 algorithm_. The result is the _code -challenge_. - -After PKCE initialization the [Web Console](/docs/getting-started/web-console/overview/) requests an -_authorization code_ from the OIDC Provider. - -It calls the Authorization endpoint with a few parameters, including the: - -- **Client Id** -- requested scopes (the list of scopes are configurable, default is `openid` - only) -- code challenge -- algorithm used to generate the code challenge from the code verifier (SHA256) - -When the Authorization Server receives the request, it checks if the user has -been authenticated already: - -- If the user has a valid session, it can be provided with an authorization code - straight away, so we jump to step 4. - -- If the user does not have a valid session yet, it will be redirected to the - Identity Provider for authentication. - -```bash title="Authorization code request example" -https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 -``` - -:::note - -Some Identity Providers require the `state` parameter in the authorization -request. It is another random value generated by the client, which the OIDC -Provider returns unchanged together with the authorization code. Checking it -protects against CSRF attacks. - -If your provider requires it, set -[`acl.oidc.state.required`](/docs/configuration/oidc/#acloidcstaterequired) to -`true`. The [Web Console](/docs/getting-started/web-console/overview/) then -generates the `state` parameter, sends it in the authorization request, and -validates the value returned by the provider. - -::: - -### 2. Prove identity - -Next, the user must prove its identity. - -This could be a username with: - -- a password, -- an OTP -- facial recognition via a mobile app -- or anything else supported by the Identity Provider. - - - -### 3. Scope consent - -After successful authentication, the user provides consent for the requested -scopes. - -The list of scopes are configurable. - -By default the Web Console requests only the `openid` scope which is mandatory -for OIDC. - -No ID Token is issued without it. - -The OIDC provider can be configured to provide the consent automatically, -without presenting the user with an additional screen in the browser. - - - -### 4. Redirection - -Consent is granted! - -The Authorization Server redirects the user back to the -[Web Console](/docs/getting-started/web-console/overview/) with the _authorization code_: - -```bash title="Authorization code response example" -https://questdb.example.com:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A -``` - -### 5. Credential request - -Now, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) requests the ID and access -tokens from the Token endpoint of the OIDC Provider with the authorization code. - -It includes the Client ID and the PKCE code verifier together with the -authorization code in the request. - -The endpoint then hashes the code verifier using the method specified previously -in step 1. - -The result must match the code challenge, also provided in step 1. - -The matching code challenge proves that the token is requested by the client -which requested the authorization code, and it was not stolen: - -```bash title="Token request example" -POST https://oidc.provider:443/as/token.oauth2 HTTP/1.1 -Content-Type: application/x-www-form-urlencoded -grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF -``` - -### 6. Credentials received - -If the PKCE check is passed, the Web Console receives the ID and access tokens. - -There is a third token in the response too, the refresh token. - -The refresh token is used by the Web Console to refresh the access token before -it expires. - -Without the refresh token mechanism, the user would be forced to re-authenticate -when the access token expires. - -The validity of the tokens are configurable inside the OIDC Provider. - -```json title="Token response example" -{ - "access_token": "gslpJtzmmi6RwaPSx0dYGD4tEkom", - "refresh_token": "FUuAAqMp6LSTKmkUd5uZuodhiE4Kr6M7Eyv.eg83ge", - "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...", - "token_type": "Bearer", - "expires_in": 300 // In seconds, thus 5 minutes -} -``` - -### 7. Database access - -With the tokens, the Web Console can interact with the database. - -A token is in the header of every request sent to QuestDB. Which of the two goes -there depends on where QuestDB reads the group memberships from, see -[Which token to send](#which-token-to-send). - -:::note - -Worried about exposing the token? An access token is usually opaque and carries -no user details, though some providers issue a JWT here too. An ID token always -does: it is a JWT whose payload is base64 encoded, not encrypted, so treat it as -you would the user's directory record. - -::: - -To carry out permission checks, the database has to know more about the user. - -For this, QuestDB has a User Info Cache. - -Steps 8 and 9 describe the default flow, where QuestDB validates the access -token by asking the OIDC Provider about it. They are skipped when the cache -already holds a valid entry for the token, and they do not happen at all when -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -is `true`, in which case QuestDB validates the ID token itself and never calls -the provider. See [Which token to send](#which-token-to-send). - -```bash title="Query request example" -https://questdb.example.com:9000/exec?query=select%20current_user() -Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom -``` - -### 8. Find user information - -No user information in the cache, or stale information? - -QuestDB uses the access token to request user information from the OIDC -Provider's User Info endpoint. - -This call also serves as token validation. - -If the token is not real or has been expired, the User Info endpoint replies -with an error: - -```bash title="User info request example" -https://oidc.provider:443/idp/userinfo.openid -Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom -``` - -### 9. Receive user information - -If the access token is valid, QuestDB receives the required user information -from the endpoint, then updates its cache. - -The cache improves performance, as QuestDB does not have to turn to the OIDC -Provider on every single request. - -Do note that cache expiry is configurable: - -```json title="User info response example with Active Directory groups" -{ - "sub": "externalUser", - "name": "External User", - "groups": [ - "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", - "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" - ] -} -``` - -### 10. Permission check - -With the help of the user information, QuestDB can carry out -[permission checks](#user-permissions). - -If the permission check is successful, the database will process the request, -and then sends the results back: - -```json title="Query response example" -{ - "query": "select current_user()", - "columns": [ - { - "name": "current_user", - "type": "STRING" - } - ], - "dataset": [["External User"]], - "count": 1, - "timestamp": -1 -} -``` - -## Settings endpoint - -QuestDB publishes the parts of its OIDC configuration that a client needs in -order to start an authentication flow. Any client can read them from the -settings endpoint, which requires no authentication: - -```bash title="Settings request example" -curl https://questdb.example.com:9000/settings -``` - -The OIDC-related entries of the response look like this, alongside other server -settings: - -```json title="Settings response example" -{ - "config": { - "acl.enabled": true, - "acl.basic.auth.realm.enabled": false, - "acl.oidc.enabled": true, - "acl.oidc.pkce.required": true, - "acl.oidc.state.required": false, - "acl.oidc.groups.encoded.in.token": false, - "acl.oidc.client.id": "questdb", - "acl.oidc.redirect.uri": "https://questdb.example.com:9000", - "acl.oidc.scope": "openid", - "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", - "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", - "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" - }, - "preferences.version": 0, - "preferences": {} -} -``` - -Each key is the name of the -[configuration option](/docs/configuration/oidc/) it carries. The -[Web Console](/docs/getting-started/web-console/overview/) reads them to build -its authorization request, and any other client can do the same instead of -hard coding the provider's details. - -| Key | Type | Present | -| --- | --- | --- | -| `acl.enabled` | boolean | always | -| `acl.basic.auth.realm.enabled` | boolean | always | -| `acl.oidc.enabled` | boolean | always | -| `acl.oidc.pkce.required` | boolean | always | -| `acl.oidc.state.required` | boolean | always | -| `acl.oidc.groups.encoded.in.token` | boolean | always | -| `acl.oidc.client.id` | string or `null` | always | -| `acl.oidc.redirect.uri` | string or `null` | always | -| `acl.oidc.scope` | string | always | -| `acl.oidc.authorization.endpoint` | string | when the endpoint is resolved | -| `acl.oidc.token.endpoint` | string | when the endpoint is resolved | -| `acl.oidc.device.authorization.endpoint` | string | when the endpoint is configured or discovered | - -The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the -[endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's -configuration document when `acl.oidc.configuration.url` is set. -[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) -is present when a device authorization endpoint has been configured, or when the -provider advertises one. QuestDB publishes it without using it, for clients -which implement the [OIDC device flow](/docs/security/oidc-device-flow/) -themselves. - -`acl.oidc.redirect.uri` is `null` unless -[`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. -A client should then fall back to its own location, as the Web Console does. - -QuestDB advertises `acl.oidc.pkce.required` and `acl.oidc.state.required`, and -enforces neither. The client generates the code verifier and the `state` value; -the provider checks the verifier, and the client checks the `state` value it -gets back. - -:::note - -Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries -are published whether or not OIDC is enabled. With OIDC disabled and -`acl.oidc.host` in use, the endpoint URLs are built from the defaults rather -than from a real provider; with `acl.oidc.configuration.url` set they are not -resolved at all, and the endpoint keys are absent from the response. - -OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be -`true`, which the same response carries. When access control is disabled, -`acl.oidc.enabled` still reports the configured value while no OIDC -authentication takes place, so read both keys. - -::: - -The path of the endpoint is set by -[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). -Setting it adds paths rather than moving the endpoint, so the default path keeps -working too. That default is `/settings` only while -[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) -is at its default, as the settings path follows the Web Console context path. -A client which cannot assume both are at their defaults should make the path -configurable. - -## Which token to send - -`acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned -by the provider belongs in the `Authorization: Bearer` header: - -| Value | Token to send | How QuestDB validates it | -| --- | --- | --- | -| `false`, the default | the access token | by calling the user info endpoint | -| `true` | the ID token | locally, by checking its signature, audience, and claims | - -Sending the wrong one fails authentication with `401 Unauthorized` and the -reason in the server log. With -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -enabled the token has to be a JWT, because QuestDB reads the group memberships -straight out of its payload. The -[Web Console](/docs/getting-started/web-console/overview/) picks the token this -way, and so should any other client. - -## OIDC device flow {#device-authorization-flow} - -For command-line applications, containers, and remote notebook kernels, where -no browser redirect can reach the client, QuestDB Enterprise supports the -OAuth 2.0 Device Authorization Grant. QuestDB does not run the flow: the client -talks to the Identity Provider directly and presents the resulting bearer -token. - -See [OIDC device flow](/docs/security/oidc-device-flow/) for the -protocol walkthrough, what to configure first, and worked examples for the -Java, Python, Rust, C and C++ clients. - -## Interactive clients - -Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an -OIDC provider. However, the level of support will vary between these tools. - -Interactive clients usually fall into one of the following categories: - -- Browser-based clients with support for HTTP redirects; this includes the Web - Console or any javascript UI -- Applications running in a browser without support for redirects, such as - Jupyter notebooks -- Non-browser based clients, usually some kind of command line interface (CLI) - or a standalone application, such as Microsoft Access - -### Browser-based clients - -If the tool is browser based and can handle HTTP redirects, it can implement two -possible flows to request an access token.: - -1. [Authorization Code](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth) - flow **(Recommended, more secure)** - -2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) - flow **(deprecated)**. It cannot use PKCE, so it does not satisfy - [`acl.oidc.pkce.required`](/docs/configuration/oidc/#acloidcpkcerequired), - which is `true` by default. - -The client can read the authorization and token endpoints, the client id and -the scopes from the [settings endpoint](#settings-endpoint) instead of hard -coding them. - -The Web Console implements the -[Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a -special version of the Authorization Code flow designed for mobile apps and -single page applications. - -Regardless of which flow is used by the web or mobile application, the requested -token can be used for authentication and authorization when communicating with -QuestDB. Select the access or ID token as described in -[Which token to send](#which-token-to-send). - -### Jupyter notebook - -Use the official Python client's -[OIDC device flow](/docs/security/oidc-device-flow/). It works when the browser -and the notebook kernel are on different machines, and does not put the user's -password in the notebook. - -Note that JupyterHub's OAuth integration (OAuthenticator) is a separate concern: -it decides who may log in to JupyterHub, and gives the notebook no token to send -to QuestDB. - -For tooling without device-flow support, a last-resort option is the -Resource Owner Password Credentials (ROPC) flow. Enabling it usually -requires additional provider configuration. - -:::caution - -The Resource Owner Password Credentials flow is legacy, and should be used as a last resort. - -::: - -The ROPC snippets below assume -`acl.oidc.groups.encoded.in.token=false` and therefore send the access token. If -the setting is `true`, the client must send the ID token instead, and the -provider must issue one for the password grant. The official device-flow client -selects the correct token automatically. - -As a fallback, the code below acquires an access token with ROPC: - -```python -from urllib import request, parse -import json - -url = "https://oidc.provider:443/as/token.oauth2" -data = parse.urlencode( { - "grant_type": "password", - "username": "testuser", - "password": "testpwd", - "scope": "openid", - "client_id": "testclient" -} ).encode() -req = request.Request(url=url, data=data) -req.add_header("Content-Type", "application/x-www-form-urlencoded") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - access_token = resp["access_token"] -``` - -This token can be used to authenticate with QuestDB: - -```python -query = parse.urlencode({ - "query": "select current_user()" -}) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") -req.add_header("Authorization", f"Bearer {access_token}") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - print(resp) -``` - -#### Externalizing credentials - -The above example saves the user's credentials into the notebook, potentially -exposing them to others. One way to improve this is to use environment variables -or files to externalize the username and password. - -Here is an example using the `dotenv` library. - -First we need to create a file named `.env` with the settings: - -```ini title=".env" -username=testuser -password=testpwd -``` - -Then load it in our notebook, and use it to request tokens: - -```python -from dotenv import load_dotenv -import os -from urllib import request, parse -import json - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -url = "https://oidc.provider:443/as/token.oauth2" -data = parse.urlencode( { - "grant_type": "password", - "username": user, - "password": pwd, - "scope": "openid", - "client_id": "testclient" -} ).encode() -req = request.Request(url=url, data=data) -req.add_header("Content-Type", "application/x-www-form-urlencoded") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - access_token = resp["access_token"] -``` - -#### Enable ROPC - -The Resource Owner Password Credentials flow can be enabled in QuestDB within -`server.conf`: - -```ini title="server.conf" -acl.oidc.ropc.flow.enabled=true -``` - -> Note that the flow also has to be configured in the OAuth2/OIDC provider! - -Now we can use Basic Authentication to simplify our code. We send the -credentials to QuestDB, and the database will validate the credentials against -the OAuth2 provider. - -```python -from dotenv import load_dotenv -import os -from urllib import request -import base64 - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -query = parse.urlencode({ - "query": "select current_user()" -}) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") -b64credentials = base64.standard_b64encode(f"{user}:{pwd}".encode()).decode() -req.add_header("Authorization", f"Basic {b64credentials}") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - print(resp) -``` - -We can also use a postgres client to connect to the database: - -:::note - -QuestDB never persists the user's credentials. - -::: - -```python -import psycopg as pg -from dotenv import load_dotenv -import os - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -conn_str = f"user={user} password={pwd} host=localhost port=8812 dbname=qdb" -with pg.connect(conn_str, autocommit=True) as connection: - with connection.cursor() as cur: - cur.execute("select current_user()") - records = cur.fetchall() - for row in records: - print(row) -``` - -### CLI, standalone applications - -For a CLI or standalone application built with an official Java, Python, Rust, -C, or C++ client, use the -[OIDC device flow](/docs/security/oidc-device-flow/). -Tools such as `psql` and Microsoft Access cannot run the flow themselves; for -those tools, acquire a token separately and use -[PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the -legacy Resource Owner Password Credentials flow. - -The user logs in with their SSO credentials, and the server validates the -details with the OAuth2 provider: - -```shell -% psql -h localhost -p 8812 -U testuser -Password for user testuser: -psql (14.2, server 11.3) -Type "help" for help. - -testldap=> -testldap=> -``` - -## Non-interactive clients - -Non-interactive clients are usually jobs or standalone applications, such as a -client for ingesting data. It is practical to manage their credentials via an -OAuth2 provider too. - -As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion. QuestDB publishes the -provider's token endpoint on the [settings endpoint](#settings-endpoint), so it -does not have to be hard coded, and the same response tells the client -[which token to send](#which-token-to-send). - -The client id and the scopes stay the client's own. `acl.oidc.client.id` and -`acl.oidc.scope` describe QuestDB's registration with the provider, not the -job's. As noted in [Architecture overview](#architecture-overview), each -application which integrates via OIDC should be given a different Client Id, so -that the two can be told apart in the provider's audit log and given different -policies. - -:::caution - -Both requests must go over `https`. The settings response names the URL this -code posts the user's password to, so anyone able to tamper with a plaintext -response can redirect the password grant to a host of their choosing. The same -applies to the connection below, which carries the resulting token. - -::: - -```python -import os -import requests -import pandas as pd -import questdb -from dotenv import load_dotenv - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -# this job's own registration in the OIDC Provider, not QuestDB's; -# the openid scope is what makes the provider issue an ID token -client_id = os.environ.get("client_id") -scope = "openid" - -settings = requests.get("https://questdb.example.com:9000/settings").json()["config"] -if not settings.get("acl.oidc.enabled"): - raise SystemExit("OIDC is not enabled on this QuestDB instance") - -# the endpoint keys are absent until QuestDB resolves them -token_endpoint = settings.get("acl.oidc.token.endpoint") -if not token_endpoint: - raise SystemExit("QuestDB has not resolved the provider's token endpoint") - -response = requests.post(token_endpoint, - data={"grant_type": "password", - "client_id": client_id, - "username": user, - "password": pwd, - "scope": scope}, - headers={"Content-Type": "application/x-www-form-urlencoded"}) -response.raise_for_status() -tokens = response.json() - -# QuestDB reads the group memberships from the ID token when this is enabled -token_key = "id_token" if settings["acl.oidc.groups.encoded.in.token"] else "access_token" -if token_key not in tokens: - raise SystemExit(f"the provider did not issue an {token_key} for the password grant") -token = tokens[token_key] - -conf = f"wss::addr=questdb.example.com:9000;token={token};" -with questdb.connect(conf) as db: - df = pd.read_csv("trades.csv") - df["timestamp"] = pd.to_datetime(df["timestamp"]) - db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") -``` - -:::note - -The Resource Owner Password Credentials flow returns an ID token only if the -provider issues one for the password grant, and only when `openid` is among the -requested scopes. Check that the provider does before relying on -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -with a non-interactive client. - -::: - -Alternatively, a user may rely on QuestDB to authenticate them via the OAuth2 -provider when the Resource Owner Password Credentials flow is enabled on the -server side: - -```python -import os -import pandas as pd -import questdb -from dotenv import load_dotenv - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -conf = f"wss::addr=questdb.example.com:9000;username={user};password={pwd};" -with questdb.connect(conf) as db: - df = pd.read_csv("trades.csv") - df["timestamp"] = pd.to_datetime(df["timestamp"]) - db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") -``` - -## OIDC for the PGWire endpoint - -If the -Resource Owner Password Credentials (ROPC) flow is not an option, we can still authenticate via OIDC on the PGWire endpoint. -However, in this case the client's responsibility to source the token required for authentication. -This method works wherever a Postgres client library is available, including jupyter notebooks. - -Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. - -The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option. Send the token -selected by [Which token to send](#which-token-to-send): the access token by -default, or the ID token when group memberships are encoded in it. - -```python -import psycopg as pg - -token = "token_requested_from_the_oauth2_provider" - -conn_str = f"user=_sso password={token} host=localhost port=8812 dbname=qdb" -with pg.connect(conn_str, autocommit=True) as connection: - with connection.cursor() as cur: - cur.execute('select current_user()') - records = cur.fetchall() - for row in records: - print(row) -``` - -## User permissions - -QuestDB requires additional user information to be able to construct the user's -access list. - -As a reminder, the access list is the list of permissions that determines what -the user can and cannot do. - -QuestDB itself does not store external users, nor their passwords or any other -authentication related detail. - -External users and their authentication methods are managed by the Identity -Provider. - -Since external users are not managed by QuestDB, permissions cannot be granted -to them directly. - -Instead, the database expects a list of groups, called the _groups claim_ to be -present in the user information. - -These external group names are mapped to QuestDB's own groups. - -The access list of the external user consists of the permissions granted to -those groups: - - - -### Mapping user permissions - -The mappings between external and QuestDB groups are managed with the following -SQL commands: - -```questdb-sql title="Create a group which is mapped to an Active Directory group" -CREATE GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -```questdb-sql title="Map an Active Directory group to an already existing QuestDB group" -ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -```questdb-sql title="Remove an Active Directory mapping without deleting the QuestDB group" -ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -QuestDB works the list of external groups out from the User Info response -message. - -If we take the example used earlier, we will see that the message contains a -claim called `groups`. Its name is set with -[`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which -has no default and must be set whenever OIDC is enabled. - -If the groups claim is missing or it is an empty list, authentication fails. The -same happens when the claim named by -[`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing or -empty. Over HTTP QuestDB replies with `401 Unauthorized`, while on the PGWire -endpoint the client receives an authentication error. Either way the reason is -only visible in the server log. - -The user has to have at least the `HTTP` permission to be able to successfully -login via the [Web Console](/docs/getting-started/web-console/overview/). - -```json title="User info response example with Active Directory groups" -{ - "sub": "externalUser", - "name": "External User", - "groups": [ - "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", - "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" - ] -} -``` - -Any change made to the user's group membership in the Identity Provider, QuestDB -will adjust the user's access list. - -:::note - -There may be a slight delay due to the User Info Cache. - -QuestDB will use the cached information until it becomes stale, and gets -updated. - -::: - -The same stands for changes made to the user's status within the Identity -Provider. - -For example, a disabled user will not be kicked out of QuestDB immediately. - -The `acl.oidc.cache.ttl` config option drives how often user information should -be synchronized with the Identity Providers. - -It should be set accordingly to your organization's policies. - -Other parts of the user information, such as the `sub` and the `name` also used -by QuestDB. - -They could be displayed in the [Web Console](/docs/getting-started/web-console/overview/), or appear in -the logs, for example. - -## Configuration options - -For all OIDC-related configuration options of QuestDB, see -[Configuration](/docs/configuration/oidc/). - -
- -## Active Directory - -The following sections are guides for setting up single sign-on (SSO) with various OAuth2 providers. - -### PingFederate - -This document helps set up SSO authentication for the Web Console in -[PingFederate](https://www.pingidentity.com/en/platform/capabilities/authentication-authority/pingfederate.html). - -It is assumed that the Azure Active Directory serves as the Identity Provider -(IdP). - -#### Set up PingFederate client - -First thing first, let's pick a name for the client! - - - -The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). - -As a result, it cannot store safely a client secret. - -Instead it can use PKCE (Proof Key for Code Exchange) to secure the flow. - -As shown above, leave the client authentication disabled. - -We also have to white list the URL of the [Web Console](/docs/getting-started/web-console/overview/) as a redirection URL: - - - -We can instruct PingFederate to automatically authorize the scopes requested by -the [Web Console](/docs/getting-started/web-console/overview/). - -The user will not be presented the extra window asking for consent after -authentication: - - - -The [Web Console](/docs/getting-started/web-console/overview/) uses the -[Authorization Code Flow](/docs/security/oidc/#authentication-and-authorization-flow), -and refreshes tokens automatically. - -Next, enable the grant types required for this flow: - - - -We've selected: - -- Authorization Code -- Refresh Token -- Access Token Validation (Client is a Resource Server) - -After that, select the token manager for the client. - -The token manager is responsible for issuing access tokens. - -All token related settings should be configured in the token manager. - - - -Finally, enable PKCE - as shown above - and save the settings. - -#### Access Token Manager settings - -QuestDB does not require any special setup regarding the access token. - -We recommend that you do not to use shorter tokens than the default 28 -characters. - -As the QuestDB [Web Console](/docs/getting-started/web-console/overview/) refreshes the token automatically, there is no need -for long-lived tokens: - - - -We've selected: - -- Token length: 28 -- Token lifetime: 5 -- Lifetime extension policy: None -- Maximum token lifetime: Null -- Lifetime extension threshold percentage: 30 - -For the next step, we tune the Authorization Server. - -#### Authorization Server settings - -These settings relate to the authorization code, refresh token and CORS. - - - -In this section, we've entered: - -- Authorization code timeout: 60 -- Authorization code entropy: 30 -- Client secret retention period: 0 - -Next, ensure the `ROLL REFRESH TOKEN VALUES` option is selected: - - - -It is also important to whitelist the [Web Console](/docs/getting-started/web-console/overview/)'s URL on the CORS list: - - - -#### Set up a Microsoft Entra ID Data Source - -PingFederate needs a Data Source setup. - -This is a secure LDAP connection to Microsoft Entra ID, formerly known as Azure -Active Directory. - -The data source needs a: - -- name -- hostname -- port -- username and password for the LDAP connection - - - -We have given it the name EntraDS and it will be applied later. - -#### Set up a Password Credential Validator - -Now that PingFederate has an LDAP connection, we can use it for authentication. - -First, create a Password Credential Validator: - - - -We've entered: - -- Instance name: EntraPCV -- Instance ID: EntraPCV -- Selected: LDAP Username Password Credential Validator -- Parent instance: None - -Furthermore, we now declare our previously created data source (`EntraDS`): - - - -This links our data store (`EntraDS`) to our PCV (`EntraPCV`). - -#### Set up an Identity Provider - -We can use our PCV once we set up an Identity Provider. - -The IdP will be used to authenticate users against Active Directory using the -LDAP connection. - -We do this in the Type subsection: - - -Next, in the IdP Adapter section... - -Click: Add a new row to Credential Validators. - -Select the PCV (`EntraPCV`) we created. - -Optionally alter number of retries: - - - -#### Add groups to OIDC policy management - -QuestDB now needs to know about the user's AD group memberships to find their -permissions. - -Groups are passed to QuestDB inside the User Info object in a custom claim. - -This has to be added in the OpenID Connect Policy Management. - -The field is Multi-Valued, because it is a list of group names. - -Under the Attribute Contract subsection, see: - - - -Next, click to the Attribute Scopes subsection. - -Ensure `groups` is among the `openid` attributes: - - - -Onwards to the Attribute Sources & User Lookup Section. - -From this view, you can add local data stores. - -Note item `test` of type of LDAP: - - - -We created it via the following choices in Add Attribute Source: - - - -Note where we specified the Data Store (`EntraDS`). - -This is also where the directory search parameters are defined. - -Back at the Attribute Sources & User Lookup Section section, note we have set -`email`. - -The source is `LDAP (test)`, while the value is `usePrincipalName`: - - - -And finally! - -In the same Attribute Sources & User Lookup Section... - -Find `groups`. - -Note the definition of Source (`LDAP (test)`) that bridges our various parts. - -The value is `memberOf`. - - - -#### Enable Resource Owner Password Credentials (ROPC) flow - -As described under [Enable ROPC](#enable-ropc), tools such as `psql` can be -integrated with the OIDC provider using the ROPC flow. - -When setting this flow up, enable the Resource Owner Password Credentials flow in the -client settings. - -Next, create a Resource Owner Credentials Grant Mapping to map values obtained from -the Password Credential Validator (PCV) into the persistent grants. - -When setting this up, select the previously created LDAP Data Source and IdP Adapter, which links -to the existing PCV. - -Then select the `username` attribute of the PCV as `USER_KEY`. - -#### QuestDB configuration for PingFederate - -The below should be set in QuestDB's `server.conf`: - -```ini title="server.conf" -# enable OIDC -acl.oidc.enabled=true - -# hostname of the PingFederate server -acl.oidc.host=pingfederate.host - -# the client id picked when setting up the client above -acl.oidc.client.id=questdb - -# the claim which contains the user's group memberships -acl.oidc.groups.claim=groups - -# enable ROPC flow -# optional, required only if ROPC is enabled in PingFederate -acl.oidc.ropc.flow.enabled=true -``` - -The endpoint defaults listed under -[Endpoints](/docs/configuration/oidc/#endpoints) already match the PingFederate -paths, so they do not have to be set. - -Alternatively, set `acl.oidc.configuration.url` instead of `acl.oidc.host` and -let QuestDB discover the endpoints from PingFederate. The two are mutually -exclusive, and the endpoint settings are not used when the configuration URL is -set. - -#### Confirm QuestDB mappings and login - -QuestDB requires a mapping, as laid out under -[Mapping user permissions](#mapping-user-permissions). - -If a given user has the HTTP permission, they will be able to now login via the -[Web Console](/docs/getting-started/web-console/overview/). - -To test, head to `http://localhost:9000` and login. - -If all has been wired up well, then login will succeed. - -
- -### Microsoft EntraId - -This document sets up SSO authentication for the [QuestDB Web Console](/docs/getting-started/web-console/overview/) in -[Microsoft EntraID](https://www.microsoft.com/en-gb/security/business/identity-access/microsoft-entra-id), formerly known as Azure AD. - - -:::tip - -To enlarge the images, click or tap them. - -::: - -#### Set up the client application in Entra ID - -First thing first, let's pick a name for the client! - -Then head to _Microsoft Entra Admin Center_, and register the application -under _Identity - App registrations - New registration_. - - - -The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). - -As a result, it cannot store safely a client secret. - -Instead, it can use PKCE (Proof Key for Code Exchange) to secure the flow. - -When registering the application, select the SPA platform. - -We also have to specify the URL of the [Web Console](/docs/getting-started/web-console/overview/) as Redirect URI. - - - -After clicking _Register_, we have created a client application with the -name _QuestDB_. - -Each application is assigned a unique id (known as Client ID in the -OAuth2 - OIDC standard). The client will identify itself with this id -when sending requests to Entra ID. - - - -We find the platform configurations under _Authentication_. This is the place where -the previously set redirect URI can be viewed and modified. We can also specify -additional redirect URIs, if necessary. - -The redirect URIs of the application are automatically eligible for the -_Authorization Code Flow with PKCE_, which is a special version of the OAuth2 standard's -Authorization Code Flow. It is specifically designed for applications where a client -secret (e.g. a password) could not be kept safely. As single page applications run in -the browser, they fall into this category. - -The redirect URIs are also added to the _CORS_ (Cross-Origin Resource Sharing) policy -of EntraID. CORS is a mechanism to allow a web page, such as the Web Console, to access -resources from a different domain than the one that served the page. In this context -this means that we let the Web Console to access Entra ID, while its origin is the -HTTP endpoint of QuestDB. - - - -If we scroll down to the bottom of this page, we can also find a section where we -can enable the _Resource Owner Password Credential Flow_. - -This OAuth2 flow is legacy, and should be enabled only if there is a requirement -of connecting to QuestDB using SSO (Single Sign-On) via clients not supporting -redirect based web flows. -This could mean a Postgres client without OAuth2 integration, such as _psql_, or -a standalone in-house client application, or could be just a jupyter notebook. - -The main issue with this flow is that the client application has to be trusted -with the user's login details. The user's credentials are passed to the -application, in this case to QuestDB, and the client application uses these -credentials to authenticate the user by forwarding them to the identity provider, -in this case to Entra ID. - -It is guaranteed that QuestDB does not store the user's credentials in any way. -They are not persisted into the database, not even in encrypted form. -The login details are treated as passthrough information. Only exception is -that server logs can contain the username, logged for audit purposes. - - - -Our next stop is the _Token configuration_, where the OAuth2/OIDC access and ID -tokens can be customized. - -Note that users can be authenticated without customized tokens, but authorization -would prove to be challenging. The user's security groups are not included -in the tokens by default. - -QuestDB can be configured to request the user's groups from the UserInfo -endpoint of the OAuth2 server, but Entra ID cannot be configured to provide -this information via the UserInfo endpoint. -Therefore, we choose to customize the tokens, QuestDB will decode and -validate the ID token, and take the group information from there. - -QuestDB authorization relies on receiving the group memberships of the user. -Entra ID groups should be mapped to QuestDB groups, and permissions can be -granted to the QuestDB groups. Detailed information about group mappings can -be found under [User permissions](#user-permissions). - - - -The customized tokens contain user information which cannot be accessed -without permission. User information is provided by Microsoft Graph, so -the client application needs specific permissions to access -Microsoft Graph APIs. - -These permissions can be configured under _API permissions_. It is important -to note that we will be setting _Delegated_ permissions here, meaning we -are not granting actual permissions to access user data. Instead, each user -logging into QuestDB will have to consent to accessing their user profile. - - - -By default, the _User.Read_ permission is added to the list, but what we -really need is: - - openid: to be able to issue ID tokens - - profile: to access user information - - offline_access: to be able to issue refresh tokens - -By clicking on _Microsoft Graph_ we can select and add these permissions. - - - -The _User.Read_ permission is not needed. It can be removed by clicking -on the `...` at the end of the row, and selecting _Remove permission_ from -the popup menu. - - - -With this we have finished setting up the QuestDB client application -in Entra ID, and now we can wire QuestDB and Entra ID together by -adding OIDC configuration to QuestDB. - -#### QuestDB configuration for Entra ID {#questdb-configuration} - -The below should be set in QuestDB's `server.conf`: - -```ini title="server.conf" -# enable OIDC -acl.oidc.enabled=true - -# the claim contains the username or user id -acl.oidc.sub.claim=name - -# the claim contains the user's group memberships -acl.oidc.groups.claim=groups - -# groups are encoded in the token -acl.oidc.groups.encoded.in.token=true - -# OIDC configuration endpoint of Entra ID -acl.oidc.configuration.url=https://login.microsoftonline.com/12345678-1234-1234-1234-123456789abc/v2.0/.well-known/openid-configuration - -# application ID taken from Entra ID -acl.oidc.client.id=8de84b90-1ea5-4e41-9e84-dba860aa01a6 - -# redirect URI, QuestDB's HTTP endpoint -acl.oidc.redirect.uri=http://localhost:9000 - -# OAuth scopes the user has to consent to -acl.oidc.scope=openid profile offline_access - -# enable ROPC flow -# optional, required only if ROPC is enabled in Entra ID -acl.oidc.ropc.flow.enabled=true -``` - -The application ID and the OIDC configuration endpoint's URL can be found -in the Overview of the application in Entra ID. - -The application ID is displayed right under the application's name, the -OIDC configuration endpoint is displayed on the panel which opens up when -the _Endpoints_ button is clicked. - - - -:::caution - -This setup relies on -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), -because Entra ID cannot serve the group memberships from its User Info -endpoint. QuestDB then validates the ID token itself and never asks Entra ID -about it, so revoking a token in Entra ID does not end the access it grants: -the token keeps working until it expires. Read that option before going to -production, and note that clients must send the ID token here, not the access -token. - -::: - -#### Map groups and grant permissions - -Now we can start QuestDB, and login with the built-in admin to create -group mappings. - -As mentioned earlier, authorization works by mapping Entra ID groups -to QuestDB groups. When the user logs in, QuestDB decodes Entra ID -group memberships from the token, then finds the QuestDB groups -mapped to them, and the user gets the permissions based on the -mapped groups. - -```questdb-sql title="Create a group which is mapped to an Entra ID group" -CREATE GROUP extUsers WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; -``` -The above command maps the Entra ID group identified by object -id `87654321-1234-1234-1234-123456789abc` to a QuestDB group called `extUsers`. - -We should grant the necessary QuestDB endpoint permissions first -to make sure users can access the Web Console, Postgres and ILP -interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/#endpoint-permissions). - -```questdb-sql title="Grant endpoint permissions" -GRANT HTTP, PGWIRE TO groupName; -``` - -Now we can grant the rest of the permissions as required. We can -grant access to tables, for example. - -```questdb-sql title="Grant database permissions" -GRANT SELECT ON table1, table2 to groupName; -``` - -#### Confirm group mappings and login - -To test, head to the Web Console and login. - -If all has been wired up well, then login will succeed, and the user -will have the access granted to them. - -
diff --git a/documentation/security/oidc/client-discovery.mdx b/documentation/security/oidc/client-discovery.mdx new file mode 100644 index 0000000000..e71e9b5c92 --- /dev/null +++ b/documentation/security/oidc/client-discovery.mdx @@ -0,0 +1,127 @@ +--- +title: OIDC client discovery and tokens +sidebar_label: Client discovery and tokens +description: How a QuestDB client reads the OIDC settings endpoint to discover the provider's endpoints, client ID and scope, and which of the two tokens to send. +--- + +A client needs two things from QuestDB before it can authenticate a user +against your Identity Provider: where the provider's endpoints are, and which +of the two tokens the provider issues QuestDB expects in the +`Authorization: Bearer` header. Both come from the unauthenticated settings +endpoint, so neither has to be hard coded. + +## Settings endpoint + +QuestDB publishes the parts of its OIDC configuration that a client needs in +order to start an authentication flow. Any client can read them from the +settings endpoint, which requires no authentication: + +```bash title="Settings request example" +curl https://questdb.example.com:9000/settings +``` + +The OIDC-related entries of the response look like this, alongside other server +settings: + +```json title="Settings response example" +{ + "config": { + "acl.enabled": true, + "acl.basic.auth.realm.enabled": false, + "acl.oidc.enabled": true, + "acl.oidc.pkce.required": true, + "acl.oidc.state.required": false, + "acl.oidc.groups.encoded.in.token": false, + "acl.oidc.client.id": "questdb", + "acl.oidc.redirect.uri": "https://questdb.example.com:9000", + "acl.oidc.scope": "openid", + "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", + "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", + "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" + }, + "preferences.version": 0, + "preferences": {} +} +``` + +Each key is the name of the +[configuration option](/docs/configuration/oidc/) it carries. The +[Web Console](/docs/getting-started/web-console/overview/) reads them to build +its authorization request, and any other client can do the same instead of +hard coding the provider's details. + +| Key | Type | Present | +| --- | --- | --- | +| `acl.enabled` | boolean | always | +| `acl.basic.auth.realm.enabled` | boolean | always | +| `acl.oidc.enabled` | boolean | always | +| `acl.oidc.pkce.required` | boolean | always | +| `acl.oidc.state.required` | boolean | always | +| `acl.oidc.groups.encoded.in.token` | boolean | always | +| `acl.oidc.client.id` | string or `null` | always | +| `acl.oidc.redirect.uri` | string or `null` | always | +| `acl.oidc.scope` | string | always | +| `acl.oidc.authorization.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.token.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.device.authorization.endpoint` | string | when the endpoint is configured or discovered | + +The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the +[endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's +configuration document when `acl.oidc.configuration.url` is set. +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) +is present when a device authorization endpoint has been configured, or when the +provider advertises one. QuestDB publishes it without using it, for clients +which implement the [OIDC device flow](/docs/security/oidc-device-flow/) +themselves. + +`acl.oidc.redirect.uri` is `null` unless +[`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. +A client should then fall back to its own location, as the Web Console does. + +QuestDB advertises `acl.oidc.pkce.required` and `acl.oidc.state.required`, and +enforces neither. The client generates the code verifier and the `state` value; +the provider checks the verifier, and the client checks the `state` value it +gets back. + +:::note + +Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries +are published whether or not OIDC is enabled. With OIDC disabled and +`acl.oidc.host` in use, the endpoint URLs are built from the defaults rather +than from a real provider; with `acl.oidc.configuration.url` set they are not +resolved at all, and the endpoint keys are absent from the response. + +OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be +`true`, which the same response carries. When access control is disabled, +`acl.oidc.enabled` still reports the configured value while no OIDC +authentication takes place, so read both keys. + +::: + +The path of the endpoint is set by +[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). +Setting it adds paths rather than moving the endpoint, so the default path keeps +working too. That default is `/settings` only while +[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) +is at its default, as the settings path follows the Web Console context path. +A client which cannot assume both are at their defaults should make the path +configurable. + +## Which token to send + +`acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned +by the provider belongs in the `Authorization: Bearer` header: + +| Value | Token to send | How QuestDB validates it | +| --- | --- | --- | +| `false`, the default | the access token | by calling the user info endpoint | +| `true` | the ID token | locally, by checking its signature, audience, and claims | + +Sending the wrong one fails authentication with `401 Unauthorized` and the +reason in the server log. With +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +enabled the token has to be a JWT, because QuestDB reads the group memberships +straight out of its payload. The +[Web Console](/docs/getting-started/web-console/overview/) picks the token this +way, and so should any other client. + diff --git a/documentation/security/oidc/client-integration.mdx b/documentation/security/oidc/client-integration.mdx new file mode 100644 index 0000000000..882c59af1a --- /dev/null +++ b/documentation/security/oidc/client-integration.mdx @@ -0,0 +1,378 @@ +--- +title: OIDC client integration patterns +sidebar_label: Client integration patterns +description: How browser clients, Jupyter notebooks, CLI tools, unattended jobs, and PGWire clients authenticate against QuestDB Enterprise with OIDC. +--- + +How a client authenticates depends on what it can do: follow a browser +redirect, prompt a person, or neither. This page covers each case, from a +browser-based UI running the Authorization Code Flow to an unattended +ingestion job holding a token it acquired out of band. + +## Interactive clients + +Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an +OIDC provider. However, the level of support will vary between these tools. + +Interactive clients usually fall into one of the following categories: + +- Browser-based clients with support for HTTP redirects; this includes the Web + Console or any javascript UI +- Applications running in a browser without support for redirects, such as + Jupyter notebooks +- Non-browser based clients, usually some kind of command line interface (CLI) + or a standalone application, such as Microsoft Access + +### Browser-based clients + +If the tool is browser based and can handle HTTP redirects, it can implement two +possible flows to request an access token.: + +1. [Authorization Code](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth) + flow **(Recommended, more secure)** + +2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) + flow **(deprecated)**. It cannot use PKCE, so it does not satisfy + [`acl.oidc.pkce.required`](/docs/configuration/oidc/#acloidcpkcerequired), + which is `true` by default. + +The client can read the authorization and token endpoints, the client id and +the scopes from the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) instead of hard +coding them. + +The Web Console implements the +[Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a +special version of the Authorization Code flow designed for mobile apps and +single page applications. + +Regardless of which flow is used by the web or mobile application, the requested +token can be used for authentication and authorization when communicating with +QuestDB. Select the access or ID token as described in +[Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +### Jupyter notebook + +Use the official Python client's +[OIDC device flow](/docs/security/oidc-device-flow/). It works when the browser +and the notebook kernel are on different machines, and does not put the user's +password in the notebook. + +Note that JupyterHub's OAuth integration (OAuthenticator) is a separate concern: +it decides who may log in to JupyterHub, and gives the notebook no token to send +to QuestDB. + +For tooling without device-flow support, a last-resort option is the +Resource Owner Password Credentials (ROPC) flow. Enabling it usually +requires additional provider configuration. + +:::caution + +The Resource Owner Password Credentials flow is legacy, and should be used as a last resort. + +::: + +The ROPC snippets below assume +`acl.oidc.groups.encoded.in.token=false` and therefore send the access token. If +the setting is `true`, the client must send the ID token instead, and the +provider must issue one for the password grant. The official device-flow client +selects the correct token automatically. + +As a fallback, the code below acquires an access token with ROPC: + +```python +from urllib import request, parse +import json + +url = "https://oidc.provider:443/as/token.oauth2" +data = parse.urlencode( { + "grant_type": "password", + "username": "testuser", + "password": "testpwd", + "scope": "openid", + "client_id": "testclient" +} ).encode() +req = request.Request(url=url, data=data) +req.add_header("Content-Type", "application/x-www-form-urlencoded") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + access_token = resp["access_token"] +``` + +This token can be used to authenticate with QuestDB: + +```python +query = parse.urlencode({ + "query": "select current_user()" +}) +req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") +req.add_header("Authorization", f"Bearer {access_token}") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + print(resp) +``` + +#### Externalizing credentials + +The above example saves the user's credentials into the notebook, potentially +exposing them to others. One way to improve this is to use environment variables +or files to externalize the username and password. + +Here is an example using the `dotenv` library. + +First we need to create a file named `.env` with the settings: + +```ini title=".env" +username=testuser +password=testpwd +``` + +Then load it in our notebook, and use it to request tokens: + +```python +from dotenv import load_dotenv +import os +from urllib import request, parse +import json + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +url = "https://oidc.provider:443/as/token.oauth2" +data = parse.urlencode( { + "grant_type": "password", + "username": user, + "password": pwd, + "scope": "openid", + "client_id": "testclient" +} ).encode() +req = request.Request(url=url, data=data) +req.add_header("Content-Type", "application/x-www-form-urlencoded") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + access_token = resp["access_token"] +``` + +#### Enable ROPC + +The Resource Owner Password Credentials flow can be enabled in QuestDB within +`server.conf`: + +```ini title="server.conf" +acl.oidc.ropc.flow.enabled=true +``` + +> Note that the flow also has to be configured in the OAuth2/OIDC provider! + +Now we can use Basic Authentication to simplify our code. We send the +credentials to QuestDB, and the database will validate the credentials against +the OAuth2 provider. + +```python +from dotenv import load_dotenv +import os +from urllib import request +import base64 + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +query = parse.urlencode({ + "query": "select current_user()" +}) +req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") +b64credentials = base64.standard_b64encode(f"{user}:{pwd}".encode()).decode() +req.add_header("Authorization", f"Basic {b64credentials}") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + print(resp) +``` + +We can also use a postgres client to connect to the database: + +:::note + +QuestDB never persists the user's credentials. + +::: + +```python +import psycopg as pg +from dotenv import load_dotenv +import os + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +conn_str = f"user={user} password={pwd} host=localhost port=8812 dbname=qdb" +with pg.connect(conn_str, autocommit=True) as connection: + with connection.cursor() as cur: + cur.execute("select current_user()") + records = cur.fetchall() + for row in records: + print(row) +``` + +### CLI, standalone applications + +For a CLI or standalone application built with an official Java, Python, Rust, +C, or C++ client, use the +[OIDC device flow](/docs/security/oidc-device-flow/). +Tools such as `psql` and Microsoft Access cannot run the flow themselves; for +those tools, acquire a token separately and use +[PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the +legacy Resource Owner Password Credentials flow. + +The user logs in with their SSO credentials, and the server validates the +details with the OAuth2 provider: + +```shell +% psql -h localhost -p 8812 -U testuser +Password for user testuser: +psql (14.2, server 11.3) +Type "help" for help. + +testldap=> +testldap=> +``` + +## Non-interactive clients + +Non-interactive clients are usually jobs or standalone applications, such as a +client for ingesting data. It is practical to manage their credentials via an +OAuth2 provider too. + +As seen in the Jupyter notebook examples, the clients can request a token +themselves and then use it to authorise data ingestion. QuestDB publishes the +provider's token endpoint on the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so it +does not have to be hard coded, and the same response tells the client +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +The client id and the scopes stay the client's own. `acl.oidc.client.id` and +`acl.oidc.scope` describe QuestDB's registration with the provider, not the +job's. As noted in [Architecture overview](/docs/security/oidc/#architecture-overview), each +application which integrates via OIDC should be given a different Client Id, so +that the two can be told apart in the provider's audit log and given different +policies. + +:::caution + +Both requests must go over `https`. The settings response names the URL this +code posts the user's password to, so anyone able to tamper with a plaintext +response can redirect the password grant to a host of their choosing. The same +applies to the connection below, which carries the resulting token. + +::: + +```python +import os +import requests +import pandas as pd +import questdb +from dotenv import load_dotenv + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +# this job's own registration in the OIDC Provider, not QuestDB's; +# the openid scope is what makes the provider issue an ID token +client_id = os.environ.get("client_id") +scope = "openid" + +settings = requests.get("https://questdb.example.com:9000/settings").json()["config"] +if not settings.get("acl.oidc.enabled"): + raise SystemExit("OIDC is not enabled on this QuestDB instance") + +# the endpoint keys are absent until QuestDB resolves them +token_endpoint = settings.get("acl.oidc.token.endpoint") +if not token_endpoint: + raise SystemExit("QuestDB has not resolved the provider's token endpoint") + +response = requests.post(token_endpoint, + data={"grant_type": "password", + "client_id": client_id, + "username": user, + "password": pwd, + "scope": scope}, + headers={"Content-Type": "application/x-www-form-urlencoded"}) +response.raise_for_status() +tokens = response.json() + +# QuestDB reads the group memberships from the ID token when this is enabled +token_key = "id_token" if settings["acl.oidc.groups.encoded.in.token"] else "access_token" +if token_key not in tokens: + raise SystemExit(f"the provider did not issue an {token_key} for the password grant") +token = tokens[token_key] + +conf = f"wss::addr=questdb.example.com:9000;token={token};" +with questdb.connect(conf) as db: + df = pd.read_csv("trades.csv") + df["timestamp"] = pd.to_datetime(df["timestamp"]) + db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") +``` + +:::note + +The Resource Owner Password Credentials flow returns an ID token only if the +provider issues one for the password grant, and only when `openid` is among the +requested scopes. Check that the provider does before relying on +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +with a non-interactive client. + +::: + +Alternatively, a user may rely on QuestDB to authenticate them via the OAuth2 +provider when the Resource Owner Password Credentials flow is enabled on the +server side: + +```python +import os +import pandas as pd +import questdb +from dotenv import load_dotenv + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +conf = f"wss::addr=questdb.example.com:9000;username={user};password={pwd};" +with questdb.connect(conf) as db: + df = pd.read_csv("trades.csv") + df["timestamp"] = pd.to_datetime(df["timestamp"]) + db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") +``` + +## OIDC for the PGWire endpoint + +If the +Resource Owner Password Credentials (ROPC) flow is not an option, we can still authenticate via OIDC on the PGWire endpoint. +However, in this case the client's responsibility to source the token required for authentication. +This method works wherever a Postgres client library is available, including jupyter notebooks. + +Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. + +The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option. Send the token +selected by [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access token by +default, or the ID token when group memberships are encoded in it. + +```python +import psycopg as pg + +token = "token_requested_from_the_oauth2_provider" + +conn_str = f"user=_sso password={token} host=localhost port=8812 dbname=qdb" +with pg.connect(conn_str, autocommit=True) as connection: + with connection.cursor() as cur: + cur.execute('select current_user()') + records = cur.fetchall() + for row in records: + print(row) +``` + diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx new file mode 100644 index 0000000000..edc0ac4f48 --- /dev/null +++ b/documentation/security/oidc/entra-id.mdx @@ -0,0 +1,286 @@ +--- +title: Set up OIDC with Microsoft Entra ID +sidebar_label: Microsoft Entra ID +description: Walkthrough for configuring single sign-on between QuestDB Enterprise and Microsoft Entra ID, covering app registration, group claims, server.conf, and permissions. +--- + +import Screenshot from "@theme/Screenshot"; + +This document sets up SSO authentication for the [QuestDB Web Console](/docs/getting-started/web-console/overview/) in +[Microsoft EntraID](https://www.microsoft.com/en-gb/security/business/identity-access/microsoft-entra-id), formerly known as Azure AD. + + +:::tip + +To enlarge the images, click or tap them. + +::: + +## Set up the client application in Entra ID + +First thing first, let's pick a name for the client! + +Then head to _Microsoft Entra Admin Center_, and register the application +under _Identity - App registrations - New registration_. + + + +The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). + +As a result, it cannot store safely a client secret. + +Instead, it can use PKCE (Proof Key for Code Exchange) to secure the flow. + +When registering the application, select the SPA platform. + +We also have to specify the URL of the [Web Console](/docs/getting-started/web-console/overview/) as Redirect URI. + + + +After clicking _Register_, we have created a client application with the +name _QuestDB_. + +Each application is assigned a unique id (known as Client ID in the +OAuth2 - OIDC standard). The client will identify itself with this id +when sending requests to Entra ID. + + + +We find the platform configurations under _Authentication_. This is the place where +the previously set redirect URI can be viewed and modified. We can also specify +additional redirect URIs, if necessary. + +The redirect URIs of the application are automatically eligible for the +_Authorization Code Flow with PKCE_, which is a special version of the OAuth2 standard's +Authorization Code Flow. It is specifically designed for applications where a client +secret (e.g. a password) could not be kept safely. As single page applications run in +the browser, they fall into this category. + +The redirect URIs are also added to the _CORS_ (Cross-Origin Resource Sharing) policy +of EntraID. CORS is a mechanism to allow a web page, such as the Web Console, to access +resources from a different domain than the one that served the page. In this context +this means that we let the Web Console to access Entra ID, while its origin is the +HTTP endpoint of QuestDB. + + + +If we scroll down to the bottom of this page, we can also find a section where we +can enable the _Resource Owner Password Credential Flow_. + +This OAuth2 flow is legacy, and should be enabled only if there is a requirement +of connecting to QuestDB using SSO (Single Sign-On) via clients not supporting +redirect based web flows. +This could mean a Postgres client without OAuth2 integration, such as _psql_, or +a standalone in-house client application, or could be just a jupyter notebook. + +The main issue with this flow is that the client application has to be trusted +with the user's login details. The user's credentials are passed to the +application, in this case to QuestDB, and the client application uses these +credentials to authenticate the user by forwarding them to the identity provider, +in this case to Entra ID. + +It is guaranteed that QuestDB does not store the user's credentials in any way. +They are not persisted into the database, not even in encrypted form. +The login details are treated as passthrough information. Only exception is +that server logs can contain the username, logged for audit purposes. + + + +Our next stop is the _Token configuration_, where the OAuth2/OIDC access and ID +tokens can be customized. + +Note that users can be authenticated without customized tokens, but authorization +would prove to be challenging. The user's security groups are not included +in the tokens by default. + +QuestDB can be configured to request the user's groups from the UserInfo +endpoint of the OAuth2 server, but Entra ID cannot be configured to provide +this information via the UserInfo endpoint. +Therefore, we choose to customize the tokens, QuestDB will decode and +validate the ID token, and take the group information from there. + +QuestDB authorization relies on receiving the group memberships of the user. +Entra ID groups should be mapped to QuestDB groups, and permissions can be +granted to the QuestDB groups. Detailed information about group mappings can +be found under [User permissions](/docs/security/oidc/group-mapping/). + + + +The customized tokens contain user information which cannot be accessed +without permission. User information is provided by Microsoft Graph, so +the client application needs specific permissions to access +Microsoft Graph APIs. + +These permissions can be configured under _API permissions_. It is important +to note that we will be setting _Delegated_ permissions here, meaning we +are not granting actual permissions to access user data. Instead, each user +logging into QuestDB will have to consent to accessing their user profile. + + + +By default, the _User.Read_ permission is added to the list, but what we +really need is: + - openid: to be able to issue ID tokens + - profile: to access user information + - offline_access: to be able to issue refresh tokens + +By clicking on _Microsoft Graph_ we can select and add these permissions. + + + +The _User.Read_ permission is not needed. It can be removed by clicking +on the `...` at the end of the row, and selecting _Remove permission_ from +the popup menu. + + + +With this we have finished setting up the QuestDB client application +in Entra ID, and now we can wire QuestDB and Entra ID together by +adding OIDC configuration to QuestDB. + +## QuestDB configuration + +The below should be set in QuestDB's `server.conf`: + +```ini title="server.conf" +# enable OIDC +acl.oidc.enabled=true + +# the claim contains the username or user id +acl.oidc.sub.claim=name + +# the claim contains the user's group memberships +acl.oidc.groups.claim=groups + +# groups are encoded in the token +acl.oidc.groups.encoded.in.token=true + +# OIDC configuration endpoint of Entra ID +acl.oidc.configuration.url=https://login.microsoftonline.com/12345678-1234-1234-1234-123456789abc/v2.0/.well-known/openid-configuration + +# application ID taken from Entra ID +acl.oidc.client.id=8de84b90-1ea5-4e41-9e84-dba860aa01a6 + +# redirect URI, QuestDB's HTTP endpoint +acl.oidc.redirect.uri=http://localhost:9000 + +# OAuth scopes the user has to consent to +acl.oidc.scope=openid profile offline_access + +# enable ROPC flow +# optional, required only if ROPC is enabled in Entra ID +acl.oidc.ropc.flow.enabled=true +``` + +The application ID and the OIDC configuration endpoint's URL can be found +in the Overview of the application in Entra ID. + +The application ID is displayed right under the application's name, the +OIDC configuration endpoint is displayed on the panel which opens up when +the _Endpoints_ button is clicked. + + + +:::caution + +This setup relies on +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), +because Entra ID cannot serve the group memberships from its User Info +endpoint. QuestDB then validates the ID token itself and never asks Entra ID +about it, so revoking a token in Entra ID does not end the access it grants: +the token keeps working until it expires. Read that option before going to +production, and note that clients must send the ID token here, not the access +token. + +::: + +## Map groups and grant permissions + +Now we can start QuestDB, and login with the built-in admin to create +group mappings. + +As mentioned earlier, authorization works by mapping Entra ID groups +to QuestDB groups. When the user logs in, QuestDB decodes Entra ID +group memberships from the token, then finds the QuestDB groups +mapped to them, and the user gets the permissions based on the +mapped groups. + +```questdb-sql title="Create a group which is mapped to an Entra ID group" +CREATE GROUP extUsers WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` +The above command maps the Entra ID group identified by object +id `87654321-1234-1234-1234-123456789abc` to a QuestDB group called `extUsers`. + +We should grant the necessary QuestDB endpoint permissions first +to make sure users can access the Web Console, Postgres and ILP +interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/#endpoint-permissions). + +```questdb-sql title="Grant endpoint permissions" +GRANT HTTP, PGWIRE TO groupName; +``` + +Now we can grant the rest of the permissions as required. We can +grant access to tables, for example. + +```questdb-sql title="Grant database permissions" +GRANT SELECT ON table1, table2 to groupName; +``` + +## Confirm group mappings and login + +To test, head to the Web Console and login. + +If all has been wired up well, then login will succeed, and the user +will have the access granted to them. + +
diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx new file mode 100644 index 0000000000..991795d77b --- /dev/null +++ b/documentation/security/oidc/group-mapping.mdx @@ -0,0 +1,119 @@ +--- +title: Mapping OIDC groups and permissions +sidebar_label: Mapping groups and permissions +description: Map Identity Provider group memberships onto QuestDB groups with EXTERNAL ALIAS, and understand how QuestDB builds a user's access list from OIDC claims. +--- + +import Screenshot from "@theme/Screenshot"; + +QuestDB does not store permissions for an OIDC user. It reads the user's group +memberships out of the Identity Provider and maps them onto QuestDB groups, +which carry the permissions. This page covers where those memberships come +from, how to create the mapping, and what happens when a claim is missing. + +## User permissions + +QuestDB requires additional user information to be able to construct the user's +access list. + +As a reminder, the access list is the list of permissions that determines what +the user can and cannot do. + +QuestDB itself does not store external users, nor their passwords or any other +authentication related detail. + +External users and their authentication methods are managed by the Identity +Provider. + +Since external users are not managed by QuestDB, permissions cannot be granted +to them directly. + +Instead, the database expects a list of groups, called the _groups claim_ to be +present in the user information. + +These external group names are mapped to QuestDB's own groups. + +The access list of the external user consists of the permissions granted to +those groups: + + + +### Mapping user permissions + +The mappings between external and QuestDB groups are managed with the following +SQL commands: + +```questdb-sql title="Create a group which is mapped to an Active Directory group" +CREATE GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +```questdb-sql title="Map an Active Directory group to an already existing QuestDB group" +ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +```questdb-sql title="Remove an Active Directory mapping without deleting the QuestDB group" +ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +QuestDB works the list of external groups out from the User Info response +message. + +If we take the example used earlier, we will see that the message contains a +claim called `groups`. Its name is set with +[`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which +has no default and must be set whenever OIDC is enabled. + +If the groups claim is missing or it is an empty list, authentication fails. The +same happens when the claim named by +[`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing or +empty. Over HTTP QuestDB replies with `401 Unauthorized`, while on the PGWire +endpoint the client receives an authentication error. Either way the reason is +only visible in the server log. + +The user has to have at least the `HTTP` permission to be able to successfully +login via the [Web Console](/docs/getting-started/web-console/overview/). + +```json title="User info response example with Active Directory groups" +{ + "sub": "externalUser", + "name": "External User", + "groups": [ + "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", + "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" + ] +} +``` + +Any change made to the user's group membership in the Identity Provider, QuestDB +will adjust the user's access list. + +:::note + +There may be a slight delay due to the User Info Cache. + +QuestDB will use the cached information until it becomes stale, and gets +updated. + +::: + +The same stands for changes made to the user's status within the Identity +Provider. + +For example, a disabled user will not be kicked out of QuestDB immediately. + +The `acl.oidc.cache.ttl` config option drives how often user information should +be synchronized with the Identity Providers. + +It should be set accordingly to your organization's policies. + +Other parts of the user information, such as the `sub` and the `name` also used +by QuestDB. + +They could be displayed in the [Web Console](/docs/getting-started/web-console/overview/), or appear in +the logs, for example. + diff --git a/documentation/security/oidc/how-sign-in-works.mdx b/documentation/security/oidc/how-sign-in-works.mdx new file mode 100644 index 0000000000..d42086524e --- /dev/null +++ b/documentation/security/oidc/how-sign-in-works.mdx @@ -0,0 +1,269 @@ +--- +title: How OIDC sign-in works +sidebar_label: How sign-in works +description: Step-by-step walkthrough of the Authorization Code Flow with PKCE that QuestDB Enterprise and the Web Console use to authenticate a user against an OIDC provider. +--- + +import Screenshot from "@theme/Screenshot"; + +The [Web Console](/docs/getting-started/web-console/overview/) signs a user in +with the Authorization Code Flow with PKCE. This page follows that exchange +end to end, from the code verifier the client generates to the permission +check QuestDB runs on the user's group memberships. Read it when you are +integrating your own browser client, or debugging a sign-in that fails partway +through. + +## Authentication and authorization flow + +The OAuth2/OIDC standard defines different ways of obtaining access and ID +tokens from the OIDC Provider, referred to as the "_flow_". + +The goal of this flow is to get the user, who is sitting in front of the Web +Console, authenticated. + +Then, it allows QuestDB to determine the user's permissions based on user +information provided by the Identity Providers. + +Specifically, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) uses the +`Authorization Code Flow with PKCE` option. + +It consists of ten steps... + +### 1. Secret generation + +First the [Web Console](/docs/getting-started/web-console/overview/) generates a cryptographically strong +random secret called the _code verifier_. + +The secret is hashed using the _SHA256 algorithm_. The result is the _code +challenge_. + +After PKCE initialization the [Web Console](/docs/getting-started/web-console/overview/) requests an +_authorization code_ from the OIDC Provider. + +It calls the Authorization endpoint with a few parameters, including the: + +- **Client Id** +- requested scopes (the list of scopes are configurable, default is `openid` + only) +- code challenge +- algorithm used to generate the code challenge from the code verifier (SHA256) + +When the Authorization Server receives the request, it checks if the user has +been authenticated already: + +- If the user has a valid session, it can be provided with an authorization code + straight away, so we jump to step 4. + +- If the user does not have a valid session yet, it will be redirected to the + Identity Provider for authentication. + +```bash title="Authorization code request example" +https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 +``` + +:::note + +Some Identity Providers require the `state` parameter in the authorization +request. It is another random value generated by the client, which the OIDC +Provider returns unchanged together with the authorization code. Checking it +protects against CSRF attacks. + +If your provider requires it, set +[`acl.oidc.state.required`](/docs/configuration/oidc/#acloidcstaterequired) to +`true`. The [Web Console](/docs/getting-started/web-console/overview/) then +generates the `state` parameter, sends it in the authorization request, and +validates the value returned by the provider. + +::: + +### 2. Prove identity + +Next, the user must prove its identity. + +This could be a username with: + +- a password, +- an OTP +- facial recognition via a mobile app +- or anything else supported by the Identity Provider. + + + +### 3. Scope consent + +After successful authentication, the user provides consent for the requested +scopes. + +The list of scopes are configurable. + +By default the Web Console requests only the `openid` scope which is mandatory +for OIDC. + +No ID Token is issued without it. + +The OIDC provider can be configured to provide the consent automatically, +without presenting the user with an additional screen in the browser. + + + +### 4. Redirection + +Consent is granted! + +The Authorization Server redirects the user back to the +[Web Console](/docs/getting-started/web-console/overview/) with the _authorization code_: + +```bash title="Authorization code response example" +https://questdb.example.com:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A +``` + +### 5. Credential request + +Now, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) requests the ID and access +tokens from the Token endpoint of the OIDC Provider with the authorization code. + +It includes the Client ID and the PKCE code verifier together with the +authorization code in the request. + +The endpoint then hashes the code verifier using the method specified previously +in step 1. + +The result must match the code challenge, also provided in step 1. + +The matching code challenge proves that the token is requested by the client +which requested the authorization code, and it was not stolen: + +```bash title="Token request example" +POST https://oidc.provider:443/as/token.oauth2 HTTP/1.1 +Content-Type: application/x-www-form-urlencoded +grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF +``` + +### 6. Credentials received + +If the PKCE check is passed, the Web Console receives the ID and access tokens. + +There is a third token in the response too, the refresh token. + +The refresh token is used by the Web Console to refresh the access token before +it expires. + +Without the refresh token mechanism, the user would be forced to re-authenticate +when the access token expires. + +The validity of the tokens are configurable inside the OIDC Provider. + +```json title="Token response example" +{ + "access_token": "gslpJtzmmi6RwaPSx0dYGD4tEkom", + "refresh_token": "FUuAAqMp6LSTKmkUd5uZuodhiE4Kr6M7Eyv.eg83ge", + "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...", + "token_type": "Bearer", + "expires_in": 300 // In seconds, thus 5 minutes +} +``` + +### 7. Database access + +With the tokens, the Web Console can interact with the database. + +A token is in the header of every request sent to QuestDB. Which of the two goes +there depends on where QuestDB reads the group memberships from, see +[Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +:::note + +Worried about exposing the token? An access token is usually opaque and carries +no user details, though some providers issue a JWT here too. An ID token always +does: it is a JWT whose payload is base64 encoded, not encrypted, so treat it as +you would the user's directory record. + +::: + +To carry out permission checks, the database has to know more about the user. + +For this, QuestDB has a User Info Cache. + +Steps 8 and 9 describe the default flow, where QuestDB validates the access +token by asking the OIDC Provider about it. They are skipped when the cache +already holds a valid entry for the token, and they do not happen at all when +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is `true`, in which case QuestDB validates the ID token itself and never calls +the provider. See [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +```bash title="Query request example" +https://questdb.example.com:9000/exec?query=select%20current_user() +Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom +``` + +### 8. Find user information + +No user information in the cache, or stale information? + +QuestDB uses the access token to request user information from the OIDC +Provider's User Info endpoint. + +This call also serves as token validation. + +If the token is not real or has been expired, the User Info endpoint replies +with an error: + +```bash title="User info request example" +https://oidc.provider:443/idp/userinfo.openid +Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom +``` + +### 9. Receive user information + +If the access token is valid, QuestDB receives the required user information +from the endpoint, then updates its cache. + +The cache improves performance, as QuestDB does not have to turn to the OIDC +Provider on every single request. + +Do note that cache expiry is configurable: + +```json title="User info response example with Active Directory groups" +{ + "sub": "externalUser", + "name": "External User", + "groups": [ + "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", + "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" + ] +} +``` + +### 10. Permission check + +With the help of the user information, QuestDB can carry out +[permission checks](/docs/security/oidc/group-mapping/). + +If the permission check is successful, the database will process the request, +and then sends the results back: + +```json title="Query response example" +{ + "query": "select current_user()", + "columns": [ + { + "name": "current_user", + "type": "STRING" + } + ], + "dataset": [["External User"]], + "count": 1, + "timestamp": -1 +} +``` + diff --git a/documentation/security/oidc/index.mdx b/documentation/security/oidc/index.mdx new file mode 100644 index 0000000000..be4ce84a27 --- /dev/null +++ b/documentation/security/oidc/index.mdx @@ -0,0 +1,143 @@ +--- +title: OpenID Connect (OIDC) +sidebar_label: Overview +slug: /security/oidc +description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, client authentication, PGWire token authentication, and group mapping. +--- + +import Screenshot from "@theme/Screenshot"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + OpenID Connect (OIDC) enables SSO authentication with external Identity Providers. + + +OpenID Connect (OIDC) integrates with Identity Providers (IdP) external to +QuestDB. + +It is a convenient way to integrate QuestDB into your enterprise environment, +and it provides SSO (Single Sign-On) for the [Web Console](/docs/getting-started/web-console/overview/). + +Microsoft Active Directory and Azure AD, for example, can be turned into an +Identity Provider. + +Specific installation steps depend on the type of the provider. + +## Architecture overview + +Altogether, the architecture appears as such: + + + +We can break it down into core components. + +### Web Console + +QuestDB's interactive UI. Users must authenticate before accessing the database +via the interface. + +The [Web Console](/docs/getting-started/web-console/overview/) uses PKCE (Proof Key for Code Exchange) to +secure the authentication and authorization flow. + +In OAuth2/OIDC terms, the [Web Console](/docs/getting-started/web-console/overview/) is referred to as +the _client_, and it is assigned an identifier: the **Client Id**. + +Each application which integrates via OIDC should be given a different **Client +Id**. + +### OIDC Provider + +Typically consists of a number of modules. + +We are interested in two of them only. + +1. The _Identity Provider_ holds user identities and user information, capable + of authenticating users, and to issue an ID Token which uniquely identify + them. + +2. The _Authorization Server_ grants access to resources, such as a database, in + the form of access tokens. + +The OIDC Provider usually integrates with a number of applications which require +different access to a number of resources. + +These clients communicate with the OIDC Provider via its endpoints. + +It exposes a number of APIs, including the Authorization, Token and User Info +endpoints. + +### QuestDB + +The database, in OAuth2/OIDC terms the _protected resource_ or _resource +server_. + +Only processes requests which contain the bearer token selected by its OIDC +configuration: normally the access token, or the ID token when group memberships +are encoded in it. + + +## Where to go next + +| Page | Covers | +| --- | --- | +| [How sign-in works](/docs/security/oidc/how-sign-in-works/) | The Authorization Code Flow with PKCE, step by step | +| [Client discovery and tokens](/docs/security/oidc/client-discovery/) | The settings endpoint, and which of the two tokens to send | +| [Device flow](/docs/security/oidc-device-flow/) | Headless sign-in for a CLI, container, or notebook kernel | +| [Client integration patterns](/docs/security/oidc/client-integration/) | Browser clients, Jupyter, CLI tools, unattended jobs, PGWire | +| [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | Turning Identity Provider groups into QuestDB permissions | +| [PingFederate](/docs/security/oidc/pingfederate/) | Provider setup walkthrough | +| [Microsoft Entra ID](/docs/security/oidc/entra-id/) | Provider setup walkthrough | + +For the `acl.oidc.*` settings, see the +[OIDC settings reference](/docs/configuration/oidc/). + +## Settings endpoint {#settings-endpoint} + +QuestDB publishes the client ID, scope, and the provider's endpoints on an +unauthenticated settings endpoint, so a client can discover them instead of +hard coding them. See +[Client discovery and tokens](/docs/security/oidc/client-discovery/#settings-endpoint). + +## Which token to send {#which-token-to-send} + +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +decides whether a client sends the access token or the ID token in the +`Authorization: Bearer` header. See +[Client discovery and tokens](/docs/security/oidc/client-discovery/#which-token-to-send). + +## OIDC device flow {#device-authorization-flow} + +For command-line applications, containers, and remote notebook kernels, where +no browser redirect can reach the client, QuestDB Enterprise supports the +OAuth 2.0 Device Authorization Grant. QuestDB does not run the flow: the client +talks to the Identity Provider directly and presents the resulting bearer +token. + +See [OIDC device flow](/docs/security/oidc-device-flow/) for the +protocol walkthrough, what to configure first, and worked examples for the +Java, Python, Rust, C and C++ clients. + + +## Where each section moved {#moved} + +These anchors are kept so that existing links keep working. Prefer the pages +above when adding new links. + +| Section | Now at | +| --- | --- | +| Authentication and authorization flow | [How sign-in works](/docs/security/oidc/how-sign-in-works/) | +| Secret generation | [How sign-in works](/docs/security/oidc/how-sign-in-works/#1-secret-generation) | +| User permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | +| Mapping user permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions) | +| Interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/) | +| Non-interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/#non-interactive-clients) | +| OIDC for the PGWire endpoint | [Client integration patterns](/docs/security/oidc/client-integration/#oidc-for-the-pgwire-endpoint) | +| PingFederate | [PingFederate](/docs/security/oidc/pingfederate/) | +| QuestDB configuration for PingFederate | [PingFederate](/docs/security/oidc/pingfederate/#questdb-configuration) | +| Microsoft Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/) | +| Configuration options | [OIDC settings](/docs/configuration/oidc/) | diff --git a/documentation/security/oidc/pingfederate.mdx b/documentation/security/oidc/pingfederate.mdx new file mode 100644 index 0000000000..21e192e061 --- /dev/null +++ b/documentation/security/oidc/pingfederate.mdx @@ -0,0 +1,380 @@ +--- +title: Set up OIDC with PingFederate +sidebar_label: PingFederate +description: Walkthrough for configuring single sign-on between QuestDB Enterprise and PingFederate, covering the client, token manager, group claims, ROPC, and server.conf. +--- + +import Screenshot from "@theme/Screenshot"; + +This document helps set up SSO authentication for the Web Console in +[PingFederate](https://www.pingidentity.com/en/platform/capabilities/authentication-authority/pingfederate.html). + +It is assumed that the Azure Active Directory serves as the Identity Provider +(IdP). + +## Set up PingFederate client + +First thing first, let's pick a name for the client! + + + +The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). + +As a result, it cannot store safely a client secret. + +Instead it can use PKCE (Proof Key for Code Exchange) to secure the flow. + +As shown above, leave the client authentication disabled. + +We also have to white list the URL of the [Web Console](/docs/getting-started/web-console/overview/) as a redirection URL: + + + +We can instruct PingFederate to automatically authorize the scopes requested by +the [Web Console](/docs/getting-started/web-console/overview/). + +The user will not be presented the extra window asking for consent after +authentication: + + + +The [Web Console](/docs/getting-started/web-console/overview/) uses the +[Authorization Code Flow](/docs/security/oidc/how-sign-in-works/), +and refreshes tokens automatically. + +Next, enable the grant types required for this flow: + + + +We've selected: + +- Authorization Code +- Refresh Token +- Access Token Validation (Client is a Resource Server) + +After that, select the token manager for the client. + +The token manager is responsible for issuing access tokens. + +All token related settings should be configured in the token manager. + + + +Finally, enable PKCE - as shown above - and save the settings. + +## Access Token Manager settings + +QuestDB does not require any special setup regarding the access token. + +We recommend that you do not to use shorter tokens than the default 28 +characters. + +As the QuestDB [Web Console](/docs/getting-started/web-console/overview/) refreshes the token automatically, there is no need +for long-lived tokens: + + + +We've selected: + +- Token length: 28 +- Token lifetime: 5 +- Lifetime extension policy: None +- Maximum token lifetime: Null +- Lifetime extension threshold percentage: 30 + +For the next step, we tune the Authorization Server. + +## Authorization Server settings + +These settings relate to the authorization code, refresh token and CORS. + + + +In this section, we've entered: + +- Authorization code timeout: 60 +- Authorization code entropy: 30 +- Client secret retention period: 0 + +Next, ensure the `ROLL REFRESH TOKEN VALUES` option is selected: + + + +It is also important to whitelist the [Web Console](/docs/getting-started/web-console/overview/)'s URL on the CORS list: + + + +## Set up a Microsoft Entra ID Data Source + +PingFederate needs a Data Source setup. + +This is a secure LDAP connection to Microsoft Entra ID, formerly known as Azure +Active Directory. + +The data source needs a: + +- name +- hostname +- port +- username and password for the LDAP connection + + + +We have given it the name EntraDS and it will be applied later. + +## Set up a Password Credential Validator + +Now that PingFederate has an LDAP connection, we can use it for authentication. + +First, create a Password Credential Validator: + + + +We've entered: + +- Instance name: EntraPCV +- Instance ID: EntraPCV +- Selected: LDAP Username Password Credential Validator +- Parent instance: None + +Furthermore, we now declare our previously created data source (`EntraDS`): + + + +This links our data store (`EntraDS`) to our PCV (`EntraPCV`). + +## Set up an Identity Provider + +We can use our PCV once we set up an Identity Provider. + +The IdP will be used to authenticate users against Active Directory using the +LDAP connection. + +We do this in the Type subsection: + + +Next, in the IdP Adapter section... + +Click: Add a new row to Credential Validators. + +Select the PCV (`EntraPCV`) we created. + +Optionally alter number of retries: + + + +## Add groups to OIDC policy management + +QuestDB now needs to know about the user's AD group memberships to find their +permissions. + +Groups are passed to QuestDB inside the User Info object in a custom claim. + +This has to be added in the OpenID Connect Policy Management. + +The field is Multi-Valued, because it is a list of group names. + +Under the Attribute Contract subsection, see: + + + +Next, click to the Attribute Scopes subsection. + +Ensure `groups` is among the `openid` attributes: + + + +Onwards to the Attribute Sources & User Lookup Section. + +From this view, you can add local data stores. + +Note item `test` of type of LDAP: + + + +We created it via the following choices in Add Attribute Source: + + + +Note where we specified the Data Store (`EntraDS`). + +This is also where the directory search parameters are defined. + +Back at the Attribute Sources & User Lookup Section section, note we have set +`email`. + +The source is `LDAP (test)`, while the value is `usePrincipalName`: + + + +And finally! + +In the same Attribute Sources & User Lookup Section... + +Find `groups`. + +Note the definition of Source (`LDAP (test)`) that bridges our various parts. + +The value is `memberOf`. + + + +## Enable Resource Owner Password Credentials (ROPC) flow + +As described under [Enable ROPC](/docs/security/oidc/client-integration/#enable-ropc), tools such as `psql` can be +integrated with the OIDC provider using the ROPC flow. + +When setting this flow up, enable the Resource Owner Password Credentials flow in the +client settings. + +Next, create a Resource Owner Credentials Grant Mapping to map values obtained from +the Password Credential Validator (PCV) into the persistent grants. + +When setting this up, select the previously created LDAP Data Source and IdP Adapter, which links +to the existing PCV. + +Then select the `username` attribute of the PCV as `USER_KEY`. + +## QuestDB configuration + +The below should be set in QuestDB's `server.conf`: + +```ini title="server.conf" +# enable OIDC +acl.oidc.enabled=true + +# hostname of the PingFederate server +acl.oidc.host=pingfederate.host + +# the client id picked when setting up the client above +acl.oidc.client.id=questdb + +# the claim which contains the user's group memberships +acl.oidc.groups.claim=groups + +# enable ROPC flow +# optional, required only if ROPC is enabled in PingFederate +acl.oidc.ropc.flow.enabled=true +``` + +The endpoint defaults listed under +[Endpoints](/docs/configuration/oidc/#endpoints) already match the PingFederate +paths, so they do not have to be set. + +Alternatively, set `acl.oidc.configuration.url` instead of `acl.oidc.host` and +let QuestDB discover the endpoints from PingFederate. The two are mutually +exclusive, and the endpoint settings are not used when the configuration URL is +set. + +## Confirm QuestDB mappings and login + +QuestDB requires a mapping, as laid out under +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). + +If a given user has the HTTP permission, they will be able to now login via the +[Web Console](/docs/getting-started/web-console/overview/). + +To test, head to `http://localhost:9000` and login. + +If all has been wired up well, then login will succeed. + diff --git a/documentation/sidebars.js b/documentation/sidebars.js index 5fa2f0f1f2..4b13ed9be8 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -719,14 +719,52 @@ module.exports = { label: "Role-Based Access Control (RBAC)", }, { - id: "security/oidc", - type: "doc", - label: "OpenID Connect (OIDC) guide", - }, - { - id: "security/oidc-device-flow", - type: "doc", - label: "OIDC device flow", + label: "OpenID Connect (OIDC)", + type: "category", + link: { type: "doc", id: "security/oidc/index" }, + items: [ + { + id: "security/oidc/how-sign-in-works", + type: "doc", + label: "How sign-in works", + }, + { + id: "security/oidc/client-discovery", + type: "doc", + label: "Client discovery and tokens", + }, + { + id: "security/oidc-device-flow", + type: "doc", + label: "Device flow", + }, + { + id: "security/oidc/client-integration", + type: "doc", + label: "Client integration patterns", + }, + { + id: "security/oidc/group-mapping", + type: "doc", + label: "Mapping groups and permissions", + }, + { + label: "Provider setup", + type: "category", + items: [ + { + id: "security/oidc/pingfederate", + type: "doc", + label: "PingFederate", + }, + { + id: "security/oidc/entra-id", + type: "doc", + label: "Microsoft Entra ID", + }, + ], + }, + ], }, { type: "doc", From 68c9c4b51655b37ebd62d6ee28a6049ab2f51b96 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 16:20:11 +0100 Subject: [PATCH 45/54] Split the RBAC page into a section Same treatment as the OIDC page: rbac.md covered principals, authentication and endpoint permissions, granting rules, the full permissions table and the worked scenarios in one file. Each now has its own page under security/rbac/, with the overview at the section root, and every inbound anchor across the SQL reference, getting-started and operations docs repointed. Two permission statements are corrected while passing through. REBASE WAL, SWITCH COLD STORAGE ROLE and SET COLD STORAGE ROLE need the SYSTEM ADMIN permission, which is grantable, rather than unspecified "database administrator privileges" outside RBAC. all_permissions() lists every permission the database supports, not the ones currently applied. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014Cz9Rp9SBqLYSUsFkGFbGN --- documentation/changelog.mdx | 2 +- documentation/connect/clients/nodejs.md | 2 +- .../connect/compatibility/rest-api.md | 2 +- .../getting-started/enterprise-quick-start.md | 4 +- .../getting-started/web-console/overview.md | 2 +- documentation/high-availability/failover.md | 4 +- documentation/operations/upgrade.md | 2 +- .../query/sql/acl/create-service-account.md | 2 +- documentation/query/sql/acl/create-user.md | 2 +- documentation/query/sql/acl/grant.md | 6 +- documentation/query/sql/acl/revoke.md | 2 +- .../query/sql/alter-table-rebase-wal.md | 9 +- .../sql/alter-table-set-storage-policy.md | 2 +- .../query/sql/switch-cold-storage-role.md | 2 +- documentation/query/sql/switch-role.md | 4 +- documentation/security/oidc/entra-id.mdx | 2 +- documentation/security/rbac.md | 717 ------------------ .../security/rbac/authentication.mdx | 121 +++ .../security/rbac/common-scenarios.mdx | 124 +++ .../security/rbac/granting-permissions.mdx | 182 +++++ documentation/security/rbac/index.mdx | 185 +++++ .../security/rbac/permissions-reference.mdx | 138 ++++ .../security/rbac/users-and-groups.mdx | 197 +++++ documentation/sidebars.js | 31 +- 24 files changed, 1001 insertions(+), 743 deletions(-) delete mode 100644 documentation/security/rbac.md create mode 100644 documentation/security/rbac/authentication.mdx create mode 100644 documentation/security/rbac/common-scenarios.mdx create mode 100644 documentation/security/rbac/granting-permissions.mdx create mode 100644 documentation/security/rbac/index.mdx create mode 100644 documentation/security/rbac/permissions-reference.mdx create mode 100644 documentation/security/rbac/users-and-groups.mdx diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index d6ae54faa6..75dd08e47c 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -50,7 +50,7 @@ This page tracks significant updates to the QuestDB documentation. ### Reference - Added the [`node_role()`](/docs/query/functions/meta/#node_role) function, which reports the replication role of the instance to any authenticated session -- [RBAC](/docs/security/rbac/#failover-operator) - Added the `SWITCH ROLE` permission to the permissions table, a failover operator scenario, and the built-in admin's break-glass role in a replicated cluster +- [RBAC](/docs/security/rbac/common-scenarios/#failover-operator) - Added the `SWITCH ROLE` permission to the permissions table, a failover operator scenario, and the built-in admin's break-glass role in a replicated cluster - [Minimal HTTP server](/docs/operations/logging-metrics/#lifecycle-endpoint) - Documented the `GET /lifecycle` component snapshot and the [`http.health.check.authentication.required`](/docs/configuration/http-min-server/#httphealthcheckauthenticationrequired) setting - [Replication metrics](/docs/operations/logging-metrics/#replication-metrics) - Added `questdb_replication_pending_upload_txn` and `questdb_backup_active_at_last_demote` - [Replication configuration](/docs/configuration/database-replication/#replicationrole) - Documented that `replication.role` is the boot role, the `primary-catchup-uploads` value, and the restart hazard after an in-place switch diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index a8e38e26c1..ed2775ab8d 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -88,7 +88,7 @@ const sender = Sender.fromEnv(); ``` When using QuestDB Enterprise, authentication can also be done via REST token. -Please check the [RBAC docs](/docs/security/rbac/#authentication) for more +Please check the [RBAC docs](/docs/security/rbac/authentication/) for more info. This client does not run an OIDC flow. QuestDB Enterprise also accepts an OIDC diff --git a/documentation/connect/compatibility/rest-api.md b/documentation/connect/compatibility/rest-api.md index 2b3fb0884e..e2355c155d 100644 --- a/documentation/connect/compatibility/rest-api.md +++ b/documentation/connect/compatibility/rest-api.md @@ -890,5 +890,5 @@ curl -G --data-urlencode "query=SELECT 1;" \ http://localhost:9000/api/v1/sql/execute ``` -Refer to the [user management](/docs/security/rbac/#user-management) page to +Refer to the [user management](/docs/security/rbac/users-and-groups/#user-management) page to learn more on how to generate a REST API token. diff --git a/documentation/getting-started/enterprise-quick-start.md b/documentation/getting-started/enterprise-quick-start.md index 2ce88e20dc..026d80bef7 100644 --- a/documentation/getting-started/enterprise-quick-start.md +++ b/documentation/getting-started/enterprise-quick-start.md @@ -186,13 +186,13 @@ GRANT ALL ON table2 TO user2 WITH GRANT OPTION; Permission grants can be specific and fine-tuned. -List the full list of applied permissions with `all_permissions()`. +List every permission the database supports with `all_permissions()`. - For the full role-based access control docs, including group management, see the [RBAC operations guide](/docs/security/rbac/). - For a full list of available permissions, see the - [permissions sub-section in the RBAC operations guide](/docs/security/rbac/#permissions). + [permissions sub-section in the RBAC operations guide](/docs/security/rbac/permissions-reference/). ## 4. Ingest data, InfluxDB Line Protocol diff --git a/documentation/getting-started/web-console/overview.md b/documentation/getting-started/web-console/overview.md index a69e9a6bc5..ff5fe17704 100644 --- a/documentation/getting-started/web-console/overview.md +++ b/documentation/getting-started/web-console/overview.md @@ -125,5 +125,5 @@ If `http.settings.readonly` configuration is set to true, instance information i ::: :::info -When using QuestDB Enterprise with Role-Based Access Control (RBAC), only the users with `SETTINGS` or `DATABASE ADMIN` permission can edit the instance information. See [Database Permissions](/docs/security/rbac/#database-permissions) for more details. +When using QuestDB Enterprise with Role-Based Access Control (RBAC), only the users with `SETTINGS` or `DATABASE ADMIN` permission can edit the instance information. See [Database Permissions](/docs/security/rbac/permissions-reference/#database-permissions) for more details. ::: diff --git a/documentation/high-availability/failover.md b/documentation/high-availability/failover.md index fdc3265be9..ee8e99374a 100644 --- a/documentation/high-availability/failover.md +++ b/documentation/high-availability/failover.md @@ -53,7 +53,7 @@ needs the `SWITCH ROLE` permission. table is suspended and the WAL lag is nil. - An account with the `SWITCH ROLE` permission and with `PGWIRE` (for SQL) or `HTTP` (for the REST endpoint). See - [Failover operator](/docs/security/rbac/#failover-operator). + [Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). - Clients configured with a multi-host address list, so writers follow the primary role on their own. See [Client failover](/docs/high-availability/client-failover/concepts/). @@ -391,5 +391,5 @@ A coordinator drives a switch as follows: restart-based migration procedures and point-in-time recovery. - [Client failover](/docs/high-availability/client-failover/concepts/) for how clients follow the primary role. -- [Failover operator](/docs/security/rbac/#failover-operator) for the account +- [Failover operator](/docs/security/rbac/common-scenarios/#failover-operator) for the account that runs switches. diff --git a/documentation/operations/upgrade.md b/documentation/operations/upgrade.md index f36890607f..c3faf41421 100644 --- a/documentation/operations/upgrade.md +++ b/documentation/operations/upgrade.md @@ -97,7 +97,7 @@ every node runs the new version. The permission is stored in a form older versions cannot read, and a `GRANT ALL` issued before the upgrade does not include it. Accounts that triggered role switches through `SYSTEM ADMIN` on 3.3.x need an explicit `GRANT SWITCH ROLE`. See -[Failover operator](/docs/security/rbac/#failover-operator). +[Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). ### QuestDB Enterprise BYOC diff --git a/documentation/query/sql/acl/create-service-account.md b/documentation/query/sql/acl/create-service-account.md index 8a6fa4fa06..964119a859 100644 --- a/documentation/query/sql/acl/create-service-account.md +++ b/documentation/query/sql/acl/create-service-account.md @@ -36,7 +36,7 @@ fails and an error is raised, unless the `IF NOT EXISTS` clause is included in the statement. Note that new service accounts can only access the database if the necessary -[endpoint permissions](/docs/security/rbac/#endpoint-permissions) have been +[endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions) have been granted. The user creating the service account automatically receives the diff --git a/documentation/query/sql/acl/create-user.md b/documentation/query/sql/acl/create-user.md index 8a8d428672..c3555ae6bc 100644 --- a/documentation/query/sql/acl/create-user.md +++ b/documentation/query/sql/acl/create-user.md @@ -37,7 +37,7 @@ fails and an error is raised, unless the `IF NOT EXISTS` clause is included in the statement. Note that new users can only access the database if the necessary -[endpoint permissions](/docs/security/rbac/#endpoint-permissions) have been +[endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions) have been granted. ## Conditional user creation diff --git a/documentation/query/sql/acl/grant.md b/documentation/query/sql/acl/grant.md index 3a2e17b45c..11f44e237a 100644 --- a/documentation/query/sql/acl/grant.md +++ b/documentation/query/sql/acl/grant.md @@ -221,7 +221,7 @@ By default, `GRANT` does not check whether entities exist, making it possible to grant permissions to users, groups or service accounts that are later created. To make sure that the target entity of the grant statement exists, use -[verification](/docs/security/rbac/#grant-verification). The +[verification](/docs/security/rbac/granting-permissions/#grant-verification). The `WITH VERIFICATION` clause enables checks on the target entity and causes the `GRANT` statement to fail if the entity does not exist. @@ -239,7 +239,7 @@ queries. Therefore when a table has a designated timestamp, granting `SELECT` or `UPDATE` permissions on any column will automatically extend those permissions to the timestamp column. These are known as -[implicit permissions](/docs/security/rbac/#implicit-permissions), and they're +[implicit permissions](/docs/security/rbac/granting-permissions/#implicit-permissions), and they're indicated by an `I` in the `origin` column of the `SHOW PERMISSIONS` output. For example, if you grant `UPDATE` permission on the `id` column of the @@ -373,7 +373,7 @@ recreated as int. ### Owner grants In QuestDB there are no owners of database objects. Instead, there are -[owner grants](/docs/security/rbac/#owner-grants). +[owner grants](/docs/security/rbac/granting-permissions/#owner-grants). An owner grant means: diff --git a/documentation/query/sql/acl/revoke.md b/documentation/query/sql/acl/revoke.md index c302b60142..6a9727c685 100644 --- a/documentation/query/sql/acl/revoke.md +++ b/documentation/query/sql/acl/revoke.md @@ -166,7 +166,7 @@ REVOKE SELECT ON products(id) FROM john; If the user has a database- or table-level permission, then revoking it on a lower level triggers -[permission level re-adjustment](/docs/security/rbac/#permission-level-re-adjustment). +[permission level re-adjustment](/docs/security/rbac/granting-permissions/#permission-level-re-adjustment). Permission is switched to lower level and `materialized`: - database level permission is pushed to table level, so e.g. SELECT will not diff --git a/documentation/query/sql/alter-table-rebase-wal.md b/documentation/query/sql/alter-table-rebase-wal.md index be4ba2dba8..b7ec3edd3c 100644 --- a/documentation/query/sql/alter-table-rebase-wal.md +++ b/documentation/query/sql/alter-table-rebase-wal.md @@ -66,10 +66,11 @@ an error: ## Permissions -`REBASE WAL` requires database administrator (system admin) privileges rather -than a table-level grant. This reflects that the operation is destructive: it -discards un-applied transactions, changes the table's internal id, and replaces -its on-disk directory. A user who can run the non-destructive +`REBASE WAL` requires the `SYSTEM ADMIN` +[permission](/docs/security/rbac/permissions-reference/) rather than a table-level grant. +This reflects that the operation is destructive: it discards un-applied +transactions, changes the table's internal id, and replaces its on-disk +directory. A user who can run the non-destructive [`RESUME WAL`](/docs/query/sql/alter-table-resume-wal/) on a table cannot necessarily run `REBASE WAL` on it. diff --git a/documentation/query/sql/alter-table-set-storage-policy.md b/documentation/query/sql/alter-table-set-storage-policy.md index 56b34d84ef..54fc4370c3 100644 --- a/documentation/query/sql/alter-table-set-storage-policy.md +++ b/documentation/query/sql/alter-table-set-storage-policy.md @@ -206,5 +206,5 @@ Stages that are not set are omitted from the output. view listing active policies - [`SHOW CREATE TABLE`](/docs/query/sql/show/#show-create-table) — displays the attached `STORAGE POLICY` clause -- [RBAC permissions](/docs/security/rbac/#permissions) — `SET`, `REMOVE`, +- [RBAC permissions](/docs/security/rbac/permissions-reference/) — `SET`, `REMOVE`, `ENABLE`, and `DISABLE STORAGE POLICY` permissions diff --git a/documentation/query/sql/switch-cold-storage-role.md b/documentation/query/sql/switch-cold-storage-role.md index e1bc3ab586..d5f531b1fb 100644 --- a/documentation/query/sql/switch-cold-storage-role.md +++ b/documentation/query/sql/switch-cold-storage-role.md @@ -59,7 +59,7 @@ A role set this way does **not** survive a restart. Update `cold.storage.role` i ### Permissions -Both statements require database administrator (system admin) privileges, including with `FORCE`. They are not grantable through [RBAC](/docs/security/rbac/) permissions. +Both statements require the `SYSTEM ADMIN` [permission](/docs/security/rbac/permissions-reference/), including with `FORCE`. There is no narrower grant for the cold storage role: granting `SYSTEM ADMIN` also grants every other system function. ## Examples diff --git a/documentation/query/sql/switch-role.md b/documentation/query/sql/switch-role.md index 3c9e3b0028..a13df89422 100644 --- a/documentation/query/sql/switch-role.md +++ b/documentation/query/sql/switch-role.md @@ -112,7 +112,7 @@ Both statements require the `SWITCH ROLE` permission, granted with `GRANT SWITCH ROLE TO entity`. `SYSTEM ADMIN` does not imply it; `DATABASE ADMIN` does. A denied session receives `Access denied for [SWITCH ROLE]`. When access control is disabled, both statements are open to every session. See -[Failover operator](/docs/security/rbac/#failover-operator). +[Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). Before version 4.0.0, both statements required `SYSTEM ADMIN`. @@ -180,7 +180,7 @@ SELECT node_role(); - [Failover and role switch](/docs/high-availability/failover/) for the full procedure, the REST endpoint, and recovery from a refused switch - [`node_role()`](/docs/query/functions/meta/#node_role) for the open role read -- [RBAC](/docs/security/rbac/#failover-operator) for the `SWITCH ROLE` permission +- [RBAC](/docs/security/rbac/common-scenarios/#failover-operator) for the `SWITCH ROLE` permission - [`replication.role`](/docs/configuration/database-replication/#replicationrole) for the boot role - [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) for diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx index edc0ac4f48..42511d5d41 100644 --- a/documentation/security/oidc/entra-id.mdx +++ b/documentation/security/oidc/entra-id.mdx @@ -263,7 +263,7 @@ id `87654321-1234-1234-1234-123456789abc` to a QuestDB group called `extUsers`. We should grant the necessary QuestDB endpoint permissions first to make sure users can access the Web Console, Postgres and ILP -interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/#endpoint-permissions). +interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions). ```questdb-sql title="Grant endpoint permissions" GRANT HTTP, PGWIRE TO groupName; diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md deleted file mode 100644 index 27707e4496..0000000000 --- a/documentation/security/rbac.md +++ /dev/null @@ -1,717 +0,0 @@ ---- -title: Role-based Access Control (RBAC) -description: - Granular access control from database level down to individual columns and - rows. Learn how to secure your QuestDB instance with users, groups, and - fine-grained permissions. ---- - -import Screenshot from "@theme/Screenshot" -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - Role-based Access Control (RBAC) provides fine-grained permissions for your QuestDB instance. - - -QuestDB Enterprise provides fine-grained access control that can restrict access -at **database**, **table**, **column**, and even **row** level (using views). - -## Quick start - -Here's a complete example to create a read-only analyst user in under a minute: - -```questdb-sql --- 1. Create the user -CREATE USER analyst WITH PASSWORD 'secure_password_here'; - --- 2. Grant endpoint access (required to connect) -GRANT PGWIRE, HTTP TO analyst; - --- 3. Grant read access to specific tables -GRANT SELECT ON trades, prices TO analyst; - --- Done! The analyst can now connect and query trades and prices tables -``` - -To verify: - -```questdb-sql -SHOW PERMISSIONS analyst; -``` - -## Access control depth - -QuestDB's access control operates across two dimensions: - -### Data access granularity - -Control *what data* users can access: - -| Level | What you can control | Example | -| ------------ | ------------------------------- | ----------------------------------------------------- | -| **Database** | All tables, global operations | `GRANT SELECT ON ALL TABLES TO user` | -| **Table** | Specific tables | `GRANT SELECT ON trades TO user` | -| **Column** | Specific columns within a table | `GRANT SELECT ON trades(ts, price) TO user` | -| **Row** | Specific rows via views | Create a view with WHERE clause, grant access to view | - -### Connection access granularity - -Control *how* users can connect: - -| Permission | Protocol | Use case | -| ---------- | ------------------------------- | ------------------------------------------ | -| `HTTP` | REST API, Web Console, ILP/HTTP | Interactive users, web applications | -| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | -| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | - -```questdb-sql --- User can connect via PostgreSQL protocol only (not web console) -GRANT PGWIRE TO analyst; - --- Service can only ingest via ILP, cannot query -GRANT ILP TO ingest_service; - --- Full interactive access -GRANT HTTP, PGWIRE TO developer; -``` - -These dimensions are independent: a user might have `SELECT` on all tables but -only be allowed to connect via `PGWIRE`, or have `INSERT` permission but only -via `ILP`. - -### Column-level access - -Restrict users to see only certain columns: - -```questdb-sql --- User can only see timestamp and price, not quantity or trader_id -GRANT SELECT ON trades(ts, price) TO analyst; -``` - -To grant on every column except a few, use the `*` wildcard with an `EXCLUDE` -list: - -```questdb-sql --- User can see all columns except trader_id -GRANT SELECT ON trades(* EXCLUDE (trader_id)) TO analyst; -``` - -The wildcard covers the columns that exist when the statement runs, not columns -added later. See -[GRANT](/docs/query/sql/acl/grant/#grant-on-all-columns-of-a-table) for details. - -### Row-level access with views - -For row-level security, create a [view](/docs/concepts/views/) that filters rows, -then grant access to the view instead of the underlying table: - -```questdb-sql --- Create a view that only shows AAPL trades -CREATE VIEW aapl_trades AS ( - SELECT * FROM trades WHERE symbol = 'AAPL' -); - --- Grant access to the view, not the base table -GRANT SELECT ON aapl_trades TO aapl_analyst; --- No GRANT on trades table = user cannot see other symbols -``` - -The user `aapl_analyst` can only see AAPL trades. They have no access to the -underlying `trades` table. - -## Common scenarios - -### Read-only analyst - -A user who can query data but cannot modify anything: - -```questdb-sql -CREATE USER analyst WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO analyst; -GRANT SELECT ON ALL TABLES TO analyst; -``` - -### Application service account - -A service account for an application that ingests data into specific tables: - -```questdb-sql -CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'pwd'; -GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access -GRANT INSERT ON sensor_data TO ingest_app; -- Can only insert into sensor_data -``` - -### Team-based access with groups - -Multiple users sharing the same permissions: - -```questdb-sql --- Create a group -CREATE GROUP trading_team; - --- Grant permissions to the group -GRANT HTTP, PGWIRE TO trading_team; -GRANT SELECT ON trades, positions TO trading_team; -GRANT INSERT ON trades TO trading_team; - --- Add users to the group - they inherit all permissions -CREATE USER alice WITH PASSWORD 'pwd1'; -CREATE USER bob WITH PASSWORD 'pwd2'; -ADD USER alice TO trading_team; -ADD USER bob TO trading_team; -``` - -### Column-level restrictions (hide sensitive data) - -Allow access to a table but hide sensitive columns: - -```questdb-sql -CREATE USER auditor WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO auditor; - --- Grant access to non-sensitive columns only -GRANT SELECT ON employees(id, name, department, hire_date) TO auditor; --- Columns salary and ssn are not granted = invisible to auditor -``` - -### Row-level security (multi-tenant) - -Different users see different subsets of data: - -```questdb-sql --- Base table has data for all regions -CREATE TABLE sales (ts TIMESTAMP, region SYMBOL, amount DOUBLE) TIMESTAMP(ts); - --- Create region-specific views -CREATE VIEW sales_emea AS (SELECT * FROM sales WHERE region = 'EMEA'); -CREATE VIEW sales_apac AS (SELECT * FROM sales WHERE region = 'APAC'); - --- Grant users access to their region only -CREATE USER emea_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO emea_manager; -GRANT SELECT ON sales_emea TO emea_manager; - -CREATE USER apac_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO apac_manager; -GRANT SELECT ON sales_apac TO apac_manager; -``` - -### Database administrator - -A user with full control (but not the built-in admin): - -```questdb-sql -CREATE USER dba WITH PASSWORD 'pwd'; -GRANT DATABASE ADMIN TO dba; -``` - -:::warning - -`DATABASE ADMIN` grants all current and future permissions. Use sparingly. - -::: - -### Failover operator - -A service account that can move the primary role between nodes, for an external -coordinator or a runbook, without any other administrative right: - -```questdb-sql -CREATE SERVICE ACCOUNT failover_bot WITH PASSWORD 'pwd'; -GRANT HTTP TO failover_bot; -- POST /lifecycle/switch on port 9003 -GRANT SWITCH ROLE TO failover_bot; -- SWITCH ROLE, SWITCH STATUS, the endpoint -``` - -`SYSTEM ADMIN` is neither required nor sufficient for a role switch, and -`DATABASE ADMIN` includes `SWITCH ROLE`. Monitoring accounts do not need it: -`node_role()` and `GET /lifecycle` are open to any authenticated principal. See -[Failover and role switch](/docs/high-availability/failover/). - -## Core concepts - - - -### Users and service accounts - -QuestDB has two types of principals: - -- **Users**: For human individuals. Can belong to multiple groups and inherit - permissions from them. Cannot be assumed by others. -- **Service accounts**: For applications. Cannot belong to groups - all - permissions must be granted directly. Can be assumed by authorized users for - testing. - -```questdb-sql -CREATE USER human_user WITH PASSWORD 'pwd'; -CREATE SERVICE ACCOUNT app_account WITH PASSWORD 'pwd'; -``` - -Names must be unique across all users, service accounts, and groups. - -#### Why service accounts? - -Service accounts provide **clean, testable application access**: - -| Aspect | User | Service Account | -| -------------------- | ------------------------------ | ---------------------- | -| Permission source | Direct + inherited from groups | Direct only | -| Can belong to groups | Yes | No | -| Can be assumed (SU) | No | Yes | -| Typical use | Human individuals | Applications, services | - -Because service accounts have no inherited permissions, their access is fully -explicit and predictable. Combined with the ability to assume them, this makes -it easy to verify exactly what an application can and cannot do: - -```questdb-sql --- Create service account with specific permissions -CREATE SERVICE ACCOUNT trading_app WITH PASSWORD 'pwd'; -GRANT ILP TO trading_app; -GRANT INSERT ON trades TO trading_app; -GRANT SELECT ON positions TO trading_app; - --- Developer can assume the service account to test its access -GRANT ASSUME SERVICE ACCOUNT trading_app TO developer; - --- Developer switches to service account context -ASSUME SERVICE ACCOUNT trading_app; --- Now operating with trading_app's exact permissions --- Test what works and what doesn't... -EXIT SERVICE ACCOUNT; -``` - -This makes service accounts ideal for applications where you need predictable, -auditable, and testable access control. - -### Groups - -Groups simplify permission management when multiple users need the same access: - -```questdb-sql -CREATE GROUP analysts; -GRANT SELECT ON ALL TABLES TO analysts; - --- All users added to this group can read all tables -ADD USER alice TO analysts; -ADD USER bob TO analysts; -``` - -Users inherit permissions from their groups. Inherited permissions cannot be -revoked directly from the user - revoke from the group instead. When a group is -dropped, all members lose the permissions they inherited from that group. - -### Authentication methods {#authentication} - - - -QuestDB supports four authentication methods: - -| Method | Use case | Endpoints | -| --------------------- | --------------------------- | ------------------------------ | -| **Password** | Interactive users | REST API, PostgreSQL Wire | -| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | -| **REST API Token** | Programmatic REST access | REST API | -| **OIDC bearer token** | SSO users, external clients | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire once enabled | - -The first three are QuestDB's own credentials, created with the statements -below. An OIDC bearer token is issued by an external Identity Provider instead, -and the user's group memberships come from the token or the provider's user -info endpoint; see [OpenID Connect](/docs/security/oidc/) and -[which token to send](/docs/security/oidc/#which-token-to-send). - -OIDC on the PostgreSQL Wire endpoint is off by default. Enable it with -[`acl.oidc.pg.token.as.password.enabled`](/docs/configuration/oidc/#acloidcpgtokenaspasswordenabled), -which accepts the token in the password field, or with -[`acl.oidc.ropc.flow.enabled`](/docs/configuration/oidc/#acloidcropcflowenabled), -which has QuestDB exchange the user's SSO credentials for a token itself. - -Users can have multiple authentication methods enabled simultaneously: - -```questdb-sql --- Add JWK token for ILP access -ALTER USER sensor_writer CREATE TOKEN TYPE JWK; - --- Add REST API token (with 30-day expiry) -ALTER USER api_user CREATE TOKEN TYPE REST WITH TTL '30d'; -``` - -:::warning - -QuestDB does not store private keys or tokens after creation. Save them -immediately - they cannot be recovered. - -::: - -:::tip - -Authentication should happen via a [secure TLS connection](/docs/security/tls/) -to protect credentials in transit. - -::: - -### Endpoint permissions - -Before a user can connect, they need endpoint permissions: - -| Permission | Allows access to | -| ---------- | -------------------------------------- | -| `HTTP` | REST API, Web Console, ILP over HTTP | -| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | -| `ILP` | InfluxDB Line Protocol TCP (port 9009) | - -```questdb-sql --- Typical setup for an interactive user -GRANT HTTP, PGWIRE TO analyst; - --- Typical setup for an ingestion service -GRANT ILP TO ingest_service; -``` - -### Built-in admin - -Every QuestDB instance starts with a built-in admin account: - -- Default username: `admin` -- Default password: `quest` - -**Change these immediately in production** via `server.conf`: - -```ini -acl.admin.user=your_admin_name -acl.admin.password=your_secure_password -``` - -The built-in admin has irrevocable root access. After creating other admin -users, disable it: - -```ini -acl.admin.user.enabled=false -``` - -In a replicated cluster, keep in mind that the built-in admin authorizes a -[role switch](/docs/high-availability/failover/) from its own credentials, -independently of the replicated access lists. It is the break-glass account -when a `SWITCH ROLE` grant has not yet replicated to the node you need to -promote. - -## Permission levels - -Permissions have different granularities determining where they can be applied: - -| Granularity | Can be granted at | -| ----------- | ------------------------------------- | -| Database | Database only | -| Table | Database or specific tables | -| Column | Database, tables, or specific columns | - -Examples: - -```questdb-sql --- Database-level: applies to all tables -GRANT SELECT ON ALL TABLES TO user; - --- Table-level: applies to specific tables -GRANT SELECT ON trades, prices TO user; - --- Column-level: applies to specific columns -GRANT SELECT ON trades(ts, symbol, price) TO user; -``` - -### The GRANT option - -When granting permissions, you can allow the recipient to grant that permission -to others: - -```questdb-sql -GRANT SELECT ON trades TO team_lead WITH GRANT OPTION; - --- team_lead can now grant SELECT on trades to others -``` - -### Owner permissions {#owner-grants} - -When a user creates a table, they automatically receive all permissions on it -with the GRANT option. This ownership does not persist - if revoked, they cannot -get it back without someone re-granting it. - -## Advanced topics - -### Permission re-adjustment {#permission-level-re-adjustment} - -Database-level permissions include access to future tables. If you revoke access -to one table, QuestDB automatically converts the database-level grant to -individual table-level grants: - -```questdb-sql -GRANT SELECT ON ALL TABLES TO user; -- Database level -REVOKE SELECT ON secret_table FROM user; - --- Result: user now has table-level SELECT on all tables EXCEPT secret_table --- Future tables will NOT be accessible -``` - -The same applies from table to column level: - -```questdb-sql -GRANT SELECT ON trades TO user; -- Table level -REVOKE SELECT ON trades(ssn) FROM user; -- Revoke one column - --- Result: user has column-level SELECT on all columns EXCEPT ssn --- Future columns will NOT be accessible -``` - -:::note - -When dropping a table, permissions on it are preserved by default (useful if -the table is recreated). Use `DROP TABLE ... CASCADE PERMISSIONS` to also -remove all associated permissions. - -::: - -### Implicit timestamp permissions {#implicit-permissions} - -If a user has SELECT or UPDATE on any column of a table, they automatically get -the same permission on the designated timestamp column. This ensures time-series -operations (SAMPLE BY, LATEST ON, etc.) work correctly. - -### Granting on non-existent objects {#grant-verification} - -You can grant permissions on tables/columns that don't exist yet: - -```questdb-sql -GRANT INSERT ON future_table TO app; --- Permission activates when future_table is created -``` - -Use `WITH VERIFICATION` to catch typos: - -```questdb-sql -GRANT SELECT ON trdaes TO user WITH VERIFICATION; --- Fails immediately because 'trdaes' doesn't exist -``` - -### Service account assumption - -Users can temporarily assume a service account's permissions for debugging: - -```questdb-sql --- Grant ability to assume -GRANT ASSUME SERVICE ACCOUNT ingest_app TO developer; - --- Developer can now switch context -ASSUME SERVICE ACCOUNT ingest_app; --- ... debug with app's permissions ... -EXIT SERVICE ACCOUNT; -``` - -## User management reference {#user-management} - -### Creating and removing principals - -```questdb-sql --- Users -CREATE USER username WITH PASSWORD 'pwd'; -DROP USER username; - --- Service accounts -CREATE SERVICE ACCOUNT appname WITH PASSWORD 'pwd'; -DROP SERVICE ACCOUNT appname; - --- Groups -CREATE GROUP groupname; -DROP GROUP groupname; -``` - -### Managing group membership - -```questdb-sql -ADD USER username TO group1, group2; -REMOVE USER username FROM group1; -``` - -### Managing authentication - -```questdb-sql --- Change password -ALTER USER username WITH PASSWORD 'new_pwd'; - --- Remove password (disables password auth) -ALTER USER username WITH NO PASSWORD; - --- Create tokens -ALTER USER username CREATE TOKEN TYPE JWK; -ALTER USER username CREATE TOKEN TYPE REST WITH TTL '30d'; -ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Auto-refresh - --- Remove tokens -ALTER USER username DROP TOKEN TYPE JWK; -ALTER USER username DROP TOKEN TYPE REST; -- Drops all REST tokens -ALTER USER username DROP TOKEN TYPE REST 'token_value_here'; -- Drop specific token -``` - -Removing all authentication methods (password and tokens) effectively disables -the user - they can no longer connect to the database. - -### Viewing information - -```questdb-sql -SHOW USERS; -- List all users -SHOW SERVICE ACCOUNTS; -- List all service accounts -SHOW GROUPS; -- List all groups -SHOW GROUPS username; -- List groups for a user -SHOW USER username; -- Show auth methods for user -SHOW PERMISSIONS username; -- Show permissions for user -``` - -Example output from `SHOW USER`: - -``` -auth_type enabled ---------- ------- -Password true -JWK Token false -REST Token true -``` - -:::note - -Viewing other users' information requires `LIST USERS` (to list all) or -`USER DETAILS` (to see details) permissions. Users can always view their own -information without these permissions. - -::: - -## Permissions reference {#permissions} - -Use `all_permissions()` to see all available permissions: - -```questdb-sql -SELECT * FROM all_permissions(); -``` - -
-Full permissions table (click to expand) - -### Database permissions - -| Permission | Level | Description | -| ------------------------- | ----------------------------------- | --------------------------------------- | -| ADD COLUMN | Database | Table | Add columns to tables | -| ADD INDEX | Database | Table | Column | Add index on symbol columns | -| ALTER COLUMN CACHE | Database | Table | Column | Enable/disable symbol caching | -| ALTER COLUMN TYPE | Database | Table | Column | Change column types | -| ATTACH PARTITION | Database | Table | Attach partitions | -| BACKUP DATABASE | Database | Create database backups | -| CANCEL ANY COPY | Database | Cancel COPY operations | -| CREATE TABLE | Database | Create tables | -| CREATE MATERIALIZED VIEW | Database | Create materialized views | -| CREATE LIVE VIEW | Database | Create live views | -| DEDUP ENABLE | Database | Table | Enable deduplication | -| DEDUP DISABLE | Database | Table | Disable deduplication | -| DETACH PARTITION | Database | Table | Detach partitions | -| DISABLE STORAGE POLICY | Database | Table | Disable storage policies | -| DROP COLUMN | Database | Table | Column | Drop columns | -| DROP INDEX | Database | Table | Column | Drop indexes | -| DROP PARTITION | Database | Table | Drop partitions | -| DROP TABLE | Database | Table | Drop tables | -| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | -| DROP LIVE VIEW | Database | Table | Drop live views | -| ENABLE STORAGE POLICY | Database | Table | Enable storage policies | -| INSERT | Database | Table | Insert data | -| REFRESH MATERIALIZED VIEW | Database | Table | Refresh materialized views | -| REINDEX | Database | Table | Column | Reindex columns | -| REMOVE STORAGE POLICY | Database | Table | Remove storage policies | -| RENAME COLUMN | Database | Table | Column | Rename columns | -| RENAME TABLE | Database | Table | Rename tables | -| RESUME WAL | Database | Table | Resume WAL processing | -| SELECT | Database | Table | Column | Read data | -| SET STORAGE POLICY | Database | Table | Set storage policies | -| SET TABLE PARAM | Database | Table | Set table parameters | -| SET TABLE TYPE | Database | Table | Change table type | -| SETTINGS | Database | Change instance settings in Web Console | -| SNAPSHOT | Database | Create snapshots | -| SQL ENGINE ADMIN | Database | List/cancel running queries | -| SWITCH ROLE | Database | Switch the replication role, read SWITCH STATUS | -| SYSTEM ADMIN | Database | System functions (reload_tls, etc.) | -| TRUNCATE TABLE | Database | Table | Truncate tables | -| UPDATE | Database | Table | Column | Update data | -| VACUUM TABLE | Database | Table | Reclaim storage | - -### User management permissions - -| Permission | Description | -| ---------------------- | --------------------------------------- | -| ADD EXTERNAL ALIAS | Create external group mappings | -| ADD PASSWORD | Set user passwords | -| ADD USER | Add users to groups | -| CREATE GROUP | Create groups | -| CREATE JWK | Create JWK tokens | -| CREATE REST TOKEN | Create REST API tokens | -| CREATE SERVICE ACCOUNT | Create service accounts | -| CREATE USER | Create users | -| DISABLE USER | Disable users | -| DROP GROUP | Drop groups | -| DROP JWK | Drop JWK tokens | -| DROP REST TOKEN | Drop REST API tokens | -| DROP SERVICE ACCOUNT | Drop service accounts | -| DROP USER | Drop users | -| ENABLE USER | Enable users | -| LIST USERS | List users/groups/service accounts | -| REMOVE EXTERNAL ALIAS | Remove external group mappings | -| REMOVE PASSWORD | Remove passwords | -| REMOVE USER | Remove users from groups | -| USER DETAILS | View user/group/service account details | - -### Special permissions - -| Permission | Description | -| -------------- | --------------------------------------------------------------------- | -| ALL | All permissions at the granted level (database/table/column) | -| DATABASE ADMIN | All permissions including future ones; can assume any service account | - -A few operations are reserved for database administrators and are not grantable as permissions at all: - -| Operation | Notes | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | -| [`ALTER TABLE REBASE WAL`](/docs/query/sql/alter-table-rebase-wal/) | Rebuilds a suspended WAL table under a fresh sequencer | -| [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) | Moves the [cold storage](/docs/concepts/cold-storage/) manager role, including with `FORCE` | - -By contrast, [`SWITCH ROLE`](/docs/query/sql/switch-role/), which moves the -replication role, is an ordinary grantable permission. - -
- -## SQL commands reference - -- [ADD USER](/docs/query/sql/acl/add-user/) -- [ALTER USER](/docs/query/sql/acl/alter-user/) -- [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) -- [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) -- [CREATE GROUP](/docs/query/sql/acl/create-group/) -- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) -- [CREATE USER](/docs/query/sql/acl/create-user/) -- [DROP GROUP](/docs/query/sql/acl/drop-group/) -- [DROP SERVICE ACCOUNT](/docs/query/sql/acl/drop-service-account/) -- [DROP USER](/docs/query/sql/acl/drop-user/) -- [EXIT SERVICE ACCOUNT](/docs/query/sql/acl/exit-service-account/) -- [GRANT](/docs/query/sql/acl/grant/) -- [GRANT ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/grant-assume-service-account/) -- [REMOVE USER](/docs/query/sql/acl/remove-user/) -- [REVOKE](/docs/query/sql/acl/revoke/) -- [REVOKE ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/revoke-assume-service-account/) -- [SHOW USER](/docs/query/sql/show/#show-user) -- [SHOW USERS](/docs/query/sql/show/#show-users) -- [SHOW GROUPS](/docs/query/sql/show/#show-groups) -- [SHOW SERVICE ACCOUNT](/docs/query/sql/show/#show-service-account) -- [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) -- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) diff --git a/documentation/security/rbac/authentication.mdx b/documentation/security/rbac/authentication.mdx new file mode 100644 index 0000000000..9951af33ec --- /dev/null +++ b/documentation/security/rbac/authentication.mdx @@ -0,0 +1,121 @@ +--- +title: RBAC authentication and endpoint access +sidebar_label: Authentication and endpoints +description: + The four authentication methods QuestDB Enterprise accepts, the HTTP, PGWIRE + and ILP endpoint permissions that gate each protocol, and how to manage + passwords and tokens. +--- + +import Screenshot from "@theme/Screenshot" + +Connecting to QuestDB Enterprise takes two things: a credential the server +accepts, and the endpoint permission for the protocol you are connecting over. A +valid password is not enough on its own. + +## Authentication methods {#authentication} + + + +QuestDB supports four authentication methods: + +| Method | Use case | Endpoints | +| --------------------- | --------------------------- | -------------------------------------------------------------------------------- | +| **Password** | Interactive users | REST API, PostgreSQL Wire | +| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | +| **REST API Token** | Programmatic REST access | REST API | +| **OIDC bearer token** | SSO users, external clients | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire once enabled | + +The first three are QuestDB's own credentials, created with the statements +below. An OIDC bearer token is issued by an external Identity Provider instead, +and the user's group memberships come from the token or the provider's user info +endpoint; see [OpenID Connect](/docs/security/oidc/) and +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +OIDC on the PostgreSQL Wire endpoint is off by default. Enable it with +[`acl.oidc.pg.token.as.password.enabled`](/docs/configuration/oidc/#acloidcpgtokenaspasswordenabled), +which accepts the token in the password field, or with +[`acl.oidc.ropc.flow.enabled`](/docs/configuration/oidc/#acloidcropcflowenabled), +which has QuestDB exchange the user's SSO credentials for a token itself. + +Users can have multiple authentication methods enabled simultaneously: + +```questdb-sql +-- Add JWK token for ILP access +ALTER USER sensor_writer CREATE TOKEN TYPE JWK; + +-- Add REST API token (with 30-day expiry) +ALTER USER api_user CREATE TOKEN TYPE REST WITH TTL '30d'; +``` + +:::warning + +QuestDB does not store private keys or tokens after creation. Save them +immediately - they cannot be recovered. + +::: + +:::tip + +Authentication should happen via a [secure TLS connection](/docs/security/tls/) +to protect credentials in transit. + +::: + +## Endpoint permissions {#endpoint-permissions} + +Before a user can connect, they need endpoint permissions: + +| Permission | Allows access to | +| ---------- | -------------------------------------- | +| `HTTP` | REST API, Web Console, ILP over HTTP | +| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | +| `ILP` | InfluxDB Line Protocol TCP (port 9009) | + +```questdb-sql +-- Typical setup for an interactive user +GRANT HTTP, PGWIRE TO analyst; + +-- Typical setup for an ingestion service +GRANT ILP TO ingest_service; +``` + +Endpoint permissions are independent of data permissions. A user may hold +`SELECT` on every table and still be unable to connect, or hold `ILP` and be +able to ingest but never query. + +## Managing authentication {#managing-authentication} + +```questdb-sql +-- Change password +ALTER USER username WITH PASSWORD 'new_pwd'; + +-- Remove password (disables password auth) +ALTER USER username WITH NO PASSWORD; + +-- Create tokens +ALTER USER username CREATE TOKEN TYPE JWK; +ALTER USER username CREATE TOKEN TYPE REST WITH TTL '30d'; +ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Auto-refresh + +-- Remove tokens +ALTER USER username DROP TOKEN TYPE JWK; +ALTER USER username DROP TOKEN TYPE REST; -- Drops all REST tokens +ALTER USER username DROP TOKEN TYPE REST 'token_value_here'; -- Drop specific token +``` + +Removing all authentication methods (password and tokens) effectively disables +the user - they can no longer connect to the database. + +## See also + +- [OpenID Connect](/docs/security/oidc/) - SSO against an external Identity + Provider +- [TLS](/docs/security/tls/) - encrypting credentials in transit +- [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) - + the principals these credentials belong to diff --git a/documentation/security/rbac/common-scenarios.mdx b/documentation/security/rbac/common-scenarios.mdx new file mode 100644 index 0000000000..b16fdfb7e4 --- /dev/null +++ b/documentation/security/rbac/common-scenarios.mdx @@ -0,0 +1,124 @@ +--- +title: RBAC common scenarios +sidebar_label: Common scenarios +description: + Worked QuestDB Enterprise permission setups for read-only analysts, ingestion + service accounts, teams, column and row restrictions, database administrators, + and failover operators. +--- + +Each scenario below is a complete, runnable setup. Adapt the names and tables; +the shape of the grants is the part worth copying. + +## Read-only analyst + +A user who can query data but cannot modify anything: + +```questdb-sql +CREATE USER analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO analyst; +GRANT SELECT ON ALL TABLES TO analyst; +``` + +## Application service account + +A service account for an application that ingests data into specific tables: + +```questdb-sql +CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'pwd'; +GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access +GRANT INSERT ON sensor_data TO ingest_app; -- Can only insert into sensor_data +``` + +## Team-based access with groups + +Multiple users sharing the same permissions: + +```questdb-sql +-- Create a group +CREATE GROUP trading_team; + +-- Grant permissions to the group +GRANT HTTP, PGWIRE TO trading_team; +GRANT SELECT ON trades, positions TO trading_team; +GRANT INSERT ON trades TO trading_team; + +-- Add users to the group - they inherit all permissions +CREATE USER alice WITH PASSWORD 'pwd1'; +CREATE USER bob WITH PASSWORD 'pwd2'; +ADD USER alice TO trading_team; +ADD USER bob TO trading_team; +``` + +## Column-level restrictions (hide sensitive data) + +Allow access to a table but hide sensitive columns: + +```questdb-sql +CREATE USER auditor WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO auditor; + +-- Grant access to non-sensitive columns only +GRANT SELECT ON employees(id, name, department, hire_date) TO auditor; +-- Columns salary and ssn are not granted = invisible to auditor +``` + +## Row-level security (multi-tenant) + +Different users see different subsets of data: + +```questdb-sql +-- Base table has data for all regions +CREATE TABLE sales (ts TIMESTAMP, region SYMBOL, amount DOUBLE) TIMESTAMP(ts); + +-- Create region-specific views +CREATE VIEW sales_emea AS (SELECT * FROM sales WHERE region = 'EMEA'); +CREATE VIEW sales_apac AS (SELECT * FROM sales WHERE region = 'APAC'); + +-- Grant users access to their region only +CREATE USER emea_manager WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO emea_manager; +GRANT SELECT ON sales_emea TO emea_manager; + +CREATE USER apac_manager WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO apac_manager; +GRANT SELECT ON sales_apac TO apac_manager; +``` + +## Database administrator + +A user with full control (but not the built-in admin): + +```questdb-sql +CREATE USER dba WITH PASSWORD 'pwd'; +GRANT DATABASE ADMIN TO dba; +``` + +:::warning + +`DATABASE ADMIN` grants all current and future permissions. Use sparingly. + +::: + +## Failover operator + +A service account that can move the primary role between nodes, for an external +coordinator or a runbook, without any other administrative right: + +```questdb-sql +CREATE SERVICE ACCOUNT failover_bot WITH PASSWORD 'pwd'; +GRANT HTTP TO failover_bot; -- POST /lifecycle/switch on port 9003 +GRANT SWITCH ROLE TO failover_bot; -- SWITCH ROLE, SWITCH STATUS, the endpoint +``` + +`SYSTEM ADMIN` is neither required nor sufficient for a role switch, and +`DATABASE ADMIN` includes `SWITCH ROLE`. Monitoring accounts do not need it: +`node_role()` and `GET /lifecycle` are open to any authenticated principal. See +[Failover and role switch](/docs/high-availability/failover/). + +## See also + +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + the semantics behind these grants +- [Permissions reference](/docs/security/rbac/permissions-reference/) - every + permission available diff --git a/documentation/security/rbac/granting-permissions.mdx b/documentation/security/rbac/granting-permissions.mdx new file mode 100644 index 0000000000..2e52d9fc22 --- /dev/null +++ b/documentation/security/rbac/granting-permissions.mdx @@ -0,0 +1,182 @@ +--- +title: Granting and revoking permissions +sidebar_label: Granting permissions +description: + How QuestDB Enterprise permissions apply at database, table and column level, + what GRANT ALL and WITH GRANT OPTION really do, and how a grant changes level + when you revoke part of it. +--- + +Permissions are granted with [`GRANT`](/docs/query/sql/acl/grant/) and removed +with [`REVOKE`](/docs/query/sql/acl/revoke/). This page covers the level a grant +applies at, the semantics that are easy to get wrong, and the behaviour that +surprises people the first time they hit it. + +For the list of permissions themselves, see the +[permissions reference](/docs/security/rbac/permissions-reference/). + +## Permission levels + +Permissions have different granularities determining where they can be applied: + +| Granularity | Can be granted at | +| ----------- | ------------------------------------- | +| Database | Database only | +| Table | Database or specific tables | +| Column | Database, tables, or specific columns | + +Examples: + +```questdb-sql +-- Database-level: applies to all tables +GRANT SELECT ON ALL TABLES TO user; + +-- Table-level: applies to specific tables +GRANT SELECT ON trades, prices TO user; + +-- Column-level: applies to specific columns +GRANT SELECT ON trades(ts, symbol, price) TO user; +``` + +### Column-level access + +Restrict users to see only certain columns: + +```questdb-sql +-- User can only see timestamp and price, not quantity or trader_id +GRANT SELECT ON trades(ts, price) TO analyst; +``` + +To grant on every column except a few, use the `*` wildcard with an `EXCLUDE` +list: + +```questdb-sql +-- User can see all columns except trader_id +GRANT SELECT ON trades(* EXCLUDE (trader_id)) TO analyst; +``` + +The wildcard covers the columns that exist when the statement runs, not columns +added later. See +[GRANT](/docs/query/sql/acl/grant/#grant-on-all-columns-of-a-table) for details. + +### Row-level access with views + +For row-level security, create a [view](/docs/concepts/views/) that filters +rows, then grant access to the view instead of the underlying table: + +```questdb-sql +-- Create a view that only shows AAPL trades +CREATE VIEW aapl_trades AS ( + SELECT * FROM trades WHERE symbol = 'AAPL' +); + +-- Grant access to the view, not the base table +GRANT SELECT ON aapl_trades TO aapl_analyst; +-- No GRANT on trades table = user cannot see other symbols +``` + +The user `aapl_analyst` can only see AAPL trades. They have no access to the +underlying `trades` table. + +Creating a view needs the `CREATE VIEW` permission; dropping or redefining one +needs `DROP VIEW` or `ALTER VIEW` on that view. + +## Granting ALL {#granting-all} + +`ALL` expands to every permission valid at the level you grant it, and at the +database level that set includes `DATABASE ADMIN`: + +```questdb-sql +-- Makes alice a full database administrator +GRANT ALL TO alice; + +-- Every permission on one table only +GRANT ALL ON trades TO alice; +``` + +:::warning + +`GRANT ALL TO ` with no table or column list is an administrator grant, +not a broad data grant. Scope it to tables, or name the permissions you mean. + +::: + +## The GRANT option + +When granting permissions, you can allow the recipient to grant that permission +to others: + +```questdb-sql +GRANT SELECT ON trades TO team_lead WITH GRANT OPTION; + +-- team_lead can now grant SELECT on trades to others +``` + +## Owner permissions {#owner-grants} + +When a user creates a table, they automatically receive all permissions on it +with the GRANT option. This ownership does not persist - if revoked, they cannot +get it back without someone re-granting it. + +## Permission re-adjustment {#permission-level-re-adjustment} + +Database-level permissions include access to future tables. If you revoke access +to one table, QuestDB automatically converts the database-level grant to +individual table-level grants: + +```questdb-sql +GRANT SELECT ON ALL TABLES TO user; -- Database level +REVOKE SELECT ON secret_table FROM user; + +-- Result: user now has table-level SELECT on all tables EXCEPT secret_table +-- Future tables will NOT be accessible +``` + +The same applies from table to column level: + +```questdb-sql +GRANT SELECT ON trades TO user; -- Table level +REVOKE SELECT ON trades(ssn) FROM user; -- Revoke one column + +-- Result: user has column-level SELECT on all columns EXCEPT ssn +-- Future columns will NOT be accessible +``` + +:::note + +When dropping a table, permissions on it are preserved by default (useful if the +table is recreated). Use `DROP TABLE ... CASCADE PERMISSIONS` to also remove all +associated permissions. + +::: + +## Implicit timestamp permissions {#implicit-permissions} + +If a user has SELECT or UPDATE on any column of a table, they automatically get +the same permission on the designated timestamp column. This ensures time-series +operations (SAMPLE BY, LATEST ON, etc.) work correctly. + +## Granting on non-existent objects {#grant-verification} + +You can grant permissions on tables/columns that don't exist yet: + +```questdb-sql +GRANT INSERT ON future_table TO app; +-- Permission activates when future_table is created +``` + +Use `WITH VERIFICATION` to catch typos: + +```questdb-sql +GRANT SELECT ON trdaes TO user WITH VERIFICATION; +-- Fails immediately because 'trdaes' doesn't exist +``` + +## See also + +- [Permissions reference](/docs/security/rbac/permissions-reference/) - every + permission and the level it can be granted at +- [Common scenarios](/docs/security/rbac/common-scenarios/) - worked grants for + analysts, service accounts, teams and admins +- [`GRANT`](/docs/query/sql/acl/grant/) and + [`REVOKE`](/docs/query/sql/acl/revoke/) statement syntax diff --git a/documentation/security/rbac/index.mdx b/documentation/security/rbac/index.mdx new file mode 100644 index 0000000000..14992466e9 --- /dev/null +++ b/documentation/security/rbac/index.mdx @@ -0,0 +1,185 @@ +--- +title: Role-based Access Control (RBAC) +sidebar_label: Overview +slug: /security/rbac +description: + Granular access control from database level down to individual columns and + rows. Learn how to secure your QuestDB instance with users, groups, and + fine-grained permissions. +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Role-based Access Control (RBAC) provides fine-grained permissions for your + QuestDB instance. + + +QuestDB Enterprise provides fine-grained access control that can restrict access +at **database**, **table**, **column**, and even **row** level (using views). + +## Quick start + +Here's a complete example to create a read-only analyst user in under a minute: + +```questdb-sql +-- 1. Create the user +CREATE USER analyst WITH PASSWORD 'secure_password_here'; + +-- 2. Grant endpoint access (required to connect) +GRANT PGWIRE, HTTP TO analyst; + +-- 3. Grant read access to specific tables +GRANT SELECT ON trades, prices TO analyst; + +-- Done! The analyst can now connect and query trades and prices tables +``` + +To verify: + +```questdb-sql +SHOW PERMISSIONS analyst; +``` + +## Access control depth + +QuestDB's access control operates across two dimensions: + +### Data access granularity + +Control _what data_ users can access: + +| Level | What you can control | Example | +| ------------ | ------------------------------- | ----------------------------------------------------- | +| **Database** | All tables, global operations | `GRANT SELECT ON ALL TABLES TO user` | +| **Table** | Specific tables | `GRANT SELECT ON trades TO user` | +| **Column** | Specific columns within a table | `GRANT SELECT ON trades(ts, price) TO user` | +| **Row** | Specific rows via views | Create a view with WHERE clause, grant access to view | + +### Connection access granularity + +Control _how_ users can connect: + +| Permission | Protocol | Use case | +| ---------- | ------------------------------- | ------------------------------------------ | +| `HTTP` | REST API, Web Console, ILP/HTTP | Interactive users, web applications | +| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | +| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | + +```questdb-sql +-- User can connect via PostgreSQL protocol only (not web console) +GRANT PGWIRE TO analyst; + +-- Service can only ingest via ILP, cannot query +GRANT ILP TO ingest_service; + +-- Full interactive access +GRANT HTTP, PGWIRE TO developer; +``` + +These dimensions are independent: a user might have `SELECT` on all tables but +only be allowed to connect via `PGWIRE`, or have `INSERT` permission but only +via `ILP`. + +## Where to go next + +| Page | Covers | +| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | +| [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) | The three principal types, and the statements that manage them | +| [Authentication and endpoint access](/docs/security/rbac/authentication/) | Passwords, JWK and REST tokens, OIDC, and the `HTTP`/`PGWIRE`/`ILP` permissions | +| [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) | Permission levels, `GRANT ALL`, the grant option, and level re-adjustment | +| [Permissions reference](/docs/security/rbac/permissions-reference/) | Every permission and the level it can be granted at | +| [Common scenarios](/docs/security/rbac/common-scenarios/) | Analysts, service accounts, teams, admins, failover operators | +| [OpenID Connect](/docs/security/oidc/) | SSO against an external Identity Provider | + +## SQL commands reference + +- [ADD USER](/docs/query/sql/acl/add-user/) +- [ALTER USER](/docs/query/sql/acl/alter-user/) +- [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) +- [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) +- [CREATE GROUP](/docs/query/sql/acl/create-group/) +- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) +- [CREATE USER](/docs/query/sql/acl/create-user/) +- [DROP GROUP](/docs/query/sql/acl/drop-group/) +- [DROP SERVICE ACCOUNT](/docs/query/sql/acl/drop-service-account/) +- [DROP USER](/docs/query/sql/acl/drop-user/) +- [EXIT SERVICE ACCOUNT](/docs/query/sql/acl/exit-service-account/) +- [GRANT](/docs/query/sql/acl/grant/) +- [GRANT ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/grant-assume-service-account/) +- [REMOVE USER](/docs/query/sql/acl/remove-user/) +- [REVOKE](/docs/query/sql/acl/revoke/) +- [REVOKE ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/revoke-assume-service-account/) +- [SHOW USER](/docs/query/sql/show/#show-user) +- [SHOW USERS](/docs/query/sql/show/#show-users) +- [SHOW GROUPS](/docs/query/sql/show/#show-groups) +- [SHOW SERVICE ACCOUNT](/docs/query/sql/show/#show-service-account) +- [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) +- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) + +## Permissions reference {#permissions} + +Every permission QuestDB supports, and the level each can be granted at, is +listed on its own page. See +[Permissions reference](/docs/security/rbac/permissions-reference/). + +## Authentication methods {#authentication} + +QuestDB accepts passwords, JWK tokens, REST API tokens, and OIDC bearer tokens. +See +[Authentication and endpoint access](/docs/security/rbac/authentication/#authentication). + +## Endpoint permissions {#endpoint-permissions} + +`HTTP`, `PGWIRE` and `ILP` decide which protocol a principal may connect over. +See +[Authentication and endpoint access](/docs/security/rbac/authentication/#endpoint-permissions). + +## User management {#user-management} + +Creating and removing principals, group membership, passwords and tokens, and +the `SHOW` statements that inspect them. See +[Users, service accounts, and groups](/docs/security/rbac/users-and-groups/#user-management). + +## Failover operator {#failover-operator} + +A service account that can move the primary role between nodes and nothing else. +See [Common scenarios](/docs/security/rbac/common-scenarios/#failover-operator). + +## Where each section moved {#moved} + +These anchors are kept so that existing links keep working. Prefer the pages +above when adding new links. + +| Section | Now at | +| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- | +| Column-level access | [Granting permissions](/docs/security/rbac/granting-permissions/#column-level-access) | +| Row-level access with views | [Granting permissions](/docs/security/rbac/granting-permissions/#row-level-access-with-views) | +| Common scenarios | [Common scenarios](/docs/security/rbac/common-scenarios/) | +| Read-only analyst | [Common scenarios](/docs/security/rbac/common-scenarios/#read-only-analyst) | +| Application service account | [Common scenarios](/docs/security/rbac/common-scenarios/#application-service-account) | +| Team-based access with groups | [Common scenarios](/docs/security/rbac/common-scenarios/#team-based-access-with-groups) | +| Column-level restrictions | [Common scenarios](/docs/security/rbac/common-scenarios/#column-level-restrictions-hide-sensitive-data) | +| Row-level security | [Common scenarios](/docs/security/rbac/common-scenarios/#row-level-security-multi-tenant) | +| Database administrator | [Common scenarios](/docs/security/rbac/common-scenarios/#database-administrator) | +| Core concepts | [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) | +| Users and service accounts | [Users and groups](/docs/security/rbac/users-and-groups/#users-and-service-accounts) | +| Why service accounts? | [Users and groups](/docs/security/rbac/users-and-groups/#why-service-accounts) | +| Groups | [Users and groups](/docs/security/rbac/users-and-groups/#groups) | +| Built-in admin | [Users and groups](/docs/security/rbac/users-and-groups/#built-in-admin) | +| Service account assumption | [Users and groups](/docs/security/rbac/users-and-groups/#service-account-assumption) | +| Creating and removing principals | [Users and groups](/docs/security/rbac/users-and-groups/#creating-and-removing-principals) | +| Managing group membership | [Users and groups](/docs/security/rbac/users-and-groups/#managing-group-membership) | +| Viewing information | [Users and groups](/docs/security/rbac/users-and-groups/#viewing-information) | +| Managing authentication | [Authentication](/docs/security/rbac/authentication/#managing-authentication) | +| Permission levels | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-levels) | +| Granting ALL | [Granting permissions](/docs/security/rbac/granting-permissions/#granting-all) | +| The GRANT option | [Granting permissions](/docs/security/rbac/granting-permissions/#the-grant-option) | +| Owner permissions | [Granting permissions](/docs/security/rbac/granting-permissions/#owner-grants) | +| Advanced topics | [Granting permissions](/docs/security/rbac/granting-permissions/) | +| Permission re-adjustment | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-level-re-adjustment) | +| Implicit timestamp permissions | [Granting permissions](/docs/security/rbac/granting-permissions/#implicit-permissions) | +| Granting on non-existent objects | [Granting permissions](/docs/security/rbac/granting-permissions/#grant-verification) | +| Database permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#database-permissions) | +| User management permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#user-management-permissions) | +| Special permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#special-permissions) | diff --git a/documentation/security/rbac/permissions-reference.mdx b/documentation/security/rbac/permissions-reference.mdx new file mode 100644 index 0000000000..e0254e5622 --- /dev/null +++ b/documentation/security/rbac/permissions-reference.mdx @@ -0,0 +1,138 @@ +--- +title: QuestDB permissions reference +sidebar_label: Permissions reference +description: + Every permission QuestDB Enterprise supports, the level each can be granted + at, and the operations reserved for SYSTEM ADMIN. +--- + +Every permission QuestDB Enterprise supports, and the level each one can be +granted at. To see the same list from a running instance, including any +permission added after this page was written: + +```questdb-sql +SELECT * FROM all_permissions(); +``` + +## Database permissions + +| Permission | Level | Description | +| ---------------------------- | ----------------------------------- | -------------------------------------------------------------- | +| ADD COLUMN | Database | Table | Add columns to tables | +| ADD INDEX | Database | Table | Column | Add index on symbol columns | +| ALTER COLUMN CACHE | Database | Table | Column | Enable/disable symbol caching | +| ALTER COLUMN TYPE | Database | Table | Column | Change column types | +| ALTER SYMBOL CAPACITY | Database | Table | Column | Change the capacity of a symbol column | +| ALTER VIEW | Database | Table | Change a view definition | +| ATTACH PARTITION | Database | Table | Attach partitions | +| BACKUP DATABASE | Database | Create database backups | +| BACKUP TABLE | Database | Table | Legacy, grants nothing: there is no `BACKUP TABLE` statement | +| CANCEL ANY COPY | Database | Cancel COPY operations | +| COMPILE VIEW | Database | Table | Recompile a view against the current schema | +| CONVERT PARTITION TO NATIVE | Database | Table | Convert Parquet partitions back to native format | +| CONVERT PARTITION TO PARQUET | Database | Table | Convert partitions to Parquet in place | +| CREATE TABLE | Database | Create tables | +| CREATE MATERIALIZED VIEW | Database | Create materialized views | +| CREATE VIEW | Database | Create views | +| CREATE LIVE VIEW | Database | Create live views | +| DEDUP ENABLE | Database | Table | Enable deduplication | +| DEDUP DISABLE | Database | Table | Disable deduplication | +| DETACH PARTITION | Database | Table | Detach partitions | +| DISABLE STORAGE POLICY | Database | Table | Disable storage policies | +| DROP COLUMN | Database | Table | Column | Drop columns | +| DROP INDEX | Database | Table | Column | Drop indexes | +| DROP PARTITION | Database | Table | Drop partitions | +| DROP TABLE | Database | Table | Drop tables | +| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | +| DROP VIEW | Database | Table | Drop views | +| DROP LIVE VIEW | Database | Table | Drop live views | +| ENABLE STORAGE POLICY | Database | Table | Enable storage policies | +| INSERT | Database | Table | Insert data | +| REFRESH MATERIALIZED VIEW | Database | Table | Refresh materialized views | +| REINDEX | Database | Table | Column | Reindex columns | +| REMOVE STORAGE POLICY | Database | Table | Remove storage policies | +| RENAME COLUMN | Database | Table | Column | Rename columns | +| RENAME TABLE | Database | Table | Rename tables | +| RESUME WAL | Database | Table | Resume WAL processing | +| SELECT | Database | Table | Column | Read data | +| SET PARQUET SETTINGS | Database | Table | Set per-column Parquet encoding, compression and bloom filters | +| SET REFRESH LIMIT | Database | Table | Set a materialized view's refresh limit | +| SET REFRESH TYPE | Database | Table | Change a materialized view's refresh strategy | +| SET STORAGE POLICY | Database | Table | Set storage policies | +| SET TABLE FORMAT | Database | Table | Switch a table's partition format between native and Parquet | +| SET TABLE PARAM | Database | Table | Set table parameters | +| SET TABLE TYPE | Database | Table | Change table type | +| SETTINGS | Database | Change instance settings in Web Console | +| SNAPSHOT | Database | Create snapshots | +| SQL ENGINE ADMIN | Database | List/cancel running queries | +| SWITCH ROLE | Database | Switch the replication role, read SWITCH STATUS | +| SYSTEM ADMIN | Database | System functions (reload_tls, etc.) | +| TRUNCATE TABLE | Database | Table | Truncate tables | +| UPDATE | Database | Table | Column | Update data | +| VACUUM TABLE | Database | Table | Reclaim storage | + +## Endpoint permissions {#endpoint-permissions} + +These control which protocol a principal may connect on. They are covered in +detail under +[authentication and endpoint access](/docs/security/rbac/authentication/#endpoint-permissions). + +| Permission | Level | Description | +| ---------- | -------- | -------------------------------------- | +| HTTP | Database | REST API, Web Console, ILP over HTTP | +| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | +| ILP | Database | InfluxDB Line Protocol TCP (port 9009) | + +## User management permissions + +| Permission | Description | +| ---------------------- | --------------------------------------- | +| ADD EXTERNAL ALIAS | Create external group mappings | +| ADD PASSWORD | Set user passwords | +| ADD USER | Add users to groups | +| CREATE GROUP | Create groups | +| CREATE JWK | Create JWK tokens | +| CREATE REST TOKEN | Create REST API tokens | +| CREATE SERVICE ACCOUNT | Create service accounts | +| CREATE USER | Create users | +| DISABLE USER | Disable users | +| DROP GROUP | Drop groups | +| DROP JWK | Drop JWK tokens | +| DROP REST TOKEN | Drop REST API tokens | +| DROP SERVICE ACCOUNT | Drop service accounts | +| DROP USER | Drop users | +| ENABLE USER | Enable users | +| LIST USERS | List users/groups/service accounts | +| REMOVE EXTERNAL ALIAS | Remove external group mappings | +| REMOVE PASSWORD | Remove passwords | +| REMOVE USER | Remove users from groups | +| USER DETAILS | View user/group/service account details | + +## Special permissions + +| Permission | Description | +| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| ALL | All permissions at the granted level (database/table/column). At the database level this includes `DATABASE ADMIN` - see [granting ALL](/docs/security/rbac/granting-permissions/#granting-all) | +| DATABASE ADMIN | All permissions including future ones; can assume any service account | + +A few destructive operations have no permission of their own. They are gated by +`SYSTEM ADMIN`, so the only way to grant them is to grant `SYSTEM ADMIN`, which +also carries every other system function: + +| Operation | Gated by | Notes | +| ----------------------------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------ | +| [`ALTER TABLE REBASE WAL`](/docs/query/sql/alter-table-rebase-wal/) | `SYSTEM ADMIN` | Rebuilds a suspended WAL table under a fresh sequencer. A table-level `RESUME WAL` grant is not enough | +| [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) | `SYSTEM ADMIN` | Moves the [cold storage](/docs/concepts/cold-storage/) manager role, including with `FORCE` | + +By contrast, [`SWITCH ROLE`](/docs/query/sql/switch-role/), which moves the +replication role, has its own dedicated permission that can be granted without +handing over the rest of `SYSTEM ADMIN`. + +## See also + +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + levels, `WITH GRANT OPTION`, and how grants shift level when you revoke +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + the `HTTP`, `PGWIRE` and `ILP` permissions in context +- [`GRANT`](/docs/query/sql/acl/grant/) and + [`REVOKE`](/docs/query/sql/acl/revoke/) statement syntax diff --git a/documentation/security/rbac/users-and-groups.mdx b/documentation/security/rbac/users-and-groups.mdx new file mode 100644 index 0000000000..af874b029d --- /dev/null +++ b/documentation/security/rbac/users-and-groups.mdx @@ -0,0 +1,197 @@ +--- +title: Users, service accounts, and groups +sidebar_label: Users and groups +description: + The three principal types in QuestDB Enterprise RBAC - users, service + accounts, and groups - how they differ, and the statements that create and + manage them. +--- + +import Screenshot from "@theme/Screenshot" + +Every permission in QuestDB is granted to a principal. This page covers the +three kinds, how they differ, and the statements that manage them. + + + +## Users and service accounts + +QuestDB has two types of principals: + +- **Users**: For human individuals. Can belong to multiple groups and inherit + permissions from them. Cannot be assumed by others. +- **Service accounts**: For applications. Cannot belong to groups - all + permissions must be granted directly. Can be assumed by authorized users for + testing. + +```questdb-sql +CREATE USER human_user WITH PASSWORD 'pwd'; +CREATE SERVICE ACCOUNT app_account WITH PASSWORD 'pwd'; +``` + +Names must be unique across all users, service accounts, and groups. + +### Why service accounts? + +Service accounts provide **clean, testable application access**: + +| Aspect | User | Service Account | +| -------------------- | ------------------------------ | ---------------------- | +| Permission source | Direct + inherited from groups | Direct only | +| Can belong to groups | Yes | No | +| Can be assumed (SU) | No | Yes | +| Typical use | Human individuals | Applications, services | + +Because service accounts have no inherited permissions, their access is fully +explicit and predictable. Combined with the ability to assume them, this makes +it easy to verify exactly what an application can and cannot do: + +```questdb-sql +-- Create service account with specific permissions +CREATE SERVICE ACCOUNT trading_app WITH PASSWORD 'pwd'; +GRANT ILP TO trading_app; +GRANT INSERT ON trades TO trading_app; +GRANT SELECT ON positions TO trading_app; + +-- Developer can assume the service account to test its access +GRANT ASSUME SERVICE ACCOUNT trading_app TO developer; + +-- Developer switches to service account context +ASSUME SERVICE ACCOUNT trading_app; +-- Now operating with trading_app's exact permissions +-- Test what works and what doesn't... +EXIT SERVICE ACCOUNT; +``` + +This makes service accounts ideal for applications where you need predictable, +auditable, and testable access control. + +### Service account assumption + +Users can temporarily assume a service account's permissions for debugging: + +```questdb-sql +-- Grant ability to assume +GRANT ASSUME SERVICE ACCOUNT ingest_app TO developer; + +-- Developer can now switch context +ASSUME SERVICE ACCOUNT ingest_app; +-- ... debug with app's permissions ... +EXIT SERVICE ACCOUNT; +``` + +## Groups + +Groups simplify permission management when multiple users need the same access: + +```questdb-sql +CREATE GROUP analysts; +GRANT SELECT ON ALL TABLES TO analysts; + +-- All users added to this group can read all tables +ADD USER alice TO analysts; +ADD USER bob TO analysts; +``` + +Users inherit permissions from their groups. Inherited permissions cannot be +revoked directly from the user - revoke from the group instead. When a group is +dropped, all members lose the permissions they inherited from that group. + +For OIDC users, group membership comes from the Identity Provider rather than +from `ADD USER`. See +[mapping groups and permissions](/docs/security/oidc/group-mapping/). + +## Built-in admin + +Every QuestDB instance starts with a built-in admin account: + +- Default username: `admin` +- Default password: `quest` + +**Change these immediately in production** via `server.conf`: + +```ini +acl.admin.user=your_admin_name +acl.admin.password=your_secure_password +``` + +The built-in admin has irrevocable root access. After creating other admin +users, disable it: + +```ini +acl.admin.user.enabled=false +``` + +In a replicated cluster, keep in mind that the built-in admin authorizes a +[role switch](/docs/high-availability/failover/) from its own credentials, +independently of the replicated access lists. It is the break-glass account when +a `SWITCH ROLE` grant has not yet replicated to the node you need to promote. + +## Management reference {#user-management} + +### Creating and removing principals + +```questdb-sql +-- Users +CREATE USER username WITH PASSWORD 'pwd'; +DROP USER username; + +-- Service accounts +CREATE SERVICE ACCOUNT appname WITH PASSWORD 'pwd'; +DROP SERVICE ACCOUNT appname; + +-- Groups +CREATE GROUP groupname; +DROP GROUP groupname; +``` + +### Managing group membership + +```questdb-sql +ADD USER username TO group1, group2; +REMOVE USER username FROM group1; +``` + +Passwords and tokens are managed with `ALTER USER`; see +[managing authentication](/docs/security/rbac/authentication/#managing-authentication). + +### Viewing information + +```questdb-sql +SHOW USERS; -- List all users +SHOW SERVICE ACCOUNTS; -- List all service accounts +SHOW GROUPS; -- List all groups +SHOW GROUPS username; -- List groups for a user +SHOW USER username; -- Show auth methods for user +SHOW PERMISSIONS username; -- Show permissions for user +``` + +Example output from `SHOW USER`: + +``` +auth_type enabled +--------- ------- +Password true +JWK Token false +REST Token true +``` + +:::note + +Viewing other users' information requires `LIST USERS` (to list all) or +`USER DETAILS` (to see details) permissions. Users can always view their own +information without these permissions. + +::: + +## See also + +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + how principals prove who they are and which protocols they may use +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + what a principal is allowed to do once connected diff --git a/documentation/sidebars.js b/documentation/sidebars.js index 4b13ed9be8..51d7306bfc 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -714,9 +714,36 @@ module.exports = { type: "category", items: [ { - id: "security/rbac", - type: "doc", label: "Role-Based Access Control (RBAC)", + type: "category", + link: { type: "doc", id: "security/rbac/index" }, + items: [ + { + id: "security/rbac/users-and-groups", + type: "doc", + label: "Users and groups", + }, + { + id: "security/rbac/authentication", + type: "doc", + label: "Authentication and endpoints", + }, + { + id: "security/rbac/granting-permissions", + type: "doc", + label: "Granting permissions", + }, + { + id: "security/rbac/permissions-reference", + type: "doc", + label: "Permissions reference", + }, + { + id: "security/rbac/common-scenarios", + type: "doc", + label: "Common scenarios", + }, + ], }, { label: "OpenID Connect (OIDC)", From 0574137b7386046320b3a5d81c742ee3e02f2ceb Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 17:31:56 +0100 Subject: [PATCH 46/54] Correct what the clients do without a terminal The sign-in prompt table claimed Rust, C and C++ fail with an interaction-required error when there is no terminal, and that Python fails unless the kernel accepts stdin, summarised as a container with no TTY failing outright in four of the five clients. Every one of those clients deliberately has no TTY check, and says so: a missing terminal is not evidence of a missing human, since a pipe into tee, a supervisor or an IDE capturing stderr all still show the code to someone. A headless container therefore prompts to a stream nobody reads and polls until the device code expires, which is the opposite of what the page promised its own headline audience. All five rows now say the call runs anyway. Python's one exception is stated for what it is: positive evidence from a notebook executor issuing allow_stdin=false, not a terminal check. The fail-fast lever moves out of the custom-prompt table into its own, in the direction that is useful. It read .interactive(true) for Rust and C++, which is the default and so a no-op, and would have left the same wrong model in place next to the corrected table above it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014Cz9Rp9SBqLYSUsFkGFbGN --- documentation/security/oidc-device-flow.mdx | 53 +++++++++++++++------ 1 file changed, 38 insertions(+), 15 deletions(-) diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 6f0886da21..e31c4fbe3f 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -191,12 +191,36 @@ call runs at all without a terminal, differs by client: | Client | Default prompt | Without a terminal | | --- | --- | --- | -| Java | prints to `System.out` | runs anyway; the client performs no terminal check | -| Python | prints to stderr, or to the frontend inside an IPython kernel | fails, unless the kernel accepts stdin | -| Rust, C, C++ | prints to stderr | fails with an interaction-required error | - -Redirecting output therefore hides the code in Java, while a container with no -TTY fails outright in the other four. +| Java | prints to `System.out` | runs anyway | +| Python | prints to stderr, or to the frontend inside an IPython kernel | runs anyway, except under a notebook executor which refuses stdin | +| Rust, C, C++ | prints to stderr | runs anyway | + +None of the clients tests for a TTY. A missing terminal is not evidence of a +missing human: a pipe into `tee`, a process supervisor, or an IDE which captures +stderr all still put the code in front of someone. The call therefore prompts +and waits whatever the process is attached to, until the user approves or the +device code expires. Redirecting or discarding the output hides the code without +shortening that wait. + +The one client which stops early does so on positive evidence rather than on a +guess. A notebook executor such as papermill, `nbclient` or +`jupyter nbconvert --execute` runs a real kernel, but issues every request with +`allow_stdin=False`, which is the frontend stating that nobody is there to +answer. The Python client reads that flag and fails with +[`OidcInteractionRequired`](#interaction-required-errors) instead of polling to +the device code's deadline. + +A headless service or a CI job should say so itself rather than rely on the +absence of a terminal. Turning the prompt off makes sign-in fail immediately +with an [interaction-required error](#interaction-required-errors): + +| Client | Fail instead of prompting | +| --- | --- | +| Java | not available | +| Python | `interactive=False` | +| Rust | `.interactive(false)` | +| C | `questdb_oidc_builder_interactive(builder, false, &error)` | +| C++ | `.interactive(false)` | All five clients also try to open the verification URL in a local browser by default. That is useful on a workstation and pointless on a remote or headless @@ -212,16 +236,15 @@ inside a Jupyter kernel. Turn it off explicitly elsewhere: | C | `questdb_oidc_builder_open_browser(builder, false, &error)` | | C++ | `.open_browser(false)` | -Override the default prompt to render the challenge elsewhere, or declare the -process interactive anyway: +Override the default prompt to render the challenge elsewhere: -| Client | Custom prompt | Force interactive | -| --- | --- | --- | -| Java | `prompt(...)` | not applicable | -| Python | `renderer=`, or `qr=True` for a QR code | `interactive=True`, and `interactive=False` to fail rather than prompt | -| Rust | `.renderer(...)` | `.interactive(true)` | -| C | `questdb_oidc_builder_event_handler()` | `questdb_oidc_builder_interactive()` | -| C++ | `.event_handler(...)` | `.interactive(true)` | +| Client | Custom prompt | +| --- | --- | +| Java | `prompt(...)` | +| Python | `renderer=`, or `qr=True` for a QR code | +| Rust | `.renderer(...)` | +| C | `questdb_oidc_builder_event_handler()` | +| C++ | `.event_handler(...)` | The Java, C and C++ clients strip terminal controls, bidi marks and zero-width characters from the prompt's text fields before the callback sees them. The Rust From d381b712c9b6ac004f8800202cb22d0bea5518c6 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 17:36:23 +0100 Subject: [PATCH 47/54] Fix the OIDC and RBAC examples which do not run Five examples across four pages either fail at runtime, fail to compile, or describe syntax the server does not have. The basic authentication example in the client integration guide calls parse.urlencode() and json.loads() while importing neither, so it raises NameError twice. Its own import list is self-contained, unlike the ROPC example above it, which continues a block that does import both. Both requests in that section posted to http://localhost:9000, one carrying an OIDC bearer token and the other base64 SSO credentials, while the caution further down the same page says both requests must go over https because a tampered plaintext response can redirect the password grant. They now use the https://questdb.example.com:9000 the rest of the page uses. The Entra ID walkthrough creates the group extUsers and then grants twice to groupName, which is never created, so running the three blocks in order fails on the second. The placeholder table names go too, for the demo tables. DROP TABLE ... CASCADE PERMISSIONS does not exist, and the default the note described is the opposite of what happens: EntDdlListener drops a table's permissions unconditionally, for tables, views, materialized views and live views alike, so a drop-and-recreate silently loses every grant. The Rust token store line used ? on at_default_location(), which returns std::io::Result. There is no From for questdb::Error, so it does not compile in the surrounding example's own signature. It now shows the map_err form the upstream oidc_device_auth.rs example uses, and names the infallible alternative. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014Cz9Rp9SBqLYSUsFkGFbGN --- documentation/connect/clients/rust.md | 17 +++++++++++++---- .../security/oidc/client-integration.mdx | 7 ++++--- documentation/security/oidc/entra-id.mdx | 4 ++-- .../security/rbac/granting-permissions.mdx | 7 ++++--- 4 files changed, 23 insertions(+), 12 deletions(-) diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 7e26898642..30a875120d 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -198,10 +198,19 @@ Otherwise add `.issuer(...)` to the builder, or set `.client_id()`, `.scope()`, `.audience()`, `.token_endpoint()` and `.device_authorization_endpoint()` yourself. -Tokens stay in memory until -`.token_store(FileTokenStore::at_default_location()?)` is added (it returns a -`std::io::Result`, so map the error into yours), which writes a -long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and +Tokens stay in memory until a token store is attached, which writes a +long-lived refresh token to disk as plaintext. `at_default_location()` returns a +`std::io::Result`, and `?` does not convert an `io::Error` into a +`questdb::Error`, so map it first: + +```rust +let store = FileTokenStore::at_default_location() + .map_err(|e| Error::new(ErrorCode::ConfigError, format!("token store: {e}")))?; +let auth = OidcDeviceAuth::from_questdb(url).token_store(store).build()?; +``` + +`FileTokenStore::at(path)` takes a directory directly and needs no mapping. See +[token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and [explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). diff --git a/documentation/security/oidc/client-integration.mdx b/documentation/security/oidc/client-integration.mdx index 882c59af1a..bcea7eabf5 100644 --- a/documentation/security/oidc/client-integration.mdx +++ b/documentation/security/oidc/client-integration.mdx @@ -105,7 +105,7 @@ This token can be used to authenticate with QuestDB: query = parse.urlencode({ "query": "select current_user()" }) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") +req = request.Request(f"https://questdb.example.com:9000/api/v1/sql/execute?{query}") req.add_header("Authorization", f"Bearer {access_token}") with request.urlopen(req) as f: body = f.read().decode(f.headers.get_content_charset()) @@ -174,8 +174,9 @@ the OAuth2 provider. ```python from dotenv import load_dotenv import os -from urllib import request +from urllib import parse, request import base64 +import json load_dotenv() user = os.environ.get("username") @@ -184,7 +185,7 @@ pwd = os.environ.get("password") query = parse.urlencode({ "query": "select current_user()" }) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") +req = request.Request(f"https://questdb.example.com:9000/api/v1/sql/execute?{query}") b64credentials = base64.standard_b64encode(f"{user}:{pwd}".encode()).decode() req.add_header("Authorization", f"Basic {b64credentials}") with request.urlopen(req) as f: diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx index 42511d5d41..40b0868b2e 100644 --- a/documentation/security/oidc/entra-id.mdx +++ b/documentation/security/oidc/entra-id.mdx @@ -266,14 +266,14 @@ to make sure users can access the Web Console, Postgres and ILP interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions). ```questdb-sql title="Grant endpoint permissions" -GRANT HTTP, PGWIRE TO groupName; +GRANT HTTP, PGWIRE TO extUsers; ``` Now we can grant the rest of the permissions as required. We can grant access to tables, for example. ```questdb-sql title="Grant database permissions" -GRANT SELECT ON table1, table2 to groupName; +GRANT SELECT ON trades, fx_trades TO extUsers; ``` ## Confirm group mappings and login diff --git a/documentation/security/rbac/granting-permissions.mdx b/documentation/security/rbac/granting-permissions.mdx index 2e52d9fc22..56969da3e6 100644 --- a/documentation/security/rbac/granting-permissions.mdx +++ b/documentation/security/rbac/granting-permissions.mdx @@ -144,9 +144,10 @@ REVOKE SELECT ON trades(ssn) FROM user; -- Revoke one column :::note -When dropping a table, permissions on it are preserved by default (useful if the -table is recreated). Use `DROP TABLE ... CASCADE PERMISSIONS` to also remove all -associated permissions. +Dropping a table removes every permission granted on it, from every user and +group. Recreating a table with the same name does not bring them back, so a +drop-and-recreate has to re-grant them. The same applies to views, materialized +views and live views. ::: From c67eef6d0b0e1600df494a94cf3fae7295b0e4e8 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 17:36:30 +0100 Subject: [PATCH 48/54] Put the OIDC and RBAC section roots back in llms.txt Turning the two guides into sidebar categories dropped both overview pages out of the generated LLM index. The category branch of generate-llms-files.js emits item.label as plain text and never reads item.link, so a category root reachable only through link: produces a heading with no URL and no description. Before the split both were type: "doc" entries and appeared as linked bullets. That silently cost the index the RBAC quick start, the access control depth tables, the only consolidated list of the ACL SQL statements, and the OIDC architecture overview. Repeating the index doc as an explicit Overview item is what the Enterprise Kubernetes Operator category already does for the same reason. It also gives each section a visible Overview row rather than leaving the root reachable only by clicking the category header, and makes the sidebar_label in both index frontmatters live instead of dead metadata. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_014Cz9Rp9SBqLYSUsFkGFbGN --- documentation/sidebars.js | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/documentation/sidebars.js b/documentation/sidebars.js index 51d7306bfc..1677cddbd4 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -718,6 +718,11 @@ module.exports = { type: "category", link: { type: "doc", id: "security/rbac/index" }, items: [ + { + id: "security/rbac/index", + type: "doc", + label: "Overview", + }, { id: "security/rbac/users-and-groups", type: "doc", @@ -750,6 +755,11 @@ module.exports = { type: "category", link: { type: "doc", id: "security/oidc/index" }, items: [ + { + id: "security/oidc/index", + type: "doc", + label: "Overview", + }, { id: "security/oidc/how-sign-in-works", type: "doc", From 964b8c4909b30c85f27b3b28ef438173da3714d5 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 18:24:14 +0100 Subject: [PATCH 49/54] docs: correct OIDC client guidance --- documentation/configuration/oidc.md | 17 ++++++++++--- documentation/connect/clients/rust.md | 2 +- documentation/security/oidc-device-flow.mdx | 24 +++++++++---------- documentation/security/oidc/group-mapping.mdx | 14 +++++++---- 4 files changed, 35 insertions(+), 22 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index e2e06b088d..d06c48b01f 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -185,9 +185,14 @@ back. - **Default**: `false` - **Reloadable**: no -When enabled, the PGWire endpoint supports OIDC authentication. The OAuth2 -token should be sent in the password field, while the username field should -contain the string `_sso`, or left empty if that is an option. +When enabled, the PGWire endpoint accepts an OAuth2 token obtained by the +client. The token should be sent in the password field, while the username field +should contain the string `_sso`, or be left empty if that is an option. + +This setting is not required for the ROPC path. To let a client such as `psql` +send the user's SSO username and password and have QuestDB exchange them for a +token, enable [`acl.oidc.ropc.flow.enabled`](#acloidcropcflowenabled) instead. +The two settings do not need to be enabled together. ### acl.oidc.pkce.required @@ -212,6 +217,12 @@ the OIDC Provider's token endpoint as a password grant, and the user is logged in if the provider issues a token. This lets clients which cannot follow a browser redirect, such as `psql`, authenticate with their SSO credentials. +For this `psql` login path, enable this setting along with the normal OIDC +configuration. +[`acl.oidc.pg.token.as.password.enabled`](#acloidcpgtokenaspasswordenabled) is +not required; that setting is for clients which obtain an OAuth2 token +themselves and send the token, rather than their SSO password, to QuestDB. + Local users are matched first, so a QuestDB user whose name also exists in the Identity Provider is authenticated against its local password, without involving the provider. diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 30a875120d..6406490b18 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -172,7 +172,7 @@ Enable the `oidc` feature to sign in an interactive user with the ```toml title="Cargo.toml" [dependencies] -questdb-rs = { version = "7", features = ["oidc"] } +questdb-rs = { version = "7.1", features = ["oidc"] } ``` `OidcDeviceAuth::from_questdb` discovers the provider endpoints, client ID, diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index e31c4fbe3f..27268663a7 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -154,7 +154,7 @@ Enable the `oidc` crate feature: ```toml title="Cargo.toml" [dependencies] -questdb-rs = { version = "7", features = ["oidc"] } +questdb-rs = { version = "7.1", features = ["oidc"] } ``` @@ -246,18 +246,17 @@ Override the default prompt to render the challenge elsewhere: | C | `questdb_oidc_builder_event_handler()` | | C++ | `.event_handler(...)` | -The Java, C and C++ clients strip terminal controls, bidi marks and zero-width -characters from the prompt's text fields before the callback sees them. The Rust -and Python clients sanitize inside their built-in renderers only, so a custom +The Java, Python, C and C++ clients strip terminal controls, bidi marks and +zero-width characters from the prompt's text fields before the callback sees +them. Rust sanitizes inside its built-in renderer only, so a custom Rust renderer receives the Identity Provider's response verbatim. :::caution -A custom Rust or Python renderer must sanitize the user code and verification -URL itself before writing them to a terminal or a DOM. Rendering them raw -reopens the spoofing surface the built-in renderers close. Rust exposes -`display_user_code()` and `display_verification_uri()` on the challenge; Python -exports `sanitize_display_text` from `questdb.auth`. +A custom Rust renderer must sanitize the user code and verification URL itself +before writing them to a terminal or a DOM. Rendering them raw reopens the +spoofing surface the built-in renderer closes. Rust exposes +`display_user_code()` and `display_verification_uri()` on the challenge. ::: @@ -268,10 +267,9 @@ the Rust challenge, and the `browser_target` key on the Python prompt event. ## Explicit configuration and endpoint pinning -Discovery supplies the client ID, scope, audience, token-selection mode, and -endpoints. Set any of them explicitly when the application has its own -registration with the provider, or when QuestDB does not publish what the client -needs: +Discovery supplies the client ID, scope, token-selection mode, and endpoints. +Set configuration explicitly when the application has its own registration with +the provider, or when QuestDB does not publish what the client needs: | Setting | Set it when | | --- | --- | diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx index 991795d77b..b943f08a00 100644 --- a/documentation/security/oidc/group-mapping.mdx +++ b/documentation/security/oidc/group-mapping.mdx @@ -60,11 +60,15 @@ ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=qu ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; ``` -QuestDB works the list of external groups out from the User Info response -message. - -If we take the example used earlier, we will see that the message contains a -claim called `groups`. Its name is set with +QuestDB can obtain the list of external groups from either the User Info +response or the ID token. By default, it reads them from the User Info +response. When +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is `true`, it reads them from the ID token instead; this mode supports providers +such as Entra ID, which cannot return groups from the User Info endpoint. + +In the User Info example used earlier, the response contains a claim called +`groups`. Its name is set with [`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which has no default and must be set whenever OIDC is enabled. From b1257cacafc7e089bc68d4153cda034f5334394b Mon Sep 17 00:00:00 2001 From: glasstiger Date: Fri, 4 Sep 2026 18:56:37 +0100 Subject: [PATCH 50/54] docs: polish OIDC and RBAC restructure --- documentation/changelog.mdx | 1 + documentation/configuration/oidc.md | 16 +++-- .../query/sql/alter-table-rebase-wal.md | 3 +- .../query/sql/switch-cold-storage-role.md | 5 +- documentation/security/oidc-device-flow.mdx | 8 +-- .../security/oidc/client-discovery.mdx | 8 +++ .../security/oidc/client-integration.mdx | 21 +++++-- documentation/security/oidc/entra-id.mdx | 9 +++ documentation/security/oidc/group-mapping.mdx | 8 +++ .../security/oidc/how-sign-in-works.mdx | 8 +++ documentation/security/oidc/index.mdx | 62 +++++++++++-------- documentation/security/oidc/pingfederate.mdx | 8 +++ .../security/rbac/authentication.mdx | 11 ++-- .../security/rbac/common-scenarios.mdx | 59 +++++++++--------- .../security/rbac/granting-permissions.mdx | 27 ++++---- documentation/security/rbac/index.mdx | 52 +++++----------- .../security/rbac/permissions-reference.mdx | 10 +-- .../security/rbac/users-and-groups.mdx | 4 +- 18 files changed, 189 insertions(+), 131 deletions(-) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 75dd08e47c..94ee0d36f4 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -90,6 +90,7 @@ This page tracks significant updates to the QuestDB documentation. ### Updated +- Restructured [OpenID Connect](/docs/security/oidc/) and [role-based access control](/docs/security/rbac/) from two monolithic guides into task-focused sections covering client flows, provider setup, group mapping, authentication, principals, permissions, and common scenarios, while preserving links to the former section anchors - [OpenID Connect (OIDC)](/docs/security/oidc/how-sign-in-works/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers - [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication - [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the expiry, the audience and the presence of the claims, but not `nbf` or `iss`. QuestDB never asks the provider about such a token, so a revoked token keeps working until it expires diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index d06c48b01f..2b7bb3ec4a 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -14,7 +14,9 @@ Management. The database can be integrated with any OAuth2/OIDC Identity Provider (IdP). For detailed information about OIDC, see the -[OpenID Connect (OIDC) integration guide](/docs/security/oidc). +[OpenID Connect (OIDC) integration guide](/docs/security/oidc/). For guidance on +choosing a flow for an application, see +[OIDC client integration patterns](/docs/security/oidc/client-integration/). ## Minimum configuration @@ -163,9 +165,15 @@ QuestDB uses the scopes in the requests it makes itself, in the run the flow themselves. For the [OIDC device flow](/docs/security/oidc-device-flow/), add -`offline_access`. Most providers, Microsoft Entra ID among them, issue a -refresh token only when that scope was requested, and without one the client -has to make the user sign in again as soon as the current token expires. +`offline_access` alongside `openid`: + +```ini title="server.conf" +acl.oidc.scope=openid offline_access +``` + +Most providers, Microsoft Entra ID among them, issue a refresh token only when +that scope was requested, and without one the client has to make the user sign +in again as soon as the current token expires. Providers which issue refresh tokens regardless ignore the extra scope, so adding it is the safe default. A client can also request it through its own `scope` override, which leaves this server-wide value, and every other flow diff --git a/documentation/query/sql/alter-table-rebase-wal.md b/documentation/query/sql/alter-table-rebase-wal.md index b7ec3edd3c..b77649c6d2 100644 --- a/documentation/query/sql/alter-table-rebase-wal.md +++ b/documentation/query/sql/alter-table-rebase-wal.md @@ -67,7 +67,8 @@ an error: ## Permissions `REBASE WAL` requires the `SYSTEM ADMIN` -[permission](/docs/security/rbac/permissions-reference/) rather than a table-level grant. +[permission](/docs/security/rbac/permissions-reference/#special-permissions) +rather than a table-level grant. This reflects that the operation is destructive: it discards un-applied transactions, changes the table's internal id, and replaces its on-disk directory. A user who can run the non-destructive diff --git a/documentation/query/sql/switch-cold-storage-role.md b/documentation/query/sql/switch-cold-storage-role.md index d5f531b1fb..7bc1a164eb 100644 --- a/documentation/query/sql/switch-cold-storage-role.md +++ b/documentation/query/sql/switch-cold-storage-role.md @@ -59,7 +59,10 @@ A role set this way does **not** survive a restart. Update `cold.storage.role` i ### Permissions -Both statements require the `SYSTEM ADMIN` [permission](/docs/security/rbac/permissions-reference/), including with `FORCE`. There is no narrower grant for the cold storage role: granting `SYSTEM ADMIN` also grants every other system function. +Both statements require the `SYSTEM ADMIN` +[permission](/docs/security/rbac/permissions-reference/#special-permissions), +including with `FORCE`. There is no narrower grant for the cold storage role: +granting `SYSTEM ADMIN` also grants every other system function. ## Examples diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc-device-flow.mdx index 27268663a7..4152105a73 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc-device-flow.mdx @@ -137,7 +137,7 @@ with in transit, so pin the provider with `issuer` as well, and pass explicitly any client ID, scope, audience, or token-selection mode you rely on. An issuer pin covers the endpoints, not those four values. - + @@ -303,7 +303,7 @@ against any server you do not fully trust, not only when an endpoint is missing. Pinning a provider and using an application's own registration: - + ```java @@ -366,7 +366,7 @@ and `.token_store(...)` respectively; the C and C++ calls above are builder methods and attach it themselves. Attach it before signing in, so the first `sign_in()` can reuse a refresh token an earlier run left behind: - + ```java @@ -432,7 +432,7 @@ handle both. Catching it and signing in again: - + ```python diff --git a/documentation/security/oidc/client-discovery.mdx b/documentation/security/oidc/client-discovery.mdx index e71e9b5c92..75b45ec542 100644 --- a/documentation/security/oidc/client-discovery.mdx +++ b/documentation/security/oidc/client-discovery.mdx @@ -125,3 +125,11 @@ straight out of its payload. The [Web Console](/docs/getting-started/web-console/overview/) picks the token this way, and so should any other client. +## See also + +- [Client integration patterns](/docs/security/oidc/client-integration/) - use + discovery and bearer tokens from browser, notebook, CLI, and PGWire clients +- [OIDC device flow](/docs/security/oidc-device-flow/) - discovery in the + official Java, Python, Rust, C, and C++ clients +- [OIDC settings reference](/docs/configuration/oidc/) - configure the values + published to clients diff --git a/documentation/security/oidc/client-integration.mdx b/documentation/security/oidc/client-integration.mdx index bcea7eabf5..fb6d15690f 100644 --- a/documentation/security/oidc/client-integration.mdx +++ b/documentation/security/oidc/client-integration.mdx @@ -352,10 +352,15 @@ with questdb.connect(conf) as db: ## OIDC for the PGWire endpoint -If the -Resource Owner Password Credentials (ROPC) flow is not an option, we can still authenticate via OIDC on the PGWire endpoint. -However, in this case the client's responsibility to source the token required for authentication. -This method works wherever a Postgres client library is available, including jupyter notebooks. +Clients such as `psql` that cannot run the device flow can still authenticate +via OIDC on the PGWire endpoint by obtaining an OAuth2 token separately and +sending it as the password. In this case, it is the client's responsibility to +source the token required for authentication. This method works wherever a +PostgreSQL client library is available, including Jupyter notebooks. + +Prefer this approach to the legacy Resource Owner Password Credentials (ROPC) +flow. If sourcing a token separately is not practical and the provider supports +ROPC, [enable QuestDB to run that flow](#enable-ropc) as the fallback. Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. @@ -377,3 +382,11 @@ with pg.connect(conn_str, autocommit=True) as connection: print(row) ``` +## See also + +- [Client discovery and tokens](/docs/security/oidc/client-discovery/) - find + the provider endpoints and select the token QuestDB expects +- [OIDC device flow](/docs/security/oidc-device-flow/) - interactive sign-in + for CLI applications, containers, and remote notebooks +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + grant the endpoint permission each client protocol requires diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx index 40b0868b2e..ea468d0f15 100644 --- a/documentation/security/oidc/entra-id.mdx +++ b/documentation/security/oidc/entra-id.mdx @@ -284,3 +284,12 @@ If all has been wired up well, then login will succeed, and the user will have the access granted to them.
+ +## See also + +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + map Entra ID group IDs to QuestDB groups +- [OIDC settings reference](/docs/configuration/oidc/) - all `acl.oidc.*` + settings and token-validation behavior +- [How sign-in works](/docs/security/oidc/how-sign-in-works/) - the browser + authorization flow used by the Web Console diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx index b943f08a00..c74da39bb5 100644 --- a/documentation/security/oidc/group-mapping.mdx +++ b/documentation/security/oidc/group-mapping.mdx @@ -121,3 +121,11 @@ by QuestDB. They could be displayed in the [Web Console](/docs/getting-started/web-console/overview/), or appear in the logs, for example. +## See also + +- [Permissions reference](/docs/security/rbac/permissions-reference/) - the + permissions external groups can receive +- [Microsoft Entra ID](/docs/security/oidc/entra-id/) - map group IDs carried + in an ID token +- [PingFederate](/docs/security/oidc/pingfederate/) - map groups returned by + the User Info endpoint diff --git a/documentation/security/oidc/how-sign-in-works.mdx b/documentation/security/oidc/how-sign-in-works.mdx index d42086524e..6fb0b94822 100644 --- a/documentation/security/oidc/how-sign-in-works.mdx +++ b/documentation/security/oidc/how-sign-in-works.mdx @@ -267,3 +267,11 @@ and then sends the results back: } ``` +## See also + +- [Client discovery and tokens](/docs/security/oidc/client-discovery/) - the + settings the Web Console and other clients discover +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + turn the final group claim into QuestDB permissions +- [OIDC settings reference](/docs/configuration/oidc/) - configure the browser + flow and provider endpoints diff --git a/documentation/security/oidc/index.mdx b/documentation/security/oidc/index.mdx index be4ce84a27..ebd7d01dfd 100644 --- a/documentation/security/oidc/index.mdx +++ b/documentation/security/oidc/index.mdx @@ -96,33 +96,6 @@ are encoded in it. For the `acl.oidc.*` settings, see the [OIDC settings reference](/docs/configuration/oidc/). -## Settings endpoint {#settings-endpoint} - -QuestDB publishes the client ID, scope, and the provider's endpoints on an -unauthenticated settings endpoint, so a client can discover them instead of -hard coding them. See -[Client discovery and tokens](/docs/security/oidc/client-discovery/#settings-endpoint). - -## Which token to send {#which-token-to-send} - -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -decides whether a client sends the access token or the ID token in the -`Authorization: Bearer` header. See -[Client discovery and tokens](/docs/security/oidc/client-discovery/#which-token-to-send). - -## OIDC device flow {#device-authorization-flow} - -For command-line applications, containers, and remote notebook kernels, where -no browser redirect can reach the client, QuestDB Enterprise supports the -OAuth 2.0 Device Authorization Grant. QuestDB does not run the flow: the client -talks to the Identity Provider directly and presents the resulting bearer -token. - -See [OIDC device flow](/docs/security/oidc-device-flow/) for the -protocol walkthrough, what to configure first, and worked examples for the -Java, Python, Rust, C and C++ clients. - - ## Where each section moved {#moved} These anchors are kept so that existing links keep working. Prefer the pages @@ -132,12 +105,47 @@ above when adding new links. | --- | --- | | Authentication and authorization flow | [How sign-in works](/docs/security/oidc/how-sign-in-works/) | | Secret generation | [How sign-in works](/docs/security/oidc/how-sign-in-works/#1-secret-generation) | +| Prove identity | [How sign-in works](/docs/security/oidc/how-sign-in-works/#2-prove-identity) | +| Scope consent | [How sign-in works](/docs/security/oidc/how-sign-in-works/#3-scope-consent) | +| Redirection | [How sign-in works](/docs/security/oidc/how-sign-in-works/#4-redirection) | +| Credential request | [How sign-in works](/docs/security/oidc/how-sign-in-works/#5-credential-request) | +| Credentials received | [How sign-in works](/docs/security/oidc/how-sign-in-works/#6-credentials-received) | +| Database access | [How sign-in works](/docs/security/oidc/how-sign-in-works/#7-database-access) | +| Find user information | [How sign-in works](/docs/security/oidc/how-sign-in-works/#8-find-user-information) | +| Receive user information | [How sign-in works](/docs/security/oidc/how-sign-in-works/#9-receive-user-information) | +| Permission check | [How sign-in works](/docs/security/oidc/how-sign-in-works/#10-permission-check) | | User permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | | Mapping user permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions) | | Interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/) | +| Browser-based clients | [Client integration patterns](/docs/security/oidc/client-integration/#browser-based-clients) | +| Jupyter notebook | [Client integration patterns](/docs/security/oidc/client-integration/#jupyter-notebook) | +| Externalizing credentials | [Client integration patterns](/docs/security/oidc/client-integration/#externalizing-credentials) | +| Enable ROPC | [Client integration patterns](/docs/security/oidc/client-integration/#enable-ropc) | +| CLI and standalone applications | [Client integration patterns](/docs/security/oidc/client-integration/#cli-standalone-applications) | | Non-interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/#non-interactive-clients) | | OIDC for the PGWire endpoint | [Client integration patterns](/docs/security/oidc/client-integration/#oidc-for-the-pgwire-endpoint) | +| Active Directory | [PingFederate](/docs/security/oidc/pingfederate/) or [Microsoft Entra ID](/docs/security/oidc/entra-id/) | | PingFederate | [PingFederate](/docs/security/oidc/pingfederate/) | +| Set up PingFederate client | [PingFederate](/docs/security/oidc/pingfederate/#set-up-pingfederate-client) | +| Access Token Manager settings | [PingFederate](/docs/security/oidc/pingfederate/#access-token-manager-settings) | +| Authorization Server settings | [PingFederate](/docs/security/oidc/pingfederate/#authorization-server-settings) | +| Set up a Microsoft Entra ID Data Source | [PingFederate](/docs/security/oidc/pingfederate/#set-up-a-microsoft-entra-id-data-source) | +| Set up a Password Credential Validator | [PingFederate](/docs/security/oidc/pingfederate/#set-up-a-password-credential-validator) | +| Set up an Identity Provider | [PingFederate](/docs/security/oidc/pingfederate/#set-up-an-identity-provider) | +| Add groups to OIDC policy management | [PingFederate](/docs/security/oidc/pingfederate/#add-groups-to-oidc-policy-management) | +| Enable Resource Owner Password Credentials flow | [PingFederate](/docs/security/oidc/pingfederate/#enable-resource-owner-password-credentials-ropc-flow) | | QuestDB configuration for PingFederate | [PingFederate](/docs/security/oidc/pingfederate/#questdb-configuration) | +| Confirm QuestDB mappings and login | [PingFederate](/docs/security/oidc/pingfederate/#confirm-questdb-mappings-and-login) | | Microsoft Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/) | +| Set up the client application in Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/#set-up-the-client-application-in-entra-id) | +| Map groups and grant permissions | [Microsoft Entra ID](/docs/security/oidc/entra-id/#map-groups-and-grant-permissions) | +| Confirm group mappings and login | [Microsoft Entra ID](/docs/security/oidc/entra-id/#confirm-group-mappings-and-login) | | Configuration options | [OIDC settings](/docs/configuration/oidc/) | + +## See also + +- [OIDC settings reference](/docs/configuration/oidc/) - configure QuestDB and + its Identity Provider connection +- [Role-based access control](/docs/security/rbac/) - grant endpoint and data + permissions after authentication +- [TLS](/docs/security/tls/) - protect credentials and tokens in transit diff --git a/documentation/security/oidc/pingfederate.mdx b/documentation/security/oidc/pingfederate.mdx index 21e192e061..5677c7f5e5 100644 --- a/documentation/security/oidc/pingfederate.mdx +++ b/documentation/security/oidc/pingfederate.mdx @@ -378,3 +378,11 @@ To test, head to `http://localhost:9000` and login. If all has been wired up well, then login will succeed. +## See also + +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + map PingFederate group names to QuestDB groups +- [OIDC settings reference](/docs/configuration/oidc/) - all `acl.oidc.*` + settings and endpoint-discovery behavior +- [How sign-in works](/docs/security/oidc/how-sign-in-works/) - the browser + authorization flow used by the Web Console diff --git a/documentation/security/rbac/authentication.mdx b/documentation/security/rbac/authentication.mdx index 9951af33ec..581728f337 100644 --- a/documentation/security/rbac/authentication.mdx +++ b/documentation/security/rbac/authentication.mdx @@ -35,6 +35,7 @@ The first three are QuestDB's own credentials, created with the statements below. An OIDC bearer token is issued by an external Identity Provider instead, and the user's group memberships come from the token or the provider's user info endpoint; see [OpenID Connect](/docs/security/oidc/) and +[client integration patterns](/docs/security/oidc/client-integration/), including [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). OIDC on the PostgreSQL Wire endpoint is off by default. Enable it with @@ -71,11 +72,11 @@ to protect credentials in transit. Before a user can connect, they need endpoint permissions: -| Permission | Allows access to | -| ---------- | -------------------------------------- | -| `HTTP` | REST API, Web Console, ILP over HTTP | -| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | -| `ILP` | InfluxDB Line Protocol TCP (port 9009) | +| Permission | Allows access to | +| ---------- | ----------------------------------------- | +| `HTTP` | REST API, Web Console, QWP, ILP over HTTP | +| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | +| `ILP` | InfluxDB Line Protocol TCP (port 9009) | ```questdb-sql -- Typical setup for an interactive user diff --git a/documentation/security/rbac/common-scenarios.mdx b/documentation/security/rbac/common-scenarios.mdx index b16fdfb7e4..3960a94ffc 100644 --- a/documentation/security/rbac/common-scenarios.mdx +++ b/documentation/security/rbac/common-scenarios.mdx @@ -7,8 +7,11 @@ description: and failover operators. --- -Each scenario below is a complete, runnable setup. Adapt the names and tables; -the shape of the grants is the part worth copying. +Each scenario below is a complete permission setup. The examples use the +`trades(timestamp, symbol, side, price, amount)` table from the +[Create your first database guide](/docs/getting-started/create-database/). +Adapt the principal and table names; the shape of the grants is the part worth +copying. ## Read-only analyst @@ -26,8 +29,8 @@ A service account for an application that ingests data into specific tables: ```questdb-sql CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'pwd'; -GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access -GRANT INSERT ON sensor_data TO ingest_app; -- Can only insert into sensor_data +GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access +GRANT INSERT ON trades TO ingest_app; -- Can only insert into trades ``` ## Team-based access with groups @@ -40,49 +43,47 @@ CREATE GROUP trading_team; -- Grant permissions to the group GRANT HTTP, PGWIRE TO trading_team; -GRANT SELECT ON trades, positions TO trading_team; +GRANT SELECT ON trades TO trading_team; GRANT INSERT ON trades TO trading_team; -- Add users to the group - they inherit all permissions -CREATE USER alice WITH PASSWORD 'pwd1'; -CREATE USER bob WITH PASSWORD 'pwd2'; -ADD USER alice TO trading_team; -ADD USER bob TO trading_team; +CREATE USER market_analyst WITH PASSWORD 'pwd1'; +CREATE USER risk_analyst WITH PASSWORD 'pwd2'; +ADD USER market_analyst TO trading_team; +ADD USER risk_analyst TO trading_team; ``` ## Column-level restrictions (hide sensitive data) -Allow access to a table but hide sensitive columns: +Allow access to selected fields while hiding columns your policy treats as +sensitive: ```questdb-sql CREATE USER auditor WITH PASSWORD 'pwd'; GRANT HTTP, PGWIRE TO auditor; --- Grant access to non-sensitive columns only -GRANT SELECT ON employees(id, name, department, hire_date) TO auditor; --- Columns salary and ssn are not granted = invisible to auditor +-- Grant access to approved columns only +GRANT SELECT ON trades(timestamp, symbol, price) TO auditor; +-- Columns side and amount are not granted = invisible to auditor ``` -## Row-level security (multi-tenant) +## Row-level security with views {#row-level-security-multi-tenant} Different users see different subsets of data: ```questdb-sql --- Base table has data for all regions -CREATE TABLE sales (ts TIMESTAMP, region SYMBOL, amount DOUBLE) TIMESTAMP(ts); - --- Create region-specific views -CREATE VIEW sales_emea AS (SELECT * FROM sales WHERE region = 'EMEA'); -CREATE VIEW sales_apac AS (SELECT * FROM sales WHERE region = 'APAC'); - --- Grant users access to their region only -CREATE USER emea_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO emea_manager; -GRANT SELECT ON sales_emea TO emea_manager; - -CREATE USER apac_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO apac_manager; -GRANT SELECT ON sales_apac TO apac_manager; +-- Create views over different subsets of the trades table +CREATE VIEW aapl_trades AS (SELECT * FROM trades WHERE symbol = 'AAPL'); +CREATE VIEW msft_trades AS (SELECT * FROM trades WHERE symbol = 'MSFT'); + +-- Grant users access to their subset only +CREATE USER aapl_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO aapl_analyst; +GRANT SELECT ON aapl_trades TO aapl_analyst; + +CREATE USER msft_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO msft_analyst; +GRANT SELECT ON msft_trades TO msft_analyst; ``` ## Database administrator diff --git a/documentation/security/rbac/granting-permissions.mdx b/documentation/security/rbac/granting-permissions.mdx index 56969da3e6..bf9b185f82 100644 --- a/documentation/security/rbac/granting-permissions.mdx +++ b/documentation/security/rbac/granting-permissions.mdx @@ -15,6 +15,9 @@ surprises people the first time they hit it. For the list of permissions themselves, see the [permissions reference](/docs/security/rbac/permissions-reference/). +The examples use the `trades(timestamp, symbol, side, price, amount)` table from +the [Create your first database guide](/docs/getting-started/create-database/). + ## Permission levels Permissions have different granularities determining where they can be applied: @@ -31,11 +34,11 @@ Examples: -- Database-level: applies to all tables GRANT SELECT ON ALL TABLES TO user; --- Table-level: applies to specific tables -GRANT SELECT ON trades, prices TO user; +-- Table-level: applies to a specific table +GRANT SELECT ON trades TO user; -- Column-level: applies to specific columns -GRANT SELECT ON trades(ts, symbol, price) TO user; +GRANT SELECT ON trades(timestamp, symbol, price) TO user; ``` ### Column-level access @@ -43,16 +46,16 @@ GRANT SELECT ON trades(ts, symbol, price) TO user; Restrict users to see only certain columns: ```questdb-sql --- User can only see timestamp and price, not quantity or trader_id -GRANT SELECT ON trades(ts, price) TO analyst; +-- User can only see timestamp and price, not symbol, side, or amount +GRANT SELECT ON trades(timestamp, price) TO analyst; ``` To grant on every column except a few, use the `*` wildcard with an `EXCLUDE` list: ```questdb-sql --- User can see all columns except trader_id -GRANT SELECT ON trades(* EXCLUDE (trader_id)) TO analyst; +-- User can see all columns except side +GRANT SELECT ON trades(* EXCLUDE (side)) TO analyst; ``` The wildcard covers the columns that exist when the statement runs, not columns @@ -87,11 +90,11 @@ needs `DROP VIEW` or `ALTER VIEW` on that view. database level that set includes `DATABASE ADMIN`: ```questdb-sql --- Makes alice a full database administrator -GRANT ALL TO alice; +-- Makes database_admin a full database administrator +GRANT ALL TO database_admin; -- Every permission on one table only -GRANT ALL ON trades TO alice; +GRANT ALL ON trades TO trade_manager; ``` :::warning @@ -136,9 +139,9 @@ The same applies from table to column level: ```questdb-sql GRANT SELECT ON trades TO user; -- Table level -REVOKE SELECT ON trades(ssn) FROM user; -- Revoke one column +REVOKE SELECT ON trades(side) FROM user; -- Revoke one column --- Result: user has column-level SELECT on all columns EXCEPT ssn +-- Result: user has column-level SELECT on all columns EXCEPT side -- Future columns will NOT be accessible ``` diff --git a/documentation/security/rbac/index.mdx b/documentation/security/rbac/index.mdx index 14992466e9..38b57d5fa6 100644 --- a/documentation/security/rbac/index.mdx +++ b/documentation/security/rbac/index.mdx @@ -21,6 +21,8 @@ at **database**, **table**, **column**, and even **row** level (using views). ## Quick start Here's a complete example to create a read-only analyst user in under a minute: +It assumes the `trades` table from the +[Create your first database guide](/docs/getting-started/create-database/). ```questdb-sql -- 1. Create the user @@ -30,9 +32,9 @@ CREATE USER analyst WITH PASSWORD 'secure_password_here'; GRANT PGWIRE, HTTP TO analyst; -- 3. Grant read access to specific tables -GRANT SELECT ON trades, prices TO analyst; +GRANT SELECT ON trades TO analyst; --- Done! The analyst can now connect and query trades and prices tables +-- Done! The analyst can now connect and query the trades table ``` To verify: @@ -53,18 +55,18 @@ Control _what data_ users can access: | ------------ | ------------------------------- | ----------------------------------------------------- | | **Database** | All tables, global operations | `GRANT SELECT ON ALL TABLES TO user` | | **Table** | Specific tables | `GRANT SELECT ON trades TO user` | -| **Column** | Specific columns within a table | `GRANT SELECT ON trades(ts, price) TO user` | +| **Column** | Specific columns within a table | `GRANT SELECT ON trades(timestamp, price) TO user` | | **Row** | Specific rows via views | Create a view with WHERE clause, grant access to view | ### Connection access granularity Control _how_ users can connect: -| Permission | Protocol | Use case | -| ---------- | ------------------------------- | ------------------------------------------ | -| `HTTP` | REST API, Web Console, ILP/HTTP | Interactive users, web applications | -| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | -| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | +| Permission | Protocol | Use case | +| ---------- | ------------------------------------ | ------------------------------------------ | +| `HTTP` | REST API, Web Console, QWP, ILP/HTTP | Interactive users, web applications | +| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | +| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | ```questdb-sql -- User can connect via PostgreSQL protocol only (not web console) @@ -117,35 +119,6 @@ via `ILP`. - [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) - [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) -## Permissions reference {#permissions} - -Every permission QuestDB supports, and the level each can be granted at, is -listed on its own page. See -[Permissions reference](/docs/security/rbac/permissions-reference/). - -## Authentication methods {#authentication} - -QuestDB accepts passwords, JWK tokens, REST API tokens, and OIDC bearer tokens. -See -[Authentication and endpoint access](/docs/security/rbac/authentication/#authentication). - -## Endpoint permissions {#endpoint-permissions} - -`HTTP`, `PGWIRE` and `ILP` decide which protocol a principal may connect over. -See -[Authentication and endpoint access](/docs/security/rbac/authentication/#endpoint-permissions). - -## User management {#user-management} - -Creating and removing principals, group membership, passwords and tokens, and -the `SHOW` statements that inspect them. See -[Users, service accounts, and groups](/docs/security/rbac/users-and-groups/#user-management). - -## Failover operator {#failover-operator} - -A service account that can move the primary role between nodes and nothing else. -See [Common scenarios](/docs/security/rbac/common-scenarios/#failover-operator). - ## Where each section moved {#moved} These anchors are kept so that existing links keep working. Prefer the pages @@ -162,16 +135,20 @@ above when adding new links. | Column-level restrictions | [Common scenarios](/docs/security/rbac/common-scenarios/#column-level-restrictions-hide-sensitive-data) | | Row-level security | [Common scenarios](/docs/security/rbac/common-scenarios/#row-level-security-multi-tenant) | | Database administrator | [Common scenarios](/docs/security/rbac/common-scenarios/#database-administrator) | +| Failover operator | [Common scenarios](/docs/security/rbac/common-scenarios/#failover-operator) | | Core concepts | [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) | | Users and service accounts | [Users and groups](/docs/security/rbac/users-and-groups/#users-and-service-accounts) | | Why service accounts? | [Users and groups](/docs/security/rbac/users-and-groups/#why-service-accounts) | | Groups | [Users and groups](/docs/security/rbac/users-and-groups/#groups) | +| Authentication methods | [Authentication](/docs/security/rbac/authentication/#authentication) | +| Endpoint permissions | [Authentication](/docs/security/rbac/authentication/#endpoint-permissions) | | Built-in admin | [Users and groups](/docs/security/rbac/users-and-groups/#built-in-admin) | | Service account assumption | [Users and groups](/docs/security/rbac/users-and-groups/#service-account-assumption) | | Creating and removing principals | [Users and groups](/docs/security/rbac/users-and-groups/#creating-and-removing-principals) | | Managing group membership | [Users and groups](/docs/security/rbac/users-and-groups/#managing-group-membership) | | Viewing information | [Users and groups](/docs/security/rbac/users-and-groups/#viewing-information) | | Managing authentication | [Authentication](/docs/security/rbac/authentication/#managing-authentication) | +| User management reference | [Users and groups](/docs/security/rbac/users-and-groups/#user-management) | | Permission levels | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-levels) | | Granting ALL | [Granting permissions](/docs/security/rbac/granting-permissions/#granting-all) | | The GRANT option | [Granting permissions](/docs/security/rbac/granting-permissions/#the-grant-option) | @@ -180,6 +157,7 @@ above when adding new links. | Permission re-adjustment | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-level-re-adjustment) | | Implicit timestamp permissions | [Granting permissions](/docs/security/rbac/granting-permissions/#implicit-permissions) | | Granting on non-existent objects | [Granting permissions](/docs/security/rbac/granting-permissions/#grant-verification) | +| Permissions reference | [Permissions reference](/docs/security/rbac/permissions-reference/) | | Database permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#database-permissions) | | User management permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#user-management-permissions) | | Special permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#special-permissions) | diff --git a/documentation/security/rbac/permissions-reference.mdx b/documentation/security/rbac/permissions-reference.mdx index e0254e5622..1c986b2a5d 100644 --- a/documentation/security/rbac/permissions-reference.mdx +++ b/documentation/security/rbac/permissions-reference.mdx @@ -77,11 +77,11 @@ These control which protocol a principal may connect on. They are covered in detail under [authentication and endpoint access](/docs/security/rbac/authentication/#endpoint-permissions). -| Permission | Level | Description | -| ---------- | -------- | -------------------------------------- | -| HTTP | Database | REST API, Web Console, ILP over HTTP | -| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | -| ILP | Database | InfluxDB Line Protocol TCP (port 9009) | +| Permission | Level | Description | +| ---------- | -------- | ----------------------------------------- | +| HTTP | Database | REST API, Web Console, QWP, ILP over HTTP | +| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | +| ILP | Database | InfluxDB Line Protocol TCP (port 9009) | ## User management permissions diff --git a/documentation/security/rbac/users-and-groups.mdx b/documentation/security/rbac/users-and-groups.mdx index af874b029d..3384bfa090 100644 --- a/documentation/security/rbac/users-and-groups.mdx +++ b/documentation/security/rbac/users-and-groups.mdx @@ -94,8 +94,8 @@ CREATE GROUP analysts; GRANT SELECT ON ALL TABLES TO analysts; -- All users added to this group can read all tables -ADD USER alice TO analysts; -ADD USER bob TO analysts; +ADD USER market_analyst TO analysts; +ADD USER risk_analyst TO analysts; ``` Users inherit permissions from their groups. Inherited permissions cannot be From ef66a91b703333ddcc743719569b596bd37c3d7c Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sat, 5 Sep 2026 00:03:38 +0100 Subject: [PATCH 51/54] docs: address OIDC and RBAC review findings --- documentation/changelog.mdx | 8 +- documentation/configuration/oidc.md | 74 ++++++--- documentation/connect/clients/c-and-cpp.md | 15 +- .../connect/clients/connect-string.md | 2 +- documentation/connect/clients/dotnet.md | 2 +- documentation/connect/clients/go.md | 2 +- documentation/connect/clients/java.md | 17 +- documentation/connect/clients/nodejs.md | 27 +++- documentation/connect/clients/python.md | 15 +- documentation/connect/clients/rust.md | 15 +- documentation/query/sql/acl/alter-group.md | 56 +++++++ documentation/query/sql/acl/create-group.md | 11 ++ .../query/sql/acl/create-service-account.md | 15 +- documentation/query/sql/acl/grant.md | 42 ++--- .../security/oidc/client-discovery.mdx | 24 ++- .../security/oidc/client-integration.mdx | 148 ++++++------------ .../device-flow.mdx} | 37 +++-- documentation/security/oidc/entra-id.mdx | 15 +- documentation/security/oidc/group-mapping.mdx | 70 +++++---- .../security/oidc/how-sign-in-works.mdx | 52 +++--- documentation/security/oidc/index.mdx | 2 +- .../security/rbac/authentication.mdx | 28 +++- .../security/rbac/common-scenarios.mdx | 16 +- documentation/security/rbac/index.mdx | 3 +- .../security/rbac/users-and-groups.mdx | 5 +- documentation/sidebars.js | 6 +- 26 files changed, 426 insertions(+), 281 deletions(-) create mode 100644 documentation/query/sql/acl/alter-group.md rename documentation/security/{oidc-device-flow.mdx => oidc/device-flow.mdx} (93%) diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 0f121fa510..f058c0ae0a 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -57,7 +57,7 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another -- [OIDC device flow](/docs/security/oidc-device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients +- [OIDC device flow](/docs/security/oidc/device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients - [OIDC settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` @@ -107,8 +107,10 @@ This page tracks significant updates to the QuestDB documentation. - Restructured [OpenID Connect](/docs/security/oidc/) and [role-based access control](/docs/security/rbac/) from two monolithic guides into task-focused sections covering client flows, provider setup, group mapping, authentication, principals, permissions, and common scenarios, while preserving links to the former section anchors - [OpenID Connect (OIDC)](/docs/security/oidc/how-sign-in-works/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers -- [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, and that sending the wrong one fails authentication -- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented what QuestDB validates when it checks a token itself: the signature, the expiry, the audience and the presence of the claims, but not `nbf` or `iss`. QuestDB never asks the provider about such a token, so a revoked token keeps working until it expires +- [OIDC client discovery](/docs/security/oidc/client-discovery/) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, that production applications need their own provider registration, and how official clients find `/settings` beneath a non-default Web Console context +- [OIDC client integration](/docs/security/oidc/client-integration/#non-interactive-clients) - Replaced password-based ROPC as the primary unattended-job example with QuestDB service-account tokens and provider-managed machine identities, and added the missing Node.js bearer-token handoff and rotation guidance +- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented local ID-token validation, including the Enterprise 4.0.1 version boundary, clock-skew and authentication-cache windows after `exp`, established-connection behavior, group-change propagation, and the requirement for a successful JWKS reload before a withdrawn key stops working +- [RBAC SQL references](/docs/security/rbac/#sql-commands-reference) - Added the missing `ALTER GROUP` reference and external-alias and service-account-password grammar, documented service-account authentication commands, and corrected grants after a table or column is dropped - Scoped three [OIDC options](/docs/configuration/oidc/) to what the server actually does: `acl.oidc.audience` is only checked when the groups are encoded in the token, the mandatory `openid` in `acl.oidc.scope` is enforced by the provider rather than at startup, and `acl.oidc.ropc.flow.enabled` makes QuestDB itself exchange HTTP basic and PGWire credentials for a token at the provider - [PingFederate SSO](/docs/security/oidc/pingfederate/#questdb-configuration) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - [Replication setup guide](/docs/high-availability/setup/#migration-procedures) - Planned primary migration now points at the in-place switch; the restart-based flow is kept for older versions and the emergency migration is marked as the lossy path diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index 2b7bb3ec4a..b53298287f 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -164,7 +164,7 @@ QuestDB uses the scopes in the requests it makes itself, in the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) for clients which run the flow themselves. -For the [OIDC device flow](/docs/security/oidc-device-flow/), add +For the [OIDC device flow](/docs/security/oidc/device-flow/), add `offline_access` alongside `openid`: ```ini title="server.conf" @@ -279,7 +279,7 @@ has no default, and QuestDB never calls it. QuestDB resolves the endpoint and publishes it on the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), for clients which implement the -[OIDC device flow](/docs/security/oidc-device-flow/) +[OIDC device flow](/docs/security/oidc/device-flow/) themselves. The official Java, Python, Rust, C, and C++ clients can discover and use the published endpoint. @@ -300,9 +300,10 @@ The keys are only used to validate tokens when With the default user info flow QuestDB validates tokens by calling the user info endpoint instead. -QuestDB downloads the keys from this endpoint at startup either way, so that the -cache is never empty. A failure to download them is logged, and does not stop -the server. +QuestDB attempts to download the keys from this endpoint at startup either way. +A failure is logged and does not stop the server. Until a download succeeds, +the cache is empty and ID-token validation fails because QuestDB cannot find the +signing key. ### acl.oidc.token.endpoint @@ -429,15 +430,28 @@ and a friendlier claim lets you keep the readable one as the principal. The Entra ID walkthrough does this, setting `acl.oidc.sub.claim=name` while the token still carries `sub` for the validator. +:::note Version requirement + +QuestDB Enterprise 4.0.1 and earlier do not validate `exp` in this mode. They +also require a claim literally named `groups` before applying the claim name +configured with `acl.oidc.groups.claim`. Upgrade to a later release before +relying on expiry validation or a differently named groups claim. + +::: + :::caution Because QuestDB never asks the provider about the token, revoking a token does -not end the access it grants: the token keeps working until its `exp` passes, -and [`acl.oidc.cache.ttl`](#acloidccachettl) does not shorten that. The only -lever is withdrawing the signing key at the provider, which cuts short every -token signed with it and takes effect within +not immediately end the access it grants. On a new authentication, QuestDB may +accept a token for up to 60 seconds after `exp` to allow for clock skew. A token +validated before or during that allowance may then remain in the authentication +cache for up to [`acl.oidc.cache.ttl`](#acloidccachettl). An established PGWire +or WebSocket connection is not closed when its token expires. + +Withdrawing a signing key can shorten this window, but only after QuestDB +successfully reloads the provider's key set. See [`acl.oidc.public.keys.expiry`](#acloidcpublickeysexpiry). Keep token lifetimes -short in the provider if you need revocation to take effect quickly. +and the authentication cache TTL short if revocation must take effect quickly. ::: @@ -462,9 +476,10 @@ fails. The same applies to the claim named by - **Default**: `30000` - **Reloadable**: no -User info cache entry TTL in milliseconds, as a plain integer only. QuestDB -caches user info responses for each valid access token. This setting controls -how often the access token is validated and user info refreshed. +OIDC authentication cache entry TTL in milliseconds, as a plain integer only. +QuestDB caches the principal and group-derived access list produced by a +successful authentication. This setting controls how often the token is checked +again and that access list is rebuilt. Set it to `0` to disable the cache, so that every request is checked again. In the default user info flow that means a call to the OIDC Provider on every @@ -473,13 +488,19 @@ request. When QuestDB checks the token locally instead, and contacts the provider only when the public keys have to be reloaded. -That local check tests the token's expiry, so an issued token is accepted until -it expires whatever this TTL is. Shortening the TTL does not make QuestDB -consult the provider, so a token revoked before its expiry is not picked up any -sooner. See +The local expiry check allows 60 seconds for clock skew. A token validated +before or during that allowance can remain accepted from the cache for up to +this TTL afterward. Setting the TTL to `0` removes that additional cache window, +but not the clock-skew allowance. It still does not make QuestDB consult the +provider, so provider-side token revocation, user disabling, and group changes +are not discovered from the same ID token. Those changes require the client to +obtain a new token. See [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) for what is and is not validated. +The TTL applies when a request or connection authenticates. It does not +continuously revalidate an established PGWire or WebSocket connection. + ### acl.oidc.public.keys.expiry - **Default**: `120000` @@ -489,12 +510,19 @@ Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. Also accepts a duration, such as `2m` or `120s`. QuestDB caches the public keys used to validate tokens issued by the OIDC -Provider, and reloads them from the public keys endpoint when the cache expires. - -Key rotation does not depend on this setting: a token signed with a key QuestDB -has not cached triggers an immediate reload. The expiry governs how long a key -the provider has already withdrawn stays usable, so lower it if signing keys are -revoked, at the cost of more requests to the endpoint. +Provider. After the cache expires, the next token validation attempts to reload +them from the public keys endpoint. + +A token signed with a key QuestDB has not cached triggers an immediate reload, +independently of this setting. QuestDB replaces the cached set only after it +downloads and parses a non-empty replacement successfully. If a reload fails, +the failure is logged and the previous keys remain available while later token +validations continue to retry. A key withdrawn by the provider can therefore +remain usable beyond this expiry during a provider or network failure. + +Lowering this setting reduces the normal delay before QuestDB notices a +withdrawn key after a successful reload, at the cost of more requests to the +endpoint. It does not provide a hard revocation deadline. Only used when [`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index 29f2456a27..f2d0e3914c 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -330,10 +330,13 @@ Handle it there, not at `connect` (see ### OIDC device flow (Enterprise) The C and C++ APIs sign in an interactive user with the -[OIDC device flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc/device-flow/). They discover the provider endpoints, client ID, scope, and the token QuestDB -expects from the server's public `/settings` endpoint. Attach the resulting auth -object to the pool so sender and reader connections share its rotating token: +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to the +builder; the client appends `/settings`. Attach the resulting auth object to the +pool so sender and reader connections share its rotating token: @@ -368,9 +371,9 @@ names are prefixed with `questdb_oidc_builder_`. Tokens stay in memory until `.default_file_token_store()` in C++, or `questdb_oidc_builder_default_file_token_store()` in C, writes a long-lived -refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and -permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and -[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). ### Other authentication limitations diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index 9e0d5babe6..7e82579591 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -223,7 +223,7 @@ WebSocket upgrade request. - Device-flow OIDC credentials are **not** connect-string keys. The Java, Python, Rust, C and C++ clients take an auth object alongside the connect string, which rotates the token on every reconnect; see - [OIDC device flow](/docs/security/oidc-device-flow/). A + [OIDC device flow](/docs/security/oidc/device-flow/). A static `token=` does not rotate, so a reconnect after expiry keeps sending the stale value. - `auth_timeout_ms` — per-host upper bound on the upgrade response read. diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index fcc4a71fb2..db8a0f2547 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 3964a03968..febe415a15 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -371,7 +371,7 @@ token, or the ID token when QuestDB reads group memberships from the token. This client does not run an OIDC flow, so acquire the token with an OAuth2 library: `golang.org/x/oauth2` implements the device grant, and QuestDB publishes the device authorization endpoint on the same settings response. The -[OIDC device flow](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) +[OIDC device flow](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) sets out the two requests, the polling interval, `slow_down`, and the device code's expiry. When the token expires or is rotated, construct a new handle with the new token. diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index debc149841..22ecbd3472 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -401,11 +401,14 @@ both the ingress and egress WebSocket upgrades. It is mutually exclusive with ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[OIDC device flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc/device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB -expects from the server's public `/settings` endpoint. The user can approve the -sign-in from a browser on any device, so this also works from a container or -remote notebook kernel: +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to +`fromQuestDB`; the client appends `/settings`. The user can approve the sign-in +from a browser on any device, so this also works from a container or remote +notebook kernel: @@ -435,9 +438,9 @@ own Client Id means pinning the endpoints rather than discovering them. Tokens stay in memory until a store is attached with `new OidcDeviceAuth.DiscoveryOptions().tokenStore(FileTokenStore.atDefaultLocation())`, -which writes a long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and -permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and -[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). +which writes a long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). ### HTTP basic auth diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index ed2775ab8d..5fcadc977a 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -87,9 +87,8 @@ const sender = Sender.fromEnv(); ... ``` -When using QuestDB Enterprise, authentication can also be done via REST token. -Please check the [RBAC docs](/docs/security/rbac/authentication/) for more -info. +When using QuestDB Enterprise, authentication can also be done via a +[REST token](/docs/security/rbac/authentication/) in the `token` setting. This client does not run an OIDC flow. QuestDB Enterprise also accepts an OIDC bearer token, which you acquire out of band from your Identity Provider: @@ -97,9 +96,29 @@ QuestDB publishes the provider's endpoints on its [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), and the same response tells you [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). See [OpenID Connect](/docs/security/oidc/), and -[implementing the flow yourself](/docs/security/oidc-device-flow/#implementing-the-flow-yourself) for the device grant's +[implementing the flow yourself](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) for the device grant's requests and polling rules. +Pass the selected bearer token to `Sender` with the same `token` setting. Use +`https::` so the credential is encrypted in transit: + +```javascript title="Use an OIDC bearer token" +const { Sender } = require("@questdb/nodejs-client") + +const token = process.env.QUESTDB_OIDC_TOKEN +if (!token) { + throw new Error("QUESTDB_OIDC_TOKEN is not set") +} + +const sender = Sender.fromConfig( + `https::addr=questdb.example.com:9000;token=${token};`, +) +``` + +The sender treats this as a static token: reconnecting does not acquire or read +a replacement. Before the token expires, finish and flush the current batch, +close the sender, acquire a new token, and construct a new sender with it. + ## Basic insert Example: inserting executed trades for cryptocurrencies. diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 3758b40b02..6f0ebcfb73 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -169,10 +169,13 @@ the full grammar. ### OIDC device flow (Enterprise) `OidcDeviceAuth` signs in an interactive user with the -[OIDC device flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc/device-flow/). It discovers the provider endpoints, client ID, scope, and the token QuestDB -expects from the server's public `/settings` endpoint. The default prompt works -in terminals and remote Jupyter kernels: +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to +`from_questdb`; the client appends `/settings`. The default prompt works in +terminals and remote Jupyter kernels: @@ -193,9 +196,9 @@ Otherwise pass `issuer=...`, or set `client_id=`, `scope=`, `audience=`, Tokens stay in memory until a store is passed as `token_store=FileTokenStore.at_default_location()`, which writes a long-lived -refresh token to disk as plaintext. See [token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and -permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and -[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). ### Other authentication limitations diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 3a1ee5f50f..44c163f9f9 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -168,7 +168,7 @@ choices have feature requirements: ### OIDC device flow (Enterprise) Enable the `oidc` feature to sign in an interactive user with the -[OIDC device flow](/docs/security/oidc-device-flow/): +[OIDC device flow](/docs/security/oidc/device-flow/): ```toml title="Cargo.toml" [dependencies] @@ -176,7 +176,10 @@ questdb-rs = { version = "7.1", features = ["oidc"] } ``` `OidcDeviceAuth::from_questdb` discovers the provider endpoints, client ID, -scope, and the token QuestDB expects from the public `/settings` endpoint: +scope, and the token QuestDB expects from the public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL; the client +appends `/settings`: @@ -210,9 +213,9 @@ let auth = OidcDeviceAuth::from_questdb(url).token_store(store).build()?; ``` `FileTokenStore::at(path)` takes a directory directly and needs no mapping. See -[token persistence](/docs/security/oidc-device-flow/#token-persistence) for the store's location and -permissions, [the sign-in prompt](/docs/security/oidc-device-flow/#the-sign-in-prompt), and -[explicit configuration](/docs/security/oidc-device-flow/#explicit-configuration-and-endpoint-pinning). +[token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). ## The pool @@ -974,7 +977,7 @@ integrations only when your application uses them: | `ndarray` | No | `Buffer::column_arr` from `ndarray` views. | | `rust_decimal` / `bigdecimal` | No | Row-buffer decimal values from those crates. Decimal strings need neither feature. | | `chrono-timestamp` | No | Timestamp values built from `chrono::DateTime`. | -| `oidc` | No | Interactive OIDC sign-in with the [device flow](/docs/security/oidc-device-flow/). Pulls in `sync-sender-http`, and needs a TLS root source for `https` discovery. | +| `oidc` | No | Interactive OIDC sign-in with the [device flow](/docs/security/oidc/device-flow/). Pulls in `sync-sender-http`, and needs a TLS root source for `https` discovery. | | `tls-native-certs` | No | TLS validation through the operating-system certificate store. | | `insecure-skip-verify` | No | `tls_verify=unsafe_off` for controlled testing only. | | `almost-all-features` | No | Client development and testing with most compatible features. It excludes Arrow and Polars. | diff --git a/documentation/query/sql/acl/alter-group.md b/documentation/query/sql/acl/alter-group.md new file mode 100644 index 0000000000..705020a18d --- /dev/null +++ b/documentation/query/sql/acl/alter-group.md @@ -0,0 +1,56 @@ +--- +title: ALTER GROUP reference +sidebar_label: ALTER GROUP +description: + "ALTER GROUP adds or removes external Identity Provider aliases from a + QuestDB group for OIDC authorization in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP` adds or removes an external Identity Provider mapping on an +existing QuestDB group. + +For an end-to-end OIDC example, see +[Mapping groups and permissions](/docs/security/oidc/group-mapping/). + +--- + +## Syntax + +```questdb-sql title="Add an external alias" +ALTER GROUP groupName WITH EXTERNAL ALIAS externalAlias; +``` + +```questdb-sql title="Remove an external alias" +ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias; +``` + +## Description + +`WITH EXTERNAL ALIAS` maps the group name or identifier in an OIDC groups claim +to `groupName`. An external user receives the QuestDB group's permissions when +their claim contains that exact alias. + +`DROP EXTERNAL ALIAS` removes the named mapping. It does not drop the QuestDB +group or change permissions granted to it. + +## Examples + +Map an Entra ID group identifier to the QuestDB group `analysts`: + +```questdb-sql +ALTER GROUP analysts +WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` + +Remove that mapping without dropping `analysts`: + +```questdb-sql +ALTER GROUP analysts +DROP EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` diff --git a/documentation/query/sql/acl/create-group.md b/documentation/query/sql/acl/create-group.md index a538cdc04e..2cdf302c6a 100644 --- a/documentation/query/sql/acl/create-group.md +++ b/documentation/query/sql/acl/create-group.md @@ -25,6 +25,10 @@ see the [RBAC operations](/docs/security/rbac) page. CREATE GROUP [IF NOT EXISTS] groupName; ``` +```questdb-sql title="Create an OIDC-mapped group" +CREATE GROUP groupName WITH EXTERNAL ALIAS externalAlias; +``` + ## Description `CREATE GROUP` adds a new user group with no permissions. @@ -37,12 +41,19 @@ the statement. Contrary to users and service accounts, it is not possible to log in as group. A group only serves as a container for permissions which are shared between users. +`WITH EXTERNAL ALIAS` maps a group name supplied by an OIDC Identity Provider +to the new QuestDB group. It cannot be combined with `IF NOT EXISTS`. Use +[`ALTER GROUP`](/docs/query/sql/acl/alter-group/) to add or remove mappings on +an existing group. + ## Examples ```questdb-sql CREATE GROUP admins; CREATE GROUP IF NOT EXISTS admins; + +CREATE GROUP analysts WITH EXTERNAL ALIAS 'identity-provider-analysts'; ``` It can be verified with: diff --git a/documentation/query/sql/acl/create-service-account.md b/documentation/query/sql/acl/create-service-account.md index 964119a859..33f1c7c517 100644 --- a/documentation/query/sql/acl/create-service-account.md +++ b/documentation/query/sql/acl/create-service-account.md @@ -23,13 +23,21 @@ see the [RBAC operations](/docs/security/rbac) page. ## Syntax ```questdb-sql -CREATE SERVICE ACCOUNT [IF NOT EXISTS] accountName [OWNED BY ownerName]; +CREATE SERVICE ACCOUNT [IF NOT EXISTS] accountName + [WITH { PASSWORD password | NO PASSWORD }] + [OWNED BY ownerName]; ``` ## Description `CREATE SERVICE ACCOUNT` adds a new service account with no permissions. +`WITH PASSWORD` enables password authentication at creation time. Omit the +clause, or use `WITH NO PASSWORD`, to create the account without a password. +Then use +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account/) to add a +password or token later. + The chosen name must be unique across all users (including the built-in admin), groups and service accounts. If the name has already been reserved, the command fails and an error is raised, unless the `IF NOT EXISTS` clause is included in @@ -53,12 +61,13 @@ an external user, because permissions cannot be granted to them. CREATE SERVICE ACCOUNT audit; CREATE SERVICE ACCOUNT IF NOT EXISTS audit; -``` +CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'change_me'; ``` + +```questdb-sql CREATE GROUP analysts; CREATE SERVICE ACCOUNT dashboard OWNED BY analysts; - ``` It can be verified with: diff --git a/documentation/query/sql/acl/grant.md b/documentation/query/sql/acl/grant.md index 11f44e237a..cccf77e1d4 100644 --- a/documentation/query/sql/acl/grant.md +++ b/documentation/query/sql/acl/grant.md @@ -326,49 +326,27 @@ ALTER TABLE countries ADD COLUMN description string; | UPDATE | countries | id | f | G | | UPDATE | countries | description | f | G | -### Grant when table or column is dropped and recreated +### Grants when a table or column is dropped and recreated {#grant-when-table-or-column-is-dropped-and-recreated} -Granted permissions are not automatically revoked when related tables or columns -are dropped. Instead, they have no effect until table or column is recreated. +Dropping a table permanently removes every permission granted on it. Recreating +a table with the same name does not restore those permissions: ```questdb-sql CREATE TABLE countries (id INT, name STRING, iso_code STRING); GRANT SELECT ON countries TO john; GRANT UPDATE ON countries(iso_code) TO john; -``` - -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | -| SELECT | countries | | f | G | -| UPDATE | countries | iso_code | f | G | - -Now, if the table is dropped, then permission stops being visible: -```questdb-sql DROP TABLE countries; -``` - -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | - -When the table is later recreated, permission are in full effect again : +CREATE TABLE countries (id INT, name STRING, iso_code STRING); -```questdb-sql -CREATE TABLE countries (id INT, name STRING, iso_code int, alpha2 STRING); +-- Required after recreation +GRANT SELECT ON countries TO john; +GRANT UPDATE ON countries(iso_code) TO john; ``` -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | -| SELECT | countries | | f | G | -| UPDATE | countries | | f | G | - -:::note - -Only the table and/or column name is used when applying permission. The type is -ignored. In the example above `iso_code` was initially of string type, then -recreated as int. - -::: +Dropping a column likewise removes permissions granted specifically on that +column. Adding a column with the same name later does not restore them. The same +drop-and-regrant rule applies to views, materialized views, and live views. ### Owner grants diff --git a/documentation/security/oidc/client-discovery.mdx b/documentation/security/oidc/client-discovery.mdx index 75b45ec542..31535414d9 100644 --- a/documentation/security/oidc/client-discovery.mdx +++ b/documentation/security/oidc/client-discovery.mdx @@ -50,6 +50,13 @@ Each key is the name of the its authorization request, and any other client can do the same instead of hard coding the provider's details. +The published `acl.oidc.client.id` and `acl.oidc.scope` belong to QuestDB's Web +Console registration. Reusing them keeps an evaluation client short, but a +production application should have its own provider registration, client ID, +redirect URIs, scopes, and policies. It can still discover the provider +endpoints and `acl.oidc.groups.encoded.in.token` here while overriding the +client ID and scope. + | Key | Type | Present | | --- | --- | --- | | `acl.enabled` | boolean | always | @@ -71,7 +78,7 @@ configuration document when `acl.oidc.configuration.url` is set. [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) is present when a device authorization endpoint has been configured, or when the provider advertises one. QuestDB publishes it without using it, for clients -which implement the [OIDC device flow](/docs/security/oidc-device-flow/) +which implement the [OIDC device flow](/docs/security/oidc/device-flow/) themselves. `acl.oidc.redirect.uri` is `null` unless @@ -107,6 +114,19 @@ is at its default, as the settings path follows the Web Console context path. A client which cannot assume both are at their defaults should make the path configurable. +The official device-flow clients append `/settings` to the QuestDB base URL +passed to `from_questdb`, `fromQuestDB`, or the corresponding C/C++ builder. If +`http.context.web.console=/console`, pass the context in that base URL: + +```text +https://questdb.example.com:9000/console +``` + +The clients then request `https://questdb.example.com:9000/console/settings`. +With a custom `http.context.settings`, QuestDB still preserves the path beneath +the Web Console context, so the same base URL works. A client which fetches the +settings document itself must use one of the exact configured paths. + ## Which token to send `acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned @@ -129,7 +149,7 @@ way, and so should any other client. - [Client integration patterns](/docs/security/oidc/client-integration/) - use discovery and bearer tokens from browser, notebook, CLI, and PGWire clients -- [OIDC device flow](/docs/security/oidc-device-flow/) - discovery in the +- [OIDC device flow](/docs/security/oidc/device-flow/) - discovery in the official Java, Python, Rust, C, and C++ clients - [OIDC settings reference](/docs/configuration/oidc/) - configure the values published to clients diff --git a/documentation/security/oidc/client-integration.mdx b/documentation/security/oidc/client-integration.mdx index fb6d15690f..d2ef171ac5 100644 --- a/documentation/security/oidc/client-integration.mdx +++ b/documentation/security/oidc/client-integration.mdx @@ -36,9 +36,13 @@ possible flows to request an access token.: [`acl.oidc.pkce.required`](/docs/configuration/oidc/#acloidcpkcerequired), which is `true` by default. -The client can read the authorization and token endpoints, the client id and -the scopes from the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) instead of hard -coding them. +The client can read the authorization and token endpoints and QuestDB's token +selection mode from the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) +instead of hard coding them. The client ID and scope published there belong to +the Web Console registration. Reuse them only for evaluation; register a +production application separately and use its own client ID, redirect URI, +scopes, and provider policies. The Web Console implements the [Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a @@ -53,7 +57,7 @@ QuestDB. Select the access or ID token as described in ### Jupyter notebook Use the official Python client's -[OIDC device flow](/docs/security/oidc-device-flow/). It works when the browser +[OIDC device flow](/docs/security/oidc/device-flow/). It works when the browser and the notebook kernel are on different machines, and does not put the user's password in the notebook. @@ -224,7 +228,7 @@ with pg.connect(conn_str, autocommit=True) as connection: For a CLI or standalone application built with an official Java, Python, Rust, C, or C++ client, use the -[OIDC device flow](/docs/security/oidc-device-flow/). +[OIDC device flow](/docs/security/oidc/device-flow/). Tools such as `psql` and Microsoft Access cannot run the flow themselves; for those tools, acquire a token separately and use [PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the @@ -245,111 +249,55 @@ testldap=> ## Non-interactive clients -Non-interactive clients are usually jobs or standalone applications, such as a -client for ingesting data. It is practical to manage their credentials via an -OAuth2 provider too. +An unattended job cannot complete device flow because that flow always requires +a person. Prefer a dedicated QuestDB service account with a REST token. This +keeps the job separate from human identities and gives it an explicit, minimal +permission set: -As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion. QuestDB publishes the -provider's token endpoint on the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so it -does not have to be hard coded, and the same response tells the client -[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). - -The client id and the scopes stay the client's own. `acl.oidc.client.id` and -`acl.oidc.scope` describe QuestDB's registration with the provider, not the -job's. As noted in [Architecture overview](/docs/security/oidc/#architecture-overview), each -application which integrates via OIDC should be given a different Client Id, so -that the two can be told apart in the provider's audit log and given different -policies. - -:::caution - -Both requests must go over `https`. The settings response names the URL this -code posts the user's password to, so anyone able to tamper with a plaintext -response can redirect the password grant to a host of their choosing. The same -applies to the connection below, which carries the resulting token. - -::: - -```python -import os -import requests -import pandas as pd -import questdb -from dotenv import load_dotenv - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -# this job's own registration in the OIDC Provider, not QuestDB's; -# the openid scope is what makes the provider issue an ID token -client_id = os.environ.get("client_id") -scope = "openid" - -settings = requests.get("https://questdb.example.com:9000/settings").json()["config"] -if not settings.get("acl.oidc.enabled"): - raise SystemExit("OIDC is not enabled on this QuestDB instance") - -# the endpoint keys are absent until QuestDB resolves them -token_endpoint = settings.get("acl.oidc.token.endpoint") -if not token_endpoint: - raise SystemExit("QuestDB has not resolved the provider's token endpoint") - -response = requests.post(token_endpoint, - data={"grant_type": "password", - "client_id": client_id, - "username": user, - "password": pwd, - "scope": scope}, - headers={"Content-Type": "application/x-www-form-urlencoded"}) -response.raise_for_status() -tokens = response.json() - -# QuestDB reads the group memberships from the ID token when this is enabled -token_key = "id_token" if settings["acl.oidc.groups.encoded.in.token"] else "access_token" -if token_key not in tokens: - raise SystemExit(f"the provider did not issue an {token_key} for the password grant") -token = tokens[token_key] - -conf = f"wss::addr=questdb.example.com:9000;token={token};" -with questdb.connect(conf) as db: - df = pd.read_csv("trades.csv") - df["timestamp"] = pd.to_datetime(df["timestamp"]) - db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") +```questdb-sql title="Create credentials for an ingestion job" +CREATE SERVICE ACCOUNT ingest_job; +GRANT HTTP TO ingest_job; +GRANT INSERT ON trades TO ingest_job; +ALTER SERVICE ACCOUNT ingest_job CREATE TOKEN TYPE REST WITH TTL '30d' REFRESH; ``` -:::note - -The Resource Owner Password Credentials flow returns an ID token only if the -provider issues one for the password grant, and only when `openid` is among the -requested scopes. Check that the provider does before relying on -[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) -with a non-interactive client. - -::: - -Alternatively, a user may rely on QuestDB to authenticate them via the OAuth2 -provider when the Resource Owner Password Credentials flow is enabled on the -server side: +The final statement displays the token once. Save it in a secrets manager and +pass it to the client over TLS, for example through a protected environment +variable: -```python +```python title="Use the service-account token" import os -import pandas as pd import questdb -from dotenv import load_dotenv -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") +token = os.environ["QUESTDB_TOKEN"] +conf = f"wss::addr=questdb.example.com:9000;token={token};" -conf = f"wss::addr=questdb.example.com:9000;username={user};password={pwd};" with questdb.connect(conf) as db: - df = pd.read_csv("trades.csv") - df["timestamp"] = pd.to_datetime(df["timestamp"]) - db.dataframe(df, table_name="trades", symbols=["symbol"], at="timestamp") + with db.query("SELECT current_user()") as result: + print(result.to_pandas()) ``` +The `REFRESH` modifier extends the REST token's expiry after successful +authentication to the primary. Review its replication behavior and revocation +commands in the +[ALTER SERVICE ACCOUNT reference](/docs/query/sql/acl/alter-service-account/#add-rest-api-token). + +If organizational policy requires provider-managed machine identities, give +the job its own confidential application or service principal and use a +machine-to-machine grant supported by that provider, commonly the OAuth 2.0 +client credentials grant. Do not use QuestDB's published Web Console client ID +or a human's username and password. The token must expose the subject and group +information required by QuestDB's configured +[validation mode](/docs/security/oidc/client-discovery/#which-token-to-send); +there is no provider-independent client-credentials setup in QuestDB. Many +providers do not issue ID tokens for client credentials, so verify this +provider-specific path before choosing encoded-token mode. + +ROPC authenticates a human by handing their password to the client. It is a +legacy fallback, not a machine-identity mechanism. Use it only for tooling that +cannot accept a separately acquired token, as described in +[Enable ROPC](#enable-ropc). + ## OIDC for the PGWire endpoint Clients such as `psql` that cannot run the device flow can still authenticate @@ -386,7 +334,7 @@ with pg.connect(conn_str, autocommit=True) as connection: - [Client discovery and tokens](/docs/security/oidc/client-discovery/) - find the provider endpoints and select the token QuestDB expects -- [OIDC device flow](/docs/security/oidc-device-flow/) - interactive sign-in +- [OIDC device flow](/docs/security/oidc/device-flow/) - interactive sign-in for CLI applications, containers, and remote notebooks - [Authentication and endpoint access](/docs/security/rbac/authentication/) - grant the endpoint permission each client protocol requires diff --git a/documentation/security/oidc-device-flow.mdx b/documentation/security/oidc/device-flow.mdx similarity index 93% rename from documentation/security/oidc-device-flow.mdx rename to documentation/security/oidc/device-flow.mdx index 4152105a73..67b0aa3d4f 100644 --- a/documentation/security/oidc-device-flow.mdx +++ b/documentation/security/oidc/device-flow.mdx @@ -5,11 +5,11 @@ description: "OIDC device flow for QuestDB Enterprise: headless SSO sign-in from import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; -import JavaExample from "../partials/_oidc.device-flow.java.partial.mdx"; -import PythonExample from "../partials/_oidc.device-flow.python.partial.mdx"; -import RustExample from "../partials/_oidc.device-flow.rust.partial.mdx"; -import CppExample from "../partials/_oidc.device-flow.cpp.partial.mdx"; -import CExample from "../partials/_oidc.device-flow.c.partial.mdx"; +import JavaExample from "../../partials/_oidc.device-flow.java.partial.mdx"; +import PythonExample from "../../partials/_oidc.device-flow.python.partial.mdx"; +import RustExample from "../../partials/_oidc.device-flow.rust.partial.mdx"; +import CppExample from "../../partials/_oidc.device-flow.cpp.partial.mdx"; +import CExample from "../../partials/_oidc.device-flow.c.partial.mdx"; import { EnterpriseNote } from "@site/src/components/EnterpriseNote" @@ -36,7 +36,7 @@ sequenceDiagram Client->>QDB: GET /settings QDB-->>Client: client ID, scope, endpoints, token mode else issuer pinned on the client - Client->>IdP: GET /.well-known/openid-configuration + Client->>IdP: GET {issuer path}/.well-known/openid-configuration IdP-->>Client: endpoints end Client->>IdP: Request device and user codes @@ -92,9 +92,9 @@ configuration-document discovery, do not set the endpoint property: all individual endpoint settings are ignored in that mode. If QuestDB does not publish the device authorization endpoint, pin the provider -with the client's `issuer` option so the client can fetch the provider's -`/.well-known/openid-configuration` document, or configure the client ID and -both endpoints explicitly. +with the client's `issuer` option so the client can fetch +`{issuer}/.well-known/openid-configuration`, including any path component in the +issuer, or configure the client ID and both endpoints explicitly. Clients reach QuestDB over `https` for discovery, so the HTTP endpoint needs TLS enabled with [`http.tls.enabled`](/docs/configuration/tls/#httptlsenabled) and a @@ -109,6 +109,12 @@ Each example discovers the OIDC client ID, scope, token-selection mode, and provider endpoints from QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), then signs in before opening the database connection. +The examples use QuestDB's default Web Console context. The clients append +`/settings` to the QuestDB URL, so include a non-default context in that URL. For +example, with `http.context.web.console=/console`, pass +`https://questdb.example.com:9000/console`. See the settings endpoint's +[path rules](/docs/security/oidc/client-discovery/#settings-endpoint). + Discovering the client ID means reusing QuestDB's own registration, which keeps the examples short but gives the application no separate identity at the provider. That is fine for evaluation; in production register the application @@ -180,8 +186,9 @@ than copying the current token into the connection string, otherwise a reconnect after expiry keeps sending the stale value. Device flow always requires a person to approve the sign-in. For unattended -services, scheduled jobs, or CI, use a service-account token or a provider flow -intended for machine identities instead. +services, scheduled jobs, or CI, use a +[QuestDB service-account token](/docs/security/oidc/client-integration/#non-interactive-clients) +or a provider flow intended for machine identities instead. ## The sign-in prompt @@ -276,7 +283,7 @@ the provider, or when QuestDB does not publish what the client needs: | `client_id` | the application has its own registration, which is the recommended setup | | `scope` | requesting `offline_access` for this application alone, rather than server-wide | | `audience` | the provider requires one, or QuestDB checks `aud` against [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | -| `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads the provider's `/.well-known/openid-configuration` instead | +| `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads `{issuer}/.well-known/openid-configuration` instead | | `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | | `groups_in_token` | overriding which of the two tokens the client selects | @@ -343,9 +350,9 @@ let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") An endpoint QuestDB advertises must sit under the pinned issuer's origin, or the client rejects it. An endpoint the client reads from the provider's own -`/.well-known/openid-configuration` is exempt, which is what makes the pin work -with providers that host their endpoints outside the issuer, such as Microsoft -Entra ID and Google. +`{issuer}/.well-known/openid-configuration` is exempt, which is what makes the +pin work with providers that host their endpoints outside the issuer, such as +Microsoft Entra ID and Google. ## Token persistence diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx index ea468d0f15..671b100510 100644 --- a/documentation/security/oidc/entra-id.mdx +++ b/documentation/security/oidc/entra-id.mdx @@ -237,10 +237,17 @@ This setup relies on [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), because Entra ID cannot serve the group memberships from its User Info endpoint. QuestDB then validates the ID token itself and never asks Entra ID -about it, so revoking a token in Entra ID does not end the access it grants: -the token keeps working until it expires. Read that option before going to -production, and note that clients must send the ID token here, not the access -token. +for current user information. Revoking a token, disabling a user, or changing +their Entra ID groups therefore does not immediately change access in QuestDB. +The client must obtain a new ID token for updated claims. + +On releases that validate `exp`, a token can still authenticate for the +60-second clock-skew allowance plus up to `acl.oidc.cache.ttl` while an earlier +validation remains cached. Existing PGWire and WebSocket connections are not +closed when the token expires. QuestDB Enterprise 4.0.1 and earlier do not +validate `exp` in this mode, so upgrade before relying on token expiry. Read the +configuration option before going to production, and note that clients must +send the ID token here, not the access token. ::: diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx index c74da39bb5..3dc3fc2578 100644 --- a/documentation/security/oidc/group-mapping.mdx +++ b/documentation/security/oidc/group-mapping.mdx @@ -45,8 +45,9 @@ those groups: ### Mapping user permissions -The mappings between external and QuestDB groups are managed with the following -SQL commands: +The mappings between external and QuestDB groups are managed with +[CREATE GROUP](/docs/query/sql/acl/create-group/) and +[ALTER GROUP](/docs/query/sql/acl/alter-group/): ```questdb-sql title="Create a group which is mapped to an Active Directory group" CREATE GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; @@ -60,6 +61,18 @@ ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=qu ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; ``` +The external alias establishes membership; it does not grant access by itself. +Grant endpoint and data permissions to the mapped QuestDB group, for example: + +```questdb-sql title="Grant Web Console and read access to the mapped group" +GRANT HTTP TO groupName; +GRANT SELECT ON trades TO groupName; +``` + +Every external user mapped to `groupName` receives those permissions. See +[Granting permissions](/docs/security/rbac/granting-permissions/) for endpoint, +table, column, and system-level examples. + QuestDB can obtain the list of external groups from either the User Info response or the ID token. By default, it reads them from the User Info response. When @@ -67,7 +80,7 @@ response. When is `true`, it reads them from the ID token instead; this mode supports providers such as Entra ID, which cannot return groups from the User Info endpoint. -In the User Info example used earlier, the response contains a claim called +In the User Info example below, the response contains a claim called `groups`. Its name is set with [`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which has no default and must be set whenever OIDC is enabled. @@ -93,36 +106,35 @@ login via the [Web Console](/docs/getting-started/web-console/overview/). } ``` -Any change made to the user's group membership in the Identity Provider, QuestDB -will adjust the user's access list. - -:::note - -There may be a slight delay due to the User Info Cache. - -QuestDB will use the cached information until it becomes stale, and gets -updated. - -::: - -The same stands for changes made to the user's status within the Identity -Provider. - -For example, a disabled user will not be kicked out of QuestDB immediately. - -The `acl.oidc.cache.ttl` config option drives how often user information should -be synchronized with the Identity Providers. - -It should be set accordingly to your organization's policies. - -Other parts of the user information, such as the `sub` and the `name` also used -by QuestDB. +How quickly an Identity Provider change takes effect depends on where QuestDB +reads the groups: + +- **User Info response**, the default: after the cached authentication entry + expires, the next authentication calls the User Info endpoint again and + rebuilds the access list. Configure that interval with + [`acl.oidc.cache.ttl`](/docs/configuration/oidc/#acloidccachettl). +- **ID token**, when `acl.oidc.groups.encoded.in.token=true`: QuestDB validates + the token locally and never asks the provider for current user information. + Cache expiry only makes QuestDB validate and read the same token again. Group + changes take effect only after the client obtains a new ID token. Disabling a + user or revoking the token at the provider is not discovered through the same + token; access ends only when a later authentication fails QuestDB's local + checks. + +In either mode, cache expiry does not terminate an established PGWire or +WebSocket connection. The change takes effect when a request or connection +authenticates with refreshed information. A disabled user or a revoked token +may therefore retain access temporarily; see the +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +security considerations for the encoded-token boundaries. -They could be displayed in the [Web Console](/docs/getting-started/web-console/overview/), or appear in -the logs, for example. +QuestDB also uses claims such as `sub` and `name` for the principal displayed in +the [Web Console](/docs/getting-started/web-console/overview/) and in audit logs. ## See also +- [Granting permissions](/docs/security/rbac/granting-permissions/) - grant + endpoint and data access to mapped QuestDB groups - [Permissions reference](/docs/security/rbac/permissions-reference/) - the permissions external groups can receive - [Microsoft Entra ID](/docs/security/oidc/entra-id/) - map group IDs carried diff --git a/documentation/security/oidc/how-sign-in-works.mdx b/documentation/security/oidc/how-sign-in-works.mdx index 6fb0b94822..fa01dc6bcd 100644 --- a/documentation/security/oidc/how-sign-in-works.mdx +++ b/documentation/security/oidc/how-sign-in-works.mdx @@ -99,13 +99,18 @@ This could be a username with: After successful authentication, the user provides consent for the requested scopes. -The list of scopes are configurable. +The list of scopes is configurable with +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). -By default the Web Console requests only the `openid` scope which is mandatory +By default the Web Console requests only the `openid` scope, which is mandatory for OIDC. No ID Token is issued without it. +A provider may require `offline_access`, explicit consent, or another policy +before issuing a refresh token. Requesting a refresh-token scope does not +guarantee that the provider returns one. + The OIDC provider can be configured to provide the consent automatically, without presenting the user with an additional screen in the browser. @@ -144,35 +149,40 @@ The matching code challenge proves that the token is requested by the client which requested the authorization code, and it was not stolen: ```bash title="Token request example" -POST https://oidc.provider:443/as/token.oauth2 HTTP/1.1 -Content-Type: application/x-www-form-urlencoded -grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF +curl --request POST https://oidc.provider:443/as/token.oauth2 \ + --header "Content-Type: application/x-www-form-urlencoded" \ + --data-urlencode "grant_type=authorization_code" \ + --data-urlencode "code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A" \ + --data-urlencode "client_id=questdb" \ + --data-urlencode "redirect_uri=https://questdb.example.com:9000" \ + --data-urlencode "code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF" ``` ### 6. Credentials received -If the PKCE check is passed, the Web Console receives the ID and access tokens. - -There is a third token in the response too, the refresh token. +If the PKCE check passes, the Web Console receives the ID and access tokens. The +provider may also return a refresh token when its scopes, consent, and policy +allow one. -The refresh token is used by the Web Console to refresh the access token before -it expires. +When present, the Web Console uses the refresh token to obtain replacement +tokens without asking the user to sign in again. Without one, the user must +authenticate again after the current tokens can no longer be used. -Without the refresh token mechanism, the user would be forced to re-authenticate -when the access token expires. +Token lifetimes are configurable in the OIDC Provider. -The validity of the tokens are configurable inside the OIDC Provider. - -```json title="Token response example" +```json title="Token response example with an optional refresh token" { "access_token": "gslpJtzmmi6RwaPSx0dYGD4tEkom", "refresh_token": "FUuAAqMp6LSTKmkUd5uZuodhiE4Kr6M7Eyv.eg83ge", "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...", "token_type": "Bearer", - "expires_in": 300 // In seconds, thus 5 minutes + "expires_in": 300 } ``` +`expires_in` is measured in seconds. The `refresh_token` field is absent when +the provider does not issue one. + ### 7. Database access With the tokens, the Web Console can interact with the database. @@ -201,8 +211,9 @@ already holds a valid entry for the token, and they do not happen at all when is `true`, in which case QuestDB validates the ID token itself and never calls the provider. See [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). -```bash title="Query request example" -https://questdb.example.com:9000/exec?query=select%20current_user() +```http title="Query request example" +GET /api/v1/sql/execute?query=select%20current_user() HTTP/1.1 +Host: questdb.example.com:9000 Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom ``` @@ -218,8 +229,9 @@ This call also serves as token validation. If the token is not real or has been expired, the User Info endpoint replies with an error: -```bash title="User info request example" -https://oidc.provider:443/idp/userinfo.openid +```http title="User info request example" +GET /idp/userinfo.openid HTTP/1.1 +Host: oidc.provider:443 Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom ``` diff --git a/documentation/security/oidc/index.mdx b/documentation/security/oidc/index.mdx index ebd7d01dfd..f456aba91a 100644 --- a/documentation/security/oidc/index.mdx +++ b/documentation/security/oidc/index.mdx @@ -87,7 +87,7 @@ are encoded in it. | --- | --- | | [How sign-in works](/docs/security/oidc/how-sign-in-works/) | The Authorization Code Flow with PKCE, step by step | | [Client discovery and tokens](/docs/security/oidc/client-discovery/) | The settings endpoint, and which of the two tokens to send | -| [Device flow](/docs/security/oidc-device-flow/) | Headless sign-in for a CLI, container, or notebook kernel | +| [Device flow](/docs/security/oidc/device-flow/) | Headless sign-in for a CLI, container, or notebook kernel | | [Client integration patterns](/docs/security/oidc/client-integration/) | Browser clients, Jupyter, CLI tools, unattended jobs, PGWire | | [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | Turning Identity Provider groups into QuestDB permissions | | [PingFederate](/docs/security/oidc/pingfederate/) | Provider setup walkthrough | diff --git a/documentation/security/rbac/authentication.mdx b/documentation/security/rbac/authentication.mdx index 581728f337..05ea46de82 100644 --- a/documentation/security/rbac/authentication.mdx +++ b/documentation/security/rbac/authentication.mdx @@ -26,9 +26,9 @@ QuestDB supports four authentication methods: | Method | Use case | Endpoints | | --------------------- | --------------------------- | -------------------------------------------------------------------------------- | -| **Password** | Interactive users | REST API, PostgreSQL Wire | +| **Password** | Interactive users | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire | | **JWK Token** | ILP ingestion | InfluxDB Line Protocol | -| **REST API Token** | Programmatic REST access | REST API | +| **REST API Token** | Programmatic HTTP access | REST API and QWP, both under the `HTTP` permission | | **OIDC bearer token** | SSO users, external clients | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire once enabled | The first three are QuestDB's own credentials, created with the statements @@ -92,7 +92,9 @@ able to ingest but never query. ## Managing authentication {#managing-authentication} -```questdb-sql +For a user: + +```questdb-sql title="Manage user authentication" -- Change password ALTER USER username WITH PASSWORD 'new_pwd'; @@ -110,8 +112,24 @@ ALTER USER username DROP TOKEN TYPE REST; -- Drops all REST tokens ALTER USER username DROP TOKEN TYPE REST 'token_value_here'; -- Drop specific token ``` -Removing all authentication methods (password and tokens) effectively disables -the user - they can no longer connect to the database. +Use the parallel `ALTER SERVICE ACCOUNT` forms for an application identity: + +```questdb-sql title="Manage service-account authentication" +ALTER SERVICE ACCOUNT service_account WITH PASSWORD 'new_pwd'; +ALTER SERVICE ACCOUNT service_account WITH NO PASSWORD; + +ALTER SERVICE ACCOUNT service_account CREATE TOKEN TYPE JWK; +ALTER SERVICE ACCOUNT service_account CREATE TOKEN TYPE REST WITH TTL '30d'; + +ALTER SERVICE ACCOUNT service_account DROP TOKEN TYPE JWK; +ALTER SERVICE ACCOUNT service_account DROP TOKEN TYPE REST; +``` + +See the [ALTER USER](/docs/query/sql/acl/alter-user/) and +[ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) references +for token refresh, expiry, and targeted REST-token removal. Removing every +authentication method effectively disables that user or service account: it can +no longer connect to the database. ## See also diff --git a/documentation/security/rbac/common-scenarios.mdx b/documentation/security/rbac/common-scenarios.mdx index 3960a94ffc..23add7aa02 100644 --- a/documentation/security/rbac/common-scenarios.mdx +++ b/documentation/security/rbac/common-scenarios.mdx @@ -73,17 +73,17 @@ Different users see different subsets of data: ```questdb-sql -- Create views over different subsets of the trades table -CREATE VIEW aapl_trades AS (SELECT * FROM trades WHERE symbol = 'AAPL'); -CREATE VIEW msft_trades AS (SELECT * FROM trades WHERE symbol = 'MSFT'); +CREATE VIEW eth_usd_trades AS (SELECT * FROM trades WHERE symbol = 'ETH-USD'); +CREATE VIEW btc_usd_trades AS (SELECT * FROM trades WHERE symbol = 'BTC-USD'); -- Grant users access to their subset only -CREATE USER aapl_analyst WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO aapl_analyst; -GRANT SELECT ON aapl_trades TO aapl_analyst; +CREATE USER eth_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO eth_analyst; +GRANT SELECT ON eth_usd_trades TO eth_analyst; -CREATE USER msft_analyst WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO msft_analyst; -GRANT SELECT ON msft_trades TO msft_analyst; +CREATE USER btc_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO btc_analyst; +GRANT SELECT ON btc_usd_trades TO btc_analyst; ``` ## Database administrator diff --git a/documentation/security/rbac/index.mdx b/documentation/security/rbac/index.mdx index 38b57d5fa6..17cbe0aa9e 100644 --- a/documentation/security/rbac/index.mdx +++ b/documentation/security/rbac/index.mdx @@ -97,8 +97,9 @@ via `ILP`. ## SQL commands reference - [ADD USER](/docs/query/sql/acl/add-user/) -- [ALTER USER](/docs/query/sql/acl/alter-user/) +- [ALTER GROUP](/docs/query/sql/acl/alter-group/) - [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) +- [ALTER USER](/docs/query/sql/acl/alter-user/) - [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) - [CREATE GROUP](/docs/query/sql/acl/create-group/) - [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) diff --git a/documentation/security/rbac/users-and-groups.mdx b/documentation/security/rbac/users-and-groups.mdx index 3384bfa090..2dcbf5b5e8 100644 --- a/documentation/security/rbac/users-and-groups.mdx +++ b/documentation/security/rbac/users-and-groups.mdx @@ -21,7 +21,7 @@ three kinds, how they differ, and the statements that manage them. ## Users and service accounts -QuestDB has two types of principals: +QuestDB has two account types: - **Users**: For human individuals. Can belong to multiple groups and inherit permissions from them. Cannot be assumed by others. @@ -157,7 +157,8 @@ ADD USER username TO group1, group2; REMOVE USER username FROM group1; ``` -Passwords and tokens are managed with `ALTER USER`; see +Passwords and tokens are managed with `ALTER USER` or +`ALTER SERVICE ACCOUNT`; see [managing authentication](/docs/security/rbac/authentication/#managing-authentication). ### Viewing information diff --git a/documentation/sidebars.js b/documentation/sidebars.js index 1677cddbd4..3eacde8f87 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -287,6 +287,10 @@ module.exports = { type: "category", label: "ALTER", items: [ + { + id: "query/sql/acl/alter-group", + type: "doc", + }, { id: "query/sql/acl/alter-service-account", type: "doc", @@ -771,7 +775,7 @@ module.exports = { label: "Client discovery and tokens", }, { - id: "security/oidc-device-flow", + id: "security/oidc/device-flow", type: "doc", label: "Device flow", }, From 23530b36e9027c6473fd88a6148600a5903c6fd9 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 6 Sep 2026 11:58:43 +0100 Subject: [PATCH 52/54] docs: document device flow in containers --- documentation/connect/clients/python.md | 5 +++ documentation/security/oidc/device-flow.mdx | 48 +++++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 6f0ebcfb73..6277e2e4c2 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -179,6 +179,11 @@ terminals and remote Jupyter kernels: +Device flow also works in Docker and Kubernetes without an inbound callback +port. Disable browser launching, expose stderr to the operator, and persist the +token store on a protected volume; see +[Docker and Kubernetes](/docs/security/oidc/device-flow/#docker-and-kubernetes). + `oidc_auth=auth` keeps shared ownership of the provider and obtains the cached or silently refreshed token for every connection and reconnect. It is mutually exclusive with a fixed `token=` setting. Silent refresh needs a refresh token, diff --git a/documentation/security/oidc/device-flow.mdx b/documentation/security/oidc/device-flow.mdx index 67b0aa3d4f..dc96a83892 100644 --- a/documentation/security/oidc/device-flow.mdx +++ b/documentation/security/oidc/device-flow.mdx @@ -272,6 +272,54 @@ clickable, use the separately vetted browser target instead: `browser_target` on the C event struct, `event_view::browser_target()` in C++, `browser_target()` on the Rust challenge, and the `browser_target` key on the Python prompt event. +## Docker and Kubernetes + +Device flow works from a Docker container or Kubernetes pod because the browser +can run on the host or another device. The container displays a verification URL +and user code while making outbound requests to QuestDB and the Identity +Provider. It does not need an inbound callback port. + +For example, a Python container can disable local browser launching and persist +tokens in a mounted directory: + +```python title="Python device flow in a container" +from questdb.auth import FileTokenStore, OidcDeviceAuth + +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + open_browser=False, + token_store=FileTokenStore("/var/lib/questdb-client/tokens"), +) +auth.sign_in() +``` + +When running device flow in a container: + +- Allow outbound HTTPS access to QuestDB and the Identity Provider. +- Keep stderr visible through attached output or container logs so the user can + read the verification URL and code. +- Set `open_browser=False`. Launching a browser is best-effort, but is not useful + inside most containers. +- Mount the token-store directory as a persistent, access-restricted volume if + credentials should survive container or pod restarts. The file store contains + an unencrypted refresh token; see [token persistence](#token-persistence). +- Call `sign_in()` before creating senders, pools, or database connections. + +The clients do not require a TTY. A detached container can start device flow and +will wait until somebody reads its logs and approves the code, or until the +device code expires. + +:::note Unattended workloads + +Device flow always needs a person to approve the sign-in. For production +services, scheduled jobs, and CI, use a +[QuestDB service-account token](/docs/security/oidc/client-integration/#non-interactive-clients) +or an OAuth client-credentials flow. Set `interactive=False` when constructing +the Python authentication object so missing credentials fail immediately rather +than waiting for approval. + +::: + ## Explicit configuration and endpoint pinning Discovery supplies the client ID, scope, token-selection mode, and endpoints. From 3165576128ac7b04edc1b27442fd51b1705f9ed9 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 6 Sep 2026 12:04:18 +0100 Subject: [PATCH 53/54] docs: add device flow example prerequisites --- documentation/security/oidc/device-flow.mdx | 35 +++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/documentation/security/oidc/device-flow.mdx b/documentation/security/oidc/device-flow.mdx index dc96a83892..5c4ff52f71 100644 --- a/documentation/security/oidc/device-flow.mdx +++ b/documentation/security/oidc/device-flow.mdx @@ -103,6 +103,41 @@ walkthroughs for [Microsoft Entra ID](/docs/security/oidc/entra-id/) and [PingFederate](/docs/security/oidc/pingfederate/). +## Prepare permissions and the example schema + +Every client example below writes one row to `trades` over QWP/WebSocket. Before +running an example, create the table with the schema from the +[Create a sample database guide](/docs/getting-started/create-database/#creating-a-table): + +```questdb-sql title="Create the example table" +CREATE TABLE trades ( + timestamp TIMESTAMP, + symbol SYMBOL, + side SYMBOL, + price DOUBLE, + amount DOUBLE +) TIMESTAMP(timestamp) PARTITION BY DAY +DEDUP UPSERT KEYS(timestamp, symbol); +``` + +Next, run the following as the built-in admin. Replace the external alias with +the exact group name or identifier that the Identity Provider places in the +user's groups claim: + +```questdb-sql title="Map the external group and grant example permissions" +CREATE GROUP oidc_ingest +WITH EXTERNAL ALIAS 'identity-provider-ingest-group'; + +GRANT HTTP TO oidc_ingest; +GRANT INSERT ON trades TO oidc_ingest; +``` + +The `HTTP` permission allows the QWP/WebSocket connection, while `INSERT` allows +the client to write to the existing table. The external alias gives those +permissions to users whose groups claim contains that value. See +[Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions) +for provider-specific group mapping details. + ## Official client examples Each example discovers the OIDC client ID, scope, token-selection mode, and From b7f2e62369a97c8fc8c3aa1da81419716d0862a9 Mon Sep 17 00:00:00 2001 From: glasstiger Date: Sun, 6 Sep 2026 12:54:13 +0100 Subject: [PATCH 54/54] docs: address OIDC and RBAC review feedback --- documentation/configuration/oidc.md | 28 +++++++++--------- documentation/connect/clients/c-and-cpp.md | 6 ++-- documentation/connect/clients/java.md | 6 ++-- documentation/connect/clients/python.md | 6 ++-- documentation/connect/clients/rust.md | 6 ++-- .../partials/_oidc.device-flow.c.partial.mdx | 29 ++++++++++++++----- documentation/query/sql/acl/alter-group.md | 8 +++++ documentation/query/sql/acl/create-group.md | 15 ++++++++-- .../query/sql/acl/create-service-account.md | 11 +++++++ .../security/oidc/client-discovery.mdx | 7 +++++ documentation/security/oidc/device-flow.mdx | 28 ++++++++++++------ documentation/security/oidc/group-mapping.mdx | 5 ++++ documentation/security/oidc/index.mdx | 3 +- .../security/rbac/authentication.mdx | 2 +- .../security/rbac/common-scenarios.mdx | 26 +++++++++++++---- .../security/rbac/granting-permissions.mdx | 12 ++++---- documentation/security/rbac/index.mdx | 8 ++--- .../security/rbac/permissions-reference.mdx | 12 ++++---- 18 files changed, 152 insertions(+), 66 deletions(-) diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index b53298287f..e1ba849229 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -153,31 +153,31 @@ location it was loaded from, without the query string or fragment - **Default**: `openid` - **Reloadable**: no -The OIDC server asks consent for the scopes listed in this property. The -scope `openid` is mandatory and must always be included. That is an OIDC -protocol requirement enforced by the provider, not a QuestDB startup check: -QuestDB passes the value on without inspecting it, so leaving `openid` out -fails at the provider rather than at startup. +The OIDC server asks consent for the scopes listed in this property. Keep +`openid` in the value for QuestDB's OIDC flows. It requests OIDC authentication +semantics and an ID token from the provider. QuestDB passes the value on without +inspecting it. Without `openid`, a provider may process the request as OAuth2 +and issue only an access token, or reject it according to provider policy. QuestDB uses the scopes in the requests it makes itself, in the [ROPC flow](#acloidcropcflowenabled), and publishes them on the [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) for clients which run the flow themselves. -For the [OIDC device flow](/docs/security/oidc/device-flow/), add -`offline_access` alongside `openid`: +For the [OIDC device flow](/docs/security/oidc/device-flow/), request the +provider's refresh-token scope, commonly `offline_access`, alongside `openid`: ```ini title="server.conf" acl.oidc.scope=openid offline_access ``` -Most providers, Microsoft Entra ID among them, issue a refresh token only when -that scope was requested, and without one the client has to make the user sign -in again as soon as the current token expires. -Providers which issue refresh tokens regardless ignore the extra scope, so -adding it is the safe default. A client can also request it through its own -`scope` override, which leaves this server-wide value, and every other flow -using it, untouched. +Requesting the scope does not guarantee a refresh token. The client +registration, user or administrator consent, and provider policy determine +whether one is issued. Some providers use a different scope, issue refresh +tokens without one, or reject an unsupported scope. Without a refresh token, +the client must ask the user to sign in again after the current token expires. +A client can request the scope through its own `scope` override, which leaves +this server-wide value, and every other flow using it, untouched. ## Authentication flows diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index f2d0e3914c..0e822078d0 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -352,8 +352,10 @@ pool so sender and reader connections share its rotating token: The pool retains the auth state and gets a cached or silently refreshed token -for every connection and reconnect. Silent refresh needs a refresh token, which -the provider issues only when `offline_access` was requested; see +for every connection and reconnect. Silent refresh needs a refresh token. +Request the provider's refresh-token scope, commonly `offline_access`; the +client registration, consent, and provider policy still determine whether one +is issued. See [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Those transport operations never prompt; they fail with `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` in C, or diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index 22ecbd3472..430bd6d27c 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -414,8 +414,10 @@ notebook kernel: Pass `auth::getToken` as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. Silent refresh needs a refresh token, which the -provider issues only when `offline_access` was requested; see +silently refreshed token. Silent refresh needs a refresh token. Request the +provider's refresh-token scope, commonly `offline_access`; the client +registration, consent, and provider policy still determine whether one is +issued. See [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call never starts an interactive flow; it throws `OidcAuthException`, which carries no distinct interaction-required type, so call `signIn()` again on the main or diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index 6277e2e4c2..4884e6ea9f 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -186,8 +186,10 @@ token store on a protected volume; see `oidc_auth=auth` keeps shared ownership of the provider and obtains the cached or silently refreshed token for every connection and reconnect. It is mutually -exclusive with a fixed `token=` setting. Silent refresh needs a refresh token, -which the provider issues only when `offline_access` was requested; see +exclusive with a fixed `token=` setting. Silent refresh needs a refresh token. +Request the provider's refresh-token scope, commonly `offline_access`; the +client registration, consent, and provider policy still determine whether one +is issued. See [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Transport operations never prompt; if they raise `OidcInteractionRequired`, importable from `questdb.auth`, call `auth.sign_in()` explicitly on the main or UI thread. diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 44c163f9f9..1d74913c93 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -185,8 +185,10 @@ appends `/settings`: Pass the closure as a provider instead of putting the current token in the connect string. Every new connection and reconnect then receives the cached or -silently refreshed token. Silent refresh needs a refresh token, which the -provider issues only when `offline_access` was requested; see +silently refreshed token. Silent refresh needs a refresh token. Request the +provider's refresh-token scope, commonly `offline_access`; the client +registration, consent, and provider policy still determine whether one is +issued. See [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call never starts an interactive flow; it returns a `questdb::Error` with `ErrorCode::AuthError`, whose `err.oidc_error()` reports diff --git a/documentation/partials/_oidc.device-flow.c.partial.mdx b/documentation/partials/_oidc.device-flow.c.partial.mdx index 057f39cddd..fac8a12018 100644 --- a/documentation/partials/_oidc.device-flow.c.partial.mdx +++ b/documentation/partials/_oidc.device-flow.c.partial.mdx @@ -1,6 +1,7 @@ ```c #include -#include // questdb_db_borrow_sender + line_sender_buffer +/* questdb_db_borrow_sender and line_sender_buffer */ +#include #include #include #include @@ -45,15 +46,27 @@ int main(void) { if (!sender) goto done; buffer = questdb_db_new_buffer(db, &error); if (!buffer) goto done; - if (!line_sender_buffer_table(buffer, QDB_TABLE_NAME_LITERAL("trades"), &error)) goto done; + if (!line_sender_buffer_table( + buffer, QDB_TABLE_NAME_LITERAL("trades"), &error)) + goto done; if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("symbol"), - QDB_UTF8_LITERAL("ETH-USDT"), &error)) goto done; + QDB_UTF8_LITERAL("ETH-USDT"), &error)) + goto done; if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("side"), - QDB_UTF8_LITERAL("sell"), &error)) goto done; - if (!line_sender_buffer_column_f64(buffer, QDB_COLUMN_NAME_LITERAL("price"), 2615.54, &error)) goto done; - if (!line_sender_buffer_column_f64(buffer, QDB_COLUMN_NAME_LITERAL("amount"), 0.00044, &error)) goto done; - if (!line_sender_buffer_at_nanos(buffer, line_sender_now_nanos(), &error)) goto done; - if (!qwp_sender_flush_buffer_and_wait(sender, buffer, qwpws_ack_level_ok, &error)) goto done; + QDB_UTF8_LITERAL("sell"), &error)) + goto done; + if (!line_sender_buffer_column_f64( + buffer, QDB_COLUMN_NAME_LITERAL("price"), 2615.54, &error)) + goto done; + if (!line_sender_buffer_column_f64( + buffer, QDB_COLUMN_NAME_LITERAL("amount"), 0.00044, &error)) + goto done; + if (!line_sender_buffer_at_nanos( + buffer, line_sender_now_nanos(), &error)) + goto done; + if (!qwp_sender_flush_buffer_and_wait( + sender, buffer, qwpws_ack_level_ok, &error)) + goto done; status = 0; done: diff --git a/documentation/query/sql/acl/alter-group.md b/documentation/query/sql/acl/alter-group.md index 705020a18d..95ad2ce882 100644 --- a/documentation/query/sql/acl/alter-group.md +++ b/documentation/query/sql/acl/alter-group.md @@ -36,9 +36,17 @@ ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias; to `groupName`. An external user receives the QuestDB group's permissions when their claim contains that exact alias. +External aliases are globally unique. If the alias is already reserved, adding +it fails instead of replacing the existing mapping. + `DROP EXTERNAL ALIAS` removes the named mapping. It does not drop the QuestDB group or change permissions granted to it. +## Permissions + +- `WITH EXTERNAL ALIAS` requires `ADD EXTERNAL ALIAS`. +- `DROP EXTERNAL ALIAS` requires `REMOVE EXTERNAL ALIAS`. + ## Examples Map an Entra ID group identifier to the QuestDB group `analysts`: diff --git a/documentation/query/sql/acl/create-group.md b/documentation/query/sql/acl/create-group.md index 2cdf302c6a..97537d2588 100644 --- a/documentation/query/sql/acl/create-group.md +++ b/documentation/query/sql/acl/create-group.md @@ -46,6 +46,14 @@ to the new QuestDB group. It cannot be combined with `IF NOT EXISTS`. Use [`ALTER GROUP`](/docs/query/sql/acl/alter-group/) to add or remove mappings on an existing group. +External aliases are globally unique. If the alias is already reserved, +`CREATE GROUP` fails instead of reusing the mapping. + +## Permissions + +Creating a group requires the `CREATE GROUP` permission. The `WITH EXTERNAL +ALIAS` form also requires `ADD EXTERNAL ALIAS`. + ## Examples ```questdb-sql @@ -64,6 +72,7 @@ SHOW GROUPS; that yields: -| name | -| ------ | -| admins | +| name | +| -------- | +| admins | +| analysts | diff --git a/documentation/query/sql/acl/create-service-account.md b/documentation/query/sql/acl/create-service-account.md index 33f1c7c517..a0d7c1fef3 100644 --- a/documentation/query/sql/acl/create-service-account.md +++ b/documentation/query/sql/acl/create-service-account.md @@ -55,6 +55,17 @@ group specified in the clause. The `OWNED BY` clause cannot be omitted if the service account is created by an external user, because permissions cannot be granted to them. +## Permissions + +Creating a service account without a password clause requires `CREATE SERVICE +ACCOUNT`. The explicit password forms need an additional permission: + +- `WITH PASSWORD` requires `ADD PASSWORD`. +- `WITH NO PASSWORD` requires `REMOVE PASSWORD`. + +Omitting the clause and writing `WITH NO PASSWORD` create the same account +state, but they do not have the same permission requirements. + ## Examples ```questdb-sql diff --git a/documentation/security/oidc/client-discovery.mdx b/documentation/security/oidc/client-discovery.mdx index 31535414d9..bc2a74b77c 100644 --- a/documentation/security/oidc/client-discovery.mdx +++ b/documentation/security/oidc/client-discovery.mdx @@ -4,6 +4,13 @@ sidebar_label: Client discovery and tokens description: How a QuestDB client reads the OIDC settings endpoint to discover the provider's endpoints, client ID and scope, and which of the two tokens to send. --- +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + OIDC client discovery and bearer-token authentication are available in + QuestDB Enterprise. + + A client needs two things from QuestDB before it can authenticate a user against your Identity Provider: where the provider's endpoints are, and which of the two tokens the provider issues QuestDB expects in the diff --git a/documentation/security/oidc/device-flow.mdx b/documentation/security/oidc/device-flow.mdx index 5c4ff52f71..b258948c2e 100644 --- a/documentation/security/oidc/device-flow.mdx +++ b/documentation/security/oidc/device-flow.mdx @@ -76,9 +76,12 @@ Before using the flow: a provider document containing `device_authorization_endpoint`, or the host-based configuration must set [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). -3. Keep `openid` in [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope), - and add `offline_access` so the provider issues a refresh token. Without one - the client must make the user sign in again at every token expiry. +3. Keep `openid` in [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) + to request OIDC authentication semantics. Also request the provider's + refresh-token scope, commonly `offline_access`. The client registration, + consent, and provider policy determine whether a refresh token is issued. + Without one, the client must ask the user to sign in again after the current + token expires. For example, a host-based configuration can add: @@ -364,7 +367,7 @@ the provider, or when QuestDB does not publish what the client needs: | Setting | Set it when | | --- | --- | | `client_id` | the application has its own registration, which is the recommended setup | -| `scope` | requesting `offline_access` for this application alone, rather than server-wide | +| `scope` | requesting the provider's refresh-token scope for this application alone, rather than server-wide | | `audience` | the provider requires one, or QuestDB checks `aud` against [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | | `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads `{issuer}/.well-known/openid-configuration` instead | | `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | @@ -431,11 +434,12 @@ let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") -An endpoint QuestDB advertises must sit under the pinned issuer's origin, or the -client rejects it. An endpoint the client reads from the provider's own -`{issuer}/.well-known/openid-configuration` is exempt, which is what makes the -pin work with providers that host their endpoints outside the issuer, such as -Microsoft Entra ID and Google. +An endpoint QuestDB advertises must share the pinned issuer's origin. When the +issuer contains a path, the endpoint must also remain beneath that path. The +client rejects an advertised endpoint which fails either check. An endpoint the +client reads from the provider's own `{issuer}/.well-known/openid-configuration` +is exempt, which is what makes the pin work with providers that host their +endpoints outside the issuer, such as Microsoft Entra ID and Google. ## Token persistence @@ -526,6 +530,9 @@ Catching it and signing in again: ```python +from questdb.auth import OidcInteractionRequired + + try: sender.flush(wait=True) except OidcInteractionRequired: @@ -592,6 +599,9 @@ decide what happens next: | `access_denied` | the user refused; stop | | `expired_token` | `expires_in` elapsed; start a new device authorization request | +Treat every other OAuth error as terminal: stop polling and return the failure +to the caller. + Polling faster than `interval`, or ignoring `slow_down`, gets the client rate limited or blocked by the provider. diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx index 3dc3fc2578..deb9a493f2 100644 --- a/documentation/security/oidc/group-mapping.mdx +++ b/documentation/security/oidc/group-mapping.mdx @@ -5,6 +5,11 @@ description: Map Identity Provider group memberships onto QuestDB groups with EX --- import Screenshot from "@theme/Screenshot"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote"; + + + OIDC group mapping and authorization are available in QuestDB Enterprise. + QuestDB does not store permissions for an OIDC user. It reads the user's group memberships out of the Identity Provider and maps them onto QuestDB groups, diff --git a/documentation/security/oidc/index.mdx b/documentation/security/oidc/index.mdx index f456aba91a..9245f08873 100644 --- a/documentation/security/oidc/index.mdx +++ b/documentation/security/oidc/index.mdx @@ -136,8 +136,9 @@ above when adding new links. | Enable Resource Owner Password Credentials flow | [PingFederate](/docs/security/oidc/pingfederate/#enable-resource-owner-password-credentials-ropc-flow) | | QuestDB configuration for PingFederate | [PingFederate](/docs/security/oidc/pingfederate/#questdb-configuration) | | Confirm QuestDB mappings and login | [PingFederate](/docs/security/oidc/pingfederate/#confirm-questdb-mappings-and-login) | -| Microsoft Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/) | +| Microsoft Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/) | | Set up the client application in Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/#set-up-the-client-application-in-entra-id) | +| QuestDB configuration | [Microsoft Entra ID](/docs/security/oidc/entra-id/#questdb-configuration) | | Map groups and grant permissions | [Microsoft Entra ID](/docs/security/oidc/entra-id/#map-groups-and-grant-permissions) | | Confirm group mappings and login | [Microsoft Entra ID](/docs/security/oidc/entra-id/#confirm-group-mappings-and-login) | | Configuration options | [OIDC settings](/docs/configuration/oidc/) | diff --git a/documentation/security/rbac/authentication.mdx b/documentation/security/rbac/authentication.mdx index 05ea46de82..3ae5f1e784 100644 --- a/documentation/security/rbac/authentication.mdx +++ b/documentation/security/rbac/authentication.mdx @@ -104,7 +104,7 @@ ALTER USER username WITH NO PASSWORD; -- Create tokens ALTER USER username CREATE TOKEN TYPE JWK; ALTER USER username CREATE TOKEN TYPE REST WITH TTL '30d'; -ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Auto-refresh +ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Sliding expiry extension -- Remove tokens ALTER USER username DROP TOKEN TYPE JWK; diff --git a/documentation/security/rbac/common-scenarios.mdx b/documentation/security/rbac/common-scenarios.mdx index 23add7aa02..55f5714f12 100644 --- a/documentation/security/rbac/common-scenarios.mdx +++ b/documentation/security/rbac/common-scenarios.mdx @@ -27,12 +27,22 @@ GRANT SELECT ON ALL TABLES TO analyst; A service account for an application that ingests data into specific tables: -```questdb-sql -CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'pwd'; +```questdb-sql title="Create an ILP/TCP service account" +CREATE SERVICE ACCOUNT ingest_app; +ALTER SERVICE ACCOUNT ingest_app CREATE TOKEN TYPE JWK; GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access GRANT INSERT ON trades TO ingest_app; -- Can only insert into trades ``` +:::warning + +Save the private key returned by `ALTER SERVICE ACCOUNT` when the token is +created. QuestDB does not store the private key, so it cannot be recovered +later. Configure the ILP/TCP client with the `ingest_app` account name and that +private key. See [RBAC authentication](/docs/security/rbac/authentication/). + +::: + ## Team-based access with groups Multiple users sharing the same permissions: @@ -73,17 +83,21 @@ Different users see different subsets of data: ```questdb-sql -- Create views over different subsets of the trades table -CREATE VIEW eth_usd_trades AS (SELECT * FROM trades WHERE symbol = 'ETH-USD'); -CREATE VIEW btc_usd_trades AS (SELECT * FROM trades WHERE symbol = 'BTC-USD'); +CREATE VIEW eth_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'ETH-USDT' +); +CREATE VIEW btc_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'BTC-USDT' +); -- Grant users access to their subset only CREATE USER eth_analyst WITH PASSWORD 'pwd'; GRANT HTTP, PGWIRE TO eth_analyst; -GRANT SELECT ON eth_usd_trades TO eth_analyst; +GRANT SELECT ON eth_usdt_trades TO eth_analyst; CREATE USER btc_analyst WITH PASSWORD 'pwd'; GRANT HTTP, PGWIRE TO btc_analyst; -GRANT SELECT ON btc_usd_trades TO btc_analyst; +GRANT SELECT ON btc_usdt_trades TO btc_analyst; ``` ## Database administrator diff --git a/documentation/security/rbac/granting-permissions.mdx b/documentation/security/rbac/granting-permissions.mdx index bf9b185f82..bbc2dfa47e 100644 --- a/documentation/security/rbac/granting-permissions.mdx +++ b/documentation/security/rbac/granting-permissions.mdx @@ -68,18 +68,18 @@ For row-level security, create a [view](/docs/concepts/views/) that filters rows, then grant access to the view instead of the underlying table: ```questdb-sql --- Create a view that only shows AAPL trades -CREATE VIEW aapl_trades AS ( - SELECT * FROM trades WHERE symbol = 'AAPL' +-- Create a view that only shows ETH-USDT trades +CREATE VIEW eth_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'ETH-USDT' ); -- Grant access to the view, not the base table -GRANT SELECT ON aapl_trades TO aapl_analyst; +GRANT SELECT ON eth_usdt_trades TO eth_analyst; -- No GRANT on trades table = user cannot see other symbols ``` -The user `aapl_analyst` can only see AAPL trades. They have no access to the -underlying `trades` table. +The user `eth_analyst` can only see `ETH-USDT` trades. They have no access to +the underlying `trades` table. Creating a view needs the `CREATE VIEW` permission; dropping or redefining one needs `DROP VIEW` or `ALTER VIEW` on that view. diff --git a/documentation/security/rbac/index.mdx b/documentation/security/rbac/index.mdx index 17cbe0aa9e..1617559a65 100644 --- a/documentation/security/rbac/index.mdx +++ b/documentation/security/rbac/index.mdx @@ -29,7 +29,7 @@ It assumes the `trades` table from the CREATE USER analyst WITH PASSWORD 'secure_password_here'; -- 2. Grant endpoint access (required to connect) -GRANT PGWIRE, HTTP TO analyst; +GRANT HTTP, PGWIRE TO analyst; -- 3. Grant read access to specific tables GRANT SELECT ON trades TO analyst; @@ -113,12 +113,12 @@ via `ILP`. - [REMOVE USER](/docs/query/sql/acl/remove-user/) - [REVOKE](/docs/query/sql/acl/revoke/) - [REVOKE ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/revoke-assume-service-account/) -- [SHOW USER](/docs/query/sql/show/#show-user) -- [SHOW USERS](/docs/query/sql/show/#show-users) - [SHOW GROUPS](/docs/query/sql/show/#show-groups) +- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) - [SHOW SERVICE ACCOUNT](/docs/query/sql/show/#show-service-account) - [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) -- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) +- [SHOW USER](/docs/query/sql/show/#show-user) +- [SHOW USERS](/docs/query/sql/show/#show-users) ## Where each section moved {#moved} diff --git a/documentation/security/rbac/permissions-reference.mdx b/documentation/security/rbac/permissions-reference.mdx index 1c986b2a5d..4f2b9f10f1 100644 --- a/documentation/security/rbac/permissions-reference.mdx +++ b/documentation/security/rbac/permissions-reference.mdx @@ -31,21 +31,21 @@ SELECT * FROM all_permissions(); | COMPILE VIEW | Database | Table | Recompile a view against the current schema | | CONVERT PARTITION TO NATIVE | Database | Table | Convert Parquet partitions back to native format | | CONVERT PARTITION TO PARQUET | Database | Table | Convert partitions to Parquet in place | -| CREATE TABLE | Database | Create tables | +| CREATE LIVE VIEW | Database | Create live views | | CREATE MATERIALIZED VIEW | Database | Create materialized views | +| CREATE TABLE | Database | Create tables | | CREATE VIEW | Database | Create views | -| CREATE LIVE VIEW | Database | Create live views | -| DEDUP ENABLE | Database | Table | Enable deduplication | | DEDUP DISABLE | Database | Table | Disable deduplication | +| DEDUP ENABLE | Database | Table | Enable deduplication | | DETACH PARTITION | Database | Table | Detach partitions | | DISABLE STORAGE POLICY | Database | Table | Disable storage policies | | DROP COLUMN | Database | Table | Column | Drop columns | | DROP INDEX | Database | Table | Column | Drop indexes | +| DROP LIVE VIEW | Database | Table | Drop live views | +| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | | DROP PARTITION | Database | Table | Drop partitions | | DROP TABLE | Database | Table | Drop tables | -| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | | DROP VIEW | Database | Table | Drop views | -| DROP LIVE VIEW | Database | Table | Drop live views | | ENABLE STORAGE POLICY | Database | Table | Enable storage policies | | INSERT | Database | Table | Insert data | | REFRESH MATERIALIZED VIEW | Database | Table | Refresh materialized views | @@ -80,8 +80,8 @@ detail under | Permission | Level | Description | | ---------- | -------- | ----------------------------------------- | | HTTP | Database | REST API, Web Console, QWP, ILP over HTTP | -| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | | ILP | Database | InfluxDB Line Protocol TCP (port 9009) | +| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | ## User management permissions