diff --git a/documentation/changelog.mdx b/documentation/changelog.mdx index 57a15c468c..f058c0ae0a 100644 --- a/documentation/changelog.mdx +++ b/documentation/changelog.mdx @@ -57,13 +57,15 @@ This page tracks significant updates to the QuestDB documentation. - [ALTER TABLE SET FORMAT](/docs/query/sql/alter-table-set-format/) - New reference page for switching a table's partition storage format between `NATIVE` and `PARQUET` - [QWP configuration](/docs/configuration/qwp/) - Server-side settings for the QWP ingestion (`/write/v4`) and query (`/read/v1`) endpoints - [Check timestamp order](/docs/cookbook/sql/time-series/check-timestamp-order/) and [Check column sort order](/docs/cookbook/sql/advanced/check-column-sort-order/) - Two cookbook recipes that detect unsorted data with `lag()`: whether a table, CSV import or external Parquet file is ordered by its timestamp, and whether one column is sorted with respect to another +- [OIDC device flow](/docs/security/oidc/device-flow/) - Interactive OIDC sign-in for headless and remote applications, including configuration, token lifecycle, a protocol walkthrough, and examples for the Java, Python, Rust, C, and C++ clients +- [OIDC settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) - How a client reads QuestDB's OIDC configuration from `/settings` to discover the authorization, token, and device authorization endpoints, and whether PKCE and the `state` parameter are required - [Kubernetes Operator](/docs/enterprise-kubernetes-operator/) - New manual for running QuestDB Enterprise clusters on Kubernetes, covering [installation](/docs/enterprise-kubernetes-operator/installation/), getting started on [AWS](/docs/enterprise-kubernetes-operator/getting-started/aws/) and [Azure](/docs/enterprise-kubernetes-operator/getting-started/azure/), [configuration](/docs/enterprise-kubernetes-operator/configuration/), [day-to-day operations](/docs/enterprise-kubernetes-operator/operations/operator/), [high availability](/docs/enterprise-kubernetes-operator/high-availability/), [known limitations](/docs/enterprise-kubernetes-operator/known-limitations/), and the full [API reference](/docs/enterprise-kubernetes-operator/reference/api/) - [ALTER TABLE SUSPEND WAL](/docs/query/sql/alter-table-suspend-wal/) - New reference page for deliberately stopping the WAL apply job, covering the quiescent-table use case that `REBASE WAL` requires, and the trap that writes to a suspended table succeed while staying invisible to queries until `RESUME WAL` ### Reference - Added the [`node_role()`](/docs/query/functions/meta/#node_role) function, which reports the replication role of the instance to any authenticated session -- [RBAC](/docs/security/rbac/#failover-operator) - Added the `SWITCH ROLE` permission to the permissions table, a failover operator scenario, and the built-in admin's break-glass role in a replicated cluster +- [RBAC](/docs/security/rbac/common-scenarios/#failover-operator) - Added the `SWITCH ROLE` permission to the permissions table, a failover operator scenario, and the built-in admin's break-glass role in a replicated cluster - [Minimal HTTP server](/docs/operations/logging-metrics/#lifecycle-endpoint) - Documented the `GET /lifecycle` component snapshot and the [`http.health.check.authentication.required`](/docs/configuration/http-min-server/#httphealthcheckauthenticationrequired) setting - [Replication metrics](/docs/operations/logging-metrics/#replication-metrics) - Added `questdb_replication_pending_upload_txn` and `questdb_backup_active_at_last_demote` - [Replication configuration](/docs/configuration/database-replication/#replicationrole) - Documented that `replication.role` is the boot role, the `primary-catchup-uploads` value, and the restart hazard after an in-place switch @@ -88,6 +90,7 @@ This page tracks significant updates to the QuestDB documentation. - Added the [`cairo.sql.parquet.cache.memory.size`](/docs/configuration/cairo-engine/) configuration property (256 MB default), deprecating the slot-based `cairo.sql.parquet.frame.cache.capacity` - Documented [`cairo.root`](/docs/configuration/cairo-engine/) absolute-path behavior: the `conf`, `import`, `export`, `tmp`, and `.checkpoint` directories become siblings of the specified directory rather than children of the server root, so leave it at the default under Docker - [read_parquet](/docs/query/functions/parquet/#designated-timestamp) - Documented nominating a designated timestamp on a Parquet file with `TIMESTAMP()` (applied directly, on a sub-query, or on a CTE), and importing a file into a table with `INSERT INTO ... SELECT` or `CREATE TABLE AS` +- Documented the OIDC configuration options that were missing from the [OIDC configuration page](/docs/configuration/oidc/): `acl.oidc.state.required`, `acl.oidc.public.keys.expiry`, `acl.oidc.response.buffer.size`, `acl.oidc.string.pool.capacity`, and `acl.oidc.device.authorization.endpoint`; also corrected `acl.oidc.pkce.enabled` to its real name `acl.oidc.pkce.required`, and documented `acl.oidc.groups.claim` as mandatory with no default - [SAMPLE BY](/docs/query/sql/sample-by/#sample-units) - Documented that the count is optional and defaults to `1`, so `SAMPLE BY h` is the same as `SAMPLE BY 1h`, and that it must be a positive integer written with no space before the unit - [REST API](/docs/connect/compatibility/rest-api/#execute) - Documented `/execute` and `/validate`, which compiles a query and returns its column metadata without running it. `/exec` is now marked deprecated in favour of `/execute` - [HTTP server configuration](/docs/configuration/http-server/#circuitbreakerbuffersize) - Flagged `circuit.breaker.buffer.size` as unsupported since QuestDB 10.0.0; a config that still sets it logs an advisory, or fails to start under `config.validation.strict` @@ -102,6 +105,14 @@ This page tracks significant updates to the QuestDB documentation. ### Updated +- Restructured [OpenID Connect](/docs/security/oidc/) and [role-based access control](/docs/security/rbac/) from two monolithic guides into task-focused sections covering client flows, provider setup, group mapping, authentication, principals, permissions, and common scenarios, while preserving links to the former section anchors +- [OpenID Connect (OIDC)](/docs/security/oidc/how-sign-in-works/#1-secret-generation) - Noted that some Identity Providers require the `state` parameter in the authorization request, and that `acl.oidc.state.required` should be set to `true` for those providers +- [OIDC client discovery](/docs/security/oidc/client-discovery/) - Documented that `acl.oidc.groups.encoded.in.token` decides whether a client sends the access token or the ID token, that production applications need their own provider registration, and how official clients find `/settings` beneath a non-default Web Console context +- [OIDC client integration](/docs/security/oidc/client-integration/#non-interactive-clients) - Replaced password-based ROPC as the primary unattended-job example with QuestDB service-account tokens and provider-managed machine identities, and added the missing Node.js bearer-token handoff and rotation guidance +- [`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) - Documented local ID-token validation, including the Enterprise 4.0.1 version boundary, clock-skew and authentication-cache windows after `exp`, established-connection behavior, group-change propagation, and the requirement for a successful JWKS reload before a withdrawn key stops working +- [RBAC SQL references](/docs/security/rbac/#sql-commands-reference) - Added the missing `ALTER GROUP` reference and external-alias and service-account-password grammar, documented service-account authentication commands, and corrected grants after a table or column is dropped +- Scoped three [OIDC options](/docs/configuration/oidc/) to what the server actually does: `acl.oidc.audience` is only checked when the groups are encoded in the token, the mandatory `openid` in `acl.oidc.scope` is enforced by the provider rather than at startup, and `acl.oidc.ropc.flow.enabled` makes QuestDB itself exchange HTTP basic and PGWire credentials for a token at the provider +- [PingFederate SSO](/docs/security/oidc/pingfederate/#questdb-configuration) - Added the missing QuestDB `server.conf` block to the walkthrough, which relied on defaults for settings that have none - [Replication setup guide](/docs/high-availability/setup/#migration-procedures) - Planned primary migration now points at the in-place switch; the restart-based flow is kept for older versions and the emergency migration is marked as the lossy path - [Error codes](/docs/troubleshooting/error-codes/#er005) - ER005 now covers the refusal of an in-place promotion, and ER006 the restart of a demoted node with a stale `replication.role` - Client libraries rewritten for the QWP binary protocol, unifying ingestion and streaming SQL queries under one handle: [Java](/docs/connect/clients/java/), [Python](/docs/connect/clients/python/), [Go](/docs/connect/clients/go/), [C & C++](/docs/connect/clients/c-and-cpp/), [Rust](/docs/connect/clients/rust/), and [.NET](/docs/connect/clients/dotnet/) diff --git a/documentation/configuration/http-server.md b/documentation/configuration/http-server.md index cb8971ee49..01baf22037 100644 --- a/documentation/configuration/http-server.md +++ b/documentation/configuration/http-server.md @@ -515,11 +515,25 @@ Context path for the file import service. ### http.context.settings -- **Default**: `/settings` +- **Default**: `/settings`, relative to + [`http.context.web.console`](#httpcontextwebconsole) - **Reloadable**: no -Context path for the service which provides server-side settings to the Web -Console. +Context path for the service that serves server-side settings to clients, such +as an OIDC client discovering the provider's endpoints from the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). + +Accepts a comma-separated list of paths. Setting it adds paths rather than +moving the service: QuestDB keeps serving the default path as well, so the +[Web Console](/docs/getting-started/web-console/overview/), which does not read +this setting, keeps working. + +The default path follows +[`http.context.web.console`](#httpcontextwebconsole), so it is `/settings` only +while that setting is at its default. Setting +`http.context.web.console=/console` makes the preserved path +`/console/settings`, and bare `/settings` is then served only if it is listed +here explicitly. ### http.context.table.status diff --git a/documentation/configuration/iam.md b/documentation/configuration/iam.md index f3ecdec571..de26f9151e 100644 --- a/documentation/configuration/iam.md +++ b/documentation/configuration/iam.md @@ -47,6 +47,10 @@ Enables or disables the built-in admin user. When enabled, the browser's basic auth popup window is used instead of the Web Console's login screen. Present for backwards compatibility only. +Cannot be enabled together with +[`acl.oidc.enabled`](/docs/configuration/oidc/#acloidcenabled). Setting both +to `true` fails server startup. + ### acl.enabled - **Default**: `true` diff --git a/documentation/configuration/oidc.md b/documentation/configuration/oidc.md index e0c5932bdc..e1ba849229 100644 --- a/documentation/configuration/oidc.md +++ b/documentation/configuration/oidc.md @@ -1,6 +1,6 @@ --- -title: OpenID Connect (OIDC) -description: Configuration settings for OpenID Connect integration in QuestDB Enterprise. +title: OIDC settings +description: "QuestDB Enterprise acl.oidc.* settings reference: minimum configuration and startup rules, endpoints, TLS, user and group claims, caching and buffers." --- :::note @@ -14,7 +14,49 @@ Management. The database can be integrated with any OAuth2/OIDC Identity Provider (IdP). For detailed information about OIDC, see the -[OpenID Connect (OIDC) integration guide](/docs/security/oidc). +[OpenID Connect (OIDC) integration guide](/docs/security/oidc/). For guidance on +choosing a flow for an application, see +[OIDC client integration patterns](/docs/security/oidc/client-integration/). + +## Minimum configuration + +OIDC requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be `true`, +which is the default. With access control disabled no OIDC authentication takes +place, but the OIDC settings are still validated at startup: with +`acl.oidc.enabled=true` the server enforces every rule below and downloads the +provider's configuration document, so an inconsistent OIDC configuration still +prevents it from starting. Set `acl.oidc.enabled=false` to stop the rules below being +enforced. `acl.oidc.configuration.url` is still parsed even then, and its +scheme must still match [`acl.oidc.tls.enabled`](#acloidctlsenabled), so a +malformed value there fails startup with OIDC switched off. + +A working setup against a Ping Identity provider needs four settings. Every +other setting has a usable default: + +```ini title="server.conf" +acl.oidc.enabled=true +acl.oidc.host=oidc.provider +acl.oidc.client.id=questdb +acl.oidc.groups.claim=groups +``` + +QuestDB refuses to start when the OIDC configuration is inconsistent. With +`acl.oidc.enabled=true`: + +- [`acl.oidc.client.id`](#acloidcclientid) and + [`acl.oidc.groups.claim`](#acloidcgroupsclaim) must be set. +- Exactly one of [`acl.oidc.host`](#acloidchost) and + [`acl.oidc.configuration.url`](#acloidcconfigurationurl) must be set. +- [`acl.basic.auth.realm.enabled`](/docs/configuration/iam/#aclbasicauthrealmenabled) + must be `false`. +- [`acl.oidc.tls.keystore.path`](#acloidctlskeystorepath) and + [`acl.oidc.tls.keystore.password`](#acloidctlskeystorepassword) must both be + set, or neither. +- [`acl.oidc.tls.enabled`](#acloidctlsenabled) must match the scheme of every + OIDC Provider URL. +- When [`acl.oidc.configuration.url`](#acloidcconfigurationurl) is set, the + document must be downloadable and parseable, and must name the authorization, + token, user info and JWKS endpoints. ## General @@ -26,6 +68,12 @@ For detailed information about OIDC, see the OAuth2 audience as set on the tokens issued by the OIDC Provider. Defaults to the client ID if not set. +Only used when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, +which is the only case in which QuestDB validates tokens itself. In the default +user info flow the OIDC Provider decides whether the token is valid, and +QuestDB does not check the audience at all. + ### acl.oidc.client.id - **Default**: none @@ -42,35 +90,53 @@ enabled. URL where the OpenID Provider's configuration information can be loaded in JSON format. Should always end with `/.well-known/openid-configuration`. +QuestDB downloads the document at startup and takes every endpoint from it, +so the settings under [Endpoints](#endpoints) and `acl.oidc.port` are not +used. The server does not start if the document cannot be downloaded or +parsed, or if it is missing the authorization, token, user info or JWKS +endpoint. + +Mutually exclusive with `acl.oidc.host`: setting both fails server startup. + ### acl.oidc.enabled - **Default**: `false` - **Reloadable**: no -Enables or disables OIDC authentication. When enabled, several other -configuration options must also be set. +Enables or disables OIDC authentication. When enabled, `acl.oidc.client.id` +and `acl.oidc.groups.claim` must also be set, along with either +`acl.oidc.host` or `acl.oidc.configuration.url`. See +[Minimum configuration](#minimum-configuration) for the full set of startup +requirements. + +OIDC cannot be enabled together with +[`acl.basic.auth.realm.enabled`](/docs/configuration/iam/#aclbasicauthrealmenabled). +Setting both to `true` fails server startup. ### acl.oidc.host - **Default**: none - **Reloadable**: no -OIDC provider hostname. Required when OIDC is enabled, unless the OIDC -configuration URL is set. +OIDC provider hostname. Required when OIDC is enabled, unless +`acl.oidc.configuration.url` is set. The two are mutually exclusive: setting +both fails server startup. ### acl.oidc.http.timeout - **Default**: `30000` - **Reloadable**: no -OIDC provider HTTP request timeout in milliseconds. +OIDC provider HTTP request timeout in milliseconds. Accepts a plain integer +only. ### acl.oidc.port - **Default**: `443` - **Reloadable**: no -OIDC provider port number. +OIDC provider port number. Not used when `acl.oidc.configuration.url` is +set, because the port is taken from the discovered endpoint URLs. ### acl.oidc.redirect.uri @@ -79,34 +145,71 @@ OIDC provider port number. The redirect URI tells the OIDC server where to redirect the user after successful authentication. If not set, the Web Console defaults it to the -location where it was loaded from (`window.location.href`). +location it was loaded from, without the query string or fragment +(`window.location.origin + window.location.pathname`). ### acl.oidc.scope - **Default**: `openid` - **Reloadable**: no -The OIDC server asks consent for the scopes listed in this property. The -scope `openid` is mandatory and must always be included. +The OIDC server asks consent for the scopes listed in this property. Keep +`openid` in the value for QuestDB's OIDC flows. It requests OIDC authentication +semantics and an ID token from the provider. QuestDB passes the value on without +inspecting it. Without `openid`, a provider may process the request as OAuth2 +and issue only an access token, or reject it according to provider policy. + +QuestDB uses the scopes in the requests it makes itself, in the +[ROPC flow](#acloidcropcflowenabled), and publishes them on the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) for clients which +run the flow themselves. + +For the [OIDC device flow](/docs/security/oidc/device-flow/), request the +provider's refresh-token scope, commonly `offline_access`, alongside `openid`: + +```ini title="server.conf" +acl.oidc.scope=openid offline_access +``` + +Requesting the scope does not guarantee a refresh token. The client +registration, user or administrator consent, and provider policy determine +whether one is issued. Some providers use a different scope, issue refresh +tokens without one, or reject an unsupported scope. Without a refresh token, +the client must ask the user to sign in again after the current token expires. +A client can request the scope through its own `scope` override, which leaves +this server-wide value, and every other flow using it, untouched. ## Authentication flows +QuestDB publishes [`acl.oidc.pkce.required`](#acloidcpkcerequired) and +[`acl.oidc.state.required`](#acloidcstaterequired) to clients through the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), and enforces +neither. The client generates the code verifier and the `state` value; the +provider checks the verifier, and the client checks the `state` value it gets +back. + ### acl.oidc.pg.token.as.password.enabled - **Default**: `false` - **Reloadable**: no -When enabled, the PGWire endpoint supports OIDC authentication. The OAuth2 -token should be sent in the password field, while the username field should -contain the string `_sso`, or left empty if that is an option. +When enabled, the PGWire endpoint accepts an OAuth2 token obtained by the +client. The token should be sent in the password field, while the username field +should contain the string `_sso`, or be left empty if that is an option. -### acl.oidc.pkce.enabled +This setting is not required for the ROPC path. To let a client such as `psql` +send the user's SSO username and password and have QuestDB exchange them for a +token, enable [`acl.oidc.ropc.flow.enabled`](#acloidcropcflowenabled) instead. +The two settings do not need to be enabled together. + +### acl.oidc.pkce.required - **Default**: `true` - **Reloadable**: no -Enables or disables PKCE for the Authorization Code Flow. This should always -be enabled in production. The Web Console is not fully secure without it. +Tells clients that PKCE is required for the Authorization Code Flow. This +should always be enabled in production. The Web Console is not fully secure +without it. ### acl.oidc.ropc.flow.enabled @@ -116,8 +219,48 @@ be enabled in production. The Web Console is not fully secure without it. Enables or disables the Resource Owner Password Credentials flow. When enabled, this flow must also be configured in the OIDC Provider. +With it enabled QuestDB runs the flow itself: a username and password arriving +over HTTP basic authentication or PGWire that match no local user are sent on to +the OIDC Provider's token endpoint as a password grant, and the user is logged +in if the provider issues a token. This lets clients which cannot follow a +browser redirect, such as `psql`, authenticate with their SSO credentials. + +For this `psql` login path, enable this setting along with the normal OIDC +configuration. +[`acl.oidc.pg.token.as.password.enabled`](#acloidcpgtokenaspasswordenabled) is +not required; that setting is for clients which obtain an OAuth2 token +themselves and send the token, rather than their SSO password, to QuestDB. + +Local users are matched first, so a QuestDB user whose name also exists in the +Identity Provider is authenticated against its local password, without involving +the provider. + +Unlike [`acl.oidc.pkce.required`](#acloidcpkcerequired) and +[`acl.oidc.state.required`](#acloidcstaterequired), this setting is not +published on the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so a client cannot +discover whether the flow is available. + +### acl.oidc.state.required + +- **Default**: `false` +- **Reloadable**: no + +Tells clients that the `state` parameter is required in the Authorization Code +Flow, which protects against CSRF attacks. Enable it if the OIDC Provider +requires the `state` parameter, or to add CSRF protection on top of PKCE. + +The [Web Console](/docs/getting-started/web-console/overview/) generates the +value, sends it in the authorization request, and checks that the provider +returns it unchanged. See +[Secret generation](/docs/security/oidc/how-sign-in-works/#1-secret-generation). + ## Endpoints +These settings apply only when the OIDC Provider is configured by host. When +`acl.oidc.configuration.url` is set, QuestDB takes every endpoint from the +provider's configuration document and the settings below are not used. + ### acl.oidc.authorization.endpoint - **Default**: `/as/authorization.oauth2` @@ -126,6 +269,23 @@ enabled, this flow must also be configured in the OIDC Provider. OIDC Authorization Endpoint. The default value should work for the Ping Identity Platform. +### acl.oidc.device.authorization.endpoint + +- **Default**: none +- **Reloadable**: no + +OIDC Device Authorization Endpoint. Unlike the other endpoint settings this one +has no default, and QuestDB never calls it. QuestDB resolves the endpoint and +publishes it on the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), for clients which +implement the +[OIDC device flow](/docs/security/oidc/device-flow/) +themselves. The official Java, Python, Rust, C, and C++ clients can discover and +use the published endpoint. + +Left unset, and absent from the provider's configuration document, the endpoint +stays unresolved and the key is omitted from the settings response. + ### acl.oidc.public.keys.endpoint - **Default**: `/pf/JWKS` @@ -135,6 +295,16 @@ JSON Web Key Set (JWKS) Endpoint. Provides the list of public keys used to decode and validate ID tokens issued by the OIDC Provider. The default value should work for the Ping Identity Platform. +The keys are only used to validate tokens when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`. +With the default user info flow QuestDB validates tokens by calling the user +info endpoint instead. + +QuestDB attempts to download the keys from this endpoint at startup either way. +A failure is logged and does not stop the server. Until a download succeeds, +the cache is empty and ID-token validation fails because QuestDB cannot find the +signing key. + ### acl.oidc.token.endpoint - **Default**: `/as/token.oauth2` @@ -167,13 +337,18 @@ Whether the OIDC provider requires a secure connection. If the OpenID Provider endpoints do not require TLS, this can be set to `false`. This is unlikely in production. +This setting must match the scheme of every OIDC Provider URL QuestDB uses, +including `acl.oidc.configuration.url` and each endpoint discovered from it. +A URL whose scheme does not match fails server startup. + ### acl.oidc.tls.keystore.password - **Default**: none - **Reloadable**: no -Keystore password. Required if a keystore file is configured and is password -protected. +Keystore password. Must be set whenever `acl.oidc.tls.keystore.path` is set. +When OIDC is enabled, setting either one without the other fails server +startup. ### acl.oidc.tls.keystore.path @@ -196,22 +371,22 @@ which it connects. ## User and group claims -### acl.oidc.cache.ttl - -- **Default**: `30000` -- **Reloadable**: no - -User info cache entry TTL in milliseconds. QuestDB caches user info responses -for each valid access token. This setting controls how often the access token -is validated and user info refreshed. - ### acl.oidc.groups.claim -- **Default**: `groups` +- **Default**: none - **Reloadable**: no The name of the custom claim in the user information that contains the -group memberships of the user. +group memberships of the user. Required when OIDC is enabled. + +If the claim is missing from the user information, or it is an empty list, +authentication fails. See +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). + +The name applies to both flows, including when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`. +The claim's value is normally an array of group names. A single bare string is +accepted too, which is how some providers send a lone group. ### acl.oidc.groups.encoded.in.token @@ -222,6 +397,64 @@ When `true`, QuestDB looks for group memberships in the ID token instead of calling the User Info endpoint. Set to `true` if the OIDC Provider encodes group memberships directly into the token. +This also changes which token the client has to send: the ID token when the +setting is `true`, the access token when it is `false`. QuestDB publishes the +setting on the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) so that clients can +[pick the right one](/docs/security/oidc/client-discovery/#which-token-to-send). + +It changes how tokens are validated too. In the default user info flow QuestDB +hands the token to the OIDC Provider on every cache miss, so the provider +decides whether it is still valid. With this setting enabled QuestDB validates +the token itself and never asks the provider about it: + +| Checked | Not checked | +| --- | --- | +| the signature, against the public key named by the token's `kid` | `nbf`, the not-before time | +| `exp`, the expiry | `iss`, the issuer | +| `aud`, against [`acl.oidc.audience`](#acloidcaudience) | | +| that `sub` and the group memberships are present | | + +Two components read the token. The signature validator requires the JWT to carry +`sub`, `aud` and `exp`, and checks the signature, the audience and the expiry +against them. It does not look at the group memberships. QuestDB then reads the +principal and the groups out of that same payload using +[`acl.oidc.sub.claim`](#acloidcsubclaim) and +[`acl.oidc.groups.claim`](#acloidcgroupsclaim), exactly as it does in the user +info flow. + +So `acl.oidc.groups.claim` may name any claim the token carries, `roles` for +example. `acl.oidc.sub.claim` may name any claim too, though the token must +still carry `sub` itself for the validator: a provider which issues both `sub` +and a friendlier claim lets you keep the readable one as the principal. The +Entra ID walkthrough does this, setting `acl.oidc.sub.claim=name` while the +token still carries `sub` for the validator. + +:::note Version requirement + +QuestDB Enterprise 4.0.1 and earlier do not validate `exp` in this mode. They +also require a claim literally named `groups` before applying the claim name +configured with `acl.oidc.groups.claim`. Upgrade to a later release before +relying on expiry validation or a differently named groups claim. + +::: + +:::caution + +Because QuestDB never asks the provider about the token, revoking a token does +not immediately end the access it grants. On a new authentication, QuestDB may +accept a token for up to 60 seconds after `exp` to allow for clock skew. A token +validated before or during that allowance may then remain in the authentication +cache for up to [`acl.oidc.cache.ttl`](#acloidccachettl). An established PGWire +or WebSocket connection is not closed when its token expires. + +Withdrawing a signing key can shorten this window, but only after QuestDB +successfully reloads the provider's key set. See +[`acl.oidc.public.keys.expiry`](#acloidcpublickeysexpiry). Keep token lifetimes +and the authentication cache TTL short if revocation must take effect quickly. + +::: + ### acl.oidc.sub.claim - **Default**: `sub` @@ -230,3 +463,87 @@ group memberships directly into the token. The name of the claim in the user information that contains the user's name. Could be a username, full name, or email. Displayed in the Web Console and logged for audit purposes. + +If the claim is missing from the user information, or empty, authentication +fails. The same applies to the claim named by +[`acl.oidc.groups.claim`](#acloidcgroupsclaim). See +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). + +## Caching and buffers + +### acl.oidc.cache.ttl + +- **Default**: `30000` +- **Reloadable**: no + +OIDC authentication cache entry TTL in milliseconds, as a plain integer only. +QuestDB caches the principal and group-derived access list produced by a +successful authentication. This setting controls how often the token is checked +again and that access list is rebuilt. + +Set it to `0` to disable the cache, so that every request is checked again. In +the default user info flow that means a call to the OIDC Provider on every +request. When +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true` +QuestDB checks the token locally instead, and contacts the provider only when +the public keys have to be reloaded. + +The local expiry check allows 60 seconds for clock skew. A token validated +before or during that allowance can remain accepted from the cache for up to +this TTL afterward. Setting the TTL to `0` removes that additional cache window, +but not the clock-skew allowance. It still does not make QuestDB consult the +provider, so provider-side token revocation, user disabling, and group changes +are not discovered from the same ID token. Those changes require the client to +obtain a new token. See +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) for what is +and is not validated. + +The TTL applies when a request or connection authenticates. It does not +continuously revalidate an established PGWire or WebSocket connection. + +### acl.oidc.public.keys.expiry + +- **Default**: `120000` +- **Reloadable**: no + +Expiry of the cached JSON Web Key Set (JWKS) in milliseconds. Also accepts a +duration, such as `2m` or `120s`. + +QuestDB caches the public keys used to validate tokens issued by the OIDC +Provider. After the cache expires, the next token validation attempts to reload +them from the public keys endpoint. + +A token signed with a key QuestDB has not cached triggers an immediate reload, +independently of this setting. QuestDB replaces the cached set only after it +downloads and parses a non-empty replacement successfully. If a reload fails, +the failure is logged and the previous keys remain available while later token +validations continue to retry. A key withdrawn by the provider can therefore +remain usable beyond this expiry during a provider or network failure. + +Lowering this setting reduces the normal delay before QuestDB notices a +withdrawn key after a successful reload, at the cost of more requests to the +endpoint. It does not provide a hard revocation deadline. + +Only used when +[`acl.oidc.groups.encoded.in.token`](#acloidcgroupsencodedintoken) is `true`, +which is the only case in which QuestDB validates token signatures itself. + +### acl.oidc.response.buffer.size + +- **Default**: `1M` +- **Reloadable**: no + +Size of the buffer used to receive and parse HTTP responses from the OIDC +Provider. Accepts a plain byte count, or a value with a `K` or `M` suffix, such +as `512K`. There is no `G` suffix. + +When a request to the OIDC Provider fails, authentication fails with it and the +reason is logged by the server. + +### acl.oidc.string.pool.capacity + +- **Default**: `128` +- **Reloadable**: no + +Initial capacity of the string pool used when parsing JSON responses received +from the OIDC Provider. diff --git a/documentation/configuration/overview.md b/documentation/configuration/overview.md index 45dbc1c738..a6dc02454e 100644 --- a/documentation/configuration/overview.md +++ b/documentation/configuration/overview.md @@ -536,7 +536,7 @@ http.net.connection.sndbuf=2m | [Logging & Metrics](/docs/configuration/logging-metrics/) | Log levels and metrics | | | [Materialized views](/docs/configuration/materialized-views/) | Materialized view refresh settings | | | [Minimal HTTP server](/docs/configuration/http-min-server/) | Health check and metrics endpoint | | -| [OpenID Connect (OIDC)](/docs/configuration/oidc/) | OIDC integration | ✓ | +| [OIDC settings](/docs/configuration/oidc/) | `acl.oidc.*` reference for OIDC authentication | ✓ | | [Parallel SQL execution](/docs/configuration/parallel-sql-execution/) | Query parallelism settings | | | [Postgres wire protocol](/docs/configuration/postgres-wire-protocol/) | PostgreSQL wire protocol connections | | | [QuestDB Wire Protocol (QWP)](/docs/configuration/qwp/) | QWP protocol limits and UDP receiver | | diff --git a/documentation/connect/clients/c-and-cpp.md b/documentation/connect/clients/c-and-cpp.md index 9b82a307ca..0e822078d0 100644 --- a/documentation/connect/clients/c-and-cpp.md +++ b/documentation/connect/clients/c-and-cpp.md @@ -7,6 +7,8 @@ description: "QuestDB C and C++ client: the questdb_db / questdb::pool connectio import Tabs from "@theme/Tabs" import TabItem from "@theme/TabItem" +import OidcDeviceFlowCpp from "../../partials/_oidc.device-flow.cpp.partial.mdx"; +import OidcDeviceFlowC from "../../partials/_oidc.device-flow.c.partial.mdx"; The C and C++ clients ingest and query over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/), a columnar binary @@ -325,16 +327,64 @@ Because the pool connects lazily, a bad credential surfaces as Handle it there, not at `connect` (see [Which errors mean what](#which-errors-mean-what)). -### Unsupported auth paths +### OIDC device flow (Enterprise) -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: +The C and C++ APIs sign in an interactive user with the +[OIDC device flow](/docs/security/oidc/device-flow/). +They discover the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to the +builder; the client appends `/settings`. Attach the resulting auth object to the +pool so sender and reader connections share its rotating token: -| Path | Status | Workaround | -|---|---|---| -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the pool when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. See the connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). | -| Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, close the pool and build a fresh one with the new token. | + + + + + + + + + + + + + +The pool retains the auth state and gets a cached or silently refreshed token +for every connection and reconnect. Silent refresh needs a refresh token. +Request the provider's refresh-token scope, commonly `offline_access`; the +client registration, consent, and provider policy still determine whether one +is issued. See +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Those transport +operations never prompt; they fail with +`QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` in C, or +`questdb::oidc::error_kind::interaction_required` in C++, and the application +calls `sign_in()` / `questdb_oidc_auth_sign_in()` explicitly on the main or UI +thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise set the builder's `issuer`, or set `client_id`, `scope`, `audience`, +`token_endpoint` and `device_authorization_endpoint` yourself; in C the same +names are prefixed with `questdb_oidc_builder_`. + +Tokens stay in memory until `.default_file_token_store()` in C++, or +`questdb_oidc_builder_default_file_token_store()` in C, writes a long-lived +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). + +### Other authentication limitations + +- Mutual TLS client certificates are not supported because the QuestDB server + does not negotiate them. Use bearer-token authentication over `wss`; see the + connect-string reference's [TLS section](/docs/connect/clients/connect-string/#tls). +- A token supplied directly through `token=...` remains fixed. Attach an OIDC + auth object for device-flow token refresh, or rebuild the pool when an + externally acquired token rotates. ## Headers diff --git a/documentation/connect/clients/connect-string.md b/documentation/connect/clients/connect-string.md index c027700dd5..7e82579591 100644 --- a/documentation/connect/clients/connect-string.md +++ b/documentation/connect/clients/connect-string.md @@ -220,6 +220,12 @@ WebSocket upgrade request. exclusive with `username` / `password`. Token auth avoids the per-request overhead of basic auth and is the recommended path for Enterprise deployments. +- Device-flow OIDC credentials are **not** connect-string keys. The Java, + Python, Rust, C and C++ clients take an auth object alongside the connect + string, which rotates the token on every reconnect; see + [OIDC device flow](/docs/security/oidc/device-flow/). A + static `token=` does not rotate, so a reconnect after expiry keeps sending + the stale value. - `auth_timeout_ms` — per-host upper bound on the upgrade response read. Does not cover TLS handshake or post-upgrade frame reads, which use OS or hard-coded defaults. Default: `15000` (15 s). diff --git a/documentation/connect/clients/dotnet.md b/documentation/connect/clients/dotnet.md index 684ea97cd3..db8a0f2547 100644 --- a/documentation/connect/clients/dotnet.md +++ b/documentation/connect/clients/dotnet.md @@ -228,7 +228,7 @@ that purpose; left unset, it inherits `auth_timeout_ms`. | Path | Status | Workaround | |---|---|---| -| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC — see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. | +| OIDC token acquisition or in-band refresh | Not supported by this client. It does not negotiate with an identity provider and has no callback to refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire a token out-of-band from your IdP, using QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) to discover the provider's authorization and token endpoints and [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send), pass it via `token=...` above, and rebuild the sender / query client when the token nears expiry. To acquire one, run the [OIDC device flow](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) with an OAuth2 library such as MSAL.NET; QuestDB publishes the device authorization endpoint on the same settings response. | | Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss://`. See the connect-string reference for the canonical statement. | | Token rotation mid-session | Not supported. Credentials are presented once during the WebSocket upgrade and are not re-sent. | On token expiry, `await sender.DisposeAsync()` and build a fresh sender with the new token. The same applies to `QueryClient`. | diff --git a/documentation/connect/clients/go.md b/documentation/connect/clients/go.md index 32c9ac9c08..febe415a15 100644 --- a/documentation/connect/clients/go.md +++ b/documentation/connect/clients/go.md @@ -360,8 +360,20 @@ db, err := qdb.Connect(ctx, The token is sent as an `Authorization: Bearer YOUR_BEARER_TOKEN` header on both the ingress and egress WebSocket upgrades. It is a **static credential**: the client sends exactly the string you pass and never refreshes or renews it. -Acquire it out of band — QuestDB Enterprise issues bearer tokens through its -[OpenID Connect flow](/docs/security/oidc/) — and manage its lifetime yourself. +Acquire it out of band, since QuestDB Enterprise issues bearer tokens through +its [OpenID Connect flow](/docs/security/oidc/), and manage its lifetime +yourself. +QuestDB publishes the provider's authorization and token endpoints on its +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), so a client can +discover them instead of hard coding them. The same response tells the client +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access +token, or the ID token when QuestDB reads group memberships from the token. +This client does not run an OIDC flow, so acquire the token with an OAuth2 +library: `golang.org/x/oauth2` implements the device grant, and QuestDB +publishes the device authorization endpoint on the same settings response. The +[OIDC device flow](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) +sets out the two requests, the polling interval, `slow_down`, and the device +code's expiry. When the token expires or is rotated, construct a new handle with the new token. An expired or rejected token surfaces as an authentication failure (see [Connection-level errors](#connection-level-errors)). It is mutually exclusive diff --git a/documentation/connect/clients/java.md b/documentation/connect/clients/java.md index 1df7d8239f..430bd6d27c 100644 --- a/documentation/connect/clients/java.md +++ b/documentation/connect/clients/java.md @@ -12,6 +12,7 @@ import TabItem from "@theme/TabItem" import SfDedupWarning from "../../partials/_sf-dedup-warning.partial.mdx" import CodeBlock from "@theme/CodeBlock" +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.java.partial.mdx"; :::note @@ -397,6 +398,52 @@ The token is sent as an `Authorization: Bearer YOUR_BEARER_TOKEN` header on both the ingress and egress WebSocket upgrades. It is mutually exclusive with `username`/`password`. +### OIDC device flow (Enterprise) + +`OidcDeviceAuth` signs in an interactive user with the +[OIDC device flow](/docs/security/oidc/device-flow/). +It discovers the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to +`fromQuestDB`; the client appends `/settings`. The user can approve the sign-in +from a browser on any device, so this also works from a container or remote +notebook kernel: + + + +Pass `auth::getToken` as a provider instead of putting the current token in the +connect string. Every new connection and reconnect then receives the cached or +silently refreshed token. Silent refresh needs a refresh token. Request the +provider's refresh-token scope, commonly `offline_access`; the client +registration, consent, and provider policy still determine whether one is +issued. See +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call +never starts an interactive flow; it throws `OidcAuthException`, which carries +no distinct interaction-required type, so call `signIn()` again on the main or +UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise pin the provider by passing +`new OidcDeviceAuth.DiscoveryOptions().issuer("https://idp")` as the second +argument to `fromQuestDB`, or set the client ID, scope and endpoints yourself +with `OidcDeviceAuth.builder()`. + +`DiscoveryOptions` carries `issuer`, `prompt`, `tokenStore`, `tlsConfig` and +`allowInsecureTransport` only. The client ID, scope, audience and +token-selection mode live on `OidcDeviceAuth.builder()`, which requires the +client ID and both endpoints together, so registering this application under its +own Client Id means pinning the endpoints rather than discovering them. + +Tokens stay in memory until a store is attached with +`new OidcDeviceAuth.DiscoveryOptions().tokenStore(FileTokenStore.atDefaultLocation())`, +which writes a long-lived refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). + ### HTTP basic auth ```java diff --git a/documentation/connect/clients/nodejs.md b/documentation/connect/clients/nodejs.md index de60fe5610..5fcadc977a 100644 --- a/documentation/connect/clients/nodejs.md +++ b/documentation/connect/clients/nodejs.md @@ -87,9 +87,37 @@ const sender = Sender.fromEnv(); ... ``` -When using QuestDB Enterprise, authentication can also be done via REST token. -Please check the [RBAC docs](/docs/security/rbac/#authentication) for more -info. +When using QuestDB Enterprise, authentication can also be done via a +[REST token](/docs/security/rbac/authentication/) in the `token` setting. + +This client does not run an OIDC flow. QuestDB Enterprise also accepts an OIDC +bearer token, which you acquire out of band from your Identity Provider: +QuestDB publishes the provider's endpoints on its +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), and the same +response tells you [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). +See [OpenID Connect](/docs/security/oidc/), and +[implementing the flow yourself](/docs/security/oidc/device-flow/#implementing-the-flow-yourself) for the device grant's +requests and polling rules. + +Pass the selected bearer token to `Sender` with the same `token` setting. Use +`https::` so the credential is encrypted in transit: + +```javascript title="Use an OIDC bearer token" +const { Sender } = require("@questdb/nodejs-client") + +const token = process.env.QUESTDB_OIDC_TOKEN +if (!token) { + throw new Error("QUESTDB_OIDC_TOKEN is not set") +} + +const sender = Sender.fromConfig( + `https::addr=questdb.example.com:9000;token=${token};`, +) +``` + +The sender treats this as a static token: reconnecting does not acquire or read +a replacement. Before the token expires, finish and flush the current batch, +close the sender, acquire a new token, and construct a new sender with it. ## Basic insert diff --git a/documentation/connect/clients/python.md b/documentation/connect/clients/python.md index be8d79aa24..4884e6ea9f 100644 --- a/documentation/connect/clients/python.md +++ b/documentation/connect/clients/python.md @@ -5,6 +5,8 @@ sidebar_label: Python description: "Use the QuestDB Python client's QuestDB pool for row, DataFrame, and Arrow ingestion plus SQL queries over QWP." --- +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.python.partial.mdx"; + The QuestDB Python client uses one `QuestDB` handle for ingestion and SQL queries over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/). Lease a short-lived sender for each unit of row-building work, bulk-load DataFrames through the handle, and run SQL with `query()`. @@ -164,16 +166,54 @@ operating-system certificate store. Override it with configuration keys: See the [connect string reference](/docs/connect/clients/connect-string/) for the full grammar. -### Unsupported auth paths - -The client supports only HTTP basic auth and static bearer-token auth. The -following are **not** supported: - -| Path | Status | Workaround | -| --- | --- | --- | -| OIDC token acquisition or in-band refresh | Not supported. The client does not negotiate with an identity provider and cannot refresh a token mid-session. | QuestDB itself supports OIDC; see [OpenID Connect](/docs/security/oidc/). Acquire an access token out-of-band from your IdP, pass it via `token=...`, and rebuild the handle when the token nears expiry. | -| Mutual TLS (client certificates) | Not supported. The QuestDB server does not negotiate client certificates regardless of client. | Use bearer-token auth over `wss`. | -| Token rotation mid-session | Not supported. The handle keeps the credentials it was built with and presents them on every connection it opens — including reconnects and failover, so an expired token also breaks mid-session reconnection. | On token expiry, close the handle and build a fresh one with the new token. | +### OIDC device flow (Enterprise) + +`OidcDeviceAuth` signs in an interactive user with the +[OIDC device flow](/docs/security/oidc/device-flow/). +It discovers the provider endpoints, client ID, scope, and the token QuestDB +expects from the server's public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL passed to +`from_questdb`; the client appends `/settings`. The default prompt works in +terminals and remote Jupyter kernels: + + + +Device flow also works in Docker and Kubernetes without an inbound callback +port. Disable browser launching, expose stderr to the operator, and persist the +token store on a protected volume; see +[Docker and Kubernetes](/docs/security/oidc/device-flow/#docker-and-kubernetes). + +`oidc_auth=auth` keeps shared ownership of the provider and obtains the cached +or silently refreshed token for every connection and reconnect. It is mutually +exclusive with a fixed `token=` setting. Silent refresh needs a refresh token. +Request the provider's refresh-token scope, commonly `offline_access`; the +client registration, consent, and provider policy still determine whether one +is issued. See +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). Transport operations +never prompt; if they raise `OidcInteractionRequired`, importable from +`questdb.auth`, call `auth.sign_in()` explicitly on the main or UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise pass `issuer=...`, or set `client_id=`, `scope=`, `audience=`, +`token_endpoint=` and `device_authorization_endpoint=` on `from_questdb`. + +Tokens stay in memory until a store is passed as +`token_store=FileTokenStore.at_default_location()`, which writes a long-lived +refresh token to disk as plaintext. See [token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). + +### Other authentication limitations + +- Mutual TLS client certificates are not supported because the QuestDB server + does not negotiate them. Use bearer-token authentication over `wss`. +- A token supplied directly through `token=...` remains fixed. Use + `oidc_auth=...` for device-flow token refresh, or close the handle and build a + new one when an externally acquired token rotates. ## The pool diff --git a/documentation/connect/clients/rust.md b/documentation/connect/clients/rust.md index 20fb11dbb2..1d74913c93 100644 --- a/documentation/connect/clients/rust.md +++ b/documentation/connect/clients/rust.md @@ -5,6 +5,8 @@ sidebar_label: Rust description: "Use the QuestDB Rust connection pool for Buffer, Chunk, Arrow, and Polars ingestion plus streaming SQL queries over QWP." --- +import OidcDeviceFlowExample from "../../partials/_oidc.device-flow.rust.partial.mdx"; + The QuestDB Rust client uses a thread-safe `QuestDb` pool for ingestion and SQL queries over [QWP](/docs/connect/wire-protocols/qwp-ingress-websocket/). Borrow a short-lived writer or reader for each unit of work, then let `Drop` return its @@ -163,6 +165,60 @@ choices have feature requirements: | `tls_roots_password=...` | Unlocks a JKS or PKCS#12 store named by `tls_roots`. | | `tls_verify=unsafe_off` | Enable `insecure-skip-verify`; use only in controlled tests. | +### OIDC device flow (Enterprise) + +Enable the `oidc` feature to sign in an interactive user with the +[OIDC device flow](/docs/security/oidc/device-flow/): + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7.1", features = ["oidc"] } +``` + +`OidcDeviceAuth::from_questdb` discovers the provider endpoints, client ID, +scope, and the token QuestDB expects from the public +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint). +Include any non-default Web Console context in the QuestDB URL; the client +appends `/settings`: + + + +Pass the closure as a provider instead of putting the current token in the +connect string. Every new connection and reconnect then receives the cached or +silently refreshed token. Silent refresh needs a refresh token. Request the +provider's refresh-token scope, commonly `offline_access`; the client +registration, consent, and provider policy still determine whether one is +issued. See +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). A transport call +never starts an interactive flow; it returns a `questdb::Error` with +`ErrorCode::AuthError`, whose `err.oidc_error()` reports +`OidcErrorKind::InteractionRequired` when a new sign-in is needed. Call +`sign_in()` explicitly on the main or UI thread. + +The Identity Provider must enable the device grant. For discovery without an +override, QuestDB must publish a device authorization endpoint, either from the +provider's configuration document or from +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +Otherwise add `.issuer(...)` to the builder, or set `.client_id()`, `.scope()`, +`.audience()`, `.token_endpoint()` and `.device_authorization_endpoint()` +yourself. + +Tokens stay in memory until a token store is attached, which writes a +long-lived refresh token to disk as plaintext. `at_default_location()` returns a +`std::io::Result`, and `?` does not convert an `io::Error` into a +`questdb::Error`, so map it first: + +```rust +let store = FileTokenStore::at_default_location() + .map_err(|e| Error::new(ErrorCode::ConfigError, format!("token store: {e}")))?; +let auth = OidcDeviceAuth::from_questdb(url).token_store(store).build()?; +``` + +`FileTokenStore::at(path)` takes a directory directly and needs no mapping. See +[token persistence](/docs/security/oidc/device-flow/#token-persistence) for the store's location and +permissions, [the sign-in prompt](/docs/security/oidc/device-flow/#the-sign-in-prompt), and +[explicit configuration](/docs/security/oidc/device-flow/#explicit-configuration-and-endpoint-pinning). + ## The pool `QuestDb` owns reusable QWP/WebSocket connections. Create one pool per @@ -923,6 +979,7 @@ integrations only when your application uses them: | `ndarray` | No | `Buffer::column_arr` from `ndarray` views. | | `rust_decimal` / `bigdecimal` | No | Row-buffer decimal values from those crates. Decimal strings need neither feature. | | `chrono-timestamp` | No | Timestamp values built from `chrono::DateTime`. | +| `oidc` | No | Interactive OIDC sign-in with the [device flow](/docs/security/oidc/device-flow/). Pulls in `sync-sender-http`, and needs a TLS root source for `https` discovery. | | `tls-native-certs` | No | TLS validation through the operating-system certificate store. | | `insecure-skip-verify` | No | `tls_verify=unsafe_off` for controlled testing only. | | `almost-all-features` | No | Client development and testing with most compatible features. It excludes Arrow and Polars. | diff --git a/documentation/connect/compatibility/rest-api.md b/documentation/connect/compatibility/rest-api.md index 033060ff2c..e2355c155d 100644 --- a/documentation/connect/compatibility/rest-api.md +++ b/documentation/connect/compatibility/rest-api.md @@ -794,11 +794,16 @@ A HTTP status code of `400` is returned with the following response body: ## Authentication -The REST API supports two authentication types: +The REST API supports three authentication types: - **HTTP basic authentication**, available in QuestDB Open Source and QuestDB Enterprise. - **Token-based authentication**, available in QuestDB Enterprise only. +- **OIDC bearer tokens**, available in QuestDB Enterprise only. These also + travel in the `Authorization: Bearer` header but are issued by an external + Identity Provider rather than by QuestDB, so they are a different credential + from the REST API token below. See [OpenID Connect](/docs/security/oidc/) and + [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). :::note @@ -885,5 +890,5 @@ curl -G --data-urlencode "query=SELECT 1;" \ http://localhost:9000/api/v1/sql/execute ``` -Refer to the [user management](/docs/security/rbac/#user-management) page to +Refer to the [user management](/docs/security/rbac/users-and-groups/#user-management) page to learn more on how to generate a REST API token. diff --git a/documentation/connect/wire-protocols/qwp-ingress-websocket.md b/documentation/connect/wire-protocols/qwp-ingress-websocket.md index e27c7ea5eb..f20ebf96fa 100644 --- a/documentation/connect/wire-protocols/qwp-ingress-websocket.md +++ b/documentation/connect/wire-protocols/qwp-ingress-websocket.md @@ -143,7 +143,10 @@ Supported methods: [HTTP basic authentication](/docs/connect/compatibility/rest-api/#http-basic-authentication). - **Token-based auth** (Enterprise only): see [Authentication via token in QuestDB Enterprise](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise). -- **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). +- **OIDC** (Enterprise only): see [OpenID Connect](/docs/security/oidc/). The + [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) publishes the + provider's authorization and token endpoints, and tells the client + [which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). A failed authentication results in a `401` or `403` HTTP response before the WebSocket connection is established. No QWP-level auth handshake exists. diff --git a/documentation/deployment/azure.md b/documentation/deployment/azure.md index 8351978eab..ebbb8ad5e6 100644 --- a/documentation/deployment/azure.md +++ b/documentation/deployment/azure.md @@ -345,6 +345,6 @@ QuestDB Enterprise adds production features for Azure: - **EntraID SSO** - Single sign-on with Microsoft Entra ID For EntraID integration, see the -[Microsoft EntraID OIDC guide](/docs/security/oidc/#microsoft-entraid). +[Microsoft EntraID OIDC guide](/docs/security/oidc/entra-id/). See [Enterprise Quick Start](/docs/getting-started/enterprise-quick-start/) for setup. diff --git a/documentation/getting-started/enterprise-quick-start.md b/documentation/getting-started/enterprise-quick-start.md index 2ce88e20dc..026d80bef7 100644 --- a/documentation/getting-started/enterprise-quick-start.md +++ b/documentation/getting-started/enterprise-quick-start.md @@ -186,13 +186,13 @@ GRANT ALL ON table2 TO user2 WITH GRANT OPTION; Permission grants can be specific and fine-tuned. -List the full list of applied permissions with `all_permissions()`. +List every permission the database supports with `all_permissions()`. - For the full role-based access control docs, including group management, see the [RBAC operations guide](/docs/security/rbac/). - For a full list of available permissions, see the - [permissions sub-section in the RBAC operations guide](/docs/security/rbac/#permissions). + [permissions sub-section in the RBAC operations guide](/docs/security/rbac/permissions-reference/). ## 4. Ingest data, InfluxDB Line Protocol diff --git a/documentation/getting-started/web-console/overview.md b/documentation/getting-started/web-console/overview.md index a69e9a6bc5..ff5fe17704 100644 --- a/documentation/getting-started/web-console/overview.md +++ b/documentation/getting-started/web-console/overview.md @@ -125,5 +125,5 @@ If `http.settings.readonly` configuration is set to true, instance information i ::: :::info -When using QuestDB Enterprise with Role-Based Access Control (RBAC), only the users with `SETTINGS` or `DATABASE ADMIN` permission can edit the instance information. See [Database Permissions](/docs/security/rbac/#database-permissions) for more details. +When using QuestDB Enterprise with Role-Based Access Control (RBAC), only the users with `SETTINGS` or `DATABASE ADMIN` permission can edit the instance information. See [Database Permissions](/docs/security/rbac/permissions-reference/#database-permissions) for more details. ::: diff --git a/documentation/high-availability/failover.md b/documentation/high-availability/failover.md index 93f58855d3..ee8e99374a 100644 --- a/documentation/high-availability/failover.md +++ b/documentation/high-availability/failover.md @@ -53,7 +53,7 @@ needs the `SWITCH ROLE` permission. table is suspended and the WAL lag is nil. - An account with the `SWITCH ROLE` permission and with `PGWIRE` (for SQL) or `HTTP` (for the REST endpoint). See - [Failover operator](/docs/security/rbac/#failover-operator). + [Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). - Clients configured with a multi-host address list, so writers follow the primary role on their own. See [Client failover](/docs/high-availability/client-failover/concepts/). @@ -284,7 +284,9 @@ The [minimal HTTP server](/docs/operations/logging-metrics/#minimal-http-server) on port 9003 exposes the same switch to external coordinators. It accepts the same credentials as the main HTTP server: HTTP basic authentication, a [REST token](/docs/connect/compatibility/rest-api/#authentication-via-token-in-questdb-enterprise) -as `Authorization: Bearer`, or an OIDC access token. The principal needs the +as `Authorization: Bearer`, or an OIDC token. Which of the two OIDC tokens to +send is decided by `acl.oidc.groups.encoded.in.token`; see +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). The principal needs the `HTTP` endpoint permission and, for the switch, `SWITCH ROLE`. With access control disabled no credentials are needed. TLS for this port is configured with the `http.min.tls.*` settings on the [TLS](/docs/configuration/tls/) page. @@ -389,5 +391,5 @@ A coordinator drives a switch as follows: restart-based migration procedures and point-in-time recovery. - [Client failover](/docs/high-availability/client-failover/concepts/) for how clients follow the primary role. -- [Failover operator](/docs/security/rbac/#failover-operator) for the account +- [Failover operator](/docs/security/rbac/common-scenarios/#failover-operator) for the account that runs switches. diff --git a/documentation/operations/upgrade.md b/documentation/operations/upgrade.md index f36890607f..c3faf41421 100644 --- a/documentation/operations/upgrade.md +++ b/documentation/operations/upgrade.md @@ -97,7 +97,7 @@ every node runs the new version. The permission is stored in a form older versions cannot read, and a `GRANT ALL` issued before the upgrade does not include it. Accounts that triggered role switches through `SYSTEM ADMIN` on 3.3.x need an explicit `GRANT SWITCH ROLE`. See -[Failover operator](/docs/security/rbac/#failover-operator). +[Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). ### QuestDB Enterprise BYOC diff --git a/documentation/partials/_oidc.device-flow.c.partial.mdx b/documentation/partials/_oidc.device-flow.c.partial.mdx new file mode 100644 index 0000000000..fac8a12018 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.c.partial.mdx @@ -0,0 +1,86 @@ +```c +#include +/* questdb_db_borrow_sender and line_sender_buffer */ +#include +#include +#include +#include + +static void show_prompt(void *data, const questdb_oidc_event *event) { + (void)data; + if (event->kind == QUESTDB_OIDC_EVENT_PROMPT) + fprintf(stderr, "Open %.*s and enter %.*s\n", + (int)event->verification_uri_len, event->verification_uri, + (int)event->user_code_len, event->user_code); +} + +int main(void) { + questdb_error *error = NULL; + questdb_oidc_builder *builder = NULL; + questdb_oidc_auth *auth = NULL; + questdb_db *db = NULL; + qwp_sender *sender = NULL; + line_sender_buffer *buffer = NULL; + int status = 1; + + const char *url = "https://questdb.example.com:9000"; + builder = questdb_oidc_builder_from_questdb(url, strlen(url), &error); + if (!builder || !questdb_oidc_builder_event_handler( + builder, show_prompt, NULL, NULL, &error)) + goto done; + + auth = questdb_oidc_builder_build(builder, &error); + if (!auth || !questdb_oidc_auth_sign_in(auth, &error)) + goto done; + + questdb_db_connect_options options; + questdb_db_connect_options_init(&options, sizeof options); + options.oidc_auth = auth; + + const char *conf = "wss::addr=questdb.example.com:9000;"; + db = questdb_db_connect_ex(conf, strlen(conf), &options, &error); + if (!db) goto done; + + /* Sender and reader borrows use the rotating token. */ + sender = questdb_db_borrow_sender(db, &error); + if (!sender) goto done; + buffer = questdb_db_new_buffer(db, &error); + if (!buffer) goto done; + if (!line_sender_buffer_table( + buffer, QDB_TABLE_NAME_LITERAL("trades"), &error)) + goto done; + if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("symbol"), + QDB_UTF8_LITERAL("ETH-USDT"), &error)) + goto done; + if (!line_sender_buffer_symbol(buffer, QDB_COLUMN_NAME_LITERAL("side"), + QDB_UTF8_LITERAL("sell"), &error)) + goto done; + if (!line_sender_buffer_column_f64( + buffer, QDB_COLUMN_NAME_LITERAL("price"), 2615.54, &error)) + goto done; + if (!line_sender_buffer_column_f64( + buffer, QDB_COLUMN_NAME_LITERAL("amount"), 0.00044, &error)) + goto done; + if (!line_sender_buffer_at_nanos( + buffer, line_sender_now_nanos(), &error)) + goto done; + if (!qwp_sender_flush_buffer_and_wait( + sender, buffer, qwpws_ack_level_ok, &error)) + goto done; + status = 0; + +done: + if (error) { + size_t len = 0; + const char *message = questdb_error_msg(error, &len); + fprintf(stderr, "OIDC device flow failed: %.*s\n", (int)len, message); + questdb_error_free(error); + } + line_sender_buffer_free(buffer); + if (sender) questdb_db_return_sender(db, sender); + questdb_db_close(db); + questdb_oidc_auth_free(auth); + questdb_oidc_builder_free(builder); + return status; +} +``` diff --git a/documentation/partials/_oidc.device-flow.cpp.partial.mdx b/documentation/partials/_oidc.device-flow.cpp.partial.mdx new file mode 100644 index 0000000000..8f2caa6ae8 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.cpp.partial.mdx @@ -0,0 +1,33 @@ +```cpp +#include // pool::borrow_sender +#include // pool::borrow_reader +#include +#include + +using namespace questdb::ingress::literals; + +int main() { + auto auth = questdb::oidc::builder::from_questdb( + "https://questdb.example.com:9000") + .event_handler([](const questdb::oidc::event_view& event) { + if (event.kind() == questdb::oidc::event_kind::prompt) + std::cerr << "Open " << event.verification_uri() + << " and enter " << event.user_code() << '\n'; + }) + .build(); + + auth.sign_in(); // the only call which may prompt or open a browser + questdb::pool pool{"wss::addr=questdb.example.com:9000;", auth}; + + auto sender = pool.borrow_sender(); + auto buffer = sender.new_buffer(); + buffer.table("trades"_tn) + .symbol("symbol"_cn, "ETH-USDT"_utf8) + .symbol("side"_cn, "sell"_utf8) + .column("price"_cn, 2615.54) + .column("amount"_cn, 0.00044) + .at(questdb::ingress::timestamp_nanos::now()); + sender.flush_and_wait(buffer); + // pool.borrow_reader() uses the same rotating token. +} +``` diff --git a/documentation/partials/_oidc.device-flow.java.partial.mdx b/documentation/partials/_oidc.device-flow.java.partial.mdx new file mode 100644 index 0000000000..2db37f1717 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.java.partial.mdx @@ -0,0 +1,26 @@ +```java +import io.questdb.client.QuestDB; +import io.questdb.client.Sender; +import io.questdb.client.cutlass.auth.OidcDeviceAuth; + +import java.time.Instant; + +try (OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000")) { + auth.signIn(); // the only call which may prompt or open a browser + + try (QuestDB db = QuestDB.connect( + "wss::addr=questdb.example.com:9000;", + auth::getToken)) { + try (Sender sender = db.borrowSender()) { + sender.table("trades") + .symbol("symbol", "ETH-USDT") + .symbol("side", "sell") + .doubleColumn("price", 2615.54) + .doubleColumn("amount", 0.00044) + .at(Instant.now()); + } + // db.borrowQuery() uses the same rotating token. + } +} +``` diff --git a/documentation/partials/_oidc.device-flow.python.partial.mdx b/documentation/partials/_oidc.device-flow.python.partial.mdx new file mode 100644 index 0000000000..ccaf6334b3 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.python.partial.mdx @@ -0,0 +1,22 @@ +```python +import questdb +from questdb import TimestampNanos +from questdb.auth import OidcDeviceAuth + +with OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000") as auth: + auth.sign_in() # the only call which may prompt or open a browser + + with questdb.connect( + "wss::addr=questdb.example.com:9000;", + oidc_auth=auth) as db: + with db.sender() as sender: + sender.row( + "trades", + symbols={"symbol": "ETH-USDT", "side": "sell"}, + columns={"price": 2615.54, "amount": 0.00044}, + at=TimestampNanos.now(), + ) + sender.flush(wait=True) + # db.dataframe() and db.query() use the same rotating token. +``` diff --git a/documentation/partials/_oidc.device-flow.rust.partial.mdx b/documentation/partials/_oidc.device-flow.rust.partial.mdx new file mode 100644 index 0000000000..f5559813f3 --- /dev/null +++ b/documentation/partials/_oidc.device-flow.rust.partial.mdx @@ -0,0 +1,37 @@ +```rust +use std::sync::Arc; +use questdb::{ + ingress::{AckLevel, TimestampNanos}, + oidc::OidcDeviceAuth, + QuestDb, +}; + +fn main() -> questdb::Result<()> { + let auth = Arc::new( + OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .build()?, + ); + auth.sign_in()?; // the only call which may prompt or open a browser + + let db = QuestDb::connect_with_token_provider( + "wss::addr=questdb.example.com:9000;", + { + let auth = Arc::clone(&auth); + move || auth.token() + }, + )?; + + let mut sender = db.borrow_sender()?; + let mut buffer = sender.new_buffer(); + buffer + .table("trades")? + .symbol("symbol", "ETH-USDT")? + .symbol("side", "sell")? + .column_f64("price", 2615.54)? + .column_f64("amount", 0.00044)? + .at(TimestampNanos::now())?; + sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok)?; + // db.borrow_reader() uses the same rotating token. + Ok(()) +} +``` diff --git a/documentation/query/sql/acl/alter-group.md b/documentation/query/sql/acl/alter-group.md new file mode 100644 index 0000000000..95ad2ce882 --- /dev/null +++ b/documentation/query/sql/acl/alter-group.md @@ -0,0 +1,64 @@ +--- +title: ALTER GROUP reference +sidebar_label: ALTER GROUP +description: + "ALTER GROUP adds or removes external Identity Provider aliases from a + QuestDB group for OIDC authorization in QuestDB Enterprise." +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + RBAC provides fine-grained database permissions management. + + +`ALTER GROUP` adds or removes an external Identity Provider mapping on an +existing QuestDB group. + +For an end-to-end OIDC example, see +[Mapping groups and permissions](/docs/security/oidc/group-mapping/). + +--- + +## Syntax + +```questdb-sql title="Add an external alias" +ALTER GROUP groupName WITH EXTERNAL ALIAS externalAlias; +``` + +```questdb-sql title="Remove an external alias" +ALTER GROUP groupName DROP EXTERNAL ALIAS externalAlias; +``` + +## Description + +`WITH EXTERNAL ALIAS` maps the group name or identifier in an OIDC groups claim +to `groupName`. An external user receives the QuestDB group's permissions when +their claim contains that exact alias. + +External aliases are globally unique. If the alias is already reserved, adding +it fails instead of replacing the existing mapping. + +`DROP EXTERNAL ALIAS` removes the named mapping. It does not drop the QuestDB +group or change permissions granted to it. + +## Permissions + +- `WITH EXTERNAL ALIAS` requires `ADD EXTERNAL ALIAS`. +- `DROP EXTERNAL ALIAS` requires `REMOVE EXTERNAL ALIAS`. + +## Examples + +Map an Entra ID group identifier to the QuestDB group `analysts`: + +```questdb-sql +ALTER GROUP analysts +WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` + +Remove that mapping without dropping `analysts`: + +```questdb-sql +ALTER GROUP analysts +DROP EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` diff --git a/documentation/query/sql/acl/create-group.md b/documentation/query/sql/acl/create-group.md index a538cdc04e..97537d2588 100644 --- a/documentation/query/sql/acl/create-group.md +++ b/documentation/query/sql/acl/create-group.md @@ -25,6 +25,10 @@ see the [RBAC operations](/docs/security/rbac) page. CREATE GROUP [IF NOT EXISTS] groupName; ``` +```questdb-sql title="Create an OIDC-mapped group" +CREATE GROUP groupName WITH EXTERNAL ALIAS externalAlias; +``` + ## Description `CREATE GROUP` adds a new user group with no permissions. @@ -37,12 +41,27 @@ the statement. Contrary to users and service accounts, it is not possible to log in as group. A group only serves as a container for permissions which are shared between users. +`WITH EXTERNAL ALIAS` maps a group name supplied by an OIDC Identity Provider +to the new QuestDB group. It cannot be combined with `IF NOT EXISTS`. Use +[`ALTER GROUP`](/docs/query/sql/acl/alter-group/) to add or remove mappings on +an existing group. + +External aliases are globally unique. If the alias is already reserved, +`CREATE GROUP` fails instead of reusing the mapping. + +## Permissions + +Creating a group requires the `CREATE GROUP` permission. The `WITH EXTERNAL +ALIAS` form also requires `ADD EXTERNAL ALIAS`. + ## Examples ```questdb-sql CREATE GROUP admins; CREATE GROUP IF NOT EXISTS admins; + +CREATE GROUP analysts WITH EXTERNAL ALIAS 'identity-provider-analysts'; ``` It can be verified with: @@ -53,6 +72,7 @@ SHOW GROUPS; that yields: -| name | -| ------ | -| admins | +| name | +| -------- | +| admins | +| analysts | diff --git a/documentation/query/sql/acl/create-service-account.md b/documentation/query/sql/acl/create-service-account.md index 8a6fa4fa06..a0d7c1fef3 100644 --- a/documentation/query/sql/acl/create-service-account.md +++ b/documentation/query/sql/acl/create-service-account.md @@ -23,20 +23,28 @@ see the [RBAC operations](/docs/security/rbac) page. ## Syntax ```questdb-sql -CREATE SERVICE ACCOUNT [IF NOT EXISTS] accountName [OWNED BY ownerName]; +CREATE SERVICE ACCOUNT [IF NOT EXISTS] accountName + [WITH { PASSWORD password | NO PASSWORD }] + [OWNED BY ownerName]; ``` ## Description `CREATE SERVICE ACCOUNT` adds a new service account with no permissions. +`WITH PASSWORD` enables password authentication at creation time. Omit the +clause, or use `WITH NO PASSWORD`, to create the account without a password. +Then use +[`ALTER SERVICE ACCOUNT`](/docs/query/sql/acl/alter-service-account/) to add a +password or token later. + The chosen name must be unique across all users (including the built-in admin), groups and service accounts. If the name has already been reserved, the command fails and an error is raised, unless the `IF NOT EXISTS` clause is included in the statement. Note that new service accounts can only access the database if the necessary -[endpoint permissions](/docs/security/rbac/#endpoint-permissions) have been +[endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions) have been granted. The user creating the service account automatically receives the @@ -47,18 +55,30 @@ group specified in the clause. The `OWNED BY` clause cannot be omitted if the service account is created by an external user, because permissions cannot be granted to them. +## Permissions + +Creating a service account without a password clause requires `CREATE SERVICE +ACCOUNT`. The explicit password forms need an additional permission: + +- `WITH PASSWORD` requires `ADD PASSWORD`. +- `WITH NO PASSWORD` requires `REMOVE PASSWORD`. + +Omitting the clause and writing `WITH NO PASSWORD` create the same account +state, but they do not have the same permission requirements. + ## Examples ```questdb-sql CREATE SERVICE ACCOUNT audit; CREATE SERVICE ACCOUNT IF NOT EXISTS audit; -``` +CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'change_me'; ``` + +```questdb-sql CREATE GROUP analysts; CREATE SERVICE ACCOUNT dashboard OWNED BY analysts; - ``` It can be verified with: diff --git a/documentation/query/sql/acl/create-user.md b/documentation/query/sql/acl/create-user.md index 8a8d428672..c3555ae6bc 100644 --- a/documentation/query/sql/acl/create-user.md +++ b/documentation/query/sql/acl/create-user.md @@ -37,7 +37,7 @@ fails and an error is raised, unless the `IF NOT EXISTS` clause is included in the statement. Note that new users can only access the database if the necessary -[endpoint permissions](/docs/security/rbac/#endpoint-permissions) have been +[endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions) have been granted. ## Conditional user creation diff --git a/documentation/query/sql/acl/grant.md b/documentation/query/sql/acl/grant.md index 3a2e17b45c..cccf77e1d4 100644 --- a/documentation/query/sql/acl/grant.md +++ b/documentation/query/sql/acl/grant.md @@ -221,7 +221,7 @@ By default, `GRANT` does not check whether entities exist, making it possible to grant permissions to users, groups or service accounts that are later created. To make sure that the target entity of the grant statement exists, use -[verification](/docs/security/rbac/#grant-verification). The +[verification](/docs/security/rbac/granting-permissions/#grant-verification). The `WITH VERIFICATION` clause enables checks on the target entity and causes the `GRANT` statement to fail if the entity does not exist. @@ -239,7 +239,7 @@ queries. Therefore when a table has a designated timestamp, granting `SELECT` or `UPDATE` permissions on any column will automatically extend those permissions to the timestamp column. These are known as -[implicit permissions](/docs/security/rbac/#implicit-permissions), and they're +[implicit permissions](/docs/security/rbac/granting-permissions/#implicit-permissions), and they're indicated by an `I` in the `origin` column of the `SHOW PERMISSIONS` output. For example, if you grant `UPDATE` permission on the `id` column of the @@ -326,54 +326,32 @@ ALTER TABLE countries ADD COLUMN description string; | UPDATE | countries | id | f | G | | UPDATE | countries | description | f | G | -### Grant when table or column is dropped and recreated +### Grants when a table or column is dropped and recreated {#grant-when-table-or-column-is-dropped-and-recreated} -Granted permissions are not automatically revoked when related tables or columns -are dropped. Instead, they have no effect until table or column is recreated. +Dropping a table permanently removes every permission granted on it. Recreating +a table with the same name does not restore those permissions: ```questdb-sql CREATE TABLE countries (id INT, name STRING, iso_code STRING); GRANT SELECT ON countries TO john; GRANT UPDATE ON countries(iso_code) TO john; -``` - -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | -| SELECT | countries | | f | G | -| UPDATE | countries | iso_code | f | G | - -Now, if the table is dropped, then permission stops being visible: -```questdb-sql DROP TABLE countries; -``` - -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | - -When the table is later recreated, permission are in full effect again : +CREATE TABLE countries (id INT, name STRING, iso_code STRING); -```questdb-sql -CREATE TABLE countries (id INT, name STRING, iso_code int, alpha2 STRING); +-- Required after recreation +GRANT SELECT ON countries TO john; +GRANT UPDATE ON countries(iso_code) TO john; ``` -| permission | table_name | column_name | grant_option | origin | -| ---------- | ---------- | ----------- | ------------ | ------ | -| SELECT | countries | | f | G | -| UPDATE | countries | | f | G | - -:::note - -Only the table and/or column name is used when applying permission. The type is -ignored. In the example above `iso_code` was initially of string type, then -recreated as int. - -::: +Dropping a column likewise removes permissions granted specifically on that +column. Adding a column with the same name later does not restore them. The same +drop-and-regrant rule applies to views, materialized views, and live views. ### Owner grants In QuestDB there are no owners of database objects. Instead, there are -[owner grants](/docs/security/rbac/#owner-grants). +[owner grants](/docs/security/rbac/granting-permissions/#owner-grants). An owner grant means: diff --git a/documentation/query/sql/acl/revoke.md b/documentation/query/sql/acl/revoke.md index c302b60142..6a9727c685 100644 --- a/documentation/query/sql/acl/revoke.md +++ b/documentation/query/sql/acl/revoke.md @@ -166,7 +166,7 @@ REVOKE SELECT ON products(id) FROM john; If the user has a database- or table-level permission, then revoking it on a lower level triggers -[permission level re-adjustment](/docs/security/rbac/#permission-level-re-adjustment). +[permission level re-adjustment](/docs/security/rbac/granting-permissions/#permission-level-re-adjustment). Permission is switched to lower level and `materialized`: - database level permission is pushed to table level, so e.g. SELECT will not diff --git a/documentation/query/sql/alter-table-rebase-wal.md b/documentation/query/sql/alter-table-rebase-wal.md index be4ba2dba8..b77649c6d2 100644 --- a/documentation/query/sql/alter-table-rebase-wal.md +++ b/documentation/query/sql/alter-table-rebase-wal.md @@ -66,10 +66,12 @@ an error: ## Permissions -`REBASE WAL` requires database administrator (system admin) privileges rather -than a table-level grant. This reflects that the operation is destructive: it -discards un-applied transactions, changes the table's internal id, and replaces -its on-disk directory. A user who can run the non-destructive +`REBASE WAL` requires the `SYSTEM ADMIN` +[permission](/docs/security/rbac/permissions-reference/#special-permissions) +rather than a table-level grant. +This reflects that the operation is destructive: it discards un-applied +transactions, changes the table's internal id, and replaces its on-disk +directory. A user who can run the non-destructive [`RESUME WAL`](/docs/query/sql/alter-table-resume-wal/) on a table cannot necessarily run `REBASE WAL` on it. diff --git a/documentation/query/sql/alter-table-set-storage-policy.md b/documentation/query/sql/alter-table-set-storage-policy.md index 56b34d84ef..54fc4370c3 100644 --- a/documentation/query/sql/alter-table-set-storage-policy.md +++ b/documentation/query/sql/alter-table-set-storage-policy.md @@ -206,5 +206,5 @@ Stages that are not set are omitted from the output. view listing active policies - [`SHOW CREATE TABLE`](/docs/query/sql/show/#show-create-table) — displays the attached `STORAGE POLICY` clause -- [RBAC permissions](/docs/security/rbac/#permissions) — `SET`, `REMOVE`, +- [RBAC permissions](/docs/security/rbac/permissions-reference/) — `SET`, `REMOVE`, `ENABLE`, and `DISABLE STORAGE POLICY` permissions diff --git a/documentation/query/sql/switch-cold-storage-role.md b/documentation/query/sql/switch-cold-storage-role.md index e1bc3ab586..7bc1a164eb 100644 --- a/documentation/query/sql/switch-cold-storage-role.md +++ b/documentation/query/sql/switch-cold-storage-role.md @@ -59,7 +59,10 @@ A role set this way does **not** survive a restart. Update `cold.storage.role` i ### Permissions -Both statements require database administrator (system admin) privileges, including with `FORCE`. They are not grantable through [RBAC](/docs/security/rbac/) permissions. +Both statements require the `SYSTEM ADMIN` +[permission](/docs/security/rbac/permissions-reference/#special-permissions), +including with `FORCE`. There is no narrower grant for the cold storage role: +granting `SYSTEM ADMIN` also grants every other system function. ## Examples diff --git a/documentation/query/sql/switch-role.md b/documentation/query/sql/switch-role.md index 3c9e3b0028..a13df89422 100644 --- a/documentation/query/sql/switch-role.md +++ b/documentation/query/sql/switch-role.md @@ -112,7 +112,7 @@ Both statements require the `SWITCH ROLE` permission, granted with `GRANT SWITCH ROLE TO entity`. `SYSTEM ADMIN` does not imply it; `DATABASE ADMIN` does. A denied session receives `Access denied for [SWITCH ROLE]`. When access control is disabled, both statements are open to every session. See -[Failover operator](/docs/security/rbac/#failover-operator). +[Failover operator](/docs/security/rbac/common-scenarios/#failover-operator). Before version 4.0.0, both statements required `SYSTEM ADMIN`. @@ -180,7 +180,7 @@ SELECT node_role(); - [Failover and role switch](/docs/high-availability/failover/) for the full procedure, the REST endpoint, and recovery from a refused switch - [`node_role()`](/docs/query/functions/meta/#node_role) for the open role read -- [RBAC](/docs/security/rbac/#failover-operator) for the `SWITCH ROLE` permission +- [RBAC](/docs/security/rbac/common-scenarios/#failover-operator) for the `SWITCH ROLE` permission - [`replication.role`](/docs/configuration/database-replication/#replicationrole) for the boot role - [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) for diff --git a/documentation/security/oidc.mdx b/documentation/security/oidc.mdx deleted file mode 100644 index 4e32f35217..0000000000 --- a/documentation/security/oidc.mdx +++ /dev/null @@ -1,1334 +0,0 @@ ---- -title: OpenID Connect (OIDC) Integration -description: Configure OpenID Connect (OIDC) integration with external Identity Providers for SSO authentication in QuestDB Enterprise Web Console. ---- - -import Screenshot from "@theme/Screenshot"; -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - OpenID Connect (OIDC) enables SSO authentication with external Identity Providers. - - -OpenID Connect (OIDC) integrates with Identity Providers (IdP) external to -QuestDB. - -It is a convenient way to integrate QuestDB into your enterprise environment, -and it provides SSO (Single Sign-On) for the [Web Console](/docs/getting-started/web-console/overview/). - -Microsoft Active Directory and Azure AD, for example, can be turned into an -Identity Provider. - -Specific installation steps depend on the type of the provider. - -## Architecture overview - -Altogether, the architecture appears as such: - - - -We can break it down into core components. - -### Web Console - -QuestDB's interactive UI. Users must authenticate before accessing the database -via the interface. - -The [Web Console](/docs/getting-started/web-console/overview/) uses PKCE (Proof Key for Code Exchange) to -secure the authentication and authorization flow. - -In OAuth2/OIDC terms, the [Web Console](/docs/getting-started/web-console/overview/) is referred to as -the _client_, and it is assigned an identifier: the **Client Id**. - -Each application which integrates via OIDC should be given a different **Client -Id**. - -### OIDC Provider - -Typically consists of a number of modules. - -We are interested in two of them only. - -1. The _Identity Provider_ holds user identities and user information, capable - of authenticating users, and to issue an ID Token which uniquely identify - them. - -2. The _Authorization Server_ grants access to resources, such as a database, in - the form of access tokens. - -The OIDC Provider usually integrates with a number of applications which require -different access to a number of resources. - -These clients communicate with the OIDC Provider via its endpoints. - -It exposes a number of APIs, including the Authorization, Token and User Info -endpoints. - -### QuestDB - -The database, in OAuth2/OIDC terms the _protected resource_ or _resource -server_. - -Only processes requests which contain a valid access token. - -## Authentication and Authorization Flow - -The OAuth2/OIDC standard defines different ways of obtaining access and ID -tokens from the OIDC Provider, referred to as the "_flow_". - -The goal of this flow is to get the user, who is sitting in front of the Web -Console, authenticated. - -Then, it allows QuestDB to determine the user's permissions based on user -information provided by the Identity Providers. - -Specifically, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) uses the -`Authorization Code Flow with PKCE` option. - -It consists of ten steps... - -### 1. Secret generation - -First the [Web Console](/docs/getting-started/web-console/overview/) generates a cryptographically strong -random secret called the _code verifier_. - -The secret is hashed using the _SHA256 algorithm_. The result is the _code -challenge_. - -After PKCE initialization the [Web Console](/docs/getting-started/web-console/overview/) requests an -_authorization code_ from the OIDC Provider. - -It calls the Authorization endpoint with a few parameters, including the: - -- **Client Id** -- requested scopes (the list of scopes are configurable, default is `openid` - only) -- code challenge -- algorithm used to generate the code challenge from the code verifier (SHA256) - -When the Authorization Server receives the request, it checks if the user has -been authenticated already: - -- If the user has a valid session, it can be provided with an authorization code - straight away, so we jump to step 4. - -- If the user does not have a valid session yet, it will be redirected to the - Identity Provider for authentication. - -```bash title="Authorization code request example" -https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.host%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 -``` - -### 2. Prove identity - -Next, the user must prove its identity. - -This could be a username with: - -- a password, -- an OTP -- facial recognition via a mobile app -- or anything else supported by the Identity Provider. - - - -### 3. Scope consent - -After successful authentication, the user provides consent for the requested -scopes. - -The list of scopes are configurable. - -By default the Web Console requests only the `openid` scope which is mandatory -for OIDC. - -No ID Token is issued without it. - -The OIDC provider can be configured to provide the consent automatically, -without presenting the user with an additional screen in the browser. - - - -### 4. Redirection - -Consent is granted! - -The Authorization Server redirects the user back to the -[Web Console](/docs/getting-started/web-console/overview/) with the _authorization code_: - -```bash title="Authorization code response example" -https://questdb.host:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A -``` - -### 5. Credential request - -Now, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) requests the ID and access -tokens from the Token endpoint of the OIDC Provider with the authorization code. - -It includes the Client ID and the PKCE code verifier together with the -authorization code in the request. - -The endpoint then hashes the code verifier using the method specified previously -in step 1. - -The result must match the code challenge, also provided in step 1. - -The matching code challenge proves that the token is requested by the client -which requested the authorization code, and it was not stolen: - -```bash title="Token request example" -POST https://oidc.provider:443/as/token.oauth2 HTTP/1.1 -Content-Type: application/x-www-form-urlencoded -grant_type=authorization_code&code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A&client_id=questdb&&redirect_uri=https%3A%2F%2Fquestdb.host%3A9000&code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF -``` - -### 6. Credentials received - -If the PKCE check is passed, the Web Console receives the ID and access tokens. - -There is a third token in the response too, the refresh token. - -The refresh token is used by the Web Console to refresh the access token before -it expires. - -Without the refresh token mechanism, the user would be forced to re-authenticate -when the access token expires. - -The validity of the tokens are configurable inside the OIDC Provider. - -```json title="Token response example" -{ - "access_token": "gslpJtzmmi6RwaPSx0dYGD4tEkom", - "refresh_token": "FUuAAqMp6LSTKmkUd5uZuodhiE4Kr6M7Eyv.eg83ge", - "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...", - "token_type": "Bearer", - "expires_in": 300 // In seconds, thus 5 minutes -} -``` - -### 7. Database access - -With the tokens, the Web Console can interact with the database. - -The access token is in the header of every request sent to QuestDB. - -> **Worried about exposing the token?** It is rather opaque and does not contain -> user details. - -To carry out permission checks, the database has to know more about the user. - -For this, QuestDB has a User Info Cache. - -If it finds a valid entry with the access token in the cache, steps 8 and 9 are -skipped: - -```bash title="Query request example" -https://questdb.host:9999/exec?query=select%20current_user() -Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom -``` - -### 8. Find user information - -No user information in the cache, or stale information? - -QuestDB uses the access token to request user information from the OIDC -Provider's User Info endpoint. - -This call also serves as token validation. - -If the token is not real or has been expired, the User Info endpoint replies -with an error: - -```bash title="User info request example" -https://oidc.provider:443/idp/userinfo.openid -Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom -``` - -### 9. Receive user information - -If the access token is valid, QuestDB receives the required user information -from the endpoint, then updates its cache. - -The cache improves performance, as QuestDB does not have to turn to the OIDC -Provider on every single request. - -Do note that cache expiry is configurable: - -```json title="User info response example with Active Directory groups" -{ - "sub": "externalUser", - "name": "External User", - "groups": [ - "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", - "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" - ] -} -``` - -### 10. Permission check - -With the help of the user information, QuestDB can carry out -[permission checks](#user-permissions). - -If the permission check is successful, the database will process the request, -and then sends the results back: - -```json title="Query response example" -{ - "query": "select current_user()", - "columns": [ - { - "name": "current_user", - "type": "STRING" - } - ], - "dataset": [["External User"]], - "count": 1, - "timestamp": -1 -} -``` - -## Interactive clients - -Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an -OIDC provider. However, the level of support will vary between these tools. - -Interactive clients usually fall into one of the following categories: - -- Browser-based clients with support for HTTP redirects; this includes the Web - Console or any javascript UI -- Applications running in a browser without support for redirects, such as - Jupyter notebooks -- Non-browser based clients, usually some kind of command line interface (CLI) - or a standalone application, such as Microsoft Access - -### Browser-based clients - -If the tool is browser based and can handle HTTP redirects, it can implement two -possible flows to request an access token.: - -1. [Authorization Code](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth) - flow **(Recommended, more secure)** - -2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) - flow - -The Web Console implements the -[Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a -special version of the Authorization Code flow designed for mobile apps and -single page applications. - -Regardless of which flow is used by the web or mobile application, the requested -access token can be used for authentication and authorization when communicating -with QuestDB as explained in the [above 7th step](#7-database-access). - -### Jupyter notebook - -JupyterHub can integrate with OAuth2 providers using OAuthenticator, as -described in its -[documentation](https://jupyterhub.readthedocs.io/en/stable/explanation/oauth.html). -The OAuthenticator documentation also contains -[examples](https://oauthenticator.readthedocs.io/en/latest/tutorials/provider-specific-setup/index.html) -using different identity providers. - -If Jupyter notebooks are used without JupyterHub, one option for OAuth2 -integration is to use the -Resource Owner Password Credentials (ROPC) flow. -It is likely that enabling this flow in your OAuth2 provider will require -additional setup. - -:::caution - -The Resource Owner Password Credentials flow is legacy, and should be used as a last resort. - -::: - -We can use the code below to acquire an access token in our notebook: - -```python -from urllib import request, parse -import json - -url = "https://oidc.provider:443/as/token.oauth2" -data = parse.urlencode( { - "grant_type": "password", - "username": "testuser", - "password": "testpwd", - "scope": "openid", - "client_id": "testclient" -} ).encode() -req = request.Request(url=url, data=data) -req.add_header("Content-Type", "application/x-www-form-urlencoded") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - access_token = resp["access_token"] -``` - -This token can be used to authenticate with QuestDB: - -```python -query = parse.urlencode({ - "query": "select current_user()" -}) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") -req.add_header("Authorization", f"Bearer {access_token}") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - print(resp) -``` - -#### Externalizing credentials - -The above example saves the user's credentials into the notebook, potentially -exposing them to others. One way to improve this is to use environment variables -or files to externalize the username and password. - -Here is an example using the `dotenv` library. - -First we need to create a file named `.env` with the settings: - -```python -username=testuser -password=testpwd -``` - -Then load it in our notebook, and use it to request tokens: - -```python -from dotenv import load_dotenv -import os -from urllib import request, parse -import json - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -url = "https://oidc.provider:443/as/token.oauth2" -data = parse.urlencode( { - "grant_type": "password", - "username": user, - "password": pwd, - "scope": "openid", - "client_id": "testclient" -} ).encode() -req = request.Request(url=url, data=data) -req.add_header("Content-Type", "application/x-www-form-urlencoded") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - access_token = resp["access_token"] -``` - -#### Enable ROPC - -The Resource Owner Password Credentials flow can be enabled in QuestDB within -`server.conf`: - -``` -acl.oidc.ropc.flow.enabled = true -``` - -> Note that the flow also has to be configured in the OAuth2/OIDC provider! - -Now we can use Basic Authentication to simplify our code. We send the -credentials to QuestDB, and the database will validate the credentials against -the OAuth2 provider. - -```python -from dotenv import load_dotenv -import os -from urllib import request -import base64 - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -query = parse.urlencode({ - "query": "select current_user()" -}) -req = request.Request(f"http://localhost:9000/api/v1/sql/execute?{query}") -b64credentials = base64.standard_b64encode(f"{user}:{pwd}".encode()).decode() -req.add_header("Authorization", f"Basic {b64credentials}") -with request.urlopen(req) as f: - body = f.read().decode(f.headers.get_content_charset()) - resp = json.loads(body) - print(resp) -``` - -We can also use a postgres client to connect to the database: - -:::note - -QuestDB never persists the user's credentials. - -::: - -```python -import psycopg as pg -from dotenv import load_dotenv -import os - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -conn_str = f"user={user} password={pwd} host=localhost port=8812 dbname=qdb" -with pg.connect(conn_str, autocommit=True) as connection: - with connection.cursor() as cur: - cur.execute("select current_user()") - records = cur.fetchall() - for row in records: - print(row) -``` - -### CLI, standalone applications - -When using CLI tools, such as `psql`, or standalone applications like Microsoft -Access, the best option may be the Resource Owner Password Credentials flow. - -The user logs in with their SSO credentials, and the server validates the -details with the OAuth2 provider: - -```shell -% psql -h localhost -p 8812 -U testuser -Password for user testuser: -psql (14.2, server 11.3) -Type "help" for help. - -testldap=> -testldap=> -``` - -## Non-interactive clients - -Non-interactive clients are usually jobs or standalone applications, such as a -client for ingesting data. It is practical to manage their credentials via an -OAuth2 provider too. - -As seen in the Jupyter notebook examples, the clients can request a token -themselves and then use it to authorise data ingestion: - -```python -import json -import os -import requests -import pandas as pd -from dotenv import load_dotenv -from questdb.ingress import Sender - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -token_endpoint = "https://oidc.provider:443/as/token.oauth2" -response = requests.post(token_endpoint, - data={"grant_type": "password", - "client_id": "testclient", - "username": user, - "password": pwd, - "scope": "openid"}, - headers={"Content-Type": "application/x-www-form-urlencoded"}) - -response_body = response.content.decode("utf-8") -tokens = json.loads(response_body) -access_token = tokens["access_token"] - -conf = f"http::addr=localhost:9000;token={access_token};" -with Sender.from_conf(conf) as sender: - df = pd.read_csv("data.csv") - df["ts"] = pd.to_datetime(df["ts"]) - sender.dataframe(df, table_name="foo", at="ts") -``` - -Alternatively, a user may rely on QuestDB to authenticate them via the OAuth2 -provider when the Resource Owner Password Credentials flow is enabled on the -server side: - -```python -import os -import pandas as pd -from dotenv import load_dotenv -from questdb.ingress import Sender - -load_dotenv() -user = os.environ.get("username") -pwd = os.environ.get("password") - -conf = f"http::addr=localhost:9000;username={user};password={pwd};" -with Sender.from_conf(conf) as sender: - df = pd.read_csv("data.csv") - df["ts"] = pd.to_datetime(df["ts"]) - sender.dataframe(df, table_name="foo", at="ts") -``` - -## OIDC for the PGWire endpoint - -If the -Resource Owner Password Credentials (ROPC) flow is not an option, we can still authenticate via OIDC on the PGWire endpoint. -However, in this case the client's responsibility to source the token required for authentication. -This method works wherever a Postgres client library is available, including jupyter notebooks. - -Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. - -The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option: - -```python -import psycopg as pg - -token = "token_requested_from_the_oauth2_provider" - -conn_str = f"user=_sso password={token} host=localhost port=8812 dbname=qdb" -with pg.connect(conn_str, autocommit=True) as connection: - with connection.cursor() as cur: - cur.execute('select current_user()') - records = cur.fetchall() - for row in records: - print(row) -``` - -## User permissions - -QuestDB requires additional user information to be able to construct the user's -access list. - -As a reminder, the access list is the list of permissions that determines what -the user can and cannot do. - -QuestDB itself does not store external users, nor their passwords or any other -authentication related detail. - -External users and their authentication methods are managed by the Identity -Provider. - -Since external users are not managed by QuestDB, permissions cannot be granted -to them directly. - -Instead, the database expects a list of groups, called the _groups claim_ to be -present in the user information. - -These external group names are mapped to QuestDB's own groups. - -The access list of the external user consists of the permissions granted to -those groups: - - - -### Mapping user permissions - -The mappings between external and QuestDB groups are managed with the following -SQL commands: - -```questdb-sql title="Create a group which is mapped to an Active Directory group" -CREATE GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -```questdb-sql title="Map an Active Directory group to an already existing QuestDB group" -ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -```questdb-sql title="Remove an Active Directory mapping without deleting the QuestDB group" -ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; -``` - -QuestDB works the list of external groups out from the User Info response -message. - -If we take the example used earlier, we will see that the message contains a -claim called `groups`. This name is configurable in QuestDB. - -If the groups claim is missing or it is an empty list, the user cannot access -the database. - -Although the user is authenticated, they have no permissions at all. - -The user has to have at least the `HTTP` permission to be able to successfully -login via the [Web Console](/docs/getting-started/web-console/overview/). - -```json title="User info response example with Active Directory groups" -{ - "sub": "externalUser", - "name": "External User", - "groups": [ - "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", - "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" - ] -} -``` - -Any change made to the user's group membership in the Identity Provider, QuestDB -will adjust the user's access list. - -:::note - -There may be a slight delay due to the User Info Cache. - -QuestDB will use the cached information until it becomes stale, and gets -updated. - -::: - -The same stands for changes made to the user's status within the Identity -Provider. - -For example, a disabled user will not be kicked out of QuestDB immediately. - -The `acl.oidc.cache.ttl` config option drives how often user information should -be synchronized with the Identity Providers. - -It should be set accordingly to your organization's policies. - -Other parts of the user information, such as the `sub` and the `name` also used -by QuestDB. - -They could be displayed in the [Web Console](/docs/getting-started/web-console/overview/), or appear in -the logs, for example. - -## Configuration options - -For all OIDC-related configuration options of QuestDB, see -[Configuration](/docs/configuration/oidc/). - -
- -## Active Directory - -The following sections are guides for setting up single sign-on (SSO) with various OAuth2 providers. - -### PingFederate - -This document helps set up SSO authentication for the Web Console in -[PingFederate](https://www.pingidentity.com/en/platform/capabilities/authentication-authority/pingfederate.html). - -It is assumed that the Azure Active Directory serves as the Identity Provider -(IdP). - -#### Set up PingFederate client - -First thing first, let's pick a name for the client! - - - -The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). - -As a result, it cannot store safely a client secret. - -Instead it can use PKCE (Proof Key for Code Exchange) to secure the flow. - -As shown above, leave the client authentication disabled. - -We also have to white list the URL of the [Web Console](/docs/getting-started/web-console/overview/) as a redirection URL: - - - -We can instruct PingFederate to automatically authorize the scopes requested by -the [Web Console](/docs/getting-started/web-console/overview/). - -The user will not be presented the extra window asking for consent after -authentication: - - - -The [Web Console](/docs/getting-started/web-console/overview/) uses the -[Authorization Code Flow](/docs/security/oidc/#authentication-and-authorization-flow), -and refreshes tokens automatically. - -Next, enable the grant types required for this flow: - - - -We've selected: - -- Authorization Code -- Refresh Token -- Access Token Validation (Client is a Resource Server) - -After that, select the token manager for the client. - -The token manager is responsible for issuing access tokens. - -All token related settings should be configured in the token manager. - - - -Finally, enable PKCE - as shown above - and save the settings. - -#### Access Token Manager settings - -QuestDB does not require any special setup regarding the access token. - -We recommend that you do not to use shorter tokens than the default 28 -characters. - -As the QuestDB [Web Console](/docs/getting-started/web-console/overview/) refreshes the token automatically, there is no need -for long-lived tokens: - - - -We've selected: - -- Token length: 28 -- Token lifetime: 5 -- Lifetime extension policy: None -- Maximum token lifetime: Null -- Lifetime extension threshold percentage: 30 - -For the next step, we tune the Authorization Server. - -#### Authorization Server settings - -These settings relate to the authorization code, refresh token and CORS. - - - -In this section, we've entered: - -- Authorization code timeout: 60 -- Authorization code entropy: 30 -- Client secret retention period: 0 - -Next, ensure the `ROLL REFRESH TOKEN VALUES` option is selected: - - - -It is also important to whitelist the [Web Console](/docs/getting-started/web-console/overview/)'s URL on the CORS list: - - - -#### Set up a Microsoft Entra ID Data Source - -PingFederate needs a Data Source setup. - -This is a secure LDAP connection to Microsoft Entra ID, formerly known as Azure -Active Directory. - -The data source needs a: - -- name -- hostname -- port -- username and password for the LDAP connection - - - -We have given it the name EntraDS and it will be applied later. - -#### Set up a Password Credential Validator - -Now that PingFederate has an LDAP connection, we can use it for authentication. - -First, create a Password Credential Validator: - - - -We've entered: - -- Instance name: EntraPCV -- Instance ID: EntraPCV -- Selected: LDAP Username Password Credential Validator -- Parent instance: None - -Furthermore, we now declare our previously created data source (`EntraDS`): - - - -This links our data store (`EntraDS`) to our PCV (`EntraPCV`). - -#### Set up an Identity Provider - -We can use our PCV once we set up an Identity Provider. - -The IdP will be used to authenticate users against Active Directory using the -LDAP connection. - -We do this in the Type subsection: - - -Next, in the IdP Adapter section... - -Click: Add a new row to Credential Validators. - -Select the PCV (`EntraPCV`) we created. - -Optionally alter number of retries: - - - -#### Add groups to OIDC policy management - -QuestDB now needs to know about the user's AD group memberships to find their -permissions. - -Groups are passed to QuestDB inside the User Info object in a custom claim. - -This has to be added in the OpenID Connect Policy Management. - -The field is Multi-Valued, because it is a list of group names. - -Under the Attribute Contract subsection, see: - - - -Next, click to the Attribute Scopes subsection. - -Ensure `groups` is among the `openid` attributes: - - - -Onwards to the Attribute Sources & User Lookup Section. - -From this view, you can add local data stores. - -Note item `test` of type of LDAP: - - - -We created it via the following choices in Add Attribute Source: - - - -Note where we specified the Data Store (`EntraDS`). - -This is also where the directory search parameters are defined. - -Back at the Attribute Sources & User Lookup Section section, note we have set -`email`. - -The source is `LDAP (test)`, while the value is `usePrincipalName`: - - - -And finally! - -In the same Attribute Sources & User Lookup Section... - -Find `groups`. - -Note the definition of Source (`LDAP (test)`) that bridges our various parts. - -The value is `memberOf`. - - - -#### Enable Resource Owner Password Credentials (ROPC) flow - -As described in the -[OIDC operations document](/docs/security/oidc/#enable-ropc) -tools - such as `psql` - can be integrated with the OIDC provider using the ROPC flow. - -When setting this flow up, enable the Resource Owner Password Credentials flow in the -client settings. - -Next, create a Resource Owner Credentials Grant Mapping to map values obtained from -the Password Credential Validator (PCV) into the persistent grants. - -When setting this up, select the previously created LDAP Data Source and IdP Adapter, which links -to the existing PCV. - -Then select the `username` attribute of the PCV as `USER_KEY`. - -#### Confirm QuestDB mappings and login - -QuestDB requires a mapping, as laid out in the -[OIDC operations document](/docs/security/oidc/#mapping-user-permissions). - -If a given user has the HTTP permission, they will be able to now login via the -[Web Console](/docs/getting-started/web-console/overview/). - -To test, head to `http://localhost:9000` and login. - -If all has been wired up well, then login will succeed. - -
- -### Microsoft EntraId - -This document sets up SSO authentication for the [QuestDB Web Console](/docs/getting-started/web-console/overview/) in -[Microsoft EntraID](https://www.microsoft.com/en-gb/security/business/identity-access/microsoft-entra-id), formerly known as Azure AD. - - -:::tip - -To enlarge the images, click or tap them. - -::: - -#### Set up the client application in Entra ID - -First thing first, let's pick a name for the client! - -Then head to _Microsoft Entra Admin Center_, and register the application -under _Identity - App registrations - New registration_. - - - -The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). - -As a result, it cannot store safely a client secret. - -Instead, it can use PKCE (Proof Key for Code Exchange) to secure the flow. - -When registering the application, select the SPA platform. - -We also have to specify the URL of the [Web Console](/docs/getting-started/web-console/overview/) as Redirect URI. - - - -After clicking _Register_, we have created a client application with the -name _QuestDB_. - -Each application is assigned a unique id (known as Client ID in the -OAuth2 - OIDC standard). The client will identify itself with this id -when sending requests to Entra ID. - - - -We find the platform configurations under _Authentication_. This is the place where -the previously set redirect URI can be viewed and modified. We can also specify -additional redirect URIs, if necessary. - -The redirect URIs of the application are automatically eligible for the -_Authorization Code Flow with PKCE_, which is a special version of the OAuth2 standard's -Authorization Code Flow. It is specifically designed for applications where a client -secret (e.g. a password) could not be kept safely. As single page applications run in -the browser, they fall into this category. - -The redirect URIs are also added to the _CORS_ (Cross-Origin Resource Sharing) policy -of EntraID. CORS is a mechanism to allow a web page, such as the Web Console, to access -resources from a different domain than the one that served the page. In this context -this means that we let the Web Console to access Entra ID, while its origin is the -HTTP endpoint of QuestDB. - - - -If we scroll down to the bottom of this page, we can also find a section where we -can enable the _Resource Owner Password Credential Flow_. - -This OAuth2 flow is legacy, and should be enabled only if there is a requirement -of connecting to QuestDB using SSO (Single Sign-On) via clients not supporting -redirect based web flows. -This could mean a Postgres client without OAuth2 integration, such as _psql_, or -a standalone in-house client application, or could be just a jupyter notebook. - -The main issue with this flow is that the client application has to be trusted -with the user's login details. The user's credentials are passed to the -application, in this case to QuestDB, and the client application uses these -credentials to authenticate the user by forwarding them to the identity provider, -in this case to Entra ID. - -It is guaranteed that QuestDB does not store the user's credentials in any way. -They are not persisted into the database, not even in encrypted form. -The login details are treated as passthrough information. Only exception is -that server logs can contain the username, logged for audit purposes. - - - -Our next stop is the _Token configuration_, where the OAuth2/OIDC access and ID -tokens can be customized. - -Note that users can be authenticated without customized tokens, but authorization -would prove to be challenging. The user's security groups are not included -in the tokens by default. - -QuestDB can be configured to request the user's groups from the UserInfo -endpoint of the OAuth2 server, but Entra ID cannot be configured to provide -this information via the UserInfo endpoint. -Therefore, we choose to customize the tokens, QuestDB will decode and -validate the ID token, and take the group information from there. - -QuestDB authorization relies on receiving the group memberships of the user. -Entra ID groups should be mapped to QuestDB groups, and permissions can be -granted to the QuestDB groups. Detailed information about group mappings can -be found in the [OIDC integration](/docs/security/oidc/#user-permissions) -documentation. - - - -The customized tokens contain user information which cannot be accessed -without permission. User information is provided by Microsoft Graph, so -the client application needs specific permissions to access -Microsoft Graph APIs. - -These permissions can be configured under _API permissions_. It is important -to note that we will be setting _Delegated_ permissions here, meaning we -are not granting actual permissions to access user data. Instead, each user -logging into QuestDB will have to consent to accessing their user profile. - - - -By default, the _User.Read_ permission is added to the list, but what we -really need is: - - openid: to be able to issue ID tokens - - profile: to access user information - - offline_access: to be able to issue refresh tokens - -By clicking on _Microsoft Graph_ we can select and add these permissions. - - - -The _User.Read_ permission is not needed. It can be removed by clicking -on the `...` at the end of the row, and selecting _Remove permission_ from -the popup menu. - - - -With this we have finished setting up the QuestDB client application -in Entra ID, and now we can wire QuestDB and Entra ID together by -adding OIDC configuration to QuestDB. - -#### QuestDB configuration - -The below should be set in QuestDB's `server.conf`: - -```shell -# enable OIDC -acl.oidc.enabled=true - -# the claim contains the username or user id -acl.oidc.sub.claim=name - -# the claim contains the user's group memberships -acl.oidc.groups.claim=groups - -# groups are encoded in the token -acl.oidc.groups.encoded.in.token=true - -# OIDC configuration endpoint of Entra ID -acl.oidc.configuration.url=https://login.microsoftonline.com/12345678-1234-1234-1234-123456789abc/v2.0/.well-known/openid-configuration - -# application ID taken from Entra ID -acl.oidc.client.id=8de84b90-1ea5-4e41-9e84-dba860aa01a6 - -# redirect URI, QuestDB's HTTP endpoint -acl.oidc.redirect.uri=http://localhost:9000 - -# OAuth scopes the user has to consent to -acl.oidc.scope=openid profile offline_access - -# enable ROPC flow -# optional, required only if ROPC is enabled in Entra ID -acl.oidc.ropc.flow.enabled=true -``` - -The application ID and the OIDC configuration endpoint's URL can be found -in the Overview of the application in Entra ID. - -The application ID is displayed right under the application's name, the -OIDC configuration endpoint is displayed on the panel which opens up when -the _Endpoints_ button is clicked. - - - -#### Map groups and grant permissions - -Now we can start QuestDB, and login with the built-in admin to create -group mappings. - -As mentioned earlier, authorization works by mapping Entra ID groups -to QuestDB groups. When the user logs in, QuestDB decodes Entra ID -group memberships from the token, then finds the QuestDB groups -mapped to them, and the user gets the permissions based on the -mapped groups. - -```questdb-sql title="Create a group which is mapped to an Entra ID group" -CREATE GROUP extUsers WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; -``` -The above command maps the Entra ID group identified by object -id `87654321-1234-1234-1234-123456789abc` to a QuestDB group called `extUsers`. - -We should grant the necessary QuestDB endpoint permissions first -to make sure users can access the Web Console, Postgres and ILP -interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/#endpoint-permissions). - -```questdb-sql title="Grant endpoint permissions" -GRANT HTTP, PGWIRE TO groupName; -``` - -Now we can grant the rest of the permissions as required. We can -grant access to tables, for example. - -```questdb-sql title="Grant database permissions" -GRANT SELECT ON table1, table2 to groupName; -``` - -#### Confirm group mappings and login - -To test, head to the Web Console and login. - -If all has been wired up well, then login will succeed, and the user -will have the access granted to them. - -
diff --git a/documentation/security/oidc/client-discovery.mdx b/documentation/security/oidc/client-discovery.mdx new file mode 100644 index 0000000000..bc2a74b77c --- /dev/null +++ b/documentation/security/oidc/client-discovery.mdx @@ -0,0 +1,162 @@ +--- +title: OIDC client discovery and tokens +sidebar_label: Client discovery and tokens +description: How a QuestDB client reads the OIDC settings endpoint to discover the provider's endpoints, client ID and scope, and which of the two tokens to send. +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + OIDC client discovery and bearer-token authentication are available in + QuestDB Enterprise. + + +A client needs two things from QuestDB before it can authenticate a user +against your Identity Provider: where the provider's endpoints are, and which +of the two tokens the provider issues QuestDB expects in the +`Authorization: Bearer` header. Both come from the unauthenticated settings +endpoint, so neither has to be hard coded. + +## Settings endpoint + +QuestDB publishes the parts of its OIDC configuration that a client needs in +order to start an authentication flow. Any client can read them from the +settings endpoint, which requires no authentication: + +```bash title="Settings request example" +curl https://questdb.example.com:9000/settings +``` + +The OIDC-related entries of the response look like this, alongside other server +settings: + +```json title="Settings response example" +{ + "config": { + "acl.enabled": true, + "acl.basic.auth.realm.enabled": false, + "acl.oidc.enabled": true, + "acl.oidc.pkce.required": true, + "acl.oidc.state.required": false, + "acl.oidc.groups.encoded.in.token": false, + "acl.oidc.client.id": "questdb", + "acl.oidc.redirect.uri": "https://questdb.example.com:9000", + "acl.oidc.scope": "openid", + "acl.oidc.authorization.endpoint": "https://oidc.provider:443/as/authorization.oauth2", + "acl.oidc.token.endpoint": "https://oidc.provider:443/as/token.oauth2", + "acl.oidc.device.authorization.endpoint": "https://oidc.provider:443/as/device_authz.oauth2" + }, + "preferences.version": 0, + "preferences": {} +} +``` + +Each key is the name of the +[configuration option](/docs/configuration/oidc/) it carries. The +[Web Console](/docs/getting-started/web-console/overview/) reads them to build +its authorization request, and any other client can do the same instead of +hard coding the provider's details. + +The published `acl.oidc.client.id` and `acl.oidc.scope` belong to QuestDB's Web +Console registration. Reusing them keeps an evaluation client short, but a +production application should have its own provider registration, client ID, +redirect URIs, scopes, and policies. It can still discover the provider +endpoints and `acl.oidc.groups.encoded.in.token` here while overriding the +client ID and scope. + +| Key | Type | Present | +| --- | --- | --- | +| `acl.enabled` | boolean | always | +| `acl.basic.auth.realm.enabled` | boolean | always | +| `acl.oidc.enabled` | boolean | always | +| `acl.oidc.pkce.required` | boolean | always | +| `acl.oidc.state.required` | boolean | always | +| `acl.oidc.groups.encoded.in.token` | boolean | always | +| `acl.oidc.client.id` | string or `null` | always | +| `acl.oidc.redirect.uri` | string or `null` | always | +| `acl.oidc.scope` | string | always | +| `acl.oidc.authorization.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.token.endpoint` | string | when the endpoint is resolved | +| `acl.oidc.device.authorization.endpoint` | string | when the endpoint is configured or discovered | + +The endpoints are absolute URLs, resolved either from `acl.oidc.host` and the +[endpoint settings](/docs/configuration/oidc/#endpoints), or from the provider's +configuration document when `acl.oidc.configuration.url` is set. +[`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint) +is present when a device authorization endpoint has been configured, or when the +provider advertises one. QuestDB publishes it without using it, for clients +which implement the [OIDC device flow](/docs/security/oidc/device-flow/) +themselves. + +`acl.oidc.redirect.uri` is `null` unless +[`acl.oidc.redirect.uri`](/docs/configuration/oidc/#acloidcredirecturi) is set. +A client should then fall back to its own location, as the Web Console does. + +QuestDB advertises `acl.oidc.pkce.required` and `acl.oidc.state.required`, and +enforces neither. The client generates the code verifier and the `state` value; +the provider checks the verifier, and the client checks the `state` value it +gets back. + +:::note + +Read `acl.oidc.enabled` before any of the other keys. The `acl.oidc.*` entries +are published whether or not OIDC is enabled. With OIDC disabled and +`acl.oidc.host` in use, the endpoint URLs are built from the defaults rather +than from a real provider; with `acl.oidc.configuration.url` set they are not +resolved at all, and the endpoint keys are absent from the response. + +OIDC also requires [`acl.enabled`](/docs/configuration/iam/#aclenabled) to be +`true`, which the same response carries. When access control is disabled, +`acl.oidc.enabled` still reports the configured value while no OIDC +authentication takes place, so read both keys. + +::: + +The path of the endpoint is set by +[`http.context.settings`](/docs/configuration/http-server/#httpcontextsettings). +Setting it adds paths rather than moving the endpoint, so the default path keeps +working too. That default is `/settings` only while +[`http.context.web.console`](/docs/configuration/http-server/#httpcontextwebconsole) +is at its default, as the settings path follows the Web Console context path. +A client which cannot assume both are at their defaults should make the path +configurable. + +The official device-flow clients append `/settings` to the QuestDB base URL +passed to `from_questdb`, `fromQuestDB`, or the corresponding C/C++ builder. If +`http.context.web.console=/console`, pass the context in that base URL: + +```text +https://questdb.example.com:9000/console +``` + +The clients then request `https://questdb.example.com:9000/console/settings`. +With a custom `http.context.settings`, QuestDB still preserves the path beneath +the Web Console context, so the same base URL works. A client which fetches the +settings document itself must use one of the exact configured paths. + +## Which token to send + +`acl.oidc.groups.encoded.in.token` tells the client which of the tokens returned +by the provider belongs in the `Authorization: Bearer` header: + +| Value | Token to send | How QuestDB validates it | +| --- | --- | --- | +| `false`, the default | the access token | by calling the user info endpoint | +| `true` | the ID token | locally, by checking its signature, audience, and claims | + +Sending the wrong one fails authentication with `401 Unauthorized` and the +reason in the server log. With +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +enabled the token has to be a JWT, because QuestDB reads the group memberships +straight out of its payload. The +[Web Console](/docs/getting-started/web-console/overview/) picks the token this +way, and so should any other client. + +## See also + +- [Client integration patterns](/docs/security/oidc/client-integration/) - use + discovery and bearer tokens from browser, notebook, CLI, and PGWire clients +- [OIDC device flow](/docs/security/oidc/device-flow/) - discovery in the + official Java, Python, Rust, C, and C++ clients +- [OIDC settings reference](/docs/configuration/oidc/) - configure the values + published to clients diff --git a/documentation/security/oidc/client-integration.mdx b/documentation/security/oidc/client-integration.mdx new file mode 100644 index 0000000000..d2ef171ac5 --- /dev/null +++ b/documentation/security/oidc/client-integration.mdx @@ -0,0 +1,340 @@ +--- +title: OIDC client integration patterns +sidebar_label: Client integration patterns +description: How browser clients, Jupyter notebooks, CLI tools, unattended jobs, and PGWire clients authenticate against QuestDB Enterprise with OIDC. +--- + +How a client authenticates depends on what it can do: follow a browser +redirect, prompt a person, or neither. This page covers each case, from a +browser-based UI running the Authorization Code Flow to an unattended +ingestion job holding a token it acquired out of band. + +## Interactive clients + +Any interactive client - a UI, Jupyter notebook, CLI - can integrate with an +OIDC provider. However, the level of support will vary between these tools. + +Interactive clients usually fall into one of the following categories: + +- Browser-based clients with support for HTTP redirects; this includes the Web + Console or any javascript UI +- Applications running in a browser without support for redirects, such as + Jupyter notebooks +- Non-browser based clients, usually some kind of command line interface (CLI) + or a standalone application, such as Microsoft Access + +### Browser-based clients + +If the tool is browser based and can handle HTTP redirects, it can implement two +possible flows to request an access token.: + +1. [Authorization Code](https://openid.net/specs/openid-connect-core-1_0.html#CodeFlowAuth) + flow **(Recommended, more secure)** + +2. [Implicit](https://openid.net/specs/openid-connect-core-1_0.html#ImplicitFlowAuth) + flow **(deprecated)**. It cannot use PKCE, so it does not satisfy + [`acl.oidc.pkce.required`](/docs/configuration/oidc/#acloidcpkcerequired), + which is `true` by default. + +The client can read the authorization and token endpoints and QuestDB's token +selection mode from the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint) +instead of hard coding them. The client ID and scope published there belong to +the Web Console registration. Reuse them only for evaluation; register a +production application separately and use its own client ID, redirect URI, +scopes, and provider policies. + +The Web Console implements the +[Authorization Code Flow with PKCE](https://oauth.net/2/pkce), which is a +special version of the Authorization Code flow designed for mobile apps and +single page applications. + +Regardless of which flow is used by the web or mobile application, the requested +token can be used for authentication and authorization when communicating with +QuestDB. Select the access or ID token as described in +[Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +### Jupyter notebook + +Use the official Python client's +[OIDC device flow](/docs/security/oidc/device-flow/). It works when the browser +and the notebook kernel are on different machines, and does not put the user's +password in the notebook. + +Note that JupyterHub's OAuth integration (OAuthenticator) is a separate concern: +it decides who may log in to JupyterHub, and gives the notebook no token to send +to QuestDB. + +For tooling without device-flow support, a last-resort option is the +Resource Owner Password Credentials (ROPC) flow. Enabling it usually +requires additional provider configuration. + +:::caution + +The Resource Owner Password Credentials flow is legacy, and should be used as a last resort. + +::: + +The ROPC snippets below assume +`acl.oidc.groups.encoded.in.token=false` and therefore send the access token. If +the setting is `true`, the client must send the ID token instead, and the +provider must issue one for the password grant. The official device-flow client +selects the correct token automatically. + +As a fallback, the code below acquires an access token with ROPC: + +```python +from urllib import request, parse +import json + +url = "https://oidc.provider:443/as/token.oauth2" +data = parse.urlencode( { + "grant_type": "password", + "username": "testuser", + "password": "testpwd", + "scope": "openid", + "client_id": "testclient" +} ).encode() +req = request.Request(url=url, data=data) +req.add_header("Content-Type", "application/x-www-form-urlencoded") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + access_token = resp["access_token"] +``` + +This token can be used to authenticate with QuestDB: + +```python +query = parse.urlencode({ + "query": "select current_user()" +}) +req = request.Request(f"https://questdb.example.com:9000/api/v1/sql/execute?{query}") +req.add_header("Authorization", f"Bearer {access_token}") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + print(resp) +``` + +#### Externalizing credentials + +The above example saves the user's credentials into the notebook, potentially +exposing them to others. One way to improve this is to use environment variables +or files to externalize the username and password. + +Here is an example using the `dotenv` library. + +First we need to create a file named `.env` with the settings: + +```ini title=".env" +username=testuser +password=testpwd +``` + +Then load it in our notebook, and use it to request tokens: + +```python +from dotenv import load_dotenv +import os +from urllib import request, parse +import json + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +url = "https://oidc.provider:443/as/token.oauth2" +data = parse.urlencode( { + "grant_type": "password", + "username": user, + "password": pwd, + "scope": "openid", + "client_id": "testclient" +} ).encode() +req = request.Request(url=url, data=data) +req.add_header("Content-Type", "application/x-www-form-urlencoded") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + access_token = resp["access_token"] +``` + +#### Enable ROPC + +The Resource Owner Password Credentials flow can be enabled in QuestDB within +`server.conf`: + +```ini title="server.conf" +acl.oidc.ropc.flow.enabled=true +``` + +> Note that the flow also has to be configured in the OAuth2/OIDC provider! + +Now we can use Basic Authentication to simplify our code. We send the +credentials to QuestDB, and the database will validate the credentials against +the OAuth2 provider. + +```python +from dotenv import load_dotenv +import os +from urllib import parse, request +import base64 +import json + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +query = parse.urlencode({ + "query": "select current_user()" +}) +req = request.Request(f"https://questdb.example.com:9000/api/v1/sql/execute?{query}") +b64credentials = base64.standard_b64encode(f"{user}:{pwd}".encode()).decode() +req.add_header("Authorization", f"Basic {b64credentials}") +with request.urlopen(req) as f: + body = f.read().decode(f.headers.get_content_charset()) + resp = json.loads(body) + print(resp) +``` + +We can also use a postgres client to connect to the database: + +:::note + +QuestDB never persists the user's credentials. + +::: + +```python +import psycopg as pg +from dotenv import load_dotenv +import os + +load_dotenv() +user = os.environ.get("username") +pwd = os.environ.get("password") + +conn_str = f"user={user} password={pwd} host=localhost port=8812 dbname=qdb" +with pg.connect(conn_str, autocommit=True) as connection: + with connection.cursor() as cur: + cur.execute("select current_user()") + records = cur.fetchall() + for row in records: + print(row) +``` + +### CLI, standalone applications + +For a CLI or standalone application built with an official Java, Python, Rust, +C, or C++ client, use the +[OIDC device flow](/docs/security/oidc/device-flow/). +Tools such as `psql` and Microsoft Access cannot run the flow themselves; for +those tools, acquire a token separately and use +[PGWire token authentication](#oidc-for-the-pgwire-endpoint), or enable the +legacy Resource Owner Password Credentials flow. + +The user logs in with their SSO credentials, and the server validates the +details with the OAuth2 provider: + +```shell +% psql -h localhost -p 8812 -U testuser +Password for user testuser: +psql (14.2, server 11.3) +Type "help" for help. + +testldap=> +testldap=> +``` + +## Non-interactive clients + +An unattended job cannot complete device flow because that flow always requires +a person. Prefer a dedicated QuestDB service account with a REST token. This +keeps the job separate from human identities and gives it an explicit, minimal +permission set: + +```questdb-sql title="Create credentials for an ingestion job" +CREATE SERVICE ACCOUNT ingest_job; +GRANT HTTP TO ingest_job; +GRANT INSERT ON trades TO ingest_job; +ALTER SERVICE ACCOUNT ingest_job CREATE TOKEN TYPE REST WITH TTL '30d' REFRESH; +``` + +The final statement displays the token once. Save it in a secrets manager and +pass it to the client over TLS, for example through a protected environment +variable: + +```python title="Use the service-account token" +import os +import questdb + +token = os.environ["QUESTDB_TOKEN"] +conf = f"wss::addr=questdb.example.com:9000;token={token};" + +with questdb.connect(conf) as db: + with db.query("SELECT current_user()") as result: + print(result.to_pandas()) +``` + +The `REFRESH` modifier extends the REST token's expiry after successful +authentication to the primary. Review its replication behavior and revocation +commands in the +[ALTER SERVICE ACCOUNT reference](/docs/query/sql/acl/alter-service-account/#add-rest-api-token). + +If organizational policy requires provider-managed machine identities, give +the job its own confidential application or service principal and use a +machine-to-machine grant supported by that provider, commonly the OAuth 2.0 +client credentials grant. Do not use QuestDB's published Web Console client ID +or a human's username and password. The token must expose the subject and group +information required by QuestDB's configured +[validation mode](/docs/security/oidc/client-discovery/#which-token-to-send); +there is no provider-independent client-credentials setup in QuestDB. Many +providers do not issue ID tokens for client credentials, so verify this +provider-specific path before choosing encoded-token mode. + +ROPC authenticates a human by handing their password to the client. It is a +legacy fallback, not a machine-identity mechanism. Use it only for tooling that +cannot accept a separately acquired token, as described in +[Enable ROPC](#enable-ropc). + +## OIDC for the PGWire endpoint + +Clients such as `psql` that cannot run the device flow can still authenticate +via OIDC on the PGWire endpoint by obtaining an OAuth2 token separately and +sending it as the password. In this case, it is the client's responsibility to +source the token required for authentication. This method works wherever a +PostgreSQL client library is available, including Jupyter notebooks. + +Prefer this approach to the legacy Resource Owner Password Credentials (ROPC) +flow. If sourcing a token separately is not practical and the provider supports +ROPC, [enable QuestDB to run that flow](#enable-ropc) as the fallback. + +Token authentication for the PGWire endpoint should be enabled by adding the `acl.oidc.pg.token.as.password.enabled=true` setting to the server configuration. + +The token should be sent in the password field, while the username field should contain the string `_sso`, or left empty if that is an option. Send the token +selected by [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access token by +default, or the ID token when group memberships are encoded in it. + +```python +import psycopg as pg + +token = "token_requested_from_the_oauth2_provider" + +conn_str = f"user=_sso password={token} host=localhost port=8812 dbname=qdb" +with pg.connect(conn_str, autocommit=True) as connection: + with connection.cursor() as cur: + cur.execute('select current_user()') + records = cur.fetchall() + for row in records: + print(row) +``` + +## See also + +- [Client discovery and tokens](/docs/security/oidc/client-discovery/) - find + the provider endpoints and select the token QuestDB expects +- [OIDC device flow](/docs/security/oidc/device-flow/) - interactive sign-in + for CLI applications, containers, and remote notebooks +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + grant the endpoint permission each client protocol requires diff --git a/documentation/security/oidc/device-flow.mdx b/documentation/security/oidc/device-flow.mdx new file mode 100644 index 0000000000..b258948c2e --- /dev/null +++ b/documentation/security/oidc/device-flow.mdx @@ -0,0 +1,622 @@ +--- +title: OIDC device flow +description: "OIDC device flow for QuestDB Enterprise: headless SSO sign-in from a CLI, container, or notebook, with Java, Python, Rust, C and C++ examples." +--- + +import Tabs from "@theme/Tabs"; +import TabItem from "@theme/TabItem"; +import JavaExample from "../../partials/_oidc.device-flow.java.partial.mdx"; +import PythonExample from "../../partials/_oidc.device-flow.python.partial.mdx"; +import RustExample from "../../partials/_oidc.device-flow.rust.partial.mdx"; +import CppExample from "../../partials/_oidc.device-flow.cpp.partial.mdx"; +import CExample from "../../partials/_oidc.device-flow.c.partial.mdx"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + The OIDC device flow signs in an interactive user from a client that + cannot receive a browser redirect. + + +The [OAuth 2.0 Device Authorization Grant](https://www.rfc-editor.org/rfc/rfc8628) +signs in a person without redirecting a browser back to the client. It is a good +fit for command-line applications, containers, and remote notebook kernels: the +client displays a verification URL and short code, and the user can authorize +from a browser on any laptop or phone. + +QuestDB does not run the device flow. The client communicates directly with the +Identity Provider, then presents the resulting bearer token to QuestDB: + +```mermaid +sequenceDiagram + participant Client as QuestDB client + participant QDB as QuestDB + participant IdP as Identity Provider + participant User + alt Discovery through QuestDB + Client->>QDB: GET /settings + QDB-->>Client: client ID, scope, endpoints, token mode + else issuer pinned on the client + Client->>IdP: GET {issuer path}/.well-known/openid-configuration + IdP-->>Client: endpoints + end + Client->>IdP: Request device and user codes + IdP-->>Client: device_code, user_code, verification URI + Client->>User: Display URL and code + par User authorizes + User->>IdP: Sign in and approve in a browser + and Client polls + loop Until approved or expired + Client->>IdP: Poll token endpoint + IdP-->>Client: authorization_pending / tokens + end + end + Client->>QDB: Authorization: Bearer selected token + Note over Client,IdP: Later: silent refresh, or an interaction-required error +``` + +The client must respect the polling interval, `slow_down` responses, and the +device code's expiry. The official clients handle those protocol details, +choose the access or ID token according to +[`acl.oidc.groups.encoded.in.token`](/docs/security/oidc/client-discovery/#which-token-to-send), cache it in memory, +and refresh it silently when the provider issues a refresh token. They never +send the device code or the user's Identity Provider password to QuestDB. + +## Configure the provider and QuestDB + +Before using the flow: + +1. Register the application as a public client with the Identity Provider and + enable the device authorization grant on it. Each application which + integrates via OIDC should have its own Client Id, so that it can be told + apart from QuestDB in the provider's audit log and given its own policies. + Pass that Client Id to the client explicitly, as described under + [Explicit configuration and endpoint pinning](#explicit-configuration-and-endpoint-pinning). +2. Make the device authorization endpoint available to clients. For the + zero-configuration examples below, `acl.oidc.configuration.url` must resolve + a provider document containing `device_authorization_endpoint`, or the + host-based configuration must set + [`acl.oidc.device.authorization.endpoint`](/docs/configuration/oidc/#acloidcdeviceauthorizationendpoint). +3. Keep `openid` in [`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope) + to request OIDC authentication semantics. Also request the provider's + refresh-token scope, commonly `offline_access`. The client registration, + consent, and provider policy determine whether a refresh token is issued. + Without one, the client must ask the user to sign in again after the current + token expires. + +For example, a host-based configuration can add: + +```ini title="server.conf" +acl.oidc.scope=openid offline_access +acl.oidc.device.authorization.endpoint=/as/device_authz.oauth2 +``` + +The exact endpoint path and refresh-token scope are provider-specific. With +configuration-document discovery, do not set the endpoint property: all +individual endpoint settings are ignored in that mode. + +If QuestDB does not publish the device authorization endpoint, pin the provider +with the client's `issuer` option so the client can fetch +`{issuer}/.well-known/openid-configuration`, including any path component in the +issuer, or configure the client ID and both endpoints explicitly. + +Clients reach QuestDB over `https` for discovery, so the HTTP endpoint needs TLS +enabled with [`http.tls.enabled`](/docs/configuration/tls/#httptlsenabled) and a +certificate. For setting OIDC up against a specific provider, the OIDC guide has +walkthroughs for +[Microsoft Entra ID](/docs/security/oidc/entra-id/) and +[PingFederate](/docs/security/oidc/pingfederate/). + +## Prepare permissions and the example schema + +Every client example below writes one row to `trades` over QWP/WebSocket. Before +running an example, create the table with the schema from the +[Create a sample database guide](/docs/getting-started/create-database/#creating-a-table): + +```questdb-sql title="Create the example table" +CREATE TABLE trades ( + timestamp TIMESTAMP, + symbol SYMBOL, + side SYMBOL, + price DOUBLE, + amount DOUBLE +) TIMESTAMP(timestamp) PARTITION BY DAY +DEDUP UPSERT KEYS(timestamp, symbol); +``` + +Next, run the following as the built-in admin. Replace the external alias with +the exact group name or identifier that the Identity Provider places in the +user's groups claim: + +```questdb-sql title="Map the external group and grant example permissions" +CREATE GROUP oidc_ingest +WITH EXTERNAL ALIAS 'identity-provider-ingest-group'; + +GRANT HTTP TO oidc_ingest; +GRANT INSERT ON trades TO oidc_ingest; +``` + +The `HTTP` permission allows the QWP/WebSocket connection, while `INSERT` allows +the client to write to the existing table. The external alias gives those +permissions to users whose groups claim contains that value. See +[Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions) +for provider-specific group mapping details. + +## Official client examples + +Each example discovers the OIDC client ID, scope, token-selection mode, and +provider endpoints from QuestDB's [settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), then +signs in before opening the database connection. + +The examples use QuestDB's default Web Console context. The clients append +`/settings` to the QuestDB URL, so include a non-default context in that URL. For +example, with `http.context.web.console=/console`, pass +`https://questdb.example.com:9000/console`. See the settings endpoint's +[path rules](/docs/security/oidc/client-discovery/#settings-endpoint). + +Discovering the client ID means reusing QuestDB's own registration, which keeps +the examples short but gives the application no separate identity at the +provider. That is fine for evaluation; in production register the application +and set its Client Id explicitly, as described below. + +The discovery URL must use `https`: reaching QuestDB over plaintext would let an +attacker replace the advertised Identity Provider endpoints and redirect the +device code and the refresh token, so the clients reject it rather than warn. +The Rust, C, C++ and Python clients accept loopback `http` for local +development; the Java client requires the opt-out below even for `localhost`. + +The opt-out covers the QuestDB discovery request only. Every client holds the +Identity Provider itself to `https` or loopback, and no flag relaxes that: + +| Client | Opt out of `https` for discovery | +| --- | --- | +| Java | `allowInsecureTransport(true)` | +| Python | `insecure=True` | +| Rust | `.allow_insecure_transport(true)` | +| C | `questdb_oidc_builder_allow_insecure_transport()` | +| C++ | `.allow_insecure_transport(true)` | + +The flag alone is not enough. Over plaintext a client also refuses to take the +credential endpoints from a `/settings` response that could have been tampered +with in transit, so pin the provider with `issuer` as well, and pass explicitly +any client ID, scope, audience, or token-selection mode you rely on. An issuer +pin covers the endpoints, not those four values. + + + + + + + + + + + + + + +Enable the `oidc` crate feature: + +```toml title="Cargo.toml" +[dependencies] +questdb-rs = { version = "7.1", features = ["oidc"] } +``` + + + + + + + + + + + + + + + + +The authentication object owns the token state, and the connection asks it for a +token on every connect and reconnect. Java and Rust take a token provider +(`auth::getToken`, a closure over `auth.token()`); Python, C and C++ take the +authentication object itself. Either way, hand the connection the object rather +than copying the current token into the connection string, otherwise a reconnect +after expiry keeps sending the stale value. + +Device flow always requires a person to approve the sign-in. For unattended +services, scheduled jobs, or CI, use a +[QuestDB service-account token](/docs/security/oidc/client-integration/#non-interactive-clients) +or a provider flow intended for machine identities instead. + +## The sign-in prompt + +Signing in renders the verification URL and the user code, then blocks until the +user approves or the device code expires. Where that text goes, and whether the +call runs at all without a terminal, differs by client: + +| Client | Default prompt | Without a terminal | +| --- | --- | --- | +| Java | prints to `System.out` | runs anyway | +| Python | prints to stderr, or to the frontend inside an IPython kernel | runs anyway, except under a notebook executor which refuses stdin | +| Rust, C, C++ | prints to stderr | runs anyway | + +None of the clients tests for a TTY. A missing terminal is not evidence of a +missing human: a pipe into `tee`, a process supervisor, or an IDE which captures +stderr all still put the code in front of someone. The call therefore prompts +and waits whatever the process is attached to, until the user approves or the +device code expires. Redirecting or discarding the output hides the code without +shortening that wait. + +The one client which stops early does so on positive evidence rather than on a +guess. A notebook executor such as papermill, `nbclient` or +`jupyter nbconvert --execute` runs a real kernel, but issues every request with +`allow_stdin=False`, which is the frontend stating that nobody is there to +answer. The Python client reads that flag and fails with +[`OidcInteractionRequired`](#interaction-required-errors) instead of polling to +the device code's deadline. + +A headless service or a CI job should say so itself rather than rely on the +absence of a terminal. Turning the prompt off makes sign-in fail immediately +with an [interaction-required error](#interaction-required-errors): + +| Client | Fail instead of prompting | +| --- | --- | +| Java | not available | +| Python | `interactive=False` | +| Rust | `.interactive(false)` | +| C | `questdb_oidc_builder_interactive(builder, false, &error)` | +| C++ | `.interactive(false)` | + +All five clients also try to open the verification URL in a local browser by +default. That is useful on a workstation and pointless on a remote or headless +host, where the browser opens on the wrong machine or a process starts that +nobody can see. Python is the exception: it suppresses the launch by itself +inside a Jupyter kernel. Turn it off explicitly elsewhere: + +| Client | Suppress the browser launch | +| --- | --- | +| Java | `prompt(DeviceCodePrompt.SYSTEM_OUT)`, or the `questdb.client.oidc.open.browser` system property set to `false` | +| Python | `open_browser=False` | +| Rust | `.open_browser(false)` | +| C | `questdb_oidc_builder_open_browser(builder, false, &error)` | +| C++ | `.open_browser(false)` | + +Override the default prompt to render the challenge elsewhere: + +| Client | Custom prompt | +| --- | --- | +| Java | `prompt(...)` | +| Python | `renderer=`, or `qr=True` for a QR code | +| Rust | `.renderer(...)` | +| C | `questdb_oidc_builder_event_handler()` | +| C++ | `.event_handler(...)` | + +The Java, Python, C and C++ clients strip terminal controls, bidi marks and +zero-width characters from the prompt's text fields before the callback sees +them. Rust sanitizes inside its built-in renderer only, so a custom Rust +renderer receives the Identity Provider's response verbatim. + +:::caution + +A custom Rust renderer must sanitize the user code and verification URL itself +before writing them to a terminal or a DOM. Rendering them raw reopens the +spoofing surface the built-in renderer closes. Rust exposes +`display_user_code()` and `display_verification_uri()` on the challenge. + +::: + +The text fields are for display only. Where a client opens the URL or makes it +clickable, use the separately vetted browser target instead: `browser_target` on +the C event struct, `event_view::browser_target()` in C++, `browser_target()` on +the Rust challenge, and the `browser_target` key on the Python prompt event. + +## Docker and Kubernetes + +Device flow works from a Docker container or Kubernetes pod because the browser +can run on the host or another device. The container displays a verification URL +and user code while making outbound requests to QuestDB and the Identity +Provider. It does not need an inbound callback port. + +For example, a Python container can disable local browser launching and persist +tokens in a mounted directory: + +```python title="Python device flow in a container" +from questdb.auth import FileTokenStore, OidcDeviceAuth + +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + open_browser=False, + token_store=FileTokenStore("/var/lib/questdb-client/tokens"), +) +auth.sign_in() +``` + +When running device flow in a container: + +- Allow outbound HTTPS access to QuestDB and the Identity Provider. +- Keep stderr visible through attached output or container logs so the user can + read the verification URL and code. +- Set `open_browser=False`. Launching a browser is best-effort, but is not useful + inside most containers. +- Mount the token-store directory as a persistent, access-restricted volume if + credentials should survive container or pod restarts. The file store contains + an unencrypted refresh token; see [token persistence](#token-persistence). +- Call `sign_in()` before creating senders, pools, or database connections. + +The clients do not require a TTY. A detached container can start device flow and +will wait until somebody reads its logs and approves the code, or until the +device code expires. + +:::note Unattended workloads + +Device flow always needs a person to approve the sign-in. For production +services, scheduled jobs, and CI, use a +[QuestDB service-account token](/docs/security/oidc/client-integration/#non-interactive-clients) +or an OAuth client-credentials flow. Set `interactive=False` when constructing +the Python authentication object so missing credentials fail immediately rather +than waiting for approval. + +::: + +## Explicit configuration and endpoint pinning + +Discovery supplies the client ID, scope, token-selection mode, and endpoints. +Set configuration explicitly when the application has its own registration with +the provider, or when QuestDB does not publish what the client needs: + +| Setting | Set it when | +| --- | --- | +| `client_id` | the application has its own registration, which is the recommended setup | +| `scope` | requesting the provider's refresh-token scope for this application alone, rather than server-wide | +| `audience` | the provider requires one, or QuestDB checks `aud` against [`acl.oidc.audience`](/docs/configuration/oidc/#acloidcaudience). QuestDB does not publish that key, so discovery cannot supply it | +| `issuer` | hardening discovery, or QuestDB publishes no device authorization endpoint so the client reads `{issuer}/.well-known/openid-configuration` instead | +| `token_endpoint`, `device_authorization_endpoint` | pinning both endpoints directly rather than through an issuer | +| `groups_in_token` | overriding which of the two tokens the client selects | + +Python passes these as keyword arguments to `from_questdb`. Rust and C++ are +builder methods in snake case, and C prefixes the same names with +`questdb_oidc_builder_`. + +Java splits them. `OidcDeviceAuth.DiscoveryOptions` carries only `issuer`, +`prompt`, `tokenStore`, `tlsConfig` and `allowInsecureTransport`; `client_id`, +`scope`, `audience` and `groups_in_token` live on `OidcDeviceAuth.builder()`, +which requires the client ID and both endpoints together. So Java cannot combine +its own client ID with endpoints discovered from `/settings`: use the builder +and pin the endpoints too. The other four clients accept a `client_id` override +on top of discovery. + +:::caution + +`/settings` needs no authentication, so it designates where your users type +their credentials and where the refresh token goes. Pinning `issuer` binds the +advertised endpoints to that issuer's origin, which is why it is worth setting +against any server you do not fully trust, not only when an endpoint is missing. + +::: + +Pinning a provider and using an application's own registration: + + + + +```java +OidcDeviceAuth auth = OidcDeviceAuth.builder() + .clientId("my-cli") + .audience("questdb-api") + .deviceAuthorizationEndpoint("https://login.example.com/oauth2/v2.0/devicecode") + .tokenEndpoint("https://login.example.com/oauth2/v2.0/token") + .build(); +``` + + + + +```python +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + issuer="https://login.example.com", + client_id="my-cli", + audience="questdb-api", +) +``` + + + + +```rust +let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .issuer("https://login.example.com") + .client_id("my-cli") + .audience("questdb-api") + .build()?; +``` + + + + +An endpoint QuestDB advertises must share the pinned issuer's origin. When the +issuer contains a path, the endpoint must also remain beneath that path. The +client rejects an advertised endpoint which fails either check. An endpoint the +client reads from the provider's own `{issuer}/.well-known/openid-configuration` +is exempt, which is what makes the pin work with providers that host their +endpoints outside the issuer, such as Microsoft Entra ID and Google. + +## Token persistence + +Tokens stay in memory unless a file token store is enabled. The store writes the +access, ID and long-lived refresh tokens as unencrypted JSON, so turn it on only +where that at-rest exposure is acceptable: + +| Client | Default location | A directory you choose | +| --- | --- | --- | +| Java | `FileTokenStore.atDefaultLocation()` | `FileTokenStore.at(path)` | +| Python | `FileTokenStore.at_default_location()` | `FileTokenStore.at(path)` | +| Rust | `FileTokenStore::at_default_location()` | `FileTokenStore::at(path)` | +| C | `questdb_oidc_builder_default_file_token_store()` | `questdb_oidc_builder_file_token_store()` | +| C++ | `.default_file_token_store()` | `.file_token_store(directory)` | + +Java, Python and Rust attach the store with `tokenStore(...)`, `token_store=` +and `.token_store(...)` respectively; the C and C++ calls above are builder +methods and attach it themselves. Attach it before signing in, so the first +`sign_in()` can reuse a refresh token an earlier run left behind: + + + + +```java +OidcDeviceAuth auth = OidcDeviceAuth.fromQuestDB( + "https://questdb.example.com:9000", + new OidcDeviceAuth.DiscoveryOptions() + .tokenStore(FileTokenStore.at(Paths.get("/var/lib/myapp/oidc")))); +``` + + + + +```python +auth = OidcDeviceAuth.from_questdb( + "https://questdb.example.com:9000", + token_store=FileTokenStore.at("/var/lib/myapp/oidc"), +) +``` + + + + +```rust +let auth = OidcDeviceAuth::from_questdb("https://questdb.example.com:9000") + .token_store(FileTokenStore::at("/var/lib/myapp/oidc")) + .build()?; +``` + + + + +The default directory is `$HOME/.questdb/oidc-tokens/`, overridden by the +`questdb.client.oidc.token.store.dir` environment variable. The name contains +dots, so most shells will not `export` it directly; set it through the container +or service manager, or pass an explicit directory instead. Java reads that same +name as a JVM system property rather than an environment variable, and falls +back to `${user.home}/.questdb/oidc-tokens/`. Resolving the default location can +fail where there is no home directory, as in a distroless container, so Rust +returns a `std::io::Result` from `at_default_location()` and an explicit +directory avoids the question. On Unix the clients create token +files with mode `0600` and the store directory with `0700`; on other platforms +the directory's existing ACL governs access, so restrict it before enabling the +store. + +## Interaction-required errors + +A transport call never prompts. When the cached token cannot be refreshed +silently, the call fails instead, and the application has to sign in again on +the main or UI thread: + +| Client | How it surfaces | +| --- | --- | +| Python | `OidcInteractionRequired`, importable from `questdb.auth` | +| Rust | a `questdb::Error` carrying `ErrorCode::AuthError`; read `err.oidc_error()` and match on `OidcErrorKind::InteractionRequired` | +| C | error kind `QUESTDB_OIDC_ERROR_INTERACTION_REQUIRED` | +| C++ | `questdb::oidc::error_kind::interaction_required` | +| Java | `OidcAuthException`, with no distinct interaction-required type to match on | + +In C++ a failure raised through an attached sender arrives as +`questdb::ingress::line_sender_error` carrying `oidc_diagnostic()`, not as +`questdb::oidc::error`, so catch the common base `const questdb::error&` to +handle both. + +Catching it and signing in again: + + + + +```python +from questdb.auth import OidcInteractionRequired + + +try: + sender.flush(wait=True) +except OidcInteractionRequired: + auth.sign_in() # on the main or UI thread + sender.flush(wait=True) +``` + + + + +```rust +fn needs_sign_in(err: &questdb::Error) -> bool { + err.code() == ErrorCode::AuthError + && matches!( + err.oidc_error().map(|e| e.kind()), + Some(OidcErrorKind::InteractionRequired) + ) +} + +if let Err(err) = sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok) { + if needs_sign_in(&err) { + auth.sign_in()?; // on the main or UI thread + sender.flush_buffer_and_wait(&mut buffer, AckLevel::Ok)?; + } else { + return Err(err); + } +} +``` + + + + +Java has no distinct type to match on, so an application that must recover +automatically has to treat any `OidcAuthException` from a transport call as a +prompt to call `signIn()` again. + +## Implementing the flow yourself + +The Go, Node.js and .NET clients do not run an OIDC flow, and neither does a +tool built directly on the REST API. Acquire the token out of band, with an +OAuth2 library that implements RFC 8628 or by making the two requests yourself, +then pass it as a static bearer token and replace it before it expires. + +Read the client ID, the scope, and the endpoints from the +[settings endpoint](/docs/security/oidc/client-discovery/#settings-endpoint), including +`acl.oidc.device.authorization.endpoint`, which QuestDB publishes without using. + +**Request a device code**, by posting `client_id` and `scope` as +`application/x-www-form-urlencoded` to the device authorization endpoint. The +response carries `device_code`, `user_code`, `verification_uri`, `expires_in`, +an optional `verification_uri_complete` with the code already embedded, and an +optional `interval` which defaults to 5 seconds. Show the URL and the user code, +sanitized, and treat them as untrusted text. + +**Poll the token endpoint** with +`grant_type=urn:ietf:params:oauth:grant-type:device_code`, the `device_code` and +the `client_id`, waiting `interval` seconds between attempts. Four error codes +decide what happens next: + +| `error` | What to do | +| --- | --- | +| `authorization_pending` | the user has not finished; poll again after `interval` | +| `slow_down` | polling too fast; add 5 seconds to `interval`, then poll again | +| `access_denied` | the user refused; stop | +| `expired_token` | `expires_in` elapsed; start a new device authorization request | + +Treat every other OAuth error as terminal: stop polling and return the failure +to the caller. + +Polling faster than `interval`, or ignoring `slow_down`, gets the client rate +limited or blocked by the provider. + +Send the token your configuration calls for, per +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send): the access +token, or the ID token where group memberships are encoded in it. Keep the +`refresh_token` if the provider issues one, and redeem it with +`grant_type=refresh_token` rather than making the user sign in again. + +[RFC 8628](https://www.rfc-editor.org/rfc/rfc8628) sections 3.1 to 3.5 give the +full request and response grammar. + +For the per-language API surface, the connection options, and the rest of each +client's behaviour, see the client pages: +[Java](/docs/connect/clients/java/#oidc-device-flow-enterprise), +[Python](/docs/connect/clients/python/#oidc-device-flow-enterprise), +[Rust](/docs/connect/clients/rust/#oidc-device-flow-enterprise), and +[C and C++](/docs/connect/clients/c-and-cpp/#oidc-device-flow-enterprise). diff --git a/documentation/security/oidc/entra-id.mdx b/documentation/security/oidc/entra-id.mdx new file mode 100644 index 0000000000..671b100510 --- /dev/null +++ b/documentation/security/oidc/entra-id.mdx @@ -0,0 +1,302 @@ +--- +title: Set up OIDC with Microsoft Entra ID +sidebar_label: Microsoft Entra ID +description: Walkthrough for configuring single sign-on between QuestDB Enterprise and Microsoft Entra ID, covering app registration, group claims, server.conf, and permissions. +--- + +import Screenshot from "@theme/Screenshot"; + +This document sets up SSO authentication for the [QuestDB Web Console](/docs/getting-started/web-console/overview/) in +[Microsoft EntraID](https://www.microsoft.com/en-gb/security/business/identity-access/microsoft-entra-id), formerly known as Azure AD. + + +:::tip + +To enlarge the images, click or tap them. + +::: + +## Set up the client application in Entra ID + +First thing first, let's pick a name for the client! + +Then head to _Microsoft Entra Admin Center_, and register the application +under _Identity - App registrations - New registration_. + + + +The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). + +As a result, it cannot store safely a client secret. + +Instead, it can use PKCE (Proof Key for Code Exchange) to secure the flow. + +When registering the application, select the SPA platform. + +We also have to specify the URL of the [Web Console](/docs/getting-started/web-console/overview/) as Redirect URI. + + + +After clicking _Register_, we have created a client application with the +name _QuestDB_. + +Each application is assigned a unique id (known as Client ID in the +OAuth2 - OIDC standard). The client will identify itself with this id +when sending requests to Entra ID. + + + +We find the platform configurations under _Authentication_. This is the place where +the previously set redirect URI can be viewed and modified. We can also specify +additional redirect URIs, if necessary. + +The redirect URIs of the application are automatically eligible for the +_Authorization Code Flow with PKCE_, which is a special version of the OAuth2 standard's +Authorization Code Flow. It is specifically designed for applications where a client +secret (e.g. a password) could not be kept safely. As single page applications run in +the browser, they fall into this category. + +The redirect URIs are also added to the _CORS_ (Cross-Origin Resource Sharing) policy +of EntraID. CORS is a mechanism to allow a web page, such as the Web Console, to access +resources from a different domain than the one that served the page. In this context +this means that we let the Web Console to access Entra ID, while its origin is the +HTTP endpoint of QuestDB. + + + +If we scroll down to the bottom of this page, we can also find a section where we +can enable the _Resource Owner Password Credential Flow_. + +This OAuth2 flow is legacy, and should be enabled only if there is a requirement +of connecting to QuestDB using SSO (Single Sign-On) via clients not supporting +redirect based web flows. +This could mean a Postgres client without OAuth2 integration, such as _psql_, or +a standalone in-house client application, or could be just a jupyter notebook. + +The main issue with this flow is that the client application has to be trusted +with the user's login details. The user's credentials are passed to the +application, in this case to QuestDB, and the client application uses these +credentials to authenticate the user by forwarding them to the identity provider, +in this case to Entra ID. + +It is guaranteed that QuestDB does not store the user's credentials in any way. +They are not persisted into the database, not even in encrypted form. +The login details are treated as passthrough information. Only exception is +that server logs can contain the username, logged for audit purposes. + + + +Our next stop is the _Token configuration_, where the OAuth2/OIDC access and ID +tokens can be customized. + +Note that users can be authenticated without customized tokens, but authorization +would prove to be challenging. The user's security groups are not included +in the tokens by default. + +QuestDB can be configured to request the user's groups from the UserInfo +endpoint of the OAuth2 server, but Entra ID cannot be configured to provide +this information via the UserInfo endpoint. +Therefore, we choose to customize the tokens, QuestDB will decode and +validate the ID token, and take the group information from there. + +QuestDB authorization relies on receiving the group memberships of the user. +Entra ID groups should be mapped to QuestDB groups, and permissions can be +granted to the QuestDB groups. Detailed information about group mappings can +be found under [User permissions](/docs/security/oidc/group-mapping/). + + + +The customized tokens contain user information which cannot be accessed +without permission. User information is provided by Microsoft Graph, so +the client application needs specific permissions to access +Microsoft Graph APIs. + +These permissions can be configured under _API permissions_. It is important +to note that we will be setting _Delegated_ permissions here, meaning we +are not granting actual permissions to access user data. Instead, each user +logging into QuestDB will have to consent to accessing their user profile. + + + +By default, the _User.Read_ permission is added to the list, but what we +really need is: + - openid: to be able to issue ID tokens + - profile: to access user information + - offline_access: to be able to issue refresh tokens + +By clicking on _Microsoft Graph_ we can select and add these permissions. + + + +The _User.Read_ permission is not needed. It can be removed by clicking +on the `...` at the end of the row, and selecting _Remove permission_ from +the popup menu. + + + +With this we have finished setting up the QuestDB client application +in Entra ID, and now we can wire QuestDB and Entra ID together by +adding OIDC configuration to QuestDB. + +## QuestDB configuration + +The below should be set in QuestDB's `server.conf`: + +```ini title="server.conf" +# enable OIDC +acl.oidc.enabled=true + +# the claim contains the username or user id +acl.oidc.sub.claim=name + +# the claim contains the user's group memberships +acl.oidc.groups.claim=groups + +# groups are encoded in the token +acl.oidc.groups.encoded.in.token=true + +# OIDC configuration endpoint of Entra ID +acl.oidc.configuration.url=https://login.microsoftonline.com/12345678-1234-1234-1234-123456789abc/v2.0/.well-known/openid-configuration + +# application ID taken from Entra ID +acl.oidc.client.id=8de84b90-1ea5-4e41-9e84-dba860aa01a6 + +# redirect URI, QuestDB's HTTP endpoint +acl.oidc.redirect.uri=http://localhost:9000 + +# OAuth scopes the user has to consent to +acl.oidc.scope=openid profile offline_access + +# enable ROPC flow +# optional, required only if ROPC is enabled in Entra ID +acl.oidc.ropc.flow.enabled=true +``` + +The application ID and the OIDC configuration endpoint's URL can be found +in the Overview of the application in Entra ID. + +The application ID is displayed right under the application's name, the +OIDC configuration endpoint is displayed on the panel which opens up when +the _Endpoints_ button is clicked. + + + +:::caution + +This setup relies on +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken), +because Entra ID cannot serve the group memberships from its User Info +endpoint. QuestDB then validates the ID token itself and never asks Entra ID +for current user information. Revoking a token, disabling a user, or changing +their Entra ID groups therefore does not immediately change access in QuestDB. +The client must obtain a new ID token for updated claims. + +On releases that validate `exp`, a token can still authenticate for the +60-second clock-skew allowance plus up to `acl.oidc.cache.ttl` while an earlier +validation remains cached. Existing PGWire and WebSocket connections are not +closed when the token expires. QuestDB Enterprise 4.0.1 and earlier do not +validate `exp` in this mode, so upgrade before relying on token expiry. Read the +configuration option before going to production, and note that clients must +send the ID token here, not the access token. + +::: + +## Map groups and grant permissions + +Now we can start QuestDB, and login with the built-in admin to create +group mappings. + +As mentioned earlier, authorization works by mapping Entra ID groups +to QuestDB groups. When the user logs in, QuestDB decodes Entra ID +group memberships from the token, then finds the QuestDB groups +mapped to them, and the user gets the permissions based on the +mapped groups. + +```questdb-sql title="Create a group which is mapped to an Entra ID group" +CREATE GROUP extUsers WITH EXTERNAL ALIAS '87654321-1234-1234-1234-123456789abc'; +``` +The above command maps the Entra ID group identified by object +id `87654321-1234-1234-1234-123456789abc` to a QuestDB group called `extUsers`. + +We should grant the necessary QuestDB endpoint permissions first +to make sure users can access the Web Console, Postgres and ILP +interfaces as required. [Read more about endpoint permissions](/docs/security/rbac/authentication/#endpoint-permissions). + +```questdb-sql title="Grant endpoint permissions" +GRANT HTTP, PGWIRE TO extUsers; +``` + +Now we can grant the rest of the permissions as required. We can +grant access to tables, for example. + +```questdb-sql title="Grant database permissions" +GRANT SELECT ON trades, fx_trades TO extUsers; +``` + +## Confirm group mappings and login + +To test, head to the Web Console and login. + +If all has been wired up well, then login will succeed, and the user +will have the access granted to them. + +
+ +## See also + +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + map Entra ID group IDs to QuestDB groups +- [OIDC settings reference](/docs/configuration/oidc/) - all `acl.oidc.*` + settings and token-validation behavior +- [How sign-in works](/docs/security/oidc/how-sign-in-works/) - the browser + authorization flow used by the Web Console diff --git a/documentation/security/oidc/group-mapping.mdx b/documentation/security/oidc/group-mapping.mdx new file mode 100644 index 0000000000..deb9a493f2 --- /dev/null +++ b/documentation/security/oidc/group-mapping.mdx @@ -0,0 +1,148 @@ +--- +title: Mapping OIDC groups and permissions +sidebar_label: Mapping groups and permissions +description: Map Identity Provider group memberships onto QuestDB groups with EXTERNAL ALIAS, and understand how QuestDB builds a user's access list from OIDC claims. +--- + +import Screenshot from "@theme/Screenshot"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote"; + + + OIDC group mapping and authorization are available in QuestDB Enterprise. + + +QuestDB does not store permissions for an OIDC user. It reads the user's group +memberships out of the Identity Provider and maps them onto QuestDB groups, +which carry the permissions. This page covers where those memberships come +from, how to create the mapping, and what happens when a claim is missing. + +## User permissions + +QuestDB requires additional user information to be able to construct the user's +access list. + +As a reminder, the access list is the list of permissions that determines what +the user can and cannot do. + +QuestDB itself does not store external users, nor their passwords or any other +authentication related detail. + +External users and their authentication methods are managed by the Identity +Provider. + +Since external users are not managed by QuestDB, permissions cannot be granted +to them directly. + +Instead, the database expects a list of groups, called the _groups claim_ to be +present in the user information. + +These external group names are mapped to QuestDB's own groups. + +The access list of the external user consists of the permissions granted to +those groups: + + + +### Mapping user permissions + +The mappings between external and QuestDB groups are managed with +[CREATE GROUP](/docs/query/sql/acl/create-group/) and +[ALTER GROUP](/docs/query/sql/acl/alter-group/): + +```questdb-sql title="Create a group which is mapped to an Active Directory group" +CREATE GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +```questdb-sql title="Map an Active Directory group to an already existing QuestDB group" +ALTER GROUP groupName WITH EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +```questdb-sql title="Remove an Active Directory mapping without deleting the QuestDB group" +ALTER GROUP groupName DROP EXTERNAL ALIAS 'CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev'; +``` + +The external alias establishes membership; it does not grant access by itself. +Grant endpoint and data permissions to the mapped QuestDB group, for example: + +```questdb-sql title="Grant Web Console and read access to the mapped group" +GRANT HTTP TO groupName; +GRANT SELECT ON trades TO groupName; +``` + +Every external user mapped to `groupName` receives those permissions. See +[Granting permissions](/docs/security/rbac/granting-permissions/) for endpoint, +table, column, and system-level examples. + +QuestDB can obtain the list of external groups from either the User Info +response or the ID token. By default, it reads them from the User Info +response. When +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is `true`, it reads them from the ID token instead; this mode supports providers +such as Entra ID, which cannot return groups from the User Info endpoint. + +In the User Info example below, the response contains a claim called +`groups`. Its name is set with +[`acl.oidc.groups.claim`](/docs/configuration/oidc/#acloidcgroupsclaim), which +has no default and must be set whenever OIDC is enabled. + +If the groups claim is missing or it is an empty list, authentication fails. The +same happens when the claim named by +[`acl.oidc.sub.claim`](/docs/configuration/oidc/#acloidcsubclaim) is missing or +empty. Over HTTP QuestDB replies with `401 Unauthorized`, while on the PGWire +endpoint the client receives an authentication error. Either way the reason is +only visible in the server log. + +The user has to have at least the `HTTP` permission to be able to successfully +login via the [Web Console](/docs/getting-started/web-console/overview/). + +```json title="User info response example with Active Directory groups" +{ + "sub": "externalUser", + "name": "External User", + "groups": [ + "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", + "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" + ] +} +``` + +How quickly an Identity Provider change takes effect depends on where QuestDB +reads the groups: + +- **User Info response**, the default: after the cached authentication entry + expires, the next authentication calls the User Info endpoint again and + rebuilds the access list. Configure that interval with + [`acl.oidc.cache.ttl`](/docs/configuration/oidc/#acloidccachettl). +- **ID token**, when `acl.oidc.groups.encoded.in.token=true`: QuestDB validates + the token locally and never asks the provider for current user information. + Cache expiry only makes QuestDB validate and read the same token again. Group + changes take effect only after the client obtains a new ID token. Disabling a + user or revoking the token at the provider is not discovered through the same + token; access ends only when a later authentication fails QuestDB's local + checks. + +In either mode, cache expiry does not terminate an established PGWire or +WebSocket connection. The change takes effect when a request or connection +authenticates with refreshed information. A disabled user or a revoked token +may therefore retain access temporarily; see the +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +security considerations for the encoded-token boundaries. + +QuestDB also uses claims such as `sub` and `name` for the principal displayed in +the [Web Console](/docs/getting-started/web-console/overview/) and in audit logs. + +## See also + +- [Granting permissions](/docs/security/rbac/granting-permissions/) - grant + endpoint and data access to mapped QuestDB groups +- [Permissions reference](/docs/security/rbac/permissions-reference/) - the + permissions external groups can receive +- [Microsoft Entra ID](/docs/security/oidc/entra-id/) - map group IDs carried + in an ID token +- [PingFederate](/docs/security/oidc/pingfederate/) - map groups returned by + the User Info endpoint diff --git a/documentation/security/oidc/how-sign-in-works.mdx b/documentation/security/oidc/how-sign-in-works.mdx new file mode 100644 index 0000000000..fa01dc6bcd --- /dev/null +++ b/documentation/security/oidc/how-sign-in-works.mdx @@ -0,0 +1,289 @@ +--- +title: How OIDC sign-in works +sidebar_label: How sign-in works +description: Step-by-step walkthrough of the Authorization Code Flow with PKCE that QuestDB Enterprise and the Web Console use to authenticate a user against an OIDC provider. +--- + +import Screenshot from "@theme/Screenshot"; + +The [Web Console](/docs/getting-started/web-console/overview/) signs a user in +with the Authorization Code Flow with PKCE. This page follows that exchange +end to end, from the code verifier the client generates to the permission +check QuestDB runs on the user's group memberships. Read it when you are +integrating your own browser client, or debugging a sign-in that fails partway +through. + +## Authentication and authorization flow + +The OAuth2/OIDC standard defines different ways of obtaining access and ID +tokens from the OIDC Provider, referred to as the "_flow_". + +The goal of this flow is to get the user, who is sitting in front of the Web +Console, authenticated. + +Then, it allows QuestDB to determine the user's permissions based on user +information provided by the Identity Providers. + +Specifically, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) uses the +`Authorization Code Flow with PKCE` option. + +It consists of ten steps... + +### 1. Secret generation + +First the [Web Console](/docs/getting-started/web-console/overview/) generates a cryptographically strong +random secret called the _code verifier_. + +The secret is hashed using the _SHA256 algorithm_. The result is the _code +challenge_. + +After PKCE initialization the [Web Console](/docs/getting-started/web-console/overview/) requests an +_authorization code_ from the OIDC Provider. + +It calls the Authorization endpoint with a few parameters, including the: + +- **Client Id** +- requested scopes (the list of scopes are configurable, default is `openid` + only) +- code challenge +- algorithm used to generate the code challenge from the code verifier (SHA256) + +When the Authorization Server receives the request, it checks if the user has +been authenticated already: + +- If the user has a valid session, it can be provided with an authorization code + straight away, so we jump to step 4. + +- If the user does not have a valid session yet, it will be redirected to the + Identity Provider for authentication. + +```bash title="Authorization code request example" +https://oidc.provider:443/as/authorization.oauth2?client_id=questdb&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Fquestdb.example.com%3A9000&code_challenge=IwZ-WuypAY3fMtvismbj1MQUe5CzMgrBa87nYcgFoLQ&code_challenge_method=S256 +``` + +:::note + +Some Identity Providers require the `state` parameter in the authorization +request. It is another random value generated by the client, which the OIDC +Provider returns unchanged together with the authorization code. Checking it +protects against CSRF attacks. + +If your provider requires it, set +[`acl.oidc.state.required`](/docs/configuration/oidc/#acloidcstaterequired) to +`true`. The [Web Console](/docs/getting-started/web-console/overview/) then +generates the `state` parameter, sends it in the authorization request, and +validates the value returned by the provider. + +::: + +### 2. Prove identity + +Next, the user must prove its identity. + +This could be a username with: + +- a password, +- an OTP +- facial recognition via a mobile app +- or anything else supported by the Identity Provider. + + + +### 3. Scope consent + +After successful authentication, the user provides consent for the requested +scopes. + +The list of scopes is configurable with +[`acl.oidc.scope`](/docs/configuration/oidc/#acloidcscope). + +By default the Web Console requests only the `openid` scope, which is mandatory +for OIDC. + +No ID Token is issued without it. + +A provider may require `offline_access`, explicit consent, or another policy +before issuing a refresh token. Requesting a refresh-token scope does not +guarantee that the provider returns one. + +The OIDC provider can be configured to provide the consent automatically, +without presenting the user with an additional screen in the browser. + + + +### 4. Redirection + +Consent is granted! + +The Authorization Server redirects the user back to the +[Web Console](/docs/getting-started/web-console/overview/) with the _authorization code_: + +```bash title="Authorization code response example" +https://questdb.example.com:9000/?code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A +``` + +### 5. Credential request + +Now, the QuestDB [Web Console](/docs/getting-started/web-console/overview/) requests the ID and access +tokens from the Token endpoint of the OIDC Provider with the authorization code. + +It includes the Client ID and the PKCE code verifier together with the +authorization code in the request. + +The endpoint then hashes the code verifier using the method specified previously +in step 1. + +The result must match the code challenge, also provided in step 1. + +The matching code challenge proves that the token is requested by the client +which requested the authorization code, and it was not stolen: + +```bash title="Token request example" +curl --request POST https://oidc.provider:443/as/token.oauth2 \ + --header "Content-Type: application/x-www-form-urlencoded" \ + --data-urlencode "grant_type=authorization_code" \ + --data-urlencode "code=1L344XEY5XRka1j4ySNa8bVQSLf71as9uGLEuv_A" \ + --data-urlencode "client_id=questdb" \ + --data-urlencode "redirect_uri=https://questdb.example.com:9000" \ + --data-urlencode "code_verifier=uGZh4sQffXLgRna7D-jtEAkuXzp7Lm_okZXBljzP38coAD44kEheIaz7Pdh98KxYtYLZHNiQPCczQYeF" +``` + +### 6. Credentials received + +If the PKCE check passes, the Web Console receives the ID and access tokens. The +provider may also return a refresh token when its scopes, consent, and policy +allow one. + +When present, the Web Console uses the refresh token to obtain replacement +tokens without asking the user to sign in again. Without one, the user must +authenticate again after the current tokens can no longer be used. + +Token lifetimes are configurable in the OIDC Provider. + +```json title="Token response example with an optional refresh token" +{ + "access_token": "gslpJtzmmi6RwaPSx0dYGD4tEkom", + "refresh_token": "FUuAAqMp6LSTKmkUd5uZuodhiE4Kr6M7Eyv.eg83ge", + "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6I...", + "token_type": "Bearer", + "expires_in": 300 +} +``` + +`expires_in` is measured in seconds. The `refresh_token` field is absent when +the provider does not issue one. + +### 7. Database access + +With the tokens, the Web Console can interact with the database. + +A token is in the header of every request sent to QuestDB. Which of the two goes +there depends on where QuestDB reads the group memberships from, see +[Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +:::note + +Worried about exposing the token? An access token is usually opaque and carries +no user details, though some providers issue a JWT here too. An ID token always +does: it is a JWT whose payload is base64 encoded, not encrypted, so treat it as +you would the user's directory record. + +::: + +To carry out permission checks, the database has to know more about the user. + +For this, QuestDB has a User Info Cache. + +Steps 8 and 9 describe the default flow, where QuestDB validates the access +token by asking the OIDC Provider about it. They are skipped when the cache +already holds a valid entry for the token, and they do not happen at all when +[`acl.oidc.groups.encoded.in.token`](/docs/configuration/oidc/#acloidcgroupsencodedintoken) +is `true`, in which case QuestDB validates the ID token itself and never calls +the provider. See [Which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +```http title="Query request example" +GET /api/v1/sql/execute?query=select%20current_user() HTTP/1.1 +Host: questdb.example.com:9000 +Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom +``` + +### 8. Find user information + +No user information in the cache, or stale information? + +QuestDB uses the access token to request user information from the OIDC +Provider's User Info endpoint. + +This call also serves as token validation. + +If the token is not real or has been expired, the User Info endpoint replies +with an error: + +```http title="User info request example" +GET /idp/userinfo.openid HTTP/1.1 +Host: oidc.provider:443 +Authorization: Bearer gslpJtzmmi6RwaPSx0dYGD4tEkom +``` + +### 9. Receive user information + +If the access token is valid, QuestDB receives the required user information +from the endpoint, then updates its cache. + +The cache improves performance, as QuestDB does not have to turn to the OIDC +Provider on every single request. + +Do note that cache expiry is configurable: + +```json title="User info response example with Active Directory groups" +{ + "sub": "externalUser", + "name": "External User", + "groups": [ + "CN=TestGroup1,OU=DC Users,DC=ad,DC=quest,DC=dev", + "CN=TestGroup2,OU=DC Users,DC=ad,DC=quest,DC=dev" + ] +} +``` + +### 10. Permission check + +With the help of the user information, QuestDB can carry out +[permission checks](/docs/security/oidc/group-mapping/). + +If the permission check is successful, the database will process the request, +and then sends the results back: + +```json title="Query response example" +{ + "query": "select current_user()", + "columns": [ + { + "name": "current_user", + "type": "STRING" + } + ], + "dataset": [["External User"]], + "count": 1, + "timestamp": -1 +} +``` + +## See also + +- [Client discovery and tokens](/docs/security/oidc/client-discovery/) - the + settings the Web Console and other clients discover +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + turn the final group claim into QuestDB permissions +- [OIDC settings reference](/docs/configuration/oidc/) - configure the browser + flow and provider endpoints diff --git a/documentation/security/oidc/index.mdx b/documentation/security/oidc/index.mdx new file mode 100644 index 0000000000..9245f08873 --- /dev/null +++ b/documentation/security/oidc/index.mdx @@ -0,0 +1,152 @@ +--- +title: OpenID Connect (OIDC) +sidebar_label: Overview +slug: /security/oidc +description: Integrate QuestDB Enterprise with an external OIDC Identity Provider for Web Console SSO, client authentication, PGWire token authentication, and group mapping. +--- + +import Screenshot from "@theme/Screenshot"; +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + OpenID Connect (OIDC) enables SSO authentication with external Identity Providers. + + +OpenID Connect (OIDC) integrates with Identity Providers (IdP) external to +QuestDB. + +It is a convenient way to integrate QuestDB into your enterprise environment, +and it provides SSO (Single Sign-On) for the [Web Console](/docs/getting-started/web-console/overview/). + +Microsoft Active Directory and Azure AD, for example, can be turned into an +Identity Provider. + +Specific installation steps depend on the type of the provider. + +## Architecture overview + +Altogether, the architecture appears as such: + + + +We can break it down into core components. + +### Web Console + +QuestDB's interactive UI. Users must authenticate before accessing the database +via the interface. + +The [Web Console](/docs/getting-started/web-console/overview/) uses PKCE (Proof Key for Code Exchange) to +secure the authentication and authorization flow. + +In OAuth2/OIDC terms, the [Web Console](/docs/getting-started/web-console/overview/) is referred to as +the _client_, and it is assigned an identifier: the **Client Id**. + +Each application which integrates via OIDC should be given a different **Client +Id**. + +### OIDC Provider + +Typically consists of a number of modules. + +We are interested in two of them only. + +1. The _Identity Provider_ holds user identities and user information, capable + of authenticating users, and to issue an ID Token which uniquely identify + them. + +2. The _Authorization Server_ grants access to resources, such as a database, in + the form of access tokens. + +The OIDC Provider usually integrates with a number of applications which require +different access to a number of resources. + +These clients communicate with the OIDC Provider via its endpoints. + +It exposes a number of APIs, including the Authorization, Token and User Info +endpoints. + +### QuestDB + +The database, in OAuth2/OIDC terms the _protected resource_ or _resource +server_. + +Only processes requests which contain the bearer token selected by its OIDC +configuration: normally the access token, or the ID token when group memberships +are encoded in it. + + +## Where to go next + +| Page | Covers | +| --- | --- | +| [How sign-in works](/docs/security/oidc/how-sign-in-works/) | The Authorization Code Flow with PKCE, step by step | +| [Client discovery and tokens](/docs/security/oidc/client-discovery/) | The settings endpoint, and which of the two tokens to send | +| [Device flow](/docs/security/oidc/device-flow/) | Headless sign-in for a CLI, container, or notebook kernel | +| [Client integration patterns](/docs/security/oidc/client-integration/) | Browser clients, Jupyter, CLI tools, unattended jobs, PGWire | +| [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | Turning Identity Provider groups into QuestDB permissions | +| [PingFederate](/docs/security/oidc/pingfederate/) | Provider setup walkthrough | +| [Microsoft Entra ID](/docs/security/oidc/entra-id/) | Provider setup walkthrough | + +For the `acl.oidc.*` settings, see the +[OIDC settings reference](/docs/configuration/oidc/). + +## Where each section moved {#moved} + +These anchors are kept so that existing links keep working. Prefer the pages +above when adding new links. + +| Section | Now at | +| --- | --- | +| Authentication and authorization flow | [How sign-in works](/docs/security/oidc/how-sign-in-works/) | +| Secret generation | [How sign-in works](/docs/security/oidc/how-sign-in-works/#1-secret-generation) | +| Prove identity | [How sign-in works](/docs/security/oidc/how-sign-in-works/#2-prove-identity) | +| Scope consent | [How sign-in works](/docs/security/oidc/how-sign-in-works/#3-scope-consent) | +| Redirection | [How sign-in works](/docs/security/oidc/how-sign-in-works/#4-redirection) | +| Credential request | [How sign-in works](/docs/security/oidc/how-sign-in-works/#5-credential-request) | +| Credentials received | [How sign-in works](/docs/security/oidc/how-sign-in-works/#6-credentials-received) | +| Database access | [How sign-in works](/docs/security/oidc/how-sign-in-works/#7-database-access) | +| Find user information | [How sign-in works](/docs/security/oidc/how-sign-in-works/#8-find-user-information) | +| Receive user information | [How sign-in works](/docs/security/oidc/how-sign-in-works/#9-receive-user-information) | +| Permission check | [How sign-in works](/docs/security/oidc/how-sign-in-works/#10-permission-check) | +| User permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/) | +| Mapping user permissions | [Mapping groups and permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions) | +| Interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/) | +| Browser-based clients | [Client integration patterns](/docs/security/oidc/client-integration/#browser-based-clients) | +| Jupyter notebook | [Client integration patterns](/docs/security/oidc/client-integration/#jupyter-notebook) | +| Externalizing credentials | [Client integration patterns](/docs/security/oidc/client-integration/#externalizing-credentials) | +| Enable ROPC | [Client integration patterns](/docs/security/oidc/client-integration/#enable-ropc) | +| CLI and standalone applications | [Client integration patterns](/docs/security/oidc/client-integration/#cli-standalone-applications) | +| Non-interactive clients | [Client integration patterns](/docs/security/oidc/client-integration/#non-interactive-clients) | +| OIDC for the PGWire endpoint | [Client integration patterns](/docs/security/oidc/client-integration/#oidc-for-the-pgwire-endpoint) | +| Active Directory | [PingFederate](/docs/security/oidc/pingfederate/) or [Microsoft Entra ID](/docs/security/oidc/entra-id/) | +| PingFederate | [PingFederate](/docs/security/oidc/pingfederate/) | +| Set up PingFederate client | [PingFederate](/docs/security/oidc/pingfederate/#set-up-pingfederate-client) | +| Access Token Manager settings | [PingFederate](/docs/security/oidc/pingfederate/#access-token-manager-settings) | +| Authorization Server settings | [PingFederate](/docs/security/oidc/pingfederate/#authorization-server-settings) | +| Set up a Microsoft Entra ID Data Source | [PingFederate](/docs/security/oidc/pingfederate/#set-up-a-microsoft-entra-id-data-source) | +| Set up a Password Credential Validator | [PingFederate](/docs/security/oidc/pingfederate/#set-up-a-password-credential-validator) | +| Set up an Identity Provider | [PingFederate](/docs/security/oidc/pingfederate/#set-up-an-identity-provider) | +| Add groups to OIDC policy management | [PingFederate](/docs/security/oidc/pingfederate/#add-groups-to-oidc-policy-management) | +| Enable Resource Owner Password Credentials flow | [PingFederate](/docs/security/oidc/pingfederate/#enable-resource-owner-password-credentials-ropc-flow) | +| QuestDB configuration for PingFederate | [PingFederate](/docs/security/oidc/pingfederate/#questdb-configuration) | +| Confirm QuestDB mappings and login | [PingFederate](/docs/security/oidc/pingfederate/#confirm-questdb-mappings-and-login) | +| Microsoft Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/) | +| Set up the client application in Entra ID | [Microsoft Entra ID](/docs/security/oidc/entra-id/#set-up-the-client-application-in-entra-id) | +| QuestDB configuration | [Microsoft Entra ID](/docs/security/oidc/entra-id/#questdb-configuration) | +| Map groups and grant permissions | [Microsoft Entra ID](/docs/security/oidc/entra-id/#map-groups-and-grant-permissions) | +| Confirm group mappings and login | [Microsoft Entra ID](/docs/security/oidc/entra-id/#confirm-group-mappings-and-login) | +| Configuration options | [OIDC settings](/docs/configuration/oidc/) | + +## See also + +- [OIDC settings reference](/docs/configuration/oidc/) - configure QuestDB and + its Identity Provider connection +- [Role-based access control](/docs/security/rbac/) - grant endpoint and data + permissions after authentication +- [TLS](/docs/security/tls/) - protect credentials and tokens in transit diff --git a/documentation/security/oidc/pingfederate.mdx b/documentation/security/oidc/pingfederate.mdx new file mode 100644 index 0000000000..5677c7f5e5 --- /dev/null +++ b/documentation/security/oidc/pingfederate.mdx @@ -0,0 +1,388 @@ +--- +title: Set up OIDC with PingFederate +sidebar_label: PingFederate +description: Walkthrough for configuring single sign-on between QuestDB Enterprise and PingFederate, covering the client, token manager, group claims, ROPC, and server.conf. +--- + +import Screenshot from "@theme/Screenshot"; + +This document helps set up SSO authentication for the Web Console in +[PingFederate](https://www.pingidentity.com/en/platform/capabilities/authentication-authority/pingfederate.html). + +It is assumed that the Azure Active Directory serves as the Identity Provider +(IdP). + +## Set up PingFederate client + +First thing first, let's pick a name for the client! + + + +The QuestDB [Web Console](/docs/getting-started/web-console/overview/) is a SPA (Single Page App). + +As a result, it cannot store safely a client secret. + +Instead it can use PKCE (Proof Key for Code Exchange) to secure the flow. + +As shown above, leave the client authentication disabled. + +We also have to white list the URL of the [Web Console](/docs/getting-started/web-console/overview/) as a redirection URL: + + + +We can instruct PingFederate to automatically authorize the scopes requested by +the [Web Console](/docs/getting-started/web-console/overview/). + +The user will not be presented the extra window asking for consent after +authentication: + + + +The [Web Console](/docs/getting-started/web-console/overview/) uses the +[Authorization Code Flow](/docs/security/oidc/how-sign-in-works/), +and refreshes tokens automatically. + +Next, enable the grant types required for this flow: + + + +We've selected: + +- Authorization Code +- Refresh Token +- Access Token Validation (Client is a Resource Server) + +After that, select the token manager for the client. + +The token manager is responsible for issuing access tokens. + +All token related settings should be configured in the token manager. + + + +Finally, enable PKCE - as shown above - and save the settings. + +## Access Token Manager settings + +QuestDB does not require any special setup regarding the access token. + +We recommend that you do not to use shorter tokens than the default 28 +characters. + +As the QuestDB [Web Console](/docs/getting-started/web-console/overview/) refreshes the token automatically, there is no need +for long-lived tokens: + + + +We've selected: + +- Token length: 28 +- Token lifetime: 5 +- Lifetime extension policy: None +- Maximum token lifetime: Null +- Lifetime extension threshold percentage: 30 + +For the next step, we tune the Authorization Server. + +## Authorization Server settings + +These settings relate to the authorization code, refresh token and CORS. + + + +In this section, we've entered: + +- Authorization code timeout: 60 +- Authorization code entropy: 30 +- Client secret retention period: 0 + +Next, ensure the `ROLL REFRESH TOKEN VALUES` option is selected: + + + +It is also important to whitelist the [Web Console](/docs/getting-started/web-console/overview/)'s URL on the CORS list: + + + +## Set up a Microsoft Entra ID Data Source + +PingFederate needs a Data Source setup. + +This is a secure LDAP connection to Microsoft Entra ID, formerly known as Azure +Active Directory. + +The data source needs a: + +- name +- hostname +- port +- username and password for the LDAP connection + + + +We have given it the name EntraDS and it will be applied later. + +## Set up a Password Credential Validator + +Now that PingFederate has an LDAP connection, we can use it for authentication. + +First, create a Password Credential Validator: + + + +We've entered: + +- Instance name: EntraPCV +- Instance ID: EntraPCV +- Selected: LDAP Username Password Credential Validator +- Parent instance: None + +Furthermore, we now declare our previously created data source (`EntraDS`): + + + +This links our data store (`EntraDS`) to our PCV (`EntraPCV`). + +## Set up an Identity Provider + +We can use our PCV once we set up an Identity Provider. + +The IdP will be used to authenticate users against Active Directory using the +LDAP connection. + +We do this in the Type subsection: + + +Next, in the IdP Adapter section... + +Click: Add a new row to Credential Validators. + +Select the PCV (`EntraPCV`) we created. + +Optionally alter number of retries: + + + +## Add groups to OIDC policy management + +QuestDB now needs to know about the user's AD group memberships to find their +permissions. + +Groups are passed to QuestDB inside the User Info object in a custom claim. + +This has to be added in the OpenID Connect Policy Management. + +The field is Multi-Valued, because it is a list of group names. + +Under the Attribute Contract subsection, see: + + + +Next, click to the Attribute Scopes subsection. + +Ensure `groups` is among the `openid` attributes: + + + +Onwards to the Attribute Sources & User Lookup Section. + +From this view, you can add local data stores. + +Note item `test` of type of LDAP: + + + +We created it via the following choices in Add Attribute Source: + + + +Note where we specified the Data Store (`EntraDS`). + +This is also where the directory search parameters are defined. + +Back at the Attribute Sources & User Lookup Section section, note we have set +`email`. + +The source is `LDAP (test)`, while the value is `usePrincipalName`: + + + +And finally! + +In the same Attribute Sources & User Lookup Section... + +Find `groups`. + +Note the definition of Source (`LDAP (test)`) that bridges our various parts. + +The value is `memberOf`. + + + +## Enable Resource Owner Password Credentials (ROPC) flow + +As described under [Enable ROPC](/docs/security/oidc/client-integration/#enable-ropc), tools such as `psql` can be +integrated with the OIDC provider using the ROPC flow. + +When setting this flow up, enable the Resource Owner Password Credentials flow in the +client settings. + +Next, create a Resource Owner Credentials Grant Mapping to map values obtained from +the Password Credential Validator (PCV) into the persistent grants. + +When setting this up, select the previously created LDAP Data Source and IdP Adapter, which links +to the existing PCV. + +Then select the `username` attribute of the PCV as `USER_KEY`. + +## QuestDB configuration + +The below should be set in QuestDB's `server.conf`: + +```ini title="server.conf" +# enable OIDC +acl.oidc.enabled=true + +# hostname of the PingFederate server +acl.oidc.host=pingfederate.host + +# the client id picked when setting up the client above +acl.oidc.client.id=questdb + +# the claim which contains the user's group memberships +acl.oidc.groups.claim=groups + +# enable ROPC flow +# optional, required only if ROPC is enabled in PingFederate +acl.oidc.ropc.flow.enabled=true +``` + +The endpoint defaults listed under +[Endpoints](/docs/configuration/oidc/#endpoints) already match the PingFederate +paths, so they do not have to be set. + +Alternatively, set `acl.oidc.configuration.url` instead of `acl.oidc.host` and +let QuestDB discover the endpoints from PingFederate. The two are mutually +exclusive, and the endpoint settings are not used when the configuration URL is +set. + +## Confirm QuestDB mappings and login + +QuestDB requires a mapping, as laid out under +[Mapping user permissions](/docs/security/oidc/group-mapping/#mapping-user-permissions). + +If a given user has the HTTP permission, they will be able to now login via the +[Web Console](/docs/getting-started/web-console/overview/). + +To test, head to `http://localhost:9000` and login. + +If all has been wired up well, then login will succeed. + +## See also + +- [Mapping OIDC groups and permissions](/docs/security/oidc/group-mapping/) - + map PingFederate group names to QuestDB groups +- [OIDC settings reference](/docs/configuration/oidc/) - all `acl.oidc.*` + settings and endpoint-discovery behavior +- [How sign-in works](/docs/security/oidc/how-sign-in-works/) - the browser + authorization flow used by the Web Console diff --git a/documentation/security/rbac.md b/documentation/security/rbac.md deleted file mode 100644 index 5b10f3c266..0000000000 --- a/documentation/security/rbac.md +++ /dev/null @@ -1,704 +0,0 @@ ---- -title: Role-based Access Control (RBAC) -description: - Granular access control from database level down to individual columns and - rows. Learn how to secure your QuestDB instance with users, groups, and - fine-grained permissions. ---- - -import Screenshot from "@theme/Screenshot" -import { EnterpriseNote } from "@site/src/components/EnterpriseNote" - - - Role-based Access Control (RBAC) provides fine-grained permissions for your QuestDB instance. - - -QuestDB Enterprise provides fine-grained access control that can restrict access -at **database**, **table**, **column**, and even **row** level (using views). - -## Quick start - -Here's a complete example to create a read-only analyst user in under a minute: - -```questdb-sql --- 1. Create the user -CREATE USER analyst WITH PASSWORD 'secure_password_here'; - --- 2. Grant endpoint access (required to connect) -GRANT PGWIRE, HTTP TO analyst; - --- 3. Grant read access to specific tables -GRANT SELECT ON trades, prices TO analyst; - --- Done! The analyst can now connect and query trades and prices tables -``` - -To verify: - -```questdb-sql -SHOW PERMISSIONS analyst; -``` - -## Access control depth - -QuestDB's access control operates across two dimensions: - -### Data access granularity - -Control *what data* users can access: - -| Level | What you can control | Example | -| ------------ | ------------------------------- | ----------------------------------------------------- | -| **Database** | All tables, global operations | `GRANT SELECT ON ALL TABLES TO user` | -| **Table** | Specific tables | `GRANT SELECT ON trades TO user` | -| **Column** | Specific columns within a table | `GRANT SELECT ON trades(ts, price) TO user` | -| **Row** | Specific rows via views | Create a view with WHERE clause, grant access to view | - -### Connection access granularity - -Control *how* users can connect: - -| Permission | Protocol | Use case | -| ---------- | ------------------------------- | ------------------------------------------ | -| `HTTP` | REST API, Web Console, ILP/HTTP | Interactive users, web applications | -| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | -| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | - -```questdb-sql --- User can connect via PostgreSQL protocol only (not web console) -GRANT PGWIRE TO analyst; - --- Service can only ingest via ILP, cannot query -GRANT ILP TO ingest_service; - --- Full interactive access -GRANT HTTP, PGWIRE TO developer; -``` - -These dimensions are independent: a user might have `SELECT` on all tables but -only be allowed to connect via `PGWIRE`, or have `INSERT` permission but only -via `ILP`. - -### Column-level access - -Restrict users to see only certain columns: - -```questdb-sql --- User can only see timestamp and price, not quantity or trader_id -GRANT SELECT ON trades(ts, price) TO analyst; -``` - -To grant on every column except a few, use the `*` wildcard with an `EXCLUDE` -list: - -```questdb-sql --- User can see all columns except trader_id -GRANT SELECT ON trades(* EXCLUDE (trader_id)) TO analyst; -``` - -The wildcard covers the columns that exist when the statement runs, not columns -added later. See -[GRANT](/docs/query/sql/acl/grant/#grant-on-all-columns-of-a-table) for details. - -### Row-level access with views - -For row-level security, create a [view](/docs/concepts/views/) that filters rows, -then grant access to the view instead of the underlying table: - -```questdb-sql --- Create a view that only shows AAPL trades -CREATE VIEW aapl_trades AS ( - SELECT * FROM trades WHERE symbol = 'AAPL' -); - --- Grant access to the view, not the base table -GRANT SELECT ON aapl_trades TO aapl_analyst; --- No GRANT on trades table = user cannot see other symbols -``` - -The user `aapl_analyst` can only see AAPL trades. They have no access to the -underlying `trades` table. - -## Common scenarios - -### Read-only analyst - -A user who can query data but cannot modify anything: - -```questdb-sql -CREATE USER analyst WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO analyst; -GRANT SELECT ON ALL TABLES TO analyst; -``` - -### Application service account - -A service account for an application that ingests data into specific tables: - -```questdb-sql -CREATE SERVICE ACCOUNT ingest_app WITH PASSWORD 'pwd'; -GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access -GRANT INSERT ON sensor_data TO ingest_app; -- Can only insert into sensor_data -``` - -### Team-based access with groups - -Multiple users sharing the same permissions: - -```questdb-sql --- Create a group -CREATE GROUP trading_team; - --- Grant permissions to the group -GRANT HTTP, PGWIRE TO trading_team; -GRANT SELECT ON trades, positions TO trading_team; -GRANT INSERT ON trades TO trading_team; - --- Add users to the group - they inherit all permissions -CREATE USER alice WITH PASSWORD 'pwd1'; -CREATE USER bob WITH PASSWORD 'pwd2'; -ADD USER alice TO trading_team; -ADD USER bob TO trading_team; -``` - -### Column-level restrictions (hide sensitive data) - -Allow access to a table but hide sensitive columns: - -```questdb-sql -CREATE USER auditor WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO auditor; - --- Grant access to non-sensitive columns only -GRANT SELECT ON employees(id, name, department, hire_date) TO auditor; --- Columns salary and ssn are not granted = invisible to auditor -``` - -### Row-level security (multi-tenant) - -Different users see different subsets of data: - -```questdb-sql --- Base table has data for all regions -CREATE TABLE sales (ts TIMESTAMP, region SYMBOL, amount DOUBLE) TIMESTAMP(ts); - --- Create region-specific views -CREATE VIEW sales_emea AS (SELECT * FROM sales WHERE region = 'EMEA'); -CREATE VIEW sales_apac AS (SELECT * FROM sales WHERE region = 'APAC'); - --- Grant users access to their region only -CREATE USER emea_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO emea_manager; -GRANT SELECT ON sales_emea TO emea_manager; - -CREATE USER apac_manager WITH PASSWORD 'pwd'; -GRANT HTTP, PGWIRE TO apac_manager; -GRANT SELECT ON sales_apac TO apac_manager; -``` - -### Database administrator - -A user with full control (but not the built-in admin): - -```questdb-sql -CREATE USER dba WITH PASSWORD 'pwd'; -GRANT DATABASE ADMIN TO dba; -``` - -:::warning - -`DATABASE ADMIN` grants all current and future permissions. Use sparingly. - -::: - -### Failover operator - -A service account that can move the primary role between nodes, for an external -coordinator or a runbook, without any other administrative right: - -```questdb-sql -CREATE SERVICE ACCOUNT failover_bot WITH PASSWORD 'pwd'; -GRANT HTTP TO failover_bot; -- POST /lifecycle/switch on port 9003 -GRANT SWITCH ROLE TO failover_bot; -- SWITCH ROLE, SWITCH STATUS, the endpoint -``` - -`SYSTEM ADMIN` is neither required nor sufficient for a role switch, and -`DATABASE ADMIN` includes `SWITCH ROLE`. Monitoring accounts do not need it: -`node_role()` and `GET /lifecycle` are open to any authenticated principal. See -[Failover and role switch](/docs/high-availability/failover/). - -## Core concepts - - - -### Users and service accounts - -QuestDB has two types of principals: - -- **Users**: For human individuals. Can belong to multiple groups and inherit - permissions from them. Cannot be assumed by others. -- **Service accounts**: For applications. Cannot belong to groups - all - permissions must be granted directly. Can be assumed by authorized users for - testing. - -```questdb-sql -CREATE USER human_user WITH PASSWORD 'pwd'; -CREATE SERVICE ACCOUNT app_account WITH PASSWORD 'pwd'; -``` - -Names must be unique across all users, service accounts, and groups. - -#### Why service accounts? - -Service accounts provide **clean, testable application access**: - -| Aspect | User | Service Account | -| -------------------- | ------------------------------ | ---------------------- | -| Permission source | Direct + inherited from groups | Direct only | -| Can belong to groups | Yes | No | -| Can be assumed (SU) | No | Yes | -| Typical use | Human individuals | Applications, services | - -Because service accounts have no inherited permissions, their access is fully -explicit and predictable. Combined with the ability to assume them, this makes -it easy to verify exactly what an application can and cannot do: - -```questdb-sql --- Create service account with specific permissions -CREATE SERVICE ACCOUNT trading_app WITH PASSWORD 'pwd'; -GRANT ILP TO trading_app; -GRANT INSERT ON trades TO trading_app; -GRANT SELECT ON positions TO trading_app; - --- Developer can assume the service account to test its access -GRANT ASSUME SERVICE ACCOUNT trading_app TO developer; - --- Developer switches to service account context -ASSUME SERVICE ACCOUNT trading_app; --- Now operating with trading_app's exact permissions --- Test what works and what doesn't... -EXIT SERVICE ACCOUNT; -``` - -This makes service accounts ideal for applications where you need predictable, -auditable, and testable access control. - -### Groups - -Groups simplify permission management when multiple users need the same access: - -```questdb-sql -CREATE GROUP analysts; -GRANT SELECT ON ALL TABLES TO analysts; - --- All users added to this group can read all tables -ADD USER alice TO analysts; -ADD USER bob TO analysts; -``` - -Users inherit permissions from their groups. Inherited permissions cannot be -revoked directly from the user - revoke from the group instead. When a group is -dropped, all members lose the permissions they inherited from that group. - -### Authentication methods {#authentication} - - - -QuestDB supports three authentication methods: - -| Method | Use case | Endpoints | -| ------------------ | ------------------------ | ------------------------- | -| **Password** | Interactive users | REST API, PostgreSQL Wire | -| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | -| **REST API Token** | Programmatic REST access | REST API | - -Users can have multiple authentication methods enabled simultaneously: - -```questdb-sql --- Add JWK token for ILP access -ALTER USER sensor_writer CREATE TOKEN TYPE JWK; - --- Add REST API token (with 30-day expiry) -ALTER USER api_user CREATE TOKEN TYPE REST WITH TTL '30d'; -``` - -:::warning - -QuestDB does not store private keys or tokens after creation. Save them -immediately - they cannot be recovered. - -::: - -:::tip - -Authentication should happen via a [secure TLS connection](/docs/security/tls/) -to protect credentials in transit. - -::: - -### Endpoint permissions - -Before a user can connect, they need endpoint permissions: - -| Permission | Allows access to | -| ---------- | -------------------------------------- | -| `HTTP` | REST API, Web Console, ILP over HTTP | -| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | -| `ILP` | InfluxDB Line Protocol TCP (port 9009) | - -```questdb-sql --- Typical setup for an interactive user -GRANT HTTP, PGWIRE TO analyst; - --- Typical setup for an ingestion service -GRANT ILP TO ingest_service; -``` - -### Built-in admin - -Every QuestDB instance starts with a built-in admin account: - -- Default username: `admin` -- Default password: `quest` - -**Change these immediately in production** via `server.conf`: - -```ini -acl.admin.user=your_admin_name -acl.admin.password=your_secure_password -``` - -The built-in admin has irrevocable root access. After creating other admin -users, disable it: - -```ini -acl.admin.user.enabled=false -``` - -In a replicated cluster, keep in mind that the built-in admin authorizes a -[role switch](/docs/high-availability/failover/) from its own credentials, -independently of the replicated access lists. It is the break-glass account -when a `SWITCH ROLE` grant has not yet replicated to the node you need to -promote. - -## Permission levels - -Permissions have different granularities determining where they can be applied: - -| Granularity | Can be granted at | -| ----------- | ------------------------------------- | -| Database | Database only | -| Table | Database or specific tables | -| Column | Database, tables, or specific columns | - -Examples: - -```questdb-sql --- Database-level: applies to all tables -GRANT SELECT ON ALL TABLES TO user; - --- Table-level: applies to specific tables -GRANT SELECT ON trades, prices TO user; - --- Column-level: applies to specific columns -GRANT SELECT ON trades(ts, symbol, price) TO user; -``` - -### The GRANT option - -When granting permissions, you can allow the recipient to grant that permission -to others: - -```questdb-sql -GRANT SELECT ON trades TO team_lead WITH GRANT OPTION; - --- team_lead can now grant SELECT on trades to others -``` - -### Owner permissions {#owner-grants} - -When a user creates a table, they automatically receive all permissions on it -with the GRANT option. This ownership does not persist - if revoked, they cannot -get it back without someone re-granting it. - -## Advanced topics - -### Permission re-adjustment {#permission-level-re-adjustment} - -Database-level permissions include access to future tables. If you revoke access -to one table, QuestDB automatically converts the database-level grant to -individual table-level grants: - -```questdb-sql -GRANT SELECT ON ALL TABLES TO user; -- Database level -REVOKE SELECT ON secret_table FROM user; - --- Result: user now has table-level SELECT on all tables EXCEPT secret_table --- Future tables will NOT be accessible -``` - -The same applies from table to column level: - -```questdb-sql -GRANT SELECT ON trades TO user; -- Table level -REVOKE SELECT ON trades(ssn) FROM user; -- Revoke one column - --- Result: user has column-level SELECT on all columns EXCEPT ssn --- Future columns will NOT be accessible -``` - -:::note - -When dropping a table, permissions on it are preserved by default (useful if -the table is recreated). Use `DROP TABLE ... CASCADE PERMISSIONS` to also -remove all associated permissions. - -::: - -### Implicit timestamp permissions {#implicit-permissions} - -If a user has SELECT or UPDATE on any column of a table, they automatically get -the same permission on the designated timestamp column. This ensures time-series -operations (SAMPLE BY, LATEST ON, etc.) work correctly. - -### Granting on non-existent objects {#grant-verification} - -You can grant permissions on tables/columns that don't exist yet: - -```questdb-sql -GRANT INSERT ON future_table TO app; --- Permission activates when future_table is created -``` - -Use `WITH VERIFICATION` to catch typos: - -```questdb-sql -GRANT SELECT ON trdaes TO user WITH VERIFICATION; --- Fails immediately because 'trdaes' doesn't exist -``` - -### Service account assumption - -Users can temporarily assume a service account's permissions for debugging: - -```questdb-sql --- Grant ability to assume -GRANT ASSUME SERVICE ACCOUNT ingest_app TO developer; - --- Developer can now switch context -ASSUME SERVICE ACCOUNT ingest_app; --- ... debug with app's permissions ... -EXIT SERVICE ACCOUNT; -``` - -## User management reference {#user-management} - -### Creating and removing principals - -```questdb-sql --- Users -CREATE USER username WITH PASSWORD 'pwd'; -DROP USER username; - --- Service accounts -CREATE SERVICE ACCOUNT appname WITH PASSWORD 'pwd'; -DROP SERVICE ACCOUNT appname; - --- Groups -CREATE GROUP groupname; -DROP GROUP groupname; -``` - -### Managing group membership - -```questdb-sql -ADD USER username TO group1, group2; -REMOVE USER username FROM group1; -``` - -### Managing authentication - -```questdb-sql --- Change password -ALTER USER username WITH PASSWORD 'new_pwd'; - --- Remove password (disables password auth) -ALTER USER username WITH NO PASSWORD; - --- Create tokens -ALTER USER username CREATE TOKEN TYPE JWK; -ALTER USER username CREATE TOKEN TYPE REST WITH TTL '30d'; -ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Auto-refresh - --- Remove tokens -ALTER USER username DROP TOKEN TYPE JWK; -ALTER USER username DROP TOKEN TYPE REST; -- Drops all REST tokens -ALTER USER username DROP TOKEN TYPE REST 'token_value_here'; -- Drop specific token -``` - -Removing all authentication methods (password and tokens) effectively disables -the user - they can no longer connect to the database. - -### Viewing information - -```questdb-sql -SHOW USERS; -- List all users -SHOW SERVICE ACCOUNTS; -- List all service accounts -SHOW GROUPS; -- List all groups -SHOW GROUPS username; -- List groups for a user -SHOW USER username; -- Show auth methods for user -SHOW PERMISSIONS username; -- Show permissions for user -``` - -Example output from `SHOW USER`: - -``` -auth_type enabled ---------- ------- -Password true -JWK Token false -REST Token true -``` - -:::note - -Viewing other users' information requires `LIST USERS` (to list all) or -`USER DETAILS` (to see details) permissions. Users can always view their own -information without these permissions. - -::: - -## Permissions reference {#permissions} - -Use `all_permissions()` to see all available permissions: - -```questdb-sql -SELECT * FROM all_permissions(); -``` - -
-Full permissions table (click to expand) - -### Database permissions - -| Permission | Level | Description | -| ------------------------- | ----------------------------------- | --------------------------------------- | -| ADD COLUMN | Database | Table | Add columns to tables | -| ADD INDEX | Database | Table | Column | Add index on symbol columns | -| ALTER COLUMN CACHE | Database | Table | Column | Enable/disable symbol caching | -| ALTER COLUMN TYPE | Database | Table | Column | Change column types | -| ATTACH PARTITION | Database | Table | Attach partitions | -| BACKUP DATABASE | Database | Create database backups | -| CANCEL ANY COPY | Database | Cancel COPY operations | -| CREATE TABLE | Database | Create tables | -| CREATE MATERIALIZED VIEW | Database | Create materialized views | -| CREATE LIVE VIEW | Database | Create live views | -| DEDUP ENABLE | Database | Table | Enable deduplication | -| DEDUP DISABLE | Database | Table | Disable deduplication | -| DETACH PARTITION | Database | Table | Detach partitions | -| DISABLE STORAGE POLICY | Database | Table | Disable storage policies | -| DROP COLUMN | Database | Table | Column | Drop columns | -| DROP INDEX | Database | Table | Column | Drop indexes | -| DROP PARTITION | Database | Table | Drop partitions | -| DROP TABLE | Database | Table | Drop tables | -| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | -| DROP LIVE VIEW | Database | Table | Drop live views | -| ENABLE STORAGE POLICY | Database | Table | Enable storage policies | -| INSERT | Database | Table | Insert data | -| REFRESH MATERIALIZED VIEW | Database | Table | Refresh materialized views | -| REINDEX | Database | Table | Column | Reindex columns | -| REMOVE STORAGE POLICY | Database | Table | Remove storage policies | -| RENAME COLUMN | Database | Table | Column | Rename columns | -| RENAME TABLE | Database | Table | Rename tables | -| RESUME WAL | Database | Table | Resume WAL processing | -| SELECT | Database | Table | Column | Read data | -| SET STORAGE POLICY | Database | Table | Set storage policies | -| SET TABLE PARAM | Database | Table | Set table parameters | -| SET TABLE TYPE | Database | Table | Change table type | -| SETTINGS | Database | Change instance settings in Web Console | -| SNAPSHOT | Database | Create snapshots | -| SQL ENGINE ADMIN | Database | List/cancel running queries | -| SWITCH ROLE | Database | Switch the replication role, read SWITCH STATUS | -| SYSTEM ADMIN | Database | System functions (reload_tls, etc.) | -| TRUNCATE TABLE | Database | Table | Truncate tables | -| UPDATE | Database | Table | Column | Update data | -| VACUUM TABLE | Database | Table | Reclaim storage | - -### User management permissions - -| Permission | Description | -| ---------------------- | --------------------------------------- | -| ADD EXTERNAL ALIAS | Create external group mappings | -| ADD PASSWORD | Set user passwords | -| ADD USER | Add users to groups | -| CREATE GROUP | Create groups | -| CREATE JWK | Create JWK tokens | -| CREATE REST TOKEN | Create REST API tokens | -| CREATE SERVICE ACCOUNT | Create service accounts | -| CREATE USER | Create users | -| DISABLE USER | Disable users | -| DROP GROUP | Drop groups | -| DROP JWK | Drop JWK tokens | -| DROP REST TOKEN | Drop REST API tokens | -| DROP SERVICE ACCOUNT | Drop service accounts | -| DROP USER | Drop users | -| ENABLE USER | Enable users | -| LIST USERS | List users/groups/service accounts | -| REMOVE EXTERNAL ALIAS | Remove external group mappings | -| REMOVE PASSWORD | Remove passwords | -| REMOVE USER | Remove users from groups | -| USER DETAILS | View user/group/service account details | - -### Special permissions - -| Permission | Description | -| -------------- | --------------------------------------------------------------------- | -| ALL | All permissions at the granted level (database/table/column) | -| DATABASE ADMIN | All permissions including future ones; can assume any service account | - -A few operations are reserved for database administrators and are not grantable as permissions at all: - -| Operation | Notes | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | -| [`ALTER TABLE REBASE WAL`](/docs/query/sql/alter-table-rebase-wal/) | Rebuilds a suspended WAL table under a fresh sequencer | -| [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) | Moves the [cold storage](/docs/concepts/cold-storage/) manager role, including with `FORCE` | - -By contrast, [`SWITCH ROLE`](/docs/query/sql/switch-role/), which moves the -replication role, is an ordinary grantable permission. - -
- -## SQL commands reference - -- [ADD USER](/docs/query/sql/acl/add-user/) -- [ALTER USER](/docs/query/sql/acl/alter-user/) -- [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) -- [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) -- [CREATE GROUP](/docs/query/sql/acl/create-group/) -- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) -- [CREATE USER](/docs/query/sql/acl/create-user/) -- [DROP GROUP](/docs/query/sql/acl/drop-group/) -- [DROP SERVICE ACCOUNT](/docs/query/sql/acl/drop-service-account/) -- [DROP USER](/docs/query/sql/acl/drop-user/) -- [EXIT SERVICE ACCOUNT](/docs/query/sql/acl/exit-service-account/) -- [GRANT](/docs/query/sql/acl/grant/) -- [GRANT ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/grant-assume-service-account/) -- [REMOVE USER](/docs/query/sql/acl/remove-user/) -- [REVOKE](/docs/query/sql/acl/revoke/) -- [REVOKE ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/revoke-assume-service-account/) -- [SHOW USER](/docs/query/sql/show/#show-user) -- [SHOW USERS](/docs/query/sql/show/#show-users) -- [SHOW GROUPS](/docs/query/sql/show/#show-groups) -- [SHOW SERVICE ACCOUNT](/docs/query/sql/show/#show-service-account) -- [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) -- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) diff --git a/documentation/security/rbac/authentication.mdx b/documentation/security/rbac/authentication.mdx new file mode 100644 index 0000000000..3ae5f1e784 --- /dev/null +++ b/documentation/security/rbac/authentication.mdx @@ -0,0 +1,140 @@ +--- +title: RBAC authentication and endpoint access +sidebar_label: Authentication and endpoints +description: + The four authentication methods QuestDB Enterprise accepts, the HTTP, PGWIRE + and ILP endpoint permissions that gate each protocol, and how to manage + passwords and tokens. +--- + +import Screenshot from "@theme/Screenshot" + +Connecting to QuestDB Enterprise takes two things: a credential the server +accepts, and the endpoint permission for the protocol you are connecting over. A +valid password is not enough on its own. + +## Authentication methods {#authentication} + + + +QuestDB supports four authentication methods: + +| Method | Use case | Endpoints | +| --------------------- | --------------------------- | -------------------------------------------------------------------------------- | +| **Password** | Interactive users | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire | +| **JWK Token** | ILP ingestion | InfluxDB Line Protocol | +| **REST API Token** | Programmatic HTTP access | REST API and QWP, both under the `HTTP` permission | +| **OIDC bearer token** | SSO users, external clients | REST API and QWP, both under the `HTTP` permission; PostgreSQL Wire once enabled | + +The first three are QuestDB's own credentials, created with the statements +below. An OIDC bearer token is issued by an external Identity Provider instead, +and the user's group memberships come from the token or the provider's user info +endpoint; see [OpenID Connect](/docs/security/oidc/) and +[client integration patterns](/docs/security/oidc/client-integration/), including +[which token to send](/docs/security/oidc/client-discovery/#which-token-to-send). + +OIDC on the PostgreSQL Wire endpoint is off by default. Enable it with +[`acl.oidc.pg.token.as.password.enabled`](/docs/configuration/oidc/#acloidcpgtokenaspasswordenabled), +which accepts the token in the password field, or with +[`acl.oidc.ropc.flow.enabled`](/docs/configuration/oidc/#acloidcropcflowenabled), +which has QuestDB exchange the user's SSO credentials for a token itself. + +Users can have multiple authentication methods enabled simultaneously: + +```questdb-sql +-- Add JWK token for ILP access +ALTER USER sensor_writer CREATE TOKEN TYPE JWK; + +-- Add REST API token (with 30-day expiry) +ALTER USER api_user CREATE TOKEN TYPE REST WITH TTL '30d'; +``` + +:::warning + +QuestDB does not store private keys or tokens after creation. Save them +immediately - they cannot be recovered. + +::: + +:::tip + +Authentication should happen via a [secure TLS connection](/docs/security/tls/) +to protect credentials in transit. + +::: + +## Endpoint permissions {#endpoint-permissions} + +Before a user can connect, they need endpoint permissions: + +| Permission | Allows access to | +| ---------- | ----------------------------------------- | +| `HTTP` | REST API, Web Console, QWP, ILP over HTTP | +| `PGWIRE` | PostgreSQL Wire Protocol (port 8812) | +| `ILP` | InfluxDB Line Protocol TCP (port 9009) | + +```questdb-sql +-- Typical setup for an interactive user +GRANT HTTP, PGWIRE TO analyst; + +-- Typical setup for an ingestion service +GRANT ILP TO ingest_service; +``` + +Endpoint permissions are independent of data permissions. A user may hold +`SELECT` on every table and still be unable to connect, or hold `ILP` and be +able to ingest but never query. + +## Managing authentication {#managing-authentication} + +For a user: + +```questdb-sql title="Manage user authentication" +-- Change password +ALTER USER username WITH PASSWORD 'new_pwd'; + +-- Remove password (disables password auth) +ALTER USER username WITH NO PASSWORD; + +-- Create tokens +ALTER USER username CREATE TOKEN TYPE JWK; +ALTER USER username CREATE TOKEN TYPE REST WITH TTL '30d'; +ALTER USER username CREATE TOKEN TYPE REST WITH TTL '1d' REFRESH; -- Sliding expiry extension + +-- Remove tokens +ALTER USER username DROP TOKEN TYPE JWK; +ALTER USER username DROP TOKEN TYPE REST; -- Drops all REST tokens +ALTER USER username DROP TOKEN TYPE REST 'token_value_here'; -- Drop specific token +``` + +Use the parallel `ALTER SERVICE ACCOUNT` forms for an application identity: + +```questdb-sql title="Manage service-account authentication" +ALTER SERVICE ACCOUNT service_account WITH PASSWORD 'new_pwd'; +ALTER SERVICE ACCOUNT service_account WITH NO PASSWORD; + +ALTER SERVICE ACCOUNT service_account CREATE TOKEN TYPE JWK; +ALTER SERVICE ACCOUNT service_account CREATE TOKEN TYPE REST WITH TTL '30d'; + +ALTER SERVICE ACCOUNT service_account DROP TOKEN TYPE JWK; +ALTER SERVICE ACCOUNT service_account DROP TOKEN TYPE REST; +``` + +See the [ALTER USER](/docs/query/sql/acl/alter-user/) and +[ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) references +for token refresh, expiry, and targeted REST-token removal. Removing every +authentication method effectively disables that user or service account: it can +no longer connect to the database. + +## See also + +- [OpenID Connect](/docs/security/oidc/) - SSO against an external Identity + Provider +- [TLS](/docs/security/tls/) - encrypting credentials in transit +- [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) - + the principals these credentials belong to diff --git a/documentation/security/rbac/common-scenarios.mdx b/documentation/security/rbac/common-scenarios.mdx new file mode 100644 index 0000000000..55f5714f12 --- /dev/null +++ b/documentation/security/rbac/common-scenarios.mdx @@ -0,0 +1,139 @@ +--- +title: RBAC common scenarios +sidebar_label: Common scenarios +description: + Worked QuestDB Enterprise permission setups for read-only analysts, ingestion + service accounts, teams, column and row restrictions, database administrators, + and failover operators. +--- + +Each scenario below is a complete permission setup. The examples use the +`trades(timestamp, symbol, side, price, amount)` table from the +[Create your first database guide](/docs/getting-started/create-database/). +Adapt the principal and table names; the shape of the grants is the part worth +copying. + +## Read-only analyst + +A user who can query data but cannot modify anything: + +```questdb-sql +CREATE USER analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO analyst; +GRANT SELECT ON ALL TABLES TO analyst; +``` + +## Application service account + +A service account for an application that ingests data into specific tables: + +```questdb-sql title="Create an ILP/TCP service account" +CREATE SERVICE ACCOUNT ingest_app; +ALTER SERVICE ACCOUNT ingest_app CREATE TOKEN TYPE JWK; +GRANT ILP TO ingest_app; -- InfluxDB Line Protocol access +GRANT INSERT ON trades TO ingest_app; -- Can only insert into trades +``` + +:::warning + +Save the private key returned by `ALTER SERVICE ACCOUNT` when the token is +created. QuestDB does not store the private key, so it cannot be recovered +later. Configure the ILP/TCP client with the `ingest_app` account name and that +private key. See [RBAC authentication](/docs/security/rbac/authentication/). + +::: + +## Team-based access with groups + +Multiple users sharing the same permissions: + +```questdb-sql +-- Create a group +CREATE GROUP trading_team; + +-- Grant permissions to the group +GRANT HTTP, PGWIRE TO trading_team; +GRANT SELECT ON trades TO trading_team; +GRANT INSERT ON trades TO trading_team; + +-- Add users to the group - they inherit all permissions +CREATE USER market_analyst WITH PASSWORD 'pwd1'; +CREATE USER risk_analyst WITH PASSWORD 'pwd2'; +ADD USER market_analyst TO trading_team; +ADD USER risk_analyst TO trading_team; +``` + +## Column-level restrictions (hide sensitive data) + +Allow access to selected fields while hiding columns your policy treats as +sensitive: + +```questdb-sql +CREATE USER auditor WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO auditor; + +-- Grant access to approved columns only +GRANT SELECT ON trades(timestamp, symbol, price) TO auditor; +-- Columns side and amount are not granted = invisible to auditor +``` + +## Row-level security with views {#row-level-security-multi-tenant} + +Different users see different subsets of data: + +```questdb-sql +-- Create views over different subsets of the trades table +CREATE VIEW eth_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'ETH-USDT' +); +CREATE VIEW btc_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'BTC-USDT' +); + +-- Grant users access to their subset only +CREATE USER eth_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO eth_analyst; +GRANT SELECT ON eth_usdt_trades TO eth_analyst; + +CREATE USER btc_analyst WITH PASSWORD 'pwd'; +GRANT HTTP, PGWIRE TO btc_analyst; +GRANT SELECT ON btc_usdt_trades TO btc_analyst; +``` + +## Database administrator + +A user with full control (but not the built-in admin): + +```questdb-sql +CREATE USER dba WITH PASSWORD 'pwd'; +GRANT DATABASE ADMIN TO dba; +``` + +:::warning + +`DATABASE ADMIN` grants all current and future permissions. Use sparingly. + +::: + +## Failover operator + +A service account that can move the primary role between nodes, for an external +coordinator or a runbook, without any other administrative right: + +```questdb-sql +CREATE SERVICE ACCOUNT failover_bot WITH PASSWORD 'pwd'; +GRANT HTTP TO failover_bot; -- POST /lifecycle/switch on port 9003 +GRANT SWITCH ROLE TO failover_bot; -- SWITCH ROLE, SWITCH STATUS, the endpoint +``` + +`SYSTEM ADMIN` is neither required nor sufficient for a role switch, and +`DATABASE ADMIN` includes `SWITCH ROLE`. Monitoring accounts do not need it: +`node_role()` and `GET /lifecycle` are open to any authenticated principal. See +[Failover and role switch](/docs/high-availability/failover/). + +## See also + +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + the semantics behind these grants +- [Permissions reference](/docs/security/rbac/permissions-reference/) - every + permission available diff --git a/documentation/security/rbac/granting-permissions.mdx b/documentation/security/rbac/granting-permissions.mdx new file mode 100644 index 0000000000..bbc2dfa47e --- /dev/null +++ b/documentation/security/rbac/granting-permissions.mdx @@ -0,0 +1,186 @@ +--- +title: Granting and revoking permissions +sidebar_label: Granting permissions +description: + How QuestDB Enterprise permissions apply at database, table and column level, + what GRANT ALL and WITH GRANT OPTION really do, and how a grant changes level + when you revoke part of it. +--- + +Permissions are granted with [`GRANT`](/docs/query/sql/acl/grant/) and removed +with [`REVOKE`](/docs/query/sql/acl/revoke/). This page covers the level a grant +applies at, the semantics that are easy to get wrong, and the behaviour that +surprises people the first time they hit it. + +For the list of permissions themselves, see the +[permissions reference](/docs/security/rbac/permissions-reference/). + +The examples use the `trades(timestamp, symbol, side, price, amount)` table from +the [Create your first database guide](/docs/getting-started/create-database/). + +## Permission levels + +Permissions have different granularities determining where they can be applied: + +| Granularity | Can be granted at | +| ----------- | ------------------------------------- | +| Database | Database only | +| Table | Database or specific tables | +| Column | Database, tables, or specific columns | + +Examples: + +```questdb-sql +-- Database-level: applies to all tables +GRANT SELECT ON ALL TABLES TO user; + +-- Table-level: applies to a specific table +GRANT SELECT ON trades TO user; + +-- Column-level: applies to specific columns +GRANT SELECT ON trades(timestamp, symbol, price) TO user; +``` + +### Column-level access + +Restrict users to see only certain columns: + +```questdb-sql +-- User can only see timestamp and price, not symbol, side, or amount +GRANT SELECT ON trades(timestamp, price) TO analyst; +``` + +To grant on every column except a few, use the `*` wildcard with an `EXCLUDE` +list: + +```questdb-sql +-- User can see all columns except side +GRANT SELECT ON trades(* EXCLUDE (side)) TO analyst; +``` + +The wildcard covers the columns that exist when the statement runs, not columns +added later. See +[GRANT](/docs/query/sql/acl/grant/#grant-on-all-columns-of-a-table) for details. + +### Row-level access with views + +For row-level security, create a [view](/docs/concepts/views/) that filters +rows, then grant access to the view instead of the underlying table: + +```questdb-sql +-- Create a view that only shows ETH-USDT trades +CREATE VIEW eth_usdt_trades AS ( + SELECT * FROM trades WHERE symbol = 'ETH-USDT' +); + +-- Grant access to the view, not the base table +GRANT SELECT ON eth_usdt_trades TO eth_analyst; +-- No GRANT on trades table = user cannot see other symbols +``` + +The user `eth_analyst` can only see `ETH-USDT` trades. They have no access to +the underlying `trades` table. + +Creating a view needs the `CREATE VIEW` permission; dropping or redefining one +needs `DROP VIEW` or `ALTER VIEW` on that view. + +## Granting ALL {#granting-all} + +`ALL` expands to every permission valid at the level you grant it, and at the +database level that set includes `DATABASE ADMIN`: + +```questdb-sql +-- Makes database_admin a full database administrator +GRANT ALL TO database_admin; + +-- Every permission on one table only +GRANT ALL ON trades TO trade_manager; +``` + +:::warning + +`GRANT ALL TO ` with no table or column list is an administrator grant, +not a broad data grant. Scope it to tables, or name the permissions you mean. + +::: + +## The GRANT option + +When granting permissions, you can allow the recipient to grant that permission +to others: + +```questdb-sql +GRANT SELECT ON trades TO team_lead WITH GRANT OPTION; + +-- team_lead can now grant SELECT on trades to others +``` + +## Owner permissions {#owner-grants} + +When a user creates a table, they automatically receive all permissions on it +with the GRANT option. This ownership does not persist - if revoked, they cannot +get it back without someone re-granting it. + +## Permission re-adjustment {#permission-level-re-adjustment} + +Database-level permissions include access to future tables. If you revoke access +to one table, QuestDB automatically converts the database-level grant to +individual table-level grants: + +```questdb-sql +GRANT SELECT ON ALL TABLES TO user; -- Database level +REVOKE SELECT ON secret_table FROM user; + +-- Result: user now has table-level SELECT on all tables EXCEPT secret_table +-- Future tables will NOT be accessible +``` + +The same applies from table to column level: + +```questdb-sql +GRANT SELECT ON trades TO user; -- Table level +REVOKE SELECT ON trades(side) FROM user; -- Revoke one column + +-- Result: user has column-level SELECT on all columns EXCEPT side +-- Future columns will NOT be accessible +``` + +:::note + +Dropping a table removes every permission granted on it, from every user and +group. Recreating a table with the same name does not bring them back, so a +drop-and-recreate has to re-grant them. The same applies to views, materialized +views and live views. + +::: + +## Implicit timestamp permissions {#implicit-permissions} + +If a user has SELECT or UPDATE on any column of a table, they automatically get +the same permission on the designated timestamp column. This ensures time-series +operations (SAMPLE BY, LATEST ON, etc.) work correctly. + +## Granting on non-existent objects {#grant-verification} + +You can grant permissions on tables/columns that don't exist yet: + +```questdb-sql +GRANT INSERT ON future_table TO app; +-- Permission activates when future_table is created +``` + +Use `WITH VERIFICATION` to catch typos: + +```questdb-sql +GRANT SELECT ON trdaes TO user WITH VERIFICATION; +-- Fails immediately because 'trdaes' doesn't exist +``` + +## See also + +- [Permissions reference](/docs/security/rbac/permissions-reference/) - every + permission and the level it can be granted at +- [Common scenarios](/docs/security/rbac/common-scenarios/) - worked grants for + analysts, service accounts, teams and admins +- [`GRANT`](/docs/query/sql/acl/grant/) and + [`REVOKE`](/docs/query/sql/acl/revoke/) statement syntax diff --git a/documentation/security/rbac/index.mdx b/documentation/security/rbac/index.mdx new file mode 100644 index 0000000000..1617559a65 --- /dev/null +++ b/documentation/security/rbac/index.mdx @@ -0,0 +1,164 @@ +--- +title: Role-based Access Control (RBAC) +sidebar_label: Overview +slug: /security/rbac +description: + Granular access control from database level down to individual columns and + rows. Learn how to secure your QuestDB instance with users, groups, and + fine-grained permissions. +--- + +import { EnterpriseNote } from "@site/src/components/EnterpriseNote" + + + Role-based Access Control (RBAC) provides fine-grained permissions for your + QuestDB instance. + + +QuestDB Enterprise provides fine-grained access control that can restrict access +at **database**, **table**, **column**, and even **row** level (using views). + +## Quick start + +Here's a complete example to create a read-only analyst user in under a minute: +It assumes the `trades` table from the +[Create your first database guide](/docs/getting-started/create-database/). + +```questdb-sql +-- 1. Create the user +CREATE USER analyst WITH PASSWORD 'secure_password_here'; + +-- 2. Grant endpoint access (required to connect) +GRANT HTTP, PGWIRE TO analyst; + +-- 3. Grant read access to specific tables +GRANT SELECT ON trades TO analyst; + +-- Done! The analyst can now connect and query the trades table +``` + +To verify: + +```questdb-sql +SHOW PERMISSIONS analyst; +``` + +## Access control depth + +QuestDB's access control operates across two dimensions: + +### Data access granularity + +Control _what data_ users can access: + +| Level | What you can control | Example | +| ------------ | ------------------------------- | ----------------------------------------------------- | +| **Database** | All tables, global operations | `GRANT SELECT ON ALL TABLES TO user` | +| **Table** | Specific tables | `GRANT SELECT ON trades TO user` | +| **Column** | Specific columns within a table | `GRANT SELECT ON trades(timestamp, price) TO user` | +| **Row** | Specific rows via views | Create a view with WHERE clause, grant access to view | + +### Connection access granularity + +Control _how_ users can connect: + +| Permission | Protocol | Use case | +| ---------- | ------------------------------------ | ------------------------------------------ | +| `HTTP` | REST API, Web Console, QWP, ILP/HTTP | Interactive users, web applications | +| `PGWIRE` | PostgreSQL Wire Protocol | SQL clients, BI tools, programmatic access | +| `ILP` | InfluxDB Line Protocol (TCP) | High-throughput data ingestion | + +```questdb-sql +-- User can connect via PostgreSQL protocol only (not web console) +GRANT PGWIRE TO analyst; + +-- Service can only ingest via ILP, cannot query +GRANT ILP TO ingest_service; + +-- Full interactive access +GRANT HTTP, PGWIRE TO developer; +``` + +These dimensions are independent: a user might have `SELECT` on all tables but +only be allowed to connect via `PGWIRE`, or have `INSERT` permission but only +via `ILP`. + +## Where to go next + +| Page | Covers | +| ------------------------------------------------------------------------------ | ------------------------------------------------------------------------------- | +| [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) | The three principal types, and the statements that manage them | +| [Authentication and endpoint access](/docs/security/rbac/authentication/) | Passwords, JWK and REST tokens, OIDC, and the `HTTP`/`PGWIRE`/`ILP` permissions | +| [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) | Permission levels, `GRANT ALL`, the grant option, and level re-adjustment | +| [Permissions reference](/docs/security/rbac/permissions-reference/) | Every permission and the level it can be granted at | +| [Common scenarios](/docs/security/rbac/common-scenarios/) | Analysts, service accounts, teams, admins, failover operators | +| [OpenID Connect](/docs/security/oidc/) | SSO against an external Identity Provider | + +## SQL commands reference + +- [ADD USER](/docs/query/sql/acl/add-user/) +- [ALTER GROUP](/docs/query/sql/acl/alter-group/) +- [ALTER SERVICE ACCOUNT](/docs/query/sql/acl/alter-service-account/) +- [ALTER USER](/docs/query/sql/acl/alter-user/) +- [ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/assume-service-account/) +- [CREATE GROUP](/docs/query/sql/acl/create-group/) +- [CREATE SERVICE ACCOUNT](/docs/query/sql/acl/create-service-account/) +- [CREATE USER](/docs/query/sql/acl/create-user/) +- [DROP GROUP](/docs/query/sql/acl/drop-group/) +- [DROP SERVICE ACCOUNT](/docs/query/sql/acl/drop-service-account/) +- [DROP USER](/docs/query/sql/acl/drop-user/) +- [EXIT SERVICE ACCOUNT](/docs/query/sql/acl/exit-service-account/) +- [GRANT](/docs/query/sql/acl/grant/) +- [GRANT ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/grant-assume-service-account/) +- [REMOVE USER](/docs/query/sql/acl/remove-user/) +- [REVOKE](/docs/query/sql/acl/revoke/) +- [REVOKE ASSUME SERVICE ACCOUNT](/docs/query/sql/acl/revoke-assume-service-account/) +- [SHOW GROUPS](/docs/query/sql/show/#show-groups) +- [SHOW PERMISSIONS](/docs/query/sql/show/#show-permissions-for-current-user) +- [SHOW SERVICE ACCOUNT](/docs/query/sql/show/#show-service-account) +- [SHOW SERVICE ACCOUNTS](/docs/query/sql/show/#show-service-accounts) +- [SHOW USER](/docs/query/sql/show/#show-user) +- [SHOW USERS](/docs/query/sql/show/#show-users) + +## Where each section moved {#moved} + +These anchors are kept so that existing links keep working. Prefer the pages +above when adding new links. + +| Section | Now at | +| ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- | +| Column-level access | [Granting permissions](/docs/security/rbac/granting-permissions/#column-level-access) | +| Row-level access with views | [Granting permissions](/docs/security/rbac/granting-permissions/#row-level-access-with-views) | +| Common scenarios | [Common scenarios](/docs/security/rbac/common-scenarios/) | +| Read-only analyst | [Common scenarios](/docs/security/rbac/common-scenarios/#read-only-analyst) | +| Application service account | [Common scenarios](/docs/security/rbac/common-scenarios/#application-service-account) | +| Team-based access with groups | [Common scenarios](/docs/security/rbac/common-scenarios/#team-based-access-with-groups) | +| Column-level restrictions | [Common scenarios](/docs/security/rbac/common-scenarios/#column-level-restrictions-hide-sensitive-data) | +| Row-level security | [Common scenarios](/docs/security/rbac/common-scenarios/#row-level-security-multi-tenant) | +| Database administrator | [Common scenarios](/docs/security/rbac/common-scenarios/#database-administrator) | +| Failover operator | [Common scenarios](/docs/security/rbac/common-scenarios/#failover-operator) | +| Core concepts | [Users, service accounts, and groups](/docs/security/rbac/users-and-groups/) | +| Users and service accounts | [Users and groups](/docs/security/rbac/users-and-groups/#users-and-service-accounts) | +| Why service accounts? | [Users and groups](/docs/security/rbac/users-and-groups/#why-service-accounts) | +| Groups | [Users and groups](/docs/security/rbac/users-and-groups/#groups) | +| Authentication methods | [Authentication](/docs/security/rbac/authentication/#authentication) | +| Endpoint permissions | [Authentication](/docs/security/rbac/authentication/#endpoint-permissions) | +| Built-in admin | [Users and groups](/docs/security/rbac/users-and-groups/#built-in-admin) | +| Service account assumption | [Users and groups](/docs/security/rbac/users-and-groups/#service-account-assumption) | +| Creating and removing principals | [Users and groups](/docs/security/rbac/users-and-groups/#creating-and-removing-principals) | +| Managing group membership | [Users and groups](/docs/security/rbac/users-and-groups/#managing-group-membership) | +| Viewing information | [Users and groups](/docs/security/rbac/users-and-groups/#viewing-information) | +| Managing authentication | [Authentication](/docs/security/rbac/authentication/#managing-authentication) | +| User management reference | [Users and groups](/docs/security/rbac/users-and-groups/#user-management) | +| Permission levels | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-levels) | +| Granting ALL | [Granting permissions](/docs/security/rbac/granting-permissions/#granting-all) | +| The GRANT option | [Granting permissions](/docs/security/rbac/granting-permissions/#the-grant-option) | +| Owner permissions | [Granting permissions](/docs/security/rbac/granting-permissions/#owner-grants) | +| Advanced topics | [Granting permissions](/docs/security/rbac/granting-permissions/) | +| Permission re-adjustment | [Granting permissions](/docs/security/rbac/granting-permissions/#permission-level-re-adjustment) | +| Implicit timestamp permissions | [Granting permissions](/docs/security/rbac/granting-permissions/#implicit-permissions) | +| Granting on non-existent objects | [Granting permissions](/docs/security/rbac/granting-permissions/#grant-verification) | +| Permissions reference | [Permissions reference](/docs/security/rbac/permissions-reference/) | +| Database permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#database-permissions) | +| User management permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#user-management-permissions) | +| Special permissions | [Permissions reference](/docs/security/rbac/permissions-reference/#special-permissions) | diff --git a/documentation/security/rbac/permissions-reference.mdx b/documentation/security/rbac/permissions-reference.mdx new file mode 100644 index 0000000000..4f2b9f10f1 --- /dev/null +++ b/documentation/security/rbac/permissions-reference.mdx @@ -0,0 +1,138 @@ +--- +title: QuestDB permissions reference +sidebar_label: Permissions reference +description: + Every permission QuestDB Enterprise supports, the level each can be granted + at, and the operations reserved for SYSTEM ADMIN. +--- + +Every permission QuestDB Enterprise supports, and the level each one can be +granted at. To see the same list from a running instance, including any +permission added after this page was written: + +```questdb-sql +SELECT * FROM all_permissions(); +``` + +## Database permissions + +| Permission | Level | Description | +| ---------------------------- | ----------------------------------- | -------------------------------------------------------------- | +| ADD COLUMN | Database | Table | Add columns to tables | +| ADD INDEX | Database | Table | Column | Add index on symbol columns | +| ALTER COLUMN CACHE | Database | Table | Column | Enable/disable symbol caching | +| ALTER COLUMN TYPE | Database | Table | Column | Change column types | +| ALTER SYMBOL CAPACITY | Database | Table | Column | Change the capacity of a symbol column | +| ALTER VIEW | Database | Table | Change a view definition | +| ATTACH PARTITION | Database | Table | Attach partitions | +| BACKUP DATABASE | Database | Create database backups | +| BACKUP TABLE | Database | Table | Legacy, grants nothing: there is no `BACKUP TABLE` statement | +| CANCEL ANY COPY | Database | Cancel COPY operations | +| COMPILE VIEW | Database | Table | Recompile a view against the current schema | +| CONVERT PARTITION TO NATIVE | Database | Table | Convert Parquet partitions back to native format | +| CONVERT PARTITION TO PARQUET | Database | Table | Convert partitions to Parquet in place | +| CREATE LIVE VIEW | Database | Create live views | +| CREATE MATERIALIZED VIEW | Database | Create materialized views | +| CREATE TABLE | Database | Create tables | +| CREATE VIEW | Database | Create views | +| DEDUP DISABLE | Database | Table | Disable deduplication | +| DEDUP ENABLE | Database | Table | Enable deduplication | +| DETACH PARTITION | Database | Table | Detach partitions | +| DISABLE STORAGE POLICY | Database | Table | Disable storage policies | +| DROP COLUMN | Database | Table | Column | Drop columns | +| DROP INDEX | Database | Table | Column | Drop indexes | +| DROP LIVE VIEW | Database | Table | Drop live views | +| DROP MATERIALIZED VIEW | Database | Table | Drop materialized views | +| DROP PARTITION | Database | Table | Drop partitions | +| DROP TABLE | Database | Table | Drop tables | +| DROP VIEW | Database | Table | Drop views | +| ENABLE STORAGE POLICY | Database | Table | Enable storage policies | +| INSERT | Database | Table | Insert data | +| REFRESH MATERIALIZED VIEW | Database | Table | Refresh materialized views | +| REINDEX | Database | Table | Column | Reindex columns | +| REMOVE STORAGE POLICY | Database | Table | Remove storage policies | +| RENAME COLUMN | Database | Table | Column | Rename columns | +| RENAME TABLE | Database | Table | Rename tables | +| RESUME WAL | Database | Table | Resume WAL processing | +| SELECT | Database | Table | Column | Read data | +| SET PARQUET SETTINGS | Database | Table | Set per-column Parquet encoding, compression and bloom filters | +| SET REFRESH LIMIT | Database | Table | Set a materialized view's refresh limit | +| SET REFRESH TYPE | Database | Table | Change a materialized view's refresh strategy | +| SET STORAGE POLICY | Database | Table | Set storage policies | +| SET TABLE FORMAT | Database | Table | Switch a table's partition format between native and Parquet | +| SET TABLE PARAM | Database | Table | Set table parameters | +| SET TABLE TYPE | Database | Table | Change table type | +| SETTINGS | Database | Change instance settings in Web Console | +| SNAPSHOT | Database | Create snapshots | +| SQL ENGINE ADMIN | Database | List/cancel running queries | +| SWITCH ROLE | Database | Switch the replication role, read SWITCH STATUS | +| SYSTEM ADMIN | Database | System functions (reload_tls, etc.) | +| TRUNCATE TABLE | Database | Table | Truncate tables | +| UPDATE | Database | Table | Column | Update data | +| VACUUM TABLE | Database | Table | Reclaim storage | + +## Endpoint permissions {#endpoint-permissions} + +These control which protocol a principal may connect on. They are covered in +detail under +[authentication and endpoint access](/docs/security/rbac/authentication/#endpoint-permissions). + +| Permission | Level | Description | +| ---------- | -------- | ----------------------------------------- | +| HTTP | Database | REST API, Web Console, QWP, ILP over HTTP | +| ILP | Database | InfluxDB Line Protocol TCP (port 9009) | +| PGWIRE | Database | PostgreSQL Wire Protocol (port 8812) | + +## User management permissions + +| Permission | Description | +| ---------------------- | --------------------------------------- | +| ADD EXTERNAL ALIAS | Create external group mappings | +| ADD PASSWORD | Set user passwords | +| ADD USER | Add users to groups | +| CREATE GROUP | Create groups | +| CREATE JWK | Create JWK tokens | +| CREATE REST TOKEN | Create REST API tokens | +| CREATE SERVICE ACCOUNT | Create service accounts | +| CREATE USER | Create users | +| DISABLE USER | Disable users | +| DROP GROUP | Drop groups | +| DROP JWK | Drop JWK tokens | +| DROP REST TOKEN | Drop REST API tokens | +| DROP SERVICE ACCOUNT | Drop service accounts | +| DROP USER | Drop users | +| ENABLE USER | Enable users | +| LIST USERS | List users/groups/service accounts | +| REMOVE EXTERNAL ALIAS | Remove external group mappings | +| REMOVE PASSWORD | Remove passwords | +| REMOVE USER | Remove users from groups | +| USER DETAILS | View user/group/service account details | + +## Special permissions + +| Permission | Description | +| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| ALL | All permissions at the granted level (database/table/column). At the database level this includes `DATABASE ADMIN` - see [granting ALL](/docs/security/rbac/granting-permissions/#granting-all) | +| DATABASE ADMIN | All permissions including future ones; can assume any service account | + +A few destructive operations have no permission of their own. They are gated by +`SYSTEM ADMIN`, so the only way to grant them is to grant `SYSTEM ADMIN`, which +also carries every other system function: + +| Operation | Gated by | Notes | +| ----------------------------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------ | +| [`ALTER TABLE REBASE WAL`](/docs/query/sql/alter-table-rebase-wal/) | `SYSTEM ADMIN` | Rebuilds a suspended WAL table under a fresh sequencer. A table-level `RESUME WAL` grant is not enough | +| [`SWITCH COLD STORAGE ROLE`](/docs/query/sql/switch-cold-storage-role/) | `SYSTEM ADMIN` | Moves the [cold storage](/docs/concepts/cold-storage/) manager role, including with `FORCE` | + +By contrast, [`SWITCH ROLE`](/docs/query/sql/switch-role/), which moves the +replication role, has its own dedicated permission that can be granted without +handing over the rest of `SYSTEM ADMIN`. + +## See also + +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + levels, `WITH GRANT OPTION`, and how grants shift level when you revoke +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + the `HTTP`, `PGWIRE` and `ILP` permissions in context +- [`GRANT`](/docs/query/sql/acl/grant/) and + [`REVOKE`](/docs/query/sql/acl/revoke/) statement syntax diff --git a/documentation/security/rbac/users-and-groups.mdx b/documentation/security/rbac/users-and-groups.mdx new file mode 100644 index 0000000000..2dcbf5b5e8 --- /dev/null +++ b/documentation/security/rbac/users-and-groups.mdx @@ -0,0 +1,198 @@ +--- +title: Users, service accounts, and groups +sidebar_label: Users and groups +description: + The three principal types in QuestDB Enterprise RBAC - users, service + accounts, and groups - how they differ, and the statements that create and + manage them. +--- + +import Screenshot from "@theme/Screenshot" + +Every permission in QuestDB is granted to a principal. This page covers the +three kinds, how they differ, and the statements that manage them. + + + +## Users and service accounts + +QuestDB has two account types: + +- **Users**: For human individuals. Can belong to multiple groups and inherit + permissions from them. Cannot be assumed by others. +- **Service accounts**: For applications. Cannot belong to groups - all + permissions must be granted directly. Can be assumed by authorized users for + testing. + +```questdb-sql +CREATE USER human_user WITH PASSWORD 'pwd'; +CREATE SERVICE ACCOUNT app_account WITH PASSWORD 'pwd'; +``` + +Names must be unique across all users, service accounts, and groups. + +### Why service accounts? + +Service accounts provide **clean, testable application access**: + +| Aspect | User | Service Account | +| -------------------- | ------------------------------ | ---------------------- | +| Permission source | Direct + inherited from groups | Direct only | +| Can belong to groups | Yes | No | +| Can be assumed (SU) | No | Yes | +| Typical use | Human individuals | Applications, services | + +Because service accounts have no inherited permissions, their access is fully +explicit and predictable. Combined with the ability to assume them, this makes +it easy to verify exactly what an application can and cannot do: + +```questdb-sql +-- Create service account with specific permissions +CREATE SERVICE ACCOUNT trading_app WITH PASSWORD 'pwd'; +GRANT ILP TO trading_app; +GRANT INSERT ON trades TO trading_app; +GRANT SELECT ON positions TO trading_app; + +-- Developer can assume the service account to test its access +GRANT ASSUME SERVICE ACCOUNT trading_app TO developer; + +-- Developer switches to service account context +ASSUME SERVICE ACCOUNT trading_app; +-- Now operating with trading_app's exact permissions +-- Test what works and what doesn't... +EXIT SERVICE ACCOUNT; +``` + +This makes service accounts ideal for applications where you need predictable, +auditable, and testable access control. + +### Service account assumption + +Users can temporarily assume a service account's permissions for debugging: + +```questdb-sql +-- Grant ability to assume +GRANT ASSUME SERVICE ACCOUNT ingest_app TO developer; + +-- Developer can now switch context +ASSUME SERVICE ACCOUNT ingest_app; +-- ... debug with app's permissions ... +EXIT SERVICE ACCOUNT; +``` + +## Groups + +Groups simplify permission management when multiple users need the same access: + +```questdb-sql +CREATE GROUP analysts; +GRANT SELECT ON ALL TABLES TO analysts; + +-- All users added to this group can read all tables +ADD USER market_analyst TO analysts; +ADD USER risk_analyst TO analysts; +``` + +Users inherit permissions from their groups. Inherited permissions cannot be +revoked directly from the user - revoke from the group instead. When a group is +dropped, all members lose the permissions they inherited from that group. + +For OIDC users, group membership comes from the Identity Provider rather than +from `ADD USER`. See +[mapping groups and permissions](/docs/security/oidc/group-mapping/). + +## Built-in admin + +Every QuestDB instance starts with a built-in admin account: + +- Default username: `admin` +- Default password: `quest` + +**Change these immediately in production** via `server.conf`: + +```ini +acl.admin.user=your_admin_name +acl.admin.password=your_secure_password +``` + +The built-in admin has irrevocable root access. After creating other admin +users, disable it: + +```ini +acl.admin.user.enabled=false +``` + +In a replicated cluster, keep in mind that the built-in admin authorizes a +[role switch](/docs/high-availability/failover/) from its own credentials, +independently of the replicated access lists. It is the break-glass account when +a `SWITCH ROLE` grant has not yet replicated to the node you need to promote. + +## Management reference {#user-management} + +### Creating and removing principals + +```questdb-sql +-- Users +CREATE USER username WITH PASSWORD 'pwd'; +DROP USER username; + +-- Service accounts +CREATE SERVICE ACCOUNT appname WITH PASSWORD 'pwd'; +DROP SERVICE ACCOUNT appname; + +-- Groups +CREATE GROUP groupname; +DROP GROUP groupname; +``` + +### Managing group membership + +```questdb-sql +ADD USER username TO group1, group2; +REMOVE USER username FROM group1; +``` + +Passwords and tokens are managed with `ALTER USER` or +`ALTER SERVICE ACCOUNT`; see +[managing authentication](/docs/security/rbac/authentication/#managing-authentication). + +### Viewing information + +```questdb-sql +SHOW USERS; -- List all users +SHOW SERVICE ACCOUNTS; -- List all service accounts +SHOW GROUPS; -- List all groups +SHOW GROUPS username; -- List groups for a user +SHOW USER username; -- Show auth methods for user +SHOW PERMISSIONS username; -- Show permissions for user +``` + +Example output from `SHOW USER`: + +``` +auth_type enabled +--------- ------- +Password true +JWK Token false +REST Token true +``` + +:::note + +Viewing other users' information requires `LIST USERS` (to list all) or +`USER DETAILS` (to see details) permissions. Users can always view their own +information without these permissions. + +::: + +## See also + +- [Authentication and endpoint access](/docs/security/rbac/authentication/) - + how principals prove who they are and which protocols they may use +- [Granting and revoking permissions](/docs/security/rbac/granting-permissions/) - + what a principal is allowed to do once connected diff --git a/documentation/sidebars.js b/documentation/sidebars.js index f0b4f587fd..3eacde8f87 100644 --- a/documentation/sidebars.js +++ b/documentation/sidebars.js @@ -287,6 +287,10 @@ module.exports = { type: "category", label: "ALTER", items: [ + { + id: "query/sql/acl/alter-group", + type: "doc", + }, { id: "query/sql/acl/alter-service-account", type: "doc", @@ -714,14 +718,94 @@ module.exports = { type: "category", items: [ { - id: "security/rbac", - type: "doc", label: "Role-Based Access Control (RBAC)", + type: "category", + link: { type: "doc", id: "security/rbac/index" }, + items: [ + { + id: "security/rbac/index", + type: "doc", + label: "Overview", + }, + { + id: "security/rbac/users-and-groups", + type: "doc", + label: "Users and groups", + }, + { + id: "security/rbac/authentication", + type: "doc", + label: "Authentication and endpoints", + }, + { + id: "security/rbac/granting-permissions", + type: "doc", + label: "Granting permissions", + }, + { + id: "security/rbac/permissions-reference", + type: "doc", + label: "Permissions reference", + }, + { + id: "security/rbac/common-scenarios", + type: "doc", + label: "Common scenarios", + }, + ], }, { - id: "security/oidc", - type: "doc", label: "OpenID Connect (OIDC)", + type: "category", + link: { type: "doc", id: "security/oidc/index" }, + items: [ + { + id: "security/oidc/index", + type: "doc", + label: "Overview", + }, + { + id: "security/oidc/how-sign-in-works", + type: "doc", + label: "How sign-in works", + }, + { + id: "security/oidc/client-discovery", + type: "doc", + label: "Client discovery and tokens", + }, + { + id: "security/oidc/device-flow", + type: "doc", + label: "Device flow", + }, + { + id: "security/oidc/client-integration", + type: "doc", + label: "Client integration patterns", + }, + { + id: "security/oidc/group-mapping", + type: "doc", + label: "Mapping groups and permissions", + }, + { + label: "Provider setup", + type: "category", + items: [ + { + id: "security/oidc/pingfederate", + type: "doc", + label: "PingFederate", + }, + { + id: "security/oidc/entra-id", + type: "doc", + label: "Microsoft Entra ID", + }, + ], + }, + ], }, { type: "doc",