Skip to content

Integrations: add adbcBridge (Apache Arrow ADBC over the PostgreSQL ODBC driver) #543

Description

@singhpratech

The Integrations section lists the tools that reach QuestDB over its PostgreSQL wire, with
pandas and Polars under data processing. I'd like to propose one more page there and can open
the PR myself — integrations/data-processing/adbcbridge.md in the shape of the Polars page,
unless you would rather it sat under "other".

adbcBridge is an Apache Arrow ADBC driver, written in C, that drives any ODBC driver and returns
Arrow record batches. Against QuestDB it goes through psqlodbc, and QuestDB 10 is one of the 53
databases it is verified on with one workload (types, NULLs, Unicode, parameters, bulk ingest,
batched reads, metadata, errors) on Linux, macOS and Windows, from Python, Rust, C#, Java and Go
with the same library. Since 10.0 the Python and Rust clients already speak Arrow natively over
QWP, so the case for this page is the other three languages, and any environment that reaches
QuestDB over the PostgreSQL wire and wants Arrow out of it. The QuestDB row of the matrix, with
the per-driver notes, is at
https://adbcbridge.org/matrix/#questdb; the two pgwire findings I filed here in August
(#7566, #7567) came out of that work.

Suggested content:

adbcBridge — Apache Arrow ADBC driver over ODBC. Reads QuestDB through the PostgreSQL ODBC
driver (psqlodbc) into Arrow record batches; bulk ingest through a parameterised INSERT (QuestDB
has its own type system behind the PG wire, so the driver uses standard-SQL DDL and
true/false boolean parameters, and leaves parameter arrays off because psqlodbc inlines their
values as string literals). Packages: PyPI adbcbridge, crates.io adbcbridge, NuGet
AdbcBridge, Maven Central org.adbcbridge:adbcbridge, Go github.com/singhpratech/adbcbridge/go.
Apache-2.0; https://github.com/singhpratech/adbcbridge

Minimal example (Python; the other four languages are one call each on the same driver):

import adbc_driver_manager.dbapi as dbapi

uri = "Driver=PostgreSQL Unicode;Server=localhost;Port=8812;Database=qdb;Uid=admin;Pwd=quest;"
with dbapi.connect(driver="odbc", db_kwargs={"uri": uri}) as conn, conn.cursor() as cur:
    cur.execute("SELECT timestamp, symbol, price FROM trades LIMIT 5")
    print(cur.fetch_arrow_table())

Two things a reader of that page would want to know, both from the matrix notes: psqlodbc's
SQLColumns does not work against QuestDB (the driver falls back to describing a zero-row
SELECT for metadata), and a prepared statement on a manual-commit connection should be freed
only after the commit, because psqlodbc frees it inside SAVEPOINT … RELEASE that QuestDB does
not support. Happy to put either or both in the entry, or leave them to the matrix page.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions