[client-v2, jdbc-v2] A caller-supplied FORMAT clause is silently ignored on ClickHouse 26.8+ (X-ClickHouse-Format wins)

[polyglotAI-bot]

Description

On ClickHouse 26.8+ the X-ClickHouse-Format request header is an explicit override that
wins over the query's FORMAT clause. client-v2 sends that header on every request, and
defaults it to RowBinaryWithNamesAndTypes when the caller sets no format. A caller who
writes SELECT ... FORMAT JSONEachRow therefore receives RowBinaryWithNamesAndTypes bytes
instead. There is no exception — only different bytes.

This contradicts the documented contract of Client.query(...):

Server response format can be specified through settings or in SQL query.
If specified in both, the sqlQuery will take precedence.

— client-v2/src/main/java/com/clickhouse/client/api/Client.java (Javadoc of query)

Related but different: issue #3068 covers the client's own internal metadata queries
(getTableSchema, getTableSchemaFromQuery, ping), which fail loudly. This report is the
caller-facing half, which fails silently. The two do not share a fix: #3068 is fixed by
changing the client's own SQL, this one is about what the client does with SQL the user wrote.

Server-side cause is the same: ClickHouse PR ClickHouse/ClickHouse#105249 turned
X-ClickHouse-Format from an alias of default_format (a fallback) into an alias of format
(an explicit override).

Steps to reproduce

1. Start clickhouse/clickhouse-server:head (26.8.1.1761).
2. Run client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings()).
3. Read response.getFormat() and the response body.

Three tests already in the repository pin this contract and turn red on 26.8, while passing
on 26.6.2.160 (same worktree, same build, only the server image changed):

Test	Result on 26.8.1.1761
client-v2 QueryTests.testSettingsNotChanged	expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]
jdbc-v2 StatementTest.testJSONEachRowFormatRequiresParserFactory	Expected SQLException (none thrown)
jdbc-v2 StatementTest.testTextFormatInResponse	Expected SQLException to be thrown, but nothing was thrown

jdbc-v2 StatementTest.testJSONEachRowFormat still passes, but for the wrong reason: its
assertions (getInt, getString) hold for either format, so the silent switch to RowBinary
is invisible to it.

The two jdbc-v2 failures show that the shipped, documented FORMAT JSONEachRow support in
the JDBC driver (jdbc_json_parser_factory, docs/client-v2-json-support.md) is inert on
26.8: response.getFormat() can no longer be JSONEachRow, so StatementImpl never selects
JSONEachRowFormatReader and returns binary-decoded rows instead.

Error Log or Exception StackTrace

No exception is thrown. That is the problem.

client-v2 QueryTests.testSettingsNotChanged:2345
  expected [JSONEachRow] but found [RowBinaryWithNamesAndTypes]

Expected Behaviour

Either of the following, but not the current silent mismatch:

• the FORMAT clause keeps taking precedence, as Client.query's Javadoc states; or
• the FORMAT clause is no longer supported and the client raises a clear error, with the
  Javadoc, docs/client-v2-json-support.md, docs/features.md and the
  examples/jdbc-v2-json-processors example updated accordingly.

Server behaviour, isolated with curl — same query, same header, two servers:

$ curl -H 'X-ClickHouse-Format: TabSeparated' ... --data-binary "SELECT 1 AS a FORMAT JSONEachRow"
26.6.2.160:    {"a":1}      <- FORMAT clause wins
26.8.1.1761:   1            <- header wins

With no header at all, 26.8.1.1761 returns {"a":1}, so FORMAT JSONEachRow itself is still
honoured by the server; only the client's unconditional header suppresses it.

Code Example

try (QueryResponse response =
         client.query("SELECT 1 AS a, 'x' AS b FORMAT JSONEachRow", new QuerySettings()).get()) {
    System.out.println(response.getFormat());
    // read response.getInputStream()
}

Observed against 26.8.1.1761:

response.getFormat() = RowBinaryWithNamesAndTypes
raw bytes (hex)      = 02 01 61 01 62 05 55 49 6e 74 38 06 53 74 72 69 6e 67 01 01 78

That is RowBinaryWithNamesAndTypes (2 columns; a/b; UInt8/String; values 1, "x").
Expected {"a":1,"b":"x"}.

Root cause

• client-v2/src/main/java/com/clickhouse/client/api/Client.java, query(...) — when the
  caller sets no format, the request settings are defaulted:
  requestSettings.setFormat(ClickHouseFormat.RowBinaryWithNamesAndTypes). The SQL text is
  never inspected for a FORMAT clause.
• client-v2/src/main/java/com/clickhouse/client/api/internal/HttpAPIClientHelper.java,
  addHeaders(...) — that setting is emitted unconditionally as
  X-ClickHouse-Format (ClickHouseHttpProto.HEADER_FORMAT).

Before 26.8 the two disagreeing instructions were harmless because the clause won. From 26.8
the header wins, so the default silently overrides the caller.

jdbc-v2 inherits this: StatementImpl.executeQueryImpl branches on response.getFormat(),
which is read back from the response header, so its JSONEachRow and text-format branches
have become unreachable.

Suggested fix

Make the header and the SQL agree, whichever direction the project prefers:

1. Keep honouring the clause. ClickHouseSqlStatement.getFormat() already extracts a
   FORMAT clause; feed it into the request settings so the header matches what the caller
   asked for. This preserves the Javadoc contract, the existing tests, and the jdbc-v2 JSON
   support, and needs no user-visible change.
2. Deprecate the clause. Detect a FORMAT clause in the caller's SQL and fail with a
   clear error pointing at QuerySettings.setFormat(...). This matches the direction stated
   on [client-v2] getTableSchema fails on ClickHouse 26.8: X-ClickHouse-Format header now overrides the query's FORMAT TSKV clause #3068 ("format should be sent via settings not in query"), but it is a breaking change
   and requires updating the Javadoc, the three tests above, docs/client-v2-json-support.md,
   docs/features.md and examples/jdbc-v2-json-processors.

What must not remain is the current state, where the client accepts the clause and quietly
returns a different format.

Contrast case that must keep working: a caller who sets the format only through
QuerySettings.setFormat(...) and writes no FORMAT clause is unaffected today and must stay
unaffected. Inserts and client-v1 already keep header and clause in agreement; r2dbc sends no
format header.

Configuration

Client Configuration

Client client = new Client.Builder()
        .addEndpoint(Protocol.HTTP, host, 8123, false)
        .setUsername("default").setPassword(password)
        .build();

Environment

• Client version: 0.10.0-rc1-SNAPSHOT, built from main at 40464dd
• Language version: Java 17
• OS: Ubuntu 24.04 (Docker)

ClickHouse Server

• ClickHouse Server version: 26.8.1.1761 (clickhouse/clickhouse-server:head) reproduces;
  26.6.2.160 does not
• ClickHouse Server non-default settings, if any: none
• CREATE TABLE statements for tables involved: none, SELECT 1 AS a, 'x' AS b is enough
• Sample data for all these tables: none needed

---

Found by automated analysis of clickhouse-java while working on #3068, and verified against a
live 26.8.1.1761 server and a 26.6.2.160 control rather than by inspection.