From c61e271cfb8c83743a0288f3550e9b5d429ad4b0 Mon Sep 17 00:00:00 2001 From: Polyglot AI <293096396+polyglotAI-bot@users.noreply.github.com> Date: Thu, 20 Aug 2026 15:11:49 +0000 Subject: [PATCH 1/5] fix(client-v2): request the format of internal queries thru settings The client sends the requested format of an operation in the X-ClickHouse-Format header of every request. getTableSchema/getTableSchemaFromQuery asked instead for TSKV with a FORMAT clause in the DESCRIBE query, and ping used a FORMAT clause too. A server before 26.8 used the format of the query, but since 26.8 the header wins, so the server answered with RowBinaryWithNamesAndTypes and the TSKV parser failed with "Failed to parse column null defined by type 'null'". The internal queries of the client now carry no FORMAT clause and set their format in the settings of the operation, so the header and the query always agree. Fixes: https://github.com/ClickHouse/clickhouse-java/issues/3068 --- CHANGELOG.md | 9 ++ .../com/clickhouse/client/api/Client.java | 23 ++-- .../client/api/RequestFormatUnitTest.java | 103 ++++++++++++++++++ .../observability/SpanRecorderTest.java | 6 +- 4 files changed, 130 insertions(+), 11 deletions(-) create mode 100644 client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ceae046d..c728260de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,15 @@ ### Bug Fixes +- **[client-v2, jdbc-v2]** Fixed `Client.getTableSchema(...)` and `Client.getTableSchemaFromQuery(...)` failing with + `Failed to parse column null defined by type 'null'` against ClickHouse `26.8+` (and `jdbc-v2` failing with it, + because `Connection`/`PreparedStatement` metadata calls use them). Both methods asked for `TSKV` with a `FORMAT` + clause in the `DESCRIBE` query, while the client sends the requested format of the operation in the + `X-ClickHouse-Format` header on every request. A server before `26.8` used the format from the query, but since + `26.8` the header wins, so the server answered with `RowBinaryWithNamesAndTypes` and the `TSKV` parser read binary + data. The internal queries of the client (the two schema calls and `ping()`) now request their format through the + settings of the operation only, so the header and the query always agree. + (https://github.com/ClickHouse/clickhouse-java/issues/3068) - **[jdbc-v2]** Fixed an `INSERT` whose values list holds a function call the bundled `ANTLR4` grammar cannot match - such as `hex(x'AB')`, valid ClickHouse the grammar has no hex string literal for - being reported to hold no function call when an `ANTLR4` parser backend is selected (`jdbc_sql_parser=ANTLR4` / `ANTLR4_PARAMS_PARSER`). Function calls in diff --git a/client-v2/src/main/java/com/clickhouse/client/api/Client.java b/client-v2/src/main/java/com/clickhouse/client/api/Client.java index 3e5764adb..884aa1f3f 100644 --- a/client-v2/src/main/java/com/clickhouse/client/api/Client.java +++ b/client-v2/src/main/java/com/clickhouse/client/api/Client.java @@ -1374,7 +1374,8 @@ public boolean ping() { public boolean ping(long timeout) { long startTime = System.nanoTime(); try { - CompletableFuture future = query("SELECT 1 FORMAT TabSeparated"); + CompletableFuture future = + query("SELECT 1", new QuerySettings().setFormat(ClickHouseFormat.TabSeparated)); try (QueryResponse response = timeout > 0 ? future.get(timeout, TimeUnit.MILLISECONDS) : future.get()) { return true; } @@ -1804,8 +1805,9 @@ public CompletableFuture query(String sqlQuery) { *

Sends SQL query to server.

* Notes: *
    - *
  • Server response format can be specified thru `settings` or in SQL query.
    • - *
  • If specified in both, the `sqlQuery` will take precedence.
    • + *
  • Server response format should be specified thru `settings` and not with a FORMAT clause in the SQL query.
    • + *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the + * format from the `sqlQuery`, a server since v26.8 uses the format from the `settings`.
    • *
* @param sqlQuery - complete SQL query. * @param settings - query operation settings. @@ -1834,8 +1836,10 @@ public CompletableFuture query(String sqlQuery, QuerySettings set * * Notes: *
    - *
  • Server response format can be specified through {@code settings} or in SQL query.
    • - *
  • If specified in both, the {@code sqlQuery} will take precedence.
    • + *
  • Server response format should be specified through {@code settings} and not with a FORMAT clause in the + * SQL query.
    • + *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the + * format from the {@code sqlQuery}, a server since v26.8 uses the format from the {@code settings}.
    • *
* * @param sqlQuery - complete SQL query. @@ -2201,7 +2205,7 @@ public TableSchema getTableSchema(String table) { * @return {@code TableSchema} - Schema of the table */ public TableSchema getTableSchema(String table, String database) { - final String sql = "DESCRIBE TABLE " + table + " FORMAT " + ClickHouseFormat.TSKV.name(); + final String sql = "DESCRIBE TABLE " + table; return getTableSchemaImpl(sql, table, null, database, null); } @@ -2215,7 +2219,7 @@ public TableSchema getTableSchemaFromQuery(String sql) { } public TableSchema getTableSchemaFromQuery(String sql, Map params) { - final String describeQuery = "DESC (" + sql + ") FORMAT " + ClickHouseFormat.TSKV.name(); + final String describeQuery = "DESC (" + sql + ")"; return getTableSchemaImpl(describeQuery, null, sql, getDefaultDatabase(), params); } @@ -2223,7 +2227,10 @@ private TableSchema getTableSchemaImpl( String describeQuery, String name, String originalQuery, String database, Map queryParams) { int operationTimeout = getOperationTimeout(); - QuerySettings settings = new QuerySettings().setDatabase(database); + // The format is requested thru settings (the X-ClickHouse-Format header) and not with a FORMAT clause: + // since v26.8 the server lets the header override the format written in the query, so a query that asks + // for one format while the client sends another in the header returns data the caller cannot parse. + QuerySettings settings = new QuerySettings().setDatabase(database).setFormat(ClickHouseFormat.TSKV); try (QueryResponse response = operationTimeout == 0 ? query(describeQuery, queryParams, settings).get() : query(describeQuery, queryParams, settings).get(operationTimeout, TimeUnit.MILLISECONDS)) { diff --git a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java new file mode 100644 index 000000000..4f8acd053 --- /dev/null +++ b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java @@ -0,0 +1,103 @@ +package com.clickhouse.client.api; + +import com.clickhouse.client.api.query.QueryResponse; +import com.clickhouse.client.api.query.QuerySettings; +import com.clickhouse.data.ClickHouseFormat; +import com.github.tomakehurst.wiremock.WireMockServer; +import com.github.tomakehurst.wiremock.client.WireMock; +import com.github.tomakehurst.wiremock.core.WireMockConfiguration; +import com.github.tomakehurst.wiremock.verification.LoggedRequest; +import org.testng.Assert; +import org.testng.annotations.AfterMethod; +import org.testng.annotations.BeforeMethod; +import org.testng.annotations.DataProvider; +import org.testng.annotations.Test; + +import java.util.List; +import java.util.concurrent.TimeUnit; +import java.util.function.Consumer; + +public class RequestFormatUnitTest { + + private static final String TSKV_RESPONSE = + "name=id\ttype=Int32\tdefault_type=\tdefault_expression=\tcomment=\tcodec_expression=\tttl_expression=\n"; + + private WireMockServer mockServer; + + private Client client; + + @BeforeMethod + public void setUp() { + mockServer = new WireMockServer(WireMockConfiguration.options().dynamicPort()); + mockServer.start(); + mockServer.stubFor(WireMock.post(WireMock.anyUrl()) + .willReturn(WireMock.aResponse().withStatus(200) + .withHeader("Content-Type", "text/plain") + .withBody(TSKV_RESPONSE))); + client = new Client.Builder() + .addEndpoint("http://localhost:" + mockServer.port()) + .setUsername("default") + .setPassword("") + .setDefaultDatabase("default") + .compressServerResponse(false) + .build(); + } + + @AfterMethod + public void tearDown() { + if (client != null) { + client.close(); + } + if (mockServer != null) { + mockServer.stop(); + } + } + + @Test(dataProvider = "requestFormatData") + public void testFormatIsRequestedWithHeaderOnly(Consumer operation, String expectedStatement, + ClickHouseFormat expectedFormat) { + operation.accept(client); + + LoggedRequest request = findRequest(expectedStatement); + Assert.assertEquals(request.getBodyAsString().trim(), expectedStatement); + Assert.assertEquals(request.getHeader("X-ClickHouse-Format"), expectedFormat.name()); + } + + @DataProvider(name = "requestFormatData") + public static Object[][] requestFormatData() { + return new Object[][]{ + {(Consumer) c -> Assert.assertEquals( + c.getTableSchema("test_table", "test_db").getColumns().size(), 1), + "DESCRIBE TABLE test_table", ClickHouseFormat.TSKV}, + {(Consumer) c -> Assert.assertEquals( + c.getTableSchemaFromQuery("SELECT id FROM test_table").getColumns().size(), 1), + "DESC (SELECT id FROM test_table)", ClickHouseFormat.TSKV}, + {(Consumer) c -> Assert.assertTrue(c.ping()), + "SELECT 1", ClickHouseFormat.TabSeparated}, + // Formats a caller asks for keep flowing through unchanged + {(Consumer) c -> runQuery(c, "SELECT 2", null), + "SELECT 2", ClickHouseFormat.RowBinaryWithNamesAndTypes}, + {(Consumer) c -> runQuery(c, "SELECT 3", + new QuerySettings().setFormat(ClickHouseFormat.JSONEachRow)), + "SELECT 3", ClickHouseFormat.JSONEachRow}, + }; + } + + private static void runQuery(Client client, String sql, QuerySettings settings) { + try (QueryResponse response = client.query(sql, settings).get(10, TimeUnit.SECONDS)) { + Assert.assertNotNull(response); + } catch (Exception e) { + throw new AssertionError("query failed: " + sql, e); + } + } + + private LoggedRequest findRequest(String statement) { + List requests = mockServer.findAll(WireMock.postRequestedFor(WireMock.anyUrl())); + for (LoggedRequest request : requests) { + if (request.getBodyAsString().trim().equals(statement)) { + return request; + } + } + throw new AssertionError("no request was sent with statement '" + statement + "', sent: " + requests); + } +} diff --git a/client-v2/src/test/java/com/clickhouse/client/observability/SpanRecorderTest.java b/client-v2/src/test/java/com/clickhouse/client/observability/SpanRecorderTest.java index 19d0ad10e..49b671eae 100644 --- a/client-v2/src/test/java/com/clickhouse/client/observability/SpanRecorderTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/observability/SpanRecorderTest.java @@ -116,7 +116,7 @@ public void testPingSpan() { // implements on top of a query CapturedSpan operationSpan = recorder.operationSpan(); Assert.assertEquals(operationSpan.getName(), "query " + database); - Assert.assertEquals(operationSpan.getAttribute(SpanAttribute.DB_QUERY_TEXT), "SELECT 1 FORMAT TabSeparated"); + Assert.assertEquals(operationSpan.getAttribute(SpanAttribute.DB_QUERY_TEXT), "SELECT 1"); Assert.assertNull(operationSpan.getAttribute(SpanAttribute.DB_OPERATION_NAME)); Assert.assertNull(operationSpan.getAttribute(SpanAttribute.DB_COLLECTION_NAME)); Assert.assertEquals(operationSpan.getEndCount(), 1); @@ -143,7 +143,7 @@ public void testTableSchemaSpanIsReportedAsQuery() { CapturedSpan operationSpan = recorder.operationSpan(); Assert.assertEquals(operationSpan.getName(), "query " + database); Assert.assertEquals(operationSpan.getAttribute(SpanAttribute.DB_QUERY_TEXT), - "DESCRIBE TABLE " + TABLE + " FORMAT TSKV"); + "DESCRIBE TABLE " + TABLE); Assert.assertNull(operationSpan.getAttribute(SpanAttribute.DB_OPERATION_NAME)); Assert.assertNull(operationSpan.getAttribute(SpanAttribute.DB_COLLECTION_NAME)); Assert.assertEquals(operationSpan.getEndCount(), 1); @@ -157,7 +157,7 @@ public void testTableSchemaFromQuerySpanIsReportedAsQuery() { CapturedSpan operationSpan = recorder.operationSpan(); Assert.assertEquals(operationSpan.getName(), "query " + database); Assert.assertEquals(operationSpan.getAttribute(SpanAttribute.DB_QUERY_TEXT), - "DESC (SELECT id FROM " + TABLE + ") FORMAT TSKV"); + "DESC (SELECT id FROM " + TABLE + ")"); Assert.assertNull(operationSpan.getAttribute(SpanAttribute.DB_OPERATION_NAME)); Assert.assertEquals(operationSpan.getEndCount(), 1); } From f449cf72814aff20a6a56bdf562b1a300b1d3f6b Mon Sep 17 00:00:00 2001 From: Polyglot AI <293096396+polyglotAI-bot@users.noreply.github.com> Date: Tue, 25 Aug 2026 01:23:05 +0000 Subject: [PATCH 2/5] fix(client-v2): send the format of a query's FORMAT clause in the header too --- .../com/clickhouse/client/api/Client.java | 16 +++- .../client/api/internal/ClientUtils.java | 94 +++++++++++++++++++ .../client/api/RequestFormatUnitTest.java | 12 +++ .../client/api/internal/ClientUtilsTest.java | 38 ++++++++ 4 files changed, 155 insertions(+), 5 deletions(-) diff --git a/client-v2/src/main/java/com/clickhouse/client/api/Client.java b/client-v2/src/main/java/com/clickhouse/client/api/Client.java index 884aa1f3f..2c830fc81 100644 --- a/client-v2/src/main/java/com/clickhouse/client/api/Client.java +++ b/client-v2/src/main/java/com/clickhouse/client/api/Client.java @@ -1806,8 +1806,8 @@ public CompletableFuture query(String sqlQuery) { * Notes: *
    *
  • Server response format should be specified thru `settings` and not with a FORMAT clause in the SQL query.
    • - *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the - * format from the `sqlQuery`, a server since v26.8 uses the format from the `settings`.
    • + *
  • A format asked for with a FORMAT clause that closes the `sqlQuery` is used when `settings` name no format.
    • + *
  • If specified in both, the `settings` take precedence.
    • *
* @param sqlQuery - complete SQL query. * @param settings - query operation settings. @@ -1838,8 +1838,9 @@ public CompletableFuture query(String sqlQuery, QuerySettings set *
    *
  • Server response format should be specified through {@code settings} and not with a FORMAT clause in the * SQL query.
    • - *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the - * format from the {@code sqlQuery}, a server since v26.8 uses the format from the {@code settings}.
    • + *
  • A format asked for with a FORMAT clause that closes the {@code sqlQuery} is used when {@code settings} + * name no format.
    • + *
  • If specified in both, the {@code settings} take precedence.
    • *
* * @param sqlQuery - complete SQL query. @@ -1854,7 +1855,12 @@ public CompletableFuture query(String sqlQuery, MapReturns the format of a trailing {@code FORMAT } clause of a statement, or {@code null} when the + * statement has no such clause or names a format this client does not know.

+ * + *

String literals, quoted identifiers and comments are skipped, so a {@code FORMAT} written inside them is + * not taken as a clause. Only a clause that closes the statement is reported: an {@code INSERT} that carries + * its data after the clause returns {@code null}.

+ * + * @param sqlQuery statement to read, may be null + * @return format of the trailing FORMAT clause or null + */ + public static ClickHouseFormat extractTrailingFormat(String sqlQuery) { + if (sqlQuery == null) { + return null; + } + + String previousWord = null; + String lastWord = null; + final int len = sqlQuery.length(); + int i = 0; + while (i < len) { + final char c = sqlQuery.charAt(i); + if (Character.isWhitespace(c) || c == ';') { + i++; + } else if (c == '-' && i + 1 < len && sqlQuery.charAt(i + 1) == '-') { + i = skipLineComment(sqlQuery, i); + } else if (c == '#') { + i = skipLineComment(sqlQuery, i); + } else if (c == '/' && i + 1 < len && sqlQuery.charAt(i + 1) == '*') { + i = skipBlockComment(sqlQuery, i); + } else if (isWordChar(c)) { + final int start = i; + while (i < len && isWordChar(sqlQuery.charAt(i))) { + i++; + } + previousWord = lastWord; + lastWord = sqlQuery.substring(start, i); + } else { + // a quoted part or any other character ends the word sequence + i = (c == '\'' || c == '"' || c == '`') ? skipQuoted(sqlQuery, i, c) : i + 1; + previousWord = lastWord; + lastWord = null; + } + } + + if (lastWord == null || !"FORMAT".equalsIgnoreCase(previousWord)) { + return null; + } + try { + return ClickHouseFormat.valueOf(lastWord); + } catch (IllegalArgumentException e) { + return null; + } + } + + private static boolean isWordChar(char c) { + return Character.isLetterOrDigit(c) || c == '_' || c == '$'; + } + + private static int skipLineComment(String str, int i) { + while (i < str.length() && str.charAt(i) != '\n') { + i++; + } + return i; + } + + private static int skipBlockComment(String str, int i) { + i += 2; + while (i + 1 < str.length() && !(str.charAt(i) == '*' && str.charAt(i + 1) == '/')) { + i++; + } + return Math.min(str.length(), i + 2); + } + + private static int skipQuoted(String str, int i, char quote) { + i++; // opening quote + while (i < str.length()) { + final char c = str.charAt(i); + if (c == '\\') { + i += 2; + } else if (c == quote) { + if (i + 1 < str.length() && str.charAt(i + 1) == quote) { + i += 2; // doubled quote is an escaped one + } else { + return i + 1; + } + } else { + i++; + } + } + return i; + } + public static void quietClose(Closeable closeable, Logger log) { if (closeable != null) { try { diff --git a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java index 4f8acd053..8f1701439 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java @@ -80,6 +80,18 @@ public static Object[][] requestFormatData() { {(Consumer) c -> runQuery(c, "SELECT 3", new QuerySettings().setFormat(ClickHouseFormat.JSONEachRow)), "SELECT 3", ClickHouseFormat.JSONEachRow}, + // A format asked for with a FORMAT clause is sent in the header too, so both agree + {(Consumer) c -> runQuery(c, "SELECT 4 FORMAT JSONEachRow", null), + "SELECT 4 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, + {(Consumer) c -> runQuery(c, "SELECT 5 FORMAT JSONEachRow", new QuerySettings()), + "SELECT 5 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, + // Settings win over a FORMAT clause + {(Consumer) c -> runQuery(c, "SELECT 6 FORMAT JSONEachRow", + new QuerySettings().setFormat(ClickHouseFormat.CSV)), + "SELECT 6 FORMAT JSONEachRow", ClickHouseFormat.CSV}, + // A FORMAT inside a literal is not a clause + {(Consumer) c -> runQuery(c, "SELECT 'x FORMAT JSONEachRow'", null), + "SELECT 'x FORMAT JSONEachRow'", ClickHouseFormat.RowBinaryWithNamesAndTypes}, }; } diff --git a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java index 88c5d8526..4e7aae965 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java @@ -1,8 +1,10 @@ package com.clickhouse.client.api.internal; +import com.clickhouse.data.ClickHouseFormat; import org.mockito.Mockito; import org.slf4j.Logger; import org.testng.Assert; +import org.testng.annotations.DataProvider; import org.testng.annotations.Test; import java.io.Closeable; @@ -45,4 +47,40 @@ public void testQuietCloseWithNull() { Mockito.verifyNoInteractions(log); Assert.assertTrue(true); } + + @Test(groups = {"unit"}, dataProvider = "trailingFormatData") + public void testExtractTrailingFormat(String sqlQuery, ClickHouseFormat expectedFormat) { + Assert.assertEquals(ClientUtils.extractTrailingFormat(sqlQuery), expectedFormat); + } + + @DataProvider(name = "trailingFormatData") + public static Object[][] trailingFormatData() { + return new Object[][]{ + {"SELECT 1 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, + {"SELECT 1 format TabSeparated", ClickHouseFormat.TabSeparated}, + {"SELECT 1\n FORMAT\tCSV \n ", ClickHouseFormat.CSV}, + {"SELECT 1 FORMAT CSV;", ClickHouseFormat.CSV}, + {"SELECT 1 SETTINGS max_block_size = 10 FORMAT Pretty", ClickHouseFormat.Pretty}, + {"SELECT 1 FORMAT CSV -- as csv", ClickHouseFormat.CSV}, + {"SELECT 1 FORMAT CSV /* as csv */", ClickHouseFormat.CSV}, + {"SELECT /* FORMAT TSKV */ 1 FORMAT CSV", ClickHouseFormat.CSV}, + // statements that name no format + {"SELECT 1", null}, + {"SELECT 1 FORMAT", null}, + {"", null}, + {null, null}, + // a format name the client does not know is left to the server + {"SELECT 1 FORMAT NoSuchFormat", null}, + // FORMAT that is not a clause of the statement + {"SELECT 'x FORMAT CSV'", null}, + {"SELECT 'it''s x FORMAT CSV'", null}, + {"SELECT 'it\\'s x FORMAT CSV'", null}, + {"SELECT 1 AS \"x FORMAT CSV\"", null}, + {"SELECT 1 AS `x FORMAT CSV`", null}, + {"SELECT 1 -- FORMAT CSV", null}, + {"SELECT 1 /* FORMAT CSV */", null}, + {"INSERT INTO t FORMAT CSV\n1,2\n", null}, + {"SELECT formatDateTime(d, '%F') FROM t", null}, + }; + } } From 4ce0695e3ae43b8d46fd994dcadb268bd587dc6a Mon Sep 17 00:00:00 2001 From: Polyglot AI <293096396+polyglotAI-bot@users.noreply.github.com> Date: Tue, 25 Aug 2026 01:49:05 +0000 Subject: [PATCH 3/5] fix(client-v2): also accept a FORMAT clause closed by a SETTINGS clause Documents the new format resolution in CHANGELOG.md and docs/features.md. --- CHANGELOG.md | 5 +- .../client/api/internal/ClientUtils.java | 56 +++++++++++++------ .../client/api/internal/ClientUtilsTest.java | 5 ++ docs/features.md | 2 +- 4 files changed, 48 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c728260de..945365302 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,7 +74,10 @@ `X-ClickHouse-Format` header on every request. A server before `26.8` used the format from the query, but since `26.8` the header wins, so the server answered with `RowBinaryWithNamesAndTypes` and the `TSKV` parser read binary data. The internal queries of the client (the two schema calls and `ping()`) now request their format through the - settings of the operation only, so the header and the query always agree. + settings of the operation only, so the header and the query always agree. For the same reason a query of a caller + that asks for a format with a `FORMAT` clause that closes the statement - like `SELECT 1 FORMAT JSONEachRow` - now + sends that format in the header too, so a `26.8+` server answers with the format the caller asked for. A format + named in the settings of the operation still takes precedence over the `FORMAT` clause of a query. (https://github.com/ClickHouse/clickhouse-java/issues/3068) - **[jdbc-v2]** Fixed an `INSERT` whose values list holds a function call the bundled `ANTLR4` grammar cannot match - such as `hex(x'AB')`, valid ClickHouse the grammar has no hex string literal for - being reported to hold no function diff --git a/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java b/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java index 2ab5838ac..8a10cf182 100644 --- a/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java +++ b/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java @@ -25,8 +25,8 @@ public static boolean isBlank(String str) { * statement has no such clause or names a format this client does not know.

* *

String literals, quoted identifiers and comments are skipped, so a {@code FORMAT} written inside them is - * not taken as a clause. Only a clause that closes the statement is reported: an {@code INSERT} that carries - * its data after the clause returns {@code null}.

+ * not taken as a clause. Only a clause that closes the statement is reported - a {@code SETTINGS} clause may + * follow it - so an {@code INSERT} that carries its data after the clause returns {@code null}.

* * @param sqlQuery statement to read, may be null * @return format of the trailing FORMAT clause or null @@ -36,45 +36,65 @@ public static ClickHouseFormat extractTrailingFormat(String sqlQuery) { return null; } - String previousWord = null; - String lastWord = null; + // state of reading a FORMAT clause: no clause seen, the name of the format is expected, the name was read, + // or a SETTINGS clause follows the name and closes the statement + int state = NO_CLAUSE; + String formatName = null; final int len = sqlQuery.length(); int i = 0; while (i < len) { final char c = sqlQuery.charAt(i); if (Character.isWhitespace(c) || c == ';') { i++; - } else if (c == '-' && i + 1 < len && sqlQuery.charAt(i + 1) == '-') { - i = skipLineComment(sqlQuery, i); - } else if (c == '#') { + continue; + } + if ((c == '-' && i + 1 < len && sqlQuery.charAt(i + 1) == '-') || c == '#') { i = skipLineComment(sqlQuery, i); - } else if (c == '/' && i + 1 < len && sqlQuery.charAt(i + 1) == '*') { + continue; + } + if (c == '/' && i + 1 < len && sqlQuery.charAt(i + 1) == '*') { i = skipBlockComment(sqlQuery, i); - } else if (isWordChar(c)) { + continue; + } + if (isWordChar(c)) { final int start = i; while (i < len && isWordChar(sqlQuery.charAt(i))) { i++; } - previousWord = lastWord; - lastWord = sqlQuery.substring(start, i); - } else { - // a quoted part or any other character ends the word sequence - i = (c == '\'' || c == '"' || c == '`') ? skipQuoted(sqlQuery, i, c) : i + 1; - previousWord = lastWord; - lastWord = null; + final String word = sqlQuery.substring(start, i); + if (state == EXPECT_NAME) { + formatName = word; + state = NAME_READ; + } else if (state == NAME_READ) { + // only a SETTINGS clause may close a statement after the format name + state = "SETTINGS".equalsIgnoreCase(word) ? IN_SETTINGS : NO_CLAUSE; + } else if (state != IN_SETTINGS && "FORMAT".equalsIgnoreCase(word)) { + state = EXPECT_NAME; + } + continue; + } + // a quoted part or any other character cannot be part of a FORMAT clause + i = (c == '\'' || c == '"' || c == '`') ? skipQuoted(sqlQuery, i, c) : i + 1; + if (state == EXPECT_NAME || state == NAME_READ) { + state = NO_CLAUSE; } } - if (lastWord == null || !"FORMAT".equalsIgnoreCase(previousWord)) { + if (formatName == null || (state != NAME_READ && state != IN_SETTINGS)) { return null; } try { - return ClickHouseFormat.valueOf(lastWord); + return ClickHouseFormat.valueOf(formatName); } catch (IllegalArgumentException e) { return null; } } + private static final int NO_CLAUSE = 0; + private static final int EXPECT_NAME = 1; + private static final int NAME_READ = 2; + private static final int IN_SETTINGS = 3; + private static boolean isWordChar(char c) { return Character.isLetterOrDigit(c) || c == '_' || c == '$'; } diff --git a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java index 4e7aae965..de2b9a7d7 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java @@ -61,6 +61,9 @@ public static Object[][] trailingFormatData() { {"SELECT 1\n FORMAT\tCSV \n ", ClickHouseFormat.CSV}, {"SELECT 1 FORMAT CSV;", ClickHouseFormat.CSV}, {"SELECT 1 SETTINGS max_block_size = 10 FORMAT Pretty", ClickHouseFormat.Pretty}, + {"SELECT 1 FORMAT CSV SETTINGS format_csv_delimiter = '|'", ClickHouseFormat.CSV}, + {"SELECT 1 FORMAT CSV\r\n", ClickHouseFormat.CSV}, + {"SELECT 1 FORMAT CSV # as csv", ClickHouseFormat.CSV}, {"SELECT 1 FORMAT CSV -- as csv", ClickHouseFormat.CSV}, {"SELECT 1 FORMAT CSV /* as csv */", ClickHouseFormat.CSV}, {"SELECT /* FORMAT TSKV */ 1 FORMAT CSV", ClickHouseFormat.CSV}, @@ -80,7 +83,9 @@ public static Object[][] trailingFormatData() { {"SELECT 1 -- FORMAT CSV", null}, {"SELECT 1 /* FORMAT CSV */", null}, {"INSERT INTO t FORMAT CSV\n1,2\n", null}, + {"INSERT INTO t FORMAT CSV\n'a',2\n", null}, {"SELECT formatDateTime(d, '%F') FROM t", null}, + {"SELECT 1 AS format FROM t LIMIT 1", null}, }; } } diff --git a/docs/features.md b/docs/features.md index ff87a167f..53fb7574f 100644 --- a/docs/features.md +++ b/docs/features.md @@ -14,7 +14,7 @@ This document lists stable, user-visible behavior in `client-v2` and `jdbc-v2` t - Proxy support: Can send requests through configured HTTP proxies, including proxy credentials. - Connection and socket tuning: Exposes pool sizing, keep-alive, reuse strategy, connect/request/socket timeouts, and low-level socket options. - Query execution: Executes SQL asynchronously and returns streaming query responses with response metadata and metrics. -- Query settings: Supports per-query database selection, output format, execution limits, roles, log comments, headers, reusable `Session` objects, session settings, server settings, and network timeout overrides. Settings explicitly set to `null` will not be sent to the server. +- Query settings: Supports per-query database selection, output format, execution limits, roles, log comments, headers, reusable `Session` objects, session settings, server settings, and network timeout overrides. Settings explicitly set to `null` will not be sent to the server. The output format of an operation is always sent in the `X-ClickHouse-Format` header, which a `26.8+` server uses in preference to the `FORMAT` clause of a query; a format asked for with a `FORMAT` clause that closes the statement of a `Client#query` call is therefore sent in that header too, and a format named in the query settings takes precedence over such a clause. - Parameterized SQL: Accepts named query parameters and can send them through supported HTTP request encodings. - Result materialization helpers: Provides streaming `Records`, generic row access, and convenience APIs that materialize all rows into generic records or typed POJOs. - Binary format readers: Reads ClickHouse binary result formats including `Native`, `RowBinary`, `RowBinaryWithNames`, and `RowBinaryWithNamesAndTypes`. From 838fab779f3ade93778d46f487e1b67ef6019623 Mon Sep 17 00:00:00 2001 From: Polyglot AI <293096396+polyglotAI-bot@users.noreply.github.com> Date: Tue, 25 Aug 2026 02:12:58 +0000 Subject: [PATCH 4/5] fix(client-v2): read the name of a FORMAT clause without regard to case The server accepts a format name in any case (FORMAT jsoneachrow works), but extractTrailingFormat resolved it with ClickHouseFormat.valueOf, which is case-sensitive. A clause written in a different case gave null, the header kept RowBinaryWithNamesAndTypes, and a 26.8 server preferred that header over the clause - so the caller got the wrong bytes. --- .../client/api/internal/ClientUtils.java | 22 ++++++++++++++----- .../client/api/RequestFormatUnitTest.java | 3 +++ .../client/api/internal/ClientUtilsTest.java | 5 +++++ 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java b/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java index 8a10cf182..9e708abd6 100644 --- a/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java +++ b/client-v2/src/main/java/com/clickhouse/client/api/internal/ClientUtils.java @@ -4,6 +4,10 @@ import org.slf4j.Logger; import java.io.Closeable; +import java.util.Collections; +import java.util.HashMap; +import java.util.Locale; +import java.util.Map; /** * Class containing utility methods used across the client. @@ -22,7 +26,8 @@ public static boolean isBlank(String str) { /** *

Returns the format of a trailing {@code FORMAT } clause of a statement, or {@code null} when the - * statement has no such clause or names a format this client does not know.

+ * statement has no such clause or names a format this client does not know. The name of the format is read + * without regard to case, as the server reads it.

* *

String literals, quoted identifiers and comments are skipped, so a {@code FORMAT} written inside them is * not taken as a clause. Only a clause that closes the statement is reported - a {@code SETTINGS} clause may @@ -83,11 +88,18 @@ public static ClickHouseFormat extractTrailingFormat(String sqlQuery) { if (formatName == null || (state != NAME_READ && state != IN_SETTINGS)) { return null; } - try { - return ClickHouseFormat.valueOf(formatName); - } catch (IllegalArgumentException e) { - return null; + // the server reads a format name without regard to case, so this client does too + return FORMATS_BY_NAME.get(formatName.toUpperCase(Locale.ROOT)); + } + + private static final Map FORMATS_BY_NAME; + + static { + Map formats = new HashMap<>(); + for (ClickHouseFormat format : ClickHouseFormat.values()) { + formats.put(format.name().toUpperCase(Locale.ROOT), format); } + FORMATS_BY_NAME = Collections.unmodifiableMap(formats); } private static final int NO_CLAUSE = 0; diff --git a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java index 8f1701439..7b6aa77bc 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java @@ -85,6 +85,9 @@ public static Object[][] requestFormatData() { "SELECT 4 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, {(Consumer) c -> runQuery(c, "SELECT 5 FORMAT JSONEachRow", new QuerySettings()), "SELECT 5 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, + // The name of the format is read without regard to case, as the server reads it + {(Consumer) c -> runQuery(c, "SELECT 7 FORMAT jsoneachrow", null), + "SELECT 7 FORMAT jsoneachrow", ClickHouseFormat.JSONEachRow}, // Settings win over a FORMAT clause {(Consumer) c -> runQuery(c, "SELECT 6 FORMAT JSONEachRow", new QuerySettings().setFormat(ClickHouseFormat.CSV)), diff --git a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java index de2b9a7d7..f57472617 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java @@ -67,6 +67,10 @@ public static Object[][] trailingFormatData() { {"SELECT 1 FORMAT CSV -- as csv", ClickHouseFormat.CSV}, {"SELECT 1 FORMAT CSV /* as csv */", ClickHouseFormat.CSV}, {"SELECT /* FORMAT TSKV */ 1 FORMAT CSV", ClickHouseFormat.CSV}, + // the name of the format is read without regard to case, as the server reads it + {"SELECT 1 FORMAT jsoneachrow", ClickHouseFormat.JSONEachRow}, + {"SELECT 1 FORMAT JsOnEaChRoW", ClickHouseFormat.JSONEachRow}, + {"SELECT 1 FORMAT TABSEPARATEDWITHNAMES", ClickHouseFormat.TabSeparatedWithNames}, // statements that name no format {"SELECT 1", null}, {"SELECT 1 FORMAT", null}, @@ -74,6 +78,7 @@ public static Object[][] trailingFormatData() { {null, null}, // a format name the client does not know is left to the server {"SELECT 1 FORMAT NoSuchFormat", null}, + {"SELECT 1 FORMAT nosuchformat", null}, // FORMAT that is not a clause of the statement {"SELECT 'x FORMAT CSV'", null}, {"SELECT 'it''s x FORMAT CSV'", null}, From 2b366cb046138a6ff38dfa8240989a3f7ffc53b8 Mon Sep 17 00:00:00 2001 From: Polyglot AI <293096396+polyglotAI-bot@users.noreply.github.com> Date: Tue, 25 Aug 2026 04:06:47 +0000 Subject: [PATCH 5/5] Address review: keep the fix to the client's own calls Reverts the resolution of a FORMAT clause written by a caller (commits f449cf7, 4ce0695 and 838fab7), which reads SQL in the client - a job of the parser. The PR again fixes only the queries the client itself sends: getTableSchema, getTableSchemaFromQuery and ping now request their format in the settings of the operation. Shortens the CHANGELOG entry and states in docs/features.md that the output format is to be set through the query settings. --- CHANGELOG.md | 14 +- .../com/clickhouse/client/api/Client.java | 16 +-- .../client/api/internal/ClientUtils.java | 126 ------------------ .../client/api/RequestFormatUnitTest.java | 15 --- .../client/api/internal/ClientUtilsTest.java | 48 ------- docs/features.md | 2 +- 6 files changed, 9 insertions(+), 212 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 945365302..ebf595410 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,17 +67,9 @@ ### Bug Fixes -- **[client-v2, jdbc-v2]** Fixed `Client.getTableSchema(...)` and `Client.getTableSchemaFromQuery(...)` failing with - `Failed to parse column null defined by type 'null'` against ClickHouse `26.8+` (and `jdbc-v2` failing with it, - because `Connection`/`PreparedStatement` metadata calls use them). Both methods asked for `TSKV` with a `FORMAT` - clause in the `DESCRIBE` query, while the client sends the requested format of the operation in the - `X-ClickHouse-Format` header on every request. A server before `26.8` used the format from the query, but since - `26.8` the header wins, so the server answered with `RowBinaryWithNamesAndTypes` and the `TSKV` parser read binary - data. The internal queries of the client (the two schema calls and `ping()`) now request their format through the - settings of the operation only, so the header and the query always agree. For the same reason a query of a caller - that asks for a format with a `FORMAT` clause that closes the statement - like `SELECT 1 FORMAT JSONEachRow` - now - sends that format in the header too, so a `26.8+` server answers with the format the caller asked for. A format - named in the settings of the operation still takes precedence over the `FORMAT` clause of a query. +- **[client-v2, jdbc-v2]** Fixed `Client.getTableSchema(...)`, `Client.getTableSchemaFromQuery(...)` and `ping()` + failing against ClickHouse `26.8+`, where the `X-ClickHouse-Format` header the client sends wins over a `FORMAT` + clause in the query. These internal queries now set their format in the settings instead of a `FORMAT` clause. (https://github.com/ClickHouse/clickhouse-java/issues/3068) - **[jdbc-v2]** Fixed an `INSERT` whose values list holds a function call the bundled `ANTLR4` grammar cannot match - such as `hex(x'AB')`, valid ClickHouse the grammar has no hex string literal for - being reported to hold no function diff --git a/client-v2/src/main/java/com/clickhouse/client/api/Client.java b/client-v2/src/main/java/com/clickhouse/client/api/Client.java index 2c830fc81..884aa1f3f 100644 --- a/client-v2/src/main/java/com/clickhouse/client/api/Client.java +++ b/client-v2/src/main/java/com/clickhouse/client/api/Client.java @@ -1806,8 +1806,8 @@ public CompletableFuture query(String sqlQuery) { * Notes: *

    *
  • Server response format should be specified thru `settings` and not with a FORMAT clause in the SQL query.
    • - *
  • A format asked for with a FORMAT clause that closes the `sqlQuery` is used when `settings` name no format.
    • - *
  • If specified in both, the `settings` take precedence.
    • + *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the + * format from the `sqlQuery`, a server since v26.8 uses the format from the `settings`.
    • *
* @param sqlQuery - complete SQL query. * @param settings - query operation settings. @@ -1838,9 +1838,8 @@ public CompletableFuture query(String sqlQuery, QuerySettings set *
    *
  • Server response format should be specified through {@code settings} and not with a FORMAT clause in the * SQL query.
    • - *
  • A format asked for with a FORMAT clause that closes the {@code sqlQuery} is used when {@code settings} - * name no format.
    • - *
  • If specified in both, the {@code settings} take precedence.
    • + *
  • If specified in both, the format that wins depends on the server version: a server before v26.8 uses the + * format from the {@code sqlQuery}, a server since v26.8 uses the format from the {@code settings}.
    • *
* * @param sqlQuery - complete SQL query. @@ -1855,12 +1854,7 @@ public CompletableFuture query(String sqlQuery, MapReturns the format of a trailing {@code FORMAT } clause of a statement, or {@code null} when the - * statement has no such clause or names a format this client does not know. The name of the format is read - * without regard to case, as the server reads it.

- * - *

String literals, quoted identifiers and comments are skipped, so a {@code FORMAT} written inside them is - * not taken as a clause. Only a clause that closes the statement is reported - a {@code SETTINGS} clause may - * follow it - so an {@code INSERT} that carries its data after the clause returns {@code null}.

- * - * @param sqlQuery statement to read, may be null - * @return format of the trailing FORMAT clause or null - */ - public static ClickHouseFormat extractTrailingFormat(String sqlQuery) { - if (sqlQuery == null) { - return null; - } - - // state of reading a FORMAT clause: no clause seen, the name of the format is expected, the name was read, - // or a SETTINGS clause follows the name and closes the statement - int state = NO_CLAUSE; - String formatName = null; - final int len = sqlQuery.length(); - int i = 0; - while (i < len) { - final char c = sqlQuery.charAt(i); - if (Character.isWhitespace(c) || c == ';') { - i++; - continue; - } - if ((c == '-' && i + 1 < len && sqlQuery.charAt(i + 1) == '-') || c == '#') { - i = skipLineComment(sqlQuery, i); - continue; - } - if (c == '/' && i + 1 < len && sqlQuery.charAt(i + 1) == '*') { - i = skipBlockComment(sqlQuery, i); - continue; - } - if (isWordChar(c)) { - final int start = i; - while (i < len && isWordChar(sqlQuery.charAt(i))) { - i++; - } - final String word = sqlQuery.substring(start, i); - if (state == EXPECT_NAME) { - formatName = word; - state = NAME_READ; - } else if (state == NAME_READ) { - // only a SETTINGS clause may close a statement after the format name - state = "SETTINGS".equalsIgnoreCase(word) ? IN_SETTINGS : NO_CLAUSE; - } else if (state != IN_SETTINGS && "FORMAT".equalsIgnoreCase(word)) { - state = EXPECT_NAME; - } - continue; - } - // a quoted part or any other character cannot be part of a FORMAT clause - i = (c == '\'' || c == '"' || c == '`') ? skipQuoted(sqlQuery, i, c) : i + 1; - if (state == EXPECT_NAME || state == NAME_READ) { - state = NO_CLAUSE; - } - } - - if (formatName == null || (state != NAME_READ && state != IN_SETTINGS)) { - return null; - } - // the server reads a format name without regard to case, so this client does too - return FORMATS_BY_NAME.get(formatName.toUpperCase(Locale.ROOT)); - } - - private static final Map FORMATS_BY_NAME; - - static { - Map formats = new HashMap<>(); - for (ClickHouseFormat format : ClickHouseFormat.values()) { - formats.put(format.name().toUpperCase(Locale.ROOT), format); - } - FORMATS_BY_NAME = Collections.unmodifiableMap(formats); - } - - private static final int NO_CLAUSE = 0; - private static final int EXPECT_NAME = 1; - private static final int NAME_READ = 2; - private static final int IN_SETTINGS = 3; - - private static boolean isWordChar(char c) { - return Character.isLetterOrDigit(c) || c == '_' || c == '$'; - } - - private static int skipLineComment(String str, int i) { - while (i < str.length() && str.charAt(i) != '\n') { - i++; - } - return i; - } - - private static int skipBlockComment(String str, int i) { - i += 2; - while (i + 1 < str.length() && !(str.charAt(i) == '*' && str.charAt(i + 1) == '/')) { - i++; - } - return Math.min(str.length(), i + 2); - } - - private static int skipQuoted(String str, int i, char quote) { - i++; // opening quote - while (i < str.length()) { - final char c = str.charAt(i); - if (c == '\\') { - i += 2; - } else if (c == quote) { - if (i + 1 < str.length() && str.charAt(i + 1) == quote) { - i += 2; // doubled quote is an escaped one - } else { - return i + 1; - } - } else { - i++; - } - } - return i; - } - public static void quietClose(Closeable closeable, Logger log) { if (closeable != null) { try { diff --git a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java index 7b6aa77bc..4f8acd053 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/RequestFormatUnitTest.java @@ -80,21 +80,6 @@ public static Object[][] requestFormatData() { {(Consumer) c -> runQuery(c, "SELECT 3", new QuerySettings().setFormat(ClickHouseFormat.JSONEachRow)), "SELECT 3", ClickHouseFormat.JSONEachRow}, - // A format asked for with a FORMAT clause is sent in the header too, so both agree - {(Consumer) c -> runQuery(c, "SELECT 4 FORMAT JSONEachRow", null), - "SELECT 4 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, - {(Consumer) c -> runQuery(c, "SELECT 5 FORMAT JSONEachRow", new QuerySettings()), - "SELECT 5 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, - // The name of the format is read without regard to case, as the server reads it - {(Consumer) c -> runQuery(c, "SELECT 7 FORMAT jsoneachrow", null), - "SELECT 7 FORMAT jsoneachrow", ClickHouseFormat.JSONEachRow}, - // Settings win over a FORMAT clause - {(Consumer) c -> runQuery(c, "SELECT 6 FORMAT JSONEachRow", - new QuerySettings().setFormat(ClickHouseFormat.CSV)), - "SELECT 6 FORMAT JSONEachRow", ClickHouseFormat.CSV}, - // A FORMAT inside a literal is not a clause - {(Consumer) c -> runQuery(c, "SELECT 'x FORMAT JSONEachRow'", null), - "SELECT 'x FORMAT JSONEachRow'", ClickHouseFormat.RowBinaryWithNamesAndTypes}, }; } diff --git a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java index f57472617..88c5d8526 100644 --- a/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java +++ b/client-v2/src/test/java/com/clickhouse/client/api/internal/ClientUtilsTest.java @@ -1,10 +1,8 @@ package com.clickhouse.client.api.internal; -import com.clickhouse.data.ClickHouseFormat; import org.mockito.Mockito; import org.slf4j.Logger; import org.testng.Assert; -import org.testng.annotations.DataProvider; import org.testng.annotations.Test; import java.io.Closeable; @@ -47,50 +45,4 @@ public void testQuietCloseWithNull() { Mockito.verifyNoInteractions(log); Assert.assertTrue(true); } - - @Test(groups = {"unit"}, dataProvider = "trailingFormatData") - public void testExtractTrailingFormat(String sqlQuery, ClickHouseFormat expectedFormat) { - Assert.assertEquals(ClientUtils.extractTrailingFormat(sqlQuery), expectedFormat); - } - - @DataProvider(name = "trailingFormatData") - public static Object[][] trailingFormatData() { - return new Object[][]{ - {"SELECT 1 FORMAT JSONEachRow", ClickHouseFormat.JSONEachRow}, - {"SELECT 1 format TabSeparated", ClickHouseFormat.TabSeparated}, - {"SELECT 1\n FORMAT\tCSV \n ", ClickHouseFormat.CSV}, - {"SELECT 1 FORMAT CSV;", ClickHouseFormat.CSV}, - {"SELECT 1 SETTINGS max_block_size = 10 FORMAT Pretty", ClickHouseFormat.Pretty}, - {"SELECT 1 FORMAT CSV SETTINGS format_csv_delimiter = '|'", ClickHouseFormat.CSV}, - {"SELECT 1 FORMAT CSV\r\n", ClickHouseFormat.CSV}, - {"SELECT 1 FORMAT CSV # as csv", ClickHouseFormat.CSV}, - {"SELECT 1 FORMAT CSV -- as csv", ClickHouseFormat.CSV}, - {"SELECT 1 FORMAT CSV /* as csv */", ClickHouseFormat.CSV}, - {"SELECT /* FORMAT TSKV */ 1 FORMAT CSV", ClickHouseFormat.CSV}, - // the name of the format is read without regard to case, as the server reads it - {"SELECT 1 FORMAT jsoneachrow", ClickHouseFormat.JSONEachRow}, - {"SELECT 1 FORMAT JsOnEaChRoW", ClickHouseFormat.JSONEachRow}, - {"SELECT 1 FORMAT TABSEPARATEDWITHNAMES", ClickHouseFormat.TabSeparatedWithNames}, - // statements that name no format - {"SELECT 1", null}, - {"SELECT 1 FORMAT", null}, - {"", null}, - {null, null}, - // a format name the client does not know is left to the server - {"SELECT 1 FORMAT NoSuchFormat", null}, - {"SELECT 1 FORMAT nosuchformat", null}, - // FORMAT that is not a clause of the statement - {"SELECT 'x FORMAT CSV'", null}, - {"SELECT 'it''s x FORMAT CSV'", null}, - {"SELECT 'it\\'s x FORMAT CSV'", null}, - {"SELECT 1 AS \"x FORMAT CSV\"", null}, - {"SELECT 1 AS `x FORMAT CSV`", null}, - {"SELECT 1 -- FORMAT CSV", null}, - {"SELECT 1 /* FORMAT CSV */", null}, - {"INSERT INTO t FORMAT CSV\n1,2\n", null}, - {"INSERT INTO t FORMAT CSV\n'a',2\n", null}, - {"SELECT formatDateTime(d, '%F') FROM t", null}, - {"SELECT 1 AS format FROM t LIMIT 1", null}, - }; - } } diff --git a/docs/features.md b/docs/features.md index 53fb7574f..1ad33f47f 100644 --- a/docs/features.md +++ b/docs/features.md @@ -14,7 +14,7 @@ This document lists stable, user-visible behavior in `client-v2` and `jdbc-v2` t - Proxy support: Can send requests through configured HTTP proxies, including proxy credentials. - Connection and socket tuning: Exposes pool sizing, keep-alive, reuse strategy, connect/request/socket timeouts, and low-level socket options. - Query execution: Executes SQL asynchronously and returns streaming query responses with response metadata and metrics. -- Query settings: Supports per-query database selection, output format, execution limits, roles, log comments, headers, reusable `Session` objects, session settings, server settings, and network timeout overrides. Settings explicitly set to `null` will not be sent to the server. The output format of an operation is always sent in the `X-ClickHouse-Format` header, which a `26.8+` server uses in preference to the `FORMAT` clause of a query; a format asked for with a `FORMAT` clause that closes the statement of a `Client#query` call is therefore sent in that header too, and a format named in the query settings takes precedence over such a clause. +- Query settings: Supports per-query database selection, output format, execution limits, roles, log comments, headers, reusable `Session` objects, session settings, server settings, and network timeout overrides. Settings explicitly set to `null` will not be sent to the server. The output format must be set through the query settings (`QuerySettings#setFormat`) and not with a `FORMAT` clause in the query: the client always sends the format of the settings in the `X-ClickHouse-Format` header, and a `26.8+` server uses that header in preference to a `FORMAT` clause in the query. - Parameterized SQL: Accepts named query parameters and can send them through supported HTTP request encodings. - Result materialization helpers: Provides streaming `Records`, generic row access, and convenience APIs that materialize all rows into generic records or typed POJOs. - Binary format readers: Reads ClickHouse binary result formats including `Native`, `RowBinary`, `RowBinaryWithNames`, and `RowBinaryWithNamesAndTypes`.