Summary
The Cosmos DB Spark connector was changed in #49700 so that an HTTP 412 Precondition Failed — returned by the service for a document excluded by a conditional patch filter (spark.cosmos.write.patch.filter) — is treated as a successful no-op skip rather than failing the write.
The Kafka sink connector supports the equivalent config (azure.cosmos.sink.write.patch.filter, see sdk/cosmos/azure-cosmos-kafka-connect/docs/configuration-reference.md:67) and has the same shouldIgnore-by-write-strategy structure, but ITEM_PATCH still falls through to default: return false and therefore fails on a filter-predicate 412.
Where
sdk/cosmos/azure-cosmos-kafka-connect/src/main/java/com/azure/cosmos/kafka/connect/implementation/sink/CosmosBulkWriter.java (~line 320):
private boolean shouldIgnore(BulkOperationFailedException failedException) {
switch (this.writeConfig.getItemWriteStrategy()) {
case ITEM_APPEND:
return KafkaCosmosExceptionsHelper.isResourceExistsException(failedException);
case ITEM_DELETE:
return KafkaCosmosExceptionsHelper.isNotFoundException(failedException);
case ITEM_DELETE_IF_NOT_MODIFIED:
return KafkaCosmosExceptionsHelper.isNotFoundException(failedException)
|| KafkaCosmosExceptionsHelper.isPreconditionFailedException(failedException);
case ITEM_OVERWRITE_IF_NOT_MODIFIED:
return KafkaCosmosExceptionsHelper.isResourceExistsException(failedException)
|| KafkaCosmosExceptionsHelper.isNotFoundException(failedException)
|| KafkaCosmosExceptionsHelper.isPreconditionFailedException(failedException);
default: // <-- ITEM_PATCH lands here
return false;
}
}
CosmosPointWriter.java (case ITEM_PATCH: ~line 68) has the same gap on the point-write path.
Why it matters
The customer scenario that motivated #49700 (see #49594) is a Kafka -> Spark -> Cosmos pipeline doing idempotent server-side increment patches guarded by a filter predicate such as NOT IS_DEFINED(last_batch_id) OR last_batch_id < <batchId>. On replay the predicate legitimately excludes already-applied documents, producing 412s.
A customer who moves that same logic to the Kafka sink connector directly — a very natural step, since it removes Spark from the pipeline — will hit exactly the bug that #49700 just fixed for Spark. The two connectors share the customer scenario, so the divergence is likely to be discovered by a customer rather than by us.
Proposed fix
Mirror the Spark behavior: treat a 412 as an ignorable no-op for ITEM_PATCH when a non-empty patch filter predicate is configured, in both CosmosBulkWriter.shouldIgnore and the ITEM_PATCH path of CosmosPointWriter.
Rationale (same as #49700): a patch filter predicate's contract is "only modify the document when the condition holds, otherwise leave it alone", so a predicate miss is an expected outcome rather than an error. This is also consistent with the connector's own existing unconditional 412-ignore for ITEM_DELETE_IF_NOT_MODIFIED and ITEM_OVERWRITE_IF_NOT_MODIFIED.
Note that etag / If-Match is not wired for the patch path in either connector today, so the filter predicate is currently the only source of a 412 on patch. If etag support is added later, the two 412 causes will need to be distinguished — worth carrying the same inline comment the Spark connector now has.
Notes
Summary
The Cosmos DB Spark connector was changed in #49700 so that an HTTP
412 Precondition Failed— returned by the service for a document excluded by a conditional patch filter (spark.cosmos.write.patch.filter) — is treated as a successful no-op skip rather than failing the write.The Kafka sink connector supports the equivalent config (
azure.cosmos.sink.write.patch.filter, seesdk/cosmos/azure-cosmos-kafka-connect/docs/configuration-reference.md:67) and has the sameshouldIgnore-by-write-strategy structure, butITEM_PATCHstill falls through todefault: return falseand therefore fails on a filter-predicate 412.Where
sdk/cosmos/azure-cosmos-kafka-connect/src/main/java/com/azure/cosmos/kafka/connect/implementation/sink/CosmosBulkWriter.java(~line 320):CosmosPointWriter.java(case ITEM_PATCH:~line 68) has the same gap on the point-write path.Why it matters
The customer scenario that motivated #49700 (see #49594) is a Kafka -> Spark -> Cosmos pipeline doing idempotent server-side
incrementpatches guarded by a filter predicate such asNOT IS_DEFINED(last_batch_id) OR last_batch_id < <batchId>. On replay the predicate legitimately excludes already-applied documents, producing 412s.A customer who moves that same logic to the Kafka sink connector directly — a very natural step, since it removes Spark from the pipeline — will hit exactly the bug that #49700 just fixed for Spark. The two connectors share the customer scenario, so the divergence is likely to be discovered by a customer rather than by us.
Proposed fix
Mirror the Spark behavior: treat a 412 as an ignorable no-op for
ITEM_PATCHwhen a non-empty patch filter predicate is configured, in bothCosmosBulkWriter.shouldIgnoreand theITEM_PATCHpath ofCosmosPointWriter.Rationale (same as #49700): a patch filter predicate's contract is "only modify the document when the condition holds, otherwise leave it alone", so a predicate miss is an expected outcome rather than an error. This is also consistent with the connector's own existing unconditional 412-ignore for
ITEM_DELETE_IF_NOT_MODIFIEDandITEM_OVERWRITE_IF_NOT_MODIFIED.Note that etag /
If-Matchis not wired for the patch path in either connector today, so the filter predicate is currently the only source of a 412 on patch. If etag support is added later, the two 412 causes will need to be distinguished — worth carrying the same inline comment the Spark connector now has.Notes
recordsSkippedmetric plus a summary log at flush), so a fully-skipped run is distinguishable from a fully-successful one.