Skip to content

fix: kll_sketch throws when a sketch is merged into itself, as in Java - #570

Open
stojkomilos wants to merge 1 commit into
apache:masterfrom
stojkomilos:stojkomilos/kll-self-merge-throw
Open

stojkomilos wants to merge 1 commit into
apache:masterfrom
stojkomilos:stojkomilos/kll-self-merge-throw

Conversation

@stojkomilos

@stojkomilos stojkomilos commented Nov 9, 2026 •

Copy link
Copy Markdown

kll_sketch::merge() now throws std::invalid_argument("A sketch cannot merge with itself.") when the other sketch is this sketch.

Java

The Java KLL sketches throw in the same case, since apache/datasketches-java#516. KllDoublesSketch, KllFloatsSketch, KllLongsSketch and KllItemsSketch all start merge() with this check, where SELF_MERGE_MSG in QuantilesAPI is "A sketch cannot merge with itself. ":

public final void merge(final KllSketch other) {
  if (readOnly || (sketchStructure != UPDATABLE)) { throw new SketchesArgumentException(TGT_IS_READ_ONLY_MSG); }
  if (this == other) { throw new SketchesArgumentException(SELF_MERGE_MSG); }
  final KllDoublesSketch othDblSk = (KllDoublesSketch)other;
  if (othDblSk.isEmpty()) { return; }

Master

On master, a self-merge sometimes works and sometimes throws "Total weight does not match N", depending on n and k. A minimal repro with the default k = 200:

#include <iostream>
#include <kll_sketch.hpp>

int main() {
  for (int n : {100, 101, 1000}) {
    datasketches::kll_sketch<float> sketch;  // k = 200
    for (int i = 0; i < n; ++i) sketch.update(static_cast<float>(i));
    try {
      sketch.merge(sketch);
      std::cout << n << " items: merged, n = " << sketch.get_n() << std::endl;
    } catch (const std::exception& e) {
      std::cout << n << " items: threw \"" << e.what() << "\"" << std::endl;
    }
  }
}

On master (c22f2b1) it prints:

100 items: merged, n = 200
101 items: threw "Total weight does not match N"
1000 items: merged, n = 2000

With this PR, all three throw "A sketch cannot merge with itself.".

Tests

  • Added a test that verifies this new throwing behaviour.

All 17 CTest suites pass with GCC 10.5 and C++11, and kll_test also passes with ASan and UBSan. Without the check, the new section fails.

merge() reads the items of the other sketch while it updates this one.
When both are the same sketch, the merge fails once it compacts, with
"Total weight does not match N", and with string items it reads
destroyed strings. The Java KLL sketches throw a
SketchesArgumentException with "A sketch cannot merge with itself."
(apache/datasketches-java#516). kll_sketch now throws
std::invalid_argument with the same message, before its other checks
as in Java, so an empty sketch merged into itself throws too.

Co-authored-by: Isaac <no-reply@databricks.com>
@stojkomilos
stojkomilos marked this pull request as ready for review November 9, 2026 12:20
stojkomilos added a commit to stojkomilos/datasketches-cpp that referenced this pull request Nov 9, 2026
apache#570 makes a self-merge throw, as Java does, so merge() is the same as
on master here. Without apache#570, a self-merge of a sketch whose buffer is
still growing can fail where master works, so this PR merges after apache#570.

Co-authored-by: Isaac <no-reply@databricks.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant