Fix six MongoDB query-semantics defects in the heuristics calculator

[LautaroPetaccio]

Six independent defects in MongoHeuristicsCalculator, each one a case where the computed truthness disagrees with what MongoDB actually does. Every claim below was checked by running the query against a real MongoDB 7.0.40 and comparing it with the calculator, rather than reasoning from the documentation.

Supersedes #1737, which carried the $all fix alone. These are folded together because they touch the same class and would otherwise need rebasing against each other; each commit stands on its own if you prefer to take them separately.

#	operator	query vs document	MongoDB	before
1	$all	{tags:{$all:["a","b","c"]}} vs ["a"]	no match	match, distance 0
1	$all	{tags:{$all:["a"]}} vs ["a","b"]	match	no match
2	$ne	{age:{$ne:"abc"}} vs {age:20}	match	no match, no gradient
3	$nin	{tags:{$nin:["a"]}} vs ["a","b"]	no match	match, distance 0
4	$not	{a:{$not:{$ne:5}}} vs {}	no match	match, distance 0
5	$all	{f:{$all:["a"]}} vs {f:"a"}	match	no match
5	$in	{tags:{$in:[["a","b"]]}} vs ["a","b"]	match	no match
6	$eq	{f:{$eq:"a"}} vs {f:["a","b"]}	match	no match
6	$ne	{f:{$ne:"a"}} vs {f:["a","b"]}	no match	match

The rows in bold are the damaging ones: a condition that no data can satisfy is reported as already covered, so the search stops generating data for it.

1. $all was quantified in the wrong direction

The score was an AND over the elements of the array held by the field, each scored by an OR over the expected values. That asks "is every stored element one of the queried values", which is the inverse of $all:

MongoDB:  ∀ e ∈ expected. ∃ a ∈ actual.   a == e   ⟺  expected ⊆ actual
Before:   ∀ a ∈ actual.   ∃ e ∈ expected. a == e   ⟺  actual ⊆ expected

The two coincide exactly when the sets are equal, which is the only case testAll covered. v6.1.1 had the correct direction; the quantifiers were transposed when the calculator moved to Truthness.

Fixing the direction also restores a usable gradient. Averaging over the document's elements meant that adding an element not in the expected list lowered the score, so the search was rewarded for shrinking the array rather than adding the missing values. Measured after the fix, for $all:[1,2,3]: 1.0000 → 0.8425 → 0.6850 → 0.2923 as more expected values go missing.

2. $ne between incomparable BSON types

computeHeuristicComparisonNonNullValues fell back to C_FALSE whenever no comparison logic was defined for a pair of types, regardless of operator. MongoDB treats values of different, incomparable BSON types as different from each other, so $ne on them holds. NOT_EQUALS_TO now returns TRUE_C in that fallback.

Ordering operators were already right: $gt/$gte/$lt/$lte genuinely do not match across BSON types, which is now pinned by a test so it does not get "fixed" later.

3. $nin ignored the elements of an array-valued field

$in checked each element; $nin passed the whole array to the comparison, which has no defined comparison against a scalar, so it fell back to false and the negation turned that into a full match. Both now share computeHeuristicForMembership and differ only by the negation, which is the point: they are definitionally complementary.

4. $not short-circuited on a missing field

$not returned true whenever the field was absent. That is right only for inner operators that do not match an absent field; $ne, $nin and $exists:false do match one, and negating them must then be false. The generic path below the shortcut already produced the correct answer in those cases, so the shortcut was discarding a correct result.

5. $all, $in and $nin each had their own idea of what "holds a value" means

MongoDB has a single rule: a field matches a value when its own value is that value, or
when it holds an array of which at least one element is. Both readings apply at once, so
{"tags": ["a","b"]} is matched both by ["a","b"] and by "a".

Each operator implemented a different half of it:

• $all handled only the array reading, so any non-array field scored false, even though
  {f:{$all:["a"]}} matches {f:"a"}.
• $in/$nin handled only the element reading, so they never compared the array itself, even
  though {tags:{$in:[["a","b"]]}} matches {tags:["a","b"]}.

computeHeuristicForMatchedValue now implements the rule once, and the three operators are
expressed in terms of it: $all is an AND over the expected values, $in an OR, and $nin the
negation of $in. That removes the special cases for scalar and empty arrays, which the rule
covers on its own, and is a net -11 lines.

Order sensitivity is covered too: ["b","a"] does not match ["a","b"], since arrays are equal
only element-by-element in order, while an array holding ["a","b"] as an element does match.

6. $eq and $ne did not apply that rule either

$eq compared the field's value as a whole, so it missed the half of the rule that looks inside
an array. $ne, being its negation, had the mirrored problem. Both now go through the same
helper, so a single value means the same thing for every operator.

This also removes a live inconsistency that predates this PR: $in with one value already
looked at array elements while $eq with the same value did not, although MongoDB treats the
two identically.

The whole-array reading already worked for $eq, since two lists were compared element by
element; only the reading that looks inside the array was missing.

Two latent crashes fixed along the way

Routing $nin through the shared membership path required guarding two cases that throw on current master:

{tags:{$in:["a"]}} against {tags: []}   -> IllegalArgumentException: null or empty Truthness instance
{tags:{$in:[]}}    against anything     -> IllegalArgumentException: null or empty Truthness instance

Both escape MongoHandler.computeFindDistance, which has no try/catch, and take the whole ExtraHeuristicsDto for that action with them. In MongoDB $in:[] matches nothing and $nin:[] matches everything, which is what the guards now produce.

A third one, older than this series and reachable from a plain {f:{$eq:[]}}, is fixed in the last commit: comparing two empty arrays took the branch for lists of equal size and aggregated over their zero elements, throwing the same exception instead of reporting the two as equal.

Tests

Unit — 21 tests added. Full Mongo suite on this branch: 293 tests, 0 failures, 0 skipped, including the Testcontainers-backed MongoHandlerTest and MongoScriptRunnerTest.

E2E — the existing endpoints exercise these operators only in shapes that were already correct: ne compares two numbers, nin is applied to a scalar field, and not wraps a $gt, which does not match an absent field. Three endpoints are added for the shapes that were broken:

endpoint	shape
neCrossType	$ne between a string field and a number
ninArrayField	$nin on a field holding an array
notExistsFalse	$not over an operator that matches an absent field

ninArrayField is deliberately not satisfied by either document that saveData inserts, since both hold "a", so it can only be covered by data EvoMaster generates itself. $all already has an endpoint and needed none.

Notes

MongoHeuristicsCalculator goes from 934 to 987 lines, ie just under the 1000-line guideline in for_developers.md. Worth flagging given how quickly this class is growing.

Point 6 is a behavioural change to the most widely used operator, since the implicit form
{f: "a"} parses to the same operation, so it is worth a closer look than the rest. It is the
last commit and can be dropped on its own if you would rather take it separately.

[LautaroPetaccio]

Closing this in favour of #1743, which carries the same findings as test cases only, with no change to the heuristics.

Each defect is there as its own @Disabled test, with the result a real MongoDB 7.0.40 gives for that exact query and document in the comment, so enabling one is a matter of deleting a single annotation. That should let the heuristic be changed in whatever order and increments suit it best.

The implementation I had written here is still on fix/mongo-query-semantics and fix/mongo-array-matching-and-parsing in my fork if it is ever useful as a reference, but it is not being proposed.