test(verify): derive an operator's configuration from its schema - #8361
Open
kz930 wants to merge 5 commits into
Open
test(verify): derive an operator's configuration from its schema#8361kz930 wants to merge 5 commits into
kz930 wants to merge 5 commits into
Conversation
An operator is configured by a form, and the form is generated from the descriptor's own annotations. The same annotations are enough to fill it without a human: an enum offers its constants, a column knob takes a column of the type its rule allows, a numeric knob its declared bounds, and a conditional field is filled only where the branch that shows it is taken. This is one file because it is one decision made repeatedly — what a knob may hold — and splitting it would put half of that decision on either side of a boundary. `@SampleColumn` names the column a knob means where a type cannot say it: the opening price among four numbers, the country code among several strings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8361 +/- ##
============================================
- Coverage 94.11% 94.10% -0.02%
- Complexity 4811 4858 +47
============================================
Files 1197 1203 +6
Lines 48813 48967 +154
Branches 5906 5930 +24
============================================
+ Hits 45939 46079 +140
+ Misses 1420 1419 -1
- Partials 1454 1469 +15
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This was referenced Sep 2, 2026
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🟢 | bs=10 sw=10 sl=64 | 419 | 0.256 | 22,396/32,421/32,421 us | 🟢 -6.9% / 🔴 +102.7% |
| 🔴 | bs=100 sw=10 sl=64 | 920 | 0.562 | 106,588/158,176/158,176 us | 🔴 +10.4% / 🔴 +42.9% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,095 | 0.669 | 903,779/1,018,362/1,018,362 us | ⚪ within ±5% / 🟢 -10.8% |
Baseline details
Latest main f34905e from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 419 tuples/sec | 425 tuples/sec | 750.25 tuples/sec | -1.4% | -44.2% |
| bs=10 sw=10 sl=64 | MB/s | 0.256 MB/s | 0.259 MB/s | 0.458 MB/s | -1.2% | -44.1% |
| bs=10 sw=10 sl=64 | p50 | 22,396 us | 21,588 us | 13,041 us | +3.7% | +71.7% |
| bs=10 sw=10 sl=64 | p95 | 32,421 us | 34,827 us | 15,996 us | -6.9% | +102.7% |
| bs=10 sw=10 sl=64 | p99 | 32,421 us | 34,827 us | 19,395 us | -6.9% | +67.2% |
| bs=100 sw=10 sl=64 | throughput | 920 tuples/sec | 954 tuples/sec | 961.14 tuples/sec | -3.6% | -4.3% |
| bs=100 sw=10 sl=64 | MB/s | 0.562 MB/s | 0.582 MB/s | 0.587 MB/s | -3.4% | -4.2% |
| bs=100 sw=10 sl=64 | p50 | 106,588 us | 103,190 us | 103,974 us | +3.3% | +2.5% |
| bs=100 sw=10 sl=64 | p95 | 158,176 us | 143,338 us | 110,662 us | +10.4% | +42.9% |
| bs=100 sw=10 sl=64 | p99 | 158,176 us | 143,338 us | 120,690 us | +10.4% | +31.1% |
| bs=1000 sw=10 sl=64 | throughput | 1,095 tuples/sec | 1,093 tuples/sec | 991.91 tuples/sec | +0.2% | +10.4% |
| bs=1000 sw=10 sl=64 | MB/s | 0.669 MB/s | 0.667 MB/s | 0.605 MB/s | +0.3% | +10.5% |
| bs=1000 sw=10 sl=64 | p50 | 903,779 us | 912,668 us | 1,013,755 us | -1.0% | -10.8% |
| bs=1000 sw=10 sl=64 | p95 | 1,018,362 us | 987,959 us | 1,056,115 us | +3.1% | -3.6% |
| bs=1000 sw=10 sl=64 | p99 | 1,018,362 us | 987,959 us | 1,082,840 us | +3.1% | -6.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,477.59,200,128000,419,0.256,22396.33,32420.87,32420.87
1,100,10,64,20,2172.96,2000,1280000,920,0.562,106588.27,158176.36,158176.36
2,1000,10,64,20,18257.07,20000,12800000,1095,0.669,903778.76,1018362.22,1018362.22A candlestick reads four prices and the day they belong to, which a type cannot tell apart; a choropleth reads a country code the same way, and a value to shade it by that a type rule does narrow. Both are what the annotation and the rule are for. These read operators that land before this change does, so they belong here rather than being added back afterwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The generator did two jobs in one file: derive a valid configuration for an operator from the annotations it already carries, and sweep that configuration into the variants a parity run needs. At two thousand lines it was the largest change in the export series and the only one that could not be read in one sitting. This change leaves the first job. The sweep follows as its own change on top of it, restoring the members verbatim. The two jobs share no member. The base pass reaches its values through buildObject, decide and valueFor; the sweep reaches its own through rowFills, columnFill and leafFill, and the four entry points that return variants are the only callers of either. Nothing was rewritten to make the seam: whole members moved out, and every line that stays is the line that was there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… above Splitting the sweep out of this file moved members by their declaration, which left the block of documentation above three of them behind. Each then sat over the member that followed, describing something else: a JSON pointer helper's text over the one that reads a field's key, a nested-row accessor's over the one that reads a field's declared example, and a free-scalar test's over the type-id registry. None of them is wrong about the member it was written for. They travel with it. The class's own header and the column-resolution comment lose the parts a reader gets from the code beside them: an assembly strategy the next twenty lines carry out, and a numbered list of the tiers an `orElse` chain already reads as. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
An operator is configured by a form, and the form is generated from the
descriptor's own annotations. The same annotations are enough to fill it without
a human: an enum offers its constants, a column knob takes a column of the type
its rule allows, a numeric knob its declared bounds, and a conditional field is
filled only where the branch that shows it is taken.
@SampleColumnnames the column a knob means where a type cannot say it: theopening price among four numbers, the country code among several strings.
This produces one configuration. Turning that configuration into the several a
parity run needs is the change after this one, which is why this PR is now half
the size it was when it was opened. The two halves share no member: this side
reaches a value through
buildObject,decideandvalueFor, and the sweepreaches its own through
rowFills,columnFillandleafFill. Nothing wasrewritten to make the seam. Whole members moved out, and every line that stays
is the line that was there.
Any related issues, documentation, discussions?
Part of #8325, which lists the set in order. This is the first half of the
seventeenth entry; the sweep follows as the second.
How was this PR tested?
Seven tests here drive the generator across the config shapes it has to handle:
an operator with no config at all, a flat one with column knobs and an enum, a
nested list of objects with a free-form value, and the three ways a column knob
resolves, by sample, by type rule, and by staying off a column a sibling took.
Two of them read
@SampleColumnon operators that carry it only after theoperator families land, earlier in the sequence than this change. Until those
merge, the two report the first column instead of the named one.
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 5)