[kz930]

What changes were proposed in this PR?

An operator that cuts a numeric column into bins and adds the bin a row fell in as a column of its own.

Two cuts, because they answer different questions. Equal width divides the span into bins of one size; equal frequency cuts at the quantiles, so the bins hold about as many rows as each other. The second is why the operator reads the whole table before it emits: a quantile is not known until the last row has arrived.

The bin reads as its own range, (2.5, 5.0], which is a label to group by rather than a number to do arithmetic on.

Any related issues, documentation, discussions?

Not part of #8325: that set makes the operators Texera already had exportable. This adds a new one, which implements the trait from the start rather than gaining it. The trait is in #8327, which has merged.

Closes #8479, the task this change is the whole of.

The same gap as #8478, on the other kind of column. Grouping by a continuous number puts almost every row in a group of its own, so counts per age bracket or per price band could not be asked for; a user wanting them had to write a Python UDF.

How was this PR tested?

Ten unit tests in the operator's own spec, over the schema it declares and the Python it emits on both paths. The parity harness runs it on four configurations, one per cut plus a hostile column name and an empty cell, comparing the engine's answer against the exported script's.

The operator in the editor, cutting score into four equal-width bins:

Each row's bin beside the value it came from, named by the range it fell in rather than by a number:

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aglinxinyuan
  You can notify them by mentioning @aglinxinyuan in a comment.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 94.54545% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.20%. Comparing base (cf6faeb) to head (c7122de).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
.../texera/amber/operator/binning/BinningOpDesc.scala	93.75%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #8481      +/-   ##
============================================
- Coverage     92.21%   92.20%   -0.02%     
- Complexity     4977     4995      +18     
============================================
  Files          1275     1277       +2     
  Lines         55649    55623      -26     
  Branches       7163     7167       +4     
============================================
- Hits          51318    51285      -33     
- Misses         2555     2557       +2     
- Partials       1776     1781       +5     

Flag	Coverage Δ		*Carryforward flag
access-control-service	77.44% <ø> (ø)
agent-service	99.53% <ø> (ø)		Carriedforward from cf6faeb
amber	86.79% <94.54%> (-0.05%)	⬇️
computing-unit-managing-service	64.02% <ø> (ø)
config-service	87.61% <ø> (ø)
file-service	82.29% <ø> (ø)
frontend	96.56% <ø> (-0.01%)	⬇️
notebook-migration-service	83.77% <ø> (ø)
pyamber	98.54% <ø> (ø)		Carriedforward from cf6faeb
workflow-compiling-service	77.95% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main cf6faeb benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	390	0.238	25,076/32,227/32,227 us	🔴 +28.5% / 🔴 +145.5%
🔴	bs=100 sw=10 sl=64	789	0.482	127,908/155,432/155,432 us	🔴 +20.8% / 🔴 +70.6%
⚪	bs=1000 sw=10 sl=64	979	0.597	1,021,095/1,063,785/1,063,785 us	⚪ within ±5% / 🔴 +21.5%

Baseline details

Latest main cf6faeb from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	390 tuples/sec	455 tuples/sec	931.98 tuples/sec	-14.3%	-58.2%
bs=10 sw=10 sl=64	MB/s	0.238 MB/s	0.278 MB/s	0.569 MB/s	-14.4%	-58.2%
bs=10 sw=10 sl=64	p50	25,076 us	19,516 us	10,818 us	+28.5%	+131.8%
bs=10 sw=10 sl=64	p95	32,227 us	32,689 us	13,127 us	-1.4%	+145.5%
bs=10 sw=10 sl=64	p99	32,227 us	32,689 us	16,504 us	-1.4%	+95.3%
bs=100 sw=10 sl=64	throughput	789 tuples/sec	891 tuples/sec	1,210 tuples/sec	-11.4%	-34.8%
bs=100 sw=10 sl=64	MB/s	0.482 MB/s	0.544 MB/s	0.739 MB/s	-11.4%	-34.8%
bs=100 sw=10 sl=64	p50	127,908 us	112,189 us	85,326 us	+14.0%	+49.9%
bs=100 sw=10 sl=64	p95	155,432 us	128,687 us	91,092 us	+20.8%	+70.6%
bs=100 sw=10 sl=64	p99	155,432 us	128,687 us	102,379 us	+20.8%	+51.8%
bs=1000 sw=10 sl=64	throughput	979 tuples/sec	988 tuples/sec	1,240 tuples/sec	-0.9%	-21.0%
bs=1000 sw=10 sl=64	MB/s	0.597 MB/s	0.603 MB/s	0.757 MB/s	-1.0%	-21.1%
bs=1000 sw=10 sl=64	p50	1,021,095 us	1,011,356 us	840,622 us	+1.0%	+21.5%
bs=1000 sw=10 sl=64	p95	1,063,785 us	1,051,421 us	889,172 us	+1.2%	+19.6%
bs=1000 sw=10 sl=64	p99	1,063,785 us	1,051,421 us	931,615 us	+1.2%	+14.2%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,512.70,200,128000,390,0.238,25076.13,32226.85,32226.85
1,100,10,64,20,2533.96,2000,1280000,789,0.482,127907.73,155432.27,155432.27
2,1000,10,64,20,20430.92,20000,12800000,979,0.597,1021095.35,1063785.19,1063785.19

[carloea2]

Additional boundary cases reproduced with compiled local tests.

[kz930]

@carloea2 the finding is fixed: a column with nothing in it skips the cut, so the rows keep null bin labels. Would you take another look when you have a moment?

[carloea2]

The all missing column case is handled now. Looks good.

[carloea2]

All 13 existing tests and formatting checks pass. A new parity test using the real table-operator lifecycle fails on stored NaNs in nullable Arrow input; the mixed-value control passes.

[kz930]

@carloea2 The stored-NaN case is fixed in 7d0ebe5, with a parity test for it. Could you take another look?

[carloea2]

The stored NaN fix passes. All 14 focused tests pass, including my original reproduction run through the real UDF lifecycle and generated export for both equal-width and equal-frequency binning. Mixed finite values agree; all stored NaNs produce empty bins. The UDF probe uses the #8766 SDK. Formatting passes. No new finding in this update.

sbt -batch 'WorkflowOperator/testOnly *BinningOpDescSpec -- -oDF' 'scalafmtCheckAll'

[carloea2]

All 13 focused tests and formatting checks pass on a fresh detached checkout. The binning implementation is unchanged from the stored-NaN fix verified earlier; this update fixes the test driver's encoding. The remaining CI failure is LoopIntegrationSpec, covered by #8938. Looks good. Leaving the draft status unchanged.

[carloea2]

Binning fails on valid LONG values.

[kz930]

@carloea2 Large LONG values now keep separate bins in both paths, and ordinary columns keep pandas' labels. Could you take another look?

[carloea2]

Equal-width binning still loses precision at the Int64 limits. The branch also conflicts with current main.

[kz930]

@carloea2 may you take another look?