Repository navigation
feat(workflow-operator): volcano plot labels follow columns - #9040
eugenegujing wants to merge 6 commits into
Conversation
Name the y column and its axis -log10(<p-value column>) instead of the fixed "-log10(pvalue)". Add optional Title, X Label, Y Label and Color Bar Label properties that override the column-derived text when set; an empty or blank field keeps the default.
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9040 +/- ##
============================================
- Coverage 91.85% 91.79% -0.06%
- Complexity 5020 5032 +12
============================================
Files 1279 1273 -6
Lines 56192 55594 -598
Branches 7320 7156 -164
============================================
- Hits 51615 51033 -582
- Misses 2741 2750 +9
+ Partials 1836 1811 -25
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 550 | 0.336 | 17,171/26,216/26,216 us | 🟢 -7.3% / 🔴 +103.4% |
| 🟢 | bs=100 sw=10 sl=64 | 1,215 | 0.741 | 81,623/96,839/96,839 us | 🟢 -22.6% / 🔴 +8.8% |
| 🔴 | bs=1000 sw=10 sl=64 | 1,385 | 0.845 | 717,743/861,116/861,116 us | 🔴 +17.5% / 🟢 -12.0% |
Baseline details
Latest main d6c9050 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 550 tuples/sec | 584 tuples/sec | 953.99 tuples/sec | -5.8% | -42.3% |
| bs=10 sw=10 sl=64 | MB/s | 0.336 MB/s | 0.357 MB/s | 0.582 MB/s | -5.9% | -42.3% |
| bs=10 sw=10 sl=64 | p50 | 17,171 us | 17,862 us | 10,584 us | -3.9% | +62.2% |
| bs=10 sw=10 sl=64 | p95 | 26,216 us | 28,267 us | 12,888 us | -7.3% | +103.4% |
| bs=10 sw=10 sl=64 | p99 | 26,216 us | 28,267 us | 16,266 us | -7.3% | +61.2% |
| bs=100 sw=10 sl=64 | throughput | 1,215 tuples/sec | 1,172 tuples/sec | 1,242 tuples/sec | +3.7% | -2.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.741 MB/s | 0.715 MB/s | 0.758 MB/s | +3.6% | -2.3% |
| bs=100 sw=10 sl=64 | p50 | 81,623 us | 78,531 us | 83,370 us | +3.9% | -2.1% |
| bs=100 sw=10 sl=64 | p95 | 96,839 us | 125,052 us | 88,966 us | -22.6% | +8.8% |
| bs=100 sw=10 sl=64 | p99 | 96,839 us | 125,052 us | 101,186 us | -22.6% | -4.3% |
| bs=1000 sw=10 sl=64 | throughput | 1,385 tuples/sec | 1,451 tuples/sec | 1,276 tuples/sec | -4.5% | +8.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.845 MB/s | 0.886 MB/s | 0.779 MB/s | -4.6% | +8.5% |
| bs=1000 sw=10 sl=64 | p50 | 717,743 us | 687,911 us | 815,662 us | +4.3% | -12.0% |
| bs=1000 sw=10 sl=64 | p95 | 861,116 us | 732,771 us | 864,800 us | +17.5% | -0.4% |
| bs=1000 sw=10 sl=64 | p99 | 861,116 us | 732,771 us | 896,884 us | +17.5% | -4.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,363.72,200,128000,550,0.336,17171.22,26215.72,26215.72
1,100,10,64,20,1646.53,2000,1280000,1215,0.741,81622.80,96838.64,96838.64
2,1000,10,64,20,14443.13,20000,12800000,1385,0.845,717742.80,861115.90,861115.90
kz930
left a comment
There was a problem hiding this comment.
The color bar always shows the effect column, which is the same column as the x axis, so is a separate Color Bar Label needed, or could it follow X Label?
…label Drop the Color Bar Label property; the points are colored by the effect column that is on the x axis, so the color bar now carries the X Label text in both the engine template and the exported script.
…is-labels # Conflicts: # common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/visualization/volcanoPlot/VolcanoPlotOpDesc.scala
What changes were proposed in this PR?
Root cause:
VolcanoPlotOpDesc.generatePythonCodehard-codes the y column as the literal"-log10(pvalue)"and plots it under that name, so the y axis always says-log10(pvalue)even when the selected column ispadj. The title is the literal"Volcano Plot"and the x axis and color bar show the raw effect column name. None of this can be changed from the property panel.The y column is now named after the selected p-value column, so the axis and the hover text say which column was transformed. Three optional label properties are added, empty by default:
Volcano Plot-log10(<p-value column>)The color bar has no label of its own: the points are colored by the effect column, the same column that is on the x axis, so the color bar carries the X Label text (or the effect column name when X Label is empty). If a later change lets the color encode a different column, a label for it would come with that change.
The labels go through
fig.update_layout(xaxis_title, yaxis_title, coloraxis_colorbar_title_text), same as Line Chart's X Label / Y Label. Each label is anEncodableString, so quotes and non-ASCII text are fine. A whitespace-only field counts as empty. The hover box still shows raw column names (plotly behaviour, same as Line Chart).Behavior change for saved workflows: a workflow with
padjas the p-value column now renders-log10(padj)instead of-log10(pvalue). That is the fix.Since #8504 merged, the operator also has an export path,
generateStandaloneCode(), which writes the same figure as a standalone script. It carried the same hard-coded"-log10(pvalue)"andtitle="Volcano Plot", so this PR applies the same change there: the y column is named after the selected p-value column and the three labels go throughfig.update_layoutas escaped Python literals (the export has no decoder, sopyStringLiteralescapes quotes and backslashes). The engine template and the exported script now label the figure the same way, and a spec checks that across the label combinations.Rebase note:
coloraxis_colorbar_title_textis emitted unconditionally, in both templates. Once the significance-coloring PR (#9039) lands, the categorical mode has nocoloraxis, so whichever branch rebases second needs to make that entry conditional.Figures below are the operator's generated Python from
mainand from this branch, run on the 430-gene example workflow from the issue; the third one has the title and both axis labels filled in.Any related issues, documentation, discussions?
Closes #9037
PR #8504 (merged) added the
generateStandaloneCode()export path with the same hard-coded"-log10(pvalue)"; the mirroring described above is the follow-up it needed.How was this PR tested?
New tests in
VolcanoPlotOpDescSpec, written before the source change:generateStandaloneCode: the y column and axis are named after the p-value column (including a name with a quote,p"adj), the labels fall back to the column-derived defaults, a set label is emitted as an escaped literal and the default title is gone, the color bar carries the X Label text, whitespace-only keeps the default, a saved workflow renders as a default-constructed operator, and a cross-path test with the title and X Label set checks that the engine template and the exported script carry the same title, axis and color bar labelsSmoke run: the generated code was executed outside Texera (pytexera stubbed, pandas + plotly 5.24.1) on the 430-gene mock table. Defaults give title
Volcano Plot, xlfc_shrunk, y-log10(padj), color barlfc_shrunk; with the labels set the figure shows the typed text; whitespace-only title and y label fall back to the defaults. The exported script for the default and the all-labels configuration parses withast.parse.Was this PR authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (claude-fable-5-1). Co-authored with Claude; the author reviewed the change before submission.