Skip to content

Foundation: Qt-free core, pytest suite, CI, and bug fixes - #1

Merged
adrobi merged 36 commits into
masterfrom
foundation
Oct 31, 2026
Merged

adrobi merged 36 commits into
masterfrom
foundation

Conversation

@adrobi

@adrobi adrobi commented Oct 31, 2026

Copy link
Copy Markdown
Owner

Summary

This PR does three things:

  • Bug fixes: a crash on an empty FASTA, IQ-TREE UFBoot runs with fewer than 1000 replicates, the non-existent HKY2+F+G4 model, and node angles in the circular layout.
  • "Foundation" refactor: spec in docs/superpowers/specs/2026-10-25-foundation-design.md, plan in docs/superpowers/plans/2026-10-25-foundation.md.
    • phyml_core no longer depends on Qt. It covers paths, settings, history, presets, alignment, launchers, the PhyML/IQ-TREE engines, the runner, the report and the CLI. pipeline.py is removed.
    • The window is split into modules (worker, log_console, settings_dialog, status_bar, params_panel, nav_panel, results_panel, fasta_input). The layout is unchanged.
    • Settings and history now live next to the exe, falling back to %APPDATA%\phymlGUI (PHYMLGUI_DATA_DIR overrides). Old files are migrated once, and damaged or unreadable files are protected from being overwritten.
    • Cancel kills the whole process tree, including MPI children. Only one run can be active at a time.
    • Tests moved to pytest (unit, gui, e2e) with needs_* binary markers. CI runs on windows-latest with Python 3.11 and 3.13, using IQ-TREE and MUSCLE pinned by SHA256.
  • RAxML-NG spike: a native Windows build. The recipe and patch are in docs/superpowers/spikes/2026-10-24-raxml-ng-windows/.

Testing

  • Locally: python -m pytest -q -rs gives 551 passed, 0 skipped (real PhyML, MS-MPI, MUSCLE and IQ-TREE).
  • Offscreen run on a temp data dir:
    • a single run and a batch;
    • cancelling an MPI run (no phyml-mpi/mpiexec/smpd left behind);
    • presets, history and switching engines.
  • Screenshots of the window before and after the split are pixel-identical.
  • Known deviation: phyml_ui/main_window.py is 462 lines against the spec's target of about 350.

🤖 Generated with Claude Code

adrobi and others added 30 commits October 24, 2026 05:22
…m the right splitter, scrollbars without the dotted page background
…tgroup, consensus, RF

- custom matplotlib renderer replaces Bio.Phylo.draw: three layouts
  (rectangular / slanted / circular with rings), leaf labels, italic
  support labels, light grid and a scale bar
- support threshold slider: branches >= threshold are accented, weaker
  ones fade; supports toggle; grid toggle
- double-click on a clade region collapses it to a single display leaf
  and back
- outgroup combo reroots the tree on the fly
- 'Consensus' button builds a majority-rule consensus from a
  *_phyml_boot_trees.txt file (Bio.Phylo.Consensus)
- 'Compare' button computes Robinson-Foulds distance (raw + normalized)
  between two Newick files with canonical split representation
… folder

- worker emits per-file status (pending/running/ok/fail); a new Queue
  tab lists the batch with live markers, and 'Retry failed files'
  restarts only the failed inputs
- run history (phyml_history.json, capped at 200): timestamp, input,
  OK count, model, duration, tree path; double-click reloads the tree;
  panel placeholder replaced by the real list
- presets: built-in Quick screening (NNI+aLRT) and Publication
  (GTR+BEST+bootstrap 1000) plus user presets saved from the current
  parameters (double-click the save row; right-click a user preset to
  delete); stored in the config
- 'Folder' status-bar button opens the output directory after a run
- engine selection in Settings (PhyML / IQ-TREE); IQ-TREE branch in the
  pipeline: -m MFP or explicit model, -T threads, --prefix, ultrafast
  bootstrap (-B) or aLRT (--alrt), per-file .iqtree/.treefile outputs
  parsed into the same stats/summary rows; ModelFinder's best-fit model
  lands in the log line, stats tab and CSV/XLSX
- engine-aware model lists (MFP + common +G4 models for IQ-TREE)
- quick alignment summary under the FASTA field: taxa count, site
  range, gap percentage
- summary is additionally written as phymlgui_summary.xlsx (openpyxl,
  optional dependency added to requirements)
- test suites moved into tests/ (smoke + e2e + gui logic + batch + tree
  viewer + iqtree); suites needing local analysis binaries print SKIP
  and exit 0 when absent, so the set is CI-friendly; scratch data stays
  in the gitignored testrun/
- tests/test_smoke.py: imports/i18n integrity, theme resolution, stats
  parsers, RF/splits math, log roles, offscreen main window — no
  binaries required
- .github/workflows/tests.yml: windows-latest smoke job on push/PR
- headless CLI: phymlgui.py --file/--batch [--config]; same pipeline,
  same i18n (locales.json now also resolved relative to the package so
  CLI works from any working directory)
- README updated for engines, viewer features, history/presets, CLI
…pane

- QPushButton.clicked(checked) leaked the bool into start_pipeline's
  files_override parameter after stage 3 (list(False) TypeError);
  connect via a no-arg lambda and guard bool defensively; the batch GUI
  test now drives the real button click so this path stays covered
- the tree pane gets a left margin so the viewer tool rows no longer
  stick to the splitter handle
…theme-aware canvas

- distance gridlines are now drawn manually at nice 1-2-5 intervals with
  numeric labels and an axis caption (ax.grid is hidden by axis_off,
  which is why the grid never appeared); circular layout gets rings
- support values moved to branch midpoints with rounded background
  badges and collision nudging that also keeps clear of taxon labels
- all paddings are proportional to tree size (the old absolute +0.5
  produced the huge empty right half on small-distance trees); leaf
  labels got a wider gap from branch tips
- grid/support checkboxes actually update the render state now (they
  were connected straight to _rerender without refreshing the fields)
- branches, labels, badges and the figure background follow the app
  theme via TreeViewerWidget.set_colors, wired from apply_theme;
  exports use the themed background instead of hardcoded white
Placing internal nodes at their deepest descendant's x made elbows
overrun short children, so branches left of the root (e.g. rodents
hanging off the basal trifurcation) drew leftward. Every node now uses
its own cumulative branch-length depth in all three layouts — verified
programmatically (0 of 14 horizontal segments go left) and visually.
…olytomy angles

- _process_one's outer except returned "" as stats, so run() died on
  stats.get() for an empty/invalid FASTA: the worker thread crashed,
  finished_signal never fired and the GUI stayed stuck with Run disabled
- IQ-TREE received the PhyML-sized replicate count (default 100) as -B,
  which UFBoot rejects ("#replicates must be >= 1000"); the value is now
  raised to 1000 via ufboot_replicates() with a log notice (en/ru)
- the IQ-TREE nucleotide model list offered "HKY2+F+G4", which IQ-TREE
  does not know; replaced with HKY+F+G4
- circular layout placed multifurcating nodes at the middle of the
  smallest gap between children instead of the middle of the arc that
  covers them (the circle minus the widest gap)

Regression tests: empty FASTA and ufboot_replicates in the smoke suite
(CI), 100-replicate run + every listed nt/aa model accepted by the real
binary in test_iqtree, polytomy angle in test_tree_viewer. The IQ-TREE
GUI test now also asserts the run status, not only the xlsx presence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Design (docs/superpowers/specs/2026-10-25-foundation-design.md), approved
section by section: Qt-free phyml_core (paths, settings, history, presets,
alignment, launchers, engines, runner), thin phyml_ui shell, data stored
next to the exe with %APPDATA% fallback and one-time legacy migration,
pytest + pytest-qt with binary markers, CI with pinned IQ-TREE/MUSCLE,
Python >= 3.11.

Spike (docs/superpowers/spikes/2026-10-24-raxml-ng-windows/): RAxML-NG
2.0.3 builds natively with MSYS2 UCRT64 using 4 configure flags and a
167-line patch; the result matches the official Linux binary (identical
bootstrap replicates, start trees and fixed-tree likelihoods, RF 0 on
all ML searches). Fixes include an RNG normalization segfault, aligned
alloc/free heap corruption and 15-bit rand() seeds that produced
duplicate bootstrap replicates on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ckouts

core.autocrlf would turn the RAxML-NG patch into CRLF on checkout, and a
CRLF patch no longer applies to the LF upstream sources (verified).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cale key check

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…migration

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… on unreadable files

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…m the record

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cy files once

The main window now takes AppPaths (detected by default) and reads and writes its
config and history through settings and HistoryStore instead of cwd-relative files.
Legacy files are copied once on start and logged. Tool paths and recent files inside
app_dir are stored relative. An unreadable config disables auto-save for the session
so defaults never replace it; save and history write errors go to the log.
History records carry the run params and per-file results; presets come from
phyml_core.presets and switch the engine. resource_path no longer depends on the cwd.
Tests get a data_dir fixture and a make_window factory that destroys its windows.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…nfig

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Native and Mpi launchers, the Engine ABC with Issue/EngineOutputs, and
PhyML(command, validate, executable, outputs, launcher, models). The
parse_stats function moves to engines/phyml.py as parse_phyml_stats and
pipeline.py imports it. Adds val_not_number, val_no_exe and val_no_mpiexec
to both locales.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rom engines

IqTree(command, validate, normalize, executable, outputs, launcher, models) with
parse_iqtree_stats and ufboot_replicates moved from pipeline.py, which imports them.
ENGINES now holds phyml and iqtree and the window asks it for the model list.
format_issue shows UI labels for parameter names in val_not_number, parse_decimal
accepts exponent notation, and param_text moves to engines/base.py.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ction

phyml_core/alignment.py: read_fasta opens the file itself (utf-8-sig, CRLF ok, no
leaked handle) and raises AlignmentError for empty or non-FASTA input; is_aligned,
clean_id, find_collisions, write_phylip (cleaned IDs, '.' -> '-', phylip-relaxed,
refuses colliding IDs with err_id_collision), summarize and the aligned/phylip path
helpers. The window's FASTA summary label now uses summarize. pipeline.py is
untouched until the new runner replaces it.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…cale encoding

read_fasta tries utf-8-sig first and falls back to the locale encoding for ANSI files
(cp1251 headers), as the old pipeline read every file; a file neither decodes gives an
AlignmentError with the decoder's message. write_phylip writes in the locale encoding
like the old conversion, so the tree viewer reads the resulting trees consistently; an
ID the codec cannot hold is an AlignmentError raised before the file is created. The
write_phylip docstring says the sequences must have equal lengths.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ess trees

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…side the runner lock

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e from the core

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… when the window closes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adrobi and others added 5 commits October 30, 2026 17:20
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…window

ParamsPanel holds the model/evolution/search groups (params() / set_params()
with one changed signal, set_engine() refills the models) and NavPanel the
recent/presets/history lists with signals. The results tabs (ResultsPanel),
the FASTA input row (FastaInput) and the settings file handling
(settings.SettingsStore, same migration/blocked-save rules) move out too.
Auto-save follows params.changed; the layout is pixel-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add packaging/third_party.json and scripts/fetch_third_party.py (SHA256-verified
downloads into bin/), a Python 3.11/3.13 CI matrix that caches bin/, README
sections on settings location and tests. The IQ-TREE GUI test now isolates the
app folder, since an empty path field falls back to bin/iqtree2.exe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve an empty mpi_np to the physical core count in the core so the CLI works on a fresh machine, report a damaged config and an unusable data folder from the CLI, show a message box when the window cannot start, fall back to mpiexec for a blank mpirun_path, use a dedicated message for history write errors, add network and job timeouts, drop dead locale keys and correct the README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
English Windows runners pipe child output as cp1252; the fake crashed printing
a Cyrillic path. The Cyrillic-path test now forces cp1252 everywhere.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adrobi
adrobi merged commit 055c5da into master Oct 31, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant