feat: add `plugboard process validate` CLI command by Copilot · Pull Request #212 · plugboard-dev/plugboard

Copilot · 2026-02-15T19:54:37Z

Summary

Adds a plugboard process validate CLI command that checks whether a YAML process config has valid topology (connected inputs, event matching, cycle detection) without running the process.

Changes

Add validate subcommand to plugboard/cli/process/__init__.py — builds the process from YAML config, runs validate_process(process.dict()), exits 0 on success or 1 with itemized errors on failure
Add two unit tests: valid config passes, mocked validation errors produce exit code 1

$ plugboard process validate tests/data/minimal-process.yaml
Validation passed

$ plugboard process validate bad-config.yaml
Validation failed:
  • Component 'x' has unconnected inputs: ['in_1']

Original prompt

This section details on the original issue you should resolve

<issue_title>Model validation (valid model topology etc)</issue_title>
<issue_description>Implement validation checks in plugboard-schemas for Process objects. Start with implementing the following checks:

Check that all component inputs are connected.

Check all components with input events, and make sure there exists in the process a component that outputs the required event type.

Check for circular connections within the process topology. These circular loops must only be considered valid if there are initial_values set on an appropriate component input within the loop.

For (3) we can use Johnson's algorithm to find all simple circuits within the process. Implement this as a utility in plugboard-schemas, i.e. try not to introduce additional dependencies. Info on the algorithm can be found at https://github.com/qpwo/python-simple-cycles/blob/master/johnson.py.</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes Model validation (valid model topology etc) #95

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

…, and validation utilities - Add _graph.py with Johnson's cycle-finding algorithm (no additional dependencies) - Add _validation.py with validation utilities for process topology: - validate_all_inputs_connected: check all component inputs are connected - validate_input_events: check input events have matching output producers - validate_no_unresolved_cycles: check circular connections have initial_values - Add model validator on ProcessSpec for circular connection detection - Export new utilities from plugboard_schemas - Add comprehensive unit tests Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

…t filter Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

toby-coleman

@copilot can you refactor as follows:

Instead of using the Pydantic validators, make all validators runtime checks.
Provide a validation utility in plugboard_schemas that be called at runtime.
- It must accept the output of process.dict() as input. The io key on the component dicts will provide the input events, output events, inputs, outputs.
Call the validator in process.init and raise an exception if it is invalid before running.

- Remove Pydantic model validator from ProcessSpec - Refactor all validators to accept process.dict() format - Add validate_process() entry point combining all checks - Call validate_process() in Process.init() raising ValidationError - Update plugboard.schemas re-exports - Update tests for new runtime validation approach Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Copilot · 2026-03-04T19:48:12Z

@copilot can you refactor as follows:

Instead of using the Pydantic validators, make all validators runtime checks.

Provide a validation utility in plugboard_schemas that be called at runtime....

Refactored in d96d7b2:

Removed the Pydantic model validator from ProcessSpec
All validators now accept process.dict() output, reading IO info from the io key on component dicts
Added validate_process() as a combined entry point in plugboard_schemas
Process.init() now calls validate_process(self.dict()) and raises ValidationError before proceeding

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

metadata.google.internal
- Triggering command: /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/bin/python /home/REDACTED/work/plugboard/plugboard/.venv/lib/python3.12/site-packages/ray/dashboard/dashboard.py --host=127.0.0.1 --port=8265 --port-retries=50 --temp-dir=/tmp/ray --log-dir=/tmp/ray/session_2026-03-04_19-44-28_997540_4481/logs --session-dir=/tmp/ray/session_2026-03-04_19-44-28_997540_4481 --logging-rotate-bytes=536870912 --logging-rotate-backup-count=5 --gcs-address=127.0.0.1:62478 --cluster-id-hex=ab1bd255a56bdfd4b906ce9485544aaa57eb6073c5f53816eae1ea00 --node-ip-address=127.0.0.1 --stdout-filepath=/tmp/ray/session_2026-03-04_19-44-28_997540_4481/logs/dashboard.out --stderr-filepath=/tmp/ray/session_2026-03-04_19-44-28_997540_4481/logs/dashboard.err k credential.userngit (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

github-actions · 2026-03-04T19:53:38Z

Benchmark comparison for 5cfd64c8 (base) vs d96d7b21 (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         459.0848 (1.0)      464.0447 (1.0)      461.2117 (1.0)      1.9287 (1.0)      461.3753 (1.0)      2.7119 (1.0)           2;0  2.1682 (1.0)           5           1
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     462.5718 (1.01)     480.1224 (1.03)     470.9983 (1.02)     6.4472 (3.34)     469.9811 (1.02)     7.8935 (2.91)          2;0  2.1231 (0.98)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

Adds a new `validate` subcommand to the CLI that checks whether a process defined in a YAML config file is valid. It builds the process and runs all topology validation checks (connected inputs, event matching, cycle detection). Exits with code 0 on success, code 1 with error details on failure. Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

github-actions · 2026-03-05T10:30:40Z

Benchmark comparison for 5cfd64c8 (base) vs 8b708b5e (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests ------------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     314.8777 (1.0)      337.5082 (1.0)      322.6089 (1.0)      9.1820 (1.25)     319.2368 (1.0)      12.1003 (1.30)          1;0  3.0997 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         323.2604 (1.03)     342.0885 (1.01)     330.6207 (1.02)     7.3363 (1.0)      331.1489 (1.04)      9.2782 (1.0)           2;0  3.0246 (0.98)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-03-10T21:04:11Z

Benchmark comparison for 5cfd64c8 (base) vs ad9cd897 (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests -----------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median               IQR            Outliers     OPS            Rounds  Iterations
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     450.8642 (1.0)      464.6043 (1.01)     455.3381 (1.0)      5.4980 (2.51)     452.9623 (1.0)      6.1325 (2.57)          1;0  2.1962 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         452.2409 (1.00)     458.2377 (1.0)      455.6421 (1.00)     2.1938 (1.0)      456.0474 (1.01)     2.3873 (1.0)           2;0  2.1947 (1.00)          5           1
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

github-actions · 2026-03-10T21:10:45Z

Benchmark comparison for 64adfa9a (base) vs b171c403 (PR)


------------------------------------------------------------------------------------------------------------------ benchmark: 2 tests ------------------------------------------------------------------------------------------------------------------
Name (time in ms)                                                                         Min                 Max                Mean            StdDev              Median                IQR            Outliers     OPS            Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_benchmark_process_run (main/.benchmarks/Linux-CPython-3.12-64bit/0001_base)     446.4639 (1.0)      462.3491 (1.0)      456.0794 (1.0)      6.8525 (3.33)     459.0854 (1.0)      11.2403 (4.11)          1;0  2.1926 (1.0)           5           1
test_benchmark_process_run (pr/.benchmarks/Linux-CPython-3.12-64bit/0001_pr)         459.0038 (1.03)     464.2752 (1.00)     462.1944 (1.01)     2.0566 (1.0)      462.1803 (1.01)      2.7321 (1.0)           2;0  2.1636 (0.99)          5           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean

codecov · 2026-03-11T00:28:22Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Initial plan

a057bd7

Copilot AI assigned Copilot and toby-coleman Feb 15, 2026

Copilot started work on behalf of toby-coleman February 15, 2026 19:54 View session

Copilot AI and others added 2 commits February 15, 2026 20:07

Address code review: remove unused ValidationError class and redundan…

fc0bf55

…t filter Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Add model validation checks for Process objects~~ feat: add process topology validation (cycle detection, input connectivity, event matching) Feb 15, 2026

Copilot AI requested a review from toby-coleman February 15, 2026 20:11

Copilot finished work on behalf of toby-coleman February 15, 2026 20:11

toby-coleman requested changes Mar 4, 2026

View reviewed changes

Copilot started work on behalf of toby-coleman March 4, 2026 19:36 View session

Copilot AI and others added 2 commits March 4, 2026 19:46

Add descriptive header to validation error message

d96d7b2

Co-authored-by: toby-coleman <13170610+toby-coleman@users.noreply.github.com>

Copilot AI requested a review from toby-coleman March 4, 2026 19:49

Copilot finished work on behalf of toby-coleman March 4, 2026 19:49

Copilot started work on behalf of toby-coleman March 4, 2026 22:23 View session

Copilot AI changed the title ~~feat: add process topology validation (cycle detection, input connectivity, event matching)~~ feat: add plugboard process validate CLI command Mar 4, 2026

Copilot finished work on behalf of toby-coleman March 4, 2026 22:29

Update tests for validation checks

ad9cd89

Merge branch 'main' into copilot/implement-model-validation-checks

b171c40

toby-coleman marked this pull request as ready for review March 11, 2026 17:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add `plugboard process validate` CLI command#212

feat: add `plugboard process validate` CLI command#212
Copilot wants to merge 8 commits intomainfrom
copilot/implement-model-validation-checks

Copilot AI commented Feb 15, 2026 •

edited

Loading

Uh oh!

toby-coleman left a comment

Uh oh!

Copilot AI commented Mar 4, 2026 •

edited

Loading

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions bot commented Mar 4, 2026

Uh oh!

github-actions bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 10, 2026

Uh oh!

github-actions bot commented Mar 10, 2026

Uh oh!

codecov bot commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Copilot AI commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Comments on the Issue (you are @copilot in this section)

Uh oh!

toby-coleman left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions bot commented Mar 4, 2026

Uh oh!

github-actions bot commented Mar 5, 2026

Uh oh!

github-actions bot commented Mar 10, 2026

Uh oh!

github-actions bot commented Mar 10, 2026

Uh oh!

codecov bot commented Mar 11, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Copilot AI commented Feb 15, 2026 •

edited

Loading

Copilot AI commented Mar 4, 2026 •

edited

Loading