Remove Mechanical Markdown from examples/ validation#977
Remove Mechanical Markdown from examples/ validation#977
Conversation
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
b92a68e to
65fc9ae
Compare
There was a problem hiding this comment.
Pull request overview
This PR migrates example validation away from Mechanical Markdown to pytest-based integration tests, so the examples/ directory is validated via native Python tests that run the examples and assert on their output.
Changes:
- Replaces the
tox -e examplesMechanical Markdown workflow with a newtox -e integrationpytest environment. - Adds
tests/integration/with aDaprRunnerhelper + one pytest file per example to validate expected output. - Updates project docs (
README.md,AGENTS.md,examples/AGENTS.md) and removesexamples/validate.sh.
Reviewed changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Removes Mechanical Markdown example envs and adds integration env running pytest. |
| pyproject.toml | Registers the custom example_dir pytest marker. |
| README.md | Updates developer instructions to run tox -e integration. |
| AGENTS.md | Updates repo contributor guidance to refer to pytest integration tests instead of Mechanical Markdown. |
| examples/AGENTS.md | Rewrites examples validation docs to describe pytest/DaprRunner approach. |
| examples/validate.sh | Removes the Mechanical Markdown validation entrypoint script. |
| CLAUDE.md | Adds repository development guidelines. |
| tests/integration/conftest.py | Introduces DaprRunner, output assertion helper, and dapr fixture/marker handling. |
| tests/integration/test_workflow.py | Adds integration coverage for the workflow examples (task chaining, fan-out/fan-in, simple). |
| tests/integration/test_w3c_tracing.py | Adds integration coverage for w3c tracing example with background receiver. |
| tests/integration/test_state_store.py | Adds integration coverage for state store example output. |
| tests/integration/test_state_store_query.py | Adds integration coverage for state store query example with MongoDB setup. |
| tests/integration/test_secret_store.py | Adds integration coverage for secret store example (with/without ACL config). |
| tests/integration/test_pubsub_streaming.py | Adds integration coverage for streaming pubsub example (subscriber/publisher). |
| tests/integration/test_pubsub_streaming_async.py | Adds integration coverage for async streaming pubsub example (subscriber/publisher). |
| tests/integration/test_pubsub_simple.py | Adds integration coverage for basic pubsub example (subscriber/publisher). |
| tests/integration/test_metadata.py | Adds integration coverage for metadata example output. |
| tests/integration/test_langgraph_checkpointer.py | Adds integration coverage for langgraph checkpointer example (skips if deps missing). |
| tests/integration/test_jobs.py | Adds integration coverage for jobs example (management + processing). |
| tests/integration/test_invoke_simple.py | Adds integration coverage for invoke-simple using direct sidecar HTTP call to avoid infinite loop. |
| tests/integration/test_invoke_custom_data.py | Adds integration coverage for invoke-custom-data (receiver + caller). |
| tests/integration/test_invoke_binding.py | Adds integration coverage for invoke-binding with Kafka docker-compose setup. |
| tests/integration/test_grpc_proxying.py | Adds integration coverage for gRPC proxying example (receiver + caller). |
| tests/integration/test_error_handling.py | Adds integration coverage for error handling example output. |
| tests/integration/test_distributed_lock.py | Adds integration coverage for distributed lock example output. |
| tests/integration/test_demo_actor.py | Adds integration coverage for demo actor example (service + client). |
| tests/integration/test_crypto.py | Adds integration coverage for crypto examples with key generation/cleanup fixture. |
| tests/integration/test_conversation.py | Adds integration coverage for conversation examples output. |
| tests/integration/test_configuration.py | Adds integration coverage for configuration example with Redis seeding + update trigger. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 30 out of 30 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def _spawn(self, args: str) -> subprocess.Popen[str]: | ||
| return subprocess.Popen( | ||
| f'dapr run {args}', | ||
| shell=True, | ||
| cwd=self._cwd, | ||
| stdout=subprocess.PIPE, | ||
| stderr=subprocess.STDOUT, | ||
| text=True, | ||
| ) |
There was a problem hiding this comment.
DaprRunner uses shell=True with interpolated strings when spawning subprocesses. Even though current call sites are test-controlled, this makes quoting brittle and can become a command-injection footgun if any arguments ever come from variables/params; prefer passing argv lists with shell=False (and use shlex.split only when necessary).
|
@seherv pls see latest copilot feedback and build failures :) |
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
Description
Removes all references to Mechanical Markdown and replaces them with pytest files that validate the output generated by the examples.
Tests are still flaky and rely on exact log output, which is not ideal. This will be addressed in a future PR.
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #972 (first iteration, do not close the issue yet)
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: