`@on_viceroy` test harness by erikrose · Pull Request #43 · fastly/compute-sdk-python

erikrose · 2026-02-17T21:47:54Z

Make it short and easy to run certain methods in a Viceroy guest.

Just decorate the method you want in Viceroy, and an ephemeral app is created; dispatch is handled; and serialization of params, return values, and exceptions is done in a standard way.

Advantages:

No on-disk project needed, which tidies up our examples dir
No more per-test-harness communication protocols
Lexical proximity of test assertions and in-Viceroy code
Tests are about 17% shorter (counting both test methods and their request handlers—even shorter if you count pyproject.toml and uv.lock).
Easier comprehension: It's simpler to trace function calls than request-routing patterns. An IDE can probably even do it.

This also adds support for request bodies in our WSGI façade, which we forgot to do.

This stacks atop #38, so review that first.

For a before and after, see the last commit, where I port the ConfigStore tests.

posborne · 2026-02-18T19:42:24Z

+try:
+    from componentize_py_types import Err  # noqa
+except ImportError:
+    sys.path.append(str(Path(__file__).parent.parent.parent / "stubs"))


This works in tree, but it is a goal of the testing infra we provide in fastly_compute to be distributable and usable by end-customers similar to how we use it in-tree.

Right. So that's another proposal I have: I think we ought to ship the stubs so customers can run typecheckers against their code. What do you think?

Yeah, I wasn't sure if we wanted to ship them or to provide the option to generate them (which should be possible). Part of the trouble is where we put them to avoid confusion -- having them on the path when you run the guest code causes problems from my recollection in one failed attempt to move things into tree (though would need to confirm again).

Since we have the version of componentize-py pinned down, we could generate them deterministically. But I'm not sure how to give customers the option to do so. Some new entrypoint or CLI flag? And then they'd have to arrange for that to be called by their testrunner (or outer build process, if they have one), and we'd have to document it and get them to read. It might be easier for everyone to just ship them as we ship the generated exceptions.

We could stick them any old place; I offer fastly_compute/stubs. It's not a package (having no __init__.py), so nobody will be tempted to import it via fastly_compute.stubs.... And it shouldn't interfere with guest-side imports because nobody put it on the PYTHONPATH. I've got that happening in tests/init.py, which I wouldn't expect guests to import. Here's the reasoning from the commit message:

Import stubs (once and only once) iff we're not under Viceroy. It turns out they are needed if we are to run typechecking on the host. In addition, some tests that can get along perfectly well with only stubs, saving a spin-up of Viceroy. Finally, a universal import of stubs on the host means we don't have to segment modules like utils.py into guest and host halves to avoid import errors.

We might be able to restate that path patch as a package-wide pytest fixture, but I'm not sure it would run early enough to satisfy imports.

posborne · 2026-02-18T19:47:08Z

+        /// Virtual environment in which to look for modules (default:
+        /// VIRTUAL_ENV env var or .venv)
+        #[arg(short, long)]
+        virtualenv: Option<PathBuf>,


So I understand, this was needed due to the acrobatics we're doing in the test harness and probably not likely to be needed by users normally (it is not for componentize-py which is the functionality I was largely reproducing).

Correct. It allows us to build against the venv the testrunner is running under, which elegantly has exactly the packages we need.

…decorator. * Add `@on_viceroy` decorator and supporting `AutoViceroyTestBase` test subclass. These work together to generate ephemeral Fastly projects under which Viceroy-side code is run. * Add `--virtualenv` option to `fastly-compute-py` so we can direct it to the env we're running under, even if it's not called ".venv" or if it's not pointed to by `VIRTUAL_ENV`. * Add Bottle to testing deps because we use it to communicate with the in-Viceroy code. * Move exceptions tests (which I'm using as a PoC) inside the package because their path needs to be importable for the machinery to work. But don't worry: we still don't ship them with the distro. * Remove unused FASTLY_COMPUTE_PY_BIN from makefile. * Import stubs (once and only once) iff we're not under Viceroy. It turns out they are needed if we are to run typechecking on the host. In addition, some tests that can get along perfectly well with only stubs, saving a spin-up of Viceroy. Finally, a universal import of stubs on the host means we don't have to segment modules like utils.py into guest and host halves to avoid import errors. * Add Jinja to config-store's uv.lock, which should have been in 87f875f. It got neglected in the rapid-fire landing of the config-store API and exception polishing.

...for consistency and to allow future use of `@on_viceroy`. We still exclude them when we publish the distribution.

This better reflects what it is: `_server` is initialized when the class is defined. More immediately, it lets `@on_viceroy` call `.request()` without needing to access a private attr.

Previously, we were just sucking in stdin, which isn't hooked up to anything useful.

In ruff's case this solves a bunch of really weird complaints about class methods: ``` ERROR Expected a callable, got `classmethod[Unknown, [**kwargs: Any], Any]` [not-callable] --> fastly_compute/tests/test_exception_remapping.py:82:13 | 82 | self.raise_int_by_surprise() ```

…er) proof of concept.

`del`ing the symbol counts as a use.

posborne

I'm OK moving forward with this as a building block, though I think a few parts of this might end up not being totally scalable:

The venv / sys.prefix bits might be finnicy in some deployments but it will be fine for our use.
As discussed, runtime path modifications are still a bit messsy, but I'm fine deferring.
Needing to compile potentially quite a few more wasms on each test run may start to add up over time. We might be able to parallelize around some of that and make it less of an issue, but we can probably deal with that as it comes. Binary caching based on a hash of the input text might also be workable (though we would need a way to also invalidate that cache), but I wouldn't build that until it becomes a sufficient annoyance.

erikrose · 2026-02-18T21:30:26Z

Thanks!

(1) sys.prefix it specced to work if we're under a virtualenv. ("If a virtual environment is in effect, this prefix will point to the virtual environment.") Outside one is the only iffy part. It may be fine.

(3) Ah, true, we would rebuild rather than using make to cache on disk. I hadn't thought of that. But yes, we could probably cache if it ever became a problem. Absent better invalidation ideas, we could key off a combination of generated text + mod dates of imports (transitively, which would be a pain). That would make it better than our make-based caching, which noticed changes in imports only insofar as we hard-coded them in the makefile. Currently we hard-code wsgi.py in there but (I just noticed) miss its from fastly_compute.exceptions.types.error import CannotRead dependency.

erikrose force-pushed the erik/in-viceroy-tests-2 branch from f0c0e2e to 14045a3 Compare February 17, 2026 22:24

erikrose mentioned this pull request Feb 17, 2026

Add logging support #35

Merged

posborne self-requested a review February 18, 2026 18:49

posborne reviewed Feb 18, 2026

View reviewed changes

Comment thread fastly_compute/testing.py Outdated

erikrose force-pushed the erik/in-viceroy-tests-2 branch 2 times, most recently from 9b42c5a to 9964d28 Compare February 18, 2026 19:37

posborne reviewed Feb 18, 2026

View reviewed changes

erikrose added 6 commits February 18, 2026 14:49

Move other tests inside the fastly_compute package...

478dd79

...for consistency and to allow future use of `@on_viceroy`. We still exclude them when we publish the distribution.

Make the server property a class method instead.

1531687

This better reflects what it is: `_server` is initialized when the class is defined. More immediately, it lets `@on_viceroy` call `.request()` without needing to access a private attr.

Support reading the HTTP request body in our WSGI wrapper.

ecb4a1b

Previously, we were just sucking in stdin, which isn't hooked up to anything useful.

Support passing params to @on_viceroy-decorated methods.

632908a

erikrose force-pushed the erik/in-viceroy-tests-2 branch from 9964d28 to c2ef65a Compare February 18, 2026 19:50

erikrose added 3 commits February 18, 2026 14:50

Port ConfigStore tests to @on_viceroy as another (in fact, a bett…

c2ef65a

…er) proof of concept.

Correct a comment: I did implement param handling.

19e8330

Don't need this "noqa" anymore.

b14cca4

`del`ing the symbol counts as a use.

posborne approved these changes Feb 18, 2026

View reviewed changes

Merge main into @on_viceroy branch to prepare for landing.

aae3e58

erikrose merged commit 0e04d5f into main Feb 18, 2026
4 checks passed

erikrose mentioned this pull request Feb 19, 2026

Ship stubs #46

Closed

erikrose deleted the erik/in-viceroy-tests-2 branch March 31, 2026 20:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`@on_viceroy` test harness#43

`@on_viceroy` test harness#43
erikrose merged 10 commits into
mainfrom
erik/in-viceroy-tests-2

erikrose commented Feb 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

posborne Feb 18, 2026

Uh oh!

erikrose Feb 18, 2026

Uh oh!

posborne Feb 18, 2026

Uh oh!

erikrose Feb 18, 2026

Uh oh!

posborne Feb 18, 2026

Uh oh!

erikrose Feb 18, 2026

Uh oh!

posborne left a comment

Uh oh!

erikrose commented Feb 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

erikrose commented Feb 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

posborne Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

erikrose Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

posborne Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

erikrose Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

posborne Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

erikrose Feb 18, 2026

Choose a reason for hiding this comment

Uh oh!

posborne left a comment

Choose a reason for hiding this comment

Uh oh!

erikrose commented Feb 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

erikrose commented Feb 17, 2026 •

edited

Loading