Skip to content

Skip signed Model Target requests against real Vuforia by default - #3522

Open
adamtheturtle wants to merge 5 commits into
mainfrom
adamtheturtle/investigate-model-target-ci-failures
Open

Skip signed Model Target requests against real Vuforia by default#3522
adamtheturtle wants to merge 5 commits into
mainfrom
adamtheturtle/investigate-model-target-ci-failures

Conversation

@adamtheturtle

@adamtheturtle adamtheturtle commented Sep 1, 2026

Copy link
Copy Markdown
Member

Signed Model Target requests (advanced datasets with a state-based configuration) consume the real Vuforia account's small, shared training allowance, which cannot be raised or reset — running them on every CI run exhausted it within hours and has kept test_model_target_web_api.py red since the #3451 credentials landed. This skips the signed cases against the real Vuforia by default while still running them against every mock backend, and adds an opt-in --verify-model-target-signing pytest option to verify them for real, for example after the allowance recovers. The unsigned cases (standard datasets, and advanced datasets without a state-based configuration) keep verifying against the real Vuforia on every run. The reasoning is documented in the skip reason, code comments, and a new section in the contributing docs.

🤖 Generated with Claude Code

@adamtheturtle @claude
Skip signed Model Target requests against real Vuforia by default
475ffe8 
Creating an advanced Model Target dataset with a state-based
configuration is a signed request which consumes the Vuforia account's
small, shared, non-resettable training allowance. Running these cases
against the real Vuforia on every CI run exhausted the allowance and
turned every run red with TRAINING_ALLOWANCE_EXCEEDED.

Skip them against the real Vuforia by default, add a
--verify-model-target-signing option to opt back in, and document why.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
adamtheturtle and others added 4 commits September 1, 2026 11:56
@adamtheturtle
Bound test endpoint request timeouts
fe5577a
@adamtheturtle @claude
Report unexpected training allowance rejections as xfail
166c0c6 
On 2026-09-01 the real Vuforia began rejecting even unsigned
(standard) state-based dataset creation with
TRAINING_ALLOWANCE_EXCEEDED, so skipping only the signed cases no
longer keeps CI green. An unexpected allowance rejection is an
account-quota condition, not a fault in the code under test, so
assert_model_target_status now reports it as an expected failure
(xfail). The affected tests pass again automatically once the
allowance recovers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@adamtheturtle
Cover Model Target CI fallback paths
d6577ff
@adamtheturtle @claude
Cover VuMark database and no-timeout delay decorator paths
2c00dc4 
The combined CI coverage check found three uncovered regions in
src/mock_vws/decorators.py: registering VuMark databases added before
the mock is entered, restoring VuMark targets after a call of a
decorated function, and the response-delay path for a request with no
client timeout. Add tests for each.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@adamtheturtle