Autoharness: support BoundedArbitrary argument types - #4693
Open
tautschnig wants to merge 2 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Kani’s autoharness generation to support argument types that implement BoundedArbitrary (e.g., Vec<T>, String, and user types deriving BoundedArbitrary) in addition to existing Arbitrary/compiler-derived Arbitrary support. It does so by adding model detection for kani::bounded_any (via a new fn_marker) and updating autoharness eligibility + MIR generation to call kani::bounded_any::<T, 4>() when appropriate, alongside the already-bounded slice/&str support.
Changes:
- Add
BoundedAnyModelmarker and compiler-side support to detect and generate harness arguments viakani::bounded_any::<T, 4>()whenT: BoundedArbitrary. - Introduce/extend Kani model functions for bounded slice and string references (
AnySliceRefModel,AnyStrRefModel) and wire them into autoharness MIR generation. - Add script-based regression tests for bounded arguments and for slice/
&strarguments; update autoharness filter expectations/docs accordingly.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/script-based-pre/cargo_autoharness_slices/src/lib.rs | New test crate exercising autoharness support for &[T]/&mut [T] and &str, including cover checks and expected failures. |
| tests/script-based-pre/cargo_autoharness_slices/slices.sh | Runs the new slices autoharness test. |
| tests/script-based-pre/cargo_autoharness_slices/slices.expected | Expected output for the slices autoharness run (incl. skip + cover statuses). |
| tests/script-based-pre/cargo_autoharness_slices/config.yml | Script-based test harness config for the slices test. |
| tests/script-based-pre/cargo_autoharness_slices/Cargo.toml | Cargo manifest for the slices test crate. |
| tests/script-based-pre/cargo_autoharness_filter/src/lib.rs | Updates the filter test to include a slice argument case as eligible. |
| tests/script-based-pre/cargo_autoharness_filter/filter.expected | Updates expected output for the filter test (now including slice eligibility). |
| tests/script-based-pre/cargo_autoharness_bounded/src/lib.rs | New test crate exercising autoharness support for BoundedArbitrary arguments (Vec, String, derived user type), plus skip for nested Vec<Vec<u8>>. |
| tests/script-based-pre/cargo_autoharness_bounded/config.yml | Script-based test harness config for the bounded test. |
| tests/script-based-pre/cargo_autoharness_bounded/Cargo.toml | Cargo manifest for the bounded test crate. |
| tests/script-based-pre/cargo_autoharness_bounded/bounded.sh | Runs the new bounded autoharness test. |
| tests/script-based-pre/cargo_autoharness_bounded/bounded.expected | Expected output for the bounded autoharness run (incl. skip + cover status). |
| library/kani_core/src/lib.rs | Adds fn_marker for kani::bounded_any so the compiler can detect BoundedArbitrary support. |
| library/kani_core/src/arbitrary.rs | Adds model helpers and markers for slice and &str generation (any_slice_ref, any_str_ref). |
| kani-compiler/src/kani_middle/transform/automatic.rs | Extends autoharness MIR generation to use slice/str models and fall back to bounded generation for T: BoundedArbitrary. |
| kani-compiler/src/kani_middle/mod.rs | Adds implements_bounded_arbitrary and extends argument-type eligibility to include BoundedArbitrary. |
| kani-compiler/src/kani_middle/kani_functions.rs | Registers new KaniModel entries: AnySliceRef, AnyStrRef, BoundedAny. |
| kani-compiler/src/kani_middle/codegen_units.rs | Updates autoharness partitioning logic to use the new unified “supported arg type” check and avoid cache poisoning. |
| docs/src/reference/experimental/autoharness.md | Documents bounded slice/str support and bounded BoundedArbitrary argument generation + caveats. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+130
to
134
| Nested slice references (e.g. `&&[u8]`) and slices inside user-defined types remain unsupported. | ||
|
|
||
| ## Limitations | ||
| ### Arguments Implementing Arbitrary | ||
| Kani will only generate an automatic harness for a function if it can represent each of its arguments nondeterministically, without bounds. |
Comment on lines
+162
to
+166
| /// models instead, which return a slice of nondeterministic length (bounded by | ||
| /// [AUTOHARNESS_SLICE_BOUND]) backed by a nondeterministic array stored in a dedicated local, | ||
| /// which stays alive for the entire harness. | ||
| /// Panics if `ty` does not implement Arbitrary (and is not a reference to a slice or str of | ||
| /// such a type). |
Comment on lines
108
to
112
| |--------------------------+----------------------------------------+----------------------------------------------------------------------------------| | ||
| | cargo_autoharness_filter | no_harness::unsupported_no_arg_name | Missing Arbitrary implementation for argument(s) _: *const i32 | | ||
| |--------------------------+----------------------------------------+----------------------------------------------------------------------------------| | ||
| | cargo_autoharness_filter | no_harness::unsupported_slice | Missing Arbitrary implementation for argument(s) _y: &[u8] | | ||
| |--------------------------+----------------------------------------+----------------------------------------------------------------------------------| | ||
| | cargo_autoharness_filter | no_harness::unsupported_vec | Missing Arbitrary implementation for argument(s) _y: std::vec::Vec<u8> | | ||
| +--------------------------+----------------------------------------+----------------------------------------------------------------------------------+ |
Functions taking &[T]/&mut [T] (with T implementing or capable of deriving Arbitrary) or &str were previously skipped with 'Missing Arbitrary implementation'; in an evaluation across the top-100 crates.io crates these argument types alone accounted for ~1400 skipped functions. With the new --bounded-arguments option, the generated harness produces a slice of nondeterministic length, backed by a nondeterministic array stored in a dedicated harness local, via two new models: AnySliceRef returns '&mut storage[..len]' with nondeterministic len (reborrowed as shared for &[T] arguments), bounded at 16 elements; AnyStrRef returns the longest valid-UTF-8 prefix of the nondeterministic bytes, bounded at 8 bytes, reusing the algorithm of String's BoundedArbitrary implementation (computing the valid prefix is a deterministic function of the bytes, which symbolic execution handles well, whereas assuming from_utf8().is_ok() is intractable even at a bound of 4 bytes; measured cost 10s@4/28s@8/180s@16, so bound 8 stays within the default 60s harness timeout). Because a verification result for such a harness only holds up to the bound -- a bug that requires a larger input will not be found -- bounded harness generation is strictly opt-in and visibly reported: - without --bounded-arguments, eligible functions are skipped with the new reason 'Requires --bounded-arguments for argument(s) ...'; - with the option, such harnesses are marked '(bounded)' in the summary table (via a new HarnessMetadata::is_bounded field), and a note after the table spells out the caveat. Both bounds sit below Kani's default unwinding bound of 20 so that loops over generated slices can be fully unwound by default. Like raw pointers (model-checking#4678), slice references are supported in argument position only, where the harness owns the backing storage. Top-level argument verdicts are no longer inserted into the shared Arbitrary cache at all, eliminating the argument-position vs. field-position cache-poisoning hazard class (implements_arbitrary memoizes its own recursion, so repeated argument types stay cheap). Towards model-checking#3832 Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Functions taking owned container arguments like Vec<T> or String were skipped with 'Missing Arbitrary implementation', even though Kani already ships BoundedArbitrary implementations for exactly these types (and a derive macro for user types). Reuse that machinery directly: when an argument type does not implement (and cannot derive) Arbitrary, but does implement BoundedArbitrary -- detected by the same Instance-resolution approach as implements_arbitrary, via a new fn_marker on kani::bounded_any -- the generated harness calls kani::bounded_any::<T, 4>() instead of kani::any(). This covers Vec<T>, String, Box<[T]>, and user types deriving BoundedArbitrary. The bound of 4 reflects measured verification cost: BoundedArbitrary values are heap allocated, and String's implementation reasons about UTF-8 validity; a bound of 8 already makes simple String harnesses exceed Kani's default 60s harness timeout. As with slices and strings, the only-valid-up-to-the-bound caveat is documented in the autoharness reference. The eligibility loop no longer inserts top-level argument verdicts into the shared Arbitrary cache at all: argument-position support (slice references, BoundedArbitrary containers) differs from field-position support, and implements_arbitrary already memoizes its own recursion, so the top-level memoization was both redundant and a poisoning hazard. Towards model-checking#3832 Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
tautschnig
force-pushed
the
autoharness-bounded-containers
branch
from
July 29, 2026 09:24
8e11ec6 to
11e22fb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Stacked on #4691 (only the last commit is new; review that one).
Autoharness can now generate harnesses for arguments whose types implement
BoundedArbitraryrather thanArbitrary— reusing Kani's existing machinery (the trait, itsVec<T>/String/Box<[T]>implementations, and#[derive(BoundedArbitrary)]) directly: the generated harness callskani::bounded_any::<T, 4>(). Detection uses the sameInstance-resolution approach asimplements_arbitrary, enabled by a newfn_markeronkani::bounded_any.Like the slice/string support in #4691, this is gated behind
--bounded-argumentsand reported: without the flag, eligible functions are skipped withRequires --bounded-arguments for argument(s) ...; with it, harnesses are marked(bounded)in the summary table and the post-table note spells out that results only hold up to the bound.The bound of 4 is measured, not guessed:
BoundedArbitraryvalues are heap allocated, andString's implementation reasons about UTF-8 validity — at bound 8, even a trivialString-consuming harness exceeds the default 60s harness timeout.Follow-up work is planned to explore unbounded generation for container types.
Testing
New script-based test
cargo_autoharness_boundedexercising both modes: the skip-reason hint without the flag; with the flag,Vec<u8>/Stringconsumption (passes), OOB indexing on possibly-empty values (fails, correctly), a user struct derivingBoundedArbitrarywith a#[bounded]field (passes), a cover check proving max-length vectors with specific contents are generated (SATISFIED), and graceful skipping ofVec<Vec<u8>>.Full autoharness suite,
verify_std_cmd/std_codegen, and compiler/driver/metadata unit tests pass.Towards #3832
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.