Skip to content

Autoharness: support BoundedArbitrary argument types - #4693

Open
tautschnig wants to merge 2 commits into
model-checking:mainfrom
tautschnig:autoharness-bounded-containers
Open

Autoharness: support BoundedArbitrary argument types#4693
tautschnig wants to merge 2 commits into
model-checking:mainfrom
tautschnig:autoharness-bounded-containers

Conversation

@tautschnig

@tautschnig tautschnig commented Jul 29, 2026

Copy link
Copy Markdown
Member

Description

Stacked on #4691 (only the last commit is new; review that one).

Autoharness can now generate harnesses for arguments whose types implement BoundedArbitrary rather than Arbitrary — reusing Kani's existing machinery (the trait, its Vec<T>/String/Box<[T]> implementations, and #[derive(BoundedArbitrary)]) directly: the generated harness calls kani::bounded_any::<T, 4>(). Detection uses the same Instance-resolution approach as implements_arbitrary, enabled by a new fn_marker on kani::bounded_any.

Like the slice/string support in #4691, this is gated behind --bounded-arguments and reported: without the flag, eligible functions are skipped with Requires --bounded-arguments for argument(s) ...; with it, harnesses are marked (bounded) in the summary table and the post-table note spells out that results only hold up to the bound.

The bound of 4 is measured, not guessed: BoundedArbitrary values are heap allocated, and String's implementation reasons about UTF-8 validity — at bound 8, even a trivial String-consuming harness exceeds the default 60s harness timeout.

Follow-up work is planned to explore unbounded generation for container types.

Testing

New script-based test cargo_autoharness_bounded exercising both modes: the skip-reason hint without the flag; with the flag, Vec<u8>/String consumption (passes), OOB indexing on possibly-empty values (fails, correctly), a user struct deriving BoundedArbitrary with a #[bounded] field (passes), a cover check proving max-length vectors with specific contents are generated (SATISFIED), and graceful skipping of Vec<Vec<u8>>.

Full autoharness suite, verify_std_cmd/std_codegen, and compiler/driver/metadata unit tests pass.

Towards #3832

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@tautschnig
tautschnig requested a review from a team as a code owner July 29, 2026 08:41
Copilot AI review requested due to automatic review settings July 29, 2026 08:41
@github-actions github-actions Bot added Z-EndToEndBenchCI Tag a PR to run benchmark CI Z-CompilerBenchCI Tag a PR to run benchmark CI labels Jul 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Kani’s autoharness generation to support argument types that implement BoundedArbitrary (e.g., Vec<T>, String, and user types deriving BoundedArbitrary) in addition to existing Arbitrary/compiler-derived Arbitrary support. It does so by adding model detection for kani::bounded_any (via a new fn_marker) and updating autoharness eligibility + MIR generation to call kani::bounded_any::<T, 4>() when appropriate, alongside the already-bounded slice/&str support.

Changes:

  • Add BoundedAnyModel marker and compiler-side support to detect and generate harness arguments via kani::bounded_any::<T, 4>() when T: BoundedArbitrary.
  • Introduce/extend Kani model functions for bounded slice and string references (AnySliceRefModel, AnyStrRefModel) and wire them into autoharness MIR generation.
  • Add script-based regression tests for bounded arguments and for slice/&str arguments; update autoharness filter expectations/docs accordingly.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/script-based-pre/cargo_autoharness_slices/src/lib.rs New test crate exercising autoharness support for &[T]/&mut [T] and &str, including cover checks and expected failures.
tests/script-based-pre/cargo_autoharness_slices/slices.sh Runs the new slices autoharness test.
tests/script-based-pre/cargo_autoharness_slices/slices.expected Expected output for the slices autoharness run (incl. skip + cover statuses).
tests/script-based-pre/cargo_autoharness_slices/config.yml Script-based test harness config for the slices test.
tests/script-based-pre/cargo_autoharness_slices/Cargo.toml Cargo manifest for the slices test crate.
tests/script-based-pre/cargo_autoharness_filter/src/lib.rs Updates the filter test to include a slice argument case as eligible.
tests/script-based-pre/cargo_autoharness_filter/filter.expected Updates expected output for the filter test (now including slice eligibility).
tests/script-based-pre/cargo_autoharness_bounded/src/lib.rs New test crate exercising autoharness support for BoundedArbitrary arguments (Vec, String, derived user type), plus skip for nested Vec<Vec<u8>>.
tests/script-based-pre/cargo_autoharness_bounded/config.yml Script-based test harness config for the bounded test.
tests/script-based-pre/cargo_autoharness_bounded/Cargo.toml Cargo manifest for the bounded test crate.
tests/script-based-pre/cargo_autoharness_bounded/bounded.sh Runs the new bounded autoharness test.
tests/script-based-pre/cargo_autoharness_bounded/bounded.expected Expected output for the bounded autoharness run (incl. skip + cover status).
library/kani_core/src/lib.rs Adds fn_marker for kani::bounded_any so the compiler can detect BoundedArbitrary support.
library/kani_core/src/arbitrary.rs Adds model helpers and markers for slice and &str generation (any_slice_ref, any_str_ref).
kani-compiler/src/kani_middle/transform/automatic.rs Extends autoharness MIR generation to use slice/str models and fall back to bounded generation for T: BoundedArbitrary.
kani-compiler/src/kani_middle/mod.rs Adds implements_bounded_arbitrary and extends argument-type eligibility to include BoundedArbitrary.
kani-compiler/src/kani_middle/kani_functions.rs Registers new KaniModel entries: AnySliceRef, AnyStrRef, BoundedAny.
kani-compiler/src/kani_middle/codegen_units.rs Updates autoharness partitioning logic to use the new unified “supported arg type” check and avoid cache poisoning.
docs/src/reference/experimental/autoharness.md Documents bounded slice/str support and bounded BoundedArbitrary argument generation + caveats.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +130 to 134
Nested slice references (e.g. `&&[u8]`) and slices inside user-defined types remain unsupported.

## Limitations
### Arguments Implementing Arbitrary
Kani will only generate an automatic harness for a function if it can represent each of its arguments nondeterministically, without bounds.
Comment on lines +162 to +166
/// models instead, which return a slice of nondeterministic length (bounded by
/// [AUTOHARNESS_SLICE_BOUND]) backed by a nondeterministic array stored in a dedicated local,
/// which stays alive for the entire harness.
/// Panics if `ty` does not implement Arbitrary (and is not a reference to a slice or str of
/// such a type).
Comment on lines 108 to 112
|--------------------------+----------------------------------------+----------------------------------------------------------------------------------|
| cargo_autoharness_filter | no_harness::unsupported_no_arg_name | Missing Arbitrary implementation for argument(s) _: *const i32 |
|--------------------------+----------------------------------------+----------------------------------------------------------------------------------|
| cargo_autoharness_filter | no_harness::unsupported_slice | Missing Arbitrary implementation for argument(s) _y: &[u8] |
|--------------------------+----------------------------------------+----------------------------------------------------------------------------------|
| cargo_autoharness_filter | no_harness::unsupported_vec | Missing Arbitrary implementation for argument(s) _y: std::vec::Vec<u8> |
+--------------------------+----------------------------------------+----------------------------------------------------------------------------------+
tautschnig and others added 2 commits July 29, 2026 09:18
Functions taking &[T]/&mut [T] (with T implementing or capable of deriving
Arbitrary) or &str were previously skipped with 'Missing Arbitrary
implementation'; in an evaluation across the top-100 crates.io crates these
argument types alone accounted for ~1400 skipped functions.

With the new --bounded-arguments option, the generated harness produces a
slice of nondeterministic length, backed by a nondeterministic array stored
in a dedicated harness local, via two new models: AnySliceRef returns
'&mut storage[..len]' with nondeterministic len (reborrowed as shared for
&[T] arguments), bounded at 16 elements; AnyStrRef returns the longest
valid-UTF-8 prefix of the nondeterministic bytes, bounded at 8 bytes,
reusing the algorithm of String's BoundedArbitrary implementation
(computing the valid prefix is a deterministic function of the bytes, which
symbolic execution handles well, whereas assuming from_utf8().is_ok() is
intractable even at a bound of 4 bytes; measured cost 10s@4/28s@8/180s@16,
so bound 8 stays within the default 60s harness timeout).

Because a verification result for such a harness only holds up to the
bound -- a bug that requires a larger input will not be found -- bounded
harness generation is strictly opt-in and visibly reported:
- without --bounded-arguments, eligible functions are skipped with the new
  reason 'Requires --bounded-arguments for argument(s) ...';
- with the option, such harnesses are marked '(bounded)' in the summary
  table (via a new HarnessMetadata::is_bounded field), and a note after the
  table spells out the caveat.

Both bounds sit below Kani's default unwinding bound of 20 so that loops
over generated slices can be fully unwound by default. Like raw pointers
(model-checking#4678), slice references are supported in argument position only, where
the harness owns the backing storage. Top-level argument verdicts are no
longer inserted into the shared Arbitrary cache at all, eliminating the
argument-position vs. field-position cache-poisoning hazard class
(implements_arbitrary memoizes its own recursion, so repeated argument
types stay cheap).

Towards model-checking#3832

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Functions taking owned container arguments like Vec<T> or String were
skipped with 'Missing Arbitrary implementation', even though Kani already
ships BoundedArbitrary implementations for exactly these types (and a
derive macro for user types).

Reuse that machinery directly: when an argument type does not implement
(and cannot derive) Arbitrary, but does implement BoundedArbitrary --
detected by the same Instance-resolution approach as implements_arbitrary,
via a new fn_marker on kani::bounded_any -- the generated harness calls
kani::bounded_any::<T, 4>() instead of kani::any(). This covers Vec<T>,
String, Box<[T]>, and user types deriving BoundedArbitrary.

The bound of 4 reflects measured verification cost: BoundedArbitrary values
are heap allocated, and String's implementation reasons about UTF-8
validity; a bound of 8 already makes simple String harnesses exceed Kani's
default 60s harness timeout. As with slices and strings, the
only-valid-up-to-the-bound caveat is documented in the autoharness
reference.

The eligibility loop no longer inserts top-level argument verdicts into the
shared Arbitrary cache at all: argument-position support (slice references,
BoundedArbitrary containers) differs from field-position support, and
implements_arbitrary already memoizes its own recursion, so the top-level
memoization was both redundant and a poisoning hazard.

Towards model-checking#3832

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Z-CompilerBenchCI Tag a PR to run benchmark CI Z-EndToEndBenchCI Tag a PR to run benchmark CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants