Skip to content

Autoharness: support &Wtf8 arguments under --bounded-arguments - #4809

Merged
feliperodri merged 4 commits into
model-checking:mainfrom
srivatsansamraj:wtf8-bounded-ref
Sep 30, 2026
Merged

feliperodri merged 4 commits into
model-checking:mainfrom
srivatsansamraj:wtf8-bounded-ref

Conversation

@srivatsansamraj

@srivatsansamraj srivatsansamraj commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

This adds a model for &Wtf8 arguments under --bounded-arguments. Before it, 32 functions in core on a whole-library verify-std run were skipped for a missing Arbitrary implementation.

WTF-8 is a superset of UTF-8. So any_wtf8_ref is Wtf8::from_str over the existing string model, any_str_ref, with the string bound. This covers only the UTF-8 subset: values holding unpaired surrogate code points are not generated, and the reference says so. Generating them needs an encoder that coalesces surrogate pairs; #4859 tracks it.

Wtf8 has no diagnostic item, so is_wtf8 matches it by name inside core::wtf8. &mut Wtf8 stays unsupported, as &mut str does. The kani crate gains #![feature(wtf8_internals)]. std does not re-export the module, so the model names the type through core.

Tests, in cargo_autoharness_wtf8 (same shape as cargo_autoharness_byte_str):

  • without --bounded-arguments, the functions are reported as requiring it;
  • with it, an unchecked index fails on the empty value;
  • covers show that the empty, the 4-byte and a specific string are generated.

This branch includes #4804 and #4806; only the last two commits are new here.

Resolves #4808

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

@srivatsansamraj
srivatsansamraj requested review from a team as code owners September 18, 2026 08:56
@github-actions github-actions Bot added Z-EndToEndBenchCI Tag a PR to run benchmark CI Z-CompilerBenchCI Tag a PR to run benchmark CI labels Sep 18, 2026
@feliperodri feliperodri added this to the Autoharness milestone Sep 20, 2026
@feliperodri feliperodri added the Z-Autoharness Issue related to autoharness subcommand label Sep 20, 2026

@feliperodri feliperodri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, with one docs line I would like before merge. This is stacked on #4804 and #4806, so it lands third.

The gap is real — 32 functions in core skipped — and the model is the right shape.

Verified rather than read:

  • Your test passes on this branch: 2 verified, the expected failure, 3/3 covers, and the without-flag skip line.
  • The coverage restriction is exactly what you describe, and I measured it: a cover for a lone surrogate (ED A0..BF ..) is UNSATISFIABLE, while empty and 4-byte covers are satisfied. So harnesses over &Wtf8 will report Success having explored only the UTF-8 half.
  • core::wtf8 does offer from_bytes_unchecked, so the conservative choice here was a choice, not a limitation — worth the praise inline.

The one thing I would fix before merge is the user-facing docs: they list &Wtf8 beside &str with only a length bound, so a reader will assume full WTF-8 coverage. One sentence naming the surrogate exclusion, as your description already does, closes it.

Also note this is behind main (now nightly-2026-09-22), and if you are rebasing the stack anyway, the inner-match drift I raised on #4804 is worth folding in here — with three ADTs sharing one guard, the unguarded catch-all now decides what an unrecognised ADT becomes.

Comment thread library/kani_core/src/arbitrary.rs Resolved
@feliperodri

Copy link
Copy Markdown
Member

@srivatsansamraj this one needs a rebase.

`&str` arguments are generated from nondeterministic harness-local storage by
`any_str_ref`, `&CStr` by `any_c_str_ref` and `&ByteStr` by `any_byte_str_ref`. `&Wtf8`
had no model and was skipped for a missing Arbitrary implementation: 32 functions in
`core` on a whole-library `verify-std` run.

WTF-8 is a superset of UTF-8, so `any_wtf8_ref` is `Wtf8::from_str` over `any_str_ref`,
with the string bound. Strings holding surrogate code points are not generated, since
`core` has no WTF-8 validator to build them soundly. Eligibility and harness generation
identify the type through `is_wtf8`, matched by name inside `core::wtf8` since the type
has no diagnostic item. `&mut Wtf8` stays unsupported, as `&mut str` does. The `kani`
crate gains `#![feature(wtf8_internals)]`; `std` does not re-export the module, so the
model names the type through `core`.

The test follows `cargo_autoharness_byte_str`: reported as requiring the flag without
it; with it, an unchecked index fails on the empty value, and covers show the empty, the
4-byte and a specific string are all generated.
The reference listed &Wtf8 beside &str with only a length bound, so a reader would assume full WTF-8 coverage; the model generates the UTF-8 subset only. The arm now repeats is_wtf8 like the other ADT arms.
auto-merge was automatically disabled September 28, 2026 07:00

Head branch was pushed to by a user without write access

@feliperodri
feliperodri added this pull request to the merge queue Sep 30, 2026
Merged via the queue into model-checking:main with commit 18ce41d Sep 30, 2026
34 checks passed
srivatsansamraj added a commit to srivatsansamraj/kani that referenced this pull request Sep 30, 2026
…checking#4862)

`&Wtf8` arguments now include unpaired surrogate code points. The model
keeps the longest well-formed WTF-8 prefix of its nondeterministic
storage, the way `any_str_ref` does for UTF-8. It no longer converts the
output of the `&str` model.

WTF-8 differs from UTF-8 in one row of the byte table: after `ED`, the
second byte may also be `A0..=BF`, which encodes a surrogate. One
sequence stays excluded: a lead surrogate (`ED A0..=AF ..`) directly
followed by a trail surrogate (`ED B0..=BF ..`). Well-formed WTF-8,
which `Wtf8::from_bytes_unchecked` requires, writes that pair as the
four-byte encoding of one supplementary code point, so the model ends
the prefix there. Every well-formed string up to the bound is still
generated: a string of length `k` arises from storage whose byte at `k`
does not continue it.

core has no WTF-8 validator, so the check is written out in the model.
Continuation bytes are checked by index, which leaves no inner loop for
symbolic execution to unwind. With this model the three harnesses in
`cargo_autoharness_wtf8` take 10 to 11 s each, against 15 to 17 s with
the `&str`-based one (one run each).

Tests:
- `tests/expected/wtf8-surrogates` calls the model at a bound of 6,
which holds two surrogates. It asserts that no lead surrogate is
followed by a trail surrogate. It covers a string ending in a lead
surrogate, one starting with a trail surrogate, and a trail followed by
a lead. With the previous model the three covers are unsatisfiable, and
without the pair check the assertion fails.
- `cargo_autoharness_wtf8` gains a cover for an unpaired surrogate,
unsatisfiable before this change.
- The docs no longer say surrogates are excluded.

This builds on model-checking#4809. Only the last commit is new here.

Resolves model-checking#4859

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 and MIT licenses.

---------

Co-authored-by: Srivatsan Samraj <321934658+srivatsansamraj@users.noreply.github.com>
srivatsansamraj pushed a commit to srivatsansamraj/kani that referenced this pull request Sep 30, 2026
…del-checking#4923)

Pass `--harness-timeout 5m` to
`script-based-pre/cargo_autoharness_fmt_impls`, as the other autoharness
tests (`bounded`, `bounds`, `byte_str`, `c_str`, `formatter`, `wtf8`)
already do.

**Why.** On the macOS x86_64 runners (`regression (macos-15-intel)`),
the `LowerExp`, `UpperExp` and `Pointer` harnesses of this test take
40–60 s, right at `kani autoharness`'s default `--harness-timeout` of 60
s. When one of them times out, CBMC reports `CBMC timed out` instead of
the expected failure, the test's `Failed Checks: "lower exp"` / `"upper
exp"` / `"pointer"` line is missing, and the test fails. The same
harnesses take about 10 s on Linux.

**Evidence.** I went through the logs of the `regression
(macos-15-intel)` jobs between 2026-09-28 and 2026-09-30. 33 of them
failed. This test failed in all 33, with 1–3 of those three harnesses
timing out each time, and in 32 of them it was the only failing test.
The one exception, model-checking#4903's merge-queue push run, also had
`cargo_autoharness_wtf8`'s `len` harness time out, at 5 m. On main
pushes alone, the job failed 7 times out of 12 on 2026-09-30. The
failures are independent of the changes under test:

- Today's automated toolchain upgrade, model-checking#4917: [run
36663872235](https://github.com/model-checking/kani/actions/runs/36663872235/job/109726991201)
(`LowExp`, `UpExp`, `Ptr` timed out; every other regression job passed)
- `main` pushes:
[36654469481](https://github.com/model-checking/kani/actions/runs/36654469481/job/109695682794),
[36656240395](https://github.com/model-checking/kani/actions/runs/36656240395/job/109701095388),
[36657849257](https://github.com/model-checking/kani/actions/runs/36657849257/job/109705923680),
[36659060566](https://github.com/model-checking/kani/actions/runs/36659060566/job/109709545862),
[36670355214](https://github.com/model-checking/kani/actions/runs/36670355214/job/109743820142),
[36674524840](https://github.com/model-checking/kani/actions/runs/36674524840/job/109756493668)
- Merge queue: model-checking#4895
[36650793954](https://github.com/model-checking/kani/actions/runs/36650793954/job/109684194253),
model-checking#4896
[36650908721](https://github.com/model-checking/kani/actions/runs/36650908721/job/109684568489),
model-checking#4809
[36654883143](https://github.com/model-checking/kani/actions/runs/36654883143/job/109696960300),
model-checking#4848
[36663305755](https://github.com/model-checking/kani/actions/runs/36663305755/job/109722546228),
model-checking#4903
[36667236036](https://github.com/model-checking/kani/actions/runs/36667236036/job/109734401431),
model-checking#4856
[36670723600](https://github.com/model-checking/kani/actions/runs/36670723600/job/109744937944),
model-checking#4881
[36596324505](https://github.com/model-checking/kani/actions/runs/36596324505/job/109502054372),
model-checking#4882
[36612997613](https://github.com/model-checking/kani/actions/runs/36612997613/job/109558873539)
- Pull requests: model-checking#4875
[36427316996](https://github.com/model-checking/kani/actions/runs/36427316996/job/109072277288),
model-checking#4895
[36443349794](https://github.com/model-checking/kani/actions/runs/36443349794/job/108999214513),
model-checking#4771
[36620184925](https://github.com/model-checking/kani/actions/runs/36620184925/job/109677883533)

The runner image version (`20260819.586`) was the same in passing and
failing runs.

**Manual testing.** Locally the three harnesses take about 10 s each.
Running the test with `--harness-timeout 3s` reproduces the CI failure
exactly: the same three harnesses time out, their `Failed Checks` lines
are missing, and the summary still reports 9 failures. With this change,
`cargo run -p compiletest -- --suite script-based-pre --mode exec
cargo_autoharness_fmt_impls` passes.

If the macOS Intel runner gets much slower, a longer timeout alone may
not be enough; making these three harnesses cheaper would be the next
step.

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 and MIT licenses.

Co-authored-by: Kiro <kiro-agent@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Z-Autoharness Issue related to autoharness subcommand Z-CompilerBenchCI Tag a PR to run benchmark CI Z-EndToEndBenchCI Tag a PR to run benchmark CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Autoharness: support &Wtf8 arguments under --bounded-arguments

3 participants