Between nightly versions 2022-01-14 and 2022-01-15, grcov 0.8.2 lost track of 82% of the coverage of BonsaiDb when using the LLVM coverage option (-Zinstrument-coverage).
Other projects of mine continue to have code coverage correctly measured. I'm not where to start narrowing this down, as I'm not getting any errors or warnings printed.
Here are the steps to get good code coverage reports out of 2022-01-14:
git clone https://github.com/khonsulabs/bonsaidb.git
cd bonsaidb
rustup toolchain install nightly-2022-01-14 --component llvm-tools-preview
LLVM_PROFILE_FILE=%m.profraw RUSTFLAGS="-Zinstrument-coverage" cargo +nightly-2022-01-14 test --workspace --all-features --all-targets
RUST_TOOLCHAIN=nightly-2022-01-14 grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --llvm -o coverage/ --ignore "target/*" --ignore "xtask/*"
If you look at the report in the coverage folder, the html report will show >80% coverage. Now, test with the next nightly:
# Clean up files
find . -name "*.profraw" -exec rm \{\} \;
rm -rf **/*.bonsaidb
rustup toolchain install nightly-2022-01-15 --component llvm-tools-preview
cargo clean
LLVM_PROFILE_FILE=%m.profraw RUSTFLAGS="-Zinstrument-coverage" cargo +nightly-2022-01-15 test --workspace --all-features --all-targets
RUST_TOOLCHAIN=nightly-2022-01-15 grcov . --binary-path ./target/debug/ -s . -t html --branch --ignore-not-existing --llvm -o coverage/ --ignore "target/*" --ignore "xtask/*"If you look at the report again, the html report now shows < 1% coverage.
The quantity of profraw files are the same, and the sizes of the files are more or less equivalent. I'm not quite sure where to start narrowing this down since it's a unique issue to this project of mine -- but the only unique part I'm aware of is its size 😅.
For now I'll pin our CI to the last working nightly. Thank you in advance for any suggestions or help!
Between nightly versions 2022-01-14 and 2022-01-15, grcov 0.8.2 lost track of 82% of the coverage of BonsaiDb when using the LLVM coverage option (
-Zinstrument-coverage).Other projects of mine continue to have code coverage correctly measured. I'm not where to start narrowing this down, as I'm not getting any errors or warnings printed.
Here are the steps to get good code coverage reports out of 2022-01-14:
If you look at the report in the coverage folder, the html report will show >80% coverage. Now, test with the next nightly:
If you look at the report again, the html report now shows < 1% coverage.
The quantity of profraw files are the same, and the sizes of the files are more or less equivalent. I'm not quite sure where to start narrowing this down since it's a unique issue to this project of mine -- but the only unique part I'm aware of is its size 😅.
For now I'll pin our CI to the last working nightly. Thank you in advance for any suggestions or help!