Skip to content

blake2 simd benches produce unexpected results #407

@nazar-pc

Description

@nazar-pc

I ran blake2 tests with and without simd feature enabled and found that behavior is strange and doesn't make a lot of sense.

Default (no simd):

$ cargo +nightly bench
...
running 8 tests
test blake2b512_10    ... bench:           8 ns/iter (+/- 0) = 1250 MB/s
test blake2b512_100   ... bench:          74 ns/iter (+/- 4) = 1351 MB/s
test blake2b512_1000  ... bench:         708 ns/iter (+/- 85) = 1412 MB/s
test blake2b512_10000 ... bench:       7,005 ns/iter (+/- 531) = 1427 MB/s
test blake2s256_10    ... bench:          13 ns/iter (+/- 0) = 769 MB/s
test blake2s256_100   ... bench:         121 ns/iter (+/- 7) = 826 MB/s
test blake2s256_1000  ... bench:       1,155 ns/iter (+/- 62) = 865 MB/s
test blake2s256_10000 ... bench:      11,791 ns/iter (+/- 1,005) = 848 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 28.22s

Simd:

$ cargo +nightly bench --features simd
...
running 8 tests
test blake2b512_10    ... bench:          11 ns/iter (+/- 0) = 909 MB/s
test blake2b512_100   ... bench:         102 ns/iter (+/- 0) = 980 MB/s
test blake2b512_1000  ... bench:       1,010 ns/iter (+/- 11) = 990 MB/s
test blake2b512_10000 ... bench:      10,076 ns/iter (+/- 68) = 992 MB/s
test blake2s256_10    ... bench:          16 ns/iter (+/- 0) = 625 MB/s
test blake2s256_100   ... bench:         152 ns/iter (+/- 1) = 657 MB/s
test blake2s256_1000  ... bench:       1,483 ns/iter (+/- 12) = 674 MB/s
test blake2s256_10000 ... bench:      14,750 ns/iter (+/- 344) = 677 MB/s

test result: ok. 0 passed; 0 failed; 0 ignored; 8 measured; 0 filtered out; finished in 2.43s

I expected simd version to produce better results and judging by the total execution time it is the case, but throughput reported by the benchmark says the opposite.

I suspect something is wrong with benchmarking code.

Using latest master (cc52373 right now) and rustc 1.65.0-nightly (20ffea693 2022-08-11).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions