Skip to content

Benchmark fixes#4194

Open
dsmiley wants to merge 8 commits intoapache:mainfrom
dsmiley:benchmark-fixes
Open

Benchmark fixes#4194
dsmiley wants to merge 8 commits intoapache:mainfrom
dsmiley:benchmark-fixes

Conversation

@dsmiley
Copy link
Contributor

@dsmiley dsmiley commented Mar 7, 2026

Our benchmark has some misc problems that have crept in because we don't have validation of them. We have a test but we don't run the benchmarks themselves. I'll separately work on getting a suitable CI job.

The following can be run in solr/benchmark, which runs all benchmarks with very dialed down settings, nearly as much as possible. Still; it does some work like indexing that can take time.

./jmh.sh -bm ss -wi 0 -i 1 -f 1 -r 5 -foe true -jvmArgs -ea

I ran that when done and examined stdout and stderr. a CI job will have to check the output for "Exception" since JMH often exits with status 0 when there were definitely problems.

I also found it necessary to use Java 25, as Java 21 on my machine was failing (at least 2x in a row) on ExitableDirectoryReaderSearch.

@@ -132,13 +132,18 @@ public IntegerMaxCardinalitySolrGen(int maxCardinality, Gen<Integer> integers) {

@Override
public Integer generate(SolrRandomnessSource in) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be called concurrently; needs to be thread-safe

public static class BenchState {

@Param({CommonParams.JAVABIN, CommonParams.JSON, "cbor", "smile", "xml", "raw"})
@Param({CommonParams.JAVABIN, CommonParams.JSON, "cbor", "smile", "xml"})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"raw" is special; isn't working (NPE) in this scenario. Don't think it makes sense to benchmark it.

throws SolrServerException, IOException {
return miniClusterState.client.request(benchState.q, collection);
NamedList<Object> response = miniClusterState.client.request(benchState.q, collection);
// consume the stream completely
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is necessary since otherwise the InputStreamResponseParser is left unconsumed and in fact it means the benchmark was mostly useless as nothing was transmitted since there was no consumer! wow

logClusterDirectorySize();

String orr = ObjectReleaseTracker.clearObjectTrackerAndCheckEmpty();
if (orr != null) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this helped track down 2 issues!


<!-- for versioning -->
<field name="_version_" type="long" indexed="true" stored="true"/>
<field name="_version_" type="long" indexed="true" stored="true" docValues="true"/>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need docValues on _version_. This ought to be a loud Solr error but it turned up here latently when a DBQ came in -- very weird.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant