Repository navigation
feat(storage): add gcp.storage.client.response.body.size metric for read operations - #14747
nidhiii-27 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request implements response body size tracking for the 'ReadObject' operation across both gRPC and JSON implementations of Google Cloud Storage, updating BaseStorageReadChannel, GrpcStorageImpl, StorageImpl, and StorageMetricsContext, along with adding corresponding unit tests. The review feedback suggests refactoring a long chain of excluded key checks in StorageMetricsContext into a static Set constant to improve maintainability, and replacing fully qualified class names with imports in the new test cases to enhance readability.
| baseAttributes.forEach( | ||
| (key, val) -> { | ||
| if (!key.equals(StorageClientMetrics.KEY_NETWORK_PATH) | ||
| && !key.equals(StorageClientMetrics.KEY_RPC_RESPONSE_STATUS_CODE) | ||
| && !key.equals(StorageClientMetrics.KEY_HTTP_RESPONSE_STATUS_CODE) | ||
| && !key.equals(StorageClientMetrics.KEY_ERROR_TYPE) | ||
| && !key.equals(StorageClientMetrics.KEY_GCP_STORAGE_OPERATION) | ||
| && !key.equals(StorageClientMetrics.KEY_GCP_CLIENT_METHOD)) { | ||
| @SuppressWarnings("unchecked") | ||
| AttributeKey<Object> objKey = (AttributeKey<Object>) key; | ||
| bodyAttrBuilder.put(objKey, val); | ||
| } | ||
| }); |
There was a problem hiding this comment.
To improve maintainability and readability, avoid duplicating the long chain of !key.equals(...) checks. Instead, define a private static final Set<AttributeKey<?>> EXCLUDED_KEYS constant containing the excluded keys, and use !EXCLUDED_KEYS.contains(key) here.
baseAttributes.forEach(
(key, val) -> {
if (!EXCLUDED_KEYS.contains(key)) {
@SuppressWarnings("unchecked")
AttributeKey<Object> objKey = (AttributeKey<Object>) key;
bodyAttrBuilder.put(objKey, val);
}
});There was a problem hiding this comment.
Addressed in commit 404687b: extracted BODY_SIZE_EXCLUDED_KEYS and a buildBodySizeAttributes(StorageClientMetrics) helper method in StorageMetricsContext and used it for both request.body.size and response.body.size.
Co-authored by AI Agent
| io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader reader = | ||
| io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader.create(); | ||
| io.opentelemetry.sdk.metrics.SdkMeterProvider provider = | ||
| io.opentelemetry.sdk.metrics.SdkMeterProvider.builder() | ||
| .registerMetricReader(reader) | ||
| .build(); | ||
| StorageClientMetrics metrics = StorageClientMetrics.create(provider, true, false); | ||
|
|
||
| StorageMetricsContext context = StorageMetricsContext.create("ReadObject"); | ||
| context.setResponseBytes(4096L); | ||
| context.recordOperation(metrics, null); | ||
|
|
||
| java.util.Collection<io.opentelemetry.sdk.metrics.data.MetricData> collected = | ||
| reader.collectAllMetrics(); | ||
| io.opentelemetry.sdk.metrics.data.MetricData bodySizeData = | ||
| collected.stream() | ||
| .filter( | ||
| m -> | ||
| m.getName() | ||
| .equals(StorageClientMetrics.METRIC_GCP_STORAGE_CLIENT_RESPONSE_BODY_SIZE)) | ||
| .findFirst() | ||
| .orElse(null); | ||
|
|
||
| assertThat(bodySizeData).isNotNull(); | ||
| assertThat(bodySizeData.getHistogramData().getPoints()).hasSize(1); | ||
| io.opentelemetry.sdk.metrics.data.HistogramPointData point = | ||
| bodySizeData.getHistogramData().getPoints().iterator().next(); |
There was a problem hiding this comment.
Avoid using fully qualified class names (such as io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader) when they can be imported. This unnecessarily reduces code readability and violates the project's general rules.
InMemoryMetricReader reader = InMemoryMetricReader.create();
SdkMeterProvider provider =
SdkMeterProvider.builder()
.registerMetricReader(reader)
.build();
StorageClientMetrics metrics = StorageClientMetrics.create(provider, true, false);
StorageMetricsContext context = StorageMetricsContext.create("ReadObject");
context.setResponseBytes(4096L);
context.recordOperation(metrics, null);
Collection<MetricData> collected = reader.collectAllMetrics();
MetricData bodySizeData =
collected.stream()
.filter(
m ->
m.getName()
.equals(StorageClientMetrics.METRIC_GCP_STORAGE_CLIENT_RESPONSE_BODY_SIZE))
.findFirst()
.orElse(null);
assertThat(bodySizeData).isNotNull();
assertThat(bodySizeData.getHistogramData().getPoints()).hasSize(1);
HistogramPointData point =
bodySizeData.getHistogramData().getPoints().iterator().next();References
- Do not use fully qualified class names if there is no class name conflict in the file and the class is already imported, as it unnecessarily reduces code readability.
There was a problem hiding this comment.
Addressed in commit 404687b: imported HistogramPointData and replaced all inline fully qualified class names with simple class names and shared helper methods (findMetric, getSingleHistogramPoint) in StorageMetricsContextTest.
Co-authored by AI Agent
There was a problem hiding this comment.
Code Review
This pull request introduces tracking and recording of response body size metrics for "ReadObject" operations across both GRPC and JSON/HTTP storage implementations. It updates BaseStorageReadChannel, GrpcStorageImpl, and StorageImpl to capture the read or downloaded bytes and set them in the StorageMetricsContext, which now records the response body size metric. Corresponding unit tests are also added to StorageMetricsContextTest. The reviewer suggested avoiding the use of fully qualified class names in the new test cases to improve code readability, recommending importing them instead.
| io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader reader = | ||
| io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader.create(); | ||
| io.opentelemetry.sdk.metrics.SdkMeterProvider provider = | ||
| io.opentelemetry.sdk.metrics.SdkMeterProvider.builder() | ||
| .registerMetricReader(reader) | ||
| .build(); | ||
| StorageClientMetrics metrics = StorageClientMetrics.create(provider, true, false); | ||
|
|
||
| StorageMetricsContext context = StorageMetricsContext.create("ReadObject"); | ||
| context.setResponseBytes(4096L); | ||
| context.recordOperation(metrics, null); | ||
|
|
||
| java.util.Collection<io.opentelemetry.sdk.metrics.data.MetricData> collected = | ||
| reader.collectAllMetrics(); | ||
| io.opentelemetry.sdk.metrics.data.MetricData bodySizeData = | ||
| collected.stream() | ||
| .filter( | ||
| m -> | ||
| m.getName() | ||
| .equals(StorageClientMetrics.METRIC_GCP_STORAGE_CLIENT_RESPONSE_BODY_SIZE)) | ||
| .findFirst() | ||
| .orElse(null); | ||
|
|
||
| assertThat(bodySizeData).isNotNull(); | ||
| assertThat(bodySizeData.getHistogramData().getPoints()).hasSize(1); | ||
| io.opentelemetry.sdk.metrics.data.HistogramPointData point = | ||
| bodySizeData.getHistogramData().getPoints().iterator().next(); | ||
| assertThat(point.getCount()).isEqualTo(1L); | ||
| assertThat(point.getSum()).isEqualTo(4096.0); | ||
| assertThat(point.getAttributes().get(StorageClientMetrics.KEY_RPC_METHOD)) | ||
| .isEqualTo("ReadObject"); |
There was a problem hiding this comment.
Avoid using fully qualified class names (such as io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader, io.opentelemetry.sdk.metrics.SdkMeterProvider, java.util.Collection, io.opentelemetry.sdk.metrics.data.MetricData, and io.opentelemetry.sdk.metrics.data.HistogramPointData) when there is no class name conflict in the file and the classes are already imported or can be imported. This unnecessarily reduces code readability. Please import these classes and use their simple names instead. This also applies to the other new tests in this file.
InMemoryMetricReader reader =
InMemoryMetricReader.create();
SdkMeterProvider provider =
SdkMeterProvider.builder()
.registerMetricReader(reader)
.build();
StorageClientMetrics metrics = StorageClientMetrics.create(provider, true, false);
StorageMetricsContext context = StorageMetricsContext.create("ReadObject");
context.setResponseBytes(4096L);
context.recordOperation(metrics, null);
Collection<MetricData> collected =
reader.collectAllMetrics();
MetricData bodySizeData =
collected.stream()
.filter(
m ->
m.getName()
.equals(StorageClientMetrics.METRIC_GCP_STORAGE_CLIENT_RESPONSE_BODY_SIZE))
.findFirst()
.orElse(null);
assertThat(bodySizeData).isNotNull();
assertThat(bodySizeData.getHistogramData().getPoints()).hasSize(1);
HistogramPointData point =
bodySizeData.getHistogramData().getPoints().iterator().next();
assertThat(point.getCount()).isEqualTo(1L);
assertThat(point.getSum()).isEqualTo(4096.0);
assertThat(point.getAttributes().get(StorageClientMetrics.KEY_RPC_METHOD))
.isEqualTo("ReadObject");References
- Do not use fully qualified class names (such as
java.time.Duration) if there is no class name conflict in the file and the class is already imported, as it unnecessarily reduces code readability.
There was a problem hiding this comment.
Addressed in commit 404687b: replaced all inline fully qualified class names with imported simple class names and shared helper methods in StorageMetricsContextTest.
Co-authored by AI Agent
…se body size - Extract BODY_SIZE_EXCLUDED_KEYS and buildBodySizeAttributes(StorageClientMetrics) in StorageMetricsContext to deduplicate request/response body size attribute filtering. - Guard GrpcStorageImpl.readAllBytes response byte recording with error == null and move countBytes helper to the private helper section. - Wire GrpcStorageImpl.blobReadSession and BlobReadSessionAdapter to track delivered response bytes across all ReadProjectionConfig types and record gcp.storage.client.response.body.size on session close. - Track lastReadError in BaseStorageReadChannel so mid-stream read errors propagate to recordOperation on close while preserving partial read byte counts. - Replace inline FQCNs with imports in StorageMetricsContextTest and add unit tests for channel reads, StorageImpl, GrpcStorageImpl, and BlobReadSessionAdapter. [Generated-by: AI]
Summary
This PR adds the
gcp.storage.client.response.body.sizemetric for read operations.Changes
StorageMetricsContext:gcp.storage.client.response.body.sizeinsiderecordOperation(...)whenoperationNameis"ReadObject"andresponseBytes > 0.BODY_SIZE_EXCLUDED_KEYSandbuildBodySizeAttributes(StorageClientMetrics)to share attribute filtering betweenrequest.body.sizeandresponse.body.size.BaseStorageReadChannel/BlobReadChannelV2/GrpcBlobReadChannel:BaseStorageReadChannel.read(ByteBuffer)viametricsContext.addResponseBytes(read).lastReadErroracrossread(ByteBuffer)so mid-stream errors are passed torecordOperation(...)onclose()while preserving partial byte counts.StorageImpl:readAllBytesanddownloadTo(usingCountingOutputStream.getCount()across retries without double-counting).GrpcStorageImpl&BlobReadSessionAdapter:GrpcStorageImpl.readAllBytes(whenerror == null) and accumulates written bytes inGrpcStorageImpl.downloadToviacountBytes.GrpcStorageImpl.blobReadSessionandBlobReadSessionAdapterto accumulate delivered bytes across allReadProjectionConfigtypes (asFutureBytes(),asFutureByteString(),asChannel(),asSeekableChannel()) and record the operation onclose()(or on session open failure).StorageMetricsContextTest:recordOperationattribute filtering,BaseStorageReadChannel(full read, partial read + early close, 0-byte close, mid-stream error),StorageImpl(readAllBytes,downloadTowith retries),BlobReadSessionAdapter(all projections and open failure), andGrpcStorageImpl(readAllBytes,downloadTo,blobReadSession).