Skip to content

feat(service): Use string maps in place of structs#144

Merged
devsjc merged 1 commit intomainfrom
devsjc/struct-map-replacement
Mar 28, 2026
Merged

feat(service): Use string maps in place of structs#144
devsjc merged 1 commit intomainfrom
devsjc/struct-map-replacement

Conversation

@devsjc
Copy link
Copy Markdown
Contributor

@devsjc devsjc commented Mar 27, 2026

Contribution Checklist

  • Have you followed the Open Climate Fix Contribution Guidelines?
  • Have you referenced the Issue this PR addresses, where applicable?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added a summary of the changes?
  • Have you written new tests for your changes, where applicable?
  • Have you successfully run make lint with your changes locally?
  • Have you successfully run make test with your changes locally?

Warning

PRs may be closed if all the above boxes are not checked.

Changes in this Pull Request

Modifies the arbitrary Structs used in the proto schema to instead be string maps. These are much less CPU-intensive to serialize and deserialize, and simpler to reason around in generated code. We have ne requirement for arbitrary-depth generic structs in our current useage pattern, so this simplifies things a bit.

Note I've done it in a non-compatible manner since we're still in an experimental release phase. As such, all clients will need to be brought up to date at the same time.

@github-actions
Copy link
Copy Markdown

Benchmark Results

Benchmark results
?   	github.com/openclimatefix/data-platform/cmd	[no test files]
?   	github.com/openclimatefix/data-platform/internal/gen/ocf/dp	[no test files]
?   	github.com/openclimatefix/data-platform/internal/interceptors	[no test files]
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/dummy	0.005s
{"level":"debug","time":"2026-03-27T23:37:48Z","message":"Completed migrations"}
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkPostgresClient/small/GetForecastAsTimeseries-4         	      73	  15229045 ns/op
BenchmarkPostgresClient/small/GetForecastAtTimestamp-4          	     286	   4057499 ns/op
BenchmarkPostgresClient/small/GetObservationsAsTimeseries-4     	     942	   1229152 ns/op
BenchmarkPostgresClient/small/CreateForecast-4                  	      96	  12197950 ns/op
PASS
ok  	github.com/openclimatefix/data-platform/internal/server/postgres	70.657s
?   	github.com/openclimatefix/data-platform/internal/server/postgres/gen	[no test files]
Benchmark vs base branch
goos: linux
goarch: amd64
pkg: github.com/openclimatefix/data-platform/internal/server/postgres
cpu: AMD EPYC 7763 64-Core Processor                
                                                   │ bench-devsjc-struct-map-replacement.txt │
                                                   │                 sec/op                  │
PostgresClient/small/GetForecastAsTimeseries-4                                  15.23m ± ∞ ¹
PostgresClient/small/GetForecastAtTimestamp-4                                   4.057m ± ∞ ¹
PostgresClient/small/GetObservationsAsTimeseries-4                              1.229m ± ∞ ¹
PostgresClient/small/CreateForecast-4                                           12.20m ± ∞ ¹
geomean                                                                         5.517m
¹ need >= 6 samples for confidence interval at level 0.95

cpu: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz
                                                   │ bench-main.txt │
                                                   │     sec/op     │
PostgresClient/small/GetForecastAsTimeseries-4         17.34m ± ∞ ¹
PostgresClient/small/GetForecastAtTimestamp-4          3.650m ± ∞ ¹
PostgresClient/small/GetObservationsAsTimeseries-4     1.029m ± ∞ ¹
PostgresClient/small/CreateForecast-4                  11.38m ± ∞ ¹
geomean                                                5.218m
¹ need >= 6 samples for confidence interval at level 0.95

@devsjc devsjc merged commit 717f55d into main Mar 28, 2026
4 checks passed
@devsjc devsjc deleted the devsjc/struct-map-replacement branch March 28, 2026 07:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant