From db58bc5a1cc135279e9ebacf19513f1070529093 Mon Sep 17 00:00:00 2001 From: Hendrik Ebbers Date: Fri, 25 Sep 2026 09:19:14 +0200 Subject: [PATCH] docs(release): start the 1.5.0 upgrade guide with the storage module MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds docs/releases/upgrade-to-1.5.md in the shape of the previous upgrade guides — a self-contained prompt an agent can run inside a consumer repo — covering the one consumer-facing change 1.5.0 carries so far. Marked as a draft, since 1.5.0 is unreleased and further sections will land with the features they describe. Writing the guard-rails section surfaced a contract violation between the two real implementations; recorded in docs/TODO.md as a fix that should land before 1.5.0 ships rather than being documented as behaviour. Co-Authored-By: Claude Opus 5 (1M context) --- docs/TODO.md | 13 +++ docs/releases/upgrade-to-1.5.md | 172 ++++++++++++++++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100644 docs/releases/upgrade-to-1.5.md diff --git a/docs/TODO.md b/docs/TODO.md index 8e93a1c..d203ac3 100644 --- a/docs/TODO.md +++ b/docs/TODO.md @@ -10,6 +10,19 @@ is everything below the wiring: reads past the end, and the incomplete-upload sweep. `S3ObjectStore`'s multipart boundary (the switch from a single `PutObject` to a multipart upload at exactly `PART_SIZE_BYTES`) is the other untested edge that matters. +- **Ranged reads diverge between the implementations, against the interface's own contract.** + `ObjectStore#get(String, long, long)` documents that "a range extending past the end returns the + available bytes rather than failing". `FileObjectStore` implements that — `offset >= size` yields + an empty stream. `S3ObjectStore` does not: it sends `Range: bytes=offset-last` and catches only + `NoSuchKeyException`, so a first-byte position at or past the object's length comes back as a raw + `S3Exception` (HTTP 416), not even wrapped as `ObjectStoreException`. A second divergence sits next + to it: for `length == 0` the file store throws `ObjectNotFoundException` when the key does not + exist ("a zero-length read of a missing key is a wrong key, not 'no bytes'"), while the S3 store + returns an empty stream without checking. Two backends behind one interface must not answer the + same call differently. **Fix before 1.5.0 ships** — the storage module is new in it, so this is + still cheap to correct. Found by reading the code while writing `docs/releases/upgrade-to-1.5.md`; + the 416 behaviour is the standard range semantics, not something that was run against a live + endpoint. - **`InMemoryObjectStore` is public API here, not a test fixture.** Its `failDeletes`, `failPuts`, `lastGetOffset` and `lastGetLength` are public mutable fields — fine inside one application, not as a published surface. `openelements.storage.type=memory` now makes it selectable in configuration, diff --git a/docs/releases/upgrade-to-1.5.md b/docs/releases/upgrade-to-1.5.md new file mode 100644 index 0000000..7f23d69 --- /dev/null +++ b/docs/releases/upgrade-to-1.5.md @@ -0,0 +1,172 @@ +# Upgrade prompt: spring-services 1.5.0 optional-module changes + +> **Draft — 1.5.0 is unreleased.** This document is written as the release takes shape; sections are +> added as features land. Everything described here is on `main` today. + +`spring-services` 1.5.0 brings one change so far. Every section below stands on its own — apply only +the ones you need: + +| Change | Spec | Nature | +| --- | --- | --- | +| Optional **object store** `ObjectStore` | — | new optional module, off by default, no schema change | + +## Object store (`spring-services-storage`) + +`spring-services` 1.5.0 adds an **optional** object-store module, `spring-services-storage`. It gives +an application one `ObjectStore` interface for binary payloads — files, images, exports, recordings — +with three implementations behind it: an S3-compatible endpoint, a local directory, and the heap. + +The feature is **off by default** and there is no database migration. Unless +`openelements.storage.type` is configured, the module registers no bean at all and there is nothing +to migrate. This guide applies only to consumers who want to turn it on. + +> Not a Java-API break: nothing existing changes. The module is purely additive. + +This file is a self-contained prompt for an agent (Claude Code, etc.) to run inside a consumer repo. + +--- + +## Prompt + +You are working inside a Spring Boot service that depends on `spring-services` and wants to store +binary objects. + +### 1. Depend on the storage module + +If you use `spring-services-all`, the module is already on the classpath — skip to step 2. Otherwise +add it (version managed by `spring-services-bom`): + +```xml + + com.open-elements + spring-services-storage + +``` + +### 2. Choose an implementation + +Unlike every other feature module, this one does **not** self-activate on classpath presence. It +ships three implementations and all three are always on the classpath, so nothing about the classpath +distinguishes them — and classpath order must not be what decides where your objects land. +`openelements.storage.type` makes the choice explicit. With it unset, no `ObjectStore` bean exists. + +**S3, or any S3-compatible endpoint** (AWS S3, Hetzner Object Storage, MinIO, …): + +```properties +openelements.storage.type=s3 +openelements.storage.s3.endpoint=https://s3.eu-central-1.amazonaws.com +openelements.storage.s3.region=eu-central-1 +openelements.storage.s3.bucket=my-objects +openelements.storage.s3.access-key=${S3_ACCESS_KEY} +openelements.storage.s3.secret-key=${S3_SECRET_KEY} +``` + +All five are required for `type=s3`. Credentials are high-value secrets — provide them from +environment variables or a secret manager, never in plaintext config. Path-style addressing is forced +and chunked encoding is disabled, because several S3-compatible providers support neither +virtual-host addressing nor trailing checksums. + +**A local directory:** + +```properties +openelements.storage.type=file +openelements.storage.file.root=/var/lib/my-app/objects +``` + +The directory is created if missing. The store owns it: it keeps objects under `root/objects/` and +in-flight writes under `root/uploads/`. + +**The heap**, for tests and local development only: + +```properties +openelements.storage.type=memory +``` + +### 3. Use it + +```java +@Service +class ExportService { + + private final ObjectStore store; + + ExportService(final ObjectStore store) { + this.store = store; + } + + void write(final String key, final InputStream data) { + store.put(key, data, "application/pdf"); // consumes and closes data + } + + InputStream read(final String key) { + return store.get(key); // you close it + } +} +``` + +The full surface: + +| Method | Notes | +| --- | --- | +| `put(key, data, contentType)` | streams in bounded parts; the payload is never held whole in memory | +| `get(key)` | the object's content stream; **the caller closes it** | +| `get(key, offset, length)` | a byte range, for seeking into a large object without reading it | +| `size(key)` | `OptionalLong` — empty means *no such object*, distinct from a zero-length one | +| `delete(key)` | idempotent; `true` if something was removed | +| `list(prefix)` | `Stream` of key, size and last-modified | +| `abortIncompleteUploadsOlderThan(instant)` | reclaims interrupted uploads; see the guard rails | + +Failures are `ObjectStoreException`; a read of a key that does not exist is `ObjectNotFoundException`. +Both are unchecked. + +### What you get + +- One interface over three backends, so the same code runs against S3 in production and a directory + or the heap in a test, decided entirely by configuration. +- Streaming in both directions. An upload of any length moves through in bounded 8 MiB parts — + single `PutObject` below that, multipart above — so payload size does not drive heap use. Note the + flip side: the S3 store allocates one 8 MiB buffer per in-flight `put`, whatever the payload's + size, so it is concurrency and not object size that sets the ceiling. +- A write that never becomes visible half-finished. The S3 store aborts its multipart upload on + failure; the file store streams into a scratch file and moves it into place, so a reader sees + either the previous object or the complete new one, never a truncated one. +- Declaring your own `ObjectStore` bean makes the library back off, whatever `type` says. + +### Guard rails / Don't do this + +- **`memory` loses every object when the process ends**, and each object sits on the heap for the + lifetime of the store. It exists so a test or a laptop needs no infrastructure. Never point a + deployed environment at it. +- **Nothing calls `abortIncompleteUploadsOlderThan` for you.** An upload interrupted mid-stream + leaves an incomplete multipart upload on S3 (or a scratch file on disk) that `list(prefix)` cannot + see — because S3 does not list them either — and that S3 **still bills you for**. If you accept + uploads, schedule this sweep yourself with a threshold comfortably past your longest legitimate + upload. The library ships no scheduler. +- **`put` consumes and closes the stream you hand it; `get` hands you one you must close.** A `get` + whose stream is never closed leaks a pooled HTTP connection on S3 and a file handle on disk. Use + try-with-resources. +- **`list(prefix)` returns a `Stream`, and on S3 it is lazy and paginated.** Consume it inside the + call rather than storing it; a prefix matching a large number of objects paginates as you iterate. +- **`delete` returning `false` is not a guarantee that nothing was there.** The S3 store asks for the + size first and then deletes, so the boolean reflects two round trips with a gap between them. Treat + it as information, not as a lock. +- **Keys are not equally portable across the implementations.** On S3 a key is an opaque string, so + `a/../b` is simply a key. In the file store a key is a path, so it is validated: an empty segment, + `.`, `..`, a backslash, or a leading/trailing `/` is rejected with `IllegalArgumentException`. Keys + that work against S3 can therefore fail against the file store. Generate keys from values you + control (UUIDs, hashes), never from unvalidated user input. +- **The file store ignores `contentType`.** A file system has nowhere to keep it and the interface + offers no way to read it back — your application stores an object's content type in its own + database. Do not expect the store to remember it. +- **There is no transaction integration.** A write that succeeded is not rolled back when the + surrounding database transaction fails. If you need the two to agree, write the object first and + record it second, then reconcile orphans with `list` — which is what + `abortIncompleteUploadsOlderThan` complements rather than replaces. +- **Maturity: the implementations themselves are not yet covered by tests.** The auto-configuration + is (activation, selection, misconfiguration), but the S3, file and in-memory stores arrived from an + application that exercised them in its own suite and the tests did not come along. Treat the file + and S3 stores as proven in one deployment rather than in this library, and weigh that before + adopting them for a new one. +- **The AWS SDK comes along even if you only use `type=file`.** `software.amazon.awssdk:s3` is a + hard dependency of the module today. Making it optional is planned; until then, budget for it in + your dependency footprint.