Skip to content

[MEDIUM] Free COPY buffers when decoders raise - #742

Merged
larskanis merged 1 commit into
ged:masterfrom
OskarEichler:codex/security-free-copy-buffer
Sep 3, 2026
Merged

[MEDIUM] Free COPY buffers when decoders raise#742
larskanis merged 1 commit into
ged:masterfrom
OskarEichler:codex/security-free-copy-buffer

Conversation

@OskarEichler

Copy link
Copy Markdown
Contributor

Summary

PG::Connection#sync_get_copy_data releases the libpq buffer after decoding, but a decoder exception bypasses PQfreemem. Repeated rejected rows therefore retain every COPY buffer and can exhaust process memory.

This change wraps decoding in rb_ensure, so the libpq buffer is released on both success and exception. Successful decoding behavior is unchanged.

Reproduction

I used a custom PG::SimpleDecoder whose decode method raises, then rescued each error while reading four local COPY streams of 32 one-MiB rows. With current master, RSS rose each round:

start:    23,264 KiB
round 1:  91,280 KiB
round 2: 124,720 KiB
round 3: 158,176 KiB
round 4: 191,632 KiB

With this patch, memory stabilized after allocator warmup:

start:    22,160 KiB
round 1:  42,064 KiB
round 2:  48,336 KiB
round 3:  57,744 KiB
round 4:  57,760 KiB

Each run called GC.start before measuring.

Verification

  • Built the native extension on Ruby 4.0.6 / PostgreSQL libpq 15.
  • Repeated the four-round memory reproduction before and after the patch.
  • Focused COPY/type-map coverage: 18 examples, 0 failures.
  • Full suite: 1,111 examples, 8 failures, 9 pending. All eight failures are the existing local pg_service/libpq 15 expectation mismatch; the COPY coverage passed.
  • git diff --check passes.

Limitations

The memory measurements used a local PostgreSQL 15 server and RSS, so allocator retention accounts for the patched run's initial rise. No new automated spec is included.

Breaking changes

None.

@larskanis
larskanis merged commit 897844d into ged:master Sep 3, 2026
13 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants