Skip to content

fix: don't buffer functions and don't keep them all in memory for duration of hashing and uploading#649

Open
pieh wants to merge 4 commits into
masterfrom
fix/dont-buffer-functions
Open

fix: don't buffer functions and don't keep them all in memory for duration of hashing and uploading#649
pieh wants to merge 4 commits into
masterfrom
fix/dont-buffer-functions

Conversation

@pieh

@pieh pieh commented Jun 28, 2026

Copy link
Copy Markdown

@netlify

netlify Bot commented Jun 28, 2026

Copy link
Copy Markdown

Deploy Preview for open-api ready!

Name Link
🔨 Latest commit f94e7bf
🔍 Latest deploy log https://app.netlify.com/projects/open-api/deploys/6a4643d3dd091d0007a51971
😎 Deploy Preview https://deploy-preview-649--open-api.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jun 28, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b9bcfbcf-70e5-4ed2-b1be-702d9289f11c

📥 Commits

Reviewing files that changed from the base of the PR and between cc7f932 and f94e7bf.

📒 Files selected for processing (2)
  • go/porcelain/deploy.go
  • go/porcelain/deploy_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • netlify/blueprints (manual)
🚧 Files skipped from review as they are similar to previous changes (2)
  • go/porcelain/deploy_test.go
  • go/porcelain/deploy.go

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes

    • Improved deployment uploads to stream packaged artifacts from disk for lower memory usage and more reliable transfers.
    • Fixed function artifact handling so reading, rewinding, and closing behave consistently whether data is buffered or path-backed.
    • Updated bundling behavior to correctly package and upload function artifacts, including during retries.
  • Tests

    • Added a regression test covering consistent read/seek/close behavior for streamed vs buffered artifacts.
    • Updated bundling-related tests to use temporary directories during packaging.

Walkthrough

The deploy bundling flow now writes function artifacts to temporary files on disk instead of relying on in-memory buffers. bundle, bundleFromManifest, and newFunctionFile accept a temp directory created by DoDeploy, and function uploads stream from FileBundle.Path. FileBundle also now supports lazy reads from disk when Buffer is nil, and tests were updated for the new temp-dir parameter and the disk-backed read/seek behavior.

Estimated code review effort: 4 (Complex) | ~40 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: streaming function artifacts instead of buffering them in memory.
Description check ✅ Passed The description is related to the changeset because it references the function upload issue this fix targets.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/dont-buffer-functions

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pieh pieh force-pushed the fix/dont-buffer-functions branch from feac31a to 3c653ec Compare June 28, 2026 19:55
@pieh pieh changed the title fix: don't buffer functions and keep them all in memory for duration of hashing and uploading fix: don't buffer functions and don't them all in memory for duration of hashing and uploading Jun 29, 2026
@pieh pieh marked this pull request as ready for review June 30, 2026 07:54
@pieh pieh requested a review from a team as a code owner June 30, 2026 07:54

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
go/porcelain/deploy.go (1)

129-133: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Make the FileBundle contract unambiguous.

Line 129 still says Path OR Buffer can be populated, but Lines 132-133 state uploads only read Path. Please update the comment so external callers don’t rely on Buffer for uploads.

Suggested docs tweak
-	// Path OR Buffer should be populated
+	// Path should be populated for uploads.
 	Path string
 
 	// Deprecated: uploads always stream from Path; this package no longer reads Buffer. It is retained
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@go/porcelain/deploy.go` around lines 129 - 133, Update the FileBundle
documentation in deploy.go so the contract is explicit that uploads use Path
only; the current comment near FileBundle still suggests “Path OR Buffer” is
valid, which conflicts with the deprecated Buffer note. Adjust the comment on
FileBundle (and its Path/Buffer fields) to state that Buffer is retained only
for backward compatibility and is not read by upload flow, so callers should
populate Path for uploads.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@go/porcelain/deploy.go`:
- Around line 270-276: The temp directory cleanup in deploy.go is ignoring the
result of os.RemoveAll, which golangci-lint flags and can hide cleanup failures.
Update the deferred cleanup around functionsTmpDir in the deploy flow to
explicitly handle and surface the RemoveAll error instead of discarding it,
using the existing bundle/deploy context to locate the cleanup path.
- Around line 966-990: The temp ZIP file in the bundle निर्माण flow is returned
before its close result is verified, so delayed write errors can be missed. In
the function that creates the archive and returns FileBundle, make sure the
temporary file close is checked before constructing the result, and propagate
any close error alongside the existing archive/IO errors. Keep the
hash/path/name logic in place, but only return the bundle after a successful
tmp.Close().

---

Nitpick comments:
In `@go/porcelain/deploy.go`:
- Around line 129-133: Update the FileBundle documentation in deploy.go so the
contract is explicit that uploads use Path only; the current comment near
FileBundle still suggests “Path OR Buffer” is valid, which conflicts with the
deprecated Buffer note. Adjust the comment on FileBundle (and its Path/Buffer
fields) to state that Buffer is retained only for backward compatibility and is
not read by upload flow, so callers should populate Path for uploads.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: cd3bfee8-0e21-4f4e-82e9-87c37138a146

📥 Commits

Reviewing files that changed from the base of the PR and between 517b027 and 3c653ec.

📒 Files selected for processing (2)
  • go/porcelain/deploy.go
  • go/porcelain/deploy_test.go
🔗 Linked repositories identified

CodeRabbit considers these linked repositories for cross-repo context during reviews:

  • netlify/blueprints (manual)

Comment thread go/porcelain/deploy.go Outdated
Comment thread go/porcelain/deploy.go Outdated
@pieh pieh force-pushed the fix/dont-buffer-functions branch from b263961 to 4acfab0 Compare June 30, 2026 11:04
@pieh pieh changed the title fix: don't buffer functions and don't them all in memory for duration of hashing and uploading fix: don't buffer functions and don't keep them all in memory for duration of hashing and uploading Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant