Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,9 @@ http = "1.4.1"
http-body-util = "0.1.3"
futures-util = { version = "0.3.32", default-features = false }

[target.'cfg(unix)'.dependencies]
forkguard = { version = "0.1.3", features = ["atfork"] }

[dev-dependencies]
magnus = { version = "0.8.2", features = ["embed"] }

Expand Down
32 changes: 32 additions & 0 deletions docs/fork-safety.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Fork safety

## Why inherited clients are rejected

wreq-ruby uses a process-wide Tokio runtime and connection pool. `fork` copies
the parent's memory, but only the thread that called `fork` continues in the
child. Tokio's worker threads are gone, and its inherited tasks, locks, and
connections are not safe to reuse.

If the parent has already loaded wreq-ruby, native HTTP operations in the child
raise `Wreq::ForkError`. This applies to new and existing clients, module
request methods, streaming request bodies, and response body methods. Retrying
the operation in the same child raises the same error.

The parent can continue using its clients. When inherited Ruby objects are
collected in the child, their native runtime state is left for the operating
system to reclaim when the process exits.

## Child processes are unsupported

A process created with `fork` must not use wreq-ruby, even when it first loads
the extension after the fork. If the parent loaded wreq-ruby, native operations
in the child raise `Wreq::ForkError`.

When the extension was not present in the parent, no wreq-ruby state or fork
marker reaches the child. The extension cannot reliably distinguish that child
from a newly started process, so this unsupported path cannot guarantee a Ruby
error and may fail inside platform libraries.

Prefork servers should use an `exec`- or spawn-based worker model when workers
need wreq-ruby. Requiring the extension again does not reset inherited runtime
state, and there is no `after_fork!` hook.
12 changes: 12 additions & 0 deletions lib/wreq.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ module Wreq
# raise ArgumentError. Known values retain the error class from their Ruby
# or native conversion, such as TypeError or Wreq::BuilderError. Validation
# finishes before network I/O.
#
# If a child process inherits wreq-ruby from its parent, requests raise
# Wreq::ForkError. Require wreq after the worker has been forked.

# Send an HTTP request.
#
Expand Down Expand Up @@ -60,6 +63,7 @@ module Wreq
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.request(method, url, **options)
end

Expand Down Expand Up @@ -93,6 +97,7 @@ def self.request(method, url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.get(url, **options)
end

Expand Down Expand Up @@ -126,6 +131,7 @@ def self.get(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.head(url, **options)
end

Expand Down Expand Up @@ -159,6 +165,7 @@ def self.head(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.post(url, **options)
end

Expand Down Expand Up @@ -192,6 +199,7 @@ def self.post(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.put(url, **options)
end

Expand Down Expand Up @@ -225,6 +233,7 @@ def self.put(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.delete(url, **options)
end

Expand Down Expand Up @@ -258,6 +267,7 @@ def self.delete(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.options(url, **options)
end

Expand Down Expand Up @@ -291,6 +301,7 @@ def self.options(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.trace(url, **options)
end

Expand Down Expand Up @@ -324,6 +335,7 @@ def self.trace(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.patch(url, **options)
end
end
Expand Down
6 changes: 6 additions & 0 deletions lib/wreq_ruby/body.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ module Wreq
#
# A sender can be attached to one request. Closing it prevents further writes but
# retains queued chunks so a request attached afterward can still drain them.
# Creating or using a sender raises Wreq::ForkError if the child inherited
# wreq-ruby from its parent.
class BodySender
# Create a bounded request-body sender.
#
Expand All @@ -25,6 +27,7 @@ class BodySender
# @return [Wreq::BodySender] A streaming request body sender
# @raise [ArgumentError] if capacity is zero, negative, or too large
# @raise [TypeError] if capacity is not an Integer
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def self.new(capacity = 8)
end

Expand All @@ -33,6 +36,7 @@ def self.new(capacity = 8)
# @param data [String] binary chunk
# @return [nil]
# @raise [IOError] if the sender or receiving side is closed
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def push(data)
end

Expand All @@ -41,6 +45,7 @@ def push(data)
# This operation is idempotent.
#
# @return [nil]
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def close
end

Expand All @@ -50,6 +55,7 @@ def close
# the receiving side.
#
# @return [Boolean]
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def closed?
end
end
Expand Down
14 changes: 14 additions & 0 deletions lib/wreq_ruby/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@ module Wreq
# native conversion, such as TypeError or Wreq::BuilderError. Request
# validation finishes before network I/O.
#
# A child process cannot create or use a client if it inherited wreq-ruby
# from its parent. These calls raise Wreq::ForkError before accessing the
# native runtime. Require wreq after the worker has been forked.
#
# @example Basic usage
# client = Wreq::Client.new
# # Use client for HTTP requests
Expand Down Expand Up @@ -165,6 +169,7 @@ class Client
# value cannot be converted or validated.
# @raise [Wreq::BuilderError, Wreq::TlsError] if the native client cannot
# be initialized.
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
#
# @example Minimal client
# client = Wreq::Client.new
Expand Down Expand Up @@ -280,6 +285,7 @@ def self.new(**options)
# or unavailable on the current platform
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def request(method, url, **options)
end

Expand Down Expand Up @@ -313,6 +319,7 @@ def request(method, url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def get(url, **options)
end

Expand Down Expand Up @@ -346,6 +353,7 @@ def get(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def head(url, **options)
end

Expand Down Expand Up @@ -379,6 +387,7 @@ def head(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def post(url, **options)
end

Expand Down Expand Up @@ -412,6 +421,7 @@ def post(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def put(url, **options)
end

Expand Down Expand Up @@ -445,6 +455,7 @@ def put(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def delete(url, **options)
end

Expand Down Expand Up @@ -478,6 +489,7 @@ def delete(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def options(url, **options)
end

Expand Down Expand Up @@ -511,6 +523,7 @@ def options(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def trace(url, **options)
end

Expand Down Expand Up @@ -544,6 +557,7 @@ def trace(url, **options)
# @return [Wreq::Response] HTTP response
# @raise [TypeError, ArgumentError, Wreq::BuilderError] if a known option
# value cannot be converted, validated, or built
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def patch(url, **options)
end
end
Expand Down
13 changes: 13 additions & 0 deletions lib/wreq_ruby/error.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,19 @@ module Wreq
# Memory allocation failed.
class MemoryError < StandardError; end

# The child process inherited wreq-ruby from its parent.
#
# Tokio worker threads do not survive fork, and inherited pooled
# connections are not safe to reuse. This error is raised before a child
# can access that state.
#
# @example
# Process.fork do
# Wreq::Client.new # Raises if the parent loaded wreq-ruby.
# end
# @see https://github.com/SearchApi/wreq-ruby/blob/main/docs/fork-safety.md
class ForkError < RuntimeError; end

# Network connection errors

# Connection to the server failed.
Expand Down
8 changes: 8 additions & 0 deletions lib/wreq_ruby/response.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ module Wreq
# access to HTTP response data including status codes, headers, body
# content, and streaming capabilities.
#
# Body methods raise Wreq::ForkError if the child inherited wreq-ruby from
# its parent.
#
# @example Basic response handling
# response = client.get("https://api.example.com")
# puts response.status.as_int # => 200
Expand Down Expand Up @@ -107,6 +110,7 @@ def cookies

# Get the response bytes as a binary string.
# @return [String] Response body as binary data
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @example
# binary_data = response.bytes
# puts binary_data.size # => 1024
Expand All @@ -122,6 +126,7 @@ def bytes
# html = response.text("ISO-8859-1")
# puts html
# @raise [Wreq::DecodingError] if body cannot be decoded with the specified encoding
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
def text(default_encoding = "UTF-8")
end

Expand All @@ -132,6 +137,7 @@ def text(default_encoding = "UTF-8")
#
# @return [Object] Parsed JSON (Hash, Array, String, Integer, Float, Boolean, nil)
# @raise [Wreq::DecodingError] if body is not valid JSON
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @example
# data = response.json
# puts data["key"]
Expand All @@ -149,6 +155,7 @@ def json
# @raise [LocalJumpError] if called without a block
# @raise [Wreq::TimeoutError, Wreq::BodyError, Wreq::ConnectionResetError, Wreq::RequestError]
# if streaming fails while reading the response body
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @example Save response to file
# File.open("output.bin", "wb") do |f|
# response.chunks { |chunk| f.write(chunk) }
Expand All @@ -165,6 +172,7 @@ def chunks
# Close the response and free associated resources.
#
# @return [void]
# @raise [Wreq::ForkError] if the child inherited wreq-ruby from its parent
# @example
# response.close
def close
Expand Down
Loading