Skip to content

Every push() leaks its curl handle and socket until cycle GC (utopia-php/fetch closures bind $this) #52

Description

@loks0n

Summary

Filed here because utopia-php/fetch is archived and read-only, and this library is the live consumer that surfaces it (Sentry adapter, new Client() per push(); the LogOwl, Raygun and AppSignal adapters share the shape).

Every Utopia\Fetch\Client::fetch() call with the default Curl adapter leaves the CurlHandle alive until PHP's cycle collector runs. Until then the handle keeps its cached connection open (visible as a CLOSE_WAIT socket once the server closes its side) and libcurl's internal wakeup eventfd. In a long-lived worker that makes one request every minute or two and allocates little, the collector can go hours between sweeps, so the process accumulates dozens to hundreds of leaked fds and sockets, then drops them all at once.

Seen in production on utopia-php/logger (Sentry adapter, one new Client() per push) inside a Swoole worker: 135 fds after one hour, 67 CLOSE_WAIT sockets, 38 eventfds, all released later in a single sweep.

Cause

Adapter\Curl::send() registers two non-static closures on the handle:

CURLOPT_HEADERFUNCTION => function ($curl, $header) use (&$responseHeaders) { ... },
CURLOPT_WRITEFUNCTION  => function ($ch, $data) use ($chunkCallback, &$responseBody, &$chunkIndex) { ... },

A non-static closure created inside an instance method binds $this. So the adapter holds the handle ($this->handle), the handle holds the closures, and the closures hold the adapter. Refcounting cannot free that cycle; __destruct only runs when the collector breaks it. Neither closure uses $this.

This is independent of Swoole: it reproduces with plain PHP and with SWOOLE_HOOK_ALL, and with HTTP/1.1 as well as HTTP/2.

Reproduction

phpswoole/swoole:6.2.2-php8.3 (also reproduced on PHP 8.5.9 / curl 8.21.0), one POST every 3 s against a local Caddy with a 2 s idle timeout so the server closes idle connections:

$push = function () use ($url) {
    $adapter = new Curl(new CurlOptions(sslVerifyPeer: false, sslVerifyHost: false));
    $client = (new Client($adapter))->setTimeout(5000)->setConnectTimeout(1000);
    return $client->fetch(url: $url, method: Client::METHOD_POST, body: ['a' => 1])->getStatusCode();
};

Counting /proc/self/fd, eventfds and CLOSE_WAIT rows in /proc/net/tcp between pushes:

start                        fds=5 eventfds=0 close_wait=0
after 4 pushes (status 200)  fds=13 eventfds=4 close_wait=3
after 8 pushes (status 200)  fds=21 eventfds=8 close_wait=8
after 12 pushes (status 200) fds=29 eventfds=12 close_wait=11
after gc_collect_cycles()    fds=5 eventfds=0 close_wait=0

Identical numbers under SWOOLE_HOOK_ALL (fds 8 → 32 → 8). With the two closures changed to static function, the same run stays flat:

after 8 pushes (status 200)  fds=5 eventfds=0 close_wait=0

Fix options

The one-line fix (static function on both callbacks) belongs in utopia-php/fetch, which is archived, so from this library's side:

  1. Drop utopia-php/fetch from the adapters and use a plain CurlHandle owned by the adapter (or utopia-php/client, which the rest of the platform has moved to). The push is a single POST with three headers.
  2. Interim: after each push call gc_collect_cycles(), which is a hack but releases the handle deterministically.

Forcing HTTP/1.1 with Connection: close does not help: the handle, its eventfd and its socket are still held until the collector runs (measured, same counts).

Same defect reaches every other utopia-php/fetch consumer: in appwrite/cloud that is 34 first-party files plus utopia-php/agents, audit, emails, pay, vcs and appwrite/server-ce.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions