Run client stacks under simulation

README.md

@@ -213,11 +213,15 @@ and the campaign results:
 ## Honest limits
 
 Code that goes through the event-loop API is supported; code that
-bypasses it is fenced: threads and executors, raw sockets, subprocesses,
-signals, and TLS raise `SimulationFenceError` rather than silently
-breaking determinism. Name resolution stays inside the simulation:
-`getaddrinfo` resolves sim host names to stable synthetic addresses and
-raises `socket.gaierror` for anything else — no real DNS, ever.
+bypasses it is fenced: threads and executors, raw socket reads and
+writes, subprocesses, signals, and loop-level TLS upgrades raise
+`SimulationFenceError` rather than silently breaking determinism.
+`sock_connect` on an `AF_INET` stream socket is the exception — it is
+simulated, so a client that connects a socket and hands it to
+`create_connection` runs, while the datagram and raw variants still fence.
+Name resolution stays inside the simulation: `getaddrinfo` resolves sim
+host names to stable synthetic addresses and raises `socket.gaierror` for
+anything else — no real DNS, ever.
 Write-side flow control is not simulated.
 The full contract is in [docs/supported-api.md](https://github.com/dhruvl/simloop/blob/main/docs/supported-api.md).
 What that contract costs real libraries — what aiohttp, anyio, websockets and

docs/compatibility.md

@@ -46,11 +46,12 @@ carries a date instead.
 | anyio | 4.14.2 | works: task group, memory object stream (one, two, three), anyio.sleep and move_on_after; virtual clock reached 1.75s | Asyncio backend only; nothing here touches a socket. |
 | redis (RESP wire protocol) | n/a | works: PING, SET and GET round trips over one connection: ['+PONG', '+OK', '0'] | Hand-rolled RESP over sim streams; no client library, no real server. |
 | websockets | 17.0.1 | works: handshake, one echoed frame ('HELLO') and close over ws:// | asyncio server and client on two sim hosts, ws:// only. |
-| aiohttp (client) | 3.14.3 | fenced: simloop does not simulate 'sock_connect'; see docs/supported-api.md for the supported asyncio subset | ClientSession GET at a sim host answered by a raw stream server. |
-| httpx | 0.28.1 | fails: AttributeError: 'NoneType' object has no attribute 'getpeername' | AsyncClient GET at a sim host answered by a raw stream server. |
+| aiohttp (client) | 3.14.3 | works: ClientSession GET returned 'hello from the simulation' | ClientSession GET at a sim host answered by a raw stream server. |
+| httpx | 0.28.1 | works: AsyncClient GET returned 'hello from the simulation' | AsyncClient GET at a sim host answered by a raw stream server. |
 
-Rows are grouped: the libraries expected to run on the loop first, then the
-client stacks expected to leave it.
+Rows are grouped: the libraries that need nothing but the loop and its
+streams first, then the client stacks that expect a socket object
+underneath them.
 
 ## Reading the rows
 
@@ -78,37 +79,67 @@ clients are built on — a length-prefixed request/response protocol on one
 long-lived connection — so the probe speaks RESP by hand against a small
 server on a second sim host, and the row claims no more than that.
 
-**aiohttp's client** leaves the simulation at its first connection attempt.
-The fence, verbatim:
+**aiohttp's client** issues its GET and reads the body back. Its connector
+resolves the name through `loop.getaddrinfo`, then hands the addresses to
+`aiohappyeyeballs`, which creates a real `AF_INET` stream socket, calls
+`loop.sock_connect` on it, and passes that socket to
+`loop.create_connection(sock=...)`. The simulation answers the sequence
+without letting the socket reach a network: `sock_connect` resolves the
+target against the host table and records it, moving no packet and no
+clock, and the `create_connection` call closes the real descriptor and
+opens a simulated connection to the recorded address, paying the same
+single round trip a direct `create_connection` would. The connector's
+`setsockopt(TCP_NODELAY)` lands on the stand-in object
+`get_extra_info("socket")` returns, which accepts option calls and does
+nothing with them.
+
+**httpx** completes the same request through httpcore and anyio. anyio
+resolves the name (the simulated resolver accepts the ASCII-encoded form
+anyio sends) and connects through `loop.create_connection`, with no socket
+of its own. The step that used to end this row comes next: httpcore asks
+the new stream who it is connected to, and anyio answers by reading the
+socket object out of `transport.get_extra_info("socket")`
+(`anyio/abc/_sockets.py`, `extra_attributes`) and calling `getpeername()`
+on it. A transport with no operating-system socket now answers with a
+stand-in that reports the peer's synthetic address and port, so the
+introspection succeeds and the response body comes back.
+
+Both client probes make one request against a responder that sends
+`Connection: close`, so neither row says anything about connection reuse.
+The piece a pool depends on is the descriptor `fileno()` returns: httpcore
+polls it to decide whether a pooled connection has died, and the
+simulation backs it with a parked descriptor the transport owns, which
+stays unreadable while the peer is alive and becomes readable once the
+peer's EOF arrives; a reset or a teardown closes it and `fileno()` returns
+`-1`, which the same poll reads as dead just as well. That contract is
+pinned by the test suite, not by these rows.
+
+Both client rows are `http://` only, and the two stacks stop differently
+on `https://`. aiohttp asks for TLS through `create_connection(ssl=...)`,
+which fences:
 
 ```
-simloop does not simulate 'sock_connect'; see docs/supported-api.md for the supported asyncio subset
+simloop does not simulate 'create_connection(ssl=...)'; see docs/supported-api.md for the supported asyncio subset
 ```
 
-Its connector resolves the name through `loop.getaddrinfo` (which the
-simulation answers), then hands the addresses to `aiohappyeyeballs`, which
-opens a real socket and calls `loop.sock_connect` on it. Raw sockets are
-fenced, so the simulation stops there rather than letting a real connection
-out.
-
-**httpx** never reaches a fence, and gets further than its verdict looks.
-It goes through httpcore and anyio; anyio resolves the name (the simulated
-resolver accepts the ASCII-encoded form anyio sends), connects through
-`loop.create_connection`, and the connection *succeeds*. The failure is
-introspection after the fact: httpcore asks the new stream for its local
-and remote addresses, anyio answers by reading the raw socket object out of
-`transport.get_extra_info("socket")` (`anyio/abc/_sockets.py`,
-`extra_attributes`), and the simulation's honest answer for a transport
-with no operating-system socket is `None`. Whether a request would complete
-past that line is untested here.
+httpx reaches no fence. httpcore wraps the byte stream with anyio's
+`TLSStream`, which drives an `ssl` memory BIO inside the process and
+sends the handshake as ordinary bytes over the simulated connection, so
+`loop.start_tls` is never called and nothing stops the attempt. Where it
+ends is up to whatever is listening: aimed at the plaintext responder
+these probes use, the handshake goes unanswered and the request dies of
+httpx's own `ConnectTimeout`. That timeout is a one-off measurement
+rather than a row — no probe on this page requests `https://`.
 
 ## Not tested
 
 - **asyncpg**: reaching its first fence needs a live PostgreSQL server to
   connect to, which no probe can provide; it is untested rather than
   fenced-or-not.
-- **TLS anywhere**: `start_tls` is fenced, so `https://` and `wss://` are out
-  of scope for every probe on this page.
+- **TLS anywhere**: no probe on this page requests `https://` or `wss://`.
+  simloop fences `start_tls` and `create_connection(ssl=...)`, but a stack
+  that runs its handshake in memory reaches neither — it reaches a simulated
+  network with nothing on it that speaks TLS unless the test puts it there.
 - Anything that reaches outside the loop by design — threads, executors,
   subprocesses, signals, real DNS. Those are fences, listed in
   [docs/supported-api.md](supported-api.md), not compatibility questions.

docs/supported-api.md

@@ -44,6 +44,8 @@ a host belong to an implicit `driver` host.
 | `loop.getaddrinfo` | Resolves against the host table, never DNS: a registered host name, its synthetic address, or a loopback-shaped name (`None`, `""`, `localhost`, `127.0.0.1`, `0.0.0.0`) meaning the calling task's own host. Returns stdlib-shaped rows — `(AF_INET, SOCK_STREAM, IPPROTO_TCP, "", (address, port))` and the `SOCK_DGRAM` / `IPPROTO_UDP` row — filtered by `family`, `type` and `proto`. Ports are numeric (`int`, a digit string, or `None` for 0); resolver `flags` have nothing to vary |
 | `loop.getnameinfo` | Reverse lookup: a synthetic address maps back to its host name, and a host name (what `get_extra_info("peername")` reports) maps to itself. `NI_NUMERICHOST` returns the address instead; services are always numeric |
 | `loop.net.address` / `hostname` | The mapping itself: every registered host owns one synthetic IPv4 address from `10.7.0.0/16` — `10.7.0.1`, `10.7.0.2`, ... handed out in registration order, starting with the implicit `driver` host |
+| `loop.sock_connect` + `create_connection(sock=...)` | The two-call connect sequence aiohttp's connector performs. On an `AF_INET` stream socket, `sock_connect` places the target in the host table and records it against the socket — no packet moves and no virtual time passes, and an address that is not a `(host, port)` pair, or a target the host table cannot place, raises `OSError` there. `create_connection(sock=...)` then claims that recorded address: it closes the real descriptor (the loop takes ownership, as the stdlib does) and opens a simulated connection, paying the same one round trip a direct connect pays, so a closed port raises `ConnectionRefusedError` from this call rather than the first. Passing a socket that no `sock_connect` on this loop parked raises `OSError`; passing `host`/`port` alongside `sock` raises `ValueError`. Binding a source address first — `TCPConnector(local_addr=...)` — is not supported: the connector binds the real socket before the simulation is consulted, and a synthetic address belongs to no real interface, so the bind fails outside the loop |
+| `transport.get_extra_info("socket")` (streams) | A stand-in object, not a network socket: `family` / `type` / `proto` report `AF_INET` / `SOCK_STREAM` / `IPPROTO_TCP`, `getsockname()` and `getpeername()` return `(synthetic address, port)` tuples for the two ends, and `setsockopt`, `shutdown` and `close` are accepted and do nothing. `fileno()` returns a parked descriptor the transport owns, created on first call, that polls unreadable while the peer is alive and readable once the peer's EOF arrives — which is how a pool that checks readability sees a dead connection. A reset, a local close or the end of the run closes the descriptor instead, and `fileno()` reports `-1` from then on; a readability poll reads that as dead too. The number itself is assigned by the operating system, so it is not reproducible across runs and is outside the determinism guarantee — nothing in a trace depends on it. No bytes ever cross it: `recv`, `send` and anything else not listed above raise `AttributeError` rather than pretend. Datagram transports still report `None` |
 
 Names and their synthetic addresses are interchangeable wherever an endpoint
 is accepted, so a client can resolve a name and connect to what it got back.
@@ -62,5 +64,17 @@ reliable by construction; and addressing is IPv4-only and entirely synthetic
 
 Anything that reaches outside the simulation raises `SimulationFenceError`:
 executors and threads (`run_in_executor`, `call_soon_threadsafe`), signal
-handlers, subprocesses, raw sockets (`sock_*`), file-descriptor callbacks
-(`add_reader` / `add_writer`), TLS upgrades, `sendfile`, and pipes.
+handlers, subprocesses, file-descriptor callbacks (`add_reader` /
+`add_writer`), loop-level TLS upgrades (`start_tls`,
+`create_connection(ssl=...)`), `sendfile`, and pipes. TLS a library
+performs in memory reaches no loop API and so reaches no fence; what that
+means in practice is in [docs/compatibility.md](compatibility.md).
+
+The socket calls are fenced with one exception. `sock_connect` on an
+`AF_INET` stream socket is simulated — it is how client stacks reach the
+network, and the table above says what it does. Every other socket kind
+fences there — datagram, raw and IPv6 alike — and the rest of the family
+stays fenced outright: `sock_recv`, `sock_recv_into`, `sock_sendall`,
+`sock_sendto`, `sock_recvfrom`, `sock_recvfrom_into`, `sock_accept` and
+`sock_sendfile`. Client stacks do not need them: the socket they connect
+is upgraded into a transport instead of being read and written directly.

src/simloop/_loop.py

@@ -6,10 +6,11 @@
 import gc
 import heapq
 import random
+import socket
 import sys
 from array import array
 from asyncio import events
-from collections.abc import Callable, Iterable
+from collections.abc import Callable, Iterable, Sequence
 from contextvars import Context
 from typing import TYPE_CHECKING, Any, NoReturn, TypeVarTuple, Unpack
 
@@ -117,6 +118,10 @@ def __init__(self, seed: int = 0) -> None:
         self._unhandled: list[BaseException] = []
         self._exception_handler: _ExceptionHandler | None = None
         self._task_factory: _TaskFactory | None = None
+        # sock_connect(sock, addr) -> create_connection(sock=sock) is how
+        # aiohttp reaches the network; the address is only visible in the
+        # first call, so it is parked here until the upgrade claims it.
+        self._sock_targets: dict[Any, tuple[Any, int]] = {}
         self._net = SimNetwork(self)
 
     @classmethod
@@ -328,6 +333,15 @@ def is_closed(self) -> bool:
     def close(self) -> None:
         if self._running:
             raise RuntimeError("cannot close a running event loop")
+        self._sock_targets.clear()
+        # Connections still open at the end of a run — what a pooling client
+        # leaves behind — hold their liveness descriptors inside the socket
+        # object's reference cycle with the transport, so the descriptors
+        # would only come back when the cycle collector ran. Nothing can poll
+        # them once the loop is closed, so release them here instead.
+        for transport in self._net._streams.values():
+            if transport._extra_socket is not None:
+                transport._extra_socket._dispose()
         self._closed = True
 
     def _check_closed(self) -> None:
@@ -386,7 +400,24 @@ async def create_connection(
         port: Any = None,
         **kwargs: Any,
     ) -> Any:
+        sock = kwargs.pop("sock", None)
         _reject_kwargs("create_connection", kwargs)
+        if sock is not None:
+            # The stdlib treats a passed-in socket as already connected and
+            # takes ownership of it. Here "connected" means sock_connect
+            # parked a target for it; the real descriptor is closed at once
+            # because the simulation only needed the address it carried.
+            if host is not None or port is not None:
+                raise ValueError(
+                    "host/port and sock can not be specified at the same time"
+                )
+            target = self._sock_targets.pop(sock, None)
+            if target is None:
+                raise OSError(
+                    "the given socket was not connected via sock_connect on this loop"
+                )
+            sock.close()
+            host, port = target
         return await self._net._open_connection(protocol_factory, host, port)
 
     async def create_server(
@@ -585,8 +616,39 @@ def sock_recv_into(self, *args: Any, **kwargs: Any) -> Any:
     def sock_sendall(self, *args: Any, **kwargs: Any) -> Any:
         _fence("sock_sendall")
 
-    def sock_connect(self, *args: Any, **kwargs: Any) -> Any:
-        _fence("sock_connect")
+    async def sock_connect(self, sock: Any, address: Any) -> None:
+        # aiohttp's connector creates a real TCP socket and connects it here
+        # before handing it to create_connection(sock=...). The simulation
+        # accepts exactly that shape; every other socket kind still fences.
+        if (
+            getattr(sock, "family", None) != socket.AF_INET
+            or getattr(sock, "type", None) != socket.SOCK_STREAM
+        ):
+            raise SimulationFenceError(
+                "simloop does not simulate 'sock_connect' for anything but "
+                "AF_INET stream sockets; see docs/supported-api.md for the "
+                "supported asyncio subset"
+            )
+        if (
+            isinstance(address, (str, bytes, bytearray))
+            or not isinstance(address, Sequence)
+            or len(address) != 2
+            or not isinstance(address[0], (str, bytes))
+            or isinstance(address[1], bool)
+            or not isinstance(address[1], int)
+        ):
+            # Caught here rather than deeper: a bare host string would be read
+            # character by character, and a string port would surface much
+            # later as a confusing error about something else entirely.
+            raise OSError(
+                f"sock_connect needs an AF_INET (host, port) address, got {address!r}"
+            )
+        host, port = address[0], address[1]
+        self._net._resolve(host)  # unknown targets fail here, loudly
+        # No packet moves and no time passes yet: the connection handshake
+        # (and its one-RTT cost) happens when create_connection claims the
+        # socket, keeping the total cost identical to a direct connect.
+        self._sock_targets[sock] = (host, port)
 
     def sock_accept(self, *args: Any, **kwargs: Any) -> Any:
         _fence("sock_accept")