Skip to content

fix(queue): stop a failed ack pinning its message in flight - #199

Open
levivannoort wants to merge 1 commit into
mainfrom
fix/queue-nats-inflight-ack-leak
Open

fix(queue): stop a failed ack pinning its message in flight#199
levivannoort wants to merge 1 commit into
mainfrom
fix/queue-nats-inflight-ack-leak

Conversation

@levivannoort

Copy link
Copy Markdown
Contributor

Closes #196.

Broker\Nats::commit() acked and then dropped the message from $inFlight:

$jsMessage->ackSync();
unset($this->inFlight[$pid]);

ackSync() is a request-reply, so a transient failure raises rather than returning a verdict — and that is precisely when the unset was skipped.

Nothing else clears the entry. reject() is the only other place that unsets, and Adapter::runPhases() deliberately does not reject after a commit failure — the work is done, so a NAK there is a duplicate guaranteed rather than a duplicate risked. The entry is left with no owner, and a long-lived worker accumulates one pinned JetStreamMessage per failed ack.

The fix

The map records that this instance owes an ack for the message, not that one succeeded, so the unset belongs in a finally. The throw still propagates, because runPhases() reporting it is the behaviour that decided not to reject.

Verification

NatsBrokerTest::testAnAckThatFailsDoesNotPinTheMessageInFlight closes the connection under the ack — the cheapest stand-in for the transient failure this guards — and asserts the map is clear afterwards. Confirmed red before the fix and green after:

in flight after receive: yes
commit threw: Utopia\NATS\Exception\ConnectionException
still in flight after the failed ack: YES -> leak     # before
still in flight after the failed ack: no  -> cleared  # after

The new test is in the e2e tier, which needs a JetStream server and cannot run on a host without ext-swoole (tests/e2e.sh boots Swoole workers), so it rests on CI — the red/green above was taken against a real server via the same steps the test performs. bin/monorepo check queue (pint, phpstan, rector) passes, and the unit tier is unchanged at 100 tests.

Found by a review sweep over main while resolving #192; not introduced by it. Two other findings from that sweep are filed as #197 and #198.

Broker\Nats::commit() acked and then dropped the message from $inFlight, so
an ack that threw left the entry behind. ackSync() is a request-reply, so a
transient failure raises rather than returning a verdict, and that is exactly
when the unset was skipped.

Nothing else clears it. reject() is the only other place that unsets, and
Adapter::runPhases() deliberately does not reject after a commit failure --
the work is done, so a NAK there is a duplicate guaranteed rather than a
duplicate risked. The entry is left with no owner, and a long-lived worker
accumulates one pinned JetStreamMessage per failed ack.

The map records that this instance owes an ack for the message, not that one
succeeded, so the unset belongs in a finally. The throw still propagates:
runPhases() reports it, which is the behaviour that decided not to reject.

Closes #196.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Broker\Nats pins a message in $inFlight when its ack fails

1 participant