Conversation
…ages in BatchJob (#9394)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9394
Problem
When
kernel.msgmaxis raised above the standard default of 8192 bytes (e.g.kernel.msgmax = 65536in containerized or high-throughput environments),SysvProcessor::submit()'s initial call tomsg_send(..., self::$typeDirect, $item)succeeds for serialized items between 8193 andmsgmaxbytes.However,
BatchJob::run()callsmsg_receive($q, 0, $type, 8192, $message, true, 0, $errorcode)with a hardcoded buffer limit of 8192 bytes and withoutMSG_NOERROR. Under System V IPCmsgrcv(2), when a message in the queue exceedsmsgszandMSG_NOERRORis not set inmsgflg, the syscall fails withE2BIG($errorcode === 7/PCNTL_E2BIG), leaving the message unconsumed at the head of the queue. Because the queue is not empty, subsequentmsg_receivecalls return immediately withE2BIG, causing a 100% CPU busy-loop and indefinitely blocking all subsequent messages.Solution
SysvProcessor:const MAX_DIRECT_SIZE = 8192.msg_send(..., self::$typeDirect, ...), check ifstrlen($serialized) <= self::MAX_DIRECT_SIZE.self::$typeFile) instead of pushing an oversized direct message to the queue, reusing$serializedand ensuring direct messages never exceed 8192 bytes.BatchJob:const MAX_MESSAGE_SIZE = 8192.run(), whenmsg_receivefails withE2BIG(checked againstPCNTL_E2BIG,SOCKET_E2BIG,MSG_E2BIG, or POSIX errno 7), drain the poison message viadrainOversizedMessage($q)usingMSG_IPC_NOWAIT | MSG_NOERRORwithout unserializing. This consumes and purges the poison message from the queue head, preventing CPU starvation.SysvProcessorTest::items()for items exceeding 8192 bytes.BatchJobTestforisMsgTooBig()anddrainOversizedMessage().Core/tests/Unitpass (537 tests, 856 assertions, 100% pass).php-tools cs-fixer(Found 0 of 471 files that can be fixed) andphpcs(0 errors, 0 warnings).