Fix parsing of tag descriptions with continuation lines starting with *#444
Open
tmdk wants to merge 1 commit intophpDocumentor:6.xfrom
Open
Fix parsing of tag descriptions with continuation lines starting with *#444tmdk wants to merge 1 commit intophpDocumentor:6.xfrom
tmdk wants to merge 1 commit intophpDocumentor:6.xfrom
Conversation
…iptions When a tag description contains continuation lines beginning with *, the * was duplicated in the parsed output. This affected block-style @param descriptions (e.g. WordPress-style docblocks with @type lines and star- prefixed list items). Root cause: tokenizeLine() split TOKEN_PHPDOC_EOL tokens with trailing whitespace into a bare EOL + TOKEN_HORIZONTAL_WS pair, placing * in both token values. When PHPStan rolled back past such a token on encountering a tag-like token (@type), joinUntil() concatenated both raw values, doubling the star. Fix: prefix every continuation line with "* " before tokenizing. The lexer always consumes the inserted "* " as part of TOKEN_PHPDOC_EOL, leaving original indentation as a separate subsequent token. An unconditional trim(..., "* \t") on every EOL token value strips the inserted "* " back out. The manual split into TOKEN_HORIZONTAL_WS is no longer needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a tag description contains continuation lines that begin with
*, the*appears twice in the parsed output. This affects block-style@paramdescriptions (common in WordPress docblocks) where list items or@typealignment uses a leading star:The parsed description for
$foowould incorrectly contain** a listinstead of* a list.Root cause
AbstractPHPStanFactory::tokenizeLine()worked around a PHPStan lexer limitation by splittingTOKEN_PHPDOC_EOLtokens that contained trailing whitespace into two tokens: a bare EOL and a separateTOKEN_HORIZONTAL_WS. This placed*in both token values. When PHPStan'sparseText()stopped early at a tag-like token (e.g.@type) and rolled back,joinUntil()concatenated both raw token values, doubling the star.Fix
Prefix every continuation line with
"* "(star + space) before tokenizing. The PHPStan lexer'sTOKEN_PHPDOC_EOLpattern matches exactly\n+ optional whitespace +*(star + one space), so it always consumes the inserted"* "as part of the EOL token, leaving the original indentation as a separate subsequent token. Atrim(..., "* \t")on every EOL token value then strips the inserted"* "back out, reducing each EOL to a plain"\n". The manual split into a separateTOKEN_HORIZONTAL_WSis no longer needed.Changes
src/DocBlock/Tags/Factory/AbstractPHPStanFactory.php— revisedtokenizeLine()tests/integration/InterpretingDocBlocksTest.php— regression test covering theblock-description case with embedded
@typelines and star-prefixed continuation lines