Skip to content

Fix duplicated output from empty match after trailing newline (--across) - #352

Open
yhuikzdtguioaert wants to merge 1 commit into
chmln:masterfrom
yhuikzdtguioaert:fix/across-trailing-newline-dup
Open

Fix duplicated output from empty match after trailing newline (--across)#352
yhuikzdtguioaert wants to merge 1 commit into
chmln:masterfrom
yhuikzdtguioaert:fix/across-trailing-newline-dup

Conversation

@yhuikzdtguioaert

Copy link
Copy Markdown

Problem

Reported in #318. In whole-input (-A/--across) mode, matching a pattern that can match the empty string duplicates the output:

$ printf 'abc\n' | sd -A '(.*)' '${1}def'
abcdef
def          # <- unexpected

$ printf 'abc\n' | sd -A '(.*)' 'def'
def
def          # <- unexpected

Using '${1}' on its own only appeared to work because the extra match expands to an empty string, so it was invisible.

Cause

In --across mode the whole input is handed to a multi-line regex. The position right after the trailing \n is treated as the start of an empty final "line", so patterns like .*, ^ and $ produce a zero-width match there and emit one extra replacement.

Fix

Skip a zero-width match located at the end of the input when the input ends with a newline. This lines up with sed:

$ printf 'abc\n' | sed -E 's/(.*)/\1def/'
abcdef

Matches that are non-empty, that are not at end-of-input, or cases where the input has no trailing newline are untouched, so genuine empty lines (e.g. abc\n\n), ^-prefixing, $-suffixing and patterns such as x* all keep their existing behaviour.

Tests

Added regression tests in sd/src/replacer/tests.rs covering the whole-input replace path, plus an end-to-end CLI test in sd-cli/tests/cli.rs. cargo test is green.

Fixes #318

In whole-input mode the multi-line regex treats the position right after
a trailing newline as the start of an empty final line. Patterns that can
match nothing there (`.*`, `^`, `$`, ...) matched it and emitted an extra
replacement, so `echo abc | sd -A '(.*)' '${1}def'` returned two lines
("abcdef" and "def") instead of one.

Ignore a zero-width match at end-of-input when the input ends in a
newline, matching sed's behaviour. Genuine empty lines and zero-width
matches elsewhere are left untouched.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Matching everything (.*) duplicates output

1 participant