Reline emits extra escape sequences when stdin/stdout/stderr are not TTY

[eregon]

Reline 0.6.0 from Ruby 3.4.7 (same with reline 0.6.3 the last release):

$ echo 'my input' > in
$ ruby -rreadline -e 'p Readline; p [STDIN,STDOUT,STDERR].map(&:tty?); p Readline.readline("> ", true)' < in |& ruby -e 'p STDIN.read'            
"Reline\n[false, false, false]\n> \e[?25l\e[1G\e[K\e[1G\e[0m> \e[0m\e[?25h\e[3G\e[?25l\e[1G\e[K\e[?25h\e[1G> my input\r\n\e[1G\"my input\"\n"

Even though STDIN, STDOUT, STDERR are all not TTY, terminal escape sequences are still included.
This causes issues for e.g. this program using Readline.readline because it's supposed to work well both if TTY or not. If not TTY, it expects no escape sequences.
That works with Readline from readline-ext but not Reline (as-is at least).

TERM=dumb doesn't help, it outputs many extra things (#660):

$ TERM=dumb ruby -rreadline -e 'p Readline; p [STDIN,STDOUT,STDERR].map(&:tty?); p Readline.readline("> ", true)' < in |& ruby -e 'p STDIN.read' 
"Reline\n[false, false, false]\n> > mmymy my imy inmy inpmy inpumy input> my input\r\n\"my input\"\n"

readline-ext for comparison works fine:

$ gem i readline-ext  
$ ruby -rreadline -e 'p Readline; p [STDIN,STDOUT,STDERR].map(&:tty?); p Readline.readline("> ", true)' < in |& ruby -e 'p STDIN.read'
"Readline\n[false, false, false]\n> my input\n\"my input\"\n"

Is there a way to achieve the same with Reline?

Related: #644

[eregon]

A workaround could be to commit the expected to include those escape sequences but it seems quite hacky:

Also diff doesn't work well with these escape sequences.

[tompng]

FYI, Readline-ext also emits escape sequences in non-tty input/output: "\b\e[K"

$ ruby -v
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]

$ printf "helo\blo\n" | ruby -rreadline -e "p Readline::VERSION; p Readline.readline '>'" | ruby -e "p STDIN.read" 
"\"8.3\"\n>helo\b\e[Klo\n\"hello\"\n"

The differences is that Reline always emits escape sequence before printing prompt string,while GNU Readline doesn't.
This is a limitation or perhaps a bug of GNU Readline.

% printf "----"; ruby -rreadline -e "p Readline.readline '> '"
----->
# Type "a", "b", "c", "d", C-a, "e", "f", "g"
--efg--> abcd

# Expected: "> efgabcd"(Reline) or "-----> efgabcd"

Here's my thought:

• Option 1: Reline should specially handle non-tty input
  • Option 1-a: Handles input escape sequences but emits nothing. "ab\C-ac\n" will return "cab"
  • Option 1-b: Reads raw input. "ab\C-ac\n" will return "ab\u0001c"
  • In either option, we need to correctly handle Reline.readmultiline case such as dynamic prompt and multiline termination check
• Option 2: Don't use Reline unless tty. User knows how to specially handle non-tty input for their various use case
  • if tty?; Reline.readmultiline; else; print '>'; gets; end is not a workaroud, but an expected usage of Reline

[eregon]

FYI, Readline-ext also emits escape sequences in non-tty input/output: "\b\e[K"

True, although it doesn't happen if the input is only printable ASCII characters, which is the most common case.

Option 1: Reline should specially handle non-tty input

Yes, I think this would be best.
Either option seems fine to me.
If STDIN is a TTY but STDOUT is not then 1-a seems better.
When STDIN is not a TTY, maybe it's expected to just be reading raw lines (1-b)? Though I would think there are use cases for 1-a there too, so I'd say 1-a seems more consistent and I suppose Reline already handles these escape sequences so it wouldn't be complicated to support.
This also means I think the check for TTY should be STDOUT.tty? (or $stdout.tty?), since that's where Reline prints the escape sequences.

I believe Option 1 is the best because Ruby programs are used to require "readline"; Readline.readline("> "), so this is more compatible and works more similar to readline-ext, when given only printable ASCII characters (the most common case).
Looking at the issue tracker, there were several other issues/discussions about this, so the current behavior seems surprising to many.

Option 2: Don't use Reline unless tty.

It's what I used to workaround for now but it feels like a hack/workaround to me, I think the Reline defaults should be better in such a case, and only require the user to do it manually if they need special handling (most don't).
Here is what I used, it's more complicated than just print '>'; gets (notably that breaks with non-empty ARGV):
truffleruby/truffleruby@3bdfcb9

  if STDIN.tty?
    code = Readline.readline('> ', true)
  else
    # Workaround for https://github.com/ruby/reline/issues/886
    print '> '
    code = STDIN.gets
    puts code
  end

A subtle part here is it should be puts code and not puts code if code (code is nil on EOF) to have a final newline on EOF.

[eregon]

• if tty?; Reline.readmultiline; else; print '>'; gets; end is not a workaroud, but an expected usage of Reline

This doesn't seem well documented, probably it should be in the README or docs if it is expected.
But option 1 seems better for everyone, and no need to document this common gotcha then as it wouldn't happen anymore.

[tompng]

I though that readline-ext handels control sequences and prints escape sequences, but it wasn't always true.
GNU-readline-backed readline-ext does that, but libedit-backed readline-ext just gets unless both stdin/stdout is a tty.

#924 will make reline use gets when input or output is not a tty.