Passing input through with `Open3` to a subprocess using `Reline.readline` crashes about reline

[MSP-Greg]

Replacing the bottom code of lib/reline.rb with the following seems to work on Windows, but not sure about other OS's and using tty to determine IO type...

Reline::IOGate = begin
  t = if Reline::Core::IS_WINDOWS
    require 'reline/windows'
    Reline::Windows
  else
    require 'reline/ansi'
    Reline::ANSI
  end
  require 'reline/general_io'
  STDOUT.tty? && STDIN.tty? ? t : Reline::GeneralIO
end

Reline.input = $stdin if Reline::IOGate == Reline::GeneralIO

FYI, with a few patches, byebug passes on Windows using Reline.

[deivid-rodriguez]

That sounds like a snippet we could perfectly add to byebug's test helper, right?

Would you like to open a PR there? We shouldn't merge it straight away since we'll need to verify that the real user experience is unchanged, but a green CI would be a great starting point.

Not sure if there's anything to be changed in reline? Certainly the "testing mode" could be easier to setup, but this is something!

[MSP-Greg]

That sounds like a snippet we could perfectly add to byebug's test helper, right?

Is the test helper loaded when Open3 is used? Don't recall...

need to verify that the real user experience is unchanged

I'm not sure what the benchmark is for a 'correct' user experience.

[deivid-rodriguez]

No benchmark, I meant that I'll use byebug with reline for a bit and verify that there's no obvious user visible changes/gotchas/bugs.

Is the test helper loaded when Open3 is used? Don't recall...

I don't think so, but we can load the snippet when needed, I think we do something similar with a simplecov helper file to make sure that coverage is tracked inside subprocesses.

Anyways, this is getting out of topic here, let's move the discussion to deivid-rodriguez/byebug#510.

[Nakilon]

Hi guys, your issue is the only one result google gives me about this:

$ ruby -e "p STDOUT.pos"
17800074

$ ruby -ropen3 -e "p Open3.capture2e 'ruby -e \"p STDOUT.pos\"'"
["-e:1:in `pos': Illegal seek @ rb_io_tell - <STDOUT> (Errno::ESPIPE)\n\tfrom -e:1:in `<main>'\n", #<Process::Status: pid 83602 exit 1>]

I face this issue in tests that launch my gem bin/ executable. It's the same in Ruby 2.3 .. 3.0

UPD: the same with backticks:

$ irb
irb(main):001:0> `ruby -e "p STDOUT.pos"`
-e:1:in `pos': Illegal seek @ rb_io_tell - <STDOUT> (Errno::ESPIPE)
	from -e:1:in `<main>'

and with IO.popen

[Nakilon]

I tried to reuse the #4796 patch. With no success.

$ cat use_ext_readline.rb 
require "readline.so"
ReadlineSo = Readline
  Object.send(:remove_const, :Readline) if Object.const_defined?(:Readline)
  Object.const_set(:Readline, ReadlineSo)
$ ruby -ropen3 -e "p Open3.capture2e 'ruby -rreline -r./use_ext_readline.rb -e \"p STDOUT.pos\"'"
["-e:1:in `pos': Illegal seek @ rb_io_tell - <STDOUT> (Errno::ESPIPE)\n\tfrom -e:1:in `<main>'\n", #<Process::Status: pid 85906 exit 1>]

$ cat use_lib_reline.rb 
    Reline.send(:remove_const, 'IOGate') if Reline.const_defined?('IOGate')
    Reline.const_set('IOGate', Reline::GeneralIO)
    Reline.send(:core).config.instance_variable_set(:@test_mode, true)
    Reline.send(:core).config.reset
    Object.send(:remove_const, :Readline) if Object.const_defined?(:Readline)
    Object.const_set(:Readline, Reline)
$ ruby -ropen3 -e "p Open3.capture2e 'ruby -rreline -r./use_lib_reline.rb -e \"p STDOUT.pos\"'"
["-e:1:in `pos': Illegal seek @ rb_io_tell - <STDOUT> (Errno::ESPIPE)\n\tfrom -e:1:in `<main>'\n", #<Process::Status: pid 85916 exit 1>]

ruby 3.0.1