rcatline mishandles multi-character line terminators

[tonycoz]

Module:

Description

When a readline is optimised to an rcatline operator the resulting operator can treat the trailing text of the SV being concatenated to as part of the record separator.

code like

$x .= readline(BAREWORD);

can be optimised into a rcatline operator, but that can incorrectly treat the trailing part of the variable ($x here) as part of the record separator.

Note that this optimisation is only done when the readline() is done from a bareword filehandle (which may be another bug, but of lesser importance)

Discovered while reviewing the sv_gets() code when working on #21877.

Steps to Reproduce

tony@venus:.../git/perl6$ cat ../21877-eol2.pl
#!perl
use Devel::Peek;

my $src = "a" x 5;
open FH, "<", \$src;

$/ = "aaaaa";
my $x = "aaaa";
$x .= <FH>;
Dump($x);
tony@venus:.../git/perl6$ ./perl -Ilib ../21877-eol2.pl
SV = PV(0x559437abff90) at 0x559437af00a8
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x559437af12b0 "aaaaa"\0
  CUR = 5
  LEN = 16

this only happens with the rcatline optimisation, if you use a non-bareword handle:

tony@venus:.../git/perl6$ cat ../21877-eol.pl
#!perl
use Devel::Peek;

my $src = "a" x 5;
open my $fh, "<", \$src;

$/ = "aaaaa";
my $x = "aaaa";
$x .= <$fh>;
Dump($x);
tony@venus:.../git/perl6$ ./perl -Ilib ../21877-eol.pl
SV = PV(0x5575ab91af90) at 0x5575ab93df58
  REFCNT = 1
  FLAGS = (POK,pPOK)
  PV = 0x5575ab94c400 "aaaaaaaaa"\0
  CUR = 9
  LEN = 16

the result is as expected.

The example uses a scalar handle for brevity, this also occurs with File::Temp handles, and presumably other handles.

Expected behavior

The text in the appended to SV shouldn't be treated as part of the record separator.

Perl configuration

<Summary of my perl5 (revision 5 version 43 subversion 4) configuration:
  Commit id: e9d9a70d1c5c0e1320520b98c9eb4b4778409137
  Platform:
    osname=linux
    osvers=6.1.0-37-amd64
    archname=x86_64-linux
    uname='linux venus 6.1.0-37-amd64 #1 smp preempt_dynamic debian 6.1.140-1 (2025-05-22) x86_64 gnulinux '
    config_args='-des -Dusedevel -DDEBUGGING'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=undef
    usemultiplicity=undef
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
  Compiler:
    cc='cc'
    ccflags ='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2'
    optimize='-O2 -g'
    cppflags='-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='12.2.0'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib/x86_64-linux-gnu /usr/lib /usr/lib64
    libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
    perllibs=-lpthread -ldl -lm -lcrypt -lutil -lc
    libc=/lib/x86_64-linux-gnu/libc.so.6
    so=so
    useshrplib=false
    libperl=libperl.a
    gnulibc_version='2.36'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='-Wl,-E'
    cccdlflags='-fPIC'
    lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl): 
  Compile-time options:
    DEBUGGING
    HAS_LONG_DOUBLE
    HAS_STRTOLD
    HAS_TIMES
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_HASH_FUNC_SIPHASH13
    PERL_HASH_USE_SBOX32
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_DEVEL
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
  Built under linux
  Compiled at Oct  3 2025 12:42:12
  %ENV:
    PERLBREW_BASHRC_VERSION="0.43"
    PERLBREW_HOME="/home/tony/.perlbrew"
    PERLBREW_MANPATH=""
    PERLBREW_PATH="/home/tony/perl5/perlbrew/bin"
    PERLBREW_ROOT="/home/tony/perl5/perlbrew"
    PERLBREW_VERSION="0.67"
  @INC:
    lib
    /usr/local/lib/perl5/site_perl/5.43.4/x86_64-linux
    /usr/local/lib/perl5/site_perl/5.43.4
    /usr/local/lib/perl5/5.43.4/x86_64-linux
    /usr/local/lib/perl5/5.43.4

[jkeenan]

In all my years of doing Perl, I never once heard of rcatline until reading this ticket. I was surprised to find that string in all these files in the core distribution ...

$ ack -l rcatline . |sort
dist/Safe/t/safeops.t
ext/Opcode/Opcode.pm
lib/B/Deparse.pm
MANIFEST
opcode.h
peep.c
pp_hot.c
pp_proto.h
pp_sys.c
regen/opcodes
t/bigmem/sv_gets.t
t/io/argv.t
t/io/utf8.t
t/lib/feature/bareword_filehandles
t/op/gv.t
t/op/lex_assign.t
t/op/override.t
t/op/readline.t
t/op/svleak.t
t/op/tie_fetch_count.t
t/perf/opcount.t
t/uni/gv.t
t/uni/readline.t

... but note that it is not once mentioned anywhere under pod/. What the hell is it?

[iabyn]

On Fri, Oct 03, 2025 at 05:29:36AM -0700, James E Keenan wrote: In all my years of doing Perl, I never once heard of *rcatline* until reading this ticket.

It's just an optimisation: $x .= <FOO> which would normally compile to a readline op and a string append op, is optimised (sometimes) into a single rcatline op.

[jkeenan]

On Fri, Oct 03, 2025 at 05:29:36AM -0700, James E Keenan wrote:
In all my years of doing Perl, I never once heard of rcatline until
reading this ticket.
It's just an optimisation:

$x .= <FOO>

which would normally compile to a readline op and a string append op,
is optimised (sometimes) into a single rcatline op.

Is there any place in our documentation where we list all ops?

[iabyn]

On Sat, Oct 04, 2025 at 04:43:42AM -0700, James E Keenan wrote: Is there any place in our documentation where we list all ops?

They're all listed in regen/opcodes. I don't think we formally document them anywhere, though.

[tonycoz]

If we were to document them I think it would just be a pointer to regen/opcodes or automatically generated from there.