illegal anon sub coredump

[p5pRT]

Migrated from rt.perl.org#17589 (status was 'resolved')

Searchable as RT17589$

[p5pRT]

From pete@skejbyparken.bnaa.dk

Created by pkelambe@cs.helsinki.fi

Immediate segfault is produced, when a file containing following
script is given to perl as command line argument.

Lines after this line are in the file.

sub a{
 sub($b);
 $b->c->d->e->f->g;
}

Lines before this line are in the file.

Same thing happened also on Debian with Perl v5.6.1 and
another RH derived system with v5.6.1 .

Perl Info

Flags:
    category=core
    severity=high

Site configuration information for perl v5.6.1:

Configured by compil at Thu Feb 21 00:59:47 EST 2002.

Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
  Platform:
    osname=linux, osvers=2.4.16-6mdksmp, archname=i386-linux
    uname='linux bi.mandrakesoft.com 2.4.16-6mdksmp #1 smp sat dec 8 04:02:48 cet 2001 i686 unknown '
    config_args='-des -Darchname=i386-linux -Dd_dosuid -Ud_csh -Duseshrplib -Doptimize=-O3 -fomit-frame-pointer -pipe -mcpu=pentiumpro -march=i586 -ffast-math -fno-strength-reduce -Dprefix=/usr -Di_ndbm -Di_gdbm -Di_shadow -Di_syslog -Uuselargefiles -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/lib/perl5/man/man3'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=undef d_sfio=undef uselargefiles=undef usesocks=undef
    use64bitint=undef use64bitall=undef uselongdouble=undef
  Compiler:
    cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include',
    optimize='-O3 -fomit-frame-pointer -pipe -mcpu=pentiumpro -march=i586 -ffast-math -fno-strength-reduce',
    cppflags='-fno-strict-aliasing -I/usr/local/include'
    ccversion='', gccversion='2.96 20000731 (Mandrake Linux 8.2 2.96-0.75mdk)', gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=4
    alignbytes=4, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -ldl -lm -lc -lcrypt -lutil
    perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
    libc=/lib/libc-2.2.4.so, so=so, useshrplib=true, libperl=libperl.so
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic -Wl,-rpath,/usr/lib/perl5/5.6.1/i386-linux/CORE'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    


@INC for perl v5.6.1:
    /usr/lib/perl5/5.6.1/i386-linux
    /usr/lib/perl5/5.6.1
    /usr/lib/perl5/site_perl/5.6.1/i386-linux
    /usr/lib/perl5/site_perl/5.6.1
    /usr/lib/perl5/site_perl
    .


Environment for perl v5.6.1:
    HOME=/home/pete
    LANG=fi_FI
    LANGUAGE=fi
    LC_COLLATE=fi_FI
    LC_CTYPE=fi_FI
    LC_MESSAGES=fi_FI
    LC_MONETARY=fi_FI
    LC_NUMERIC=fi_FI
    LC_TIME=fi_FI
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/home/pete/bin:/home/pete/usr/bin:/home/pete/sbin:/home/pete/usr/sbin:/sbin:/usr/sbin:/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin:/usr/games
    PERL_BADLANG (unset)
    SHELL=/bin/bash

[p5pRT]

From kane@cpan.org

Pete Via Rt wrote​:

# New Ticket Created by (Pete)
# Please include the string​: [perl #17589]
# in the subject line of all future correspondence about this issue.
# <URL​: http​://rt.perl.org/rt2/Ticket/Display.html?id=17589 >

~s segfault by short script
This is a bug report for perl from pkelambe@​cs.helsinki.fi,
generated with the help of perlbug 1.33 running under perl v5.6.1.

-----------------------------------------------------------------
[Please enter your report here]

Immediate segfault is produced, when a file containing following
script is given to perl as command line argument.

Lines after this line are in the file.
sub a{
sub($b);
$b->c->d->e->f->g;
}
Lines before this line are in the file.

Same thing happened also on Debian with Perl v5.6.1 and
another RH derived system with v5.6.1 .

[Please do not change anything below this line]

the problem can be reliably reproduced on all versions of perl I run on
this machine​:
[kane@​gremlin tmp]$ uname -a
Linux gremlin 2.4.7-10 #1 Thu Sep 6 17​:27​:27 EDT 2001 i686 unknown

[kane@​gremlin tmp]$ more t
sub a{
  sub($b);
  $b->c->d->e->f->g;
}

[kane@​gremlin tmp]$ perl5.5.3 t
syntax error at t line 2, near "sub($b);"
Segmentation fault
[kane@​gremlin tmp]$ perl5.6.0 t
syntax error at t line 2, near "sub($b);"
Segmentation fault
[kane@​gremlin tmp]$ perl5.6.1 t
Segmentation fault
[kane@​gremlin tmp]$ perl5.7.3 t
Segmentation fault
[kane@​gremlin tmp]$ perl5.8.0 t
Segmentation fault
[kane@​gremlin tmp]$

[kane@​gremlin tmp]$ perl5.5.3 -w t
syntax error at t line 2, near "sub($b);"
Segmentation fault
[kane@​gremlin tmp]$ perl5.6.0 -w t
syntax error at t line 2, near "sub($b);"
Segmentation fault
[kane@​gremlin tmp]$ perl5.6.1 -w t
Segmentation fault
[kane@​gremlin tmp]$ perl5.6.2 -w t
bash​: perl5.6.2​: command not found
[kane@​gremlin tmp]$ perl5.7.3 -w t
Illegal character in prototype for ? : $b at t line 2.
Segmentation fault
[kane@​gremlin tmp]$ perl5.8.0 -w t
Illegal character in prototype for ? : $b at t line 2.
Segmentation fault
[kane@​gremlin tmp]$

The offending line lies in the improper prototyping (as warnings under
5.7.3 and 5.8.0 already tell us). Commenting out 'sub($b)' makes it stop
segfaulting on all platforms​:

[kane@​gremlin tmp]$ more t
sub a{
  #sub($b);
  $b->c->d->e->f->g;
}

I hope this is of use for those that will actually *debug* this ;)

-jos

--

Jos Boumans

How do I prove I am not crazy to people who are?

CPANPLUS http​://cpanplus.sf.net
Just another perl hacker http​://japh.nu

[p5pRT]

From @hvds

"Jos I. Boumans" <kane@​cpan.org> wrote​:
:Pete Via Rt wrote​:
:> Immediate segfault is produced, when a file containing following
:> script is given to perl as command line argument.
:>
:> Lines after this line are in the file.
:> sub a{
:> sub($b);
:> $b->c->d->e->f->g;
:> }
:> Lines before this line are in the file.
[...]
:the problem can be reliably reproduced on all versions of perl I run on
:this machine​:
[...]
:The offending line lies in the improper prototyping (as warnings under
:5.7.3 and 5.8.0 already tell us).

Strangely, no; if you remove the prototype it still segfaults​:
  crypt% perl -we 'sub a { sub; $b->c->d->e->f; }'
  Segmentation fault (core dumped)
  crypt%

I'm not sure that I quite understand what -DX info tells me​:
  Pad 0x8145148[0x814bbc8] new​: padlist=0x8145130 name=0x814513c flags=0x0
  Pad 0x815185c[0x8148b70] new​: padlist=0x8151844 name=0x8151850 flags=0x6
  Pad 0x8151898[0x8148bb0] new​: padlist=0x8151880 name=0x815188c flags=0x6
  Pad 0x8151898[0x8148bb0] alloc​: 1 for entersub
  Pad 0x8151898[0x8148bb0] alloc​: 2 for entersub
  Pad 0x8151898[0x8148bb0] alloc​: 3 for entersub
  Pad 0x8151898[0x8153080] alloc​: 4 for entersub
  Pad 0x815185c[0x8148b70] free​: 1
  Pad 0x815185c[0x8148b70] free​: 2
  Pad 0x815185c[0x8148b70] free​: 3
  Pad 0x815185c[0x8148b70] free​: 4
  Segmentation fault (core dumped)
..... but my guess is that the anonymous sub declaration (for which we've
queued a 'syntax error' message that we never get to print) is leaving
us in an unstable state - possibly by leaving one but not the other of
curpad and comppad wrong.

Hugo

[p5pRT]

From @iabyn

On Mon, Oct 21, 2002 at 03​:34​:30PM +0100, hv@​crypt.org wrote​:

Strangely, no; if you remove the prototype it still segfaults​:
crypt% perl -we 'sub a { sub; $b->c->d->e->f; }'
Segmentation fault (core dumped)
crypt%

I'm not sure that I quite understand what -DX info tells me​:
Pad 0x8145148[0x814bbc8] new​: padlist=0x8145130 name=0x814513c flags=0x0
Pad 0x815185c[0x8148b70] new​: padlist=0x8151844 name=0x8151850 flags=0x6
Pad 0x8151898[0x8148bb0] new​: padlist=0x8151880 name=0x815188c flags=0x6
Pad 0x8151898[0x8148bb0] alloc​: 1 for entersub
Pad 0x8151898[0x8148bb0] alloc​: 2 for entersub
Pad 0x8151898[0x8148bb0] alloc​: 3 for entersub
Pad 0x8151898[0x8153080] alloc​: 4 for entersub
Pad 0x815185c[0x8148b70] free​: 1
Pad 0x815185c[0x8148b70] free​: 2
Pad 0x815185c[0x8148b70] free​: 3
Pad 0x815185c[0x8148b70] free​: 4
Segmentation fault (core dumped)

0x123[0x456] shows comppad and curpad;
'new​:' shows the creation a new padlist (for a new CV) - one for main, one
for 'sub a' and one for 'sub;'

when 'sub;' fails, comp/curpad aren't restored to the middle padlist,
so all the allocs are done in the wrong pad.

A simpler example​:

gizmo [sql]$ perl5.8.0 -DX -we 'sub a { my $x; sub; my $y }'
Pad 0x1de5d8 alloc 1 for padsv
Pad 0x1de650 alloc 1 for padsv
Pad 0x1de5d8 free 1
Pad 0x1de5d8 free 1

$x and $y get put in different pads

Probably best to leave off trying to fix it until my curpad==AvARRAY(comppad)
patch goes in.

--
"Strange women lying in ponds distributing swords is no basis for a system
of government. Supreme executive power derives from a mandate from the
masses, not from some farcical aquatic ceremony."
Dennis - Monty Python and the Holy Grail.

[p5pRT]

From @iabyn

On Mon, Oct 21, 2002 at 04​:08​:23PM +0100, Dave Mitchell wrote​:

when 'sub;' fails, comp/curpad aren't restored to the middle padlist,
so all the allocs are done in the wrong pad.

The problem with "sub;" is that the scope isn't restored​:

in perly.y,

  | ANONSUB startanonsub proto subattrlist block %prec '('
  { $$ = newANONATTRSUB($2, $3, $4, $5); }

startanonsub​: /* NULL */ /* start an anonymous subroutine scope */
  { $$ = start_subparse(FALSE, CVf_ANON); }

When "sub;" is parsed, start_subparse() is called but newANONATTRSUB()
isn't, which means that the LEAVE_SCOPE() in newANONATTRSUB() isn't
called, which means that the new scope and pad set up by start_subparse()
remain.

Presumably either toke.c has to clever ebough to spot the error early
and so avoid start_subparse() being called, or yacc has to be told
that on error, to do something like { $$ = LEAVE_SCOPE($2) }

But since I don't really understand either the 'error' token in yacc, nor
the complexities of toke.c, I'm hoping that someone else can fix this!

--
Nothing ventured, nothing lost.

[p5pRT]

From @rgs

Dave Mitchell wrote​:

The problem with "sub;" is that the scope isn't restored​:

in perly.y,

|    ANONSUB startanonsub proto subattrlist block    %prec '\('
            \{ $$ = newANONATTRSUB\($2\, $3\, $4\, $5\); \}

startanonsub​: /* NULL */ /* start an anonymous subroutine scope */
{ $$ = start_subparse(FALSE, CVf_ANON); }

When "sub;" is parsed, start_subparse() is called but newANONATTRSUB()
isn't, which means that the LEAVE_SCOPE() in newANONATTRSUB() isn't
called, which means that the new scope and pad set up by start_subparse()
remain.

Presumably either toke.c has to clever ebough to spot the error early
and so avoid start_subparse() being called, or yacc has to be told
that on error, to do something like { $$ = LEAVE_SCOPE($2) }

Why not turning "sub;" and "sub (some_proto);" into syntax errors ?
Well, in fact, they are already syntax errors, but we can make them
fail faster (i.e. from the lexer :)

(tests to be added in t/op/anonsub.t, but I prefer to see if this
can be ported to test.pl)

(do you like the error message ? I think a better wording can be found.)

Index​: pod/perldiag.pod

--- pod/perldiag.pod
+++ pod/perldiag.pod Thu Oct 24 09​:30​:22 2002
@​@​ -1691,6 +1691,10 @​@​
names (like C<$A​::B>). You've exceeded Perl's limits. Future versions
of Perl are likely to eliminate these arbitrary limitations.

+=item Illegal empty anonymous sub
+
+(F) You can't have a forward declaration for an anonymous sub.
+
=item Illegal binary digit %s

(F) You used a digit other than 0 or 1 in a binary number.
Index​: toke.c

--- toke.c
+++ toke.c Thu Oct 24 09​:48​:44 2002
@​@​ -5056,6 +5056,8 @​@​

  if (*s == '​:' && s[1] != '​:')
  PL_expect = attrful;
+ else if (!have_name && *s == ';' && key == KEY_sub)
+ Perl_croak(aTHX_ "Illegal empty anonymous sub");

  if (have_proto) {
  PL_nextval[PL_nexttoke].opval =
End of Patch.

[p5pRT]

From goldbb2@earthlink.net

Rafael Garcia-Suarez wrote​:
[snip]

Why not turning "sub;" and "sub (some_proto);" into syntax errors ?
Well, in fact, they are already syntax errors, but we can make them
fail faster (i.e. from the lexer :)
[snip]
if (*s == '​:' && s[1] != '​:')
PL_expect = attrful;
+ else if (!have_name && *s == ';' && key == KEY_sub)
+ Perl_croak(aTHX_ "Illegal empty anonymous sub");

This only covers "sub;" and "sub (stuff);", it doesn't cover things like
"sub, stuff", or "{sub}" or "sub && stuff", etc. In other words,
instead of checking for the presence of ";", you should check for the
abscence of "{".

+ else if (!have_name && *s != '{' && key == KEY_sub)
+ Perl_croak(aTHX_ "Illegal empty anonymous sub");

[untested]

PS​: At what point are comments stripped out? That is, if someone has
"sub\n#stuff\n{", will *s be '{', or will it be '#' ?

--
my $n = 2; print +(split //, 'e,4c3H r ktulrnsJ2tPaeh'
...."\n1oa! er")[map $n = ($n * 24 + 30) % 31, (42) x 26]

[p5pRT]

From @hvds

Benjamin Goldberg <goldbb2@​earthlink.net> wrote​:
:PS​: At what point are comments stripped out? That is, if someone has
:"sub\n#stuff\n{", will *s be '{', or will it be '#' ?

skipspace() also skips comments.

Hugo

[p5pRT]

From @rgs

hv@​crypt.org wrote​:

Benjamin Goldberg <goldbb2@​earthlink.net> wrote​:
:PS​: At what point are comments stripped out? That is, if someone has
:"sub\n#stuff\n{", will *s be '{', or will it be '#' ?

skipspace() also skips comments.

Here's a patch. It makes "sub;" and similar report an error immediately
from the lexer. However the solution is not complete, it doesn't
handle the case "sub ($) : lvalue;" (marked as a TODO test), because
it's handled by the lexer in separate tokens, hence more difficult to
work out.

Also, t/op/anonsub.t should not be ported to t/test.pl, because
test.pl uses closures.

Inline Patch

--- src/pod/perldiag.pod.orig	Thu Oct 17 22:26:59 2002
+++ src/pod/perldiag.pod	Mon Oct 28 23:06:22 2002
@@ -1714,6 +1714,10 @@ =item Illegal character in prototype for
 (W syntax) An illegal character was found in a prototype declaration.  Legal
 characters in prototypes are $, @, %, *, ;, [, ], &, and \.
 
+=item Illegal declaration of anonymous subroutine
+
+(F) You can't have a forward declaration for an anonymous subroutine.
+
 =item Illegal division by zero
 
 (F) You tried to divide a number by 0.  Either something was wrong in
--- src/t/op/anonsub.t.orig	Tue Oct  1 22:06:16 2002
+++ src/t/op/anonsub.t	Mon Oct 28 23:26:13 2002
@@ -12,7 +12,7 @@
 
 undef $/;
 @prgs = split "\n########\n", <DATA>;
-print "1..", scalar @prgs, "\n";
+print "1..", 6 + scalar @prgs, "\n";
 
 $tmpfile = "asubtmp000";
 1 while -f ++$tmpfile;
@@ -49,6 +49,30 @@
        print "not ";
     }
     print "ok ", ++$i, "\n";
+}
+
+sub test_invalid_decl {
+    my ($code,$todo) = @_;
+    $todo //= '';
+    eval $code;
+    if ($@ =~ /^Illegal declaration of anonymous subroutine at/) {
+	print "ok ", ++$i, " - '$code' is illegal$todo\n";
+    } else {
+	print "not ok ", ++$i, " - '$code' is illegal$todo\n# GOT: $@";
+    }
+}
+
+test_invalid_decl('sub;');
+test_invalid_decl('sub ($) ;');
+test_invalid_decl('{ $x = sub }');
+test_invalid_decl('sub ($) && 1');
+test_invalid_decl('sub ($) : lvalue;',' # TODO');
+
+eval "sub #foo\n{print 1}";
+if ($@ eq '') {
+    print "ok ", ++$i, "\n";
+} else {
+    print "not ok ", ++$i, "\n# GOT: $@";
 }
 
 __END__
--- src/toke.c.orig	Fri Oct 25 21:54:51 2002
+++ src/toke.c	Mon Oct 28 23:10:34 2002
@@ -5057,6 +5057,8 @@ Perl_yylex(pTHX)
 
 		if (*s == ':' && s[1] != ':')
 		    PL_expect = attrful;
+		else if (!have_name && *s != '{' && key == KEY_sub)
+		    Perl_croak(aTHX_ "Illegal declaration of anonymous subroutine");
 
 		if (have_proto) {
 		    PL_nextval[PL_nexttoke].opval =
End of Patch.

[p5pRT]

From @tamias

On Mon, Oct 28, 2002 at 11​:27​:27PM +0100, Rafael Garcia-Suarez wrote​:

Here's a patch. It makes "sub;" and similar report an error immediately
from the lexer. However the solution is not complete, it doesn't
handle the case "sub ($) : lvalue;" (marked as a TODO test), because
it's handled by the lexer in separate tokens, hence more difficult to
work out.

Also, t/op/anonsub.t should not be ported to t/test.pl, because
test.pl uses closures.

--- src/pod/perldiag.pod.orig Thu Oct 17 22​:26​:59 2002
+++ src/pod/perldiag.pod Mon Oct 28 23​:06​:22 2002
@​@​ -1714,6 +1714,10 @​@​ =item Illegal character in prototype for
(W syntax) An illegal character was found in a prototype declaration. Legal
characters in prototypes are $, @​, %, *, ;, [, ], &, and \.

+=item Illegal declaration of anonymous subroutine
+
+(F) You can't have a forward declaration for an anonymous subroutine.
+

This error message seems rather cryptic to me. In particular, I'm
imagining a novice programmer making this mistake, reading the error
message and the explanation, and still having no idea how to fix it.

How about something like​:

sub must be followed by a subroutine declaration or definition.

Ronald

[p5pRT]

From @hvds

Ronald J Kimball <rjk@​linguist.Thayer.dartmouth.edu> wrote​:
:On Mon, Oct 28, 2002 at 11​:27​:27PM +0100, Rafael Garcia-Suarez wrote​:
:> +=item Illegal declaration of anonymous subroutine
:> +
:> +(F) You can't have a forward declaration for an anonymous subroutine.
:
:This error message seems rather cryptic to me. In particular, I'm
:imagining a novice programmer making this mistake, reading the error
:message and the explanation, and still having no idea how to fix it.
:
:How about something like​:
:
:sub must be followed by a subroutine declaration or definition.

I'm not convinced that will help the novice programmer any more.
How about​:
  When using the C<sub> keyword to declare an anonymous subroutine,
  you must always specify a block of code. See L<perl???>.

Hugo

[p5pRT]

From @rgs

hv@​crypt.org wrote​:

Ronald J Kimball <rjk@​linguist.Thayer.dartmouth.edu> wrote​:
:On Mon, Oct 28, 2002 at 11​:27​:27PM +0100, Rafael Garcia-Suarez wrote​:
:> +=item Illegal declaration of anonymous subroutine
:> +
:> +(F) You can't have a forward declaration for an anonymous subroutine.
:
:This error message seems rather cryptic to me. In particular, I'm
:imagining a novice programmer making this mistake, reading the error
:message and the explanation, and still having no idea how to fix it.
:
:How about something like​:
:
:sub must be followed by a subroutine declaration or definition.

I'm not convinced that will help the novice programmer any more.
How about​:
When using the C<sub> keyword to declare an anonymous subroutine,
you must always specify a block of code. See L<perl???>.

Something like this applied as change #18072.

[p5pRT]

From @jhi

Since this isssue was fixed, I'm marking the problem ticket
as resolved.

[p5pRT]

@jhi - Status changed from 'new' to 'resolved'