exec & Posix::_exit not well ported to Windows

[p5pRT]

Migrated from rt.perl.org#75080 (status was 'open')

Searchable as RT75080$

[p5pRT]

From occitan@esperanto.org

Hi!

The build tool makepp does a lot of fork/exec or (an extension to make)
fork/perl-code/_exit. We use _exit because we have a lot of data and
short-circuiting destruction saves noticeable time.

Meanwhile under MinGW and Cygwin this works as expected — the libc
emulates this well. But under Strawberry or ActiveState both exec and
_exit terminate all "processes" together. OTOH plain exit unwinds all
of the call stack and complains about freeing something twice — might be
our bug since it was never intended to happen, though I don't see how
plain perl can cause that.

For many years we have lived with not having -j parallel builds. But
multi-core is becoming more common under Windows, making this problem a
show stopper for efficient builds.

Since the libc seems to cope just fine, wouldn't it be an option to
use/copy their fork/exec/exit/_exit?

coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer

--
lerne / learn / apprends / lär dig / ucz się Esperanto​:
  http​://lernu.net /http​://ikurso.net

[p5pRT]

From @smpeters

On Thu May 13 04​:02​:09 2010, occitan@​esperanto.org wrote​:

Hi!

The build tool makepp does a lot of fork/exec or (an extension to make)
fork/perl-code/_exit. We use _exit because we have a lot of data and
short-circuiting destruction saves noticeable time.

Meanwhile under MinGW and Cygwin this works as expected — the libc
emulates this well. But under Strawberry or ActiveState both exec and
_exit terminate all "processes" together. OTOH plain exit unwinds all
of the call stack and complains about freeing something twice — might be
our bug since it was never intended to happen, though I don't see how
plain perl can cause that.

For many years we have lived with not having -j parallel builds. But
multi-core is becoming more common under Windows, making this problem a
show stopper for efficient builds.

Since the libc seems to cope just fine, wouldn't it be an option to
use/copy their fork/exec/exit/_exit?

coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer

Unfortuantely, makepp (http​://makepp.sourceforge.net/) is not supported
here. I would suggest contacting the authors of makepp directly.

Thanks,

Steve Peters

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

@smpeters - Status changed from 'open' to 'rejected'

[p5pRT]

From occitan@esperanto.org

  la 08/14/2010 09​:54 PM Steve Peters via RT skribis​:

On Thu May 13 04​:02​:09 2010, occitan@​esperanto.org wrote​:

Hi!

The build tool makepp does a lot of fork/exec or (an extension to make)
fork/perl-code/_exit. We use _exit because we have a lot of data and
short-circuiting destruction saves noticeable time.

Meanwhile under MinGW and Cygwin this works as expected — the libc
emulates this well. But under Strawberry or ActiveState both exec and
_exit terminate all "processes" together. OTOH plain exit unwinds all
of the call stack and complains about freeing something twice — might be
our bug since it was never intended to happen, though I don't see how
plain perl can cause that.

For many years we have lived with not having -j parallel builds. But
multi-core is becoming more common under Windows, making this problem a
show stopper for efficient builds.

Since the libc seems to cope just fine, wouldn't it be an option to
use/copy their fork/exec/exit/_exit?

coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer

Unfortuantely, makepp (http​://makepp.sourceforge.net/) is not supported
here. I would suggest contacting the authors of makepp directly.

I am one of the makepp developers. This is not about a makepp bug, but about
an *ActiveState/Strawberry Perl problem*, which prevents makepp from reaching
its full potential on native Windows. The emulation of
fork/exec/exit/POSIX​::_exit is not good enough for parallel make, but Cygwin
and MinGW show that it is possible even on Windows!

Dual- or quadcore CPUs are almost normal, meaning this is no longer just a
nice to have, but a necessity. So I want the native Windows port to do it the
same way, possibly borrowing these functions from glibc, if needed.

coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer

--
lerne / learn / apprends / lär dig / ucz się Esperanto​:
  http​://lernu.net / http​://ikurso.net

[p5pRT]

From @smpeters

On Sat Aug 14 17​:04​:18 2010, occitan@​esperanto.org wrote​:

la 08/14/2010 09​:54 PM Steve Peters via RT skribis​:

On Thu May 13 04​:02​:09 2010, occitan@​esperanto.org wrote​:

Hi!

The build tool makepp does a lot of fork/exec or (an extension to
make)
fork/perl-code/_exit. We use _exit because we have a lot of data
and
short-circuiting destruction saves noticeable time.

Meanwhile under MinGW and Cygwin this works as expected — the libc
emulates this well. But under Strawberry or ActiveState both exec
and
_exit terminate all "processes" together. OTOH plain exit unwinds
all
of the call stack and complains about freeing something twice —
might be
our bug since it was never intended to happen, though I don't see
how
plain perl can cause that.

For many years we have lived with not having -j parallel builds.
But
multi-core is becoming more common under Windows, making this
problem a
show stopper for efficient builds.

Since the libc seems to cope just fine, wouldn't it be an option to
use/copy their fork/exec/exit/_exit?

coralament / best Grötens / liebe Grüße / best regards / elkorajn
salutojn
Daniel Pfeiffer

Unfortuantely, makepp (http​://makepp.sourceforge.net/) is not
supported
here. I would suggest contacting the authors of makepp directly.

I am one of the makepp developers. This is not about a makepp bug,
but about
an *ActiveState/Strawberry Perl problem*, which prevents makepp from
reaching
its full potential on native Windows. The emulation of
fork/exec/exit/POSIX​::_exit is not good enough for parallel make, but
Cygwin
and MinGW show that it is possible even on Windows!

Dual- or quadcore CPUs are almost normal, meaning this is no longer
just a
nice to have, but a necessity. So I want the native Windows port to
do it the
same way, possibly borrowing these functions from glibc, if needed.

Borrowing from glibc would significantly effect the licensing of Perl,
so that is not a workable option.

[p5pRT]

@smpeters - Status changed from 'rejected' to 'open'

[p5pRT]

From occitan@esperanto.org

  Hi Steve,

la 08/18/2010 04​:32 PM Steve Peters via RT skribis​:

On Sat Aug 14 17​:04​:18 2010, occitan@​esperanto.org wrote​:

la 08/14/2010 09​:54 PM Steve Peters via RT skribis​:

On Thu May 13 04​:02​:09 2010, occitan@​esperanto.org wrote​:

Hi!

The build tool makepp does a lot of fork/exec or (an extension to
make)
fork/perl-code/_exit. We use _exit because we have a lot of data
and
short-circuiting destruction saves noticeable time.

Meanwhile under MinGW and Cygwin this works as expected — the libc
emulates this well. But under Strawberry or ActiveState both exec
and
_exit terminate all "processes" together. OTOH plain exit unwinds
all
of the call stack and complains about freeing something twice —
might be
our bug since it was never intended to happen, though I don't see
how
plain perl can cause that.

For many years we have lived with not having -j parallel builds.
But
multi-core is becoming more common under Windows, making this
problem a
show stopper for efficient builds.

Since the libc seems to cope just fine, wouldn't it be an option to
use/copy their fork/exec/exit/_exit?

coralament / best Grötens / liebe Grüße / best regards / elkorajn
salutojn
Daniel Pfeiffer

Unfortuantely, makepp (http​://makepp.sourceforge.net/) is not
supported
here. I would suggest contacting the authors of makepp directly.
I am one of the makepp developers. This is not about a makepp bug,
but about
an *ActiveState/Strawberry Perl problem*, which prevents makepp from
reaching
its full potential on native Windows. The emulation of
fork/exec/exit/POSIX​::_exit is not good enough for parallel make, but
Cygwin
and MinGW show that it is possible even on Windows!

Dual- or quadcore CPUs are almost normal, meaning this is no longer
just a
nice to have, but a necessity. So I want the native Windows port to
do it the
same way, possibly borrowing these functions from glibc, if needed.

Borrowing from glibc would significantly effect the licensing of Perl,
so that is not a workable option.
Yeah, well, GPL3 is not Artistic. I'm no lawyer, but how about analyzing how
they do it, and then reimplementing that?

If it helps, I can supply perl one liners that show what is not working. On
Windows devolopment I'm hopeless though.

coralament / best Grötens / liebe Grüße / best regards / elkorajn salutojn
Daniel Pfeiffer

--
lerne / learn / apprends / lär dig / ucz się Esperanto​:
  http​://lernu.net / http​://ikurso.net

[p5pRT]

From ben@morrow.me.uk

Quoth perlbug-followup@​perl.org​:

On Sat Aug 14 17​:04​:18 2010, occitan@​esperanto.org wrote​:

I am one of the makepp developers. This is not about a makepp bug,
but about
an *ActiveState/Strawberry Perl problem*, which prevents makepp from
reaching
its full potential on native Windows. The emulation of
fork/exec/exit/POSIX​::_exit is not good enough for parallel make, but
Cygwin
and MinGW show that it is possible even on Windows!

Dual- or quadcore CPUs are almost normal, meaning this is no longer
just a
nice to have, but a necessity. So I want the native Windows port to
do it the
same way, possibly borrowing these functions from glibc, if needed.

Borrowing from glibc would significantly effect the licensing of Perl,
so that is not a workable option.

More importantly, if you want Cygwin, use Cygwin. Win32 perl is (or
should be) an attempt to provide as much of Perl as possible *on Win32*
without a complex emulation layer underneath (ithreads and pseudofork
notwithstanding).

Ben

[jkeenan]

This ticket was unintentionally closed during the RT-to-GH transition in October 2019. Its status had been changed in RT from 'rejected' to 'open' in August 2010, but no reason for the re-opening was provided.

I am re-opening this ticket and self-assigning for the purpose of allowing anyone who wants to keep this ticket open the opportunity to speak up. Otherwise will close within 7 days.