How do I configure with ASAN?

[jkeenan]

More than once, I have had to ask, "How do I invoke ./Configure so as to build with ASAN?" (Most recently: #22586 (comment)) But I never seem to succeed. Can we straighten this out so that more people can explore address sanitizers?

My environment:

$ uname -mrs
Linux 6.8.0-40-generic x86_64

$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ git describe
v5.41.3

My most recent attempt (from #22586 (comment)):

$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=clang
...

./Configure gets almost all the way through, then segfaults:

...
Run make depend now? [y]  
clang -c -DPERL_CORE -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall generate_uudmap.c
clang -o generate_uudmap -fsanitize=address -fstack-protector-strong -L/usr/local/lib generate_uudmap.o -lpthread -ldl -lm -lcrypt -lutil -lc 
./generate_uudmap uudmap.h bitcount.h mg_data.h
make: *** [Makefile:346: bitcount.h] Segmentation fault (core dumped)
...

I should note in passing that it feels awkward to me to delay calling -Dcc=clang until after I've called -Accflags and -Aldflags. But when I put -Dcc=clang where it feels more natural, ./Configure quickly collapses:

$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address
...
Let me guess what the preprocessor flags are...
Any additional ld flags (NOT including libraries)?
[ -fsanitize=address -fstack-protector-strong -L/usr/local/lib]  
Checking your choice of C compiler and flags for coherency...

I've tried to compile and run the following simple program:

#include <stdio.h>
int main() { printf("Ok\n"); return(0); }

I used the command:

	clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc
	 ./try

and I got the following output:

Segmentation fault (core dumped)
The program compiled OK, but exited with status 139.
You have a problem.  Shall I abort Configure [y]  
Ok.  Stopping Configure.

Next, let's try the above only with gcc instead of clang:

$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=gcc -Accflags=-fsanitize=address -Aldflags=-fsanitize=address
...
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
...
^C

I get what appears to be an infinite loop of those DEADLYSIGNALs.

And finally, putting gcc at the end of the invocation:

$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=gcc
...
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer:DEADLYSIGNAL
...
^C

I have a vague recollection that in the past I was able to complete ./Configure only to have my machine grind to a halt during make. But now I can't even get through ./Configure!

Guidance requested.

[khwilliamson]

On 9/12/24 16:09, James E Keenan wrote: More than once, I have had to ask, "How do I invoke |./Configure| so as to build with ASAN?" (Most recently: #22586 (comment) <#22586 (comment)>) But I never seem to succeed. Can we straighten this out so that more people can explore address sanitizers? My environment: |$ uname -mrs Linux 6.8.0-40-generic x86_64 $ clang --version Ubuntu clang version 14.0.0-1ubuntu1.1 Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin $ gcc --version gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. $ git describe v5.41.3 | My most recent attempt (from #22586 (comment) <#22586 (comment)>): |$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=clang ... | |./Configure| gets almost all the way through, then segfaults:

This is my precise invocation that worked just now bin/bash ./Configure -des -Uversiononly '-Dprefix=/home/khw/perl/devel' -Dusedevel '-Doptimize=-O0' -DDEBUGGING $'-A\'optimize=-g3\'' $'-A\'optimize=-ggdb3\'' $'-A\'optimize=-g3\'' $'-Accflags=\'-fsanitize=address\'' $'-Accflags=\'-fsanitize-address-use-after-scope\'' $'-Accflags=\'-fno-omit-frame-pointer\'' $'-Accflags=\'-fno-common\'' $'-Accflags=\'-ftrapv\'' $'-Accflags=\'-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore\'' $'-Accflags=\'-Wno-deprecated\'' $'-Accflags=\'-DNO_NL_LOCALE_NAME\'' $'-Dman1dir=\'none\'' $'-Dman3dir=\'none\'' $'-Aldflags=\'-fsanitize=address\'' $'-Alddlflags=\'-fsanitize=address\'' $'-Aldflags=\'-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore\'' $'-Alddlflags=\'-fsanitize-blacklist=/home/khw/perl/locales/straight/asan_ignore\'' $'-Alddlflags=\'-shared\'' '-Dcc=clang' -Dusecbacktrace -Dusethreads Some of these are obsolete or not generally applicable. asan_ignore is shipped with perl.

…

|... Run make depend now? [y] clang -c -DPERL_CORE -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -Wall generate_uudmap.c clang -o generate_uudmap -fsanitize=address -fstack-protector-strong -L/usr/local/lib generate_uudmap.o -lpthread -ldl -lm -lcrypt -lutil -lc ./generate_uudmap uudmap.h bitcount.h mg_data.h make: *** [Makefile:346: bitcount.h] Segmentation fault (core dumped) ... | I should note in passing that it feels awkward to me to delay calling |-Dcc=clang| until *after* I've called |-Accflags| and |-Aldflags|. But when I put |-Dcc=clang| where it feels more natural, |./Configure| quickly collapses: |$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address ... Let me guess what the preprocessor flags are... Any additional ld flags (NOT including libraries)? [ -fsanitize=address -fstack-protector-strong -L/usr/local/lib] Checking your choice of C compiler and flags for coherency... I've tried to compile and run the following simple program: #include <stdio.h> int main() { printf("Ok\n"); return(0); } I used the command: clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc ./try and I got the following output: Segmentation fault (core dumped) The program compiled OK, but exited with status 139. You have a problem. Shall I abort Configure [y] Ok. Stopping Configure. | Next, let's try the above only with |gcc| instead of |clang|: |$ git clean -dfx; sh ./Configure -des -Dusedevel -Dcc=gcc -Accflags=-fsanitize=address -Aldflags=-fsanitize=address ... AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL ... ^C | I get what appears to be an infinite loop of those |DEADLYSIGNAL|s. And finally, putting |gcc| at the end of the invocation: |$ git clean -dfx; sh ./Configure -des -Dusedevel -Accflags=-fsanitize=address -Aldflags=-fsanitize=address -Dcc=gcc ... AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL AddressSanitizer:DEADLYSIGNAL ... ^C | I have a vague recollection that in the past I was able to complete |./Configure| only to have my machine grind to a halt during |make|. But now I can't even get through |./Configure|! Guidance requested. — Reply to this email directly, view it on GitHub <#22588>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2DH5PT3MK7LDFQTBQP6TZWIGKRAVCNFSM6AAAAABOEFI2BSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGUZDGNBUGQ3DQMQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mauke]

I can't reproduce your segfaults. I've just built perl from a clean git directory using these exact commands:

./Configure -Dcc=clang -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des
make

Unsurprisingly, gcc also works here:

./Configure -Dcc=gcc -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des
make

In either case, I get a working perl:

$ ./perl -v

This is perl 5, version 41, subversion 4 (v5.41.4 (v5.41.3-102-gbd12780ddd)) built for x86_64-linux
...

My environment:

$ uname -mrs
Linux 6.8.0-105041-tuxedo x86_64

$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ git describe 
v5.41.3-102-gbd12780ddd

[jkeenan]

On my Ubuntu Linux laptop (where the original complaint) was run, I tried @mauke's invocation and got the same failures as before.

Much to my surprise, I got farther on two servers to which I have access.

Debian 11

$ uname -mrs
Linux 6.1.0-23-amd64 x86_64

$ gcc --version | head -n 1
gcc (Debian 12.2.0-14) 12.2.0

$ clang --version
Debian clang version 14.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ ./Configure -Dcc=gcc -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des && make test_prep
[# Completed successfully ]
$ make test_harness
...
Test Summary Report
-------------------
re/pat.t                                                           (Wstat: 9 (Signal: KILL) Tests: 276 Failed: 0)
  Non-zero wait status: 9
  Parse errors: Bad plan.  You planned 1295 tests but ran 276.
re/pat_psycho.t                                                    (Wstat: 9 (Signal: KILL) Tests: 1 Failed: 0)
  Non-zero wait status: 9
  Parse errors: Bad plan.  You planned 15 tests but ran 1.
Files=2889, Tests=1184831, 1088 wallclock secs (149.93 usr 40.12 sys + 1170.84 cusr 369.65 csys = 1730.54 CPU)
Result: FAIL

ISTR that at times in the past we've seen failures in t/re/pat.t and t/re/pat_psycho.t under ASAN.

FreeBSD 13

$ uname -mrs
FreeBSD 13.3-RELEASE-p6 amd64

$ gcc --version | head -n 1
gcc (FreeBSD Ports Collection) 13.2.0

$ clang --version
FreeBSD clang version 17.0.6 (https://github.com/llvm/llvm-project.git llvmorg-17.0.6-0-g6009708b4367)
Target: x86_64-unknown-freebsd13.3
Thread model: posix
InstalledDir: /usr/bin

$ ./Configure -Dcc=clang -Accflags='-fsanitize=address' -Aldflags='-fsanitize=address' -Dusedevel -des && make
...
	Everything is up to date. Type 'make test' to run test suite.

$ make test_prep
...
$ make test_harness
...
All tests successful.
Files=2900, Tests=1186258, 823 wallclock secs (264.85 usr 46.89 sys + 2030.75 cusr 373.27 csys = 2715.77 CPU)
Result: PASS

I have a suspicion that this is related to these problems reported to the 'actions' project on GitHub:

actions/runner-images#9491

actions/runner-images#9524

... but the issues discussed there are way outside my area of expertise.

[iabyn]

On Mon, Sep 16, 2024 at 01:15:46PM -0700, James E Keenan wrote: On my Ubuntu Linux laptop (where the original complaint) was run, I tried @mauke's invocation and got the same failures as before. Much to my surprise, I got farther on two servers to which I have access.

Note that ASAN requires a *lot* more memory than a normal build. For example on my 16Gb laptop I normally build+test with -j16, but under ASAN I run with -j4 - any more, and I see tests start to fail.

…

-- The optimist believes that he lives in the best of all possible worlds. As does the pessimist.

[tonycoz]

I've tried to compile and run the following simple program:

#include <stdio.h>
int main() { printf("Ok\n"); return(0); }

I used the command:

	clang -o try -O2 -fsanitize=address -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -fsanitize=address -fstack-protector-strong -L/usr/local/lib try.c -lpthread -ldl -lm -lcrypt -lutil -lc
	 ./try

What do you see if you try copying the code to a file and compiling/running it yourself? Does it differ if you remove all the -l arguments?

If that's failing it seems like you have a toolchain problem, not a perl problem.

[jkeenan]

What do you see if you try copying the code to a file and compiling/running it yourself? Does it differ if you remove all the -l arguments?

If that's failing it seems like you have a toolchain problem, not a perl problem.

$ uname -mrs
Linux 6.8.0-45-generic x86_64
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ cat gh-22588-try.c 
#include <stdio.h>
int main() { printf("Ok\n"); return(0); }

$ clang -o try -O2 -fsanitize=address -fno-strict-aliasing \
-pipe -fstack-protector-strong -I/usr/local/include \
-fsanitize=address -fstack-protector-strong \
-L/usr/local/lib $P5P_DIR/gh-22588-try.c \
-lpthread -ldl -lm -lcrypt -lutil -lc

$ ./try
Ok

$ rm ./try
$ clang -o try -O2 -fsanitize=address -fno-strict-aliasing \
-pipe -fstack-protector-strong -I/usr/local/include \
-fsanitize=address -fstack-protector-strong \
-L/usr/local/lib $P5P_DIR/gh-22588-try.c
$ ./try
Ok

[tonycoz]

Exactly which OS are you running this on? Ubuntu something, but which version?

[jkeenan]

Exactly which OS are you running this on? Ubuntu something, but which version?

22.04 LTS

$  uname -a
Linux tpadx1 6.8.0-45-generic #45~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep 11 15:25:05 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

I expect to upgrade to 24.04 within 2 months.

[tonycoz]

I suspect you have a damaged install, you could try:

sudo dpkg -V

though you'll probably see it complain about many config files.

[jkeenan]

I upgraded to Ubuntu 24.04 LTS last week, which brought along upgrades to clang and gcc.

$ uname -mrs
Linux 6.8.0-47-generic x86_64

$ lsb_release -a
...
Description:	Ubuntu 24.04.1 LTS
...

$ gcc --version
gcc (Ubuntu 13.2.0-23ubuntu4) 13.2.0
...

$ clang --version
Ubuntu clang version 18.1.3 (1ubuntu1)
...

I was able to configure and build (thru make test_prep) successfully with these commands:

$ git describe
v5.41.5-17-g161ff7336a

$ ./perl -Ilib -V:config_args
config_args='-des -Dusedevel -Dcc=clang -Accflags=-fsanitize=address -Aldflags=-fsanitize=address';

As expected, make test_harness experienced failures, likely due to memory, on programs like t/re/pat_psycho.t and slowed the machine to a mini-crawl, so I killed the harness.

[khwilliamson]

@jkeenan is this closable?