[flang] Slow compilation and high memory usage for htdp.f

[hawkinsp]

The attached file is from HTDP version 3.2.5.
htdp.zip

flang-new built from head as of today takes at least 2 minutes and more than 60GB of memory to compile this file. I don't know how much it takes in the end since I killed it once the memory usage reached that level.

Repro withflang-new -c htdp.f

gfortran 12.2.0 compiles it reasonably quickly and compactly (25.6s, 566MB memory usage):

$ /usr/bin/time -v gfortran -c htdp.f
...
        Command being timed: "gfortran -c htdp.f"
        User time (seconds): 25.56
        System time (seconds): 0.46
        Percent of CPU this job got: 99%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:26.09
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 566388
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 134119
        Voluntary context switches: 11
        Involuntary context switches: 84
        Swaps: 0
        File system inputs: 0
        File system outputs: 21352
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

[hakostra]

I cloned the Github HTDP repo and tried the compilation with Intel ifx, GNU gfortran and LLVM flang-new.

Compile times for the entire HTDP codebase sources, with -O2 as the only flag:

GNU gfortran:

	Command being timed: "make"
	User time (seconds): 4.24
	System time (seconds): 0.22
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.46
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 495872
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 196172
	Voluntary context switches: 69
	Involuntary context switches: 326
	Swaps: 0
	File system inputs: 0
	File system outputs: 27616
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Intel ifx:

	Command being timed: "make"
	User time (seconds): 2.27
	System time (seconds): 0.37
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.65
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 405724
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 3
	Minor (reclaiming a frame) page faults: 195540
	Voluntary context switches: 125
	Involuntary context switches: 129
	Swaps: 0
	File system inputs: 0
	File system outputs: 12072
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

LLVM flang-new:

	Command being timed: "make"
	User time (seconds): 434.62
	System time (seconds): 1.24
	Percent of CPU this job got: 100%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 7:15.73
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 786676
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 17
	Minor (reclaiming a frame) page faults: 314315
	Voluntary context switches: 12860
	Involuntary context switches: 24228
	Swaps: 0
	File system inputs: 0
	File system outputs: 12416
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

So it seems maximum memory usage is reasonable and in the same order of magnitude between these three compilers now.

The compile time for flang-new is still way higher than ifx and gfortran, but that is not due to the file htdp.f, this compiles in a second (roughly) in all three compilers now. The culprit is initvl.f:

GNU gfortran:

$ /usr/bin/time -v gfortran -O2 -c initvl.f
	Command being timed: "gfortran -O2 -c initvl.f"
	User time (seconds): 2.36
	System time (seconds): 0.15
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.52
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 493056
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 122104
	Voluntary context switches: 7
	Involuntary context switches: 353
	Swaps: 0
	File system inputs: 0
	File system outputs: 14840
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Intel ifx:

$ /usr/bin/time -v ifx -O2 -c initvl.f
	Command being timed: "ifx -O2 -c initvl.f"
	User time (seconds): 0.73
	System time (seconds): 0.19
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.93
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 403684
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 100200
	Voluntary context switches: 15
	Involuntary context switches: 79
	Swaps: 0
	File system inputs: 0
	File system outputs: 3352
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

LLVM flang-new:

$ /usr/bin/time -v flang-new -O2 -c initvl.f
	Command being timed: "flang-new -O2 -c initvl.f"
	User time (seconds): 415.05
	System time (seconds): 0.82
	Percent of CPU this job got: 99%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 6:56.33
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 769988
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 192278
	Voluntary context switches: 206
	Involuntary context switches: 29628
	Swaps: 0
	File system inputs: 0
	File system outputs: 3288
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

The initvl.f seems to be basically a big data block with static numerical data...

As an important side-note, I would like to add that there are some linker warnings when linking the final executable with flang-new:

flang-new -O2  -o htdp htdp.o initvl.o initeq.o initbd.o initps.o  
/usr/bin/ld: warning: size of symbol `qparm_' changed from 199084 in htdp.o to 199088 in initeq.o
/usr/bin/ld: warning: size of symbol `psgrid_' changed from 44 in htdp.o to 48 in initps.o

I cannot see any obvious faults in the code. These might be worth looking into as a separate issue. Since I do not know this code I never tried to run the final executable.

I did all of this testing with:

$ flang-new --version
flang-new version 20.0.0git (https://github.com/llvm/llvm-project.git cd634f57c10dedbe4f908889dece2c4460b702c9)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/llvm/llvm-project/install/bin

[swatheesh-mcw]

Adding a reproducer which might be related to the mentioned slow compilation issue:

module derived_type_mod
    implicit none
    integer, parameter :: dim = 50000
    type :: type1
        integer :: var
    end type type1
    type :: type2
        type(type1), dimension(:), allocatable :: member1
    end type type2
    type(type2), dimension(dim) :: array
end module derived_type_mod

While analyzing, I could see that the slow compilation is happening on MLIR when translating MLIR (LLVM IR dialect) to LLVM IR. This slow code generation is observed for higher dimensions of derived type array especially if the member (member1) of derived type (type2) is allocatable and type of the member is of derived type (type1). If the type of member is of common types (like integer, real etc.,) or if the member has static size, then the slow code generation is not observed. It seems initialization in ModuleTranslation::convertGlobalsAndAliases() function in ModuleTranslation.cpp on MLIR which is responsible for conversion of globals and aliases to corresponding LLVM IR is causing the slow down for this reproducer.

The same reproducer compiles fine with gfortran, ifx and ifort compilers.

[DanielCChen]

This Issue might also be related #153197, where the code is doing explicit initialization to a large array.