[Fortran/gfortran] Enable some "compile" tests

Enable the 'compile' tests from all the directories under Fortran/gfortran/regression
(except for the tests in regression itself). Disable those tests that fail.

Author: tarunprabhu

Fortran/gfortran/README.md

@@ -4,36 +4,41 @@ This is the top-level directory for tests imported from
 [GCC](https://github.com/gcc-mirror/gcc). The test files are contained within
 two subdirectories:
 
-    - `regression`: Contains the gfortran [tests](https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite/gfortran.dg).
-    - `torture`: Contains the gfortran [torture tests](https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite/gfortran.fortran-torture).
+- `regression`: Contains the gfortran [tests](https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite/gfortran.dg).
+- `torture`: Contains the gfortran [torture tests](https://github.com/gcc-mirror/gcc/tree/master/gcc/testsuite/gfortran.fortran-torture).
 
 The tests in both `regression` (and its subdirectories) and `torture` can be
 classified roughly as _compile_ tests and _execute_ tests. The _compile_ tests
 generally check the compiler's error/warning messages and, in some cases,
 optimization logs. The _execute_ tests are end-to-end tests that check the
 behavior of the binary produced by the compiler.
 
-Currently, only the _execute_ tests are supported in `regression`. Both 
+Currently, only the _execute_ tests are supported in `regression`. Both
 `compile` and `execute` tests have been enabled in `torture`.
 
 Of the supported tests, a number of tests have been disabled. There are four
 categories of such tests:
 
-    - *Unsupported*: These are tests that use non-standard extensions/intrinsics
-    that are not currently supported by flang. Unless those non-standard
-    features are supported in the future, these tests will never be enabled.
+- *Unsupported*: These are tests that use non-standard extensions/intrinsics
+that are not currently supported by flang. Unless those non-standard
+features are supported in the future, these tests will never be enabled.
 
-    - *Unimplemented*: These tests hit a "not yet implemented" assertion within
-    flang.
+- *Unimplemented*: These tests hit a "not yet implemented" assertion within
+flang.
 
-    - *Skipped*: These tests cause some form of compiler error. Some trigger an
-    assertion within the compiler. Others are legal Fortran programs, but
-    nevertheless cause a semantic error, most likely due to unimplemented
-    features.
+- *Skipped*: These tests cause some form of compiler error. Some trigger an
+assertion within the compiler. Others are legal Fortran programs, but
+nevertheless cause a semantic error, most likely due to unimplemented
+features.
 
-    - *Failing*: These tests fail at runtime - some crash on execution, others
-    produce incorrect/unexpected output. This could be a result of a bug in the
-    compiler/code generator or the runtime.
+- *Failing*: These tests fail at test-time.
+
+  - For "execute" tests, some crash on execution, others produce
+  incorrect/unexpected output. This could be a result of a bug in the
+  compiler/code generator or the runtime.
+
+  - For "compile" tests, this could be because the compilation succeeds when it
+  is expected to fail, or vice versa.
 
 Over time, the number of tests in the *unimplemented*, *skipped*, and *failing*
 categories should decrease. Eventually, only the *unsupported* category should
@@ -42,8 +47,8 @@ remain.
 
 ### _Compile_ tests ###
 
-The _compile_ tests are "built" when the whole test suite is built at which 
-time a compilation log is saved. At testing time, the log is checked to 
+The _compile_ tests are "built" when the whole test suite is built at which
+time a compilation log is saved. At testing time, the log is checked to
 determine whether the test should pass or fail. If the test is expected to pass,
 but the compilation log contains errors, the test will be deemed to have failed
 and vice versa. The _compile_ test are supported in `torture`, but not in
@@ -66,11 +71,11 @@ default.
 In order to enable the disabled tests, one or more of the following options can
 be passed to `cmake`:
 
-    - `TEST_SUITE_FORTRAN_FORCE_ALL_TESTS`: Enable all disabled tests.
-    - `TEST_SUITE_FORTRAN_FORCE_UNSUPPORTED_TESTS`: Enable only the *unsupported* tests.
-    - `TEST_SUITE_FORTRAN_FORCE_UNIMPLEMENTED_TESTS`: Enable only the *unimplemented* tests.
-    - `TEST_SUITE_FORTRAN_FORCE_SKIPPED_TESTS`: Enable only the *skipped* tests.
-    - `TEST_SUITE_FORTRAN_FORCE_FAILING_TESTS`: Enable only the *failing* tests.
+- `TEST_SUITE_FORTRAN_FORCE_ALL_TESTS`: Enable all disabled tests.
+- `TEST_SUITE_FORTRAN_FORCE_UNSUPPORTED_TESTS`: Enable only the *unsupported* tests.
+- `TEST_SUITE_FORTRAN_FORCE_UNIMPLEMENTED_TESTS`: Enable only the *unimplemented* tests.
+- `TEST_SUITE_FORTRAN_FORCE_SKIPPED_TESTS`: Enable only the *skipped* tests.
+- `TEST_SUITE_FORTRAN_FORCE_FAILING_TESTS`: Enable only the *failing* tests.
 
 Some of the tests require the `ISO_Fortran_binding.h` header file. `cmake` will
 look for this file in the `include` directory of the `flang` installation
@@ -116,3 +121,57 @@ manually periodically.
 
 The test files in `regression` and `torture` *must not* be modified.
 
+
+### TODO's ###
+
+If some of the items listed here are implemented, even in part, it should
+allows us to make better use of the test-suite.
+
+Several DejaGNU directives from the test files are currently ignored. In some
+cases, those directives check that the language feature/optimization being
+exercised by the tests is actually handled correctly. By ignoring them, we are
+simply checking that `flang` (or the code produced by it) does not crash at
+build/test time. In the case of the _compile_ tests where this is the case, we
+could have situations where the test passes because the compilation succeeded,
+not because the compiler actually did the right thing - for instance, the tests
+in `gfortran/regression/vect` check if the code was correctly vectorized. We
+could pass those tests just by failing to crash - not because `flang` actually
+vectorized the code.
+
+It is not clear how much effort would be involved in correctly handling all the
+DejaGNU  directives.
+
+### `dg-error` directive ###
+
+The `dg-error` directive indicates that the test should fail to compile with a
+particular error. Obviously, this is a `gfortran`-specific error. `flang` may
+not have a direct equivalent i.e. it may produce a more general error message
+(or maybe even a more specific one if `gfortran` is the one with the more
+general error message). For now, when a`dg-error` is encountered, the test is
+marked as `expect-error`. At test time, we only check if "some" error (that was
+not a crash) occurred. This can cause false-negatives, particularly in the
+OpenMP (and perhaps even OpenACC) tests. This is where some directives/clauses
+are currently not implemented which results in a parse error (as opposed to the
+triggering of a "not-yet-implemented" assertion) which is also deemed an "error",
+thereby causing the test to pass.
+
+#### `scan-tree-dump` directive ####
+
+In the _compile_ tests, the `dg-final { scan-tree-dump* ...}` directives are
+ignored. The `scan-tree-dump*` checks GCC's internal tree structure to ensure
+that the specific language feature/optimization the test was meant to exercise
+was handled correctly (see, for example, `regression/volatile_7.f90`).
+
+The tests instruct GCC to write out the internal representation to file and scan
+the file for the presence or absence of certain text. To capture the same
+behavior here, we would need to parse and translate the internal representation
+of GCC to an equivalent representation in LLVM IR.
+
+### `target` directive ###
+
+The `target` directive is used to restrict tests to run on certain
+platforms/systems. Currently, the target directive is ignored entirely and the
+tests are always run. Currently, the gfortran tests are only enabled on *nix on
+x86-64 and aarch64 and ignoring the directive seems to be ok. As support for
+more systems and architectures are added, these directives will need to be
+handled correctly.

Fortran/gfortran/regression/analyzer/CMakeLists.txt

@@ -9,5 +9,4 @@
 set(TESTS)
 gfortran_populate_tests(TESTS)
 gfortran_add_execute_tests_from("${TESTS}")
-# TODO: Enable the "compile" tests.
-# gfortran_add_compile_tests_from("${TESTS}")
+gfortran_add_compile_tests_from("${TESTS}")

Fortran/gfortran/regression/analyzer/DisabledFiles.cmake

@@ -9,11 +9,21 @@
 # There are currently no unsupported files.
 set(UNSUPPORTED_FILES "")
 
-# There are currently no unimplemented files.
-set(UNIMPLEMENTED_FILES "")
+# These tests are disabled because they trigger "not yet implemented"
+# assertions.
+file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
+  # unimplemented: allocatable components in derived type assignment
+  pr93777.f90
+
+  # unimplemented: gather rhs LEN parameters in assignment to allocatable
+  deferred_character_25.f90
+)
 
 # There are currently no skipped files.
 set(SKIPPED_FILES "")
 
-# There are currently no failing files.
-set(FAILING_FILES "")
+# There tests fail when they are expected to pass.
+file(GLOB FAILING_FILES CONFIGURE_DEPENDS
+  # These files fail to compile when compilation is expected to succeed.
+  malloc.f90
+  malloc-example.f90)

Fortran/gfortran/regression/asan/CMakeLists.txt

@@ -9,5 +9,4 @@
 set(TESTS)
 gfortran_populate_tests(TESTS)
 gfortran_add_execute_tests_from("${TESTS}")
-# TODO: Enable the "compile" tests.
-# gfortran_add_compile_tests_from("${TESTS}")
+gfortran_add_compile_tests_from("${TESTS}")

Fortran/gfortran/regression/asan/DisabledFiles.cmake

@@ -9,8 +9,13 @@
 # There are currently no unsupported files.
 set(UNSUPPORTED_FILES "")
 
-# There are currently no unimplemented files.
-set(UNIMPLEMENTED_FILES "")
+# These tests are disabled because they trigger a "not yet implemented"
+# assertion.
+file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
+  # unimplemented: character array expression temp with dynamic length
+  associate_58.f90
+  associate_59.f90
+)
 
 # These tests trigger internal compiler errors.
 file(GLOB SKIPPED_FILES CONFIGURE_DEPENDS

Fortran/gfortran/regression/c-interop/CMakeLists.txt

@@ -9,5 +9,4 @@
 set(TESTS)
 gfortran_populate_tests(TESTS)
 gfortran_add_execute_tests_from("${TESTS}")
-# TODO: Enable the "compile" tests.
-# gfortran_add_compile_tests_from("${TESTS}")
+gfortran_add_compile_tests_from("${TESTS}")

Fortran/gfortran/regression/c-interop/DisabledFiles.cmake

@@ -20,6 +20,7 @@ file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
   argument-association-assumed-rank-6.f90
   argument-association-assumed-rank-7.f90
   argument-association-assumed-rank-8.f90
+  c535a-1.f90
   cf-out-descriptor-6.f90
   contiguous-1.f90
   contiguous-2.f90
@@ -51,6 +52,14 @@ file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
   pr103390-7.f90
   pr103390-8.f90
   pr103390-9.f90
+
+  # unimplemented: support for polymorphic types
+  c407a-1.f90
+  c407b-1.f90
+
+  # unimplemented: procedure pointer arguments
+  removed-restrictions-2.f90
+  removed-restrictions-4.f90
 )
 
 # These tests are skipped because they trigger internal compiler errors.
@@ -127,5 +136,12 @@ file(GLOB SKIPPED_FILES CONFIGURE_DEPENDS
   section-1p.f90
 )
 
-# There are currently no failing files.
-set(FAILING_FILES "")
+# These tests fail when they are expected to pass.
+file(GLOB FAILING_FILES CONFIGURE_DEPENDS
+  # These files are expected to compile, but fail to do so.
+  c535b-1.f90
+
+  # These files are expected to fail to compile, but succeed instead.
+  c516.f90
+  c524a.f90
+)

Fortran/gfortran/regression/coarray/CMakeLists.txt

@@ -9,5 +9,4 @@
 set(TESTS)
 gfortran_populate_tests(TESTS)
 gfortran_add_execute_tests_from("${TESTS}")
-# TODO: Enable the "compile" tests.
-# gfortran_add_compile_tests_from("${TESTS}")
+gfortran_add_compile_tests_from("${TESTS}")

Fortran/gfortran/regression/coarray/DisabledFiles.cmake

@@ -12,10 +12,12 @@ set(UNSUPPORTED_FILES "")
 # These tests trigger "not yet implemented" assertions in flang.
 file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
   # unimplemented: support for polymorphic types
+  alloc_comp_2.f90
   poly_run_3.f90
 
   # unimplemented: coarray allocation
   allocate_errgmsg.f90
+  array_temporary.f90
   get_array.f90
   lock_2.f90
   move_alloc_1.f90
@@ -28,13 +30,15 @@ file(GLOB UNIMPLEMENTED_FILES CONFIGURE_DEPENDS
 
   # unimplemented: coarray in procedure interface
   dummy_1.f90
+  dummy_2.f90
   poly_run_1.f90
   pr107441-caf.f90
   ptr_comp_1.f08
   ptr_comp_2.f08
   scalar_alloc_1.f90
 
   # unimplemented: intrinsic: '[SYM]'
+  array_temporary-1.f90
   coindexed_1.f90
   collectives_1.f90
   collectives_2.f90
@@ -124,6 +128,24 @@ file(GLOB SKIPPED_FILES CONFIGURE_DEPENDS
   # error: '[SYM]' is a non-ALLOCATABLE coarray and must have an explicit
   # coshape
   poly_run_2.f90
+
+  # --------------------------------------------------------------------------
+  #
+  # These tests are skipped because they are intended to be additional sources
+  # for codimension2.f90. Because of the way the test-suite is setup, it
+  # tries to compile all the files - even those that do not have a do-compile
+  # directive. These files are intended to be compiled together with
+  # codimension2.f90 because they use a module that is created by the former.
+  # If compiled separately, they do not find the module and fail to compile.
+  codimension_2a.f90
+  codimension_2b.f90
+
+  # --------------------------------------------------------------------------
+  #
+  # This is a multi-file test, but cannot find the additional sources
+  # codimension_2a.f90 and codimension_2b.f90. This is almost certainly a bug
+  # in the build files in llvm-test-suite/Fortran/gfortran.
+  codimension_2.f90
 )
 
 # These tests fail at runtime when they should pass.

Fortran/gfortran/regression/debug/CMakeLists.txt

@@ -9,5 +9,4 @@
 set(TESTS)
 gfortran_populate_tests(TESTS)
 gfortran_add_execute_tests_from("${TESTS}")
-# TODO: Enable the "compile" tests.
-# gfortran_add_compile_tests_from("${TESTS}")
+gfortran_add_compile_tests_from("${TESTS}")