Relaced UnknownDetails with MiscDetails

Author: NimishMishra

clang-tools-extra/docs/doxygen.cfg.in

@@ -1230,7 +1230,7 @@ CHM_FILE               =
 HHC_LOCATION           =
 
 # The GENERATE_CHI flag controls if a separate .chi index file is generated (
-# YES) or that it should be included in the master .chm file ( NO).
+# YES) or that it should be included in the main .chm file ( NO).
 # The default value is: NO.
 # This tag requires that the tag GENERATE_HTMLHELP is set to YES.
 

clang/docs/doxygen.cfg.in

@@ -1219,7 +1219,7 @@ CHM_FILE               =
 HHC_LOCATION           =
 
 # The GENERATE_CHI flag controls if a separate .chi index file is generated (
-# YES) or that it should be included in the master .chm file ( NO).
+# YES) or that it should be included in the main .chm file ( NO).
 # The default value is: NO.
 # This tag requires that the tag GENERATE_HTMLHELP is set to YES.
 

flang/docs/FlangDriver.md

@@ -13,24 +13,107 @@
    :local:
 ```
 
+
+> **_NOTE:_** This document assumes that Flang's drivers can already generate code and
+> produce executables. However, this is still work-in-progress. By making this
+> assumption, we are able to prepare this document ahead-of-time and to provide
+> an overview of the design that we are working towards.
+
 There are two main drivers in Flang:
 * the compiler driver, `flang-new`
 * the frontend driver, `flang-new -fc1`
 
-The compiler driver will allow you to control all compilation phases (i.e.
-preprocessing, frontend code-generation, middlend/backend code-optimisation and
-lowering, linking). For frontend specific tasks, the compiler driver creates a
-Fortran compilation job and delegates it to `flang-new -fc1`, the frontend driver.
+> **_NOTE:_** The diagrams in this document refer to `flang` as opposed to
+> `flang-new`. This is because the diagrams reflect the final design that we
+> are still working towards. See the note on [the flang script](https://github.com/llvm/llvm-project/blob/main/flang/docs/FlangDriver.md#the-flang-script)
+> below for more context.
+
+The **compiler driver** will allow you to control all compilation phases (e.g.
+preprocessing, semantic checks, code-generation, code-optimisation, lowering
+and linking). For frontend specific tasks, the compiler driver creates a
+Fortran compilation job and delegates it to `flang-new -fc1`, the frontend
+driver. For linking, it creates a linker job and calls an external linker (e.g.
+LLVM's [`lld`](https://lld.llvm.org/)). It can also call other tools such as
+external assemblers (e.g. [`as`](https://www.gnu.org/software/binutils/)). In
+Clang, the compiler driver can also link the generated binaries with LLVM's
+static analysis/sanitizer libraries (e.g.
+[MemorySanitizer](https://clang.llvm.org/docs/MemorySanitizer.html)). This is
+not yet available in Flang, but will be relatively easy to support once such
+libraries become available. Flang's compiler driver is intended for Flang's
+end-users - its interface needs to remain stable. Otherwise, Flang's users will
+have to adjust their build scripts every time a compiler flag is changed.
+
+| ![Compiler Driver](compiler_driver.png) |
+|:--:|
+| *Flang’s compiler driver and the **tools** that it runs* |
+
+The **frontend driver** glues together and drives all of the Flang's frontend
+libraries. As such, it provides an easy-to-use and intuitive interface to the
+frontend. It uses MLIR and LLVM for code-generation and can be viewed as a
+driver for Flang, LLVM and MLIR libraries. Contrary to the compiler driver, it
+is not capable of calling any external tools (including linkers).  It is aware
+of all the frontend internals that are "hidden" from the compiler driver. It
+accepts many frontend-specific options not available in `flang-new` and as such
+it provides a finer control over the frontend. Note that this tool is mostly
+intended for Flang developers. In particular, there are no guarantees about the
+stability of its interface and compiler developers can use it to experiment
+with new flags.
+
+| ![Frontend Driver](frontend_driver.png) |
+|:-:|
+| *Flang's frontend driver and the **libraries** that it drives* |
+
+Note that similarly to `-Xclang` in `clang`, you can use `-Xflang` to forward a
+frontend specific flag from the _compiler_ directly to the _frontend_ driver,
+e.g.:
+
+```lang=bash
+flang-new -Xflang -fdebug-dump-parse-tree input.f95
+```
 
-The frontend driver glues all of the frontend libraries together and provides
-an easy-to-use and intuitive interface to the frontend. It accepts many
-frontend-specific options not available in `flang-new` and as such it provides a
-finer control over the frontend. Similarly to `-Xclang` in `clang`, you can use
-`-Xflang` to forward the frontend specific flags from the compiler directly to
-the frontend driver.
+In the invocation above, `-fdebug-dump-parse-tree` is forwarded to `flang-new
+-fc1`. Without the forwarding flag, `-Xflang`, you would see the following
+warning:
 
-## Compiler Driver
+```lang=bash
+flang-new: warning: argument unused during compilation:
+```
+
+As `-fdebug-dump-parse-tree` is only supported by `flang-new -fc1`, `flang-new`
+will ignore it when used without `Xflang`.
+
+## Why Do We Need Two Drivers?
+As hinted above, `flang-new` and `flang-new -fc1` are two separate tools. The
+fact that these tools are accessed through one binary, `flang-new`, is just an
+implementation detail. Each tool has a separate list of options, albeit defined
+in the same file: `clang/include/clang/Driver/Options.td`.
+
+The separation helps us split various tasks and allows us to implement more
+specialised tools. In particular, `flang-new` is not aware of various
+compilation phases within the frontend (e.g. scanning, parsing or semantic
+checks). It does not have to be. Conversely, the frontend driver, `flang-new
+-fc1`, needs not to be concerned with linkers or other external tools like
+assemblers. Nor does it need to know where to look for various systems
+libraries, which is usually OS and platform specific.
+
+One helpful way of differentiating these tools is to keep in mind that:
+
+* the compiler driver is an end-user tool
+* frontend driver is a compiler developer tool with many additional options,
+
+Also, Since the compiler driver can call external tools, e.g. linkers, it can
+be used to generate **executables**. The frontend driver cannot call external
+tools and hence can only generate **object files**. A similar model is
+implemented in Clang (`clang` vs `clang -cc1` vs `clang -cc1as`), which is
+based on the [architecture of
+GCC](https://en.wikibooks.org/wiki/GNU_C_Compiler_Internals/GNU_C_Compiler_Architecture).
+In fact, Flang needs to adhere to this model in order to be able to re-use
+Clang's driver library. If you are more familiar with the [architecture of
+GFortran](https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gfortran/About-GNU-Fortran.html)
+than Clang, then `flang-new` corresponds to `gfortran` and `flang-new -fc1` to
+`f951`.
 
+## Compiler Driver
 The main entry point for Flang's compiler driver is implemented in
 `flang/tools/flang-driver/driver.cpp`.  Flang's compiler driver is implemented
 in terms of Clang's driver library, `clangDriver`. This approach allows us to:
@@ -92,9 +175,9 @@ You can read more on the design of `clangDriver` in Clang's [Driver Design &
 Internals](https://clang.llvm.org/docs/DriverInternals.html).
 
 ## Frontend Driver
-Flang's frontend driver is the main interface between end-users and the Flang
-frontend. The high-level design is similar to Clang's frontend driver, `clang
--cc1` and consists of the following classes:
+Flang's frontend driver is the main interface between compiler developers and
+the Flang frontend. The high-level design is similar to Clang's frontend
+driver, `clang -cc1` and consists of the following classes:
 * `CompilerInstance`, which is a helper class that encapsulates and manages
   various objects that are always required by the frontend (e.g. `AllSources`,
   `AllCookedSources, `Parsing`, `CompilerInvocation`, etc.). In most cases

flang/docs/compiler_driver.png

@@ -1222,7 +1222,7 @@ CHM_FILE               =
 HHC_LOCATION           =
 
 # The GENERATE_CHI flag controls if a separate .chi index file is generated (
-# YES) or that it should be included in the master .chm file ( NO).
+# YES) or that it should be included in the main .chm file ( NO).
 # The default value is: NO.
 # This tag requires that the tag GENERATE_HTMLHELP is set to YES.
 

flang/docs/doxygen.cfg.in

@@ -156,11 +156,6 @@ bool OpenMPCounterVisitor::Pre(const OpenMPConstruct &c) {
   ompWrapperStack.push_back(ow);
   return true;
 }
-bool OpenMPCounterVisitor::Pre(const OmpEndLoopDirective &c) { return true; }
-bool OpenMPCounterVisitor::Pre(const DoConstruct &) {
-  loopLogRecordStack.push_back(curLoopLogRecord);
-  return true;
-}
 
 void OpenMPCounterVisitor::Post(const OpenMPDeclarativeConstruct &) {
   PostConstructsCommon();
@@ -178,27 +173,11 @@ void OpenMPCounterVisitor::PostConstructsCommon() {
       clauseStrings[curConstruct]};
   constructClauses.push_back(r);
 
-  // Keep track of loop log records if it can potentially have the
-  // nowait clause added on later.
-  if (const auto *oc = std::get_if<const OpenMPConstruct *>(curConstruct)) {
-    if (const auto *olc = std::get_if<OpenMPLoopConstruct>(&(*oc)->u)) {
-      const auto &beginLoopDir{
-          std::get<Fortran::parser::OmpBeginLoopDirective>(olc->t)};
-      const auto &beginDir{
-          std::get<Fortran::parser::OmpLoopDirective>(beginLoopDir.t)};
-      if (beginDir.v == llvm::omp::Directive::OMPD_do ||
-          beginDir.v == llvm::omp::Directive::OMPD_do_simd) {
-        curLoopLogRecord = &constructClauses.back();
-      }
-    }
-  }
-
   auto it = clauseStrings.find(curConstruct);
   clauseStrings.erase(it);
   ompWrapperStack.pop_back();
   delete curConstruct;
 }
-void OpenMPCounterVisitor::Post(const OmpEndLoopDirective &c) {}
 
 void OpenMPCounterVisitor::Post(const OmpProcBindClause::Type &c) {
   clauseDetails += "type=" + OmpProcBindClause::EnumToString(c) + ";";
@@ -242,26 +221,9 @@ void OpenMPCounterVisitor::Post(const OmpClause &c) {
   clauseDetails.clear();
 }
 void OpenMPCounterVisitor::PostClauseCommon(const ClauseInfo &ci) {
-  // The end loop construct (!$omp end do) can contain a nowait clause.
-  // The flang parser does not parse the end loop construct as part of
-  // the OpenMP construct for the loop construct. So the end loop is left
-  // hanging as a separate executable construct. If a nowait clause is seen in
-  // an end loop construct we have to find the associated loop construct and
-  // add nowait to its list of clauses. Note: This is not a bug in flang, the
-  // parse tree is corrected during semantic analysis.
-  if (ci.clause == "nowait") {
-    assert(curLoopLogRecord &&
-        "loop Construct should be visited before a nowait clause");
-    curLoopLogRecord->clauses.push_back(ci);
-  } else {
-    assert(!ompWrapperStack.empty() &&
-        "Construct should be visited before clause");
-    clauseStrings[ompWrapperStack.back()].push_back(ci);
-  }
-}
-void OpenMPCounterVisitor::Post(const DoConstruct &) {
-  curLoopLogRecord = loopLogRecordStack.back();
-  loopLogRecordStack.pop_back();
+  assert(
+      !ompWrapperStack.empty() && "Construct should be visited before clause");
+  clauseStrings[ompWrapperStack.back()].push_back(ci);
 }
 } // namespace parser
 } // namespace Fortran

flang/docs/frontend_driver.png

@@ -17,7 +17,6 @@
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
 
-#include <deque>
 #include <string>
 
 namespace Fortran {
@@ -62,13 +61,10 @@ struct OpenMPCounterVisitor {
   template <typename A> void Post(const A &) {}
   bool Pre(const OpenMPDeclarativeConstruct &c);
   bool Pre(const OpenMPConstruct &c);
-  bool Pre(const OmpEndLoopDirective &c);
-  bool Pre(const DoConstruct &);
 
   void Post(const OpenMPDeclarativeConstruct &);
   void Post(const OpenMPConstruct &);
   void PostConstructsCommon();
-  void Post(const OmpEndLoopDirective &c);
 
   void Post(const OmpProcBindClause::Type &c);
   void Post(const OmpDefaultClause::Type &c);
@@ -83,20 +79,9 @@ struct OpenMPCounterVisitor {
   void Post(const OmpCancelType::Type &c);
   void Post(const OmpClause &c);
   void PostClauseCommon(const ClauseInfo &ci);
-  void Post(const DoConstruct &);
 
   std::string clauseDetails{""};
-
-  // curLoopLogRecord and loopLogRecordStack store
-  // pointers to this datastructure's entries. Hence a
-  // vector cannot be used since pointers are invalidated
-  // on resize. Next best option seems to be deque. Also a
-  // list cannot be used since YAML gen requires a
-  // datastructure which can be accessed through indices.
-  std::deque<LogRecord> constructClauses;
-
-  LogRecord *curLoopLogRecord{nullptr};
-  llvm::SmallVector<LogRecord *> loopLogRecordStack;
+  llvm::SmallVector<LogRecord> constructClauses;
   llvm::SmallVector<OmpWrapperType *> ompWrapperStack;
   llvm::DenseMap<OmpWrapperType *, llvm::SmallVector<ClauseInfo>> clauseStrings;
   Parsing *parsing{nullptr};

flang/examples/flang-omp-report-plugin/flang-omp-report-visitor.cpp

@@ -33,10 +33,6 @@ namespace llvm {
 namespace yaml {
 using llvm::yaml::IO;
 using llvm::yaml::MappingTraits;
-template <typename T>
-struct SequenceTraits<std::deque<T>,
-    std::enable_if_t<CheckIsBool<SequenceElementTraits<T>::flow>::value>>
-    : SequenceTraitsImpl<std::deque<T>, SequenceElementTraits<T>::flow> {};
 template <> struct MappingTraits<ClauseInfo> {
   static void mapping(IO &io, ClauseInfo &info) {
     io.mapRequired("clause", info.clause);

flang/examples/flang-omp-report-plugin/flang-omp-report-visitor.h

@@ -0,0 +1,2 @@
+ruamel.yaml==0.17.16
+ruamel.yaml.clib==0.2.6