Probably final report version

Author: BryanAbate

tastydoc/report/report.pdf

@@ -1,5 +1,31 @@
 \documentclass{report}
 \usepackage{listings}
+\usepackage[dvipsnames]{xcolor}
+
+\lstset{ 
+  backgroundcolor=\color{white},   % choose the background color; you must add \usepackage{color} or \usepackage{xcolor}; should come as last argument
+  basicstyle=\footnotesize,        % the size of the fonts that are used for the code
+  breakatwhitespace=false,         % sets if automatic breaks should only happen at whitespace
+  breaklines=true,                 % sets automatic line breaking
+  captionpos=b,                    % sets the caption-position to bottom
+  commentstyle=\color{green},    % comment style
+  escapeinside={\%*}{*)},          % if you want to add LaTeX within your code
+  extendedchars=true,              % lets you use non-ASCII characters; for 8-bits encodings only, does not work with UTF-8
+  frame=single,	                   % adds a frame around the code
+  keepspaces=true,                 % keeps spaces in text, useful for keeping indentation of code (possibly needs columns=flexible)
+  keywordstyle=\color{blue},       % keyword style
+  language=Scala,                  % the language of the code
+  numbers=none,                    % where to put the line-numbers; possible values are (none, left, right)
+  rulecolor=\color{black},         % if not set, the frame-color may be changed on line-breaks within not-black text (e.g. comments (green here))
+  showspaces=false,                % show spaces everywhere adding particular underscores; it overrides 'showstringspaces'
+  showstringspaces=false,          % underline spaces within strings only
+  showtabs=false,                  % show tabs within strings adding particular underscores
+  stepnumber=2,                    % the step between two line-numbers. If it's 1, each line will be numbered
+  stringstyle=\color{mymauve},     % string literal style
+  tabsize=2,	                   % sets default tabsize to 2 spaces
+  title=\lstname                   % show the filename of files included with \lstinputlisting; also try caption instead of title
+}
+
 \usepackage{hyperref}
 \usepackage{tikz}
 \usetikzlibrary{shapes.geometric, arrows, positioning}
@@ -26,8 +52,10 @@
 
 \chapter{Introduction}
 
+The repository for this project can be found here: \url{https://github.com/BryanAbate/dotty/tree/tastydoc/tastydoc}
+
 A documentation tool is a program generating information (often in the form of HTML files) about projects. Usually, that tool includes method signatures, user-written documentation, links to other pages of the documentation, etc.
-In Dotty, the current tool for generating Scala project documentation is called Dottydoc. The tool introduced here, Tastydoc, aims to improve on Dottydoc and replace it.
+In Dotty, the current tool for generating Scala projects documentation is called Dottydoc. The tool introduced here, Tastydoc, aims to improve on Dottydoc and replace it.
 
 Tastydoc uses TASTy files. These files contain information about the source code of a program. They become output when a program is compiled. Tastydoc consumes such files to extract information about the code, making its use completely independent of the compiler.
 The tool tries to be as close as possible to the structure proposed in Dottydoc so that it can reuse some portions of its code with minor modifications.
@@ -36,7 +64,7 @@ \chapter{Introduction}
 
 The tool was developed with a few goals in mind. First, it should be independent of the compiler, hence the use of TASTy files. It should also produce humanly consumable files, hence the choice of Markdown as an output. Finally, it should have as many features as Dottydoc and be as easily maintainable.
 
-The report is organized in the following structure: First, we will talk about the features of the tool. Then, we will give a high-level and a low-level architectural descriptions. Dottydoc and Tastydoc will then be compared. We will finally talk about the problems encountered during the development and the further work needed.
+The report is organized in the following structure: First, we will talk about the features of the tool. Then, we will give a high-level and a low-level architectural description. Dottydoc and Tastydoc will then be compared. We will finally talk about the problems encountered during the development and the further work needed.
 
 \chapter{Features}
 We will list here the most important features of Tastydoc. For a comparison with Dottydoc features see \autoref{sec:comparison}.
@@ -47,30 +75,15 @@ \chapter{Features}
 For classes, traits, and objects the following are also available (when applicable): members, parents, constructors, known subclasses and companion.
 Packages know all their members as long as all corresponding TASTy files are passed as arguments to the tool.
 
-User-written documentation, including individual access to defined @ such as \texttt{@param} (except \texttt{@usecase}). Two syntaxes are supported, which are the old Scaladoc Wiki-style syntax as well as Dottydoc Markdown syntax. Linking to entities (classes, methods, etc.) is possible inside the user documentation in the Wiki-style syntax using the notation like \texttt{[[scala.foo.bar]]}.
+User-written documentation, including individual access to defined @ such as \texttt{@param} (except \texttt{@usecase}) is available. Two syntaxes are supported, which are the old Scaladoc Wiki-style syntax as well as Dottydoc Markdown syntax. Linking to entities (classes, methods, etc.) is possible inside the user documentation in the Wiki-style syntax using the notation like \texttt{[[scala.foo.bar]]}.
 
 \paragraph{TASTy}
 The tool relies on consuming TASTy files and is hence independent from the compiler.
 
 \paragraph{Linking}
-Tastydoc is able to produce links to types defined outside of the current TASTy files (as well as the one inside the file). This is particularly useful for linking to the types of the parameters and return types, companions, scope modifiers, and parents.
-
-\paragraph{Output}
-The tool produces Markdown documentation files that are easy to modify and read.
-
-\chapter{Architecture}
-\section{High-level architectural descriptions}
-\subsection{TASTy}
-TASTy is a data format aimed at serializing compiler information about code. Dotty produces these files for each class, object without companion class or trait, trait and package if it contains something else than classes, traits or objects. Given its nature, TASTy is perfect to extract information.
-This is done through the use of the reflect API and the making of a class extending \texttt{TastyConsumer}\footnote{\texttt{scala/tasty/file/TastyConsumer.scala}}.
-
-\subsection{High-level API}
-The tool offers an interface that is easy to use (called Representation, see \autoref{sec:representation}) to access all the information about an entity in a comprehensible format for anyone willing to write their own documentation tool without worrying to consume TASTy files.
-
-\subsection{Reuse of Dottydoc intermediate representation}
-The intermediate structure (Representation) is similar to Dottydoc \texttt{Entity}\footnote{\texttt{dotty/tools/dottydoc/Entity.scala}} allowing to easily reuse parts of Dottydoc code with only minor modifications. Right now, the tool uses parts of Dottydoc code for parsing user documentation. However, it could produce the same HTML files as Dottydoc without relying on compiler internals. Especially since Dottydoc already has a complex templating mechanism that allows to attach blog posts to documentation and a search engine.
+Tastydoc is able to produce links to types defined outside of the current TASTy files (as well as the one inside the file). This is particularly useful for linking to the types of the parameters and the return types, companions, scope modifiers, and parents.
 
-\subsection{Markdown}
+\paragraph{Markdown}
 Outputting Markdown instead of HTML has several advantages. It is easy to edit by hand, easily previewed, and users can easily add their own texts or files to it. A user could, for example, write a Markdown file and link it in the documentation of a class or do the opposite and link a documentation page in a Markdown file.
 
 Markdown files can easily be uploaded for preview to GitHub or GitLab. In that way, it is very easy to upload documentation files along with the code of a project.
@@ -79,8 +92,8 @@ \subsection{Markdown}
 \begin{center}
     \begin{tikzpicture}[node distance=2cm]
         \node (write) [process] {User writes code and documentation};
-        \node (compile) [process, below of=write] {Compiler produces Tasty files};
-        \node (tool) [process, below of=compile] {Tastydoc consumes Tasty files and produces Markdown};
+        \node (compile) [process, below of=write] {Compiler produces TASTy files};
+        \node (tool) [process, below of=compile] {Tastydoc consumes TASTy files and produces Markdown};
         \node (user) [process, below of=tool] {User edits Markdown files or add files as he wishes};
         \node (HTML) [process, below of=user, right of=user, xshift=1.2cm] {Markdown files are consumed and transformed to another format(HTML, PDF, etc.)};
         \node (upload) [process, below of=user, left of=user, xshift=-1.2cm] {Markdown files are uploaded to a git hosting service or a website};
@@ -93,6 +106,18 @@ \subsection{Markdown}
     \end{tikzpicture}
 \end{center}
 
+\chapter{Architecture}
+\section{High-level architectural descriptions}
+\subsection{TASTy}
+TASTy is a data format aimed at serializing compiler information about code. Dotty produces these files for each class, object without companion class or trait, trait and package if it contains something else than classes, traits or objects. Given its nature, TASTy is perfect to extract information.
+This is done through the use of the reflect API and the making of a class extending \texttt{TastyConsumer}\footnote{\texttt{scala/tasty/file/TastyConsumer.scala}}.
+
+\subsection{High-level API}
+The tool offers an interface that is easy to use (called Representation, see \autoref{sec:representation}) to access all the information about an entity in a comprehensible format for anyone willing to write their own documentation tool without worrying to consume TASTy files.
+
+\subsection{Reuse of Dottydoc intermediate representation}
+The intermediate structure (Representation) is similar to Dottydoc \texttt{Entity}\footnote{\texttt{dotty/tools/dottydoc/Entity.scala}} allowing to easily reuse parts of Dottydoc code with only minor modifications. Right now, the tool uses parts of Dottydoc code for parsing user documentation. However, it could produce the same HTML files as Dottydoc without relying on compiler internals. Especially since Dottydoc already has a complex templating mechanism that allows to attach blog posts to documentation and a search engine.
+
 \section{Low-level architecture}
 \subsection{TastyExtractor}
 File: \texttt{dotty/tastydoc/TastyExtractor.scala}
@@ -159,18 +184,18 @@ \subsection{Representation}
 extends Representation with Modifiers
 with TypeParams 
 \end{lstlisting}
-    \item \texttt{EmulatedPackageRepresentation} This Representation allow to regroup every PackageRepresentation of the same package under the same Representation. In fact, Tasty will output a Tasty file for each entity in a package and the top of the \texttt{reflect.Tree} is a \texttt{reflect.PackageClause}. Thus, there will be multiple \texttt{PackageRepresentation} of the same package when converting multiple classes of the same package. That latter point can be problematic in regard to usability. This Representation counters that issue. When using an \texttt{EmulatedPackageRepresentation}, it looks to the user as if he were using a \texttt{PackageRepresentation} containing all the members of the package.
+    \item \texttt{EmulatedPackageRepresentation} This Representation allows to regroup every PackageRepresentation of the same package under the same Representation. In fact, Tasty will output a Tasty file for each entity in a package and the top of the \texttt{reflect.Tree} is a \texttt{reflect.PackageClause}. Thus, there will be multiple \texttt{PackageRepresentation} of the same package when converting multiple classes of the same package. That latter point can be problematic in regard to usability. This Representation counters that issue. When using an \texttt{EmulatedPackageRepresentation}, it looks to the user as if he were using a \texttt{PackageRepresentation} containing all the members of the package.
 \end{itemize}
 
 \subsection{DocPrinter}
 File: \texttt{dotty/tastydoc/DocPrinter.scala}
 
-Object with methods that formats Representations and References to Markdown and writes them to files. Basically, it handles all the formatting and printing logic of the tool.
+Object with methods that formats Representations and References to Markdown and writes them to files. Basically, it handles all the formatting and writting logic of the tool.
 
 \subsection{mdscala}
 File: \texttt{dotty/tastydoc/mdscala.scala}
 
-Object that has helper methods for outputting markdown. Unfortunately, it does not handle escaping as it is a non-trivial task and would require a significant amount of time (see \autoref{sec:problems}).
+Object that has helper methods for outputting Markdown. Unfortunately, it does not handle escaping as it is a non-trivial task and would require a significant amount of time (see \autoref{sec:problems}).
 
 \subsection{TastyDocConsumer}
 File: \texttt{dotty/tastydoc/TastyDocConsumer.scala}
@@ -184,7 +209,7 @@ \subsection{Main}
 Command line parameters are:
 \begin{itemize}
     \item \textbf{--syntax} \{\textit{wiki} or \textit{markdown}\} Syntax to use for user documentation parsing
-    \item \textbf{--packagestolink} \{\textit{regex1 regex2 ...}\} Regexes to specify which packages should be linked when formatting References
+    \item \textbf{--packagestolink} \{\textit{regex1 regex2 ...}\} Regexes of packages or entities (example: \texttt{scala.collection.*}). Only the types with a path matching these regexes will produce links in the documentation files
     \item \textbf{--classpath} \{\textit{URI}\} Extra classpath for input files
     \item \textbf{-i} \{\textit{file1 file2 ...}\} TASTy files
     \item \textbf{-d} \{\textit{dir1 dir2 ...}\} Directories to recursively find TASTy files
@@ -212,7 +237,7 @@ \section{Workflow}
     \end{tikzpicture}
 \end{center}
 
-As we can see, parsing user documentation is only done after converting to Representation. Indeed, in order to link methods, classes, etc. from user documentation we need to have first converted everything to Representation. Otherwise, there is no way to know what is linked (a method? a class? an object?).
+As we can see, parsing user documentation is only done after converting to Representation. Indeed, in order to link methods, classes, etc. from user documentation we need to have first converted everything to Representation because the Wiki-style syntax does not tell us what is linked (a method? a class? an object?).
 
 
 \chapter{Comparison between Dottydoc and Tastydoc}
@@ -284,7 +309,7 @@ \chapter{Problems}
 
 To mitigate this problem, we first output types and then values so that when linking it links to the first one appearing in the page which would be a type and it is what we want in most cases. However, this does not solve the problem of linking to type parameters and of linking to overloaded methods.
 
-\chapter{Remaining work}
+\chapter{Further work}
 Although the project is nearly complete, some work needs to be done to perfect the tool.
 
 \paragraph{Markdown escaping}
@@ -294,7 +319,7 @@ \chapter{Remaining work}
 Type Lambdas are complex types to handle and convert to References, right now they are converted to constant References. This is not the expected behaviour: the output is not the one we expect and no linking can be done this way. It is, however, acceptable for a first version as type lambdas do not appear that often.
 
 \paragraph{Complex types}
-Assume the following type:
+Assume the following code:
 \begin{lstlisting}[language=scala]
     class Graph {
         type Node = Int