[docs] Overhaul the documentation (#136)

There were multiple mentions of how to run a server, all of them
slightly different. This reorganizes the documentation to have a
single place explaining how to run a server, either locally or
in production via Docker.

This also moves the documentation of the deprecated `nts` test
suite to a less prominent location -- we should actually see if
that can be removed entirely.

Finally, it updates the Quickstart section of "Running tests" to
ensure that the commands actually run on anyone's machine. A
quickstart should not require custom setup.

Author: ldionne

deployment/app_wrapper.py

@@ -3,13 +3,12 @@
 Concepts
 ========
 
-LNT's data model is pretty simple, and just following the :ref:`quickstart` can
-get you going with performance testing. Moving beyond that, it is useful to have
-an understanding of some of the core concepts in LNT. This can help you get the
-most out of LNT.
+LNT's data model is pretty simple and can be grasped intuitively. However, for
+more advanced usage, it is useful to have an understanding of some of the core
+concepts in LNT. This can help you get the most out of LNT.
 
-Orders Machines and Tests
--------------------------
+Orders, Machines and Tests
+--------------------------
 
 LNT's data model was designed to track the performance of a system in many
 configurations over its evolution.  In LNT, an Order is the x-axis of your
@@ -19,15 +18,15 @@ also be used to represent treatments, such as a/b.  You can put anything you
 want into LNT as an order, as long as it can be sorted by Python's sort
 function.
 
-A Machine in LNT is the logical bucket which results are categorized by. 
+A Machine in LNT is the logical bucket which results are categorized by.
 Comparing results from the same machine is easy, across machines is harder.
 Sometimes machine can literally be a machine, but more abstractly, it can be any
 configuration you are interested in tracking. For example, to store results
-from an Arm test machine, you could have a machine call "ArmMachine"; but, you 
+from an Arm test machine, you could have a machine called "ArmMachine"; but, you
 may want to break machines up further for example "ArmMachine-Release"
 "ArmMachine-Debug", when you compile the thing you want to test in two modes.
 When doing testing of LLVM, we often string all the useful parameters of the
-configuration into one machines name:: 
+configuration into the name of the machine::
 
     <hardware>-<arch>-<optimization level>-<branch-name>
 
@@ -37,7 +36,7 @@ Runs and Samples
 ----------------
 
 Samples are the actual data points LNT collects. Samples have a value, and
-belong to a metric, for example a 4.00 second (value) compile time (metric).  
+belong to a metric, for example a 4.00 second (value) compile time (metric).
 Runs are the unit in which data is submitted.  A Run represents one run through
 a set of tests.  A run has a Order which it was run
 on, a Machine it ran on, and a set of Tests that were run, and for each Test
@@ -54,7 +53,7 @@ Test Suites
 LNT uses the idea of a Test Suite to control what metrics are collected.  Simply,
 the test suite acts as a definition of the data that should be stored about
 the tests that are being run.  LNT currently comes with two default test suites.
-The Nightly Test Suite (NTS) (which is run far more often than nightly now), 
+The Nightly Test Suite (NTS) (which is run far more often than nightly now),
 collects 6 metrics per test: compile time, compile status, execution time, execution
 status, score and size.  The Compile (compile) Test Suite, is focused on metrics
 for compile quality: wall, system and user compile time, compile memory usage

docs/concepts.rst

@@ -8,22 +8,22 @@ Contents
 
    intro
 
-   quickstart
+   tests
 
-   tools
+   running_server
 
-   tests
+   importing_data
+
+   tools
 
    concepts
 
    api
 
-   importing_data
+   profiles
 
    developer_guide
 
-   profiles
-   
 Indices and tables
 ==================
 
@@ -39,4 +39,3 @@ Module Listing
    :maxdepth: 2
 
    modules/testing
-   

docs/contents.rst

@@ -6,20 +6,20 @@ Importing Data
 Importing Data in a Text File
 -----------------------------
 
-The LNT importreport command will import data in a simple text file format. The
+The ``lnt importreport`` command will import data in a simple text file format. The
 command takes a space separated key value file and creates an LNT report file,
 which can be submitted to a LNT server.  Example input file::
 
-    foo.exec 123
+    foo.execution_time 123
     bar.size 456
     foo/bar/baz.size 789
 
-The format is "test-name.metric", so exec and size are valid metrics for the
-test suite you are submitting to.
+The format is ``test-name.metric value``, so ``execution_time`` and ``size`` must be valid
+metrics for the test suite you are submitting to.
 
 Example::
 
-    echo -n "foo.exec 25\nbar.score 24.2\nbar/baz.size 110.0\n" > results.txt
+    echo -n "foo.execution_time 25\nbar.score 24.2\nbar/baz.size 110.0\n" > results.txt
     lnt importreport --machine=my-machine-name --order=1234 --testsuite=nts results.txt report.json
     lnt submit http://mylnt.com/db_default/submitRun report.json
 
@@ -28,7 +28,7 @@ Example::
 LNT Report File Format
 ----------------------
 
-The lnt importreport tool is an easy way to import data into LNTs test format.
+The ``lnt importreport`` tool is an easy way to import data into LNT's test format.
 You can also create LNTs report data directly for additional flexibility.
 
 First, make sure you've understood the underlying :ref:`concepts` used by LNT.

docs/importing_data.rst

@@ -8,116 +8,38 @@ of two main parts, a web application for accessing and visualizing performance
 data, and command line utilities to allow users to generate and submit test
 results to the server.
 
-The package was originally written for use in testing LLVM compiler
-technologies, but is designed to be usable for the performance testing of any
-software.
-
-If you are an LLVM developer who is mostly interested in just using LNT to run
-the test-suite against some compiler, then you should fast forward to the
-:ref:`quickstart` or to the information on :ref:`tests`.
-
-LNT uses a simple and extensible format for interchanging data between the test
-producers and the server; this allows the LNT server to receive and store data
-for a wide variety of applications.
+The package was originally written for use in testing LLVM compiler technologies,
+but is designed to be usable for the performance testing of any software. LNT uses
+a simple and extensible format for interchanging data between the test producers and
+the server; this allows the LNT server to receive and store data for a wide variety
+of applications.
 
 Both the LNT client and server are written in Python, however the test data
 itself can be passed in one of several formats, including property lists and
 JSON. This makes it easy to produce test results from almost any language.
 
+.. _installation:
 
 Installation
 ------------
 
-If you are only interested in using LNT to run tests locally, see the
-:ref:`quickstart`.
-
-If you want to run an LNT server, you will need to perform the following
-additional steps:
-
- 2. Create a new LNT installation::
-
-      lnt create path/to/install-dir
-
-    This will create the LNT configuration file, the default database, and a
-    .wsgi wrapper to create the application. You can execute the generated app
-    directly to run with the builtin web server, or use::
-
-      lnt runserver path/to/install-dir
-
-    which provides additional command line options. Neither of these servers is
-    recommended for production use.
-
- 3. Edit the generated 'lnt.cfg' file if necessary, for example to:
-
-    a. Update the databases list.
-
-    b. Update the public URL the server is visible at.
-
-    c. Update the nt_emailer configuration.
-
- 4. Add the 'lnt.wsgi' app to your Apache configuration. You should set also
-    configure the WSGIDaemonProcess and WSGIProcessGroup variables if not
-    already done.
-
-    If running in a virtualenv you will need to configure that as well; see the
-    `modwsgi wiki <http://code.google.com/p/modwsgi/wiki/VirtualEnvironments>`_.
-
-For production servers, you should consider using a full DBMS like PostgreSQL.
-To create an LNT instance with PostgreSQL backend, you need to do this instead:
+You can install the latest stable release of LNT from PyPI. We recommend doing
+that from a virtual environment::
 
- 1. Create an LNT database in PostgreSQL, also make sure the user has
-    write permission to the database::
+   python3 -m venv .venv
+   source .venv/bin/activate
+   pip install llvm-lnt
 
-      CREATE DATABASE "lnt.db"
+This will install the client-side tools. If you want to run a production server,
+you should instead install ``llvm-lnt`` while including the server-side optional
+requirements::
 
- 2. Then create LNT installation::
+   pip install "llvm-lnt[server]"
 
-      lnt create path/to/install-dir --db-dir postgresql://user@host
+That's it! ``lnt`` should now be accessible from the virtual environment.
 
- 3. Run server normally::
-
-      lnt runserver path/to/install-dir
-
-Architecture
-------------
-
-The LNT web app is currently implemented as a Flask WSGI web app, with Jinja2
-for the templating engine. My hope is to eventually move to a more AJAXy web
-interface.
-
-The database layer uses SQLAlchemy for its ORM, and is typically backed by
-SQLite, although I have tested on MySQL on the past, and supporting other
-databases should be trivial. My plan is to always support SQLite as this allows
-the possibility of developers easily running their own LNT installation for
-viewing nightly test results, and to run with whatever DB makes the most sense
-on the server.
-
-Running a LNT Server Locally
-----------------------------
-
-LNT can accommodate many more users in the production config.  In production:
-- Postgres or MySQL should be used as the database.
-- A proper wsgi server should be used, in front of a proxy like Nginx or Apache.
-
-To install the extra packages for the server config::
-
-    virtualenv venv
-    . ./venv/bin/activate
-    pip install -r requirements.server.txt
-    lnt create path/to/data_dir --db-dir postgresql://user@host  # data_dir path will be where lnt data will go.
-    cd deployment
-    # Now edit app_wrapper.py to have your path/to/data_dir path and the log-file below.
-    gunicorn app_wrapper:app --bind 0.0.0.0:8000 --workers 8 --timeout 300 --name lnt_server --log-file /var/log/lnt/lnt.log --access-logfile /var/log/lnt/gunicorn_access.log --max-requests 250000
-
-
-Running a LNT Server via Docker
--------------------------------
-
-We provide a Docker Compose setup with Docker containers that can be used to
-easily bring up a fully working production server within minutes. The container
-can be built and run with::
-
-   docker compose --file docker/compose.yaml --env-file <secrets> up
-
-``<secrets>`` should be the path to a file containing environment variables
-required by the containers. Please refer to the Docker Compose file for details.
+If you are an LLVM developer who is mostly interested in just using LNT to run
+the test-suite against some compiler, then you should fast forward to the section
+on :ref:`running tests <tests>`. If you want to run your own LNT server, jump to
+the section on :ref:`running a server <running_server>`. Otherwise, jump to the
+:ref:`table of contents <contents>` to get started.