refactor: merge cli (#65)

* refactor: add cli

* fix: app not launching

* refactor: `app` command -> `gui`

* chore: remove redundant text

* feat: remove launch app if else

* feat: update launch app pattern with public

* docs: update documentation with new CLI tool usage

---------

Co-authored-by: Christoph Brunken <[email protected]>

Author: lwalew

README.md

@@ -47,28 +47,36 @@ can be found [here](https://instadeep.com/).
 
 ## 🚀 Usage
 
-MLIPAudit can be used as two separate CLI tools, the benchmarking script and
-a UI app for visualization of results. Furthermore, for advanced users that want to add
-their own benchmarks or create their own app with our existing benchmark classes, we
-also offer to use MLIPAudit as a library.
+MLIPAudit can be used via its CLI tool `mlipaudit`, which can carry out two main tasks:
+the benchmarking task and a graphical UI app for visualization of results. Furthermore,
+for advanced users that want to add their own benchmarks or create their own app with
+our existing benchmark classes, we also offer to use MLIPAudit as a library.
 
-### CLI Tools
-
-After installation via pip, the `mlipaudit` command line tool is available. It executes
-a benchmark run and can be configured via some command line arguments. Run the following
-to obtain an overview of these configuration options:
+After installation via pip, the `mlipaudit` command is available in your terminal.
+Run the following to obtain an overview of two main tasks, `benchmark` and `gui`:
 
 ```bash
 mlipaudit -h
 ```
 
-The `-h` flag prints the help message of the script with the info on how to use it.
+The `-h` flag prints the help message with the info on how to use the tool.
+See below, for details on the two available tasks.
+
+### The benchmarking task
+
+The first task is `benchmark`. It executes a benchmark run and can be configured
+via some command line arguments. To print the help message for this specific task,
+run:
+
+```bash
+mlipaudit benchmark -h
+```
 
 For example, to launch a full benchmark for a model located at `/path/to/model.zip`,
 you can run:
 
 ```bash
-mlipaudit -m /path/to/model.zip -o /path/to/output
+mlipaudit benchmark -m /path/to/model.zip -o /path/to/output
 ```
 
 In this case, benchmark results are written to the directory `/path/to/output`. In this
@@ -81,10 +89,19 @@ For a tutorial on how to run models that are not native to the
 [mlip](https://github.com/instadeepai/mlip) library, see
 [this](https://instadeep.com/) section of our documentation.
 
-To visualize the detailed results (potentially of multiple models), run:
+### The graphical user interface
+
+To visualize the detailed results (potentially of multiple models), the `gui` task can
+be run. To get more information, run:
+
+```bash
+mlipaudit gui -h
+```
+
+For example, to display the results stored at `/path/to/output`, execute:
 
 ```bash
-mlipauditapp /path/to/output
+mlipaudit gui /path/to/output
 ```
 
 This should automatically open a webpage in your browser with a graphical user interface
@@ -111,7 +128,7 @@ documentation for details on the available functions.
 You can use these functions to build your own benchmarking script and GUI pages for our
 app. For inspiration, we recommend to take a look at the main scripts for
 these tools in this repo, located at `src/mlipaudit/main.py` and
-`src/mlipaudit/app.py`, respectively.`
+`src/mlipaudit/app.py`, respectively.
 
 ## 🤗 Data
 

docs/source/index.rst

@@ -19,7 +19,7 @@ line. For example,
 
 .. code-block:: bash
 
-    mlipaudit -m /path/to/visnet.zip /path/to/mace.zip -o /path/to/output
+    mlipaudit benchmark -m /path/to/visnet.zip /path/to/mace.zip -o /path/to/output
 
 runs the complete benchmark suite for two models, `visnet` and `mace` and
 stores the results in JSON files in the `/path/to/output` directory. **The results**
@@ -31,7 +31,7 @@ To visualize these results, we provide a graphical user interface based on
 
 .. code-block:: bash
 
-    mlipauditapp /path/to/output
+    mlipaudit gui /path/to/output
 
 to launch the app (opens a browser window automatically and displays the UI).
 

docs/source/tutorials/cli/index.rst

@@ -4,19 +4,25 @@ Tutorial: CLI tools
 ===================
 
 After installation and activating the respective Python environment, the command line
-tools `mlipaudit` and `mlipauditapp` should be available:
+tool `mlipaudit` should be available with two tasks:
 
-* `mlipaudit`: The **benchmarking CLI tool**. It runs the full or partial benchmark
-  suite for one or more models. Results will be stored locally in multiple JSON files
-  in an intuitive directory structure.
-* `mlipauditapp`: The **UI app** for visualization of the results. Running it opens a
+* `mlipaudit benchmark`: The **benchmarking CLI task**. It runs the full or partial
+  benchmark suite for one or more models. Results will be stored locally in multiple
+  JSON files in an intuitive directory structure.
+* `mlipaudit gui`: The **UI app** for visualization of the results. Running it opens a
   browser window and displays the web app. Implementation is based
   on `streamlit <https://streamlit.io/>`_.
 
-Benchmarking CLI tool
----------------------
+Benchmarking task
+-----------------
 
-The tool has the following command line options:
+The benchmarking CLI task is invoked by running
+
+.. code-block:: bash
+
+    mlipaudit benchmark [OPTIONS]
+
+and has the following command line options:
 
 * `-h / --help`: Prints info on usage of tool into terminal.
 * `-m / --models`: Paths to the
@@ -56,7 +62,7 @@ For example, if you want to run the entire benchmark suite for two models, say
 
 .. code-block:: bash
 
-    mlipaudit -m /path/to/visnet_1.zip /path/to/mace_2.zip -o /path/to/output
+    mlipaudit benchmark -m /path/to/visnet_1.zip /path/to/mace_2.zip -o /path/to/output
 
 The output directory then contains an intuitive folder structure of models and
 benchmarks with the aforementioned `result.json` files. Each of these files will
@@ -83,9 +89,10 @@ by running
 
 .. code-block:: bash
 
-    mlipauditapp /path/to/output
+    mlipaudit gui /path/to/output
 
-in the terminal. This should open a browser window automatically.
+in the terminal. This should open a browser window automatically. More information
+can be obtained by running `mlipaudit gui -h`.
 
 The landing page of the app will provide you with some basic information about the app
 and with a table of all the evaluated models with their overall score.
@@ -139,7 +146,7 @@ You can now run your benchmarks like this:
 
 .. code-block:: bash
 
-    mlipaudit -m /path/to/my_model.py -o /path/to/output
+    mlipaudit benchmark -m /path/to/my_model.py -o /path/to/output
 
 Note that the model name that will be assigned to the model will be `my_model`.
 

pyproject.toml

@@ -20,7 +20,6 @@ dependencies = [
 
 [project.scripts]
 mlipaudit = "mlipaudit.main:main"
-mlipauditapp = "mlipaudit.app:launch_app"
 
 [build-system]
 requires = ["hatchling"]

src/mlipaudit/app.py

@@ -18,7 +18,6 @@
 from typing import Callable
 
 import streamlit as st
-from streamlit import runtime as st_runtime
 from streamlit.web import cli as st_cli
 
 from mlipaudit.benchmark import BenchmarkResult
@@ -66,25 +65,41 @@ def _get_pages_for_category(
     ]
 
 
-def main():
-    """Main of our UI app.
+def _parse_app_args(argvs: list[str]) -> tuple[str, bool]:
+    """Parse the command line arguments for the app.
+
+    Args:
+        argvs: The command line arguments.
+
+    Returns:
+        The parsed arguments.
 
     Raises:
         RuntimeError: if results directory is not passed as argument.
     """
-    if len(sys.argv) < 2:
+    if len(argvs) < 2:
         raise RuntimeError(
             "You must provide the results directory as a command line argument, "
-            "like this: mlipauditapp /path/to/results"
+            "like this: mlipaudit gui /path/to/results"
         )
     is_public = False
-    if len(sys.argv) == 3 and sys.argv[2] == "__hf":
+    if len(argvs) == 3 and argvs[2] == "__public":
         is_public = True
-    else:
-        if not Path(sys.argv[1]).exists():
-            raise RuntimeError("The specified results directory does not exist.")
 
-    results_dir = sys.argv[1]
+    if not Path(argvs[1]).exists():
+        raise RuntimeError("The specified results directory does not exist.")
+
+    results_dir = argvs[1]
+    return results_dir, is_public
+
+
+def main() -> None:
+    """Main of our UI app.
+
+    Raises:
+        RuntimeError: if results directory is not passed as argument.
+    """
+    results_dir, is_public = _parse_app_args(argvs=sys.argv)
 
     results = load_benchmark_results_from_disk(results_dir, BENCHMARKS)
     scores = load_scores_from_disk(scores_dir=results_dir)
@@ -102,7 +117,7 @@ def main():
         benchmark_pages[name] = st.Page(
             functools.partial(
                 page_wrapper.get_page_func(),
-                data_func=_data_func_from_key(name, results),
+                data_func=_data_func_from_key(name, results),  # type: ignore
             ),
             title=name.replace("_", " ").capitalize(),
             url_path=name,
@@ -148,15 +163,14 @@ def main():
     pg.run()
 
 
-def launch_app():
+def launch_app(results_dir: str, is_public: bool) -> None:
     """Figures out whether run by streamlit or not. Then calls `main()`."""
-    if st_runtime.exists():
-        main()
-    else:
-        original_args_without_exec = sys.argv[1:]
-        sys.argv = ["streamlit", "run", __file__] + original_args_without_exec
-        sys.exit(st_cli.main())
+    args = [results_dir]
+    if is_public:
+        args.append("__public")
+    sys.argv = ["streamlit", "run", __file__] + args
+    sys.exit(st_cli.main())
 
 
 if __name__ == "__main__":
-    launch_app()
+    main()