Merge branch 'main' into more_flexible_folder_structure

Author: enryH

.github/workflows/cdci.yml

@@ -1,4 +1,4 @@
-name: Python package
+name: CICD Python Package
 
 on:
   push:

.github/workflows/docs.yml

@@ -1,4 +1,4 @@
-name: Test documentation building, and publish report to GitHub Pages
+name: Docs and Report Examples
 
 on:
   push:
@@ -10,7 +10,7 @@ on:
 
 jobs:
   test:
-    name: Test
+    name: Test docs and report examples
     runs-on: ubuntu-latest
     strategy:
       matrix:

CONTRIBUTING.md

@@ -27,16 +27,17 @@ you to share any suggestions or observations you have about this project.
 
 Here are the ways you can submit your suggestions and contribute to the project:
 
-1. **Reporting Issues or Suggesting Improvements:** If you have a  [GitHub][github] account 
-   (or are willing to [open one][github-join]) but are unfamiliar with Git, you can report 
-   bugs or suggest improvements by [creating an issue][new-issue]. This GitHub feature allows 
-   for discussion threads on reported issues and proposed enhancements.
+### 1. Reporting Issues or Suggesting Improvements
 
-2. **Submitting Changes via Pull Requests:** If you are comfortable using Git and would like to
-   add or modify a functionality, you can submit a **pull request (PR)**. Instructions on how to contribute this way are provided in the next section.
+If you have a  [GitHub][github] account (or are willing to [open one][github-join]) but are unfamiliar with Git, you can report bugs or suggest improvements by [creating an issue][new-issue]. This GitHub feature allows for discussion threads on reported issues and proposed enhancements.
 
-3. **Providing Feedback via Email:** If you don’t have a GitHub account and are
-   unfamiliar with Git, you can send feedback via email to [[email protected]][contact]. However, using GitHub is preferred, as it allows us to respond more quickly and track discussions openly.
+### 2. Submitting Changes via Pull Requests
+
+If you are comfortable using Git and would like to add or modify a functionality, you can submit a **pull request (PR)**. Instructions on how to contribute this way are provided in the next section.
+
+### 3. Providing Feedback via Email
+
+If you don’t have a GitHub account and are unfamiliar with Git, you can send feedback via email to [[email protected]][contact]. However, using GitHub is preferred, as it allows us to respond more quickly and track discussions openly.
 
 > [!NOTE]
 > The documentation for [Git][git-docs] and [GitHub][github-docs] are easy to follow, and you can learn the basics using their official guides.

README.md

@@ -7,7 +7,7 @@
 | Information           | Links                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
 | :-------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | **Package**           | [![PyPI Latest Release](https://img.shields.io/pypi/v/vuegen.svg)][vuegen-pypi] [![Conda Latest Release](https://img.shields.io/conda/v/bioconda/vuegen.svg)][vuegen-conda] [![Supported versions](https://img.shields.io/pypi/pyversions/vuegen.svg)][vuegen-pypi] [![Docker Repository on Quay](https://quay.io/repository/dtu_biosustain_dsp/vuegen/status "Docker Repository on Quay")][vuegen-docker-quay] [![License](https://img.shields.io/github/license/Multiomics-Analytics-Group/vuegen)][vuegen-license] |
-| **Documentation**     | [![made-with-sphinx-doc](https://img.shields.io/badge/Made%20with-Sphinx-1f425f.svg)](https://www.sphinx-doc.org/) ![Docs](https://readthedocs.org/projects/vuegen/badge/?style=flat) [![View - Documentation](https://img.shields.io/badge/view-Documentation-blue?style=flat)][vuegen-docs]                                                                                                                                                                                                                         |
+| **Documentation**     | [![View - Documentation](https://img.shields.io/badge/view-Documentation-blue?style=flat)][vuegen-docs] [![made-with-sphinx-doc](https://img.shields.io/badge/Made%20with-Sphinx-1f425f.svg)](https://www.sphinx-doc.org/) ![Docs](https://readthedocs.org/projects/vuegen/badge/?style=flat)                                                                                                                                                                                                                           |
 | **Build**             | [![CI](https://github.com/Multiomics-Analytics-Group/vuegen/actions/workflows/cdci.yml/badge.svg)][ci-gh-action] [![Docs](https://github.com/Multiomics-Analytics-Group/vuegen/actions/workflows/docs.yml/badge.svg)][ci-docs]                                                                                                                                                                                                                                                                                        |
 | **Examples**          | [![HTML5](https://img.shields.io/badge/html5-%23E34F26.svg?style=for-the-badge&logo=html5&logoColor=white)][emp-html-demo] [![Streamlit](https://img.shields.io/badge/Streamlit-%23FE4B4B.svg?style=for-the-badge&logo=streamlit&logoColor=white)][emp-st-demo]                                                                                                                                                                                                                                                       |
 | **Discuss on GitHub** | [![GitHub issues](https://img.shields.io/github/issues/Multiomics-Analytics-Group/vuegen)][issues] [![GitHub pull requests](https://img.shields.io/github/issues-pr/Multiomics-Analytics-Group/vuegen)][pulls]                                                                                                                                                                                                                                                                                                        |
@@ -98,7 +98,7 @@ If you prefer not to install VueGen on your system, a pre-configured Docker cont
 
 ### Nextflow and nf-core
 
-VueGen is also available as a [nf-core][nfcore] module, customised for compatibility with the [Nextflow][nextflow] environment. This module is designed to automate report generation from outputs produced by other modules, subworkflows, or pipelines. The code and documentation for the nf-core module are available in the [nf-VueGen repository][nf-vuegen].
+VueGen is also available as a [nf-core][nfcore] module, customised for compatibility with the [Nextflow][nextflow] environment. This module is designed to automate report generation from outputs produced by other modules, subworkflows, or pipelines. You can read the offical documentation for the nf-core module [here](nf-vuegen-nf-core). Also, the source code and detailed documentation are available in the [nf-VueGen repository][nf-vuegen].
 
 ## Execution
 
@@ -238,7 +238,7 @@ More information regarding the app and builds can be found in the
 
 VueGen’s functionality is demonstrated through two case studies:
 
-**1. Predefined Directory**
+### 1. Predefined Directory
 
 This introductory case study uses a predefined directory with plots, dataframes, Markdown, and HTML components. Users can generate reports in different formats and modify the configuration file to customize the report structure.
 
@@ -247,7 +247,7 @@ This introductory case study uses a predefined directory with plots, dataframes,
 > [!NOTE]
 > The [configuration file][predef-dir-config] is available in the `docs/example_config_files` folder, and the [directory][predef-dir] with example data is in the `docs/example_data` folder.
 
-**2. Earth Microbiome Project Data**
+### 2. Earth Microbiome Project Data
 
 This advanced case study demonstrates the application of VueGen in a real-world scenario using data from the [Earth Microbiome Project (EMP)][emp]. The EMP is an initiative to characterize global microbial taxonomic and functional diversity. The notebook process the EMP data, create plots, dataframes, and other components, and organize outputs within a directory to produce reports. Report content and structure can be adapted by modifying the configuration file. Each report consists of sections on exploratory data analysis, metagenomics, and network analysis.
 
@@ -257,10 +257,12 @@ This advanced case study demonstrates the application of VueGen in a real-world
 > The EMP case study is available online as [HTML][emp-html-demo] and [Streamlit][emp-st-demo] reports.
 > The [configuration file][emp-config] is available in the `docs/example_config_files` folder, and the [directory][emp-dir] with example data is in the `docs/example_data` folder.
 
-**3. ChatBot Component**
+### 3. ChatBot Component
+
+This case study highlights VueGen’s capability to embed a chatbot component into a report subsection, 
+enabling interactive conversations inside the report. This component is streamlit-specific and is not 
+available for other report types.
 
-This case study highlights VueGen’s capability to embed a chatbot component into a report subsection,
-enabling interactive conversations inside the report.
 
 Two API modes are supported:
 
@@ -290,9 +292,17 @@ to support more flexible and general-purpose response formats in future releases
 
 Once a Streamlit report is generated, it can be deployed as a web application to make it accessible online. There are multiple ways to achieve this:
 
-- **Streamlit Community Cloud**: Deploy your report easily using [Streamlit Cloud][st-cloud], as demonstrated in the [EMP VueGen Demo][emp-st-demo]. The process involves moving the necessary scripts, data, and a requirements.txt file into a GitHub repository. Then, the app can be deployed via the Streamlit Cloud interface. The deployment example is available in the `streamlit-report-example` branch.
-- **Standalone Executables**: Convert your Streamlit application into a desktop app by packaging it as an executable file for different operating systems. A detailed explanation of this process can be found in this [Streamlit forum post][st-forum-exe].
-- **Stlite**: Run Streamlit apps directly in the browser with [stlite][stlite], a WebAssembly port of Streamlit powered by Pyodide, eliminating the need for a server. It also allows packaging apps as standalone desktop executables using stlite desktop.
+### Streamlit Community Cloud 
+
+Deploy your report easily using [Streamlit Cloud][st-cloud], as demonstrated in the [EMP VueGen Demo][emp-st-demo]. The process involves moving the necessary scripts, data, and a requirements.txt file into a GitHub repository. Then, the app can be deployed via the Streamlit Cloud interface. The deployment example is available in the `streamlit-report-example` branch.
+
+### Standalone Executables
+
+Convert your Streamlit application into a desktop app by packaging it as an executable file for different operating systems. A detailed explanation of this process can be found in this [Streamlit forum post][st-forum-exe].
+
+### Stlite
+
+Run Streamlit apps directly in the browser with [stlite][stlite_repo], a WebAssembly port of Streamlit powered by Pyodide, eliminating the need for a server. It also allows packaging apps as standalone desktop executables using stlite desktop.
 
 These options provide flexibility depending on whether the goal is online accessibility, lightweight execution, or local application distribution.
 
@@ -357,6 +367,7 @@ We appreciate your feedback! If you have any comments, suggestions, or run into
 [nfcore]: https://nf-co.re/
 [nextflow]: https://www.nextflow.io/
 [nf-vuegen]: https://github.com/Multiomics-Analytics-Group/nf-vuegen/
+[nf-vuegen-nf-core]: https://nf-co.re/modules/vuegen/
 [colab_badge]: https://colab.research.google.com/assets/colab-badge.svg
 [colab_link_intro_demo]: https://colab.research.google.com/github/Multiomics-Analytics-Group/vuegen/blob/main/docs/vuegen_basic_case_study.ipynb
 [predef-dir-config]: https://github.com/Multiomics-Analytics-Group/vuegen/blob/main/docs/example_config_files/Basic_example_vuegen_demo_notebook_config.yaml
@@ -366,7 +377,7 @@ We appreciate your feedback! If you have any comments, suggestions, or run into
 [emp-config]: https://github.com/Multiomics-Analytics-Group/vuegen/blob/main/docs/example_config_files/Earth_microbiome_vuegen_demo_notebook_config
 [emp-dir]: https://github.com/Multiomics-Analytics-Group/vuegen/blob/main/docs/example_data/Earth_microbiome_vuegen_demo_notebook
 [st-cloud]: https://streamlit.io/cloud
-[stlite]: https://github.com/whitphx/stlite
+[stlite_repo]: https://github.com/whitphx/stlite
 [st-forum-exe]: https://discuss.streamlit.io/t/streamlit-deployment-as-an-executable-file-exe-for-windows-macos-and-android/6812
 [Mona]: https://multiomics-analytics-group.github.io/
 [Biosustain]: https://www.biosustain.dtu.dk/

docs/README.md

@@ -2,9 +2,9 @@
 
 In order to build the docs you need to 
 
-  1. install sphinx and additional support packages
-  2. build the package reference files
-  3. run sphinx to create a local html version
+  1. Install sphinx and additional support packages
+  2. Build the package reference files
+  3. Run sphinx to create a local html version
 
 The documentation is build using readthedocs automatically.
 
@@ -31,7 +31,9 @@ sphinx-apidoc --force --implicit-namespaces --module-first -o reference ../src/v
 sphinx-build -n -W --keep-going -b html ./ ./_build/
 ```
 
-## Include repo README.md into docs
+## Include repo README into docs
+
+The README is included in the `Overview` section of the docs. We created a [Python script](https://github.com/Multiomics-Analytics-Group/vuegen/blob/split-readme-docs/docs/split_readme.py) to split the README sections into separate md files, stored in `docs/sections_readme`. The `index.md` file contains the structure of the docs with the generated sections and additional information.
 
 Relative links are used in the main README, which need to be resolved when building. It's
 possible to include the a `relative-docs` option if one uses `index.md` ([see docs](https://myst-parser.readthedocs.io/en/latest/faq/index.html#include-a-file-from-outside-the-docs-folder-like-readme-md)). This does not work

docs/conf.py

@@ -38,12 +38,13 @@
     "sphinx.ext.intersphinx",
     "sphinx_new_tab_link",
     "myst_nb",
+    "sphinx_copybutton",
 ]
 
 #  https://myst-nb.readthedocs.io/en/latest/computation/execute.html
 nb_execution_mode = "auto"
 
-myst_enable_extensions = ["dollarmath", "amsmath"]
+myst_enable_extensions = ["dollarmath", "amsmath", "colon_fence"]
 
 # Plolty support through require javascript library
 # https://myst-nb.readthedocs.io/en/latest/render/interactive.html#plotly
@@ -98,13 +99,13 @@
 # https://github.com/executablebooks/MyST-NB/blob/master/docs/conf.py
 # html_title = ""
 html_theme = "sphinx_book_theme"
-# html_logo = "_static/logo-wide.svg"
-# html_favicon = "_static/logo-square.svg"
+html_logo = "https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg"
+html_favicon = "https://raw.githubusercontent.com/Multiomics-Analytics-Group/vuegen/main/docs/images/vuegen_logo.svg"
 html_theme_options = {
     "github_url": "https://github.com/Multiomics-Analytics-Group/vuegen",
     "repository_url": "https://github.com/Multiomics-Analytics-Group/vuegen",
     "repository_branch": "main",
-    "home_page_in_toc": True,
+    "home_page_in_toc": False,
     "path_to_docs": "docs",
     "show_navbar_depth": 1,
     "use_edit_page_button": True,
@@ -136,7 +137,16 @@
     PROJECT_ROOT = Path(__file__).parent.parent
     PACKAGE_ROOT = PROJECT_ROOT / "src" / "vuegen"
 
+    def run_split_readme(_):
+        print("[conf.py] Splitting README.md into sections...")
+        from split_readme import process_readme
+
+        readme_path = PROJECT_ROOT / "README.md"
+        output_dir = PROJECT_ROOT / "docs" / "sections_readme"
+        process_readme(readme_path, output_dir)
+
     def run_apidoc(_):
+        print("[conf.py] Running sphinx-apidoc...")
         from sphinx.ext import apidoc
 
         apidoc.main(
@@ -154,4 +164,5 @@ def run_apidoc(_):
         )
 
     def setup(app):
+        app.connect("builder-inited", run_split_readme)
         app.connect("builder-inited", run_apidoc)

docs/example_report.md

@@ -1,6 +1,3 @@
-# Vuegen Case Study - View HTML Report
+# Earth Microbiome Project Case Study - HTML and Streamlit Example Reports
 
-The stable version of the current `html` report generated in the example notebook
-using `vuegen` can be viewed at
-
-[multiomics-analytics-group.github.io/vuegen/](https://multiomics-analytics-group.github.io/vuegen/)
+The Earth Microbiome Project case study generated in the example notebook using `vuegen` is available online as [HTML](https://multiomics-analytics-group.github.io/vuegen/) and [Streamlit](https://earth-microbiome-vuegen-demo.streamlit.app/) reports.

docs/images/vuegen_graph_abstract.png

@@ -1,14 +1,28 @@
-# Overview
-
 <!-- https://myst-parser.readthedocs.io/en/latest/faq/index.html
 #include-a-file-from-outside-the-docs-folder-like-readme-md -->
 
-```{include} ../README.md
-:start-line: 0
+```{include} ./sections_readme/home_page.md
+:caption: VueGen
 :relative-docs: docs
 :relative-images:
 ```
 
+```{toctree} 
+:maxdepth: 1
+:caption: Overview
+
+sections_readme/about
+sections_readme/installation
+sections_readme/execution
+sections_readme/gui
+sections_readme/case_studies
+sections_readme/web_app_deploy
+sections_readme/contributing
+sections_readme/citation
+sections_readme/credits
+sections_readme/contact
+```
+
 ```{toctree}
 :maxdepth: 2
 
@@ -20,8 +34,12 @@ vuegen_demo
 :caption: Building a report
 
 vuegen_basic_case_study
+vuegen_basic_case_study_configfile
 vuegen_case_study_earth_microbiome
+vuegen_case_study_earth_microbiome_configfile
 example_report
+vuegen_APICall_configfile
+vuegen_Chatbot_configfile
 ```
 
 ```{toctree}