Merge branch 'main' into add-local-contrib-guide

Author: Cli4d

.github/workflows/gh-pages.yml

@@ -43,7 +43,7 @@ jobs:
           hugo config
 
       - name: Build with Hugo
-        run: hugo --minify
+        run: npm run build
 
       - name: Build JSDoc
         run: npm run jsdoc

.github/workflows/test.yml

@@ -29,6 +29,8 @@ jobs:
           hugo-version: '0.119.0'
           extended: true # Prevent error building site: POSTCSS: failed to transform, see https://gohugo.io/troubleshooting/faq/#i-get-this-feature-is-not-available-in-your-current-hugo-version
       - name: Build with Hugo
-        run: hugo --minify
+        run: npm run build
+      - name: Build JSDoc
+        run: npm run jsdoc
       - name: Test broken links
         run: npm run test:links

README.md

@@ -1,5 +1,11 @@
 Open Terms Archive documentation.
 
+## Styleguide
+
+### Order of menu sections
+
+Pages and sections are ordered on the “end user” to “core team member” spectrum: the more up a page is in the menu, the more it is targeted at consumer end users, the more low it is the more it is targeted at committed contributors.
+
 ## Installation
 
 ### Dependencies

content/_index.en.md

@@ -21,30 +21,23 @@ _Words in bold are [business domain names](https://en.wikipedia.org/wiki/Domain-
 
 ## Main concepts
 
-### Instances
+### Collection
 
-Open Terms Archive is a decentralised system.
-
-It aims at enabling any entity to **track** **terms** on its own and at federating a number of public **instances** in a single ecosystem to maximise discoverability, collaboration and political power. To that end, the Open Terms Archive **engine** can be run on any server, thus making it a dedicated **instance**.
-
-> Federated public instances can be [found on GitHub](
-https://github.com/OpenTermsArchive?q=declarations).
-
-### Collections
-
-An **instance** **tracks** **terms** of a single **collection**.
+Open Terms Archive is a decentralised system. It aims at enabling any entity to **track** **terms** on its own. To that end, the Open Terms Archive **engine** can be run on any server, thus making it a dedicated **instance**. An **instance** **tracks** **terms** within a single **collection**.
 
 A **collection** is characterised by a **scope** across **dimensions** that describe the **terms** it **tracks**, such as **language**, **jurisdiction** and **industry**.
 
-> Federated public collections can be [found on GitHub](https://github.com/OpenTermsArchive?q=versions).
-
 #### Example scope
 
 > The terms tracked in this collection are:
 > - Of dating services used in Europe.
 > - In the European Union and Switzerland jurisdictions.
 > - In English, unless no English version exists, in which case the primary official language of the jurisdiction of incorporation of the service operator will be used.
 
+### Federation
+
+In order to maximise discoverability, collaboration and political power, public **collections** are **federated** within a single ecosystem. This makes their data mutually discoverable and enables mutualising effort.
+
 ### Terms types
 
 To distinguish between the different **terms** of a **service**, each has a **type**, such as “Terms of Service”, “Privacy Policy”, “Developer Agreement”…
@@ -105,9 +98,8 @@ Such a dataset can be generated from **versions** alone. If **snapshots** and **
 
 This documentation describes how to execute the **engine** independently from any specific **instance**. For other use cases, other parts of the documentation could be more relevant:
 
-- to contribute **declarations** to an existing **instance**, see [how to contribute terms]({{< relref "contributing-terms" >}});
-- to create a new **collection**, see the [collection bootstrap](https://github.com/OpenTermsArchive/template-declarations) script;
-- to create a new public **instance**, see the [governance]({{< relref "governance" >}}) documentation.
+- to contribute **declarations** to an existing **collection**, see [contributing terms]({{< relref "contributing-terms" >}});
+- to create a new **collection**, see [creating a new collection]({{< relref "collections/create" >}}).
 
 ### Requirements
 
@@ -284,68 +276,6 @@ The server `<port>` and `<basePath>` are defined in the [configuration](#configu
 
 > For example, with the default configuration, the list of services can be found at [`http://localhost:3000/api/v1/services`](http://localhost:3000/api/v1/services).
 
-## API
-
-Open Terms Archive collections can be explored locally over a [Node.js](#nodejs-api), or remotely over a [Web API](#web-api). As Open Terms Archive is decentralised, each [instance](#instances) embarks its own API. The documentation relevant to the specific version of the engine on that instance is provided on that instance itself.
-
-### Web API
-
-The Web API exposes JSON data over HTTP(S). Its [OpenAPI](https://swagger.io/specification/) specification can be found at `http://localhost:<port>/<basePath>/<API version>/docs`.
-
-That endpoint exposes both the OpenAPI specification if the requested `Content-Type` is JSON, and a Swagger UI for visual and interactive documentation otherwise.
-
-> For example, the [documentation](http://162.19.74.224/api/v1/docs) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations) is publicly available for exploration.
-
-### Node.js API
-
-As a Node module dependency, the engine exposes a JavaScript API that can be called in your own code. The following modules are available.
-
-#### `fetch`
-
-The `fetch` module gets the MIME type and content of a document from its URL
-
-```js
-import fetch from '@opentermsarchive/engine/fetch';
-```
-
-Documentation on how to use `fetch` is provided [as JSDoc](https://docs.opentermsarchive.org/jsdoc/).
-
-##### Headless browser management
-
-If you pass the `executeClientScripts` option to `fetch`, a headless browser will be used to download and execute the page before serialising its DOM. For performance reasons, the starting and stopping of the browser is your responsibility to avoid instantiating a browser on each fetch. Here is an example on how to use this feature:
-
-```js
-import fetch, { launchHeadlessBrowser, stopHeadlessBrowser } from '@opentermsarchive/engine/fetch';
-
-await launchHeadlessBrowser();
-await fetch({ executeClientScripts: true, ... });
-await fetch({ executeClientScripts: true, ... });
-await fetch({ executeClientScripts: true, ... });
-await stopHeadlessBrowser();
-```
-
-The `fetch` module options are defined as a [`node-config` submodule](https://github.com/node-config/node-config/wiki/Sub-Module-Configuration). The default `fetcher` configuration can be overridden by adding a `fetcher` object to the [local configuration file](#configuration-file).
-
-#### `extract`
-
-The `extract` module transforms HTML or PDF content into a Markdown string according to a [declaration](#declarations).
-
-```js
-import extract from '@opentermsarchive/engine/extract';
-```
-
-The `extract` function documentation is available [as JSDoc](https://docs.opentermsarchive.org/jsdoc/).
-
-#### `SourceDocument`
-
-The `SourceDocument` class encapsulates information about a terms' source document tracked by Open Terms Archive.
-
-```js
-import SourceDocument from '@opentermsarchive/engine/sourceDocument';
-```
-
-The `SourceDocument` format is defined [in source code](https://github.com/OpenTermsArchive/engine/tree/main/src/archivist/services/sourceDocument.js).
-
 - - -
 
 ## Configuring
@@ -393,13 +323,12 @@ The default configuration can be found in `config/default.json`. The full refere
       "sendWarnings": "Boolean. Set to true to also send email in case of warning",
     }
   },
-  "tracker": { // Tracking mechanism to create GitHub issues when terms content is inaccessible
+  "reporter": { // Reporter mechanism to create GitHub issues when terms content is inaccessible
     "githubIssues": {
-      "repository": "GitHub repository where to create isssues",
-      "label": {
-        "name": "Label to attach to bot-created issues. This specific label will be created automatically in the target repository",
-        "color": "The hexadecimal color code for the label, without the leading #",
-        "description": "A short description of the label"
+      "repositories": {
+        "declarations": "GitHub repository where to create issues; expected format: <owner>/<repository>",
+        "versions": "GitHub repository of versions associated with the declarations; expected format: <owner>/<repository>",
+        "snapshots": "GitHub repository of snapshots associated with the declarations; expected format: <owner>/<repository>"
       }
     }
   },

content/api/_index.md

@@ -0,0 +1,4 @@
+---
+title: API
+weight: 6
+---

content/api/collection.en.md

@@ -0,0 +1,14 @@
+---
+title: Collection
+weight: 2
+---
+
+# Collection Web API [Beta]
+
+As Open Terms Archive is decentralised, each instance embarks its own API. The documentation relevant to the specific version of the engine on that instance is provided on that instance itself.
+
+The Collection API exposes JSON data over HTTP. Its [OpenAPI](https://swagger.io/specification/) specification can be found at `http://localhost:<port>/<basePath>/<API version>/docs`.
+
+That endpoint exposes both the OpenAPI specification if the requested `Content-Type` is JSON, and a Swagger UI for visual and interactive documentation otherwise.
+
+> For example, the [documentation](http://162.19.74.224/api/v1/docs) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations) is publicly available for exploration.

content/api/federated.en.md

@@ -0,0 +1,24 @@
+---
+title: Federated
+weight: 3
+---
+
+# Federated Web API [Beta]
+
+Open Terms Archive is a decentralised system that tracks collections of services' terms across multiple servers. Each collection operates its own API, and the federated API unifies search and discovery across collections, fostering collaboration with external applications.
+
+The Federated Web API exposes JSON data over HTTP. Its [documentation](http://51.89.136.45/v1/docs/) is provided in a dedicated, interactive interface.
+
+That endpoint exposes both the [OpenAPI](https://swagger.io/specification/) specification if the requested `Content-Type` is JSON, and a Swagger UI for visual and interactive documentation otherwise.
+
+## Beta
+
+This API is offered as a preview, based on a first use case [defined](https://github.com/OpenTermsArchive/engine/issues/1016) with partner [ToS;DR](https://tosdr.org). Unexpected problems or missing functionality may arise. Please provide feedback through [issues](https://github.com/OpenTermsArchive/federated-api/issues) in the dedicated repository.
+
+## Source code
+
+The codebase for the Federated API is available on [`github.com/OpenTermsArchive/federated-api`](https://github.com/OpenTermsArchive/federated-api).
+
+## Deploying
+
+Deployment recipes are available in a [dedicated repository](https://github.com/OpenTermsArchive/deployment). Look at the [Federated API section](https://github.com/OpenTermsArchive/deployment#federated-api-application) on the README to know how to deploy the API.

content/api/node.en.md

@@ -0,0 +1,54 @@
+---
+title: Node
+weight: 1
+---
+
+# Node.js API [Beta]
+
+As a Node module dependency, the engine exposes a JavaScript API that can be called in your own code. The following modules are available.
+
+## `fetch`
+
+The `fetch` module gets the MIME type and content of a document from its URL
+
+```js
+import fetch from '@opentermsarchive/engine/fetch';
+```
+
+Documentation on how to use `fetch` is provided [as JSDoc](/jsdoc/index.html).
+
+### Headless browser management
+
+If you pass the `executeClientScripts` option to `fetch`, a headless browser will be used to download and execute the page before serialising its DOM. For performance reasons, the starting and stopping of the browser is your responsibility to avoid instantiating a browser on each fetch. Here is an example on how to use this feature:
+
+```js
+import fetch, { launchHeadlessBrowser, stopHeadlessBrowser } from '@opentermsarchive/engine/fetch';
+
+await launchHeadlessBrowser();
+await fetch({ executeClientScripts: true, ... });
+await fetch({ executeClientScripts: true, ... });
+await fetch({ executeClientScripts: true, ... });
+await stopHeadlessBrowser();
+```
+
+The `fetch` module options are defined as a [`node-config` submodule](https://github.com/node-config/node-config/wiki/Sub-Module-Configuration). The default `fetcher` configuration can be overridden by adding a `fetcher` object to the local configuration file.
+
+## `extract`
+
+The `extract` module transforms HTML or PDF content into a Markdown string according to a declaration.
+
+```js
+import extract from '@opentermsarchive/engine/extract';
+```
+
+The `extract` function documentation is available [as JSDoc](/jsdoc/index.html).
+
+## `SourceDocument`
+
+The `SourceDocument` class encapsulates information about a terms' source document tracked by Open Terms Archive.
+
+```js
+import SourceDocument from '@opentermsarchive/engine/sourceDocument';
+```
+
+The `SourceDocument` format is defined [in source code](https://github.com/OpenTermsArchive/engine/tree/main/src/archivist/services/sourceDocument.js).

content/collections/_index.md

@@ -0,0 +1,4 @@
+---
+title: Collections
+weight: 5
+---