diff --git a/.markdownlint.yml b/.markdownlint.yml index 5ed0a27b..4a52dc42 100644 --- a/.markdownlint.yml +++ b/.markdownlint.yml @@ -14,6 +14,7 @@ no-inline-html: allowed_elements: - abbr - i + - filename no-duplicate-header: false diff --git a/config/_default/module.yaml b/config/_default/module.yaml new file mode 100644 index 00000000..1557d6e5 --- /dev/null +++ b/config/_default/module.yaml @@ -0,0 +1,5 @@ +mounts: + - source: assets + target: assets + - source: node_modules/minireset.css/minireset.css + target: assets/css/vendor/minireset.css diff --git a/content/_index.en.md b/content/_index.en.md deleted file mode 100644 index cd12f475..00000000 --- a/content/_index.en.md +++ /dev/null @@ -1,480 +0,0 @@ ---- -title: "Getting started" -weight: 1 ---- - -_The document you are reading now is targeted at developers wanting to use or contribute to the engine of [Open Terms Archive](https://opentermsarchive.org). For a high-level overview of Open Terms Archive’s wider goals and processes, please read its [public homepage](https://opentermsarchive.org)._ - -# Open Terms Archive Engine - -This codebase is a Node.js module enabling downloading, archiving and publishing versions of documents obtained online. It can be used independently from the Open Terms Archive ecosystem. - -- - - - -## Motivation - -_Words in bold are [business domain names](https://en.wikipedia.org/wiki/Domain-driven_design)._ - -**Services** have **terms** written in **documents**, contractual (Terms of Services, Privacy Policy…) or not (Community Guidelines, Deceased User Policy…), that can change over time. Open Terms Archive enables users rights advocates, regulatory bodies and interested citizens to follow the **changes** to these **terms**, to be notified whenever a new **version** is published, to explore their entire **history** and to collaborate in analysing them. This free and open-source engine is developed to support these goals. - -- - - - -## Main concepts - -### Collection - -Open Terms Archive is a decentralised system. It aims at enabling any entity to **track** **terms** on its own. To that end, the Open Terms Archive **engine** can be run on any server, thus making it a dedicated **instance**. An **instance** **tracks** **terms** within a single **collection**. - -A **collection** is characterised by a **scope** across **dimensions** that describe the **terms** it **tracks**, such as **language**, **jurisdiction** and **industry**. - -#### Example scope - -> The terms tracked in this collection are: -> - Of dating services used in Europe. -> - In the European Union and Switzerland jurisdictions. -> - In English, unless no English version exists, in which case the primary official language of the jurisdiction of incorporation of the service operator will be used. - -### Federation - -In order to maximise discoverability, collaboration and political power, public **collections** are **federated** within a single ecosystem. This makes their data mutually discoverable and enables mutualising effort. - -### Terms types - -To distinguish between the different **terms** of a **service**, each has a **type**, such as “Terms of Service”, “Privacy Policy”, “Developer Agreement”… - -This **type** matches the topic, but not necessarily the title the **service** gives to it. Unifying the **types** enables comparing **terms** across **services**. - -> More information on terms types can be found in the [dedicated repository](https://github.com/OpenTermsArchive/terms-types). They are published on NPM under [`@opentermsarchive/terms-types`](https://www.npmjs.com/package/@opentermsarchive/terms-types), enabling standardisation and interoperability beyond the Open Terms Archive engine. - -### Declarations - -The **terms** that constitute a **collection** are defined in simple JSON files called **declarations**. - -A **declaration** also contains some metadata on the **service** on which the **terms** apply. - -> Here is an example declaration tracking the Privacy Policy of Open Terms Archive: -> -> ```json -> { -> "name": "Open Terms Archive", -> "documents": { -> "Privacy Policy": { -> "fetch": "https://opentermsarchive.org/en/privacy-policy", -> "select": ".textcontent" -> } -> } -> } -> ``` - -- - - - -## Add terms to a collection - -Open Terms Archive **acquires** **terms** to deliver an explorable **history** of **changes**. This can be done in two ways: - -1. For the present and future, by **tracking**. -2. For the past, by **importing** from an existing **fonds** such as [ToSBack](https://tosback.org), the [Internet Archive](https://web.archive.org/), [Common Crawl](https://commoncrawl.org) or any other in-house format. - -### Tracking terms - -In order to **track** the **changes** of **terms**, the **engine** **records** a **snapshot** of **documents** that contain them by **fetching** their web **location** several times a day. The **engine** then **extracts** a **version** from this **snapshot** by: - -1. **Selecting** the subset of the **document** (or **documents**) that contains the **terms** (instead of, e.g., navigation menus, footers, cookies banners…). -2. **Removing insignificant content**, that is residual content in this subset that is not part of the **terms** (e.g. ads, illustrative pictures, internal navigation links…). -3. **Filtering noise** that can emerge in the **terms** by preventing parts that change frequently from triggering false positives for **changes** (e.g. tracker identifiers in links, relative dates…). The **engine** can execute custom **filters** written in JavaScript to that end. - -After these steps, if **changes** are spotted in the resulting **terms**, a new **version** is **recorded**. - -Preserving **snapshots** enables recovering after the fact information potentially lost in the **extraction** step: if **declarations** were wrong, they can be **maintained** and corrected **versions** can be **extracted** from the original **snapshots**. - -### Importing terms - -Existing **fonds** can be prepared for easier analysis by unifying their format to the **Open Terms Archive dataset format**. This unique format enables building interoperable tools, fostering collaboration across reusers. -Such a dataset can be generated from **versions** alone. If **snapshots** and **declarations** can be retrieved from the **fonds** too, then a full-fledged **collection** can be created. - -- - - - -## Use the engine - -This documentation describes how to execute the **engine** independently from any specific **instance**. For other use cases, other parts of the documentation could be more relevant: - -- to contribute **declarations** to an existing **collection**, see the [Terms reference]({{< relref "terms/reference" >}}); -- to create a new **collection**, see [creating a new collection]({{< relref "collections/create" >}}). - -### Requirements - -This module is tested to work across operating systems (continuous testing on UNIX, macOS and Windows). - -A [Node.js](https://nodejs.org/en/download/) runtime is required to execute this engine. - -![Supported Node.js version can be found in the package.json file](https://img.shields.io/node/v/@opentermsarchive/engine?color=informational&label=Supported%20Node.js%20version) - -### Getting started - -This engine is published as a [module on NPM](https://npmjs.com/package/@opentermsarchive/engine). The recommended install is as a dependency in a `package.json` file, next to a folder containing [declaration files](#declarations). - -```sh -npm install --save @opentermsarchive/engine -mkdir declarations -``` - -In an editor, create the following declaration file in `declarations/Open Terms Archive.json` to track the terms of the Open Terms Archive website: - -```json -{ - "name": "Open Terms Archive", - "documents": { - "Privacy Policy": { - "fetch": "https://opentermsarchive.org/en/privacy-policy", - "select": ".textcontent" - } - } -} -``` - -In the terminal: - -```sh -npx ota track -``` - -The tracked terms can be found in the `data` folder. - -This quick example aimed at letting you try the engine quickly. Most likely, you will simply `npm install` from an existing collection, or create a new collection from the [collection template](https://github.com/OpenTermsArchive/template-declarations). - -### CLI - -Once the engine module is installed as a dependency within another module, the `ota` command with the following subcommands is available. - -In these commands: - -- **``** is the case sensitive name of the service declaration file without the extension. For example, for `Twitter.json`, the service ID is `Twitter`. -- **``** is the property name used under the `documents` property in the declaration to declare a terms. For example, in the getting started declaration, the terms type declared is `Privacy Policy`. - -#### `ota track` - -```sh -npx ota track -``` - -[Track](#tracking-terms) the current terms of services according to provided declarations. - -The declarations, snapshots and versions paths are defined in the [configuration](#configuring). - -> Note that the snapshots and versions will be recorded at the moment the command is executed, on top of the existing local history. If a shared history already exists and the goal is to add on top of it, that history has to be downloaded before executing that command. - -##### Recap of available options - -```sh -npx ota track --help -``` - -##### Track terms of specific services - -```sh -npx ota track --services "" [""...] -``` - -##### Track specific terms of specific services - -```sh -npx ota track --services "" [""...] --types "" [""...] -``` - -##### Track terms four times a day - -```sh -npx ota track --schedule -``` - -#### `ota validate` - -```sh -npx ota validate [--services ...] [--types ...] -``` - -Check that all declarations allow recording a snapshot and a version properly. - -If one or several `` are provided, check only those services. - -##### Validate schema only - -```sh -npx ota validate --schema-only [--services ...] [--types ...] -``` - -Check that all declarations are readable by the engine. - -Allows for a much faster check of declarations, but does not check that the terms are actually accessible. - -If one or several `` are provided, check only those services. - -##### Validate modified terms only - -```sh -npx ota validate --modified -``` - -Run [`ota validate`](#ota-validate) only on files that have been modified in Git. - -#### `ota lint` - -```sh -npx ota lint [--services ...] [--fix] [--modified] -``` - -Test the the format of declarations' normalisation. - -Use `--fix` to automatically correct formatting mistakes and ensure that all declarations are standardised. - -If one or several `` are provided, check only those services. - -#### `ota dataset` - -Export the versions dataset into a ZIP file and publish it to GitHub releases. - -The dataset title and the URL of the versions repository are defined in the [configuration](#configuring). - -To export the dataset into a local ZIP file: - -```sh -npx ota dataset [--file ] -``` - -To export the dataset into a ZIP file and publish it on GitHub releases: - -```sh -GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --publish -``` - -The `GITHUB_TOKEN` can also be defined in a [`.env` file](#environment-variables). - -To export, publish the dataset and remove the local copy that was created after it has been uploaded: - -```sh -GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --publish --remove-local-copy -``` - -##### Publish dataset on monday every week - -To schedule export, publishing and local copy removal: - -```sh -GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --schedule --publish --remove-local-copy -``` - -#### `ota serve` - -```sh -npx ota serve -``` - -Start the collection [Web API](#web-api) server. - -The Web API will be available under `http://localhost:///`. -The server `` and `` are defined in the [configuration](#configuring). - -> For example, with the default configuration, the list of services can be found at [`http://localhost:3000/api/v1/services`](http://localhost:3000/api/v1/services). - -- - - - -## Configuring - -### Configuration file - -The default configuration can be found in `config/default.json`. The full reference is given below. You are unlikely to want to edit all of these elements. - -```js -{ - "@opentermsarchive/engine": { - "trackingSchedule": "Cron expression to define the tracking schedule; see below", - "services": { - "declarationsPath": "Directory containing services declarations and associated filters" - }, - "recorder": { - "versions": { - "storage": { - "": "Storage repository configuration object; see below" - } - }, - "snapshots": { - "storage": { - "": "Storage repository configuration object; see below" - } - } - }, - "fetcher": { - "waitForElementsTimeout": "Maximum time (in milliseconds) to wait for elements to be present in the page when fetching document in a headless browser" - "navigationTimeout": "Maximum time (in milliseconds) to wait for page to load", - "language": "Language (in ISO 639-1 format) to pass in request headers" - }, - "notifier": { // Notify specified mailing lists when new versions are recorded - "sendInBlue": { // SendInBlue API Key is defined in environment variables, see the “Environment variables” section below - "updatesListId": "SendInBlue contacts list ID of persons to notify on terms updates", - "updateTemplateId": "SendInBlue email template ID used for updates notifications" - } - }, - "logger": { // Logging mechanism to be notified upon error - "smtp": { - "host": "SMTP server hostname", - "username": "User for server authentication" // Password for server authentication is defined in environment variables, see the “Environment variables” section below - }, - "sendMailOnError": { // Can be set to `false` if sending email on error is not needed - "to": "The address to send the email to in case of an error", - "from": "The address from which to send the email", - "sendWarnings": "Boolean. Set to true to also send email in case of warning", - }, - "timestampPrefix": "Boolean. Set to true to prefix the timestamp of the error message with the current date (in ISO 8601 format YYYY-MM-DDTHH:MM:SSZ); see below" - }, - "reporter": { // Reporter mechanism to create GitHub issues when terms content is inaccessible - "type": "The type of reporter to use; can be 'github' or 'gitlab'", - "repositories": { - "declarations": "GitHub or GitLab repository where to create issues; expected format: /", - "versions": "GitHub or GitLab repository of versions associated with the declarations; expected format: /", - "snapshots": "GitHub or GitLab repository of snapshots associated with the declarations; expected format: /" - } - "baseURL": "Base URL of the GitLab instance (only relevant if 'type' is 'gitlab')", - "apiBaseURL": "Base URL of the GitLab API (only relevant if 'type' is 'gitlab')" - }, - "dataset": { // Release mechanism to create dataset periodically - "title": "Title of the dataset; recommended to be the name of the instance that generated it", - "versionsRepositoryURL": "GitHub or GitLab repository where the dataset will be published as a release; recommended to be the versions repository for discoverability and tagging purposes", - "publishingSchedule": "Cron expression to define the dataset publishing schedule; see below", - "apiBaseURL": "Base URL of the GitLab API (only relevant if 'type' is 'gitlab')" - }, - "collection-api": { // Collection metadata API - "port": "The port number on which the API will listen for incoming requests", - "basePath": "The base path for the API endpoints" - } - } -} -``` - -The default configuration is merged with (and overridden by) environment-specific configuration that can be specified at startup with the `NODE_ENV` environment variable. See [node-config](https://github.com/node-config/node-config) for more information about configuration files. - -For development, in order to have a local configuration that overrides the existing config, it is recommended to create a `config/development.json` file. - -#### Logging - -Setting `logger.timestampPrefix` to `false` can be useful when logs are being collected by a process manager (like PM2) that already adds timestamps to log entries. This avoids duplicate timestamps in the logs. - -#### Schedules - -Schedules for tracking and dataset publication are defined using Cron expressions. - -A Cron expression is a string comprised of five or six fields separated by spaces, each representing a different unit of time: minute, hour, day of the month, month, and day of the week (and optionally, year). For example, the expression `30 */12 * * *` means "at minute 30 past every 12th hour of every day." - -Here are some valid examples of Cron expressions and what they represent: - -- `0 0 * * *`: Run at midnight every day. -- `0 */6 * * *`: Run every 6 hours. -- `30 2 * * MON`: Run at 2:30 AM every Monday. -. -Some online tools, such as [crontab.guru](https://crontab.guru), provide a user-friendly interface to create and validate Cron expressions. - -#### Storage repositories - -Two storage repositories are currently supported: Git and MongoDB. Each one can be used independently for versions and snapshots. - -##### Git - -```json -{ - … - "storage": { - "git": { - "path": "Versions database directory path, relative to the root of this project", - "publish": "Boolean. Set to true to push changes to the origin of the cloned repository at the end of every run. Recommended for production only.", - "snapshotIdentiferTemplate": "Text. Template used to explicit where to find the referenced snapshot id. Must contain a %SNAPSHOT_ID that will be replaced by the snapshot ID. Only useful for versions", - "author": { - "name": "Name to which changes in tracked terms will be credited", - "email": "Email to which changes in tracked terms will be credited" - } - } - } - … -} -``` -##### MongoDB - -```json -{ - … - "storage": { - "mongo": { - "connectionURI": "URI for defining connection to the MongoDB instance. See https://docs.mongodb.com/manual/reference/connection-string/", - "database": "Database name", - "collection": "MongoDB collection name; not to be confused with the Open Terms Archive collection" - } - } - … -} -``` - -### Environment variables - -Environment variables can be passed in the command-line or provided in a `.env` file at the root of the repository. See `.env.example` for an example of such a file. - -- `OTA_ENGINE_SMTP_PASSWORD`: a password for email server authentication, in order to send email notifications. -- `OTA_ENGINE_SENDINBLUE_API_KEY`: a SendInBlue API key, in order to send email notifications with that service. -- `OTA_ENGINE_GITHUB_TOKEN`: a token with repository privileges to access the [GitHub API](https://github.com/settings/tokens) to create issues and publish dataset releases. -- `OTA_ENGINE_GITLAB_TOKEN`: a token with repository privileges to access the [GitLab API](https://gitlab.com/profile/personal_access_tokens) to create issues. -- `OTA_ENGINE_GITLAB_RELEASES_TOKEN`: a token with repository privileges to access the [GitLab API](https://gitlab.com/profile/personal_access_tokens) to publish dataset releases. - -If both `OTA_ENGINE_GITHUB_TOKEN` and `OTA_ENGINE_GITLAB_TOKEN` are defined, GitHub takes precedence for dataset publishing. - -If an outgoing HTTP/HTTPS proxy to access the Internet is required, it is possible to provide it through the `HTTP_PROXY` and `HTTPS_PROXY` environment variable. - -- - - - -## Deploying - -Deployment recipes are available in a [dedicated repository](https://github.com/OpenTermsArchive/deployment). Look at the [README](https://github.com/OpenTermsArchive/deployment#readme) to know how to deploy the engine. - -- - - - -## Contributing - -### Getting a copy - -In order to edit the code of the engine itself, an editable and executable copy is necessary. - -First of all, follow the [requirements](#requirements) above. Then, clone the repository: - -```sh -git clone https://github.com/OpenTermsArchive/engine.git -cd engine -``` - -Install dependencies: - -```sh -npm install -``` - -### Testing - -If changes are made to the engine, check that all parts covered by tests still work properly: - -```sh -npm test -``` - -If existing features are changed or new ones are added, relevant tests must be added too. - -### Suggesting changes - -To contribute to the core engine of Open Terms Archive, see the [CONTRIBUTING](https://github.com/OpenTermsArchive/engine/blob/main/CONTRIBUTING.md) file. You will need knowledge of JavaScript and Node.js. - -### Sponsorship and partnerships - -Beyond individual contributions, we need funds and committed partners to pay for a core team to maintain and grow Open Terms Archive. If you know of opportunities, please let us know over email at `contact@[project name without spaces].org`! - -- - - - -## License - -The code for this software is distributed under the [European Union Public Licence (EUPL) v1.2](https://joinup.ec.europa.eu/collection/eupl/eupl-text-eupl-12). In short, this [means](https://choosealicense.com/licenses/eupl-1.2/) you are allowed to read, use, modify and redistribute this source code, as long as you as you credit “Open Terms Archive Contributors” and make available any change you make to it under similar conditions. - -Contact the core team over email at `contact@[project name without spaces].org` if you have any specific need or question regarding licensing. - diff --git a/content/_index.md b/content/_index.md new file mode 100644 index 00000000..7c12ffe7 --- /dev/null +++ b/content/_index.md @@ -0,0 +1,39 @@ +--- +title: Open Terms Archive documentation +linkTitle: Homepage +weight: 1 +--- + +# Open Terms Archive documentation + +Open Terms Archive is a decentralised system that tracks collections of services' terms across multiple servers. Each collection exposes the data it collects through datasets and through its own API. The Federation API unifies search and discovery across collections. + +## Required minimal terminology + +Before diving into the details, let's first understand the essential terminology used throughout this documentation. + +- **Terms:** the contractual documents published by services (such as Terms of Service, Privacy Policy, Community Guidelines…) that users agree to. +- **Collection:** a set of tracked terms grouped by a specific scope (such as language, jurisdiction, industry…). +- **Declaration:** a JSON file that defines which terms to track for a service and how to track them. +- **Snapshot:** the original source document (HTML, PDF…) from which the terms will be extracted. +- **Version:** the textual content of the terms after filtering out irrelevant content (navigation menu, advertisements…) from a snapshot. + +## Documentation structure + +This documentation follows the [Diátaxis](https://diataxis.fr) approach and structures content in different categories: + +- **Tutorials:** step-by-step learning guides that help beginners get started with Open Terms Archive, providing foundational knowledge and hands-on experience. +- **How-to guides:** task-focused instructions that help experienced users accomplish specific goals efficiently and effectively. +- **Reference:** comprehensive technical documentation detailing configuration options and specifications for advanced users. +- **Explanations:** background knowledge that enables understanding the constraints and how choices that are made. + +### Table of contents + +- **Analysis:** guidance on how to analyze terms changes, from navigating through the history of tracked documents to publishing memos about significant changes. +- **Community:** information on how to participate in the Open Terms Archive community. +- **Terms:** guidance on tracking and maintaining terms declarations. +- **Collections:** guidance on creating and managing collections of tracked terms. +- **Federation:** constraints and benefits of joining the Open Terms Archive federation. +- **Deployment:** guidance on operating an Open Terms Archive instance. +- **Programmatic access:** documentation on the engine CLI and APIs. +- **Concepts and principles:** main concepts, terminology and fundamental design principles of Open Terms Archive. diff --git a/content/analysis/_index.md b/content/analysis/_index.md new file mode 100644 index 00000000..06716769 --- /dev/null +++ b/content/analysis/_index.md @@ -0,0 +1,5 @@ +--- +title: "Analysis" +weight: 2 +--- + diff --git a/content/analysis/how-to/_index.md b/content/analysis/how-to/_index.md new file mode 100644 index 00000000..537c3cba --- /dev/null +++ b/content/analysis/how-to/_index.md @@ -0,0 +1,4 @@ +--- +title: How to +weight: 1 +--- diff --git a/content/terms/how-to-navigate-history.md b/content/analysis/how-to/navigate-history.md similarity index 97% rename from content/terms/how-to-navigate-history.md rename to content/analysis/how-to/navigate-history.md index 0a0fb1d8..14fb5efb 100644 --- a/content/terms/how-to-navigate-history.md +++ b/content/analysis/how-to/navigate-history.md @@ -1,7 +1,9 @@ --- -title: "How to navigate history" +title: Navigate history weight: 1 -aliases: /navigate-history/ +aliases: + - /navigate-history/ + - /terms/how-to-navigate-history/ --- # How to navigate history diff --git a/content/memos/how-to-publish.en.md b/content/analysis/how-to/publish-memo.md similarity index 95% rename from content/memos/how-to-publish.en.md rename to content/analysis/how-to/publish-memo.md index 2c5f67d4..31e57696 100644 --- a/content/memos/how-to-publish.en.md +++ b/content/analysis/how-to/publish-memo.md @@ -1,5 +1,6 @@ --- -title: "How to publish a memo" +title: Publish a memo +aliases: /memos/how-to-publish/ --- # How to publish a memo @@ -10,7 +11,7 @@ No technical skills are required. ## Prerequisites -- A drafted memo compliant with the [copywriting reference]({{< relref "/memos/copywriting-reference" >}}). +- A drafted memo compliant with the [copywriting reference]({{< relref "/analysis/reference/copywriting" >}}). ## 1. Send diff --git a/content/analysis/reference/_index.md b/content/analysis/reference/_index.md new file mode 100644 index 00000000..6f4fd0f8 --- /dev/null +++ b/content/analysis/reference/_index.md @@ -0,0 +1,4 @@ +--- +title: Reference +weight: 2 +--- diff --git a/content/memos/copywriting-reference.en.md b/content/analysis/reference/copywriting.md similarity index 99% rename from content/memos/copywriting-reference.en.md rename to content/analysis/reference/copywriting.md index 5c6999c5..08c81c22 100644 --- a/content/memos/copywriting-reference.en.md +++ b/content/analysis/reference/copywriting.md @@ -1,5 +1,6 @@ --- -title: "Copywriting reference" +title: Copywriting +aliases: /memos/copywriting-reference/ --- # Copywriting reference diff --git a/content/api/_index.md b/content/api/_index.md index 2b9b2cea..99349a05 100644 --- a/content/api/_index.md +++ b/content/api/_index.md @@ -1,4 +1,4 @@ --- -title: API -weight: 4 +title: Programmatic access +weight: 8 --- diff --git a/content/api/cli.md b/content/api/cli.md new file mode 100644 index 00000000..74af7d61 --- /dev/null +++ b/content/api/cli.md @@ -0,0 +1,63 @@ +--- +title: Command line interface +weight: 1 +--- + +# Command line interface + +Once the engine module is installed as a dependency within another module, the `ota` command with the following subcommands is available. + +In these commands: + +- `` is the case sensitive name of the service declaration file without the extension. For example, for `Twitter.json`, the service ID is `Twitter`. +- `` is the property name used under the `documents` property in the declaration to declare a terms. For example, in the getting started declaration, the terms type declared is `Privacy Policy`. + +## Tracking terms + +{{< configOption name="ota track" description="Track the current terms of services according to provided declarations. The declarations, snapshots and versions paths are defined in the configuration." example="`npx ota track`" >}} + +> Note that the snapshots and versions will be recorded at the moment the command is executed, on top of the existing local history. If a shared history already exists and the goal is to add on top of it, that history has to be downloaded before executing that command. + +{{< configOption name="ota track --help" description="Show help and available options for track command" example="`npx ota track --help`" >}} + +{{< configOption name="ota track [--services ...]" description="Track terms of specific services only" example="`npx ota track --services \"Facebook\" \"LinkedIn\"`" >}} + +{{< configOption name="ota track [--services ...] [--types ...]" description="Track specific terms types of specific services only" example="`npx ota track --services \"Facebook\" \"LinkedIn\" --types \"Privacy Policy\" \"Terms of Service\"`" >}} + +{{< configOption name="ota track --schedule [--services ...] [--types ...]" description="Track terms on the schedule defined in the configuration" example="`npx ota track --schedule`" >}} + +## Validating declarations + +{{< configOption name="ota validate [--services ...] [--types ...]" description="Check that all declarations allow recording a snapshot and a version properly. If service IDs are provided, check only those services." example="`npx ota validate --services \"Facebook\" \"LinkedIn\" --types \"Privacy Policy\" \"Terms of Service\"`" >}} + +{{< configOption name="ota validate --schema-only [--services ...] [--types ...]" description="Check that all declarations are readable by the engine. Allows for a much faster check of declarations, but does not check that the terms are actually accessible." example="`npx ota validate --schema-only --services \"Facebook\" \"LinkedIn\" --types \"Privacy Policy\" \"Terms of Service\"`" >}} + +{{< configOption name="ota validate --modified" description="Run ota validate only on files that have been modified in Git" example="`npx ota validate --modified`" >}} + +## Linting declarations + +{{< configOption name="ota lint [--services ...]" description="Test the format of declarations' normalisation." example="`npx ota lint --services \"Facebook\" \"LinkedIn\"`" >}} + +{{< configOption name="ota lint --fix [--services ...]" description="Automatically correct formatting mistakes and ensure that all declarations are standardised" example="`npx ota lint --fix`" >}} + +{{< configOption name="ota lint --modified" description="Run ota lint only on files that have been modified in Git" example="`npx ota lint --modified`" >}} + +## Publishing dataset + +{{< configOption name="ota dataset [--file ]" description="Export the versions dataset into a ZIP file and publish it to GitHub releases. The dataset title and the URL of the versions repository are defined in the configuration." example="`npx ota dataset --file dataset.zip`" >}} + +To export the dataset into a ZIP file and publish it on GitHub releases: + +{{< configOption name="ota dataset --publish [--file ]" description="Export and publish dataset to GitHub releases" example="`GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --publish`" >}} + +The `GITHUB_TOKEN` can also be defined in a [`.env` file]({{< relref "collections/reference/environment-variables" >}}). + +To export, publish the dataset and remove the local copy that was created after it has been uploaded: + +{{< configOption name="ota dataset --publish --remove-local-copy [--file ]" description="Export, publish dataset and remove local copy after upload" example="`GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --publish --remove-local-copy`" >}} + +{{< configOption name="ota dataset --schedule [--file ]" description="Schedule export, publishing and local copy removal" example="`GITHUB_TOKEN=ghp_XXXXXXXXX npx ota dataset --schedule --publish --remove-local-copy`" >}} + +## Exposing the collection API + +{{< configOption name="ota serve" description="Start the collection Web API server. The Web API will be available under `:///`. The server port and base path are defined in the configuration." example="`npx ota serve`" >}} diff --git a/content/api/collection.en.md b/content/api/collection.md similarity index 93% rename from content/api/collection.en.md rename to content/api/collection.md index e5dfc8bc..f6cdcd4d 100644 --- a/content/api/collection.en.md +++ b/content/api/collection.md @@ -1,9 +1,9 @@ --- -title: Collection +title: Collection REST API weight: 2 --- -# Collection Web API [Beta] +# Collection REST API As Open Terms Archive is decentralised, each instance embarks its own API. The documentation relevant to the specific version of the engine on that instance is provided on that instance itself. diff --git a/content/api/federation.en.md b/content/api/federation.md similarity index 99% rename from content/api/federation.en.md rename to content/api/federation.md index 9f61c34b..0a53f42d 100644 --- a/content/api/federation.en.md +++ b/content/api/federation.md @@ -1,10 +1,10 @@ --- -title: Federation +title: Federation REST API weight: 3 aliases: /api/federated/ --- -# Federation API +# Federation REST API Open Terms Archive is a decentralised system that tracks collections of services' terms across multiple servers. Each collection operates its own API, and the Federation API unifies search and discovery across collections, fostering collaboration with external applications. diff --git a/content/api/node.en.md b/content/api/node.md similarity index 72% rename from content/api/node.en.md rename to content/api/node.md index 4dcf2433..32312dde 100644 --- a/content/api/node.en.md +++ b/content/api/node.md @@ -1,15 +1,15 @@ --- -title: Node +title: Node.js module weight: 1 --- -# Node.js API [Beta] +# Node.js module -As a Node module dependency, the engine exposes a JavaScript API that can be called in your own code. The following modules are available. +As a Node module dependency, the engine exposes a JavaScript API that can be called in your own code. ## `fetch` -The `fetch` module gets the MIME type and content of a document from its URL +`fetch` gets the MIME type and content of a document from its URL ```js import fetch from '@opentermsarchive/engine/fetch'; @@ -31,11 +31,11 @@ await fetch({ executeClientScripts: true, ... }); await stopHeadlessBrowser(); ``` -The `fetch` module options are defined as a [`node-config` submodule](https://github.com/node-config/node-config/wiki/Sub-Module-Configuration). The default `fetcher` configuration can be overridden by adding a `fetcher` object to the local configuration file. +The `fetch` options are defined as a [`node-config` submodule](https://github.com/node-config/node-config/wiki/Sub-Module-Configuration). The default `fetcher` configuration can be overridden by adding a `fetcher` object to the local configuration file. ## `extract` -The `extract` module transforms HTML or PDF content into a Markdown string according to a declaration. +`extract` transforms HTML or PDF content into a Markdown string according to a declaration. ```js import extract from '@opentermsarchive/engine/extract'; diff --git a/content/collections/_index.md b/content/collections/_index.md index 317649b2..dfa6330b 100644 --- a/content/collections/_index.md +++ b/content/collections/_index.md @@ -1,4 +1,4 @@ --- title: Collections -weight: 3 +weight: 5 --- diff --git a/content/collections/create.en.md b/content/collections/create.en.md deleted file mode 100644 index aee02682..00000000 --- a/content/collections/create.en.md +++ /dev/null @@ -1,394 +0,0 @@ ---- -title: Creating a collection -weight: 3 ---- - -# Creating a collection - -You are considering creating a new collection to track terms with Open Terms Archive? Amazing! - -## Define metadata - -First of all, define the [metadata]({{< relref "collections/metadata" >}}) of the collection you would like to create and keep them ready to be used when setting up the repositories. - -## Check existing collections - -Now that you have a clear idea what you would like to track, double-check that there are no [existing federated collections](https://opentermsarchive.org/#collections) that you could contribute to. If you have a doubt about whether some terms you want to track would fit a collection, reach out to the collection maintainers. - -If no existing collection could be a good host for the terms you would like to track, then it is relevant to create your own. - -## Inform the community - -Starting a new collection is an exciting endeavour, and would strongly benefit from the support of the community who already maintains existing collections. It is strongly recommended to share your intention to create a new collection as early as possible in the process, to get support and identify potential partners. - -You can inform the community by posting on the instant messaging system, or [sending an email](mailto:contact@opentermsarchive.org) to the core team. - -## Define governance - -Setting up and maintaining a collection over time needs fulfilling certain tasks on a regular basis. These tasks are handled through roles. To make sure that all these roles are covered, define the [governance]({{< relref "collections/governance" >}}) of your collection. - -At any time, feel free to ask for help or partners in the community. - -## Create repositories - -Collections rely on three git repositories being set up to hold the data. - -The instructions below assume the usage of GitHub to host repositories. If you don't use GitHub, try to set up the equivalent metadata in your git hosting platform. Contributions to the documentation to make it independent from GitHub are very welcome! - -### Declarations - -Create the collection declarations repository by using the [`demo-declarations`](https://github.com/OpenTermsArchive/demo-declarations) repository as template. - -- Go to the [`demo-declarations` repository](https://github.com/OpenTermsArchive/demo-declarations) -- Click on the “Use this template” dropdown and select “Create a new repository” -- Set the repository name to `-declarations`. For example: `pga-declarations`. -- When redirected to the newly generated repository, wait a minute or two for the automatic setup to run. You can check the status of the `first-time-setup` GitHub action to make sure that everything ran fine. - -#### Fill the “About” section - -- Click on the little cogwheel icon next to the “About” block. -- Set the description to “Declarations for ``. Maintained by ``.” -- Set website to `https://opentermsarchive.org`, or any other relevant dedicated website. -- Add the following tags: `terms-of-service`, `terms-of-service-agreements`, `terms-and-conditions`, `open-terms-archive`. -- Uncheck “Releases”, “Packages” and “Deployments”. - -#### Define repository settings - -These settings ease the whole contribution process. - -- In “General → Features” - - Disable “Wikis”. - - Disable “Projects”. -- In “General → Pull Requests” - - Check only the “Allow squash merging” option, and set it to “Default to pull request title and commit details”. - - Enable “Allow auto-merge”. - - Enable “Automatically delete head branches”. -- In “Branches” - - Add a branch protection rule for `main`. - - Check “Require a pull request before merging”, check "Require approvals" and set “Required number of approvals before merging” to 1. - - Check “Require status checks to pass before merging” and add `validate_modified_declarations` and `validate_schema` as required status checks. -- In “Actions → General → Actions permissions” - - Select “Allow all actions and reusable workflows”. - -#### Remove default labels - -Issues labels will be added by the engine as problems are encountered when tracking. The default labels offered by GitHub, such as `question` or `wontfix`, are relevant for software development but less so for the process prescribed by Open Terms Archive. - -- Remove all default labels. - -#### Update README - -- Update the README file with proper metadata: topic, maintainers, jurisdictions, languages… - -#### Update metadata - -- Update the metadata file `metadata.yml` with the collection metadata you defined earlier. - -### Snapshots - -Create the snapshots repository by using the [`demo-snapshots` repository](https://github.com/OpenTermsArchive/demo-snapshots) as template: - -- Go to the [`demo-snapshots` repository](https://github.com/OpenTermsArchive/demo-snapshots) -- Click on the “Use this template” dropdown and select “Create a new repository” -- Set the repository name to `-snapshots`. -- When redirected to the newly generated repository, wait a minute or two for the automatic setup to run. You can check the status of the `first-time-setup` GitHub action to make sure that everything ran fine. - -#### Fill the “About” section - -- Set the description: “Documents snapshots for ``. Maintained by ``.” -- Set website to `https://opentermsarchive.org`. -- Add the following tags: `terms-of-service`, `terms-of-service-agreements`, `terms-and-conditions`, `open-terms-archive`. -- Uncheck “Releases”, “Packages” and “Deployments”. - -#### Define repository settings - -These settings aim at minimising the otherwise overwhelming amount of information and click targets. - -- In “General → Features” - - Uncheck “Wikis”. - - Uncheck “Issues”. - - Uncheck “Discussions”. - - Uncheck “Projects”. -- In “Actions → General → Actions permissions” - - Select “Disable actions”. - -### Versions - -Create the versions repository by using the [`demo-versions` repository](https://github.com/OpenTermsArchive/demo-versions) as template: - -- Go to the [`demo-versions` repository](https://github.com/OpenTermsArchive/demo-versions) -- Click on the “Use this template” dropdown and select “Create a new repository” -- Set the repository name to `-versions`. -- When redirected to the newly generated repository, wait a minute or two for the automatic setup to run. You can check the status of the `first-time-setup` GitHub action to make sure that everything ran fine. - -#### Fill the “About” section - -- Set the description: “Terms versions for ``. Maintained by ``.” -- Set website to `{{< ref "terms/how-to-navigate-history" >}}` -- Add the following tags: `terms-of-service`, `terms-of-service-agreements`, `terms-and-conditions`, `open-terms-archive`. -- Uncheck “Packages”. -- Uncheck “Deployments”. - -#### Define repository settings - -These settings aim at minimising the otherwise overwhelming amount of information and click targets. - -- In “General → Features” - - Uncheck “Wikis”. - - Uncheck “Issues”. - - Uncheck “Discussions”. - - Uncheck “Projects”. -- In “Actions → General → Actions permissions” - - Select “Disable actions”. - -#### Update README - -- Update the README file with proper metadata. - -## Set up GitHub teams - -For collections to be included in the Open Terms Archive organisation only. For third parties, handle rights however you see fit. - -- [Create a new collection team](https://github.com/orgs/OpenTermsArchive/new-team) -- Give it the name of the collection -- Set as avatar the collection icon from the website -- Set as description: “Maintainers of the `` collection” -- Add selected members to the team -- Add the declarations repository to the collection team, with “Maintain” access rights -- Add the snapshots repository to the collection team, with “Triage” access rights (giving them more would enable them to corrupt data) -- Add the versions repository to the collection team, with “Triage” access rights (giving them more would enable them to corrupt data) -- Add the declarations, snapshots and versions repositories to the Bots team with “Write” access - -## Set up deployment - -### Check server configuration - -Before proceeding with deployment, ensure that the server meets the following requirements: - -- Verify that the server provides an Ed25519 fingerprint for its SSH host key: - - - Retrieve the Ed25519 SSH host key from the server `ssh-keyscan -t ed25519 ` - - If the server has an Ed25519 SSH host key defined, the output will display the corresponding fingerprint. It will look something like this: - - ```shell - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJM6fCKWkiKv+uysoHsklIAuUOH6Dpc3crzHxk7GwrD - ``` - - - If no output is displayed or if the output does not include an Ed25519 SSH host key fingerprint, it indicates that the server does not have an Ed25519 SSH host key defined. Here's an example of the output when no Ed25519 SSH host key is defined: - - ```shell - # SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDqruXk1P6vIVvN2i6ffLO6TlYCcC6lqF3oBYT7sC+nfIb5C9HYsUFWptSxohOy41wV3AbSzjHqEjxCt9MeJ4HXrLItti8Qr3fRBYgs45+L44bMZ9sA/EO+YiXQU9cQJb2qjK5EYQyFyEnGUMbh+zzRiCRQTI0A2nlnJ9zWQJr/4jIrlNJ6N0lV1GQzN/iIpCJto6ZmhbEgW5W3H+zB5qj72uKoyLlm8Lh+AF5ljtTnOuXgrh2nN6EN1hjRf52VtQZ93guIBkn5+riZ3gYYp3fl4sYbIAZRRs5rOcyFk9d/jo+kBw/Pxht4KABVHJHMPQ9cI2Fn2VbEOvZ+RrWLXc5Am3qwbUWpqYmp7n7wwdTrkYeCBMsXk7xQl5TJh+5Rkr6k0YRkcbvP+J+I1TJwob1DOyWBpRA3v9LYimEmy9eheQuKYzH5sQ/0r/7ZwhBL5/lB5kpv3kmwA7DZy1J5UbgChtSey3Du0N+6p/vgfybIgcZD5Csz8+dF3c+gZBCfRd4XpKgLoxLPZO69tM8/3z/3W0gOfXgEw6QKwJ6KoFXeBdRG9c/CCsR8dF3iIeZYWZvj+8nC/Y7hF6Dedr/6CHc0O4xwqE0GQzF3YUZI7HcqjxIIFsIsG+loUGWYB7a0HHn0FrAq79Q== - ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLe8sKzXq4KReWp0Dz1lC8AKOcYNtPuk7GOqJRSVGkG1xRhP94gReTp7S1WnF6LgFt3vlC2k62BkSoXgryY3+8= - ``` - - - If the server does not have an Ed25519 SSH host key defined, once logged in on your server, you can generate one: `sudo ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ed25519_key` - - Restart the SSH service to apply the changes: `sudo systemctl restart ssh` - -- Make sure that a non-root user has been set up on the server, if not you can create by following: - - - Once logged in on your server, create a new user: `adduser `. Follow the prompts to set a password for the new user and provide any additional information if required. - - Grant sudo privileges: `usermod -aG sudo `. - -- Grant `sudo` rights to the designated user without requiring a password prompt. To achieve this: - - - Open the file `/etc/sudoers` for editing. - - Navigate to the section titled `# Allow members of group sudo to execute any command`. - - Add the following line at the end of this section: - - ```shell - ALL=(ALL) NOPASSWD:ALL - ``` - -### Define the inventory - -On your local machine: - -- Clone the `-declarations` repository -- Update the following entries in the inventory file `deployment/inventory.yml`: - - `` (example: `162.19.74.224`) - - `ansible_user: ` (example: `debian`) - - `ed25519_fingerprint: ` obtained with `ssh-keyscan -t ed25519 ` (example: `AAAAC3NzaC1lZDI1ETE5AAAAIJkjE2KIbUcoClK+lKLR5ZvmdXMD/eXWghHdenFeJz4c`) - -### Add fingerprint to GitHub -declarations settings - -- Log in on GitHub with a user account that has admin privileges for the `-declarations` repository -- Go to `https://github.com/OpenTermsArchive/-declarations/settings/secrets/actions` -- Create the `SERVER_FINGERPRINT` [secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) by using the previously obtained ed25519 fingerprint - -### Set up a SSH deployment key - -This key will enable automated deployment via GitHub Actions. - -#### Create the SSH key - -- Connect to the server: `ssh @` -- Create a new deployment SSH key: `ssh-keygen -t ed25519 -q -N "" -f ~/.ssh/ota-deploy` -- Add the public key to `authorized_keys`: `cat ~/.ssh/ota-deploy.pub >> ~/.ssh/authorized_keys` - -#### Add key to GitHub -declarations repository secrets - -- Log in on GitHub using a user account with admin privileges for the `-declarations` repository -- Go to `https://github.com/OpenTermsArchive/-declarations/settings/secrets/actions` -- Create the `SERVER_SSH_KEY` secret with the previously generated deployment private key - -#### Back key up - -On your local machine: - -- Clone the [`engine.wiki`](https://github.com/OpenTermsArchive/engine/wiki) -- Open the shared passwords database `database.kdbx` with [KeePassXC](https://keepassxc.org) -- Create the `Collection: ` folder -- Inside this folder, add an entry with the title `Deployment SSH key` -- Attach public `ota-deploy.pub` and private `ota-deploy` key files to the entry -- Save, commit and push - -### Allow OTA-Bot to create issues and publish dataset on GitHub - -#### Create a fine-grained repo-scoped token - -- Log in on GitHub as OTA-Bot user -- Go to -- Set the “Token name”: “Issue/Release on `` collection” -- Leave blank “Description” -- Set the expiry date to the maximum possible (one year later at the time of writing) -- In “Resource owner”, select the “OpenTermsArchive“ organization -- Leave blank the following text area: `Describe why your personal access token needs access to the OpenTermsArchive organization.` -- In “Repository access”, select “Only select repositories” -- In “Select repositories”, Select “``-declarations” and ``-versions” -- In “Permissions” → “Repository permissions”, select “Contents — Access: Read and write” -- In “Permissions” → “Repository permissions”, select “Issues — Access: Read and write” -- Validate with “Generate token and request access” - -#### Backup the token - -On your local machine: - -- Open the shared passwords database `database.kdbx` with [KeePassXC](https://keepassxc.org) -- Inside the `Collection: ` folder, add an entry with the title `GitHub Token` -- Copy the previously generated token in the `Password` field -- Save, commit and push - -#### Validate the token - -- Log in on GitHub using a user account with admin privileges for the OpenTermsArchive organization -- Go to -- Select the pending request -- Approve it - -### Add secrets - -#### Generate vault key - -On your local machine: - -- Open the shared passwords database `database.kdbx` with [KeePassXC](https://keepassxc.org) -- Inside the `Collection: ` folder, add an entry with the title `Vault key` -- In the password field, click on the icon on the right to generate a new password -- Generate a new password that does not contain any quote or backtick -- Save, commit and push - -#### Create vault key file - -On your local machine: - -- Go to the `-declarations` repository -- Go to `deployment` folder -- Create a `vault.key` file -- Copy the previously generated password inside - -#### Add vault key to GitHub -declarations settings - -- Log in on GitHub using a user account with admin privileges for the `-declarations` repository -- Go to `https://github.com/OpenTermsArchive/-declarations/settings/secrets/actions` -- Create the `ANSIBLE_VAULT_KEY` [secrets](https://docs.github.com/en/actions/security-guides/encrypted-secrets#creating-encrypted-secrets-for-a-repository) by using the previously generated vault key - -#### Encrypt token and update deployment - -On your local machine: - -- Go to the `-declarations` repository -- Go to `deployment` folder -- Paste the token in a `.env` file under the name `OTA_ENGINE_GITHUB_TOKEN` -- Encrypt `.env`: `ansible-vault encrypt .env` - -### Set up an OTA-Bot SSH key specific to this collection - -#### Create the SSH key - -- Create a new SSH key: `ssh-keygen -t ed25519 -C bot@opentermsarchive.org -P "" -f ./-key` -- Copy the generated private key file - -#### Encrypt private key and update deployment - -On your local machine: - -- Go to the `-declarations` repository -- Go to `deployment` folder -- Paste the private key file -- Rename it into `github-bot-private-key` -- Encrypt key `github-bot-private-key`: `ansible-vault encrypt github-bot-private-key` - -#### Back key up - -On your local machine: - -- Inside the [`engine.wiki`](https://github.com/OpenTermsArchive/engine/wiki) -- Open the shared passwords database `database.kdbx` with [KeePassXC](https://keepassxc.org) -- Inside the `Collection: ` folder, add an entry with the title `OTA-Bot GitHub SSH key` -- Attach public `-key.pub` and private `-key` key files to the entry -- Save, commit and push - -#### Attach the key to OTA-Bot GitHub user - -- Log in on GitHub as OTA-Bot user -- Go to -- Set the “Title”: “`` collection” -- Paste the public key in “Key” -- Validate with “Add SSH Key” - -### Enable sending error reports over email - -#### Create an SMTP key - -Create an SMTP key to allow sending error notifications by email. - -- Log in on Brevo as Open Terms Archive admin user -- Go to -- “Generate a new SMTP key” -- Set the SMTP key name “Name your SMTP key”: `` collection” -- Validate with “Generate” - -#### Backup key - -On your local machine: - -- Inside the [`engine.wiki`](https://github.com/OpenTermsArchive/engine/wiki) -- Open the shared passwords database `database.kdbx` with [KeePassXC](https://keepassxc.org) -- Inside the `Collection: ` folder, add an entry with the title `SMTP Key` -- Copy the previously generated key in the `Password` field -- Save, commit and push - -#### Encrypt key and update deployment - -On your local machine: - -- Go to the `-declarations` repository -- Go to `deployment` folder -- Decrypt `.env` if necessary: `ansible-vault decrypt .env` -- Paste the key in a `.env` file under the name `OTA_ENGINE_SMTP_PASSWORD` -- Encrypt `.env`: `ansible-vault encrypt .env` - -## Test - -### Via GitHub Actions - -- Check that the `deploy` action ran properly on the declarations repository. - -### Locally - -To test deployment from your local machine, your SSH keys must be authorized to connect to the server. - -- `cd -declarations/deployment` -- `ansible-galaxy collection install -r requirements.yml` -- `ansible-playbook opentermsarchive.deployment.deploy` diff --git a/content/collections/federation.en.md b/content/collections/federation.en.md deleted file mode 100644 index 2783492e..00000000 --- a/content/collections/federation.en.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Federation -weight: 4 ---- - -# Federation - -Open Terms Archive is a decentralised system. It aims at enabling any entity set up their own collections and track terms on their own. - -In order to maximise discoverability, collaboration and political power, public collections are federated within a single ecosystem. This makes their data mutually discoverable and enables mutualising effort. - -## Benefits - -A collection that **joins** the **federation** enjoys the following benefits: - -1. Visibility on the Open Terms Archive website lists of collections and datasets. -2. Access to the Open Terms Archive GitHub organisation, administered by the Open Terms Archive core team. -3. Collection logo provided by the Open Terms Archive core team. -4. Referencing in the official [collections list](https://opentermsarchive.org/collections.json), enabling off-the-shelf discovery in the [Federation API]({{< relref "api/federation" >}}). -5. Referencing in the official [datasets list](https://opentermsarchive.org/datasets), providing visibility to analysts. -6. Dedicated channel on the Open Terms Archive instant messaging system. -7. API uptime tracking. -8. Public announcement through all Open Terms Archive communication channels upon joining. - -## Criteria - -A **collection** can **join** the Open Terms Archive **federation** if it abides by the following quality criteria: - -1. Clearly defined [collection metadata]({{< relref "collections/metadata" >}}). -2. Clearly defined [collection governance]({{< relref "collections/governance" >}}). -3. The vast majority of **versions** are readable, as evidenced by a sample assessment. -4. **Frequency** of at least one track a day, as evidenced by snapshots. -5. Public and open-licensed **declarations**, as evidenced by the `LICENSE` file in the declarations repository. -6. Public and open-licensed **snapshots**, as evidenced by the `LICENSE` file in the snapshots repository. -7. Public and open-licensed **versions**, as evidenced by the `LICENSE` file in the versions repository. -8. Regular, public, and open-licensed **dataset** releases, as evidenced by the `LICENSE` file in the datasets. -9. Publicly accessible API with median response time under 20ms, as evidenced by uptime tracking logs. -10. Abide by all Open Terms Archive software and data licenses. - -## How to join - -An official request for joining the federation can be sent by the collection administrator or curator to the core team over email, GitHub or instant messages. That request should contain all necessary content or links that enable assessing the criteria. - -The core team will confirm reception of the request and assess the criteria within 4 weeks. The result of the assessment will be provided to the requester through the same means the request was sent, along with a positive decision or a list of improvements that need to be done before a new request can be sent. - -Once a positive decision has been issued, the core team will provide the requester with a timeline for joining the federation, on which they will jointly agree. - -## Disclaimer - -Please note that establishing the federation is an ongoing effort. While criteria are assessed and refined to strike the right balance between accessibility and quality, the Open Terms Archive core team is responsible for assessing which collections may join the federation, and has the right to update the criteria as requests for joining are made. - -The core team has the right to re-assess the federated status of any collection at any given time. In case the federated status of a collection is changed, the collection administrator will be informed immediately, and if possible before the action is made. diff --git a/content/collections/how-to/_index.md b/content/collections/how-to/_index.md new file mode 100644 index 00000000..e5d7a8d9 --- /dev/null +++ b/content/collections/how-to/_index.md @@ -0,0 +1,5 @@ +--- +title: How to +weight: 2 +--- + diff --git a/content/collections/how-to/create-repositories.md b/content/collections/how-to/create-repositories.md new file mode 100644 index 00000000..4d383376 --- /dev/null +++ b/content/collections/how-to/create-repositories.md @@ -0,0 +1,118 @@ +--- +title: Create repositories +weight: 1 +--- + +# How to create collection repositories + +Collections in Open Terms Archive rely on three Git repositories to hold the data: +- Declarations repository: stores the declarations that define terms to track +- Snapshots repository: stores raw snapshots of tracked documents +- Versions repository: stores processed versions of tracked documents + +This guide assumes you use GitHub. For other Git platforms, adapt these steps accordingly. + +## Prerequisites + +Before starting, ensure you have: +- A GitHub account with permissions to create repositories +- Your collection ID and name defined +- Your collection metadata prepared + +## Create declarations repository + +### Create from template + +1. Go to the [`demo-declarations`](https://github.com/OpenTermsArchive/demo-declarations) repository +2. Click "Use this template" dropdown and select "Create a new repository" +3. Name it `-declarations` (e.g. `pga-declarations`) +4. Wait 1-2 minutes for automatic setup to complete (check `first-time-setup` action status) + +### Configure repository settings + +1. Set up the "About" section: + - Click the cogwheel icon next to "About" + - Add description: "Declarations for ``. Maintained by ``." + - Set website: `https://opentermsarchive.org` + - Add tags: `terms-of-service`, `terms-of-service-agreements`, `terms-and-conditions`, `open-terms-archive` + - Uncheck "Releases", "Packages" and "Deployments" + +2. Configure features: + - In "General → Features": + - Disable "Wikis" and "Projects" + - In "General → Pull Requests": + - Enable only "Allow squash merging" with "Default to pull request title and commit details" + - Enable "Allow auto-merge" + - Enable "Automatically delete head branches" + - In "Branches": + - Add branch protection for `main` + - Require pull request with 1 approval + - Require status checks: `validate_modified_declarations` and `validate_schema` + - In "Actions": + - Allow all actions and reusable workflows + +3. Remove all default labels + +4. Update content: + - Update README with collection metadata + - Update `metadata.yml` with collection metadata + +## Create snapshots repository + +### Create from template + +1. Go to [`demo-snapshots`](https://github.com/OpenTermsArchive/demo-snapshots) +2. Click "Use this template" and select "Create a new repository" +3. Name it `-snapshots` +4. Wait for automatic setup to complete + +### Configure repository + +1. Set up "About" section: + - Add description: "Documents snapshots for ``. Maintained by ``." + - Set website: `https://opentermsarchive.org` + - Add standard tags + - Uncheck "Releases", "Packages" and "Deployments" + +2. Configure features: + - Disable Wikis, Issues, Discussions, and Projects + - Disable GitHub Actions + +## Create versions repository + +### Create from template + +1. Go to [`demo-versions`](https://github.com/OpenTermsArchive/demo-versions) +2. Click "Use this template" and select "Create a new repository" +3. Name it `-versions` +4. Wait for automatic setup to complete + +### Configure repository + +1. Set up "About" section: + - Add description: "Terms versions for ``. Maintained by ``." + - Set website to terms navigation documentation + - Add standard tags + - Uncheck "Packages" and "Deployments" + +2. Configure features: + - Disable Wikis, Issues, Discussions, and Projects + - Disable GitHub Actions + +3. Update README with collection metadata + +## Set up GitHub teams + +For collections within the Open Terms Archive organization: + +1. [Create a new collection team](https://github.com/orgs/OpenTermsArchive/new-team) +2. Configure team: + - Name it after the collection + - Set collection icon as avatar + - Description: "Maintainers of the `` collection" +3. Set repository access: + - Declarations repository: "Maintain" access + - Snapshots repository: "Triage" access + - Versions repository: "Triage" access +4. Add team members +5. Add repositories to Bots team with "Write" access diff --git a/content/collections/how-to/create.md b/content/collections/how-to/create.md new file mode 100644 index 00000000..2fb1ccd0 --- /dev/null +++ b/content/collections/how-to/create.md @@ -0,0 +1,34 @@ +--- +title: Create a collection +weight: 3 +--- + +# How to create a collection + +You are considering creating a new collection to track terms with Open Terms Archive? Amazing! + +## Define metadata + +First of all, define the [metadata]({{< relref "collections/reference/metadata" >}}) of the collection you would like to create and keep them ready to be used when setting up the repositories. + +## Check existing collections + +Now that you have a clear idea what you would like to track, double-check that there are no [existing federated collections](https://opentermsarchive.org/#collections) that you could contribute to. If you have a doubt about whether some terms you want to track would fit a collection, reach out to the collection maintainers. + +If no existing collection could be a good host for the terms you would like to track, then it is relevant to create your own. + +## Inform the community + +Starting a new collection is an exciting endeavour, and would strongly benefit from the support of the community who already maintains existing collections. It is strongly recommended to share your intention to create a new collection as early as possible in the process, to get support and identify potential partners. + +You can inform the community by posting on the instant messaging system, or [sending an email](mailto:contact@opentermsarchive.org) to the core team. + +## Define governance + +Setting up and maintaining a collection over time needs fulfilling certain tasks on a regular basis. These tasks are handled through roles. To make sure that all these roles are covered, define the [governance]({{< relref "collections/reference/governance" >}}) of your collection. + +At any time, feel free to ask for help or partners in the community. + +## Create repositories + +See the [guide to create repositories]({{< relref "collections/how-to/create-repositories" >}}) for your collection. diff --git a/content/collections/how-to/define-metadata.md b/content/collections/how-to/define-metadata.md new file mode 100644 index 00000000..bffe5c67 --- /dev/null +++ b/content/collections/how-to/define-metadata.md @@ -0,0 +1,70 @@ +--- +title: Define metadata +weight: 2 +--- + +# How to define metadata + +This guide will help you define the metadata for your Open Terms Archive collection. + +## Prerequisites + +- A text editor. +- Basic understanding of the [YAML syntax](https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html). +- Your collection's basic information ready. + +## Steps + +1. Create a new file named `metadata.yml` in your collection's root directory. + +2. Define the required basic information: + + ```yaml + name: "Your Collection Name" # Keep it under 3 words + id: "your-collection-id" # Create a simple dash-separated identifier + tagline: "Brief description" # Write a one-line collection summary + languages: ["en"] # List ISO 639 language codes + jurisdictions: ["US"] # List ISO 3166-2 country codes + ``` + +3. Add detailed description and repositories URLs: + + ```yaml + description: "A more detailed explanation of your collection's purpose" + dataset: "https://github.com/YourOrg/your-collection-versions/releases" + declarations: "https://github.com/YourOrg/your-collection-declarations" + versions: "https://github.com/YourOrg/your-collection-versions" + snapshots: "https://github.com/YourOrg/your-collection-snapshots" + ``` + +4. Define tracking periods: + + ```yaml + trackingPeriods: + - startDate: "YYYY-MM-DD" # When tracking begins + schedule: "0 0 * * *" # Cron expression for frequency + serverLocation: "City, CC" # Server location (City, Country code) + ``` + +5. Define governance structure: + + ```yaml + governance: + - name: "Host entity" + url: "https://entity-website.com" + logo: "https://path-to-logo.png" + roles: ["curator", "maintainer"] + - name: "Admin entity" + url: "https://admin-website.com" + logo: "https://path-to-logo.png" + roles: ["administrator"] + ``` + +6. Add optional fields as needed: + + ```yaml + logo: "https://path-to-collection-logo.png" + donations: "https://donation-page-url.com" + ``` + +7. Validate your YAML syntax using a YAML validator. diff --git a/content/collections/metadata.en.md b/content/collections/metadata.en.md deleted file mode 100644 index fe18caaa..00000000 --- a/content/collections/metadata.en.md +++ /dev/null @@ -1,154 +0,0 @@ ---- -title: Metadata -weight: 1 ---- - -# Collection metadata - -A collection is defined by the following metadata. - -## Required fields - -| Field | Type | Description | -|----------------|-----------------|---------------------------------------| -| `name` | string | Display name of the collection (max 3 words) | -| `id` | string | Unique identifier derived from name (acronyms, dash-separated) | -| `tagline` | string | Concise description of collection topic | -| `languages` | Array of strings| [ISO 639 language codes](https://en.wikipedia.org/wiki/ISO_639) allowed in collection | -| `jurisdictions`| Array of strings| [ISO 3166-2 country codes](https://en.wikipedia.org/wiki/ISO_3166-2) for covered jurisdictions | - -Example: - -```yaml -name: Platform Governance Archive -id: pga -tagline: Largest global social media -languages: [en, fr, de] -jurisdictions: [EU] -``` - -## Additional fields - -Additional fields are optional but highly recommended to facilitate better discovery, comprehension, and utilization of the collection. - -| Field | Type | Description | -|----------------|-----------------|---------------------------------------| -| `description` | string | Detailed context beyond tagline | -| `dataset` | url | URL to released versions dataset | -| `declarations` | url | URL to declarations repository | -| `versions` | url | URL to versions repository | -| `snapshots` | url | URL to snapshots repository | -| `logo` | url | URL to the collection's logo. Optimized PNG transparent image (min width 240px) | -| `donation` | url | URL to donation page | -| `trackingPeriods`| array of tracking periods objects | see [Tracking periods](#tracking-periods) section | -| `governance` | governance object | see [Governance](#governance) section | - -Example: - -```yaml -description: Comprehensive collection of terms of service from major global social media platforms. -dataset: https://github.com/OpenTermsArchive/demo-versions/releases -declarations: https://github.com/OpenTermsArchive/demo-declarations -versions: https://github.com/OpenTermsArchive/demo-versions -snapshots: https://github.com/OpenTermsArchive/demo-snapshots -logo: https://opentermsarchive.org/images/collections/demo.png -donation: https://opencollective.com/opentermsarchive -``` - -### Tracking periods - -Information about when and how terms are tracked: - -| Field | Type | Description | -|---------------|--------|--------------------------------------------| -| `startDate` | date ([ISO 8601 format: YYYY-MM-DD](https://en.wikipedia.org/wiki/ISO_8601)) | The date when tracking started | -| `endDate` | date ([ISO 8601 format: YYYY-MM-DD](https://en.wikipedia.org/wiki/ISO_8601)) | The date when tracking ended or will end. If not specified, tracking is ongoing. | -| `schedule` | string | A [cron expression](https://en.wikipedia.org/wiki/Cron#Cron_expression) that defines the tracking frequency | -| `serverLocation`| string (city name, [ISO 3166-2 country code](https://en.wikipedia.org/wiki/ISO_3166-2)) | The geographic location of the tracking server | - -Example: - -```yaml -trackingPeriods: - - startDate: 2023-01-01 - endDate: 2024-12-31 - schedule: "0 0 * * *" - serverLocation: London, GB -``` - -### Governance - -Organizations responsible for collection roles: - -| Field | Type | Description | -|---------------|-----------------|----------------------------------------| -| `hosts` | Array of organizations objects| Organizations hosting the collection | -| `administrators`| Array of organizations objects| Organizations managing the collection | -| `curators` | Array of organizations objects| Organizations curating the content | -| `maintainers` | Array of organizations objects| Organizations maintaining the technical aspects | -| `sponsors` | Array of organizations objects| Organizations providing support | - -Each organization in the governance roles is defined by the following fields: - -| Field | Type | Description | -|---------------|--------|-------------------------------------------| -| `name` | string | Name of the organization | -| `url` | url | Website URL of the organization (optional) | -| `logo` | url | URL to the organization's logo. Optimized PNG transparent image (min width 240px) (optional) | - -Example: - -```yaml -governance: - administrators: - - name: Open Terms Archive - url: https://opentermsarchive.org/ - logo: https://opentermsarchive.org/images/logo/logo-open-terms-archive-black.png - curators: [] - maintainers: [] - hosts: [] - sponsors: [] -``` - -## Complete metadata example - -```yaml -id: demo -name: Demo -tagline: Services needed to operate the Open Terms Archive engine -description: This demo collection serves as a reference implementation… -dataset: https://github.com/OpenTermsArchive/demo-versions/releases -declarations: https://github.com/OpenTermsArchive/demo-declarations -versions: https://github.com/OpenTermsArchive/demo-versions -snapshots: https://github.com/OpenTermsArchive/demo-snapshots -logo: https://opentermsarchive.org/images/collections/demo.png -donation: https://opencollective.com/opentermsarchive -languages: [en] -jurisdictions: [EU] -trackingPeriods: - - startDate: 2023-01-01 - endDate: 2024-12-31 - schedule: "0 0 * * *" - serverLocation: London, GB -governance: - hosts: - - name: Ministry for Europe and Foreign Affairs - url: https://www.diplomatie.gouv.fr/en/ - logo: https://opentermsarchive.org/images/contributors/meae.png - administrators: - - name: Open Terms Archive - url: https://opentermsarchive.org/ - logo: https://opentermsarchive.org/images/logo/logo-open-terms-archive-black.png - curators: - - name: Open Terms Archive - url: https://opentermsarchive.org/ - logo: https://opentermsarchive.org/images/logo/logo-open-terms-archive-black.png - maintainers: - - name: Open Terms Archive - url: https://opentermsarchive.org/ - logo: https://opentermsarchive.org/images/logo/logo-open-terms-archive-black.png - sponsors: - - name: Ministry for Europe and Foreign Affairs - url: https://www.diplomatie.gouv.fr/en/ - logo: https://opentermsarchive.org/images/contributors/meae.png -``` diff --git a/content/collections/reference/_index.md b/content/collections/reference/_index.md new file mode 100644 index 00000000..63aae4a6 --- /dev/null +++ b/content/collections/reference/_index.md @@ -0,0 +1,4 @@ +--- +title: Reference +weight: 3 +--- diff --git a/content/collections/reference/configuration.md b/content/collections/reference/configuration.md new file mode 100644 index 00000000..9ff14ca2 --- /dev/null +++ b/content/collections/reference/configuration.md @@ -0,0 +1,302 @@ +--- +title: Configuration +weight: 2 +--- + +# Configuration options + +This reference documentation details all available configuration options that can be specified in a collection's configuration file to configure the Open Terms Archive engine. + +As an example, see the [production configuration file](https://github.com/OpenTermsArchive/demo-declarations/blob/main/config/production.json) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations). + +## Options + +{{< configOption + name="trackingSchedule" + type="string" + description="Defines how often the engine should check for changes in terms. Uses standard cron syntax to set the schedule." + default="`30 */12 * * *` (runs every 12 hours at minute 30)" +>}} + +{{< configOption + name="collectionPath" + type="string" + description="Path to the collection's directory containing declarations directory and metadata file, relative to the engine execution location" + example="`../collections/demo-declarations`" + default="`./`" +>}} + +### Recorder + +The recorder section manages how versions and snapshots of terms are stored, supporting multiple storage backends. + +{{< configOption + name="recorder.versions.storage" + type="object" + description="Configuration for storing versions. Supports Git and MongoDB. See [Storage Repositories](#storage-repositories) for more information." +>}} + +{{< configOption + name="recorder.snapshots.storage" + type="object" + description="Configuration for storing snapshots. Supports Git and MongoDB. See [Storage Repositories](#storage-repositories) for more information." +>}} + +### Fetcher + +The fetcher section configures how the engine retrieves documents from the web. + +{{< configOption + name="fetcher.waitForElementsTimeout" + type="number" + description="Maximum wait time for elements to appear in a page (milliseconds)." + default="`10000`" +>}} + +{{< configOption + name="fetcher.navigationTimeout" + type="number" + description="Maximum wait time for a page to load (milliseconds)." + default="`30000`" +>}} + +{{< configOption + name="fetcher.language" + type="string" + description="Language code (ISO 639-1) for request headers." + default="`en`" +>}} + +### Notifier + +The notifier section sets up how notifications are sent when new versions of terms are recorded. + +{{< configOption + name="notifier.sendInBlue.updatesListId" + type="string" + description="SendInBlue contacts list ID of persons to notify on terms updates." + default="`850`" +>}} + +{{< configOption + name="notifier.sendInBlue.updateTemplateId" + type="string" + description="SendInBlue email template ID used for updates notifications." + default="`7`" +>}} + +### Logger + +The logger section configures logging and error notification settings. + +{{< configOption + name="logger.smtp.host" + type="string" + description="SMTP server hostname." + default="`smtp-relay.sendinblue.com`" +>}} + +{{< configOption + name="logger.smtp.username" + type="string" + description="Username for SMTP server authentication." + default="`admin@opentermsarchive.org`" +>}} + +{{< configOption + name="logger.sendMailOnError.to" + type="string" + description="Email address for error notifications." + example="`admin@example.com`" +>}} + +{{< configOption + name="logger.sendMailOnError.from" + type="string" + description="Sender email address for error notifications." + example="`noreply@example.com`" +>}} + +{{< configOption + name="logger.sendMailOnError.sendWarnings" + type="boolean" + description="Set to true to also send email in case of warning." + default="`false`" +>}} + +{{< configOption + name="logger.timestampPrefix" + type="boolean" + description="Set to false to avoid duplicate timestamps if logs are managed by a process manager." + default="`true`" +>}} + +### Reporter + +The reporter section manages how issues are reported when terms content is inaccessible, supporting GitHub and GitLab. + +{{< configOption + name="reporter.type" + type="string" + description="Type of reporter" + example="`github`" + allowedValues="`github`, `gitlab`" +>}} + +{{< configOption + name="reporter.repositories.declarations" + type="string" + description="Repository for creating issues." + example="`OpenTermsArchive/demo-declarations`" +>}} + +{{< configOption + name="reporter.repositories.versions" + type="string" + description="Repository for versions." + example="`OpenTermsArchive/demo-versions`" +>}} + +{{< configOption + name="reporter.repositories.snapshots" + type="string" + description="Repository for snapshots." + example="`OpenTermsArchive/demo-snapshots`" +>}} + +{{< configOption + name="reporter.baseURL" + type="string" + description="Base URL for GitLab (if applicable)." + example="`https://gitlab.example.com`" +>}} + +{{< configOption + name="reporter.apiBaseURL" + type="string" + description="API base URL for GitLab (if applicable)." + example="`https://api.gitlab.example.com`" +>}} + +### Dataset + +The dataset section configures how datasets are published. + +{{< configOption + name="dataset.title" + type="string" + description="Title of the dataset." + default="`sandbox`" +>}} + +{{< configOption + name="dataset.versionsRepositoryURL" + type="string" + description="Repository URL for dataset releases." + default="`https://github.com/OpenTermsArchive/sandbox`" +>}} + +{{< configOption + name="dataset.publishingSchedule" + type="string" + description="Cron expression for dataset publishing." + default="`30 8 * * MON` (runs every Monday at 8:30 AM)" +>}} + +### Collection API + +The collection API section sets the parameters for the API server. + +{{< configOption + name="collection-api.api.port" + type="number" + description="Port number for the API server." + example="`8080`" + required=true +>}} + +{{< configOption + name="collection-api.api.basePath" + type="string" + description="Base path for API endpoints." + example="`/collection-api`" + required=true +>}} + +--- + +## Storage Repositories + +The storage repositories section set the parameters for supported backends for storing versions and snapshots, supporting Git and MongoDB. + +{{< configOption + name="storage.type" + type="string" + description="Type of storage backend." + default="`git`" + allowedValues="`git`, `mongo`" +>}} + +### Git + +The Git storage configuration allows to store versions in a Git repository. + +{{< configOption + name="storage.git.path" + type="string" + description="Path to the versions database directory." + default="`./data/versions`" +>}} + +{{< configOption + name="storage.git.publish" + type="boolean" + description="Boolean to push changes to the origin." + default="`false`" +>}} + +{{< configOption + name="storage.git.snapshotIdentiferTemplate" + type="string" + description="Template for snapshot ID reference. `%SNAPSHOT_ID` will be replaced with the actual snapshot ID." + default="`./data/snapshots/%SNAPSHOT_ID`" +>}} + +{{< configOption + name="storage.git.author.name" + type="string" + description="Author name for changes." + default="`Open Terms Archive Bot`" +>}} + +{{< configOption + name="storage.git.author.email" + type="string" + description="Author email for changes." + default="`bot@opentermsarchive.org`" +>}} + +### MongoDB + +The MongoDB storage configuration allows to store versions in a MongoDB database. + +{{< configOption + name="storage.mongo.connectionURI" + type="string" + description="MongoDB connection URI." + default="`mongodb://127.0.0.1:27017`" +>}} + +{{< configOption + name="storage.mongo.database" + type="string" + description="Database name." + default="`open-terms-archive`" +>}} + +{{< configOption + name="storage.mongo.collection" + type="string" + description="Collection name." + default="`snapshots`" +>}} diff --git a/content/collections/reference/environment-variables.md b/content/collections/reference/environment-variables.md new file mode 100644 index 00000000..16f2d415 --- /dev/null +++ b/content/collections/reference/environment-variables.md @@ -0,0 +1,26 @@ +--- +title: Environment variables +weight: 3 +--- + +## Environment variables + +This reference documentation details all available environment variables that can be used to configure sensitive information. These variables can be defined in a `.env` file at the root of the collection repository. + +### Engine + +{{< configOption name="OTA_ENGINE_SMTP_PASSWORD" type="string" description="SMTP password for email error notifications." >}} + +{{< configOption name="OTA_ENGINE_SENDINBLUE_API_KEY" type="string" description="API key for SendInBlue." >}} + +{{< configOption name="OTA_ENGINE_GITHUB_TOKEN" type="string" description="GitHub token for API access." >}} + +{{< configOption name="OTA_ENGINE_GITLAB_TOKEN" type="string" description="GitLab token for API access." >}} + +{{< configOption name="OTA_ENGINE_GITLAB_RELEASES_TOKEN" type="string" description="GitLab token for dataset releases." >}} + +--- + +### Federation API + +{{< configOption name="OTA_FEDERATION_API_SMTP_PASSWORD" type="string" description="SMTP password for email error notifications." >}} diff --git a/content/collections/governance.en.md b/content/collections/reference/governance.md similarity index 79% rename from content/collections/governance.en.md rename to content/collections/reference/governance.md index 769fc571..1019b9a7 100644 --- a/content/collections/governance.en.md +++ b/content/collections/reference/governance.md @@ -1,32 +1,35 @@ --- title: Governance -weight: 2 +weight: 4 +aliases: /collections/governance/ --- # Collection governance Setting up and maintaining a collection needs fulfilling certain tasks. These tasks are handled through roles. Each of these roles can be volunteer or paid, and can be handled by one single or several different entities. The Open Terms Archive core team provides processes and tools to support all of these roles. -## Host +This reference documentation details all available roles that can be involved in a collection's governance to ensure its proper operation and maintenance. + +### Host This role ensures that a server and internet access is available to run the engine on and fetch the terms, either by using its own infrastructure or renting a server on a hosting provider. -## Administrator +### Administrator This role takes responsibility for ensuring that the engine and associated software tools are functional and up to date. -## Curator +### Curator This role decides which services and terms are welcome in the collection. -## Maintainer +### Maintainer This role guarantees the quality of the tracked versions by reviewing contributions. -## Contributor +### Contributor This role adds declarations and keeps them up to date. -## Sponsor +### Sponsor This optional role supports the existence of the collection by funding other roles, providing agent time, political support, or any other relevant non-operational contribution. diff --git a/content/collections/reference/metadata.md b/content/collections/reference/metadata.md new file mode 100644 index 00000000..d4bbd1a7 --- /dev/null +++ b/content/collections/reference/metadata.md @@ -0,0 +1,190 @@ +--- +title: Metadata +weight: 1 +aliases: /collections/metadata/ +--- + +# Collection metadata + +This reference documentation details all available metadata fields that can be specified in a collection's metadata file. + +As an example, see the [complete metadata file](https://github.com/OpenTermsArchive/demo-declarations/blob/main/metadata.yml) of the [Demo collection](https://github.com/OpenTermsArchive/demo-declarations). + + +## Fields + +{{< configOption + name="id" + type="string" + description="Unique identifier derived from name (acronyms, dash-separated)" + example="`pga`" + required=true +>}} + +{{< configOption + name="name" + type="string" + description="Display name of the collection (max 3 words)" + example="`Platform Governance Archive`" + required=true +>}} + +{{< configOption + name="tagline" + type="string" + description="Concise description of collection topic" + example="`Major global social media services`" + required=true +>}} + +{{< configOption + name="languages" + type="Array of strings" + description="[ISO 639 language codes](https://en.wikipedia.org/wiki/ISO_639) allowed in collection" + example="`[en, fr, de]`" + required=true +>}} + +{{< configOption + name="jurisdictions" + type="Array of strings" + description="[ISO 3166-2 country codes](https://en.wikipedia.org/wiki/ISO_3166-2) for covered jurisdictions" + example="`[EU]`" + required=true +>}} + +{{< configOption + name="description" + type="string" + description="Detailed description of the collection" + example=`The **Platform Governance Archive** (PGA) collection tracks the terms of major global social media services. + +This data is maintained and analysed by the [Platform Governance Archive](https://www.platformgovernancearchive.org/) at the Universität Bremen's [Center for Media Communication and Information Research (ZeMKI)](https://www.uni-bremen.de/zemki). + +This initiative offers researchers, journalists and citizens the tools to analyze how platforms structure and regulate communication and interaction in our societies. + +It also aims to promote greater transparency and accountability of these powerful digital services.` +>}} + +{{< configOption + name="dataset" + type="url" + description="URL to released versions dataset" + example="`https://github.com/OpenTermsArchive/pga-versions/releases`" +>}} + +{{< configOption + name="declarations" + type="url" + description="URL to declarations repository" + example="`https://github.com/OpenTermsArchive/pga-declarations`" +>}} + +{{< configOption + name="versions" + type="url" + description="URL to versions repository" + example="`https://github.com/OpenTermsArchive/pga-versions`" +>}} + +{{< configOption + name="snapshots" + type="url" + description="URL to snapshots repository" + example="`https://github.com/OpenTermsArchive/pga-snapshots`" +>}} + +{{< configOption + name="logo" + type="url" + description="URL to the collection's logo. Optimized PNG transparent image (min width 240px)" + example="`https://opentermsarchive.org/images/collections/pga.png`" +>}} + +{{< configOption + name="donations" + type="url" + description="URL to the donations page" + example="`https://opencollective.com/opentermsarchive`" +>}} + +{{< configOption + name="trackingPeriods" + type="array of tracking periods objects" + description="Tracking periods object, see [Tracking periods](#tracking-periods) section" +>}} + +{{< configOption + name="governance" + type="array of entity objects" + description="Entity object, see [Entity](#entity) section" +>}} + +--- + +### Tracking periods + +{{< configOption + name="startDate" + type="date" + description="The date when tracking started (ISO 8601 format: YYYY-MM-DD)" + example="`2023-01-01`" + required=true +>}} + +{{< configOption + name="endDate" + type="date" + description="The date when tracking ended or will end. If not specified, tracking is ongoing." + example="`2024-12-31`" +>}} + +{{< configOption + name="schedule" + type="string" + description="A [cron expression](https://en.wikipedia.org/wiki/Cron#Cron_expression) that defines the tracking frequency" + example="`0 0 * * *`" + required=true +>}} + +{{< configOption + name="serverLocation" + type="string" + description="The geographic location of the tracking server (city name, ISO 3166-2 country code)" + example="`London, GB`" + required=true +>}} + +--- + +### Entity + +{{< configOption + name="name" + type="string" + description="Name of the organization" + example="`Open Terms Archive`" + required=true +>}} + +{{< configOption + name="url" + type="url" + description="Website URL of the organization" + example="`https://opentermsarchive.org/`" +>}} + +{{< configOption + name="logo" + type="url" + description="URL to the organization's logo. Optimized PNG transparent image (min width 240px)" + example="`https://opentermsarchive.org/images/logo/logo-open-terms-archive-black.png`" +>}} + +{{< configOption + name="roles" + type="string" + description="Roles of the entity within the governance" + allowedValues="`host`, `administrator`, `curator`, `maintainer`, `sponsor`" + required=true +>}} diff --git a/content/collections/tutorials/_index.md b/content/collections/tutorials/_index.md new file mode 100644 index 00000000..18583df4 --- /dev/null +++ b/content/collections/tutorials/_index.md @@ -0,0 +1,4 @@ +--- +title: Tutorials +weight: 1 +--- diff --git a/content/collections/tutorials/create.md b/content/collections/tutorials/create.md new file mode 100644 index 00000000..f54816ce --- /dev/null +++ b/content/collections/tutorials/create.md @@ -0,0 +1,93 @@ +--- +title: Create your first collection +weight: 1 +aliases: /collections/create/ +--- + +# Create your first collection + +This tutorial will guide you through creating your first Open Terms Archive collection. + +By the end, you'll have a working collection that tracks changes to a service's privacy policy. You will also have a basic understanding of how to create a collection. + +## Prerequisites + +- [Node.js](https://nodejs.org/en) is installed on your system. +- You have basic familiarity with the command line. +- You know how to use a text editor. + +## Create a collection + +### Step 1: Set up the directory structure + +1. Create a new directory: + ```bash + mkdir ota-tutorial-declarations + cd ota-tutorial-declarations + ``` + +2. Create a `declarations` directory inside the project. This is where you will declare the service and terms you want to track: + ```bash + mkdir declarations + ``` + +3. Create the configuration file for the collection: + ```bash + mkdir config + ``` + +### Step 2: Create the service declaration + +4. Create a file `declarations/Open Terms Archive.json` with the following content. For detailed instructions on how to structure it, follow the [Tracking terms tutorial]({{< relref "/terms/tutorials/track" >}}): + ```json + { + "name": "Open Terms Archive", + "documents": { + "Privacy Policy": { + "fetch": "https://opentermsarchive.org/en/privacy-policy", + "select": ".textcontent" + } + } + } + ``` + +### Step 2: Create the metadata file + +5. Create a file `metadata.yaml`: + ```yaml + id: ota-tutorial + name: Tutorial collection + tagline: Learn how to create a collection + description: | + A step-by-step tutorial collection that guides through creating an Open Terms Archive collection. + Track terms and conditions from websites while learning the basics of declarations, configuration, and metadata. + languages: [en] + jurisdictions: [EU] + ``` + +### Step 3: Create the configuration file + +6. Create a file `config/development.json` and set the tracking schedule to every minute: + ```json + { + "trackingSchedule": "* * * * *" + } + ``` + +### Step 4: Install and run the engine + +7. Install the Open Terms Archive engine: + ```bash + npm install --save @opentermsarchive/engine + ``` + +8. Start the scheduled tracking of the declared terms: + ```bash + npx ota track --schedule + ``` + +9. After one minute, check the results: + - Check the extracted version, which should contain the Privacy Policy of Open Terms Archive in Markdown format without any other content (no header, footer…): `./data/versions/Open Terms Archive/Privacy Policy.md`. + - Check the snapshot, which is the original HTML document of the Open Terms Archive Privacy Policy: `./data/snapshots/Open Terms Archive/Privacy Policy.html`. + +Congratulations! You have created your first collection. diff --git a/content/community/_index.md b/content/community/_index.md index 0aa00fc9..c1b85428 100644 --- a/content/community/_index.md +++ b/content/community/_index.md @@ -1,4 +1,4 @@ --- title: "Community" -weight: 7 +weight: 3 --- diff --git a/content/community/how-to/_index.md b/content/community/how-to/_index.md new file mode 100644 index 00000000..537c3cba --- /dev/null +++ b/content/community/how-to/_index.md @@ -0,0 +1,4 @@ +--- +title: How to +weight: 1 +--- diff --git a/content/community/adding-channels.png b/content/community/how-to/adding-channels.png similarity index 100% rename from content/community/adding-channels.png rename to content/community/how-to/adding-channels.png diff --git a/content/community/channel-options.jpg b/content/community/how-to/channel-options.jpg similarity index 100% rename from content/community/channel-options.jpg rename to content/community/how-to/channel-options.jpg diff --git a/content/community/edit-profile-page.jpg b/content/community/how-to/edit-profile-page.jpg similarity index 100% rename from content/community/edit-profile-page.jpg rename to content/community/how-to/edit-profile-page.jpg diff --git a/content/community/how-to-join-mattermost.md b/content/community/how-to/join-mattermost.md similarity index 92% rename from content/community/how-to-join-mattermost.md rename to content/community/how-to/join-mattermost.md index a1d28db8..5c3709bf 100644 --- a/content/community/how-to-join-mattermost.md +++ b/content/community/how-to/join-mattermost.md @@ -1,12 +1,11 @@ --- -title: How to join Mattermost +title: Join Mattermost +aliases: /community/how-to-join-mattermost/ --- # How to join Mattermost -Welcome to Open Terms Archive. This document provides guidelines for onboarding new members and contributors joining the Open Terms Archive community on Mattermost. - -Open Terms Archive welcomes various contributors, including individuals, sponsors, and other entities interested in transparency from the opaqueness of terms by Big Tech services. +The Open Terms Archive community collaborates on GitHub, but concentrates interactions on an instant messaging platform, Mattermost. Every contributor and user is welcome to join the community. ## Mattermost platform diff --git a/content/community/joinning-channels.png b/content/community/how-to/joinning-channels.png similarity index 100% rename from content/community/joinning-channels.png rename to content/community/how-to/joinning-channels.png diff --git a/content/community/landing-page.png b/content/community/how-to/landing-page.png similarity index 100% rename from content/community/landing-page.png rename to content/community/how-to/landing-page.png diff --git a/content/community/leave-the-canal.jpg b/content/community/how-to/leave-the-canal.jpg similarity index 100% rename from content/community/leave-the-canal.jpg rename to content/community/how-to/leave-the-canal.jpg diff --git a/content/community/profile-page.jpg b/content/community/how-to/profile-page.jpg similarity index 100% rename from content/community/profile-page.jpg rename to content/community/how-to/profile-page.jpg diff --git a/content/community/see-channels.jpg b/content/community/how-to/see-channels.jpg similarity index 100% rename from content/community/see-channels.jpg rename to content/community/how-to/see-channels.jpg diff --git a/content/concepts/_index.md b/content/concepts/_index.md new file mode 100644 index 00000000..78a4c739 --- /dev/null +++ b/content/concepts/_index.md @@ -0,0 +1,4 @@ +--- +title: Concepts and principles +weight: 10 +--- diff --git a/content/design-principles.en.md b/content/concepts/design-principles.md similarity index 98% rename from content/design-principles.en.md rename to content/concepts/design-principles.md index a6bb3134..83bfd12f 100644 --- a/content/design-principles.en.md +++ b/content/concepts/design-principles.md @@ -1,5 +1,7 @@ --- title: "Design principles" +weight: 2 +aliases: /design-principles/ --- # Design principles diff --git a/content/concepts/main.md b/content/concepts/main.md new file mode 100644 index 00000000..0e8ac472 --- /dev/null +++ b/content/concepts/main.md @@ -0,0 +1,82 @@ +--- +title: Main concepts +weight: 1 +--- + +## Main concepts + +Words in bold are [business domain names](https://en.wikipedia.org/wiki/Domain-driven_design). + +**Services** have **terms** written in **documents**, contractual (Terms of Services, Privacy Policy…) or not (Community Guidelines, Deceased User Policy…), that can change over time. Open Terms Archive enables users rights advocates, regulatory bodies and interested citizens to follow the **changes** to these **terms**, to be notified whenever a new **version** is published, to explore their entire **history** and to collaborate in analysing them. This free and open-source engine is developed to support these goals. + +### Collection + +Open Terms Archive is a decentralised system. It aims at enabling any entity to **track** **terms** on its own. To that end, the Open Terms Archive **engine** can be run on any server, thus making it a dedicated **instance**. An **instance** **tracks** **terms** within a single **collection**. + +A **collection** is characterised by a **scope** across **dimensions** that describe the **terms** it **tracks**, such as **language**, **jurisdiction** and **industry**. + +#### Example scope + +> The terms tracked in this collection are: + +> - Of dating services used in Europe. +> - In the European Union and Switzerland jurisdictions. +> - In English, unless no English version exists, in which case the primary official language of the jurisdiction of incorporation of the service operator will be used. + +### Federation + +In order to maximise discoverability, collaboration and political power, public **collections** are **federated** within a single ecosystem. This makes their data mutually discoverable and enables mutualising effort. + +### Terms types + +To distinguish between the different **terms** of a **service**, each has a **type**, such as “Terms of Service”, “Privacy Policy”, “Developer Agreement”… + +This **type** matches the topic, but not necessarily the title the **service** gives to it. Unifying the **types** enables comparing **terms** across **services**. + +> More information on terms types can be found in the [dedicated repository](https://github.com/OpenTermsArchive/terms-types). They are published on NPM under [`@opentermsarchive/terms-types`](https://www.npmjs.com/package/@opentermsarchive/terms-types), enabling standardisation and interoperability beyond the Open Terms Archive engine. + +### Declarations + +The **terms** that constitute a **collection** are defined in simple JSON files called **declarations**. + +A **declaration** also contains some metadata on the **service** on which the **terms** apply. + +> Here is an example declaration tracking the Privacy Policy of Open Terms Archive: +> +> ```json +> { +> "name": "Open Terms Archive", +> "documents": { +> "Privacy Policy": { +> "fetch": "https://opentermsarchive.org/en/privacy-policy", +> "select": ".textcontent" +> } +> } +> } +> ``` + +- - - + +## Add terms to a collection + +Open Terms Archive **acquires** **terms** to deliver an explorable **history** of **changes**. This can be done in two ways: + +1. For the present and future, by **tracking**. +2. For the past, by **importing** from an existing **fonds** such as [ToSBack](https://tosback.org), the [Internet Archive](https://web.archive.org/), [Common Crawl](https://commoncrawl.org) or any other in-house format. + +### Tracking terms + +In order to **track** the **changes** of **terms**, the **engine** **records** a **snapshot** of **documents** that contain them by **fetching** their web **location** several times a day. The **engine** then **extracts** a **version** from this **snapshot** by: + +1. **Selecting** the subset of the **document** (or **documents**) that contains the **terms** (instead of, e.g., navigation menus, footers, cookies banners…). +2. **Removing insignificant content**, that is residual content in this subset that is not part of the **terms** (e.g. ads, illustrative pictures, internal navigation links…). +3. **Filtering noise** that can emerge in the **terms** by preventing parts that change frequently from triggering false positives for **changes** (e.g. tracker identifiers in links, relative dates…). The **engine** can execute custom **filters** written in JavaScript to that end. + +After these steps, if **changes** are spotted in the resulting **terms**, a new **version** is **recorded**. + +Preserving **snapshots** enables recovering after the fact information potentially lost in the **extraction** step: if **declarations** were wrong, they can be **maintained** and corrected **versions** can be **extracted** from the original **snapshots**. + +### Importing terms + +Existing **fonds** can be prepared for easier analysis by unifying their format to the **Open Terms Archive dataset format**. This unique format enables building interoperable tools, fostering collaboration across reusers. +Such a dataset can be generated from **versions** alone. If **snapshots** and **declarations** can be retrieved from the **fonds** too, then a full-fledged **collection** can be created. diff --git a/content/deployment/_index.md b/content/deployment/_index.md new file mode 100644 index 00000000..7eb4a20a --- /dev/null +++ b/content/deployment/_index.md @@ -0,0 +1,4 @@ +--- +title: Deployment +weight: 7 +--- diff --git a/content/deployment/how-to/_index.md b/content/deployment/how-to/_index.md new file mode 100644 index 00000000..537c3cba --- /dev/null +++ b/content/deployment/how-to/_index.md @@ -0,0 +1,4 @@ +--- +title: How to +weight: 1 +--- diff --git a/content/deployment/how-to/deploy.md b/content/deployment/how-to/deploy.md new file mode 100644 index 00000000..ed9f6356 --- /dev/null +++ b/content/deployment/how-to/deploy.md @@ -0,0 +1,149 @@ +--- +title: Deploy a collection +weight: 1 +--- + +# How to deploy a collection + +This guide will help you deploy an Open Terms Archive collection to a server. + +## 1. Configure the server + +First, ensure your server provides unsupervised access: + +1. Check the SSH host key: + ```shell + ssh-keyscan --type=ed25519 + ``` + If no Ed25519 key appears, generate one on the server: + ```shell + sudo ssh-keygen --type=ed25519 --file=/etc/ssh/ssh_host_ed25519_key + sudo systemctl restart ssh + ``` + +2. Create a non-root user if needed: + ```shell + adduser + usermod --append --groups=sudo + ``` + +3. Grant passwordless sudo access: + ```shell + # Add to /etc/sudoers: + ALL=(ALL) NOPASSWD:ALL + ``` + +## 2. Set up the deployment configuration + +1. Clone the collection declarations repository: + ```shell + git clone https://github.com/OpenTermsArchive/-declarations.git + ``` + +2. Configure the inventory file `deployment/inventory.yml`: + ```yaml + : "your.server.ip" + ansible_user: "your_username" + ed25519_fingerprint: "your_ssh_fingerprint" + ``` + +3. Add the server fingerprint to GitHub: + - Go to `https://github.com/OpenTermsArchive/-declarations/settings/secrets/actions` + - Create a new secret named `SERVER_FINGERPRINT` with your Ed25519 fingerprint + +## 3. Configure SSH deployment keys + +1. On the server, generate a deployment key: + ```shell + ssh-keygen --type=ed25519 --quiet --passphrase="" --file=~/.ssh/ota-deploy + cat ~/.ssh/ota-deploy.pub >> ~/.ssh/authorized_keys + ``` + +2. Add the private key to GitHub: + - Go to the repository secrets + - Create `SERVER_SSH_KEY` with the private key content + +3. Back up the keys: + - Store both public and private keys in the shared password database + - Create an entry titled "Deployment SSH key" in the collection folder + +## 4. Set up GitHub permissions + +1. Create a fine-grained GitHub token: + - Log in as OTA-Bot + - Create a new token at github.com/settings/personal-access-tokens/new + - Set repository access for both declarations and versions repos + - Grant "Contents" and "Issues" write permissions + +2. Back up the token: + - Store it in the shared password database under "GitHub Token" + +3. Get the token approved: + - Have an organization admin approve the token request + +## 5. Configure secrets + +1. Generate and store a vault key: + - Generate a secure password without quotes/backticks + - Store it in the password database + - Create `deployment/vault.key` with the password + - Add it as `ANSIBLE_VAULT_KEY` in GitHub secrets + +2. Store GitHub token: + ``` + # In deployment/.env: + OTA_ENGINE_GITHUB_TOKEN=your_token + ``` + +3. Encrypt the `.env` file: + ```shell + ansible-vault encrypt .env + ``` + +## 6. Set up collection-specific SSH key + +1. Generate a new key: + ```shell + ssh-keygen --type=ed25519 --comment=bot@opentermsarchive.org --passphrase="" --file=./-key + ``` + +2. Encrypt and store the private key: + ```shell + # Copy private key to deployment/github-bot-private-key + ansible-vault encrypt github-bot-private-key + ``` + +3. Add the public key to OTA-Bot's GitHub account: + - Go to github.com/settings/ssh/new + - Add the public key with title " collection" + +## 7. Configure email notifications + +1. Generate SMTP credentials: + - Create a new SMTP key in Brevo + - Name it " collection" + +2. Store the credentials: + ```shell + # In deployment/.env: + OTA_ENGINE_SMTP_PASSWORD=your_smtp_key + ``` + +3. Encrypt the `.env` file: + ```shell + ansible-vault encrypt .env + ``` + +## 8. Test the deployment + +1. Via GitHub Actions: + - Check that the `deploy` action completes successfully + +2. Via local deployment: + ```shell + cd -declarations/deployment + ansible-galaxy collection install --requirements-file requirements.yml + ansible-playbook opentermsarchive.deployment.deploy + ``` + +If all steps complete successfully, your collection should now be properly deployed and running. diff --git a/content/deployment/how-to/setup-github-teams.md b/content/deployment/how-to/setup-github-teams.md new file mode 100644 index 00000000..f2cf3b8f --- /dev/null +++ b/content/deployment/how-to/setup-github-teams.md @@ -0,0 +1,50 @@ +--- +title: Set up GitHub teams +weight: 2 +--- + +# How to set up GitHub teams for a collection + +This guide explains how to configure GitHub teams and permissions for a new collection in the Open Terms Archive organization. For third-party collections outside the organization, you can handle permissions according to your own needs. + +## Prerequisites + +- Admin access to the OpenTermsArchive GitHub organization +- The collection name and icon +- List of team members to add +- The three collection repositories (declarations, snapshots, versions) already created + +## Steps + +1. Create the collection team: + 1. Go to [GitHub's new team page](https://github.com/orgs/OpenTermsArchive/new-team) + 2. Enter the collection name as the team name + 3. Upload the collection icon from the website as the team avatar + 4. Set the description to: "Maintainers of the `` collection" + 5. Click "Create team" + +2. Add team members: + 1. Navigate to the team's "Members" tab + 2. Click "Add member" + 3. Search and select each member to add + 4. Confirm their addition + +3. Configure repository access for the collection team: + 1. Go to the team's "Repositories" tab + 2. Add the declarations repository with "Maintain" access + 3. Add the snapshots repository with "Triage" access + 4. Add the versions repository with "Triage" access + + > Note: Snapshots and versions repositories are limited to "Triage" access to prevent data corruption + +4. Configure bot access: + 1. Navigate to the Bots team + 2. Go to the "Repositories" tab + 3. Add all three repositories (declarations, snapshots, versions) with "Write" access + +## Verification + +After completing these steps, verify that: +- The team appears in the organization's teams list +- Team members can access all repositories with appropriate permissions +- The Bots team has write access to all repositories diff --git a/content/memos/_index.md b/content/federation/_index.md similarity index 50% rename from content/memos/_index.md rename to content/federation/_index.md index d63eb02d..9a8971f5 100644 --- a/content/memos/_index.md +++ b/content/federation/_index.md @@ -1,4 +1,4 @@ --- -title: "Memos" +title: Federation weight: 6 --- diff --git a/content/federation/benefits.md b/content/federation/benefits.md new file mode 100644 index 00000000..75c4d5d0 --- /dev/null +++ b/content/federation/benefits.md @@ -0,0 +1,26 @@ +--- +title: Benefits +weight: 1 +aliases: /collections/federation/ +--- + +# Open Terms Archive federation + +Open Terms Archive is a decentralised system. It aims at enabling any entity set up their own collections and track terms on their own. + +In order to maximise **discoverability**, **collaboration** and **political power**, public collections are federated within a single ecosystem. This makes their data mutually discoverable and enables mutualising effort. + +## Benefits of joining the federation + +A collection that joins the federation enjoys the following benefits: + +1. Visibility on the Open Terms Archive website lists of collections and datasets. +2. Access to the Open Terms Archive GitHub organisation, administered by the Open Terms Archive core team. +3. Collection logo provided by the Open Terms Archive core team. +4. Referencing in the official [collections list](https://opentermsarchive.org/collections.json), enabling off-the-shelf discovery in the [Federation API]({{< relref "api/federation" >}}). +5. Referencing in the official [datasets list](https://opentermsarchive.org/datasets), providing visibility to analysts. +6. Dedicated channel on the Open Terms Archive instant messaging system. +7. API uptime tracking. +8. Public announcement through all Open Terms Archive communication channels upon joining. + +By joining the Open Terms Archive federation, your collection becomes part of a **dynamic network** that leverages **shared resources** and **collective visibility** to drive **greater impact**. diff --git a/content/federation/how-to/_index.md b/content/federation/how-to/_index.md new file mode 100644 index 00000000..537c3cba --- /dev/null +++ b/content/federation/how-to/_index.md @@ -0,0 +1,4 @@ +--- +title: How to +weight: 1 +--- diff --git a/content/federation/how-to/join.md b/content/federation/how-to/join.md new file mode 100644 index 00000000..7e07c071 --- /dev/null +++ b/content/federation/how-to/join.md @@ -0,0 +1,12 @@ +--- +title: Join the federation +weight: 3 +--- + +# How to join the federation + +An official request for joining the federation can be sent by the collection administrator or curator to the core team over email, GitHub or instant messages. That request should contain all necessary content or links that enable assessing the criteria. + +The core team will confirm reception of the request and assess the criteria within 4 weeks. The result of the assessment will be provided to the requester through the same means the request was sent, along with a positive decision or a list of improvements that need to be done before a new request can be sent. + +Once a positive decision has been issued, the core team will provide the requester with a timeline for joining the federation, on which they will jointly agree. diff --git a/content/federation/reference/_index.md b/content/federation/reference/_index.md new file mode 100644 index 00000000..6f4fd0f8 --- /dev/null +++ b/content/federation/reference/_index.md @@ -0,0 +1,4 @@ +--- +title: Reference +weight: 2 +--- diff --git a/content/federation/reference/criteria.md b/content/federation/reference/criteria.md new file mode 100644 index 00000000..a4c5a831 --- /dev/null +++ b/content/federation/reference/criteria.md @@ -0,0 +1,27 @@ +--- +title: Criteria +weight: 3 +--- + +# Federation criteria + +A **collection** can **join** the Open Terms Archive **federation** if it abides by the following quality criteria: + +1. Clearly defined [collection metadata]({{< relref "collections/reference/metadata" >}}). +2. Clearly defined [collection governance]({{< relref "collections/reference/governance" >}}). +3. The vast majority of **versions** are readable, as evidenced by a sample assessment. +4. **Frequency** of at least one track a day, as evidenced by snapshots. +5. Public and open-licensed **declarations**, as evidenced by the `LICENSE` file in the declarations repository. +6. Public and open-licensed **snapshots**, as evidenced by the `LICENSE` file in the snapshots repository. +7. Public and open-licensed **versions**, as evidenced by the `LICENSE` file in the versions repository. +8. Regular, public, and open-licensed **dataset** releases, as evidenced by the `LICENSE` file in the datasets. +9. Publicly accessible API with median response time under 20ms, as evidenced by uptime tracking logs. +10. Abide by all Open Terms Archive software and data licenses. + +For detailed instructions on joining the federation, refer to the [How to Join the Federation guide]({{< relref "federation/how-to/join" >}}). + +## Disclaimer + +Please note that establishing the federation is an ongoing effort. While criteria are assessed and refined to strike the right balance between accessibility and quality, the Open Terms Archive core team is responsible for assessing which collections may join the federation, and has the right to update the criteria as requests for joining are made. + +The core team has the right to re-assess the federated status of any collection at any given time. In case the federated status of a collection is changed, the collection administrator will be informed immediately, and if possible before the action is made. diff --git a/content/terms/_index.md b/content/terms/_index.md index 396ccf30..420599f2 100644 --- a/content/terms/_index.md +++ b/content/terms/_index.md @@ -1,4 +1,4 @@ --- title: "Terms" -weight: 2 +weight: 4 --- diff --git a/content/terms/declarations-maintenance.en.md b/content/terms/declarations-maintenance.md similarity index 67% rename from content/terms/declarations-maintenance.en.md rename to content/terms/declarations-maintenance.md index b3fff54a..c36c22f0 100644 --- a/content/terms/declarations-maintenance.en.md +++ b/content/terms/declarations-maintenance.md @@ -1,6 +1,6 @@ --- title: "Declarations maintenance" -weight: 3 +weight: 6 --- # Declarations maintenance @@ -88,40 +88,3 @@ export const removeSharesButton = [ }, ]; ``` - -## How to terminate a service - -If the service provider stops offering a service, the associated terms will become unavailable. To mark that service termination in Open Terms Archive and ensure tracking tentatives are stopped, while maintaining the possibility to explore the history: - -1. Move the existing documents declaration to the service [history file](#terms-declaration-history). -2. Update the declaration to stop tracking all terms, by removing every `` entries from the `documents` key in the declaration: - -```json -{ - "name": "", - "documents": {} -} -``` - -## How to rename a service - -The consensus is to consider that a service provider renaming a service (for example, `Twitter` to `X`) is akin to terminating the previous service and opening a new one. Therefore, to apply a service renaming, open a pull request that both [terminates the previous service](#how-to-terminate-a-service) and adds a new [service declaration]({{< relref "terms/tracking-new-terms#declaring-a-new-service" >}}) with the new service name. You can reuse the `documents` part of the original declaration, but should double-check that the selectors and URLs still match, as a service rename is most often accompanied by a new page layout, a new domain name, and sometimes entirely new terms. - -## How to get the `validUntil` date from an issue - -1. Go to the open issue and scroll to the last action from `OTA-Bot` (comment or issue opening). - - ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment.png) - - Note that sometimes: - - - the date is not quite precise, it can be written "last month" - ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment-last-month.png) - - the last comment may be the announcement of the reopening of the issue - ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment-reopened-issue.png) - -2. On the date of this comment, right-click "inspect" to open your browser's code inspector. - -3. The value to use as `validUntil` is the value of the `datetime` attribute from the `` element. - - ![]({{< relref "." >}}declarations-maintenance-inspector-open-get-datetime.png) diff --git a/content/guidelines/_index.md b/content/terms/guidelines/_index.md similarity index 100% rename from content/guidelines/_index.md rename to content/terms/guidelines/_index.md diff --git a/content/guidelines/choosing-selectors.en.md b/content/terms/guidelines/choosing-selectors.md similarity index 99% rename from content/guidelines/choosing-selectors.en.md rename to content/terms/guidelines/choosing-selectors.md index f3e9a6cc..5b309088 100644 --- a/content/guidelines/choosing-selectors.en.md +++ b/content/terms/guidelines/choosing-selectors.md @@ -1,5 +1,6 @@ --- title: "Choosing selectors" +aliases: /guidelines/choosing-selectors/ --- # Choosing selectors diff --git a/content/guidelines/declaring.en.md b/content/terms/guidelines/declaring.md similarity index 89% rename from content/guidelines/declaring.en.md rename to content/terms/guidelines/declaring.md index 14cc60c0..019a5a29 100644 --- a/content/guidelines/declaring.en.md +++ b/content/terms/guidelines/declaring.md @@ -1,10 +1,9 @@ --- -title: "Declaring documents" +title: "Declaring terms" +aliases: /guidelines/declaring/ --- -This document presents practical guidelines, is edited collaboratively and is not normative. Normative constraints are exposed in the [Terms reference]({{< relref "terms/reference" >}}). - -# Declaring documents +# Declaring terms ## Service name @@ -31,7 +30,7 @@ This document presents practical guidelines, is edited collaboratively and is no ### Provider prefixing -- If you encounter a document you want to add to a service, yet find that it would override an already-declared document for this service such as Terms of Service or Privacy Policy, and that the only solution you see would be to create a new terms type that would contain the name of the feature, then it is likely you should declare a new service, potentially duplicating existing documents. +- If you encounter terms you want to add to a service, yet find that it would override already-declared terms for this service such as Terms of Service or Privacy Policy, and that the only solution you see would be to create a new terms type that would contain the name of the feature, then it is likely you should declare a new service, potentially duplicating existing terms. > Example: the Facebook Community Payments terms are Terms of Service. The only way to declare them in the Facebook service would be to add a “Community Payments Terms” terms type as they would otherwise conflict with Facebook's Terms of Service. It is better to declare a new service called “Facebook Payments” with its own Terms of Service. It turns out that this service also has a developer agreement, independent from the main Facebook service. @@ -49,13 +48,13 @@ Noise is unwanted content in versions. ### Irrelevant content -The first type of noise we try to remove is content that is not **relevant legally speaking**, and that harms **document readability**. +The first type of noise we try to remove is content that is not **relevant legally speaking**, and that harms **terms readability**. **CSS selectors** are a first step as they permit to select an area instead of the whole page, but they let pass through content such as headers, footers, buttons, drop-down lists... **Filtering** permits to get rid of the remaining irrelevant content. -A drop-down list let user select which document he would like to see but this list doesn't interest us in the final document. +A drop-down list lets users select which terms they would like to see but this list doesn't interest us in the final version. **HTML file :** diff --git a/content/guidelines/reviewing.en.md b/content/terms/guidelines/reviewing.md similarity index 98% rename from content/guidelines/reviewing.en.md rename to content/terms/guidelines/reviewing.md index 1eef2c7b..1f58e455 100644 --- a/content/guidelines/reviewing.en.md +++ b/content/terms/guidelines/reviewing.md @@ -1,5 +1,6 @@ --- title: "Reviewing contributions" +aliases: /guidelines/reviewing/ --- # Reviewing contributions @@ -52,10 +53,10 @@ Your focus should be on two aspects: accuracy and quality. 1. Click on the _Inspect the declaration_ link to view the declaration in a graphical user interface. 2. Use the link provided in the URL section of the contribution tool to check out the original document. -3. Verify that the name of the service matches the JSON file and complies with the [guidelines]({{< relref "guidelines/declaring#service-name" >}}). +3. Verify that the name of the service matches the JSON file and complies with the [guidelines]({{< relref "terms/guidelines/declaring#service-name" >}}). 4. Quickly scan the document to ensure that the correct term type has been selected. To determine the term type, consider who the intended audience is and what the document is discussing. You can also refer to the [terms types list](https://github.com/OpenTermsArchive/terms-types/blob/main/termsTypes.json) to find the best term type for the document. 5. Confirm that the selected area of the document contains only one term type and does not include any other types. -6. Check both the significant and insignificant parts of the document. Ensure that the suggested selectors abide by the [selectors guidelines]({{< relref "guidelines/choosing-selectors" >}}). +6. Check both the significant and insignificant parts of the document. Ensure that the suggested selectors abide by the [selectors guidelines]({{< relref "terms/guidelines/choosing-selectors" >}}). - Ensure that the significant parts do not include navigation items, contact links, or other insignificant details that may cause confusion by triggering a change detection when the legal terms have actually not been updated. 7. Verify the version of the document in the contribution tool by clicking on the _Verify version_ button. 8. Ensure that all checks generated by the OTA-bot are manually checked. @@ -77,7 +78,7 @@ For pull requests that update declarations, you should focus should be on two th ### Step-by-step Review Guide 1. Click on the inspect the declaration suggestion link to view contribution using the contribution tool. -2. Check both the significant and insignificant parts of the document. Ensure that the suggested selectors abide by the [selectors guidelines]({{< relref "guidelines/choosing-selectors" >}}). +2. Check both the significant and insignificant parts of the document. Ensure that the suggested selectors abide by the [selectors guidelines]({{< relref "terms/guidelines/choosing-selectors" >}}). 3. Verify the version of the document in the contribution tool by clicking on the `verify version` button. 4. Open the issue linked with the pull request. Confirm the date when the declaration was last tracked from the bot's comment. 5. Compare it with the `validUntil` property in the history file under the `Files changes` section of the pull request. If the dates are the same, proceed to approve the pull request. diff --git a/content/guidelines/targeting.en.md b/content/terms/guidelines/targeting.md similarity index 90% rename from content/guidelines/targeting.en.md rename to content/terms/guidelines/targeting.md index 3805ac4f..7f1e4cfe 100644 --- a/content/guidelines/targeting.en.md +++ b/content/terms/guidelines/targeting.md @@ -1,5 +1,6 @@ --- title: "Targeting" +aliases: /guidelines/targeting/ --- # What to track? diff --git a/content/terms/how-to-be-notified-of-terms-changes.en.md b/content/terms/how-to-be-notified-of-terms-changes.en.md deleted file mode 100644 index 781b90ba..00000000 --- a/content/terms/how-to-be-notified-of-terms-changes.en.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: "How to be notified of terms changes" -aliases: /subscribe-rss/ ---- - -# How to be notified of terms changes - -An RSS feed is a type of web page that contains information about the latest content published by a website, such as the date of publication and the address where you can view it. When this resource is updated, a feed reader app automatically notifies you and you can see the update. You can receive notification for a specific service or document by subscribing to RSS feeds. - -- - - - -## For a specific document - -To find out the address of the RSS feed you want to subscribe to: - -1. [Navigate]({{< relref "terms/how-to-navigate-history" >}}) to the page with the history of changes you are interested in. - - For example, for the GitHub Privacy Policy of the Demo instance, this would be [this page](https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md). -2. Copy the address of that page from your browser’s address bar. - - For example, for the GitHub Privacy Policy of the Demo instance, this would be `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md`. -3. Append `.atom` at the end of this address. - - For example, for the GitHub Privacy Policy of the Demo instance, this would become `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md.atom`. -4. Subscribe your RSS feed reader to the resulting address. - -- - - - -## For all the documents of a service - -Simply navigate to the history of changes for the service you are interested in and follow the same procedure as for a specific document. - -For example, for all GitHub documents of the Demo instance, you would obtain `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub.atom`. - -- - - - -## For all the documents of an instance - -Simply append `commits.atom` to the URL of the repository. - -For example, for all documents of the Demo instance, you would use `https://github.com/OpenTermsArchive/demo-versions/commits.atom`. diff --git a/content/terms/how-to/_index.md b/content/terms/how-to/_index.md new file mode 100644 index 00000000..6b204c17 --- /dev/null +++ b/content/terms/how-to/_index.md @@ -0,0 +1,4 @@ +--- +title: How to +weight: 2 +--- diff --git a/content/terms/how-to-add-terms-using-with-the-graphical-contribution-interface.md b/content/terms/how-to/add-terms-using-UI.md similarity index 83% rename from content/terms/how-to-add-terms-using-with-the-graphical-contribution-interface.md rename to content/terms/how-to/add-terms-using-UI.md index aca6e5c8..813541b7 100644 --- a/content/terms/how-to-add-terms-using-with-the-graphical-contribution-interface.md +++ b/content/terms/how-to/add-terms-using-UI.md @@ -1,5 +1,7 @@ --- -title: "How to add terms with the graphical contribution interface" +title: Add terms with the graphical contribution interface +linkTitle: Add terms with UI +aliases: /terms/how-to-add-terms-using-with-the-graphical-contribution-interface/ --- # How to add terms with the graphical contribution interface diff --git a/content/terms/how-to/be-notified.md b/content/terms/how-to/be-notified.md new file mode 100644 index 00000000..b9b9dd5f --- /dev/null +++ b/content/terms/how-to/be-notified.md @@ -0,0 +1,40 @@ +--- +title: Be notified of terms changes +aliases: + - /subscribe-rss/ + - /terms/how-to-be-notified-of-terms-changes/ +--- + +# How to be notified of terms changes + +An RSS feed is a type of web page that contains information about the latest content published by a website, such as the date of publication and the address where you can view it. When this resource is updated, a feed reader app automatically notifies you and you can see the update. You can receive notification for a specific service or document by subscribing to RSS feeds. + +- - - + +## For specific terms + +To find out the address of the RSS feed you want to subscribe to: + +1. [Navigate]({{< relref "/analysis/how-to/navigate-history" >}}) to the page with the history of changes you are interested in. + - For example, for the GitHub Privacy Policy of the Demo collection, this would be [this page](https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md). +2. Copy the address of that page from your browser’s address bar. + - For example, for the GitHub Privacy Policy of the Demo collection, this would be `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md`. +3. Append `.atom` at the end of this address. + - For example, for the GitHub Privacy Policy of the Demo collection, this would become `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub/Privacy%20Policy.md.atom`. +4. Subscribe your RSS feed reader to the resulting address. + +- - - + +## For all the terms of a service + +Simply navigate to the history of changes for the service you are interested in and follow the same procedure as for a specific document. + +For example, for all terms of the Demo collection, you would obtain `https://github.com/OpenTermsArchive/demo-versions/commits/main/GitHub.atom`. + +- - - + +## For all terms of a collection + +Simply append `commits.atom` to the URL of the repository. + +For example, for all terms of the Demo collection, you would use `https://github.com/OpenTermsArchive/demo-versions/commits.atom`. diff --git a/content/terms/declarations-maintenance-inspector-open-get-datetime.png b/content/terms/how-to/declarations-maintenance-inspector-open-get-datetime.png similarity index 100% rename from content/terms/declarations-maintenance-inspector-open-get-datetime.png rename to content/terms/how-to/declarations-maintenance-inspector-open-get-datetime.png diff --git a/content/terms/declarations-maintenance-ota-bot-comment-last-month.png b/content/terms/how-to/declarations-maintenance-ota-bot-comment-last-month.png similarity index 100% rename from content/terms/declarations-maintenance-ota-bot-comment-last-month.png rename to content/terms/how-to/declarations-maintenance-ota-bot-comment-last-month.png diff --git a/content/terms/declarations-maintenance-ota-bot-comment-reopened-issue.png b/content/terms/how-to/declarations-maintenance-ota-bot-comment-reopened-issue.png similarity index 100% rename from content/terms/declarations-maintenance-ota-bot-comment-reopened-issue.png rename to content/terms/how-to/declarations-maintenance-ota-bot-comment-reopened-issue.png diff --git a/content/terms/declarations-maintenance-ota-bot-comment.png b/content/terms/how-to/declarations-maintenance-ota-bot-comment.png similarity index 100% rename from content/terms/declarations-maintenance-ota-bot-comment.png rename to content/terms/how-to/declarations-maintenance-ota-bot-comment.png diff --git a/content/terms/how-to/get-the-validUntil-date-from-an-issue.md b/content/terms/how-to/get-the-validUntil-date-from-an-issue.md new file mode 100644 index 00000000..4478d4b2 --- /dev/null +++ b/content/terms/how-to/get-the-validUntil-date-from-an-issue.md @@ -0,0 +1,25 @@ +--- +title: How to get the `validUntil` +linkTitle: Get the `validUntil` date +--- + +# How to get the `validUntil` date + +## From a GitHub issue + +1. Go to the open issue and scroll to the last action from `OTA-Bot` (comment or issue opening). + + ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment.png) + + Note that sometimes: + + - the date is not quite precise, it can be written "last month" + ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment-last-month.png) + - the last comment may be the announcement of the reopening of the issue + ![]({{< relref "." >}}declarations-maintenance-ota-bot-comment-reopened-issue.png) + +2. On the date of this comment, right-click "inspect" to open your browser's code inspector. + +3. The value to use as `validUntil` is the value of the `datetime` attribute from the `` element. + + ![]({{< relref "." >}}declarations-maintenance-inspector-open-get-datetime.png) diff --git a/content/terms/how-to/rename-a-service.md b/content/terms/how-to/rename-a-service.md new file mode 100644 index 00000000..2a22e189 --- /dev/null +++ b/content/terms/how-to/rename-a-service.md @@ -0,0 +1,8 @@ +--- +title: How to rename a service +linkTitle: Rename a service +--- + +# How to rename a service + +The consensus is to consider that a service provider renaming a service (for example, `Twitter` to `X`) is akin to terminating the previous service and opening a new one. Therefore, to apply a service renaming, open a pull request that both [terminates the previous service](#how-to-terminate-a-service) and adds a new [service declaration]({{< relref "terms/how-to/track-new-terms#declaring-a-new-service" >}}) with the new service name. You can reuse the `documents` part of the original declaration, but should double-check that the selectors and URLs still match, as a service rename is most often accompanied by a new page layout, a new domain name, and sometimes entirely new terms. diff --git a/content/terms/how-to/terminate-a-service.md b/content/terms/how-to/terminate-a-service.md new file mode 100644 index 00000000..764cd44d --- /dev/null +++ b/content/terms/how-to/terminate-a-service.md @@ -0,0 +1,18 @@ +--- +title: How to terminate a service +linkTitle: Terminate a service +--- + +# How to terminate a service + +If the service provider stops offering a service, the associated terms will become unavailable. To mark that service termination in Open Terms Archive and ensure tracking tentatives are stopped, while maintaining the possibility to explore the history: + +1. Move the existing documents declaration to the service [history file](#terms-declaration-history). +2. Update the declaration to stop tracking all terms, by removing every `` entries from the `documents` key in the declaration: + +```json +{ + "name": "", + "documents": {} +} +``` diff --git a/content/terms/tracking-new-terms.en.md b/content/terms/how-to/track-new-terms.md similarity index 89% rename from content/terms/tracking-new-terms.en.md rename to content/terms/how-to/track-new-terms.md index c232076d..157e2ce6 100644 --- a/content/terms/tracking-new-terms.en.md +++ b/content/terms/how-to/track-new-terms.md @@ -1,10 +1,10 @@ --- -title: "Tracking new terms" +title: Track new terms weight: 2 aliases: /contributing-terms/ --- -# Tracking new terms +# How to track new terms Tracking terms is done by _declaring_ them and the service they are associated with. Such a declaration is achieved by editing JSON files in the [`declarations`](https://github.com/OpenTermsArchive/contrib-declarations/tree/main/declarations) folder. @@ -34,7 +34,7 @@ The service name is exposed to end users. It should reflect as closely as possib - _Example: `Firebase` (by Google) → `Firebase`_. - _Example: `App Store` (by Apple) → `App Store`_. -> If you have a hard time finding the service name, check out the [practical guidelines to find the service name]({{< relref "guidelines/declaring#service-name" >}}), and feel free to mention your uncertainties in the pull request! We will help you improve the service name if necessary 🙂 +> If you have a hard time finding the service name, check out the [practical guidelines to find the service name]({{< relref "/terms/guidelines/declaring#service-name" >}}), and feel free to mention your uncertainties in the pull request! We will help you improve the service name if necessary 🙂 ## Service ID @@ -54,7 +54,7 @@ The service ID is exposed to developers. It should be easy to handle with script - _Example: `App Store` → `App Store`_. - _Example: `DeviantArt` → `DeviantArt`_. -> If you have a hard time defining the service ID, check out the [practical guidelines to derive the ID from the service name]({{< relref "guidelines/declaring#service-id" >}}), and feel free to mention your uncertainties in the pull request! We will help you improve the service ID if necessary 🙂 +> If you have a hard time defining the service ID, check out the [practical guidelines to derive the ID from the service name]({{< relref "/terms/guidelines/declaring#service-id" >}}), and feel free to mention your uncertainties in the pull request! We will help you improve the service ID if necessary 🙂 > More details on the ID and naming constraints and recommendations can be found in the relevant [decision record](https://github.com/OpenTermsArchive/engine/blob/main/decision-records/0001-service-name-and-id.md). diff --git a/content/terms/reference/_index.md b/content/terms/reference/_index.md new file mode 100644 index 00000000..63aae4a6 --- /dev/null +++ b/content/terms/reference/_index.md @@ -0,0 +1,4 @@ +--- +title: Reference +weight: 3 +--- diff --git a/content/terms/reference.en.md b/content/terms/reference/declaration.md similarity index 98% rename from content/terms/reference.en.md rename to content/terms/reference/declaration.md index dd76df72..c9558b62 100644 --- a/content/terms/reference.en.md +++ b/content/terms/reference/declaration.md @@ -1,10 +1,9 @@ --- -title: "Reference" -html_title: "Terms reference" -weight: 4 +title: "Declaration format" +aliases: "/terms/reference/" --- -# Terms reference +# Terms declaration format reference Terms are declared in a service declaration file, under the `documents` property. @@ -33,7 +32,7 @@ Let’s start by defining these keys! This property should simply contain the URL at which the terms you want to track can be downloaded. HTML and PDF files are supported. -When terms coexist in different languages and jurisdictions, please refer to the [scope of the collection]({{< relref "collections/metadata" >}}) to which you are contributing. This scope is usually defined in the README. +When terms coexist in different languages and jurisdictions, please refer to the [scope of the collection]({{< relref "collections/reference/metadata" >}}) to which you are contributing. This scope is usually defined in the README. ## `select` @@ -197,7 +196,7 @@ Each filter is exposed as a named function export that takes a `document` parame > The `document` parameter is actually a [JSDOM](https://github.com/jsdom/jsdom) document instance. -You can learn more about usual noise and ways to handle it [in the guidelines]({{< relref "guidelines/declaring#usual-noise" >}}). +You can learn more about usual noise and ways to handle it [in the guidelines]({{< relref "/terms/guidelines/declaring#usual-noise" >}}). ### Example diff --git a/content/terms/tutorials/_index.md b/content/terms/tutorials/_index.md new file mode 100644 index 00000000..18583df4 --- /dev/null +++ b/content/terms/tutorials/_index.md @@ -0,0 +1,4 @@ +--- +title: Tutorials +weight: 1 +--- diff --git a/content/terms/tutorials/track.md b/content/terms/tutorials/track.md new file mode 100644 index 00000000..f89efe25 --- /dev/null +++ b/content/terms/tutorials/track.md @@ -0,0 +1,79 @@ +--- +title: Track your first terms +--- + +# Track your first terms + +This tutorial will guide you through tracking your first terms. + +By the end, you'll have tracked a service's privacy policy. You will also have a basic understanding of how to run the engine to track terms. + +## Prerequisites + +- [Node.js](https://nodejs.org/en) is installed on your system. +- You have basic familiarity with the command line. +- You have basic familiarity with HTML and CSS selectors. +- You know how to use a text editor. + +## Track terms + +### Step 1: Set up the directory structure + +1. Create a new directory: + ```bash + mkdir ota-tutorial-declarations + cd ota-tutorial-declarations + ``` + +2. Create a `declarations` directory inside the project. This is where you will declare the service and terms you want to track: + ```bash + mkdir declarations + ``` + +### Step 2: Create the service declaration + +For this tutorial, we will use the Privacy Policy of Open Terms Archive as an example. + +Create a file `declarations/Open Terms Archive.json`. The name of the file is the name of the service that will be tracked. The first thing to declare is the tracked service name: + ```json + { + "name": "Open Terms Archive" + } + ``` + +Now, you can add terms you want to track to the declaration. For this example, we will use the Privacy Policy. + +You can go on the Open Terms Archive [website](https://opentermsarchive.org/) and copy the URL of its [Privacy Policy](https://opentermsarchive.org/en/privacy-policy/) to fill the `fetch` field. + +And you can inspect the HTML of the page to get the selector of the content you want to extract to fill the `select` field. + +The resulting declaration should look something like this: + ```json + { + "name": "Open Terms Archive", + "documents": { + "Privacy Policy": { + "fetch": "https://opentermsarchive.org/en/privacy-policy", + "select": ".textcontent" + } + } + } + ``` + +### Step 3: Install and run the engine + +1. Install the Open Terms Archive engine: + ```bash + npm install --save @opentermsarchive/engine + ``` + +2. Start a one time tracking of the declared terms: + ```bash + npx ota track + ``` + +3. Verify the results: + - Check the extracted version, which should contain the Privacy Policy of Open Terms Archive in Markdown format without any other content (no header, footer…): `./data/versions/Open Terms Archive/Privacy Policy.md`. + - Check the snapshot, which is the original HTML document of the Open Terms Archive Privacy Policy: `./data/snapshots/Open Terms Archive/Privacy Policy.html`. + +Congratulations! You have tracked your first terms. diff --git a/package-lock.json b/package-lock.json index 0a00a23f..8a320678 100644 --- a/package-lock.json +++ b/package-lock.json @@ -12,7 +12,6 @@ "minireset.css": "^0.0.7", "postcss": "^8.4.19", "postcss-cli": "^10.1.0", - "postcss-import": "^15.0.1", "postcss-mixins": "^9.0.4", "postcss-preset-env": "^7.8.3" }, @@ -1946,7 +1945,8 @@ "node_modules/function-bind": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz", - "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==" + "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==", + "dev": true }, "node_modules/function.prototype.name": { "version": "1.1.5", @@ -2236,6 +2236,7 @@ "version": "1.0.3", "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", "integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==", + "dev": true, "dependencies": { "function-bind": "^1.1.1" }, @@ -2545,6 +2546,7 @@ "version": "2.11.0", "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.11.0.tgz", "integrity": "sha512-RRjxlvLDkD1YJwDbroBHMb+cukurkDWNyHx7D3oNB5x9rb5ogcksMC5wHCadcXoo67gVr/+3GFySh3134zi6rw==", + "dev": true, "dependencies": { "has": "^1.0.3" }, @@ -4076,7 +4078,8 @@ "node_modules/path-parse": { "version": "1.0.7", "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz", - "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==" + "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==", + "dev": true }, "node_modules/path-scurry": { "version": "1.10.1", @@ -4455,22 +4458,6 @@ "postcss": "^8.2" } }, - "node_modules/postcss-import": { - "version": "15.1.0", - "resolved": "https://registry.npmjs.org/postcss-import/-/postcss-import-15.1.0.tgz", - "integrity": "sha512-hpr+J05B2FVYUAXHeK1YyI267J/dDDhMU6B6civm8hSY1jYJnBXxzKDKDswzJmtLHryrjhnDjqqp/49t8FALew==", - "dependencies": { - "postcss-value-parser": "^4.0.0", - "read-cache": "^1.0.0", - "resolve": "^1.1.7" - }, - "engines": { - "node": ">=14.0.0" - }, - "peerDependencies": { - "postcss": "^8.0.0" - } - }, "node_modules/postcss-initial": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/postcss-initial/-/postcss-initial-4.0.1.tgz", @@ -5134,6 +5121,7 @@ "version": "1.22.1", "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.1.tgz", "integrity": "sha512-nBpuuYuY5jFsli/JIs1oldw6fOQCBioohqWZg/2hiaOybXOft4lonv85uDOKXdf8rhyK159cxU5cDcK/NKk8zw==", + "dev": true, "dependencies": { "is-core-module": "^2.9.0", "path-parse": "^1.0.7", @@ -5656,6 +5644,7 @@ "version": "1.0.0", "resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz", "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==", + "dev": true, "engines": { "node": ">= 0.4" }, @@ -7395,7 +7384,8 @@ "function-bind": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz", - "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==" + "integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==", + "dev": true }, "function.prototype.name": { "version": "1.1.5", @@ -7610,6 +7600,7 @@ "version": "1.0.3", "resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz", "integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==", + "dev": true, "requires": { "function-bind": "^1.1.1" } @@ -7821,6 +7812,7 @@ "version": "2.11.0", "resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.11.0.tgz", "integrity": "sha512-RRjxlvLDkD1YJwDbroBHMb+cukurkDWNyHx7D3oNB5x9rb5ogcksMC5wHCadcXoo67gVr/+3GFySh3134zi6rw==", + "dev": true, "requires": { "has": "^1.0.3" } @@ -8898,7 +8890,8 @@ "path-parse": { "version": "1.0.7", "resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz", - "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==" + "integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==", + "dev": true }, "path-scurry": { "version": "1.10.1", @@ -9098,16 +9091,6 @@ "postcss-value-parser": "^4.2.0" } }, - "postcss-import": { - "version": "15.1.0", - "resolved": "https://registry.npmjs.org/postcss-import/-/postcss-import-15.1.0.tgz", - "integrity": "sha512-hpr+J05B2FVYUAXHeK1YyI267J/dDDhMU6B6civm8hSY1jYJnBXxzKDKDswzJmtLHryrjhnDjqqp/49t8FALew==", - "requires": { - "postcss-value-parser": "^4.0.0", - "read-cache": "^1.0.0", - "resolve": "^1.1.7" - } - }, "postcss-initial": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/postcss-initial/-/postcss-initial-4.0.1.tgz", @@ -9521,6 +9504,7 @@ "version": "1.22.1", "resolved": "https://registry.npmjs.org/resolve/-/resolve-1.22.1.tgz", "integrity": "sha512-nBpuuYuY5jFsli/JIs1oldw6fOQCBioohqWZg/2hiaOybXOft4lonv85uDOKXdf8rhyK159cxU5cDcK/NKk8zw==", + "dev": true, "requires": { "is-core-module": "^2.9.0", "path-parse": "^1.0.7", @@ -9895,7 +9879,8 @@ "supports-preserve-symlinks-flag": { "version": "1.0.0", "resolved": "https://registry.npmjs.org/supports-preserve-symlinks-flag/-/supports-preserve-symlinks-flag-1.0.0.tgz", - "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==" + "integrity": "sha512-ot0WnXS9fgdkgIcePe6RHNk1WA8+muPa6cSjeR3V8K27q9BB1rTE3R1p7Hv0z1ZyAc8s6Vvv8DIyWf681MAt0w==", + "dev": true }, "svg-tags": { "version": "1.0.0", diff --git a/package.json b/package.json index b6a014c2..bb8ac59e 100644 --- a/package.json +++ b/package.json @@ -20,7 +20,7 @@ "lint:markdown": "markdownlint *.md content/**/*.md", "lint:css": "stylelint \"themes/opentermsarchive/assets/css/*.css\"", "lint:js": "eslint themes/opentermsarchive/assets/js/", - "start:dev": "hugo serve --watch --logLevel debug --disableFastRender", + "start:dev": "hugo serve --watch --logLevel debug --disableFastRender --ignoreCache", "jsdoc": "npm install @opentermsarchive/engine@latest --no-save && jsdoc --configure .jsdoc.json" }, "devDependencies": { @@ -38,7 +38,6 @@ "minireset.css": "^0.0.7", "postcss": "^8.4.19", "postcss-cli": "^10.1.0", - "postcss-import": "^15.0.1", "postcss-mixins": "^9.0.4", "postcss-preset-env": "^7.8.3" } diff --git a/themes/opentermsarchive/assets/css/components/aside.css b/themes/opentermsarchive/assets/css/components/aside.css new file mode 100644 index 00000000..b44c0e5e --- /dev/null +++ b/themes/opentermsarchive/assets/css/components/aside.css @@ -0,0 +1,112 @@ +.aside { + flex-shrink: 0; + margin-right:var(--mXL); + @mixin grid 3, 2; + border:1px solid var(--colorBlack200); + border-radius:4px; +} + +@media (--tabletSmall) { + .aside { + max-width: 100%; + margin-bottom:var(--m2XL); + } +} + +.aside_nav{ + ul{ + color:var(--colorBlack600); + display:flex; + flex-direction:column; + + li{ + border-bottom:1px solid var(--colorBlack200); + + a{ + padding: var(--mXS) var(--mS); + display:inline-block; + width:100%; + + &:hover{ + background-color:var(--colorBlack200); + } + } + + &:last-child{ + border:none; + } + + details{ + position: relative; + + summary{ + padding: var(--mXS) var(--mS); + list-style: none; + cursor: pointer; + + &:hover{ + background-color:var(--colorBlack200); + } + + &::marker, + &::-webkit-details-marker{ + display: none; + } + + & svg{ + position: absolute; + right: 10px; + top:14px; + font-size:0.8em; + color:var(--colorBlack400); + transition: transform 0.2s ease; + } + } + + &[open]{ + summary > svg{ + transform: rotate(180deg); + } + } + } + + span{ + display:inline-block; + } + } + + ul{ + padding-left:var(--mS); + padding-top:var(--m2XS); + padding-bottom:var(--m2XS); + + li{ + border:none; + padding-top:var(--m3XS); + padding-bottom:var(--m3XS); + + a{ + padding: 0; + + &:hover{ + background-color: transparent; + } + } + + ul{ + padding-top:0; + padding-bottom:0; + + li{ + padding:0; + } + } + } + } + } +} + +.aside_item-current{ + border-left:2px solid var(--colorPrimary); +} + diff --git a/themes/opentermsarchive/assets/css/components/aside/aside.css b/themes/opentermsarchive/assets/css/components/aside/aside.css deleted file mode 100644 index 51960917..00000000 --- a/themes/opentermsarchive/assets/css/components/aside/aside.css +++ /dev/null @@ -1,53 +0,0 @@ -.aside { - flex-shrink: 0; - margin-right:var(--mXL); - @mixin grid 3, 2; - border:1px solid var(--colorBlack200); - border-radius:4px; -} - -.aside_items{ - -} - -.aside_item{ - padding: var(--mXS) var(--mS); - border-bottom:1px solid var(--colorBlack200); - - &:last-child{ - border:none; - } - - & #TableOfContents{ - padding-left:var(--mXS); - - & > ul > li > ul{ - padding-left:var(--mXS); - } - } -} - -.aside_subitems{ - padding-left:var(--mXS); -} - -.aside_item-current{ - background-color:var(--colorBlack200); - & > a, - & > span{ - font-weight:bold; - } -} - -.aside_item_section{ - color:var(--colorBlack600); - width:100%; - display:inline-block; -} - -@media (--tabletSmall) { - .aside { - max-width: 100%; - margin-bottom:var(--m2XL); - } -} diff --git a/themes/opentermsarchive/assets/css/components/configOption.css b/themes/opentermsarchive/assets/css/components/configOption.css new file mode 100644 index 00000000..a0276f9b --- /dev/null +++ b/themes/opentermsarchive/assets/css/components/configOption.css @@ -0,0 +1,47 @@ +.config-option { + padding: 2rem 0; +} + +.config-option + .config-option { + box-shadow: inset 0 1px var(--colorBlack200); +} + +.config-option .config-option-name code { + font-weight: 600; +} + +.config-option-type { + color: var(--colorBlack600); + padding-left: 1rem; + text-transform: lowercase; +} + +.config-option-required { + padding-left: 1rem; + text-transform: lowercase; + color: rgb(154, 103, 0); +} + +.config-option-details { + display: flex; +} + +.config-option-description, +.config-option-default, +.config-option-example, +.config-option-allowed-values { + padding: 1rem 0.4em; +} + +.config-option-example code, +.config-option-default code, +.config-option-allowed-values code { + margin-left: 0.4em; + display: inline-block; +} + +.config-option-example, +.config-option-default, +.config-option-allowed-values { + font-size: 0.85em; +} diff --git a/themes/opentermsarchive/assets/css/components/container/container.css b/themes/opentermsarchive/assets/css/components/container.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/container/container.css rename to themes/opentermsarchive/assets/css/components/container.css diff --git a/themes/opentermsarchive/assets/css/components/divider/divider.css b/themes/opentermsarchive/assets/css/components/divider.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/divider/divider.css rename to themes/opentermsarchive/assets/css/components/divider.css diff --git a/themes/opentermsarchive/assets/css/components/footer/footer.css b/themes/opentermsarchive/assets/css/components/footer.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/footer/footer.css rename to themes/opentermsarchive/assets/css/components/footer.css diff --git a/themes/opentermsarchive/assets/css/components/header/header.css b/themes/opentermsarchive/assets/css/components/header.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/header/header.css rename to themes/opentermsarchive/assets/css/components/header.css diff --git a/themes/opentermsarchive/assets/css/components/headerMenu/headerMenu.css b/themes/opentermsarchive/assets/css/components/headerMenu.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/headerMenu/headerMenu.css rename to themes/opentermsarchive/assets/css/components/headerMenu.css diff --git a/themes/opentermsarchive/assets/css/components/languageSwitcher/languageSwitcher.css b/themes/opentermsarchive/assets/css/components/languageSwitcher.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/languageSwitcher/languageSwitcher.css rename to themes/opentermsarchive/assets/css/components/languageSwitcher.css diff --git a/themes/opentermsarchive/assets/css/components/logo/logo.css b/themes/opentermsarchive/assets/css/components/logo.css similarity index 100% rename from themes/opentermsarchive/assets/css/components/logo/logo.css rename to themes/opentermsarchive/assets/css/components/logo.css diff --git a/themes/opentermsarchive/assets/css/components/textContent/textContent.css b/themes/opentermsarchive/assets/css/components/textContent.css similarity index 99% rename from themes/opentermsarchive/assets/css/components/textContent/textContent.css rename to themes/opentermsarchive/assets/css/components/textContent.css index c7d428f9..a00c84ab 100644 --- a/themes/opentermsarchive/assets/css/components/textContent/textContent.css +++ b/themes/opentermsarchive/assets/css/components/textContent.css @@ -3,6 +3,7 @@ padding:var(--mL); overflow-x: hidden; max-width: 100%; + width: 100%; & hr { clear: both; diff --git a/themes/opentermsarchive/assets/css/loader.css b/themes/opentermsarchive/assets/css/loader.css index 597eb617..e5446ce1 100644 --- a/themes/opentermsarchive/assets/css/loader.css +++ b/themes/opentermsarchive/assets/css/loader.css @@ -1,37 +1,38 @@ /* Reset */ -@import "node_modules/minireset.css/minireset.css"; +@import "/vendor/minireset.css"; /* Mixins */ -@import "themes/opentermsarchive/assets/css/mixins.css"; +@import "mixins.css"; /* Custom media queries and custom properties */ -@import "themes/opentermsarchive/assets/css/custom-properties.css"; -@import "themes/opentermsarchive/assets/css/custom-media-queries.css"; +@import "custom-properties.css"; +@import "custom-media-queries.css"; /* Base */ -@import "themes/opentermsarchive/assets/css/base.css"; +@import "base.css"; /* Elements */ -@import "themes/opentermsarchive/assets/css/elements/utility.css"; -@import "themes/opentermsarchive/assets/css/elements/sprite.css"; -@import "themes/opentermsarchive/assets/css/elements/buttons.css"; -@import "themes/opentermsarchive/assets/css/elements/fonts.css"; -@import "themes/opentermsarchive/assets/css/elements/icons.css"; -@import "themes/opentermsarchive/assets/css/elements/inlines.css"; -@import "themes/opentermsarchive/assets/css/elements/titles.css"; -@import "themes/opentermsarchive/assets/css/elements/links.css"; -@import "themes/opentermsarchive/assets/css/elements/form.css"; -@import "themes/opentermsarchive/assets/css/elements/blockquote.css"; -@import "themes/opentermsarchive/assets/css/elements/table.css"; -@import "themes/opentermsarchive/assets/css/elements/syntax.css"; +@import "/elements/utility.css"; +@import "/elements/sprite.css"; +@import "/elements/buttons.css"; +@import "/elements/fonts.css"; +@import "/elements/icons.css"; +@import "/elements/inlines.css"; +@import "/elements/titles.css"; +@import "/elements/links.css"; +@import "/elements/form.css"; +@import "/elements/blockquote.css"; +@import "/elements/table.css"; +@import "/elements/syntax.css"; /* Components */ -@import "themes/opentermsarchive/assets/css/components/logo/logo.css"; -@import "themes/opentermsarchive/assets/css/components/headerMenu/headerMenu.css"; -@import "themes/opentermsarchive/assets/css/components/languageSwitcher/languageSwitcher.css"; -@import "themes/opentermsarchive/assets/css/components/container/container.css"; -@import "themes/opentermsarchive/assets/css/components/header/header.css"; -@import "themes/opentermsarchive/assets/css/components/footer/footer.css"; -@import "themes/opentermsarchive/assets/css/components/divider/divider.css"; -@import "themes/opentermsarchive/assets/css/components/textContent/textContent.css"; -@import "themes/opentermsarchive/assets/css/components/aside/aside.css"; +@import "/components/logo.css"; +@import "/components/headerMenu.css"; +@import "/components/languageSwitcher.css"; +@import "/components/container.css"; +@import "/components/header.css"; +@import "/components/footer.css"; +@import "/components/divider.css"; +@import "/components/textContent.css"; +@import "/components/aside.css"; +@import "/components/configOption.css"; diff --git a/themes/opentermsarchive/assets/css/postcss.config.js b/themes/opentermsarchive/assets/css/postcss.config.js index d57c05aa..bad1e627 100644 --- a/themes/opentermsarchive/assets/css/postcss.config.js +++ b/themes/opentermsarchive/assets/css/postcss.config.js @@ -1,16 +1,9 @@ -const themeDir = __dirname + '/../../'; +const themeDir = `${__dirname}/../../`; module.exports = { plugins: [ - require('postcss-mixins')({ - mixinsFiles: [themeDir + 'assets/css/mixins.css'], - }), - require('postcss-import')({ - path: [themeDir], - }), - require('autoprefixer')({ - path: [themeDir], - }), + require('postcss-mixins')({ mixinsFiles: [`${themeDir}assets/css/mixins.css`] }), + require('autoprefixer')({ path: [themeDir] }), require('postcss-preset-env')({ path: [themeDir], stage: 1, diff --git a/themes/opentermsarchive/layouts/partials/aside.html b/themes/opentermsarchive/layouts/partials/aside.html index d59d946d..232d3f49 100644 --- a/themes/opentermsarchive/layouts/partials/aside.html +++ b/themes/opentermsarchive/layouts/partials/aside.html @@ -1,33 +1,36 @@