-
Notifications
You must be signed in to change notification settings - Fork 5
Add data.gouv dataset publishing documentation #204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| --- | ||
| title: Publish to data.gouv.fr | ||
| weight: 6 | ||
| --- | ||
|
|
||
| # How to publish datasets to data.gouv.fr | ||
|
|
||
| This guide explains how to configure your collection to automatically publish datasets to [data.gouv.fr](https://www.data.gouv.fr/), the French government's open data platform. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - A [data.gouv.fr](https://www.data.gouv.fr/) account | ||
| - An API key from your data.gouv.fr account settings | ||
| - Either an existing dataset or an organization where you can create datasets | ||
|
|
||
| ## Choose your approach | ||
|
|
||
| There are two ways to publish datasets to data.gouv.fr: | ||
|
|
||
| ### Option 1: Automatically create a dataset in an organization | ||
|
|
||
| This approach is suitable when you want the system to automatically create and manage the dataset within your organization. | ||
|
|
||
| 1. Find your organization on [data.gouv.fr](https://www.data.gouv.fr/) (or [demo.data.gouv.fr](https://demo.data.gouv.fr/) for testing) | ||
| 2. Copy the organization ID or slug from the URL (e.g., `open-terms-archive`) | ||
| 3. Add it to your configuration at `dataset.datagouv.organizationIdOrSlug` | ||
| 4. Set `dataset.title` in your configuration (this will be used as the dataset title) | ||
|
|
||
| The dataset will be automatically created if it doesn't already exist in the organization. | ||
|
|
||
| ### Option 2: Use an existing dataset | ||
|
|
||
| This approach is suitable when you already have a dataset created on data.gouv.fr and want to update it automatically. | ||
|
|
||
| 1. Create a dataset on [data.gouv.fr](https://www.data.gouv.fr/) (or [demo.data.gouv.fr](https://demo.data.gouv.fr/) for testing) | ||
| 2. Copy the dataset ID from the Informations tab on the dataset page (e.g., `6914a64b17a0a61222`) | ||
| 3. Add it to your configuration at `dataset.datagouv.datasetId` | ||
|
|
||
| ## Configure your collection | ||
|
|
||
| ### 1. Add configuration settings | ||
|
|
||
| In your collection's configuration file (e.g., `config/production.json`), add the `datagouv` settings under the `dataset` section: | ||
|
|
||
| **For Option 1 (automatic creation):** | ||
|
|
||
| ```json | ||
| { | ||
| "dataset": { | ||
| "title": "<collection_name> collection dataset", | ||
| "datagouv": { | ||
| "organizationIdOrSlug": "open-terms-archive" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| **For Option 2 (existing dataset):** | ||
|
|
||
| ```json | ||
| { | ||
| "dataset": { | ||
| "title": "<collection_name> collection dataset", | ||
| "datagouv": { | ||
| "datasetId": "6914a64b17a0a91bb0a61222" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### 3. Configure for testing (optional) | ||
|
|
||
| If you want to test with the demo environment first, add `useDemo`: | ||
|
|
||
| ```json | ||
| { | ||
| "dataset": { | ||
| "title": "<collection_name> collection dataset", | ||
| "datagouv": { | ||
| "organizationIdOrSlug": "open-terms-archive", | ||
| "useDemo": true | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### 4. Set the API key | ||
|
|
||
| Create a `.env` file at the root of your collection repository (if it doesn't already exist) and add your data.gouv.fr API key: | ||
|
|
||
| ``` | ||
| OTA_ENGINE_DATAGOUV_API_KEY=your_api_key_here | ||
| ``` | ||
|
|
||
| ## Test your configuration | ||
|
|
||
| You can test your configuration by manually publishing a dataset: | ||
|
|
||
| ```bash | ||
| npx ota dataset --publish | ||
| ``` | ||
|
|
||
| This will create and publish a dataset to data.gouv.fr. Check the output to verify the dataset was published successfully. | ||
|
|
||
| ## Set up automatic publishing | ||
|
|
||
| To automatically publish datasets on a schedule, use the `--schedule` flag: | ||
|
|
||
| ```bash | ||
| npx ota dataset --schedule --publish --remove-local-copy | ||
| ``` | ||
|
|
||
| This will publish datasets according to the schedule defined in your configuration (by default, every Monday at 8:30 AM). | ||
|
|
||
| ## Publishing to multiple platforms | ||
|
|
||
| data.gouv.fr publishing can be used alongside GitHub or GitLab releases. Simply configure both platforms and datasets will be published to all configured platforms simultaneously. | ||
|
|
||
| See the [configuration reference]({{< relref "collections/reference/configuration#datagouv-fr-publishing" >}}) for all available options. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The display of examples does not seem to take double dashes into account.