Update docs (#634)

* Update notes on release workflow to better match current practice

* Add note on turning the other services back on

Author: PGijsbers

docs/developer/releases.md

@@ -11,56 +11,54 @@ The project loosely uses [Semantic Versions](https://semver.org) where the patch
     developed. For now, we make sure all URLs are at least under a version suffix, which makes
     support in the future possible.
 
-Breaking changes of a resource include deleting a field, changing the name of an existing field,
-or changing the datatype of a field. Adding new fields is not a breaking change.
+We do our best to version API changes such that users can opt out of breaking changes.
+We consider the following breaking changes:
 
-On a breaking change for a resource (e.g. for Dataset), a new router with a new version should
-be created. The existing router should be deprecated, and rewritten so that it can handle the
-new metadata of the database. This deprecation of a router will be visible in the Swagger
-documentation. Calls to a deprecated router will still work, but a response header "Deprecated"
-will be added with the deprecation date. The deprecated router will then be deleted on the next
-release.
+ - removing a field from the input
+ - changing the name of a field
+ - changing the datatype of a field
+ - changing or removing the default of a field
 
-On non-breaking changes of a resource, a new version is not needed for the corresponding router.
-
-Example:
-- Start www.aiod.eu/api/datasets/v0
-- Release 1: www.aiod.eu/api/datasets/v0 (no breaking changes)
-- Release 2:
-    - www.aiod.eu/api/datasets/v0 (deprecated)
-    - www.aiod.eu/api/datasets/v1
-- Release 3: www.aiod.eu/api/datasets/v1
-
-## Changelog
-
-As changelog we use the Github tags. For each release, a release branch should be created with a
-bumped version in the pyproject.toml, and merged with the master. The tag should contain a
-message detailing all the breaking and non-breaking changes. This message should adhere to the
-guiding principles as described in https://keepachangelog.com/.
-
-- Show all tags: https://github.com/aiondemand/AIOD-rest-api/tags
-- Show a specific tag: https://github.com/aiondemand/AIOD-rest-api/releases/tag/0.3.20220501
-
-This information can also be extracted using the Github REST API.
+We generally try to avoid putting breaking changes into already released versions.
+There can be some exceptions based on the developers' judgement, but such a decision should not
+be made lightly and clearly communicated. Once the project matures more, one should avoid these
+changes altogether. Some changes may be backported in a compatible manner,
+for example instead of changing the name for a field, you could add a field with the new name without removing the old one.
+This gives people time to move from one name to the other ([example](https://github.com/aiondemand/AIOD-rest-api/pull/594)).
 
+The entire API is versioned together. This is because of the interconnectedness of assets.
+If one versioned asset types independently, but we return e.g., `Person`s that are member of an `Organisation`
+as part of the organisations endpoint, we would need a way to specify versions for both `Organisation` and `Person`
+in the request. Because there are many high-level connections (e.g., `has_part` and `is_part_of`) we felt it is
+better to version the entire API. However, while we internally have *some* support for schema migrations (
+[example](https://github.com/aiondemand/AIOD-rest-api/blob/ff24e3c4278194b405bc49adc93991704e9c0e1d/src/database/model/project/project.py#L122)
+), there are still issues when they are retrieved through indirect means ([#633](https://github.com/aiondemand/AIOD-rest-api/issues/633)).
 
 ## Creating a release
 To create a new release:
 
 1. Make sure all requested functionality is merged with the `develop` branch.
-2. Create a release branch from develop: `git checkout -b release/[VERSION]`. Example of version: `1.1.20231129`
-3. Update the version in `pyproject.toml` on the release branch.
-4. Test all (most of) the functionality. Checkout the project in a new directory and remove all
-   your local images, and make sure it works out-of-the box.
-5. Go to https://github.com/aiondemand/AIOD-rest-api/releases and draft a new release from the
-   release branch. Look at all closed PRs and create a changelog.
-6. Create a PR from release branch to develop. Make sure that when it is merged, the release branch is preserved.
-7. Deploy on the server(s):
-    - Check which services currently work (before the update). It's a sanity check for if a service _doesn't_ work later.
-    - Bring the services down.
-    - Update the code on the server by checking out the release.
-    - If the release contains new configuration options or configuration defaults, make sure that the override files are updated as needed.
-    - Start the services without connectors.
-    - Make sure the latest database migrations are applied: see ["Schema Migrations"](schema/migration.md#update-the-database)
-    - Restart the services with connectors.
-9. Notify everyone (e.g., in the API channel in Slack).
+2. Make sure the version in `pyproject.toml` is updated in the `develop` branch.
+3. Make a release on GitHub: https://github.com/aiondemand/AIOD-rest-api/releases
+4. Make sure the release notes follow a similar style, use the merged PRs since last release (use e.g. filter `merged:>=2025-08-2`).
+
+Deploy the new release to the test server. If that works (preferably leave it up for a while), then deploy to production.
+Deploying an update:
+
+ 1. Create an additional database backup: `./scripts/mysql_dump.sh`
+ 1. Check which services currently work (before the update). It's a sanity check for if a service _doesn't_ work later.
+ 1. See which files, if any, are changed locally: `git status`
+   - Stash the changes, or keep track of them in some way. Try to make a note to update the software so that local changes are not necessary.
+ 1. Check out the release (e.g., `git fetch origin && git checkout v2.0.20250802`)
+   - If absolutely necessary, reapply the changes. Again: make a note to avoid needing any local changes next time.
+ 1. If the release contains new configuration options or configuration defaults, make sure that the override files are updated as needed.
+ 1. Rebuild the images: `./scripts/build.sh`
+ 1. Build the alembic image separately (it's not included in the above script), see `docker build -f alembic/Dockerfile . -t aiod-migration` ([docs](schema/migration.md).
+ 1. Bring down the services: `./scripts/down.sh`
+ 1. Bring the database back up: `docker compose --env-file=.env --env-file=override.env up sqlserver -d`
+-  Runs the [migrations](schema/migration.md), if any: `docker run -v $(pwd)/alembic:/alembic:ro  -v $(pwd)/src:/app -it --network aiod-rest-api_default  aiod-migration`
+- Start the remainder of the services: `./scripts/up.sh`
+- To make sure the latest taxonomy is also included:
+  - Copy the `export/taxonomies.json` file from the `metadata-schema` repository and have it update on startup, or
+  - run the taxonomy service independently (preferred, but currently broken?)
+- Notify everyone (e.g., in the API channel in Slack).