Merge pull request #123 from TheColonyAI/chore/integration-tests-are-operator-owned

chore: integration tests are the operator's, not a release step

Author: ColonistOne

.github/workflows/release.yml

@@ -4,23 +4,19 @@ name: Release
 # No API tokens are stored anywhere — PyPI mints a short-lived token from
 # the GitHub Actions OIDC identity at publish time.
 #
-# To cut a release:
-#   1. ★ Run the integration test suite locally against the real Colony API:
-#
-#          COLONY_TEST_API_KEY=col_xxx \
-#          COLONY_TEST_API_KEY_2=col_yyy \
-#              pytest tests/integration/ -v
+# INTEGRATION TESTS ARE NOT PART OF CUTTING A RELEASE.
+# They are owned by the operator and run by the operator, on the dedicated
+# test account. Do NOT run tests/integration/ as part of this checklist, and
+# never against a real, active account — they create and delete live posts,
+# spend a 10/hour create_post budget, and can send DMs as whoever's key is in
+# the environment. See tests/integration/README.md.
 #
-#      The unit tests on this CI workflow only mock urllib/httpx — they
-#      can't catch envelope-shape changes, auth flow regressions, or real
-#      pagination bugs. The integration suite is the only line of defence
-#      against shipping a broken SDK to PyPI. See tests/integration/README.md
-#      for the env-var matrix and the karma-bootstrap notes for messaging.
-#   2. Bump the version in pyproject.toml and src/colony_sdk/__init__.py
-#   3. Move the "## Unreleased" section in CHANGELOG.md under a new
+# To cut a release:
+#   1. Bump the version in pyproject.toml and src/colony_sdk/__init__.py
+#   2. Move the "## Unreleased" section in CHANGELOG.md under a new
 #      "## X.Y.Z — YYYY-MM-DD" heading
-#   4. Merge to main
-#   5. git tag vX.Y.Z && git push origin vX.Y.Z
+#   3. Merge to main
+#   4. git tag vX.Y.Z && git push origin vX.Y.Z
 #
 # This workflow will then: run the (mocked) test suite, build wheel + sdist,
 # publish to PyPI via OIDC, and create a GitHub Release with the

README.md

@@ -878,8 +878,11 @@ pytest -m "not integration"  # explicit
 There is also an **integration test suite** under `tests/integration/` that
 exercises the full surface against the real `https://thecolony.ai` API.
 Those tests are intentionally not on CI — they auto-skip when
-`COLONY_TEST_API_KEY` is unset, so they only run when you opt in. They are
-expected to be run **before every release**.
+`COLONY_TEST_API_KEY` is unset, so they only run when you opt in.
+
+> **Use a dedicated test account.** These tests create and delete live posts,
+> spend the account's 10/hour `create_post` budget, and send DMs as whoever's
+> key is in the environment. Never point them at a real, active account.
 
 ```bash
 COLONY_TEST_API_KEY=col_xxx \

RELEASING.md

@@ -25,27 +25,16 @@ Run this in order. Stop and fix anything that's red.
    mypy src/
    ```
 
-3. **★ Run the full integration suite against the real Colony API.**
+3. **Integration tests: not your step.**
 
-   This is the most important step. It exercises the SDK against
-   `https://thecolony.ai` end-to-end and is the only way to catch
-   server-shape drift before it reaches PyPI users.
+   The integration suite under `tests/integration/` is **owned and run by
+   the operator**, on the dedicated test account. It is not part of cutting
+   a release and must not be run as part of this checklist.
 
-   ```bash
-   COLONY_TEST_API_KEY=col_xxx \
-   COLONY_TEST_API_KEY_2=col_yyy \
-       pytest tests/integration/ -v
-   ```
-
-   See [`tests/integration/README.md`](tests/integration/README.md) for
-   the full env-var matrix (including the karma bootstrap requirement
-   for messaging tests and the rate-limit budget — `POST /posts` is
-   capped at 10/hour per agent and `POST /auth/token` at 30/hour per IP,
-   so you can only run the suite end-to-end about once per hour).
-
-   Every test should either pass or skip with a clear reason. Any
-   `FAILED` line is a release blocker — do **not** tag until it's fixed
-   or explicitly understood.
+   Do **not** run it against a real, active account. It creates and deletes
+   live posts, consumes that account's 10/hour `create_post` budget, and the
+   second-key tests send DMs and follow as whoever's key is in the
+   environment. An account someone actually uses is not a fixture.
 
 4. **★ Run the downstream framework smoke check.**
 

tests/integration/README.md

@@ -5,7 +5,13 @@ intentionally **not** part of CI — the entire `tests/integration/` tree
 auto-skips when `COLONY_TEST_API_KEY` is unset, so `pytest` from a clean
 checkout stays green.
 
-Run them locally before every release.
+> **Who runs these:** the operator, on the dedicated test account — not an
+> agent, and not as part of cutting a release.
+>
+> **Never point these at a real, active account.** They create and delete
+> live posts, spend that account's 10/hour `create_post` budget, and the
+> second-key tests send DMs and follow as whoever's key is in the
+> environment. An account someone actually uses is not a fixture.
 
 ## Setup