Give the CI something real to check, and let it start (#1166)

* Replace the committed API key with a placeholder

Both data_folder/secrets.yaml and the example next to it shipped a real-looking
OpenAI key. It is dead and answers 401, but it should never have been in the
repository, and anyone copying the file inherits a broken default instead of a
clear instruction to put their own key there.

Not rewriting history over it. This repository has thousands of forks, a force
push would break every one of them, and the old blob stays reachable by SHA on
GitHub regardless, so a rewrite would buy nothing and cost a lot.

* Give the CI something real to check, and let it start

The workflow had never run a single job. The repository only allowed actions
defined inside itself, so actions/checkout was refused before any job was
created, and every run ended as a startup failure. That setting now allows
GitHub-owned actions and nothing else.

Starting was only half of it. The workflow ran pytest against a repository with
no tests, which exits 5, so it would have gone red the moment it worked. So
there are tests now, and they check the thing that actually breaks for people:
the config files this repository ships have to satisfy the validator this
repository ships, since copying them is the first step of the setup. One of them
refuses a secrets file that holds anything looking like a real key, which is the
regression that was sitting in here until today.

Two of the tests feed the validator a deliberately broken config and require it
to raise, so the suite cannot pass against a validator that accepts anything.

Also pins Python to 3.12 rather than tracking whatever is newest, adds an import
of the app as a smoke check, and declares inquirer, which main.py imports and
which until now arrived only as somebody else's transitive dependency.

Author: feder-cr

.github/workflows/ci.yml

@@ -11,15 +11,18 @@ jobs:
 
     steps:
       - name: Checkout code
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
 
       - name: Set up Python
-        uses: actions/setup-python@v3
+        uses: actions/setup-python@v5
         with:
-          python-version: '3.x'
+          python-version: '3.12'
 
       - name: Install dependencies
         run: pip install -r requirements.txt
 
+      - name: The app has to import on a clean machine
+        run: python -c "import main"
+
       - name: Run tests
-        run: pytest 
+        run: pytest -q

conftest.py

@@ -0,0 +1,6 @@
+import sys
+from pathlib import Path
+
+# The tests import main.py, which lives at the repository root. pytest puts the
+# test file's own directory on sys.path, not this one, so it goes on explicitly.
+sys.path.insert(0, str(Path(__file__).parent))

requirements.txt

@@ -28,4 +28,5 @@ webdriver-manager==4.0.2
 pytest
 pytest-mock
 pytest-cov
-undetected-chromedriver==3.5.5
+undetected-chromedriver==3.5.5
+inquirer

tests/test_shipped_config.py

@@ -0,0 +1,65 @@
+"""The config files this repository ships have to satisfy the validator it ships.
+
+A broken example config is invisible to whoever breaks it and fatal to every new
+user, because copying it is the first thing the setup instructions ask for. The
+same goes for a committed credential: one was sitting in both secrets files for
+a long time and nothing in here was looking.
+"""
+
+from pathlib import Path
+
+import pytest
+import yaml
+
+import main
+
+SHIPPED = ["data_folder", "data_folder_example"]
+
+
+@pytest.mark.parametrize("folder", SHIPPED)
+def test_shipped_work_preferences_pass_the_validator(folder):
+    config = main.ConfigValidator.validate_config(
+        Path(folder) / "work_preferences.yaml"
+    )
+    assert config["positions"], "positions cannot be empty or the run has nothing to do"
+
+
+@pytest.mark.parametrize("folder", SHIPPED)
+def test_shipped_secrets_provide_a_key(folder):
+    assert main.ConfigValidator.validate_secrets(Path(folder) / "secrets.yaml")
+
+
+@pytest.mark.parametrize("folder", SHIPPED)
+def test_shipped_secrets_are_a_placeholder_and_not_a_real_key(folder):
+    key = main.ConfigValidator.validate_secrets(Path(folder) / "secrets.yaml")
+    assert not key.startswith("sk-"), (
+        f"{folder}/secrets.yaml looks like a real API key rather than a placeholder. "
+        "Credentials must never be committed here: put yours in the file locally and "
+        "keep it out of the diff."
+    )
+
+
+def test_the_validator_rejects_a_config_with_a_required_key_missing(tmp_path):
+    """Without this, every assertion above would also pass against a validator
+    that accepts anything at all."""
+    config = yaml.safe_load(
+        Path("data_folder_example/work_preferences.yaml").read_text(encoding="utf-8")
+    )
+    del config["positions"]
+    broken = tmp_path / "work_preferences.yaml"
+    broken.write_text(yaml.safe_dump(config), encoding="utf-8")
+
+    with pytest.raises(main.ConfigError):
+        main.ConfigValidator.validate_config(broken)
+
+
+def test_the_validator_rejects_a_config_with_a_wrong_type(tmp_path):
+    config = yaml.safe_load(
+        Path("data_folder_example/work_preferences.yaml").read_text(encoding="utf-8")
+    )
+    config["distance"] = "not a number"
+    broken = tmp_path / "work_preferences.yaml"
+    broken.write_text(yaml.safe_dump(config), encoding="utf-8")
+
+    with pytest.raises(main.ConfigError):
+        main.ConfigValidator.validate_config(broken)