Skip to content

Add CI test that validates all FML manifests are parseable by FmlClient #15020

@jaredlockhart

Description

@jaredlockhart

Because

  • When application-services added stricter FML validation (bug 2020683, gecko-pref and default are now mutually exclusive), the Fenix v149.0.0 manifests became unparseable by FmlClient
  • We had no unit test that actually loads each versioned FML manifest with FmlClient — the existing "Check Local Feature Manifests" CI job only checks that manifests are up-to-date with what the manifest tool generates, not that they pass FML validation
  • This gap meant the breakage was only caught by integration tests timing out on a 500, not by a clear unit test failure
  • See PR chore(nimbus): Update application-services #15002 and issue FmlClient constructor crash causes 500 when manifest has validation errors #15018 for the full investigation

This issue

  • Add a unit test in experimenter/features/tests/test_features.py (alongside TestValidateFeatureManifests) that iterates all Application × Channel × Version combinations and calls FmlClient(path, channel) for each manifest file
  • Use self.subTest() so failures identify the exact app/channel/version
  • This ensures that application-services updates and manifest syncs are caught early with a clear error message

Blocked by

  • Bug 2025587 — the Fenix v149.0.0 manifest needs to be fixed upstream first (remove default from gecko-nimbus-validation.test-preference), otherwise this test will fail immediately on the current manifests

┆Issue is synchronized with this Jira Task

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions