Remove system_dependencies (#105)

* remove system_dependencies from System model

* Update tests for `system_dependencies` removal

This also includes the additional validation of circular dependencies for ingress and egress. There was some previous discussion around the need for this and wanting these fields to be intentionally permissible so this commit may be reverted prior to merge.

* remove validation of ingress/egress

* changelog updates

* Update src/fideslang/models.py

Co-authored-by: Thomas <[email protected]>

---------

Co-authored-by: Thomas <[email protected]>

Author: SteveDMurphy

CHANGELOG.md

@@ -6,99 +6,108 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/)
 
 The types of changes are:
 
-* `Added` for new features.
-* `Changed` for changes in existing functionality.
-* `Developer Experience` for changes in developer workflow or tooling.
-* `Deprecated` for soon-to-be removed features.
-* `Removed` for now removed features.
-* `Fixed` for any bug fixes.
-* `Security` in case of vulnerabilities.
+- `Added` for new features.
+- `Changed` for changes in existing functionality.
+- `Developer Experience` for changes in developer workflow or tooling.
+- `Deprecated` for soon-to-be removed features.
+- `Removed` for now removed features.
+- `Fixed` for any bug fixes.
+- `Security` in case of vulnerabilities.
 
 ## [Unreleased](https://github.com/ethyca/fideslang/compare/1.3.2...main)
 
 ### Changed
-* Make `PrivacyDeclaration` use pydantic `orm_mode` [#101](https://github.com/ethyca/fideslang/pull/101)
+
+- Make `PrivacyDeclaration` use pydantic `orm_mode` [#101](https://github.com/ethyca/fideslang/pull/101)
+
+### Remove
+
+- The `system_dependencies` field of `System` resources [#105](https://github.com/ethyca/fideslang/pull/105)
 
 ## [1.3.3](https://github.com/ethyca/fideslang/compare/1.3.2...1.3.3)
 
 ### Changed
-* Make `PrivacyDeclation.name` optional [#97](https://github.com/ethyca/fideslang/pull/97)
+
+- Make `PrivacyDeclation.name` optional [#97](https://github.com/ethyca/fideslang/pull/97)
 
 ## [1.3.2](https://github.com/ethyca/fideslang/compare/1.3.1...1.3.2)
 
 ### Changed
-* Update css to brand colors, edit footer [#87](https://github.com/ethyca/fideslang/pull/87)
-* Moved over DSR concepts into Fideslang. Expanded allowable characters for FideKey and added additional Dataset validation. [#95](https://github.com/ethyca/fideslang/pull/95)
-* Docs css and link updates [#93](https://github.com/ethyca/fideslang/pull/93)
+
+- Update css to brand colors, edit footer [#87](https://github.com/ethyca/fideslang/pull/87)
+- Moved over DSR concepts into Fideslang. Expanded allowable characters for FideKey and added additional Dataset validation. [#95](https://github.com/ethyca/fideslang/pull/95)
+- Docs css and link updates [#93](https://github.com/ethyca/fideslang/pull/93)
 
 ## [1.3.1](https://github.com/ethyca/fideslang/compare/1.3.0...1.3.1)
 
 ### Fixed
 
-* `DataFlow` resource models included in `System` resource models are now exported to valid YAML [#89](https://github.com/ethyca/fideslang/pull/89)
+- `DataFlow` resource models included in `System` resource models are now exported to valid YAML [#89](https://github.com/ethyca/fideslang/pull/89)
 
 ## [1.3.0](https://github.com/ethyca/fideslang/compare/1.2.0...1.3.0)
 
 ### Added
 
-* The `DataFlow` resource model defines a resource with which a `System` resource may communicate [#85](https://github.com/ethyca/fideslang/pull/85)
-* `PrivacyDeclaration`s may define `egress` and `ingress`, to contextualize communications with other resources [#85](https://github.com/ethyca/fideslang/pull/85)
+- The `DataFlow` resource model defines a resource with which a `System` resource may communicate [#85](https://github.com/ethyca/fideslang/pull/85)
+- `PrivacyDeclaration`s may define `egress` and `ingress`, to contextualize communications with other resources [#85](https://github.com/ethyca/fideslang/pull/85)
 
 ### Deprecated
 
-* The `dataset_references` field of `PrivacyDeclaration` resources [#85](https://github.com/ethyca/fideslang/pull/85)
-* The `system_dependencies` field of `System` resources [#85](https://github.com/ethyca/fideslang/pull/85)
+- The `dataset_references` field of `PrivacyDeclaration` resources [#85](https://github.com/ethyca/fideslang/pull/85)
+- The `system_dependencies` field of `System` resources [#85](https://github.com/ethyca/fideslang/pull/85)
 
 ### Developer Experience
 
-* The `DataFlow` resource model is exposed when importing `fideslang` [#85](https://github.com/ethyca/fideslang/pull/85)
+- The `DataFlow` resource model is exposed when importing `fideslang` [#85](https://github.com/ethyca/fideslang/pull/85)
 
 ### Docs
-* Updated the brand colors and footer on the docs site [#87](https://github.com/ethyca/fideslang/pull/87)
+
+- Updated the brand colors and footer on the docs site [#87](https://github.com/ethyca/fideslang/pull/87)
+
 ### Fixed
 
-* Fixed broken links in docs [#74](https://github.com/ethyca/fideslang/pull/74)
-* Pydantic 1.10.0 was causing issues so specified the pydantic version needs to be less than 1.10.0 [#79](https://github.com/ethyca/fideslang/pull/79)
-* Resolved a circular import in `default_taxonomy.py` [#85](https://github.com/ethyca/fideslang/pull/85)
+- Fixed broken links in docs [#74](https://github.com/ethyca/fideslang/pull/74)
+- Pydantic 1.10.0 was causing issues so specified the pydantic version needs to be less than 1.10.0 [#79](https://github.com/ethyca/fideslang/pull/79)
+- Resolved a circular import in `default_taxonomy.py` [#85](https://github.com/ethyca/fideslang/pull/85)
 
 ## [1.2.0](https://github.com/ethyca/fideslang/compare/1.1.0...1.2.0)
 
 ### Added
 
-* New field `is_default` added to DataCategory, DataUse, DataSubject, and DataQualifier [#68](https://github.com/ethyca/fideslang/pull/68)
-* Return invalid key values as part of the stack trace for easier debugging [#55](https://github.com/ethyca/fideslang/pull/55)
+- New field `is_default` added to DataCategory, DataUse, DataSubject, and DataQualifier [#68](https://github.com/ethyca/fideslang/pull/68)
+- Return invalid key values as part of the stack trace for easier debugging [#55](https://github.com/ethyca/fideslang/pull/55)
 
 ### Docs
 
-* Updated documentation for new Data Category and Use taxonomy [#69](https://github.com/ethyca/fideslang/pull/69)
+- Updated documentation for new Data Category and Use taxonomy [#69](https://github.com/ethyca/fideslang/pull/69)
 
 ### Changed
 
-* Docker images now use Debian `bullseye` instead of `buster`
+- Docker images now use Debian `bullseye` instead of `buster`
 
 ### Fixed
 
-* Add setuptools to dev-requirements to fix versioneer error [#72](https://github.com/ethyca/fideslang/pull/72)
+- Add setuptools to dev-requirements to fix versioneer error [#72](https://github.com/ethyca/fideslang/pull/72)
 
 ## 1.1.0
 
 ### Changed
 
-* Simplification of Data Categories and Data Uses [#62](https://github.com/ethyca/fideslang/pull/62)
+- Simplification of Data Categories and Data Uses [#62](https://github.com/ethyca/fideslang/pull/62)
 
 ## 1.0.0
 
 ### Added
 
-* There is now a `tags` field on the `FidesModel` model [#45](https://github.com/ethyca/fideslang/pull/45)
-* Add DatasetFieldBase model [#49](https://github.com/ethyca/fideslang/pull/49)
+- There is now a `tags` field on the `FidesModel` model [#45](https://github.com/ethyca/fideslang/pull/45)
+- Add DatasetFieldBase model [#49](https://github.com/ethyca/fideslang/pull/49)
 
 ## 0.9.0
 
 ### Added
 
-* Created the fideslang standalone python module
+- Created the fideslang standalone python module
 
 ### Developer Experience
 
-* Added a py.typed file
+- Added a py.typed file

src/fideslang/models.py

@@ -937,9 +937,6 @@ class System(FidesModel):
     privacy_declarations: List[PrivacyDeclaration] = Field(
         description=PrivacyDeclaration.__doc__,
     )
-    system_dependencies: Optional[List[FidesKey]] = Field(
-        description="A list of fides keys to model dependencies."
-    )
     joint_controller: Optional[ContactDetails] = Field(
         description=ContactDetails.__doc__,
     )
@@ -959,10 +956,6 @@ class System(FidesModel):
         "privacy_declarations", allow_reuse=True
     )(sort_list_objects_by_name)
 
-    _no_self_reference: classmethod = validator(
-        "system_dependencies", allow_reuse=True, each_item=True
-    )(no_self_reference)
-
     _check_valid_country_code: classmethod = country_code_validator
 
     @validator("privacy_declarations", each_item=True)
@@ -993,21 +986,6 @@ def privacy_declarations_reference_data_flows(
 
         return value
 
-    @validator("system_dependencies")
-    @classmethod
-    def deprecate_system_dependencies(cls, value: List[FidesKey]) -> List[FidesKey]:
-        """
-        Warn that the `system_dependencies` field is deprecated, if set.
-        """
-
-        if value is not None:
-            warn(
-                "The system_dependencies field is deprecated, and will be removed in a future version of fideslang. Use the 'egress' and 'ingress` fields instead.",
-                DeprecationWarning,
-            )
-
-        return value
-
     class Config:
         """Class for the System config"""
 

src/fideslang/validation.py

@@ -51,7 +51,6 @@ def no_self_reference(value: FidesKey, values: Dict) -> FidesKey:
 
     i.e. DataCategory.parent_key != DataCategory.fides_key
     """
-
     fides_key = FidesKey.validate(values.get("fides_key", ""))
     if value == fides_key:
         raise FidesValidationError("FidesKey can not self-reference!")

tests/fideslang/test_models.py

@@ -91,45 +91,6 @@ def test_system_valid(self) -> None:
             tags=["some", "tags"],
         )
 
-    def test_system_dependencies_deprecation(self) -> None:
-        with deprecated_call(match="system_dependencies"):
-            assert System(
-                description="Test Policy",
-                egress=[
-                    DataFlow(
-                        fides_key="test_system_2",
-                        type="system",
-                        data_categories=[],
-                    )
-                ],
-                fides_key="test_system",
-                ingress=[
-                    DataFlow(
-                        fides_key="test_system_3",
-                        type="system",
-                        data_categories=[],
-                    )
-                ],
-                meta={"some": "meta stuff"},
-                name="Test System",
-                organization_fides_key=1,
-                privacy_declarations=[
-                    PrivacyDeclaration(
-                        data_categories=[],
-                        data_qualifier="aggregated_data",
-                        data_subjects=[],
-                        data_use="provide",
-                        egress=["test_system_2"],
-                        ingress=["test_system_3"],
-                        name="declaration-name",
-                    )
-                ],
-                registry_id=1,
-                system_dependencies=[],
-                system_type="SYSTEM",
-                tags=["some", "tags"],
-            )
-
     def test_system_valid_no_egress_or_ingress(self) -> None:
         assert System(
             description="Test Policy",

tests/fideslang/test_relationships.py

@@ -3,6 +3,7 @@
 from fideslang import relationships
 from fideslang.models import (
     DataCategory,
+    DataFlow,
     Dataset,
     DatasetCollection,
     DatasetField,
@@ -34,7 +35,7 @@ def test_find_referenced_fides_keys_2():
         name="test_dc",
         fides_key="test_dc",
         description="test description",
-        system_dependencies=["key_1", "key_2"],
+        egress=[DataFlow(fides_key="key_1", type="system", data_categories=None), DataFlow(fides_key="key_2", type="system", data_categories=None)],
         system_type="test",
         privacy_declarations=None,
     )
@@ -65,7 +66,7 @@ def test_get_referenced_missing_keys():
                 name="test_system",
                 fides_key="test_system",
                 description="test description",
-                system_dependencies=["key_3", "key_4"],
+                egress=[DataFlow(fides_key="key_3", type="system", data_categories=None), DataFlow(fides_key="key_4", type="system", data_categories=None)],
                 system_type="test",
                 privacy_declarations=None,
             )

tests/fideslang/test_validation.py

@@ -2,22 +2,23 @@
 from pydantic import ValidationError
 
 from fideslang.models import (
+    CollectionMeta,
     DataCategory,
+    DataFlow,
+    Dataset,
+    DatasetCollection,
+    DatasetField,
+    DatasetMetadata,
     DataUse,
+    FidesCollectionKey,
+    FidesDatasetReference,
+    FidesMeta,
     FidesModel,
     Policy,
     PolicyRule,
     PrivacyDeclaration,
     PrivacyRule,
     System,
-    FidesDatasetReference,
-    FidesMeta,
-    Dataset,
-    DatasetMetadata,
-    DatasetCollection,
-    CollectionMeta,
-    DatasetField,
-    FidesCollectionKey,
 )
 from fideslang.validation import FidesKey, FidesValidationError, valid_data_type
 
@@ -225,36 +226,11 @@ def test_create_valid_system():
                 dataset_references=[],
             )
         ],
-        system_dependencies=["another_system", "yet_another_system"],
+        egress=[DataFlow(fides_key="another_system", type="system", data_categories=None), DataFlow(fides_key="yet_another_system", type="system", data_categories=None)],
     )
     assert True
 
 
-@pytest.mark.unit
-def test_circular_dependency_system():
-    with pytest.raises(ValidationError):
-        System(
-            organization_fides_key=1,
-            registryId=1,
-            fides_key="test_system",
-            system_type="SYSTEM",
-            name="Test System",
-            description="Test Policy",
-            privacy_declarations=[
-                PrivacyDeclaration(
-                    name="declaration-name",
-                    data_categories=[],
-                    data_use="provide.service",
-                    data_subjects=[],
-                    data_qualifier="aggregated_data",
-                    dataset_references=["test_system"],
-                )
-            ],
-            system_dependencies=["test_system"],
-        )
-    assert True
-
-
 @pytest.mark.unit
 @pytest.mark.parametrize("country_code", ["United States", "US", "usa"])
 def test_invalid_country_identifier(country_code: str):