Merge remote-tracking branch 'origin/develop' into develop-1.1

Author: dgarros

.vale/styles/Infrahub/sentence-case.yml

@@ -49,6 +49,7 @@ exceptions:
   - JetStream
   - Jinja
   - Jinja2
+  - JWT
   - Namespace
   - NATS
   - Node
@@ -63,9 +64,11 @@ exceptions:
   - RFile
   - SDK
   - Single sign-on
+  - SSO
   - TLS
   - Tony Stark
   - TransformPython
+  - UI
   - Vale
   - VS Code
   - VS Code extensions

.vale/styles/spelling-exceptions.txt

@@ -4,6 +4,7 @@ APIs
 artifact_definitions
 artifact_name
 async
+Authentik
 boolean
 check_definitions
 class_name

.yamllint.yml

@@ -10,6 +10,7 @@ ignore: |
   **/node_modules
   # https://github.com/sbaudoin/yamllint/issues/16
   /helm/templates
+  /python_sdk
 
 rules:
   new-lines: disable

CHANGELOG.md

@@ -11,6 +11,73 @@ This project uses [*towncrier*](https://towncrier.readthedocs.io/) and the chang
 
 <!-- towncrier release notes start -->
 
+## [1.0.1](https://github.com/opsmill/infrahub/tree/v1.0.1) - 2024-10-31
+
+### Fixed
+
+- When a user is not logged in and the branch name is not found, hide the quick-create action and display the message: 'No branch found' ([#4801](https://github.com/opsmill/infrahub/issues/4801))
+- Fix automation to trigger generation of artifacts after merging a branch ([#4804](https://github.com/opsmill/infrahub/issues/4804))
+- Avoid sending an empty list to the load schema API on repository import if it's not required
+- Update demo environment to work with Infrahub 1.0
+
+## [1.0.0](https://github.com/opsmill/infrahub/tree/v1.0.0) - 2024-10-30
+
+### Removed
+
+- Remove previously deprecated GET API endpoint "/api/schema/" ([#3884](https://github.com/opsmill/infrahub/issues/3884))
+
+### Deprecated
+
+- Marked CoreAccount.role as deprecated
+  Due to the new permissions framework the account roles "admin" / "read-only" / "read-write" are deprecated and will be removed in Infrahub 1.1
+
+### Added
+
+- Reworked branch selector:
+  - Redesigned the UI
+  - Added filter for branch
+  - Improved accessibility & keyboard navigation
+  - Improved UX on new branch form
+  - Added quick link to view all branches
+- Add support to sign in with OAuth2 and Open ID Connect (OIDC) ([#1568](https://github.com/opsmill/infrahub/issues/1568))
+- Add internal HTTP adapter to allow for generic access from Infrahub ([#3302](https://github.com/opsmill/infrahub/issues/3302))
+- Add support to search a node by human friendly ID within a GraphQL query ([#3908](https://github.com/opsmill/infrahub/issues/3908))
+- Added link to our Discord server in the account menu
+- Added permissions framework for global and object kind level permissions
+
+  In this first iteration the object permissions are applied to nodes as a whole, in upcoming versions it will be possible to define attribute level permissions as well.
+- New permissions system in UI:
+  - Implemented CRUD views for managing accounts, groups, roles, and permissions
+  - Updated all components to support new permission system
+  - Added dynamic message display according to user access levels
+
+### Fixed
+
+- The `infrahub-git` agent service has been renamed to `task-worker` in docker compose and the command to start it has been updated as well ([#1075](https://github.com/opsmill/infrahub/issues/1075))
+- Add ability to import repositories with default branch other than 'main' ([#3435](https://github.com/opsmill/infrahub/issues/3435))
+- Disable approve/merge/close buttons for merged Proposed Changes ([#3495](https://github.com/opsmill/infrahub/issues/3495))
+- Fixed regex validation for List type attributes ([#3929](https://github.com/opsmill/infrahub/issues/3929))
+- Allow users to run artifacts and generators on nodes without name attribute ([#4062](https://github.com/opsmill/infrahub/issues/4062))
+- In the schema, properly delete inherited attribute and relationship on Node when the original attribute or relationship are being deleted on the Generic ([#4301](https://github.com/opsmill/infrahub/issues/4301))
+- "Retry All" button for checks is bigger ([#4315](https://github.com/opsmill/infrahub/issues/4315))
+- Add a size restriction on common attribute kinds. Only TextArea and JSON support large values ([#4432](https://github.com/opsmill/infrahub/issues/4432))
+- The HFID of a related node is properly returned via GraphQL in all scenarios ([#4482](https://github.com/opsmill/infrahub/issues/4482))
+- Add full validation to BranchMerge and BranchRebase mutations ([#4595](https://github.com/opsmill/infrahub/issues/4595))
+- Report user-friendly error for invalid uniqueness_constraints when loading schemas ([#4677](https://github.com/opsmill/infrahub/issues/4677))
+- Fixed pagination query for nodes with order_by clause using non unique attributes ([#4700](https://github.com/opsmill/infrahub/issues/4700))
+- Fixed schema migration when an attribute previously present on a node is added back ([#4727](https://github.com/opsmill/infrahub/issues/4727))
+- Add order_weight property to multiple attributes and relationships in the demo schema to improve how some models are displayed in the list views
+- Changed the Python SDK connection timeout to 60s
+- Fix metric missing the query name in Prometheus data
+- Fixes an issue where docker compose would output ANSI control characters that don't support it
+- Prevent temporary directories generated by Docusaurus to be imported by Docker
+
+## [0.16.4](https://github.com/opsmill/infrahub/tree/v0.16.4) - 2024-10-17
+
+### Fixed
+
+- Fixed an issue on the UI where a new relationship was being added to the main branch instead of the current branch. ([#4598](https://github.com/opsmill/infrahub/issues/4598))
+
 ## [0.16.3](https://github.com/opsmill/infrahub/tree/v0.16.3) - 2024-10-10
 
 ### Removed

backend/infrahub/config.py

@@ -197,9 +197,9 @@ class DatabaseSettings(BaseSettings):
     tls_insecure: bool = Field(default=False, description="Indicates if TLS certificates are verified")
     tls_ca_file: Optional[str] = Field(default=None, description="File path to CA cert or bundle in PEM format")
     query_size_limit: int = Field(
-        default=5000,
+        default=5_000,
         ge=1,
-        le=20000,
+        le=20_000,
         description="The max number of records to fetch in a single query before performing internal pagination.",
     )
     max_depth_search_hierarchy: int = Field(

backend/infrahub/core/attribute.py

@@ -396,12 +396,15 @@ async def _update(self, db: InfrahubDatabase, at: Optional[Timestamp] = None) ->
         self.validate(value=self.value, name=self.name, schema=self.schema)
 
         # Check if the current value is still the default one
-        if (
-            self.is_default
-            and (self.schema.default_value is not None and self.schema.default_value != self.value)
-            or (self.schema.default_value is None and self.value is not None)
-        ):
-            self.is_default = False
+        if self.is_default:
+            if isinstance(self.value, Enum):
+                has_default_value = self.schema.default_value == self.value.value
+            else:
+                has_default_value = self.schema.default_value == self.value
+            if (self.schema.default_value is not None and not has_default_value) or (
+                self.schema.default_value is None and self.value is not None
+            ):
+                self.is_default = False
 
         query = await NodeListGetAttributeQuery.init(
             db=db,

backend/infrahub/core/branch/tasks.py

@@ -5,12 +5,17 @@
 from infrahub import lock
 from infrahub.core import registry
 from infrahub.core.branch import Branch
+from infrahub.core.diff.coordinator import DiffCoordinator
 from infrahub.core.diff.ipam_diff_parser import IpamDiffParser
+from infrahub.core.diff.merger.merger import DiffMerger
+from infrahub.core.diff.repository.repository import DiffRepository
 from infrahub.core.merge import BranchMerger
 from infrahub.core.migrations.schema.models import SchemaApplyMigrationData
 from infrahub.core.migrations.schema.tasks import schema_apply_migrations
+from infrahub.core.validators.determiner import ConstraintValidatorDeterminer
 from infrahub.core.validators.models.validate_migration import SchemaValidateMigrationData
 from infrahub.core.validators.tasks import schema_validate_migrations
+from infrahub.dependencies.registry import get_component_registry
 from infrahub.exceptions import ValidationError
 from infrahub.log import get_log_data
 from infrahub.message_bus import Meta, messages
@@ -27,15 +32,38 @@ async def rebase_branch(branch: str) -> None:
     await add_branch_tag(branch_name=branch)
 
     obj = await Branch.get_by_name(db=service.database, name=branch)
-    merger = BranchMerger(db=service.database, source_branch=obj, service=service)
+    base_branch = await Branch.get_by_name(db=service.database, name=registry.default_branch)
+    component_registry = get_component_registry()
+    diff_coordinator = await component_registry.get_component(DiffCoordinator, db=service.database, branch=obj)
+    diff_merger = await component_registry.get_component(DiffMerger, db=service.database, branch=obj)
+    merger = BranchMerger(
+        db=service.database,
+        diff_coordinator=diff_coordinator,
+        diff_merger=diff_merger,
+        source_branch=obj,
+        service=service,
+    )
+    diff_repository = await component_registry.get_component(DiffRepository, db=service.database, branch=obj)
+    enriched_diff = await diff_coordinator.update_branch_diff(base_branch=base_branch, diff_branch=obj)
+    if enriched_diff.get_all_conflicts():
+        raise ValidationError(
+            f"Branch {obj.name} contains conflicts with the default branch that must be addressed."
+            " Please review the diff for details and manually update the conflicts before rebasing."
+        )
+    node_diff_field_summaries = await diff_repository.get_node_field_summaries(
+        diff_branch_name=enriched_diff.diff_branch_name, diff_id=enriched_diff.uuid
+    )
+
+    candidate_schema = merger.get_candidate_schema()
+    determiner = ConstraintValidatorDeterminer(schema_branch=candidate_schema)
+    constraints = await determiner.get_constraints(node_diffs=node_diff_field_summaries)
 
     # If there are some changes related to the schema between this branch and main, we need to
-    #  - Run all the validations to ensure everything if correct before rebasing the branch
+    #  - Run all the validations to ensure everything is correct before rebasing the branch
     #  - Run all the migrations after the rebase
     if obj.has_schema_changes:
-        candidate_schema = merger.get_candidate_schema()
-        constraints = await merger.calculate_validations(target_schema=candidate_schema)
-
+        constraints += await merger.calculate_validations(target_schema=candidate_schema)
+    if constraints:
         error_messages = await schema_validate_migrations(
             message=SchemaValidateMigrationData(branch=obj, schema_branch=candidate_schema, constraints=constraints)
         )
@@ -44,25 +72,26 @@ async def rebase_branch(branch: str) -> None:
 
     schema_in_main_before = merger.destination_schema.duplicate()
 
-    async with service.database.start_transaction() as dbt:
-        await obj.rebase(db=dbt)
-        log.info("Branch successfully rebased")
-
-    if obj.has_schema_changes:
-        # NOTE there is a bit additional work in order to calculate a proper diff that will
-        # allow us to pull only the part of the schema that has changed, for now the safest option is to pull
-        # Everything
-        # schema_diff = await merger.has_schema_changes()
-        # TODO Would be good to convert this part to a Prefect Task in order to track it properly
-        updated_schema = await registry.schema.load_schema_from_db(
-            db=service.database,
-            branch=obj,
-            # schema=merger.source_schema.duplicate(),
-            # schema_diff=schema_diff,
-        )
-        registry.schema.set_schema_branch(name=obj.name, schema=updated_schema)
-        obj.update_schema_hash()
-        await obj.save(db=service.database)
+    async with lock.registry.global_graph_lock():
+        async with service.database.start_transaction() as dbt:
+            await obj.rebase(db=dbt)
+            log.info("Branch successfully rebased")
+
+        if obj.has_schema_changes:
+            # NOTE there is a bit additional work in order to calculate a proper diff that will
+            # allow us to pull only the part of the schema that has changed, for now the safest option is to pull
+            # Everything
+            # schema_diff = await merger.has_schema_changes()
+            # TODO Would be good to convert this part to a Prefect Task in order to track it properly
+            updated_schema = await registry.schema.load_schema_from_db(
+                db=service.database,
+                branch=obj,
+                # schema=merger.source_schema.duplicate(),
+                # schema_diff=schema_diff,
+            )
+            registry.schema.set_schema_branch(name=obj.name, schema=updated_schema)
+            obj.update_schema_hash()
+            await obj.save(db=service.database)
 
         # Execute the migrations
         migrations = await merger.calculate_migrations(target_schema=updated_schema)
@@ -116,11 +145,16 @@ async def merge_branch(branch: str, conflict_resolution: dict[str, bool] | None
     await add_branch_tag(branch_name=registry.default_branch)
 
     obj = await Branch.get_by_name(db=service.database, name=branch)
+    component_registry = get_component_registry()
 
     merger: BranchMerger | None = None
     async with lock.registry.global_graph_lock():
         async with service.database.start_transaction() as db:
-            merger = BranchMerger(db=db, source_branch=obj, service=service)
+            diff_coordinator = await component_registry.get_component(DiffCoordinator, db=db, branch=obj)
+            diff_merger = await component_registry.get_component(DiffMerger, db=db, branch=obj)
+            merger = BranchMerger(
+                db=db, diff_coordinator=diff_coordinator, diff_merger=diff_merger, source_branch=obj, service=service
+            )
             await merger.merge(conflict_resolution=conflict_resolution)
             await merger.update_schema()
 

backend/infrahub/core/diff/data_check_synchronizer.py

@@ -5,6 +5,7 @@
 from infrahub.core.manager import NodeManager
 from infrahub.core.node import Node
 from infrahub.database import InfrahubDatabase
+from infrahub.exceptions import SchemaNotFoundError
 
 from .conflicts_extractor import DiffConflictsExtractor
 from .model.path import ConflictSelection, EnrichedDiffConflict, EnrichedDiffRoot
@@ -22,14 +23,18 @@ def __init__(
         self.conflict_recorder = conflict_recorder
 
     async def synchronize(self, enriched_diff: EnrichedDiffRoot) -> list[Node]:
-        proposed_changes = await NodeManager.query(
-            db=self.db,
-            schema=InfrahubKind.PROPOSEDCHANGE,
-            filters={"source_branch": enriched_diff.diff_branch_name, "state": ProposedChangeState.OPEN},
-        )
+        try:
+            proposed_changes = await NodeManager.query(
+                db=self.db,
+                schema=InfrahubKind.PROPOSEDCHANGE,
+                filters={"source_branch": enriched_diff.diff_branch_name, "state": ProposedChangeState.OPEN},
+            )
+        except SchemaNotFoundError:
+            # if the CoreProposedChange schema does not exist, then there's nothing to do
+            proposed_changes = []
         if not proposed_changes:
             return []
-        enriched_conflicts = enriched_diff.get_all_conflicts()
+        enriched_conflicts_map = enriched_diff.get_all_conflicts()
         data_conflicts = await self.conflicts_extractor.get_data_conflicts(enriched_diff_root=enriched_diff)
         all_data_checks = []
         for pc in proposed_changes:
@@ -38,7 +43,7 @@ async def synchronize(self, enriched_diff: EnrichedDiffRoot) -> list[Node]:
             )
             all_data_checks.extend(core_data_checks)
             core_data_checks_by_id = {cdc.enriched_conflict_id.value: cdc for cdc in core_data_checks}  # type: ignore[attr-defined]
-            enriched_conflicts_by_id = {ec.uuid: ec for ec in enriched_conflicts}
+            enriched_conflicts_by_id = {ec.uuid: ec for ec in enriched_conflicts_map.values()}
             for conflict_id, core_data_check in core_data_checks_by_id.items():
                 enriched_conflict = enriched_conflicts_by_id.get(conflict_id)
                 if not enriched_conflict: