KnowledgeBase Resource

[nesymno]

Issue #2117

Description of changes:

• Added KnowledgeBase resource

TODO:

• Add hooks for tags
• Add e2e tests

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@knottnt Hey! That is my first contribution to you guys, so can you check it out and give some advices, maybe I did something wrong. At all, I also want to add e2e tests and cover all endpoints from KnowledgeBase resource.

[ack-prow]

Hi @nesymno. Thanks for your PR.

I'm waiting for a aws-controllers-k8s member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[nesymno]

/assign @knottnt

[knottnt]

Left a few comments.

[knottnt]

This field path should be ignored so that it isn't included in the spec. An idempotency token isn't something the user should be setting the in the Spec.

[knottnt]

Were you seeing the code-generator fail to recognize these operations? I believe it should pick these up automatically.

[nesymno]

I had no errors during the code-generation. But I got it, will remove this.

[knottnt]

For terminal codes we only want to specify errors where the user definitely must change the resource's Spec in order for the controller to reconcile. Examples, of this would invalid inputs such as the ValidationException. Generally, we don't include ResourceNotFoundException as this may not require a change to resource's Spec. For example the missing resource may be created in the future.

You can find more details here.

[knottnt]

To fully implement resource tagging I believe you'll need to add some custom hooks similar to those used for the Agent resource.

[knottnt]

It looks like the AuthConfiguration.Type field is having a trailing underscore added due to "type" being a Go keyword. In these cases we prefer to force the Go json tag back to the "type". You can see an example of this in the kafka-controller's generator.yaml. It looks like there are a few other instances of type_ in the Spec besides this one.

[nesymno]

@knottnt Yeah. What about a case, where field is a slice of structs that contain type_?
I didn't find anything about that in docs and I also checked out a lot of another controllers, but didn't find same case.

That is how I doing that now:

KnowledgeBaseConfiguration.SQLKnowledgeBaseConfiguration.RedshiftConfiguration.StorageConfigurations.Type:
        go_tag: json:"type,omitempty"

Where StorageConfigurations = []struct{Type: *string}, but looks like it doesn't work

I saw that I could use type: []*struct{Type: *string} (for example), but how to say to generator that you need to change json tags for each slice element with name 'Type'?

[knottnt]

Testing locally it looks like the below works for setting the json tag for the individual elements.

KnowledgeBaseConfiguration.SQLKnowledgeBaseConfiguration.RedshiftConfiguration.StorageConfigurations.Type:
        go_tag: json:"type,omitempty"

[nesymno]

Okay. I just checked generated yaml and it still contains type_ for the list item.

[knottnt]

Thanks @nesymno for the contribution! For the E2E tests you can look at the Agent tests for an example. If you have any questions feel free to reach out!

[ack-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nesymno
Once this PR has been reviewed and has the lgtm label, please ask for approval from knottnt by writing /assign @knottnt in a comment. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[nesymno]

@knottnt I have stuck on e2e tests. Can you check them pls? I know, that tags related stuff will not work now, but crud tests also fails and I got next error:

_________________________________________________________________________________ ERROR at setup of TestKnowledgeBase.test_crud _________________________________________________________________________________
[gw2] linux -- Python 3.11.13 /usr/local/bin/python3.11

    @pytest.fixture(scope="module")
    def simple_knowledge_base():
        knowledge_base_name = random_suffix_name("bedrock-test-kb", 32)
        knowledge_base_description = "Test knowledge base for e2e testing"
        knowledge_base_role_arn = get_bootstrap_resources().AgentRole.arn
        embedding_model_arn = "arn:aws:bedrock:us-west-2::foundation-model/amazon.titan-embed-text-v1"
        opensearch_collection_arn = "arn:aws:aoss:us-west-2:050752643586:collection/kqs7wb5kssgmnlomcb7e"
        vector_index_name = "e2e-index"
        knowledge_base_tag_key = "test1"
        knowledge_base_tag_value = "value1"
    
        replacements = REPLACEMENT_VALUES.copy()
        replacements["KNOWLEDGE_BASE_NAME"] = knowledge_base_name
        replacements["KNOWLEDGE_BASE_DESCRIPTION"] = knowledge_base_description
        replacements["KNOWLEDGE_BASE_ROLE_ARN"] = knowledge_base_role_arn
        replacements["EMBEDDING_MODEL_ARN"] = embedding_model_arn
        replacements["OPENSEARCH_COLLECTION_ARN"] = opensearch_collection_arn
        replacements["VECTOR_INDEX_NAME"] = vector_index_name
        replacements["TAG_KEY_1"] = knowledge_base_tag_key
        replacements["TAG_VALUE_1"] = knowledge_base_tag_value
    
        resource_data = load_resource(
            "knowledge_base",
            additional_replacements=replacements,
        )
    
        ref = k8s.CustomResourceReference(
            CRD_GROUP,
            CRD_VERSION,
            KNOWLEDGE_BASE_RESOURCE_PLURAL,
            knowledge_base_name,
            namespace="default",
        )
    
        logger.info("Creating KnowledgeBase %s", knowledge_base_name)
        k8s.create_custom_resource(ref, resource_data)
        cr = k8s.wait_resource_consumed_by_controller(ref)
    
        assert k8s.get_resource_exists(ref)
        assert cr is not None
        assert "status" in cr
>       assert "knowledgeBaseID" in cr["status"]
E       AssertionError: assert 'knowledgeBaseID' in {'ackResourceMetadata': {'ownerAccountID': '050752643586', 'region': 'us-west-2'}, 'conditions': [{'message': 'ValidationException: The knowledge base storage configuration provided is invalid... Request failed: [security_exception] 403 Forbidden', 'status': 'True', 'type': 'ACK.Terminal'}, {'lastTransitionTime': '2025-06-20T09:07:21Z', 'message': 'Resource not synced', 'reason': 'resource is in terminal condition', 'status': 'False', ...}]}                                                                                                                                  

tests/test_knowledge_base.py:79: AssertionError

As for me, yaml looks good, no? The error code is 403, so maybe I need to fix role or policies? Help me pls.