Skip to content

Commit 229074c

Browse files
ananth102ananth102
authored
Unit tests for Notebook Instance (#83)
#82 is required for this PR. This pull request has unit tests for the Notebook Instance, currently it has about 75% coverage and it takes about 0.1 seconds to run. Here are all the tests: Create: - Create=InvalidInput - Given one of the parameters is invalid, ko.Status shows a terminal condition - Create=Valid - Testing a successful create call ReadOne: - ReadOne=AfterCreate - Testing readOne right after create, the status should be in Pending with Resource synced being false. - ReadOne=Deleting - Testing readOne when deleting, resource synced should be false. - ReadOne=Fail - Testing readone when Describe fails, an error should be returned. - ReadOne=FailedState - Testing how readone handles the Failed State, it should have a failure reason. - ReadOne=InService - Testing how readone handles the InService state. Resource Synced should be true. - ReadOne=StoppedAfterUpdateACK - Testing how readone handles the Stopped state after it's been updated. In this test the Notebook Instance has been stopped by the controller. - ReadOne=StoppedAfterUpdateHuman - Testing how readone handles the Stopped state when stopped by a human for an update, StoppedByControllerMeta should not be in the status. - ReadOne=StoppedBeforeUpdateACK - Testing how readone handles the Stopped state when it's stopped right before an update. The StoppedByControllerMetadata status should be in UpdatePending. - ReadOne=StoppingDeleting - Testing how readone handles the Stopping state before going into deleting. The Resource synced condition should be set to False. - ReadOne=StoppingUpdate - Testing how readone handles the Stopping state before going into Update. The resource synced condition should be False and the StoppedByControllerMetadata status should be set to UpdatePending. - ReadOne=UpdatingUser - Testing how readone handles the Updating status when a user stops the Notebook. StoppedByControllerMetadata should not be in the status field. - ReadOne=UpdatingUserStopping- Testing if the readone call updates the Notebook's status from stopped to updating, when it recieves information from the server. - ReadOne=StartFail - Testing how readone would behave if start failed, test should return an error. Update: - Update=Pending - This test checks if the Notebook requeues while in Pending state. - Update=Stopping - This test checks if the Notebook requeues while in Stopping state. - Update=Updating - This test checks if the Notebook requeues while in updating state. - Update=InService - This test checks if the controller stops the Notebook. Note: Although a stop controller call has been made runtime has not performed an sdk.Find() as a result the NotebookInstanceStatus will be InService. - Update=Stopped - ByControllerUser - This test checks if the controller updates and does not set the StoppedByControllerMetadata status. - Update=InvalidInput - This test checks what the controller would do if update failed. It should return an error. - Update=Dissassociated - This test checks if the dissassociated parameters are being handled right. - Update=StopError - This test checks if the controller returns an error if StopNotebookInstance fails. Delete: - Delete=InService - This test checks if the Notebook stops before deleting. - Delete=Deleting - This test checks if the Notebook requeues while deleting - Delete=Pending - This test checks if the Notebook requeues while in pending state - Delete=Stopping - This test checks if the Notebook requeues while stopping - Delete=Updating - This test checks if the Notebook requeues while updating. - Delete=Stopped - This test checks if the Notebook requeues right after making the delete call. - Delete=Fail - This test checks if the Notebook returns an error if delete fails. - Delete=StopFail - This test checks if the Notebook returns an error if delete fails. Nomenclature: For readone/delete/update output nb_(operation)__(suffix) For update input: nb_update_<desired/latest>_(suffix) Directory structure: testdata - v1alpha - (folder which has yaml files) - crud operation - desired (folder which has input yaml files) - observed (folder which has output yaml files) - sdkapi - readOne - (server responses for readone) - create - (server responses for create) TODO: ReadOne Replace status with annotation in the following files: - testdata/v1alpha1/readone/desired/nb_readone_stopped_after_update_ack.yaml - testdata/v1alpha1/readone/desired/nb_readone_stopped_before_update_ack.yaml - testdata/v1alpha1/readone/desired/nb_readone_stopping_update.yaml - testdata/v1alpha1/readone/observed/nb_readone_stopped_before_update_ack.yaml - testdata/v1alpha1/readone/observed/nb_readone_stopped_before_update_ack.yaml - testdata/v1alpha1/readone/observed/nb_readone_stopping_update.yaml Update tests: Replace status with annotation in the following files: - testdata/v1alpha1/update/desired/nb_update_latest_stopped_controller_input.yaml - testdata/v1alpha1/update/desired/nb_update_latest_stopped_dissassociated.yaml - testdata/v1alpha1/update/desired/nb_update_latest_stopping_input.yaml - testdata/v1alpha1/update/observed/nb_update_inservice.yaml - testdata/v1alpha1/update/observed/nb_update_stopped_controller.yaml Delete tests: Add tests for complete deletion(once unit test supports that)
1 parent fced166 commit 229074c

File tree

64 files changed

+1718
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

64 files changed

+1718
-0
lines changed

‎pkg/resource/notebook_instance/manager_test_suite_test.go

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License"). You may
4+
// not use this file except in compliance with the License. A copy of the
5+
// License is located at
6+
//
7+
// http://aws.amazon.com/apache2.0/
8+
//
9+
// or in the "license" file accompanying this file. This file is distributed
10+
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
11+
// express or implied. See the License for the specific language governing
12+
// permissions and limitations under the License.
13+
14+
package notebook_instance
15+
16+
import (
17+
"errors"
18+
"fmt"
19+
20+
"path/filepath"
21+
"testing"
22+
23+
ackv1alpha1 "github.com/aws-controllers-k8s/runtime/apis/core/v1alpha1"
24+
ackmetrics "github.com/aws-controllers-k8s/runtime/pkg/metrics"
25+
acktypes "github.com/aws-controllers-k8s/runtime/pkg/types"
26+
"github.com/aws-controllers-k8s/sagemaker-controller/pkg/testutil"
27+
mocksvcsdkapi "github.com/aws-controllers-k8s/sagemaker-controller/test/mocks/aws-sdk-go/sagemaker"
28+
svcsdk "github.com/aws/aws-sdk-go/service/sagemaker"
29+
"github.com/ghodss/yaml"
30+
"github.com/google/go-cmp/cmp"
31+
"github.com/google/go-cmp/cmp/cmpopts"
32+
"go.uber.org/zap/zapcore"
33+
ctrlrtzap "sigs.k8s.io/controller-runtime/pkg/log/zap"
34+
)
35+
36+
// provideResourceManagerWithMockSDKAPI accepts MockSageMakerAPI and returns pointer to resourceManager
37+
// the returned resourceManager is configured to use mockapi api.
38+
func provideResourceManagerWithMockSDKAPI(mockSageMakerAPI *mocksvcsdkapi.SageMakerAPI) *resourceManager {
39+
zapOptions := ctrlrtzap.Options{
40+
Development: true,
41+
Level: zapcore.InfoLevel,
42+
}
43+
fakeLogger := ctrlrtzap.New(ctrlrtzap.UseFlagOptions(&zapOptions))
44+
return &resourceManager{
45+
rr: nil,
46+
awsAccountID: "",
47+
awsRegion: "",
48+
sess: nil,
49+
sdkapi: mockSageMakerAPI,
50+
log: fakeLogger,
51+
metrics: ackmetrics.NewMetrics("sagemaker"),
52+
}
53+
}
54+
55+
// TestNotebookInstance runs the test suite for feature group
56+
func TestNotebookInstanceTestSuite(t *testing.T) {
57+
defer func() {
58+
if r := recover(); r != nil {
59+
fmt.Println(testutil.RecoverPanicString, r)
60+
t.Fail()
61+
}
62+
}()
63+
var ts = testutil.TestSuite{}
64+
testutil.LoadFromFixture(filepath.Join("testdata", "test_suite.yaml"), &ts)
65+
var delegate = testRunnerDelegate{t: t}
66+
var runner = testutil.TestSuiteRunner{TestSuite: &ts, Delegate: &delegate}
67+
runner.RunTests()
68+
}
69+
70+
// testRunnerDelegate implements testutil.TestRunnerDelegate
71+
type testRunnerDelegate struct {
72+
t *testing.T
73+
}
74+
75+
func (d *testRunnerDelegate) ResourceDescriptor() acktypes.AWSResourceDescriptor {
76+
return &resourceDescriptor{}
77+
}
78+
79+
func (d *testRunnerDelegate) ResourceManager(mocksdkapi *mocksvcsdkapi.SageMakerAPI) acktypes.AWSResourceManager {
80+
return provideResourceManagerWithMockSDKAPI(mocksdkapi)
81+
}
82+
83+
func (d *testRunnerDelegate) GoTestRunner() *testing.T {
84+
return d.t
85+
}
86+
87+
func (d *testRunnerDelegate) EmptyServiceAPIOutput(apiName string) (interface{}, error) {
88+
if apiName == "" {
89+
return nil, errors.New("no API name specified")
90+
}
91+
//TODO: use reflection, template to auto generate this block/method.
92+
switch apiName {
93+
case "CreateNotebookInstanceWithContext":
94+
var output svcsdk.CreateNotebookInstanceOutput
95+
return &output, nil
96+
case "DeleteNotebookInstanceWithContext":
97+
var output svcsdk.DeleteNotebookInstanceOutput
98+
return &output, nil
99+
case "DescribeNotebookInstanceWithContext":
100+
var output svcsdk.DescribeNotebookInstanceOutput
101+
return &output, nil
102+
case "UpdateNotebookInstanceWithContext":
103+
var output svcsdk.UpdateNotebookInstanceOutput
104+
return &output, nil
105+
case "StartNotebookInstance":
106+
var output svcsdk.StartNotebookInstanceOutput
107+
return &output, nil
108+
case "StopNotebookInstance":
109+
var output svcsdk.StopNotebookInstanceOutput
110+
return &output, nil
111+
}
112+
return nil, errors.New(fmt.Sprintf("no matching API name found for: %s", apiName))
113+
}
114+
115+
func (d *testRunnerDelegate) Equal(a acktypes.AWSResource, b acktypes.AWSResource) bool {
116+
ac := a.(*resource)
117+
bc := b.(*resource)
118+
// Ignore LastTransitionTime since it gets updated each run.
119+
opts := []cmp.Option{cmpopts.EquateEmpty(), cmpopts.IgnoreFields(ackv1alpha1.Condition{}, "LastTransitionTime")}
120+
121+
if cmp.Equal(ac.ko.Status, bc.ko.Status, opts...) {
122+
return true
123+
} else {
124+
fmt.Printf("Difference (-expected +actual):\n\n")
125+
fmt.Println(cmp.Diff(ac.ko.Status, bc.ko.Status, opts...))
126+
return false
127+
}
128+
}
129+
130+
// Checks to see if the given yaml file, with name stored as expectation,
131+
// matches the yaml marshal of the AWSResource stored as actual.
132+
func (d *testRunnerDelegate) YamlEqual(expectation string, actual acktypes.AWSResource) bool {
133+
// Build a tmp file for the actual yaml.
134+
actualResource := actual.(*resource)
135+
actualYamlByteArray, _ := yaml.Marshal(actualResource.ko)
136+
return testutil.IsYamlEqual(&expectation, &actualYamlByteArray)
137+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/create/create_success.json

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
{
2+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test"
3+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/delete_resp.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Deleting",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_deleting.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Deleting",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_failed_state.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": "VolumeError",
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Failed",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_inservice.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": ["ml.eia1.medium"],
3+
"AdditionalCodeRepositories": ["code-repo"],
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": "my-code",
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": "op",
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": "my-lfc",
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "InService",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": ["woof"],
19+
"SubnetId": "abc",
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_pending_after_create.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Pending",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_stopped.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Stopped",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 8
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_stopped_after_update_ack.json

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
{
2+
"AcceleratorTypes": null,
3+
"AdditionalCodeRepositories": null,
4+
"CreationTime": "0001-01-01T00:00:00Z",
5+
"DefaultCodeRepository": null,
6+
"DirectInternetAccess": "Enabled",
7+
"FailureReason": null,
8+
"InstanceType": "ml.t2.medium",
9+
"KmsKeyId": null,
10+
"LastModifiedTime": "2021-08-15T22:06:18.767Z",
11+
"NetworkInterfaceId": null,
12+
"NotebookInstanceArn": "arn:aws:sagemaker:us-east-2:1234:notebook-instance/test",
13+
"NotebookInstanceLifecycleConfigName": null,
14+
"NotebookInstanceName": "test",
15+
"NotebookInstanceStatus": "Stopped",
16+
"RoleArn": "arn:aws:iam::1234:role/service-role/AmazonSageMakerServiceCatalogProductsUseRole",
17+
"RootAccess": "Enabled",
18+
"SecurityGroups": null,
19+
"SubnetId": null,
20+
"Url": "test.notebook.us-east-2.sagemaker.aws",
21+
"VolumeSizeInGB": 9
22+
}

‎pkg/resource/notebook_instance/testdata/sdkapi/readone/readone_stopped_after_update_human.json

Whitespace-only changes.

0 commit comments

Comments
 (0)