Skip to content

Commit 22bc30c

Browse files
jialei-chenedwardmediaEdward Sun
authored
Add a resource: Discovery Engine Data Store (GoogleCloudPlatform#9860)
* Create a configuration file product.yaml for product discoveryengine * Create resource DataStore configuration file template DataStore.yaml * Modify resource DataStore configuration file template to match the API resource's documented behavior * Add each API resource field to configuration file properties attribute * Add IAM support in Terraform to match DataStore API resource IAM support * Polish field contents in product configuration file. * Modify DataStore.yaml for LRO handling, parameters matching and readability improvement. * Add a DataStore create test. * Modify product name from DiscoveryEngine to Discoveryengine so that generated resource name would be google_discoveryengine_data_store * change to GA in doc (GoogleCloudPlatform#9491) Co-authored-by: Edward Sun <sunedward@google.com> * Add value of to template into the generated tests to fix error 'fmt.Sprintf format %s reads arg #3, but call has 2 args'. * Remove Api::Product::ApiReference to match the change GoogleCloudPlatform#9536 * Add var collection_id to unit test examples in resource config. * Fix argument missing error in unit tests. * Update resource create unit test case fields to contain display_name, industry_vertical, content_config. * Fix errors in acceptance tests: now acc test fails at invalid parent field of CreateDataStoreRequest. * Set collection_id as default_collection. * Remove uncessary variables under unit test example. * Fix acceptance tests issues by (1) removing iam_policy and (2) adding import_format. * Clean up configs: (1) update api version for tpg-beta to v1alpha; (2) remove fields for std Update method; (3) rename structured datastore example. * Add examples and tests for all types of DataStore. * Remove redundant field project from create test examples. * Fix lint-yaml check failure. * Add field solution_types to acceptance tests to satisfy coverage requirements. * Clean up industry_vertical enum values. * Add a new query parameter createAdvancedSiteSearch to create_url. * Add a create test for advanced site search data store. * Add IAM roles/discoveryengine.admin. * Remove role config from .ci/infra/terraform/main.tf * Remove beta version in product.yaml because both ga and beta versions have the same base_url. * Rename product from Discoveryengine to DiscoveryEngine. * Remove unnecessary configs and improve description readability. * Mark property solution_types as immutable. * Add update related settings and update test. * Add delete_url for resource's Delete method. * Remove all but one test example. * Rename test example to make it more generic. * Modify first template in update test to omit all non-required fields. * Fix the update test: update display_name not data_store_id. * Modify property description: remove brackets. * Update API version from v1alpha to v1. * Support regional APIs. * Modify API reference link. --------- Co-authored-by: Edward Sun <42220489+edwardmedia@users.noreply.github.com> Co-authored-by: Edward Sun <sunedward@google.com>
1 parent 2295649 commit 22bc30c

File tree

4 files changed

+245
-0
lines changed

4 files changed

+245
-0
lines changed

‎mmv1/products/discoveryengine/DataStore.yaml‎

Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
# Copyright 2023 Google Inc.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
--- !ruby/object:Api::Resource
15+
name: 'DataStore'
16+
description: |
17+
Data store is a collection of websites and documents used to find answers for
18+
end-user's questions in Discovery Engine (a.k.a. Vertex AI Search and
19+
Conversation).
20+
references: !ruby/object:Api::Resource::ReferenceLinks
21+
guides:
22+
'Create a search data store': 'https://cloud.google.com/generative-ai-app-builder/docs/create-data-store-es'
23+
api: 'https://cloud.google.com/generative-ai-app-builder/docs/reference/rest/v1/projects.locations.collections.dataStores'
24+
25+
base_url: 'projects/{{project}}/locations/{{location}}/collections/default_collection/dataStores'
26+
self_link: 'projects/{{project}}/locations/{{location}}/collections/default_collection/dataStores/{{data_store_id}}'
27+
create_url: 'projects/{{project}}/locations/{{location}}/collections/default_collection/dataStores?dataStoreId={{data_store_id}}&createAdvancedSiteSearch={{create_advanced_site_search}}'
28+
update_verb: :PATCH
29+
update_mask: true
30+
delete_url: 'projects/{{project}}/locations/{{location}}/collections/default_collection/dataStores/{{data_store_id}}'
31+
import_format:
32+
[
33+
'projects/{{project}}/locations/{{location}}/collections/default_collection/dataStores/{{data_store_id}}',
34+
]
35+
36+
autogen_async: true
37+
async: !ruby/object:Api::OpAsync
38+
operation: !ruby/object:Api::OpAsync::Operation
39+
path: "name"
40+
base_url: "{{op_id}}"
41+
wait_ms: 1000
42+
timeouts: !ruby/object:Api::Timeouts
43+
insert_minutes: 60
44+
update_minutes: 60
45+
delete_minutes: 60
46+
result: !ruby/object:Api::OpAsync::Result
47+
path: "response"
48+
status: !ruby/object:Api::OpAsync::Status
49+
path: "done"
50+
complete: true
51+
allowed:
52+
- true
53+
- false
54+
error: !ruby/object:Api::OpAsync::Error
55+
path: "error"
56+
message: "message"
57+
58+
examples:
59+
- !ruby/object:Provider::Terraform::Examples
60+
name: "discoveryengine_datastore_basic"
61+
primary_resource_id: 'basic'
62+
primary_resource_name:
63+
'fmt.Sprintf("tf_test_data_store%s", context["random_suffix"])'
64+
vars:
65+
data_store_id: "data-store-id"
66+
parameters:
67+
- !ruby/object:Api::Type::String
68+
name: 'location'
69+
required: true
70+
immutable: true
71+
url_param_only: true
72+
description: |
73+
The geographic location where the data store should reside. The value can
74+
only be one of "global", "us" and "eu".
75+
- !ruby/object:Api::Type::String
76+
name: 'dataStoreId'
77+
required: true
78+
immutable: true
79+
url_param_only: true
80+
description: |
81+
The unique id of the data store.
82+
- !ruby/object:Api::Type::Boolean
83+
name: 'createAdvancedSiteSearch'
84+
default_value: false
85+
url_param_only: true
86+
description: |
87+
If true, an advanced data store for site search will be created. If the
88+
data store is not configured as site search (GENERIC vertical and
89+
PUBLIC_WEBSITE contentConfig), this flag will be ignored.
90+
91+
properties:
92+
- !ruby/object:Api::Type::String
93+
name: 'name'
94+
description: |
95+
The unique full resource name of the data store. Values are of the format
96+
`projects/{project}/locations/{location}/collections/{collection_id}/dataStores/{data_store_id}`.
97+
This field must be a UTF-8 encoded string with a length limit of 1024
98+
characters.
99+
output: true
100+
- !ruby/object:Api::Type::String
101+
name: 'displayName'
102+
description: |
103+
The display name of the data store. This field must be a UTF-8 encoded
104+
string with a length limit of 128 characters.
105+
required: true
106+
- !ruby/object:Api::Type::Enum
107+
name: 'industryVertical'
108+
description: |
109+
The industry vertical that the data store registers.
110+
values:
111+
- :GENERIC
112+
- :MEDIA
113+
immutable: true
114+
required: true
115+
- !ruby/object:Api::Type::Array
116+
name: 'solutionTypes'
117+
description: |
118+
The solutions that the data store enrolls.
119+
item_type: !ruby/object:Api::Type::Enum
120+
name: 'solutionType'
121+
description: |
122+
The type of solution.
123+
values:
124+
- :SOLUTION_TYPE_RECOMMENDATION
125+
- :SOLUTION_TYPE_SEARCH
126+
- :SOLUTION_TYPE_CHAT
127+
immutable: true
128+
- !ruby/object:Api::Type::String
129+
name: 'defaultSchemaId'
130+
description: |
131+
The id of the default Schema associated with this data store.
132+
output: true
133+
- !ruby/object:Api::Type::Enum
134+
name: 'contentConfig'
135+
description: |
136+
The content config of the data store.
137+
values:
138+
- :NO_CONTENT
139+
- :CONTENT_REQUIRED
140+
- :PUBLIC_WEBSITE
141+
immutable: true
142+
required: true
143+
- !ruby/object:Api::Type::Time
144+
name: "createTime"
145+
description: |
146+
Timestamp when the DataStore was created.
147+
output: true

‎mmv1/products/discoveryengine/product.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Copyright 2023 Google Inc.
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
--- !ruby/object:Api::Product
15+
name: DiscoveryEngine
16+
display_name: Discovery Engine
17+
versions:
18+
- !ruby/object:Api::Product::Version
19+
name: ga
20+
base_url: https://{{location}}-discoveryengine.googleapis.com/v1/
21+
scopes:
22+
- https://www.googleapis.com/auth/cloud-platform

‎mmv1/templates/terraform/examples/discoveryengine_datastore_basic.tf.erb‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
resource "google_discovery_engine_data_store" "basic" {
2+
location = "global"
3+
data_store_id = "<%= ctx[:vars]['data_store_id'] %>"
4+
display_name = "tf-test-structured-datastore"
5+
industry_vertical = "GENERIC"
6+
content_config = "NO_CONTENT"
7+
solution_types = ["SOLUTION_TYPE_SEARCH"]
8+
create_advanced_site_search = false
9+
}

‎mmv1/third_party/terraform/services/discoveryengine/resource_discovery_engine_data_store_test.go‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
package discoveryengine_test
2+
3+
import (
4+
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
5+
"github.com/hashicorp/terraform-provider-google/google/acctest"
6+
"testing"
7+
)
8+
9+
func TestAccDiscoveryEngineDataStore_discoveryengineDatastoreBasicExample_update(t *testing.T) {
10+
t.Parallel()
11+
12+
context := map[string]interface{}{
13+
"random_suffix": acctest.RandString(t, 10),
14+
}
15+
16+
acctest.VcrTest(t, resource.TestCase{
17+
PreCheck: func() { acctest.AccTestPreCheck(t) },
18+
ProtoV5ProviderFactories: acctest.ProtoV5ProviderFactories(t),
19+
CheckDestroy: testAccCheckDiscoveryEngineDataStoreDestroyProducer(t),
20+
Steps: []resource.TestStep{
21+
{
22+
Config: testAccDiscoveryEngineDataStore_discoveryengineDatastoreBasicExample_basic(context),
23+
},
24+
{
25+
ResourceName: "google_discovery_engine_data_store.basic",
26+
ImportState: true,
27+
ImportStateVerify: true,
28+
ImportStateVerifyIgnore: []string{"location", "data_store_id", "create_advanced_site_search"},
29+
},
30+
{
31+
Config: testAccDiscoveryEngineDataStore_discoveryengineDatastoreBasicExample_update(context),
32+
},
33+
{
34+
ResourceName: "google_discovery_engine_data_store.basic",
35+
ImportState: true,
36+
ImportStateVerify: true,
37+
ImportStateVerifyIgnore: []string{"location", "data_store_id", "create_advanced_site_search"},
38+
},
39+
},
40+
})
41+
}
42+
43+
func testAccDiscoveryEngineDataStore_discoveryengineDatastoreBasicExample_basic(context map[string]interface{}) string {
44+
return acctest.Nprintf(`
45+
resource "google_discovery_engine_data_store" "basic" {
46+
location = "global"
47+
data_store_id = "tf-test-data-store-id%{random_suffix}"
48+
display_name = "tf-test-structured-datastore"
49+
industry_vertical = "GENERIC"
50+
content_config = "NO_CONTENT"
51+
}
52+
`, context)
53+
}
54+
55+
func testAccDiscoveryEngineDataStore_discoveryengineDatastoreBasicExample_update(context map[string]interface{}) string {
56+
return acctest.Nprintf(`
57+
resource "google_discovery_engine_data_store" "basic" {
58+
location = "global"
59+
data_store_id = "tf-test-data-store-id%{random_suffix}"
60+
display_name = "updated-tf-test-structured-datastore"
61+
industry_vertical = "GENERIC"
62+
content_config = "NO_CONTENT"
63+
solution_types = ["SOLUTION_TYPE_SEARCH"]
64+
create_advanced_site_search = false
65+
}
66+
`, context)
67+
}

0 commit comments

Comments
 (0)