-
Notifications
You must be signed in to change notification settings - Fork 399
feat(docs): add devsite-help and snippets for Retries, Update Masks, and OCC #15308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 2 commits
cd55d59
8621a55
cfcd79c
6e594a6
4b30c4c
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| # Optimistic Concurrency Control (OCC) Loop for IAM | ||
|
|
||
| ## Introduction to OCC | ||
|
|
||
| Optimistic Concurrency Control (OCC) is a strategy used to manage shared resources and prevent "lost updates" or race conditions when multiple users or processes attempt to modify the same resource simultaneously. | ||
|
|
||
| In the context of Google Cloud .NET libraries, IAM Policy objects contain an `Etag` property. When calling `SetIamPolicy`, the client library includes this `Etag`. If the server detects that the `Etag` provided does not match the current version on the server, it throws an RPC exception with the status `Aborted` or `FailedPrecondition`. | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| ## Implementing the OCC Loop | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| The core of the implementation is a `while` loop wrapped in a `try-catch` block handling specific gRPC exceptions. | ||
|
|
||
| ### **Steps of the Loop** | ||
|
|
||
| | Step | Action | C\# Implementation | | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| | ----- | ----- | ----- | | ||
| | **1\. Read** | Fetch the current IAM Policy. | `await client.GetIamPolicyAsync(name)` | | ||
| | **2\. Modify** | Apply changes to the `Policy` object. | Modify `policy.Bindings` collection. | | ||
| | **3\. Write** | Attempt to set the policy. | `await client.SetIamPolicyAsync(name, policy)` | | ||
| | **4\. Retry** | Catch specific `RpcException`. | `catch (RpcException ex) when (ex.StatusCode == StatusCode.Aborted)` | | ||
|
|
||
| ## Examples | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| The following example demonstrates how to implement the OCC loop using the `Google.Cloud.ResourceManager.V3` library. | ||
|
|
||
| [!code-cs[](../examples/help.OccForIam.txt#OccForIam)] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # Configuring Retries and Timeouts | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| In the Google Cloud C\# Client Libraries, configuring retry logic is managed by setting properties on the client configuration object or by explicitly passing `CallSettings` to the RPC method. | ||
|
|
||
| ## **Global Client Configuration** | ||
|
|
||
| You can configure the default retry behavior for an entire client instance by modifying the client's internal configuration object during creation. This is useful for uniformly applying a specific policy across all calls made by that client. | ||
|
|
||
| The retry settings control the exponential backoff strategy, which determines how long the client waits between retry attempts after a recoverable failure. | ||
|
|
||
| [!code-cs[](../examples/help.Retries.txt#ServiceSettingsRetries)] | ||
|
|
||
| ## **Per-Call Configuration (Recommended)** | ||
|
|
||
| For most use cases, it is recommended to override settings for specific, high-contention, or time-sensitive calls using `CallSettings`. This avoids changing the default behavior of the entire client. | ||
|
|
||
| You pass the `CallSettings` object as an optional final argument to the asynchronous method. | ||
|
|
||
| [!code-cs[](../examples/help.Retries.txt#CallSettingsRetries)] | ||
|
|
||
| ## **Key RetrySettings Properties** | ||
|
|
||
| When constructing `RetrySettings.FromBackoff`, you can use the following parameters to fine-tune the exponential backoff strategy: | ||
|
|
||
| | Parameter | Type | Description | | ||
| | ----- | ----- | ----- | | ||
| | `maxAttempts` | `int` | The maximum number of retry attempts. | | ||
| | `initialDelay` | `TimeSpan` | Wait time before the first retry. | | ||
| | `delayMultiplier` | `double` | Multiplier applied to the delay after each failure (e.g., 2.0). | | ||
| | `maxDelay` | `TimeSpan` | The maximum wait time between any two retries. | | ||
| | `totalTimeout` | `TimeSpan` | Total time allowed for the request (including all retries) before giving up. | | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Update Masks | ||
|
|
||
| ## Optimizing Resource Updates with FieldMask | ||
|
|
||
| When updating resources (often corresponding to HTTP PATCH requests) in Google Cloud APIs, you generally use an **Update Mask** (represented by `Google.Protobuf.WellKnownTypes.FieldMask`). | ||
|
|
||
| The purpose of the Update Mask is to tell the server exactly which fields you intend to modify, preventing accidental overwrites or resetting of other fields that you did not include in your request object. | ||
|
|
||
| By providing a `FieldMask`, you explicitly declare the subset of fields in the resource object that should be updated. | ||
|
|
||
| ## Constructing and Applying the FieldMask | ||
|
|
||
| The `FieldMask` object is part of the Google Protobuf library and contains a collection of strings representing the fields to be modified. | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| **Crucial Point:** The strings in the `FieldMask.Paths` collection **must** correspond to the **snake\_case** field names defined in the Protocol Buffer (protobuf) schema, not the C\# property names (which are PascalCase). | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| # Examples | ||
|
|
||
| [!code-cs[](../examples/help.UpdateMask.txt#UpdateMasks)] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| // Copyright 2025 Google Inc. All Rights Reserved. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| using Google.Api.Gax.ResourceNames; | ||
| using Google.Cloud.Iam.V1; | ||
| using Google.Cloud.ResourceManager.V3; | ||
| using Grpc.Core; | ||
|
|
||
| namespace Google.Cloud.Tools.Snippets | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| public class OccForIamSnippets | ||
| { | ||
| public async Task<int> OccForIam() | ||
| { | ||
| string projectId = "your-project-id"; | ||
| string role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"; | ||
| string member = "user:betterbrent@google.com"; | ||
|
||
| int maxRetries = 5; | ||
|
|
||
| // Sample: OccForIam | ||
| // Required using directives: | ||
| // using Google.Api.Gax.ResourceNames; | ||
| // using Google.Cloud.Iam.V1; | ||
| // using Google.Cloud.ResourceManager.V3; | ||
| // using Grpc.Core; | ||
|
|
||
| // Setup Client | ||
| ProjectsClient client = await ProjectsClient.CreateAsync(); | ||
| ProjectName resourceName = ProjectName.FromProject(projectId); | ||
|
|
||
| int retries = 0; | ||
|
|
||
| // --- START OCC LOOP --- | ||
| while (retries < maxRetries) | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| try | ||
| { | ||
| // READ: Get the current policy (includes Etag) | ||
| Console.WriteLine($"Attempt {retries + 1}: Reading policy for {resourceName}..."); | ||
| Policy policy = await client.GetIamPolicyAsync(resourceName); | ||
|
|
||
| // MODIFY: Apply changes to the local Policy object | ||
| Binding binding = policy.Bindings.FirstOrDefault(b => b.Role == role); | ||
|
|
||
| if (binding != null) | ||
| { | ||
| if (!binding.Members.Contains(member)) | ||
| { | ||
| binding.Members.Add(member); | ||
| } | ||
| } | ||
| else | ||
| { | ||
| policy.Bindings.Add(new Binding | ||
| { | ||
| Role = role, | ||
| Members = { member } | ||
| }); | ||
| } | ||
|
|
||
| // WRITE: Attempt to set the modified policy | ||
| // The 'policy' object contains the original 'Etag' from Step 1. | ||
| Console.WriteLine($"Attempt {retries + 1}: Writing modified policy..."); | ||
| Policy updatedPolicy = await client.SetIamPolicyAsync(resourceName, policy); | ||
|
|
||
| // SUCCESS | ||
| Console.WriteLine("Successfully updated IAM policy."); | ||
| return 0; | ||
| } | ||
| catch (RpcException ex) when ( | ||
| ex.StatusCode == StatusCode.Aborted || | ||
| ex.StatusCode == StatusCode.FailedPrecondition) | ||
amanda-tarafa marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| // RETRY LOGIC | ||
| retries++; | ||
| Console.WriteLine($"Concurrency conflict (Etag mismatch). Retrying... ({retries}/{maxRetries})"); | ||
|
|
||
| if (retries >= maxRetries) | ||
| { | ||
| Console.WriteLine("Failed to update policy after max retries."); | ||
| throw; | ||
| } | ||
|
|
||
| // Simple backoff | ||
| await Task.Delay(100 * retries); | ||
| } | ||
| } | ||
| // --- END OCC LOOP --- | ||
|
|
||
| return 1; // non-zero, failed | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| // End sample | ||
| } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| // Copyright 2025 Google Inc. All Rights Reserved. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| using Google.Cloud.SecretManager.V1; | ||
| using Google.Api.Gax.Grpc; | ||
| using Google.Api.Gax; | ||
| using Grpc.Core; | ||
|
|
||
| namespace Google.Cloud.Tools.Snippets | ||
| { | ||
| public class RetriesSnippets | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| public async Task ServiceSettingsRetries() | ||
| { | ||
| // Sample: ServiceSettingsRetries | ||
| // Required using directives: | ||
| // using Google.Cloud.SecretManager.V1; | ||
| // using Google.Api.Gax.Grpc; | ||
| // using Google.Api.Gax; | ||
| // using Grpc.Core; | ||
|
|
||
| // Define custom retry settings | ||
| var retrySettings = RetrySettings.FromExponentialBackoff( | ||
| maxAttempts: 3, // Maximum number of attempts | ||
| initialBackoff: TimeSpan.FromMilliseconds(500), // Wait 0.5s before the first retry | ||
| maxBackoff: TimeSpan.FromSeconds(5), // Cap the wait at 5s | ||
| backoffMultiplier: 2.0, // Double the wait time on each failure | ||
| retryFilter: RetrySettings.FilterForStatusCodes(StatusCode.Unavailable, StatusCode.DeadlineExceeded) | ||
| ); | ||
|
|
||
| // Apply your custom retry settings to the specific method | ||
| var settings = new SecretManagerServiceSettings(); | ||
| settings.ListSecretsSettings = settings.ListSecretsSettings.WithRetry(retrySettings); | ||
|
|
||
| // Create a new client with the custom configuration | ||
| var builder = new SecretManagerServiceClientBuilder | ||
| { | ||
| Settings = settings | ||
| }; | ||
|
|
||
| var client = await builder.BuildAsync(); | ||
|
|
||
| Console.WriteLine("Client created with custom retry settings."); | ||
| // End sample | ||
| } | ||
| public async Task CallSettingsRetries() | ||
| { | ||
| string projectId = "your-project-id"; | ||
|
|
||
| // Sample: CallSettingsRetries | ||
| // Required using directives: | ||
| // using Google.Cloud.SecretManager.V1; | ||
| // using Google.Api.Gax; | ||
| // using Google.Protobuf.WellKnownTypes; | ||
| // using Google.Api.Gax.ResourceNames; | ||
|
|
||
| // Define custom retry settings | ||
| var retrySettings = RetrySettings.FromExponentialBackoff( | ||
| maxAttempts: 3, | ||
| initialBackoff: TimeSpan.FromSeconds(0.5), | ||
| maxBackoff: TimeSpan.FromSeconds(5), | ||
| backoffMultiplier: 2.0, | ||
| retryFilter: RetrySettings.FilterForStatusCodes(StatusCode.Unavailable, StatusCode.DeadlineExceeded) | ||
| ); | ||
|
|
||
| // Create CallSettings | ||
| var callSettings = CallSettings.FromRetry(retrySettings); | ||
|
|
||
| // Prepare the request | ||
| ProjectName parent = ProjectName.FromProject(projectId); | ||
| ListSecretsRequest request = new ListSecretsRequest { ParentAsProjectName = parent }; | ||
|
|
||
| // Call the method, passing the CallSettings | ||
| var client = await SecretManagerServiceClient.CreateAsync(); | ||
| var response = client.ListSecrets(request, callSettings); | ||
|
|
||
| Console.WriteLine("RPC called with custom retry settings."); | ||
| // End sample | ||
| } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // Copyright 2025 Google Inc. All Rights Reserved. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| using Google.Cloud.SecretManager.V1; | ||
| using Google.Protobuf.WellKnownTypes; | ||
| using Grpc.Core; | ||
|
|
||
| namespace Google.Cloud.Tools.Snippets | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| public class UpdateMaskSnippets | ||
| { | ||
| public async Task UpdateMasks() | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| string projectId = "your-project-id"; | ||
| string secretId = "test-secret"; | ||
|
||
|
|
||
| // Sample: UpdateMasks | ||
| // Required using directives: | ||
| // using Google.Cloud.SecretManager.V1; | ||
| // using Google.Protobuf.WellKnownTypes; | ||
| // using Grpc.Core; | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| // Setup Client | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| SecretManagerServiceClient client = await SecretManagerServiceClient.CreateAsync(); | ||
|
|
||
| // Prepare the resource with NEW values | ||
| Secret secret = new Secret | ||
| { | ||
| // Set the full resource name | ||
| SecretName = SecretName.FromProjectSecret(projectId, secretId), | ||
|
|
||
bshaffer marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| // Populate ONLY the fields we intend to change | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Labels = { { "env", "production" } } | ||
| }; | ||
|
|
||
| // Create the FieldMask | ||
| FieldMask updateMask = new FieldMask(); | ||
|
|
||
| // Add the Protobuf field name in snake_case | ||
| // Note: The C# property is 'Labels', but the proto field name is 'labels'. | ||
| updateMask.Paths.Add("labels"); | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
|
|
||
| // Prepare the Request object | ||
| UpdateSecretRequest request = new UpdateSecretRequest | ||
| { | ||
| Secret = secret, | ||
| UpdateMask = updateMask | ||
| }; | ||
|
|
||
| // Call the API | ||
| try | ||
bshaffer marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| { | ||
| Secret updatedSecret = await client.UpdateSecretAsync(request); | ||
| Console.WriteLine($"Secret labels updated successfully. New Etag: {updatedSecret.Etag}"); | ||
| } | ||
| catch (RpcException ex) | ||
| { | ||
| Console.WriteLine($"Update failed: {ex.Status.Detail}"); | ||
| } | ||
|
|
||
| // End sample | ||
| } | ||
| } | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.