Skip to content

Conversation

@rudrakhsha-crest
Copy link
Contributor

@rudrakhsha-crest rudrakhsha-crest commented Mar 28, 2025

Description

Added samples for floor setting operations along with test cases.

Checklist

@rudrakhsha-crest rudrakhsha-crest requested review from a team as code owners March 28, 2025 16:01
@snippet-bot
Copy link

snippet-bot bot commented Mar 28, 2025

Here is the summary of changes.

You are about to add 6 region tags.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@product-auto-label product-auto-label bot added the samples Issues that are directly related to samples. label Mar 28, 2025
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @rudrakhsha-crest, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

This pull request introduces samples for floor setting operations in the Model Armor API, along with corresponding test cases. The changes include adding new files for creating, getting, and updating floor settings at the organization, folder, and project levels. Additionally, the pull request updates the CODEOWNERS file and blunderbuss.yml to include the model-armor API.

Highlights

  • Floor Setting Samples: Adds samples for getting and updating floor settings for organizations, folders, and projects.
  • Test Cases: Includes test cases to validate the functionality of the new floor setting samples.
  • Configuration Updates: Updates the CODEOWNERS file and blunderbuss.yml to include the model-armor API.

Changelog

Click here to see the changelog
  • .github/blunderbuss.yml
    • Adds modelarmor API to assign_issues_by and assign_prs_by sections, routing to the GoogleCloudPlatform/cloud-modelarmor-team.
  • CODEOWNERS
    • Adds model-armor codeowners, assigning responsibility to @GoogleCloudPlatform/nodejs-samples-reviewers, @GoogleCloudPlatform/cloud-samples-reviewers, and @GoogleCloudPlatform/cloud-modelarmor-team.
  • model-armor/package.json
    • Creates a package.json file for the model-armor samples, defining dependencies, scripts, and metadata.
  • model-armor/snippets/createTemplate.js
    • Adds a snippet for creating a Model Armor template with Responsible AI (RAI) filters.
  • model-armor/snippets/createTemplateWithAdvancedSdp.js
    • Adds a snippet for creating a Model Armor template with advanced SDP settings enabled.
  • model-armor/snippets/createTemplateWithBasicSdp.js
    • Adds a snippet for creating a Model Armor template with basic SDP settings enabled.
  • model-armor/snippets/createTemplateWithLabels.js
    • Adds a snippet for creating a Model Armor template with Responsible AI (RAI) filters and custom labels.
  • model-armor/snippets/createTemplateWithMetadata.js
    • Adds a snippet for creating a new model armor template with template metadata.
  • model-armor/snippets/deleteTemplate.js
    • Adds a snippet for deleting a Model Armor template.
  • model-armor/snippets/getFolderFloorSettings.js
    • Adds a snippet for retrieving the floor settings for a Google Cloud folder.
  • model-armor/snippets/getOrganizationFloorSettings.js
    • Adds a snippet for retrieving the floor settings for a Google Cloud organization.
  • model-armor/snippets/getProjectFloorSettings.js
    • Adds a snippet for retrieving the floor settings for a Google Cloud project.
  • model-armor/snippets/getTemplate.js
    • Adds a snippet for retrieving a Model Armor template by its ID.
  • model-armor/snippets/listTemplates.js
    • Adds a snippet for listing all Model Armor templates in a specified project and location.
  • model-armor/snippets/listTemplatesWithFilter.js
    • Adds a snippet for listing Model Armor templates that match a specific filter criteria.
  • model-armor/snippets/updateFolderFloorSettings.js
    • Adds a snippet for updating the floor settings of a folder in Model Armor.
  • model-armor/snippets/updateOrganizationFloorSettings.js
    • Adds a snippet for updating the floor settings of an organization in Model Armor.
  • model-armor/snippets/updateProjectFloorSettings.js
    • Adds a snippet for updating the floor settings of a project in Model Armor.
  • model-armor/snippets/updateTemplate.js
    • Adds a snippet for updating an existing model armor template.
  • model-armor/snippets/updateTemplateLabels.js
    • Adds a snippet for updating the labels of an existing model armor template.
  • model-armor/snippets/updateTemplateMetadata.js
    • Adds a snippet for updating the metadata of an existing model armor template.
  • model-armor/snippets/updateTemplateWithMaskConfiguration.js
    • Adds a snippet for updating an existing model armor template with a specific update mask.
  • model-armor/test/.eslintrc.yml
    • Adds an .eslintrc.yml file for the test directory, configuring the environment for Mocha tests.
  • model-armor/test/modelarmor.test.js
    • Adds a test suite for Model Armor, including tests for getting and updating floor settings at the organization, folder, and project levels.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.


In realms of code, where models reside,
Armor we forge, with filters applied.
Templates we craft, with labels so fine,
Protecting the output, making it shine.
Floor settings we tweak, with utmost care,
Ensuring the safety, beyond compare.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces samples for floor setting operations in Model Armor, along with corresponding test cases. The addition of these samples enhances the usability and understanding of the Model Armor API. The code appears well-structured and follows general best practices. However, there are a few areas that could benefit from improvement, particularly in terms of error handling and documentation clarity.

Summary of Findings

  • Missing Error Handling: The code lacks explicit error handling for the getFloorSetting method calls in the get floor settings samples. If the API call fails, the application might crash or behave unexpectedly. Adding error handling would make the samples more robust.
  • Inconsistent use of ModelArmorClient: In getFolderFloorSettings.js, the ModelArmorClient is imported from @google-cloud/modelarmor.v1, but in getOrganizationFloorSettings.js and getProjectFloorSettings.js, it's imported from @google-cloud/modelarmor. This inconsistency should be addressed for better maintainability.
  • Copyright Year: The copyright year is set to 2025 in all the new files. Please confirm that this is correct, or update it to the current year.

Merge Readiness

The pull request is almost ready for merging. However, the missing error handling in the get floor settings samples should be addressed before merging. I am unable to directly approve the pull request, and recommend that others review and approve this code before merging. I recommend that the pull request not be merged until the high severity issues are addressed.

@rudrakhsha-crest rudrakhsha-crest changed the title feat(modelarmor): Added floor settings samples for model armor feat(modelarmor): Added floor settings samples Apr 16, 2025
@rudrakhsha-crest rudrakhsha-crest force-pushed the model-armor-floor-settings branch from e4b81d5 to 73935d9 Compare April 17, 2025 11:07
@rudrakhsha-crest rudrakhsha-crest changed the title feat(modelarmor): Added floor settings samples feat(modelarmor): Added snippets for floor settings Apr 18, 2025
@rudrakhsha-crest rudrakhsha-crest force-pushed the model-armor-floor-settings branch from 73935d9 to 313bd64 Compare April 18, 2025 12:47
@harshnasitcrest harshnasitcrest requested a review from a team as a code owner May 19, 2025 15:05
@harshnasitcrest
Copy link
Contributor

Please set MA_FOLDER_ID and MA_ORG_ID to values from https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/model_armor/snippets/noxfile_config.py#L42-L43 before merging this PR.

@glasnt
Copy link
Contributor

glasnt commented May 30, 2025

Pending tests resolution, this can be merged

@harshnasitcrest
Copy link
Contributor

@glasnt Only floor setting tests are failing due to permission issues.

@utsav1810 - I found this service account [email protected] being used in the tests from job logs.. Could you please add it to the relevant folder and organization? Once done, we can try re-running the tests.

@utsav1810
Copy link

@harshnasitcrest I added permissions for [email protected]. We can try running the tests.

@utsav1810 utsav1810 added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels May 30, 2025
@harshnasitcrest
Copy link
Contributor

@harshnasitcrest I added permissions for [email protected]. We can try running the tests.

@utsav1810 The error persists. Does it take some time to propogate permissions?
Screenshot 2025-05-30 at 2 53 42 PM

@glasnt
Copy link
Contributor

glasnt commented Jun 2, 2025

It looks like the tests might have changed, and you're expecting the project to have locations/global floor settings to get and disable, but never create. Does something need to be created on the project to support this test?

@harshnasitcrest
Copy link
Contributor

It looks like the tests might have changed, and you're expecting the project to have locations/global floor settings to get and disable, but never create. Does something need to be created on the project to support this test?

@glasnt - No, floor setting does not need to be created, you can only set or view the settings.
Ref:

@glasnt
Copy link
Contributor

glasnt commented Jun 3, 2025

I'm not seeing the permissions being active in the project. I'm okay with adding modelarmor.floorSettingsViewer, but I'm not sure I want our testing changing global resources, so can we mock the update test somehow?

@harshnasitcrest
Copy link
Contributor

I'm not seeing the permissions being active in the project. I'm okay with adding modelarmor.floorSettingsViewer, but I'm not sure I want our testing changing global resources, so can we mock the update test somehow?

This project, folder and organization is specifically provisioned for code samples tests. We're okay with updating the floor settings on that. @utsav1810 Please confirm.

@utsav1810
Copy link

utsav1810 commented Jun 3, 2025

@harshnasitcrest The org and folder are dedicated for Model Armor - we can update global resources here, no problem.
For project, I am fine if we mock the test as per @glasnt 's suggestion.

@utsav1810
Copy link

Update - Had a quick chat with @glasnt offline. I will add the missing permissions on the project. The tests should work fine after that.
@harshnasitcrest No need to mock the tests. We will proceed with the same way we did for python tests.

@utsav1810
Copy link

@glasnt The cl for permission addition is submitted. Tests are still failing though. Can you check if the permissions are reflected in the project now?

@glasnt glasnt added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Jun 9, 2025
templatesToDelete.push(templateName);
it('should get project floor settings', () => {
const output = execSync(
`node snippets/getProjectFloorSettings.js ${projectId}`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests are failing because this invocation of the test doesn't return anything.

Example:

  3) Model Armor tests
       should get project floor settings:
     AssertionError: expected '' to match /projects\/long-door-651\/locations\/global\/floorSetting/

Other tests in this file use the require('../snippets/..'); format, based on samples that return values that can be verified for success. In the current format, the files don't return text, so can't be validated.

An example edit that would update these tests to at least a common format:

     assert.match(output, new RegExp(expectedName.replace(/\//g, '\\/')));
   });
 
-  it('should get project floor settings', () => {
-    const output = execSync(
-      `node snippets/getProjectFloorSettings.js ${projectId}`
-    ).toString();
+  it('should get project floor settings', async () => {
+      const getProjectFloorSettings = require('../snippets/getProjectFloorSettings');
+
+      const output = await getProjectFloorSettings(
+        projectId,
+      );
 
     // Check for expected name format in output
     const expectedName = `projects/${projectId}/locations/global/floorSetting`;
-    assert.match(output, new RegExp(expectedName.replace(/\//g, '\\/')));
+    assert.match(output.name, new RegExp(expectedName.replace(/\//g, '\\/')));
   });
 
   it('should update organization floor settings', () => {

Copy link
Contributor

@harshnasitcrest harshnasitcrest Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's an ongoing issue in Model Armor API involving floor settings. As per Eric's recommendation, keeping this PR on hold until then

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As per approval from Utsav (Model Armor team), we'll remove the floor settings tests along with TODO comment and proceed further.

Copy link
Contributor

@harshnasitcrest harshnasitcrest Jun 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, @utsav1810 Could you add kokoro labels to run the tests?

Tests are passing on local:
Screenshot 2025-06-11 at 4 23 43 PM

@utsav1810 utsav1810 added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels Jun 11, 2025
@utsav1810 utsav1810 merged commit 595f2e9 into GoogleCloudPlatform:main Jun 11, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. samples Issues that are directly related to samples.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants