Clarification Needed on Data Handling and Retention Policies

[aanshshah]

Clarification Needed on Data Handling and Retention Policies

Overview

When users and enterprises evaluate using the library with sensitive data and they need clarification on how data is handled during the optimization process, particularly regarding data retention and privacy compliance.

Questions/Concerns

1. Data Transmission: When using the optimizer, training data and prompts are sent to AWS Bedrock. Are there any logs or retained copies of this data?

2. Data Retention:

   • How long is training data retained in AWS services during optimization?
   • Are intermediate results from MIPROv2 optimization stored anywhere?
   • Is there a way to ensure complete data deletion after optimization?

3. Compliance Considerations:

   • For GDPR/HIPAA compliance, we need to ensure no PII is retained
   • Can you provide guidance on using the optimizer with regulated data?
   • Are there any data residency controls available?

4. Audit Trail: Is there any logging of what data was used for optimization that could potentially expose sensitive information?

Example Use Case

Optimize prompts for a healthcare chatbot that processes patient queries. What assurances are there around data handling?

Requested Information

• Documentation on data handling practices
• Best practices for using the optimizer with sensitive data
• Configuration options for enhanced privacy (if available)
• Confirmation of AWS Bedrock's data handling for optimization requests

Suggestions

1. Add a section in the README about data privacy and security
2. Provide configuration options to disable any logging/retention
3. Consider adding local optimization options that don't require cloud calls
4. Document which AWS services are involved in the data flow

Current Documentation Gap

The current README mentions AWS credentials and Bedrock access but doesn't address:

• Data retention policies
• Privacy considerations
• Compliance guidance
• Security best practices for sensitive data

This information would be valuable for enterprise adoption and security reviews.

[VikramShenoy97]

Thank you for your input! Please see the responses to your questions below

Data Transmission: When using the optimizer, training data and prompts are sent to AWS Bedrock. Are there any logs or retained copies of this data?

Response: Bedrock by default disables model invocation logging. Unless you enable it and store it in S3 or CloudWatch, data will not be stored in your account.

How long is training data retained in AWS services during optimization?

Response: Data is not retained unless the user enables Bedrock Model Invocation logging. If the user chooses to enable it and use S3 they can define their own retention settings for the model logs.

Are intermediate results from MIPROv2 optimization stored anywhere?

Response: No the intermediate results of MIPROv2 are not stored anywhere.

Is there a way to ensure complete data deletion after optimization?

Response: Data is not retained or stored. Everything is in memory.

Can you provide guidance on using the optimizer with regulated data?

Response: Please follow your companies best practices with data handling. Here's Amazon Bedrock's Data Protection Guide

Are there any data residency controls available?

Response: The users have complete control on where the data resides. The prompt optimizer simply reads the data in memory and does not store it on physical storage during optimization runs. Users can define their own data residency controls for the stored data.

Is there any logging of what data was used for optimization that could potentially expose sensitive information?

Response: Bedrock by default disables model invocation logging. Unless you enable it and store it in S3 or CloudWatch, data will not be stored in your account.

Document which AWS services are involved in the data flow
Response: The optimizer only uses Bedrock.

As suggested, we will be adding a section in our README linking to Bedrock's Model Invocation Logging. We will create a PR to have that in the README this week.

[amazeAmazing]

Thank you for your suggestion!

I made the README change: #29 and will close this issue.