Skip to content

Commit 376c99f

Browse files
amber-cccamber-ccc
authored
2024-11-15-preview and 2024-11-01 updates for Conversations Runtime (Azure#47814)
* Updated the dotnet sdk based on the new commit * - adjust test examples based on the new generated sdk * updated samples and snippets * updated code based on new commit id * modified the ChangeLog * remove breaking changes and Bugs Fixed sections * update sdk based on the latest commit id * updated class name based on new commit id * updated the snippets * add one more version 2024-11-01 in TestBase and add test for ConversationPiiWithCharacterMask * -adjust the version to 2024-11-01 in the test * Modified the assertion in conversation pii test with MaskPolicyType * add async test for ConversationPii_WithCharacterMaskPolicy * add sync and async test for ConversationPii_WithEntityMaskPolicy * add sync and async test for ConversationPii_WithNoMaskPolicy * revert ConversationsClientLiveTests * modify new test data for previous tests * revert previous tests to main version. * testing * test for authentication * - update the assets.json * update recording for some old version apis * update assets.json * Succesfully run all the local tests and update the assets.json * adding new tests to clientLiveTests and update assets.json * remove the duplicate snippets * update some remaining test recordings to 2024-11-15-preview * udpate the new test for MaskWithEntity * Update the snippets for new feature tests * update the snippets for the general README file. * updated the change log
1 parent 7efce99 commit 376c99f

File tree

38 files changed

+4879
-278
lines changed

38 files changed

+4879
-278
lines changed

‎sdk/cognitivelanguage/Azure.AI.Language.Conversations/CHANGELOG.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,19 @@
11
# Release History
22

3-
## 2.0.0-beta.2 (Unreleased)
3+
## 2.0.0-beta.2 (2025-02-03)
44

55
### Features Added
66

7-
- Exposed `JsonModelWriteCore` for model serialization procedure.
8-
9-
### Breaking Changes
10-
11-
### Bugs Fixed
7+
- Added three differenct type of Redaction Policy `CharacterMaskPolicyType`, `EntityMaskTypePolicyType` and `NoMaskPolicyType` for the function `AnalyzeConversations`
8+
- Added support for analyze-conversation API Versions
9+
- 2024-11-01
10+
- 2024-11-15-preview
1211

1312
### Other Changes
1413

14+
- Changed property `CreditCardNumberValue` to `CreditCardValue` and `PhoneNumberValue` to `PhoneValue` for class `ConversationPiiCategoryExclusions`
15+
- Added a new `Instruction` property to class `ConversationSummarizationActionContent`
16+
1517
## 2.0.0-beta.1 (2024-08-01)
1618

1719
- Added support for service version 2024-05-01.

‎sdk/cognitivelanguage/Azure.AI.Language.Conversations/README.md‎

Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -482,6 +482,225 @@ foreach (AnalyzeConversationOperationResult operationResult in operationState.Ac
482482
}
483483
```
484484

485+
### Extract PII from a conversation With Character Mask Policy
486+
487+
To detect and redact PII in a conversation using Character Mask Policy, you can use the `AnalyzeConversationsAsync` method overload with an action of type `PiiOperationAction` and utilize `CharacterMaskPolicyType` to define the RedactionCharacter (e.g., an asterisk *) to replace sensitive information. The method returns a `Response<AnalyzeConversationOperationState>`::
488+
489+
```C# Snippet:AnalyzeConversation_ConversationPiiWithCharacterMaskPolicy
490+
var redactionPolicy = new CharacterMaskPolicyType
491+
{
492+
RedactionCharacter = RedactionCharacter.Asterisk
493+
};
494+
495+
// Simulate input conversation
496+
MultiLanguageConversationInput input = new MultiLanguageConversationInput(
497+
new List<ConversationInput>
498+
{
499+
new TextConversation("1", "en", new List<TextConversationItem>
500+
{
501+
new TextConversationItem(id: "1", participantId: "Agent_1", text: "Can you provide your name?"),
502+
new TextConversationItem(id: "2", participantId: "Customer_1", text: "Hi, my name is John Doe."),
503+
new TextConversationItem(id: "3", participantId: "Agent_1", text: "Thank you John, that has been updated in our system.")
504+
})
505+
});
506+
507+
// Add action with CharacterMaskPolicyType
508+
List<AnalyzeConversationOperationAction> actions = new List<AnalyzeConversationOperationAction>
509+
{
510+
new PiiOperationAction
511+
{
512+
ActionContent = new ConversationPiiActionContent
513+
{
514+
RedactionPolicy = redactionPolicy
515+
},
516+
Name = "Conversation PII with Character Mask Policy"
517+
}
518+
};
519+
520+
// Create input for analysis
521+
AnalyzeConversationOperationInput data = new AnalyzeConversationOperationInput(input, actions);
522+
523+
// Act: Perform the PII analysis
524+
Response<AnalyzeConversationOperationState> analyzeConversationOperation = await client.AnalyzeConversationsAsync(data);
525+
AnalyzeConversationOperationState operationState = analyzeConversationOperation.Value;
526+
// Assert: Validate the results
527+
foreach (AnalyzeConversationOperationResult operationResult in operationState.Actions.Items)
528+
{
529+
Console.WriteLine($"Operation action name: {operationResult.Name}");
530+
531+
if (operationResult is ConversationPiiOperationResult piiOperationResult)
532+
{
533+
foreach (ConversationalPiiResult conversation in piiOperationResult.Results.Conversations)
534+
{
535+
Console.WriteLine($"Conversation: #{conversation.Id}");
536+
foreach (ConversationPiiItemResult item in conversation.ConversationItems)
537+
{
538+
string redactedText = item.RedactedContent?.Text ?? string.Empty;
539+
Console.WriteLine($"Redacted Text: {redactedText}");
540+
541+
// Only verify redaction if the original sentence had PII
542+
if (item.Entities.Any())
543+
{
544+
foreach (var entity in item.Entities)
545+
{
546+
Assert.That(redactedText, Does.Not.Contain(entity.Text),
547+
$"Expected entity '{entity.Text}' to be redacted but found in: {redactedText}");
548+
549+
Assert.That(redactedText, Does.Contain("*"),
550+
$"Expected redacted text to contain '*' but got: {redactedText}");
551+
}
552+
}
553+
}
554+
}
555+
}
556+
}
557+
```
558+
559+
### Extract PII from a conversation With Entity Mask Policy
560+
561+
To detect and redact PII in a conversation using Character Mask Policy, you can use the `AnalyzeConversationsAsync` method overload with an action of type `PiiOperationAction` and utilize `EntityMaskTypePolicyType` as Mask Policy to replace sensitive information. The method returns a `Response<AnalyzeConversationOperationState>`::
562+
563+
```C# Snippet:AnalyzeConversation_ConversationPiiWithEntityMaskPolicy
564+
var redactionPolicy = new EntityMaskTypePolicyType();
565+
566+
MultiLanguageConversationInput input = new MultiLanguageConversationInput(
567+
new List<ConversationInput>
568+
{
569+
new TextConversation("1", "en", new List<TextConversationItem>
570+
{
571+
new TextConversationItem(id: "1", participantId: "Agent_1", text: "Can you provide your name?"),
572+
new TextConversationItem(id: "2", participantId: "Customer_1", text: "Hi, my name is John Doe."),
573+
new TextConversationItem(id: "3", participantId: "Agent_1", text: "Thank you John, that has been updated in our system.")
574+
})
575+
});
576+
577+
// Add action with EntityMaskTypePolicyType
578+
List<AnalyzeConversationOperationAction> actions = new List<AnalyzeConversationOperationAction>
579+
{
580+
new PiiOperationAction
581+
{
582+
ActionContent = new ConversationPiiActionContent
583+
{
584+
RedactionPolicy = redactionPolicy
585+
},
586+
Name = "Conversation PII with Entity Mask Policy"
587+
}
588+
};
589+
590+
// Create input for analysis
591+
AnalyzeConversationOperationInput data = new AnalyzeConversationOperationInput(input, actions);
592+
593+
// Act: Perform the PII analysis
594+
Response<AnalyzeConversationOperationState> analyzeConversationOperation = await client.AnalyzeConversationsAsync(data);
595+
AnalyzeConversationOperationState operationState = analyzeConversationOperation.Value;
596+
597+
// Assert: Validate the results
598+
foreach (AnalyzeConversationOperationResult operationResult in operationState.Actions.Items)
599+
{
600+
Console.WriteLine($"Operation action name: {operationResult.Name}");
601+
602+
if (operationResult is ConversationPiiOperationResult piiOperationResult)
603+
{
604+
foreach (ConversationalPiiResult conversation in piiOperationResult.Results.Conversations)
605+
{
606+
Console.WriteLine($"Conversation: #{conversation.Id}");
607+
foreach (ConversationPiiItemResult item in conversation.ConversationItems)
608+
{
609+
string redactedText = item.RedactedContent?.Text ?? string.Empty;
610+
Console.WriteLine($"Redacted Text: {redactedText}");
611+
612+
// Only verify redaction if the original sentence had PII
613+
if (item.Entities.Any())
614+
{
615+
foreach (var entity in item.Entities)
616+
{
617+
Assert.That(redactedText, Does.Not.Contain(entity.Text),
618+
$"Expected entity '{entity.Text}' to be redacted but found in: {redactedText}");
619+
620+
// Case-insensitive pattern to match entity mask variations
621+
string expectedMaskPattern = $@"\[{entity.Category}-?\d*\]";
622+
623+
// Perform case-insensitive regex match
624+
StringAssert.IsMatch("(?i)" + expectedMaskPattern, redactedText,
625+
$"Expected redacted text to contain an entity mask similar to '[{entity.Category}]' but got: {redactedText}");
626+
}
627+
}
628+
}
629+
}
630+
}
631+
}
632+
```
633+
634+
### Extract PII from a conversation With Character Mask Policy
635+
636+
To detect and redact PII in a conversation using No Mask Policy, you can use the `AnalyzeConversationsAsync` method overload with an action of type `PiiOperationAction` and utilize `NoMaskPolicyType` as Mask Policy. The method returns a `Response<AnalyzeConversationOperationState>`::
637+
638+
```C# Snippet:AnalyzeConversation_ConversationPiiWithNoMaskPolicy
639+
var redactionPolicy = new NoMaskPolicyType();
640+
641+
// Simulate input conversation
642+
MultiLanguageConversationInput input = new MultiLanguageConversationInput(
643+
new List<ConversationInput>
644+
{
645+
new TextConversation("1", "en", new List<TextConversationItem>
646+
{
647+
new TextConversationItem(id: "1", participantId: "Agent_1", text: "Can you provide your name?"),
648+
new TextConversationItem(id: "2", participantId: "Customer_1", text: "Hi, my name is John Doe."),
649+
new TextConversationItem(id: "3", participantId: "Agent_1", text: "Thank you John, that has been updated in our system.")
650+
})
651+
});
652+
653+
// Add action with NoMaskPolicyType
654+
List<AnalyzeConversationOperationAction> actions = new List<AnalyzeConversationOperationAction>
655+
{
656+
new PiiOperationAction
657+
{
658+
ActionContent = new ConversationPiiActionContent
659+
{
660+
RedactionPolicy = redactionPolicy
661+
},
662+
Name = "Conversation PII with No Mask Policy"
663+
}
664+
};
665+
666+
// Create input for analysis
667+
AnalyzeConversationOperationInput data = new AnalyzeConversationOperationInput(input, actions);
668+
669+
// Act: Perform the PII analysis
670+
Response<AnalyzeConversationOperationState> analyzeConversationOperation = await client.AnalyzeConversationsAsync(data);
671+
AnalyzeConversationOperationState operationState = analyzeConversationOperation.Value;
672+
673+
// Assert: Validate the results
674+
foreach (AnalyzeConversationOperationResult operationResult in operationState.Actions.Items)
675+
{
676+
Console.WriteLine($"Operation action name: {operationResult.Name}");
677+
678+
if (operationResult is ConversationPiiOperationResult piiOperationResult)
679+
{
680+
foreach (ConversationalPiiResult conversation in piiOperationResult.Results.Conversations)
681+
{
682+
Console.WriteLine($"Conversation: #{conversation.Id}");
683+
foreach (ConversationPiiItemResult item in conversation.ConversationItems)
684+
{
685+
string originalText = item.RedactedContent?.Text ?? string.Empty;
686+
Console.WriteLine($"Original Text: {originalText}");
687+
688+
// Ensure PII is detected
689+
if (item.Entities.Any())
690+
{
691+
foreach (var entity in item.Entities)
692+
{
693+
detectedEntities.Add(entity.Text);
694+
Assert.That(originalText, Does.Contain(entity.Text),
695+
$"Expected entity '{entity.Text}' to be present but was not found in: {originalText}");
696+
}
697+
}
698+
}
699+
}
700+
}
701+
}
702+
```
703+
485704
### Additional samples
486705

487706
Browse our [samples][conversationanalysis_samples] for more examples of how to analyze conversations.

0 commit comments

Comments
 (0)