Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 17 additions & 26 deletions sdk/ai/azure-ai-voicelive/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,25 @@
# Release History

## 1.2.0 (2025-12-10)
## 1.2.0b3 (Unreleased)

### Features Added

- **MCP (Model Context Protocol) Support**: Added comprehensive support for Model Context Protocol integration:
- `MCPServer` tool type for defining MCP server configurations with authorization, headers, and approval requirements
- `MCPTool` model for representing MCP tool definitions with input schemas and annotations
- `MCPApprovalType` enum for controlling approval workflows (`never`, `always`, or tool-specific)
- New item types: `MCPApprovalResponseRequestItem`, `ResponseMCPApprovalRequestItem`, `ResponseMCPApprovalResponseItem`, `ResponseMCPCallItem`, and `ResponseMCPListToolItem`
- New server events: `ServerEventMcpListToolsInProgress`, `ServerEventMcpListToolsCompleted`, `ServerEventMcpListToolsFailed`, `ServerEventResponseMcpCallArgumentsDelta`, `ServerEventResponseMcpCallArgumentsDone`, `ServerEventResponseMcpCallInProgress`, `ServerEventResponseMcpCallCompleted`, and `ServerEventResponseMcpCallFailed`
- Client event `MCP_APPROVAL_RESPONSE` for responding to approval requests
- Enhanced `ItemType` enum with MCP-related types: `mcp_list_tools`, `mcp_call`, `mcp_approval_request`, and `mcp_approval_response`
- **Enhanced Avatar Configuration**: Expanded avatar functionality with new configuration options:
- Added `AvatarConfigTypes` enum with support for `video-avatar` and `photo-avatar` types
- Added `PhotoAvatarBaseModes` enum for photo avatar base models (e.g., `vasa-1`)
- Added `AvatarOutputProtocol` enum for avatar streaming protocols (`webrtc`, `websocket`)
- Enhanced `AvatarConfig` model with new properties: `type`, `model`, and `output_protocol`
- **Image Content Support**: Added support for image inputs in conversations:
- New `RequestImageContentPart` model for including images in requests
- New `RequestImageContentPartDetail` enum for controlling image detail levels (`auto`, `low`, `high`)
- Added `INPUT_IMAGE` to `ContentPartType` enum
- Enhanced token details models (`InputTokenDetails`, `CachedTokenDetails`) with `image_tokens` tracking
- **Enhanced OpenAI Voices**: Added new OpenAI voice options:
- Added `marin` and `cedar` voices to `OpenAIVoiceName` enum
- **Extended Azure Personal Voice Configuration**: Enhanced `AzurePersonalVoice` with additional customization options:
- Added support for custom lexicon via `custom_lexicon_url`
- Added `prefer_locales` for locale preferences
- Added `locale`, `style`, `pitch`, `rate`, and `volume` properties for fine-tuned voice control
- **Pre-generated Assistant Messages**: Added support for pre-generated assistant messages in `ResponseCreateParams` via the `pre_generated_assistant_message` property
- **Manual Turn Taking Control**: Added `NoTurnDetection` class to allow manual control of turn taking
- `TurnDetectionType.NONE` enum value for explicitly disabling automatic turn detection
- Enables client-managed conversation flow without server-side voice activity detection
- **Array Encoding Support**: Enhanced model serialization to support delimited string arrays
- Support for pipe-delimited, space-delimited, comma-delimited, and newline-delimited array formats
- Automatic encoding/decoding of string arrays to delimited strings in model properties

### Other Changes

- **Security Enhancement**: Replaced `eval()` usage in serialization code with explicit type checking
- **Dependency Update**: Updated `azure-core` minimum version requirement from `1.35.0` to `1.36.0`
- **Development Status**: Reverted package development status to Beta (was incorrectly set to Production/Stable)

### Bug Fixes

- Fixed deserialization handling for mutable types (dict, list, set) to properly track changes in model objects

## 1.2.0b2 (2025-11-20)

Expand Down
1 change: 1 addition & 0 deletions sdk/ai/azure-ai-voicelive/apiview-properties.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@
"azure.ai.voicelive.models.MCPApprovalResponseRequestItem": "VoiceLive.MCPApprovalResponseRequestItem",
"azure.ai.voicelive.models.MCPServer": "VoiceLive.MCPServer",
"azure.ai.voicelive.models.MCPTool": "VoiceLive.MCPTool",
"azure.ai.voicelive.models.NoTurnDetection": "VoiceLive.NoTurnDetection",
"azure.ai.voicelive.models.OpenAIVoice": "VoiceLive.OpenAIVoice",
"azure.ai.voicelive.models.OutputTextContentPart": "VoiceLive.OutputTextContentPart",
"azure.ai.voicelive.models.OutputTokenDetails": "VoiceLive.OutputTokenDetails",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,21 @@ def default(self, o): # pylint: disable=too-many-return-statements
r"(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)\s\d{4}\s\d{2}:\d{2}:\d{2}\sGMT"
)

_ARRAY_ENCODE_MAPPING = {
"pipeDelimited": "|",
"spaceDelimited": " ",
"commaDelimited": ",",
"newlineDelimited": "\n",
}


def _deserialize_array_encoded(delimit: str, attr):
if isinstance(attr, str):
if attr == "":
return []
return attr.split(delimit)
return attr

Comment on lines +174 to +188
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new array encoding/decoding functionality (lines 174-188, 330-334, 532-534, 819-829) lacks test coverage. Since the repository has comprehensive test coverage for serialization (test_unit_serialization.py), tests should be added for the new delimited array formats (pipe, space, comma, newline) to ensure proper encoding and decoding.

Copilot uses AI. Check for mistakes.

def _deserialize_datetime(attr: typing.Union[str, datetime]) -> datetime:
"""Deserialize ISO-8601 formatted string into Datetime object.
Expand Down Expand Up @@ -315,6 +330,8 @@ def _deserialize_int_as_str(attr):
def get_deserializer(annotation: typing.Any, rf: typing.Optional["_RestField"] = None):
if annotation is int and rf and rf._format == "str":
return _deserialize_int_as_str
if annotation is str and rf and rf._format in _ARRAY_ENCODE_MAPPING:
return functools.partial(_deserialize_array_encoded, _ARRAY_ENCODE_MAPPING[rf._format])
if rf and rf._format:
return _DESERIALIZE_MAPPING_WITHFORMAT.get(rf._format)
return _DESERIALIZE_MAPPING.get(annotation) # pyright: ignore
Expand Down Expand Up @@ -353,9 +370,39 @@ def __contains__(self, key: typing.Any) -> bool:
return key in self._data

def __getitem__(self, key: str) -> typing.Any:
# If this key has been deserialized (for mutable types), we need to handle serialization
if hasattr(self, "_attr_to_rest_field"):
cache_attr = f"_deserialized_{key}"
if hasattr(self, cache_attr):
rf = _get_rest_field(getattr(self, "_attr_to_rest_field"), key)
if rf:
value = self._data.get(key)
if isinstance(value, (dict, list, set)):
# For mutable types, serialize and return
# But also update _data with serialized form and clear flag
# so mutations via this returned value affect _data
serialized = _serialize(value, rf._format)
# If serialized form is same type (no transformation needed),
# return _data directly so mutations work
if isinstance(serialized, type(value)) and serialized == value:
return self._data.get(key)
# Otherwise return serialized copy and clear flag
try:
object.__delattr__(self, cache_attr)
except AttributeError:
pass
# Store serialized form back
self._data[key] = serialized
return serialized
return self._data.__getitem__(key)

def __setitem__(self, key: str, value: typing.Any) -> None:
# Clear any cached deserialized value when setting through dictionary access
cache_attr = f"_deserialized_{key}"
try:
object.__delattr__(self, cache_attr)
except AttributeError:
pass
self._data.__setitem__(key, value)
Comment on lines 372 to 406
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new mutable type deserialization caching mechanism (lines 372-406, 1073-1106) lacks test coverage. Since the repository has comprehensive test coverage for deserialization functionality, tests should be added to verify that mutations to deserialized mutable types (dict, list, set) are properly tracked and reflected in the model's _data.

Copilot uses AI. Check for mistakes.

def __delitem__(self, key: str) -> None:
Expand Down Expand Up @@ -483,6 +530,8 @@ def _is_model(obj: typing.Any) -> bool:

def _serialize(o, format: typing.Optional[str] = None): # pylint: disable=too-many-return-statements
if isinstance(o, list):
if format in _ARRAY_ENCODE_MAPPING and all(isinstance(x, str) for x in o):
return _ARRAY_ENCODE_MAPPING[format].join(o)
return [_serialize(x, format) for x in o]
if isinstance(o, dict):
return {k: _serialize(v, format) for k, v in o.items()}
Expand Down Expand Up @@ -767,6 +816,17 @@ def _deserialize_sequence(
return obj
if isinstance(obj, ET.Element):
obj = list(obj)
try:
if (
isinstance(obj, str)
and isinstance(deserializer, functools.partial)
and isinstance(deserializer.args[0], functools.partial)
and deserializer.args[0].func == _deserialize_array_encoded # pylint: disable=comparison-with-callable
):
# encoded string may be deserialized to sequence
return deserializer(obj)
except: # pylint: disable=bare-except
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare except clause catches all exceptions without discrimination, which can hide unexpected errors and make debugging difficult. Replace with specific exception types (e.g., AttributeError, TypeError) that might be raised when checking the deserializer structure.

Suggested change
except: # pylint: disable=bare-except
except (AttributeError, TypeError, IndexError):

Copilot uses AI. Check for mistakes.
pass
return type(obj)(_deserialize(deserializer, entry, module) for entry in obj)


Expand Down Expand Up @@ -998,7 +1058,11 @@ def __init__(

@property
def _class_type(self) -> typing.Any:
return getattr(self._type, "args", [None])[0]
result = getattr(self._type, "args", [None])[0]
# type may be wrapped by nested functools.partial so we need to check for that
if isinstance(result, functools.partial):
return getattr(result, "args", [None])[0]
return result

@property
def _rest_name(self) -> str:
Expand All @@ -1009,14 +1073,37 @@ def _rest_name(self) -> str:
def __get__(self, obj: Model, type=None): # pylint: disable=redefined-builtin
# by this point, type and rest_name will have a value bc we default
# them in __new__ of the Model class
item = obj.get(self._rest_name)
# Use _data.get() directly to avoid triggering __getitem__ which clears the cache
item = obj._data.get(self._rest_name)
if item is None:
return item
if self._is_model:
return item
return _deserialize(self._type, _serialize(item, self._format), rf=self)

# For mutable types, we want mutations to directly affect _data
# Check if we've already deserialized this value
cache_attr = f"_deserialized_{self._rest_name}"
if hasattr(obj, cache_attr):
# Return the value from _data directly (it's been deserialized in place)
return obj._data.get(self._rest_name)

deserialized = _deserialize(self._type, _serialize(item, self._format), rf=self)

# For mutable types, store the deserialized value back in _data
# so mutations directly affect _data
if isinstance(deserialized, (dict, list, set)):
obj._data[self._rest_name] = deserialized
object.__setattr__(obj, cache_attr, True) # Mark as deserialized
return deserialized

return deserialized

def __set__(self, obj: Model, value) -> None:
# Clear the cached deserialized object when setting a new value
cache_attr = f"_deserialized_{self._rest_name}"
if hasattr(obj, cache_attr):
object.__delattr__(obj, cache_attr)

if value is None:
# we want to wipe out entries if users set attr to None
try:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -821,13 +821,20 @@ def serialize_basic(cls, data, data_type, **kwargs):
:param str data_type: Type of object in the iterable.
:rtype: str, int, float, bool
:return: serialized object
:raises TypeError: raise if data_type is not one of str, int, float, bool.
"""
custom_serializer = cls._get_custom_serializers(data_type, **kwargs)
if custom_serializer:
return custom_serializer(data)
if data_type == "str":
return cls.serialize_unicode(data)
return eval(data_type)(data) # nosec # pylint: disable=eval-used
if data_type == "int":
return int(data)
if data_type == "float":
return float(data)
if data_type == "bool":
return bool(data)
raise TypeError("Unknown basic data type: {}".format(data_type))
Comment on lines +831 to +837
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The replacement of eval() with explicit type checking (lines 831-837) lacks test coverage for the new error path. Since the repository has comprehensive test coverage for serialization, add tests to verify that calling serialize_basic with unsupported data types (not str, int, float, bool) raises a TypeError with an appropriate error message.

Copilot uses AI. Check for mistakes.

@classmethod
def serialize_unicode(cls, data):
Expand Down Expand Up @@ -1757,7 +1764,7 @@ def deserialize_basic(self, attr, data_type): # pylint: disable=too-many-return
:param str data_type: deserialization data type.
:return: Deserialized basic type.
:rtype: str, int, float or bool
:raises TypeError: if string format is not valid.
:raises TypeError: if string format is not valid or data_type is not one of str, int, float, bool.
"""
# If we're here, data is supposed to be a basic type.
# If it's still an XML node, take the text
Expand All @@ -1783,7 +1790,11 @@ def deserialize_basic(self, attr, data_type): # pylint: disable=too-many-return

if data_type == "str":
return self.deserialize_unicode(attr)
return eval(data_type)(attr) # nosec # pylint: disable=eval-used
if data_type == "int":
return int(attr)
if data_type == "float":
return float(attr)
raise TypeError("Unknown basic data type: {}".format(data_type))
Comment on lines +1793 to +1797
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The replacement of eval() with explicit type checking (lines 1793-1797) lacks test coverage for the new error path. Since the repository has comprehensive test coverage for deserialization, add tests to verify that calling deserialize_basic with unsupported data types (not str, int, float, bool) raises a TypeError with an appropriate error message.

Copilot uses AI. Check for mistakes.

@staticmethod
def deserialize_unicode(data):
Expand Down
2 changes: 1 addition & 1 deletion sdk/ai/azure-ai-voicelive/azure/ai/voicelive/_version.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@
# Changes may cause incorrect behavior and will be lost if the code is regenerated.
# --------------------------------------------------------------------------

VERSION = "1.2.0"
VERSION = "1.2.0b3"
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
MCPTool,
MessageContentPart,
MessageItem,
NoTurnDetection,
OpenAIVoice,
OutputTextContentPart,
OutputTokenDetails,
Expand Down Expand Up @@ -238,6 +239,7 @@
"MCPTool",
"MessageContentPart",
"MessageItem",
"NoTurnDetection",
"OpenAIVoice",
"OutputTextContentPart",
"OutputTokenDetails",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,7 @@ class ToolType(str, Enum, metaclass=CaseInsensitiveEnumMeta):
class TurnDetectionType(str, Enum, metaclass=CaseInsensitiveEnumMeta):
"""Type of TurnDetectionType."""

NONE = "none"
SERVER_VAD = "server_vad"
AZURE_SEMANTIC_VAD = "azure_semantic_vad"
AZURE_SEMANTIC_VAD_EN = "azure_semantic_vad_en"
Expand Down
43 changes: 31 additions & 12 deletions sdk/ai/azure-ai-voicelive/azure/ai/voicelive/models/_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -777,17 +777,17 @@ class TurnDetection(_Model):
"""Top-level union for turn detection configuration.

You probably want to use the sub-classes and not this class directly. Known sub-classes are:
AzureSemanticVad, AzureSemanticVadEn, AzureSemanticVadMultilingual, ServerVad
AzureSemanticVad, AzureSemanticVadEn, AzureSemanticVadMultilingual, NoTurnDetection, ServerVad

:ivar type: Required. Known values are: "server_vad", "azure_semantic_vad",
:ivar type: Required. Known values are: "none", "server_vad", "azure_semantic_vad",
"azure_semantic_vad_en", and "azure_semantic_vad_multilingual".
:vartype type: str or ~azure.ai.voicelive.models.TurnDetectionType
"""

__mapping__: dict[str, _Model] = {}
type: str = rest_discriminator(name="type", visibility=["read", "create", "update", "delete", "query"])
"""Required. Known values are: \"server_vad\", \"azure_semantic_vad\", \"azure_semantic_vad_en\",
and \"azure_semantic_vad_multilingual\"."""
"""Required. Known values are: \"none\", \"server_vad\", \"azure_semantic_vad\",
\"azure_semantic_vad_en\", and \"azure_semantic_vad_multilingual\"."""

@overload
def __init__(
Expand Down Expand Up @@ -2489,6 +2489,33 @@ def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)


class NoTurnDetection(TurnDetection, discriminator="none"):
"""No turn detection - client manages turn taking.

:ivar type: Required.
:vartype type: str or ~azure.ai.voicelive.models.NONE
Copy link

Copilot AI Jan 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation reference uses "~azure.ai.voicelive.models.NONE" which appears to be incorrect. It should reference the TurnDetectionType.NONE enum value properly, similar to how other models reference their types (e.g., using "str or ~azure.ai.voicelive.models.TurnDetectionType").

Suggested change
:vartype type: str or ~azure.ai.voicelive.models.NONE
:vartype type: str or ~azure.ai.voicelive.models.TurnDetectionType

Copilot uses AI. Check for mistakes.
"""

type: Literal[TurnDetectionType.NONE] = rest_discriminator(name="type", visibility=["read", "create", "update", "delete", "query"]) # type: ignore
"""Required."""

@overload
def __init__(
self,
) -> None: ...

@overload
def __init__(self, mapping: Mapping[str, Any]) -> None:
"""
:param mapping: raw JSON to initialize the model.
:type mapping: Mapping[str, Any]
"""

def __init__(self, *args: Any, **kwargs: Any) -> None:
super().__init__(*args, **kwargs)
self.type = TurnDetectionType.NONE # type: ignore


class OpenAIVoice(_Model):
"""OpenAI voice configuration with explicit type field.

Expand Down Expand Up @@ -2681,8 +2708,6 @@ class RequestSession(_Model):
:vartype instructions: str
:ivar input_audio_sampling_rate: Input audio sampling rate in Hz. Available values:



* For pcm16: 8000, 16000, 24000

* For g711_alaw/g711_ulaw: 8000.
Expand Down Expand Up @@ -2737,8 +2762,6 @@ class RequestSession(_Model):
input_audio_sampling_rate: Optional[int] = rest_field(visibility=["read", "create", "update", "delete", "query"])
"""Input audio sampling rate in Hz. Available values:



* For pcm16: 8000, 16000, 24000

* For g711_alaw/g711_ulaw: 8000."""
Expand Down Expand Up @@ -3733,8 +3756,6 @@ class ResponseSession(_Model):
:vartype instructions: str
:ivar input_audio_sampling_rate: Input audio sampling rate in Hz. Available values:



* For pcm16: 8000, 16000, 24000

* For g711_alaw/g711_ulaw: 8000.
Expand Down Expand Up @@ -3793,8 +3814,6 @@ class ResponseSession(_Model):
input_audio_sampling_rate: Optional[int] = rest_field(visibility=["read", "create", "update", "delete", "query"])
"""Input audio sampling rate in Hz. Available values:



* For pcm16: 8000, 16000, 24000

* For g711_alaw/g711_ulaw: 8000."""
Expand Down
4 changes: 2 additions & 2 deletions sdk/ai/azure-ai-voicelive/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ authors = [
description = "Microsoft Corporation Azure Ai Voicelive Client Library for Python"
license = "MIT"
classifiers = [
"Development Status :: 5 - Production/Stable",
"Development Status :: 4 - Beta",
"Programming Language :: Python",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3",
Expand All @@ -32,7 +32,7 @@ keywords = ["azure", "azure sdk"]

dependencies = [
"isodate>=0.6.1",
"azure-core>=1.35.0",
"azure-core>=1.36.0",
"typing-extensions>=4.6.0",
]
dynamic = [
Expand Down
Loading
Loading