Recap for October 2nd, 2024 - Semantic Kernel Office Hours (US/EMEA)

[sophialagerkranspandey]

Semantic Kernel Office Hours for US/EMEA - October 2nd, 2024 recording: https://www.youtube.com/watch?v=v4rJG6cBRYA&list=PLmsFUfdnGr3xAtM3ofjpEkIWnuSA2d-i-&index=34

If you weren't able to attend office hours but still have questions, please post them here: https://github.com/microsoft/semantic-kernel/discussions

Meeting notes:

• Office Hours Overview: Sophia announced the weekly office hour sessions for US and EMEA customers every Wednesday from 8:00 to 9:00 AM Pacific Time, and monthly sessions for APAC customers on the second Wednesday of each month from 4:00 to 5:00 PM Pacific Time.
• New Releases and Updates: Sophia highlighted the new releases of 1.21.1 and 1.10.1, which include bug fixes, updates for unit testing for the kernel, Jason schema builder, and implementing generic data model structure for Azure Cosmos DB. She also mentioned the new release of 1.10.1 in Python.
• Documentation Restructuring: Sophia shared that the team has been restructuring the documentation based on community feedback, introducing four different frameworks, and updating content. She appreciated the feedback and mentioned that more updates will be coming in the following weeks.
• Ongoing Updates: Sophia noted that the team is still working on updates and that more changes will be rolled out in the coming weeks.
• New Blogs and Downloads: Sophia announced new blogs from engineers on structured outputs available in Python and .NET, and the milestone of semantic kernel Python reaching 1,000,000 downloads. She also mentioned the release of Java 1.3.0 and content in various languages.
• Structured Outputs in .NET: Dmytro presented a blog post on using structured outputs in .NET with open AI models, explaining different approaches and limitations. He encouraged the community to check the blog post for more details.
• Blog Post Overview: Dmytro presented a blog post released the previous day, detailing how to use structured outputs in .NET with open AI models.
• Approaches: Dmytro explained different approaches for using structured outputs, including constructing JSON schema manually and using response models directly.
• Limitations: Dmytro discussed limitations, such as the lack of support for certain JSON schema features in open AI structured outputs, and provided workarounds.
• Function Calling: Dmytro described the use of structured outputs with function calling, providing an example of categorizing emails based on their content.
• Structured Outputs in Python: Evan Mattson discussed the blog post on structured outputs in Python, highlighting the use of pedantic models and the ability to pass in a non-pedantic model. He provided an example of asking a math question and encouraged the community to try it out and provide feedback.
• Blog Post Overview: Evan Mattson discussed a blog post on structured outputs in Python, highlighting the use of Pydantic models for response formats.
• Pydantic Models: Evan explained that the open AI SDK supports setting up Pydantic models as the response format, providing validation for the JSON schema.
• Example: Evan provided an example of asking a math question and receiving a structured response with detailed steps and the final answer.
• Feedback Request: Evan encouraged the community to try out the examples and provide feedback through discussions or issues.
• Open AI and Azure AI Updates: Roger shared that Azure AI and Open AI released their stable packages, removing many preview features. The team is currently discussing strategies for versioning the connectors to minimize customer impact.
• Open Q&A Session: During the open Q&A session, participants asked questions about tracking tokens during streaming, debugging agent calls, and token compression. The team provided insights and offered to follow up offline for further assistance.
• Token Tracking: Jeevan asked about tracking tokens during streaming, particularly for images. Roger explained that the current approach is to use the image's pixel size for estimation and suggested following up on GitHub for more details.
• Debugging Agent Calls: Garry inquired about debugging agent calls and tracking prompts. Chris explained the use of prompt filters and the importance of ensuring the correct kernel configuration. He also mentioned the potential use of open telemetry for tracking.
• Token Compression: Garry asked about token compression techniques. Mark and Dmytro discussed prompt compression and mentioned that there are examples in the repository. They also noted the importance of maintaining observability and security.