Commit 33f431e
feat(openai) - Support audio input and output modality
- Added support for audio input and output in OpenAI Chat Completion API
- Introduced new audio-related parameters, enums, and record types
- Updated ChatCompletionMessage, ChatCompletionChunk, and related classes
- Added new AudioParameters, AudioOutput, and InputAudio record types
- Implemented method to handle audio media content conversion
- Included new model enum for GPT-4o audio preview
- Extended existing API classes to accommodate audio modalities
- Modified usage tracking and metadata classes to handle audio-specific token details
- Improved ModelOptionsUtils with additional JSON utility methods
Tests:
- Updated test classes to validate audio input and output functionality
- Added integration tests for multimodal audio input with streaming and non-streaming methods
- Created parameterized tests for audio-enabled models
- Enhanced OpenAI API integration tests to cover audio-related scenarios
Docs:
- Updated documentation in spring-ai-docs to explain audio multimodality support
Resolves #15601 parent be0f9fb commit 33f431e
File tree
13 files changed
+403
-72
lines changed- models/spring-ai-openai/src
- main
- java/org/springframework/ai/openai
- api
- resources
- test/java/org/springframework/ai/openai
- api
- tool
- chat
- metadata
- spring-ai-core/src/main/java/org/springframework/ai/model
- spring-ai-docs/src/main/antora/modules/ROOT/pages/api/chat
13 files changed
+403
-72
lines changedLines changed: 29 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
| 58 | + | |
58 | 59 | | |
59 | 60 | | |
60 | 61 | | |
| |||
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
| 80 | + | |
79 | 81 | | |
80 | 82 | | |
81 | 83 | | |
| |||
406 | 408 | | |
407 | 409 | | |
408 | 410 | | |
409 | | - | |
| 411 | + | |
410 | 412 | | |
411 | 413 | | |
412 | 414 | | |
| |||
423 | 425 | | |
424 | 426 | | |
425 | 427 | | |
426 | | - | |
427 | | - | |
428 | | - | |
429 | | - | |
430 | | - | |
| 428 | + | |
431 | 429 | | |
432 | 430 | | |
433 | 431 | | |
| |||
446 | 444 | | |
447 | 445 | | |
448 | 446 | | |
449 | | - | |
| 447 | + | |
450 | 448 | | |
451 | 449 | | |
452 | 450 | | |
| |||
456 | 454 | | |
457 | 455 | | |
458 | 456 | | |
459 | | - | |
| 457 | + | |
460 | 458 | | |
461 | 459 | | |
462 | 460 | | |
| |||
508 | 506 | | |
509 | 507 | | |
510 | 508 | | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
511 | 532 | | |
512 | 533 | | |
513 | 534 | | |
| |||
0 commit comments