SoundFlow v1.3.0

[LSXPrime]

Release Notes - SoundFlow v1.3.0

1480 Hours of Development - 1480 Hours of Goodbye.
That is the effort poured into this release. Version 1.3.0 is not just an update; it is a complete transformation of the library from a playback engine into a comprehensive creative audio framework.

This release introduces a pluggable codec architecture (FFmpeg), a complete MIDI sequencing and routing engine, a polyphonic synthesizer with SoundFont support, and a comprehensive metadata reading/writing API.

---

🌟 Key Highlights

• FFmpeg Codecs: Native support for decoding and encoding major formats (MP3, AAC, OGG, Opus, FLAC, ALAC) via a new pluggable extension.
• MIDI & Synthesis: A complete MIDI I/O backend, a built-in polyphonic Synthesizer, and SoundFont (.sf2) playback capability.
• Advanced Metadata: Read and write metadata tags (ID3v2, Vorbis Comments, MP4 Atoms) across all supported formats.
• MIDI Mapping: A reflection-based system to bind MIDI controllers to any audio parameter (Volume, Cutoff, etc.) in real-time.

---

📢 An Update, and a Very Long Goodbye

So, this update is finally out. It took 1480 hours of actual work to get here, a true labor of love (at the current moment, I am sure I am just a shut-in nerd). Hope you enjoy it, because... things are about to get interesting...

And now, for the main event: I have a mandatory engagement coming up. I’m heading off for a little vacation known as forced military service. This "vacation" lasts either 1 year (if the cosmos smiles upon me), or 3 years (which is, well, my actual suicide trigger).

The best part? No internet-enabled phone, no computer access. Total blackout.

So, this is almost certainly the last update you’ll see for a very long time. I might manage to squeeze one final push out before my mandatory long goodbye, but don't count on it.

I did the best I could with the time I had left. Enjoy it.

And to everyone who starred SoundFlow and actually used it - thank you. Seriously. Maybe someday I'll be that legendary unpaid open-source developer who finally breaks the entire modern infrastructure, too.

Oh, and Check out my other project, LSXPrime/ProseFlow. It's a system-wide, privacy-focused Copilot that's actually helpful, and I’m sure it will be a great assist to some of you.

---

🚀 New Features

🎧 Codecs & Formats (SoundFlow.Codecs.FFMpeg)

• Pluggable Architecture: AudioEngine now supports ICodecFactory registration, allowing dynamic addition of format support.
• New Extension: Introduced SoundFlow.Codecs.FFMpeg.
  • Decoders: MP3, AAC, OGG Vorbis, Opus, FLAC, ALAC, AC3, WAV, AIFF, and more.
  • Encoders: Write audio streams to MP3, AAC/M4A, FLAC, and WAV.
• Automatic Format Detection: AssetDataProvider and StreamDataProvider now probe streams to determine the correct codec automatically based on file headers.

🎹 MIDI & Synthesis (SoundFlow.Midi & SoundFlow.Synthesis)

• PortMidi Backend: Cross-platform MIDI I/O support via SoundFlow.Midi.PortMidi.
• MidiManager: A central hub for routing MIDI messages between physical devices and internal components.
• Synthesizer Component:
  • Multi-timbral, polyphonic synthesis engine.
  • Support for SoundFonts (.sf2) via SoundFontBank.
  • MPE (MIDI Polyphonic Expression) support for per-note pitch bend, pressure, and timbre.
  • Built-in ADSR envelopes, LFOs, and Filter modulation.
• MIDI Modifiers: Real-time MIDI effects including:
  • ArpeggiatorModifier (Up, Down, Random patterns).
  • HarmonizerModifier (Chord generation).
  • RandomizerModifier (Velocity/Pitch randomization).
  • VelocityModifier (Curves and scaling).
  • TransposeModifier.

📝 Metadata API (SoundFlow.Metadata)

• Unified Reader/Writer: Static SoundMetadataReader and SoundMetadataWriter classes.
• Format Support:
  • MP3: ID3v1 and ID3v2.3/2.4 reading and writing.
  • FLAC/OGG: Vorbis Comments and embedded pictures.
  • MP4/M4A: iTunes-style metadata atoms (moov/udta/ilst).
  • WAV/AIFF: RIFF INFO chunks and ID3 chunks.
• Cue Sheets: Support for parsing embedded Cue Sheets in FLAC and WAV.

🎛️ Core & Audio Processing

• Composition Overhaul:
  • Split responsibilities into CompositionEditor, CompositionRecorder (MIDI recording), and CompositionRenderer.
  • Added MidiTrack and MidiSegment alongside existing Audio tracks.
  • Project serialization now saves MIDI data, mappings, and tempo maps.
• ChannelMixer: New SIMD-accelerated (AVX/SSE) utility for efficient Mono/Stereo/Surround upmixing and downmixing.
• ResamplerModifier: Real-time variable speed/pitch resampling effect.
• MIDI Mapping: Use [ControllableParameter] attribute on properties to expose them to the new MidiMappingManager. Bind physical knobs to software parameters with Linear or Logarithmic scaling.

---

📚 Learning Resources & Samples

The documentation website has been fully updated to reflect the v1.3.0 API changes and new features.

We have also added three new sample projects to help you get started with the new capabilities:

• SoundFlow.Samples.Midi.BasicSequencing: Demonstrates how to programmatically create MIDI tracks, notes, and playback via the Synthesizer.
• SoundFlow.Samples.Midi.PropertyMapping: Shows how to map external MIDI controllers to software parameters (like Filter Cutoff) using the MidiMappingManager.
• SoundFlow.Samples.Midi.SynthesisModifiers: Illustrates how to use the Arpeggiator and Harmonizer modifiers to generate complex performances from simple input.

---

🛠 Improvements

• SIMD Math: Updated MathHelper and ChannelMixer to leverage hardware intrinsics for faster mixing and FFT operations.
• Logging: Replaced Console.WriteLine calls with a centralized SoundFlow.Utils.Log class. Subscribe to Log.OnLog to redirect internal engine logs to your own logging framework.
• Native Builds: Completely restructured the native C/C++ layer with dedicated CMake projects for miniaudio, portmidi, and ffmpeg-codec, including automated NuGet packaging via .targets.

---

📦 New & Updated NuGet Packages

All SoundFlow packages have been updated to version 1.3.0 and are available for download. This release introduces two major extensions required to unlock the new MIDI and Codec capabilities.

New Packages

• SoundFlow.Codecs.FFMpeg
  • Adds support for decoding and encoding MP3, AAC, OGG, Opus, FLAC, ALAC, and more.
• SoundFlow.Midi.PortMidi
  • Provides the cross-platform backend for MIDI I/O, device management, and synchronization.

Updated Packages

• SoundFlow (Core Library)
• SoundFlow.Extensions.WebRtc.Apm (Echo Cancellation & Noise Suppression)

---

⚠️ Breaking Changes

Version 1.3.0 introduces significant structural changes. If you are migrating from v1.2.x, please check the detailed migration guide:

https://lsxprime.github.io/soundflow-docs/#/docs/1.3.0/migration-guide

---

This discussion was created from the release SoundFlow v1.3.0.

[LSXPrime]

This release has fixed some NativeAot warnings related to device management. However, there are a few more warnings remaining related to composition persistence.

0>CompositionProjectManager.cs(186,20): Warning IL2026 : Using member 'System.Text.Json.JsonSerializer.Serialize<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.
0>CompositionProjectManager.cs(383,27): Warning IL2026 : Using member 'System.Text.Json.JsonSerializer.Deserialize<TValue>(String, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.
0>CompositionProjectManager.cs(781,46): Warning IL2026 : Using member 'System.Text.Json.JsonSerializer.SerializeToElement<TValue>(TValue, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.
0>CompositionProjectManager.cs(879,45): Warning IL2026 : Using member 'System.Text.Json.JsonSerializer.Deserialize(String, Type, JsonSerializerOptions)' which has 'RequiresUnreferencedCodeAttribute' can break functionality when trimming application code. JSON serialization and deserialization might require types that cannot be statically analyzed. Use the overload that takes a JsonTypeInfo or JsonSerializerContext, or make sure all of the required types are preserved.
0>CompositionProjectManager.cs(821,30): Warning IL2057 : Unrecognized value passed to the parameter 'typeName' of method 'System.Type.GetType(String)'. It's not possible to guarantee the availability of the target type.

These warnings (IL2026) are flagging potential runtime failures if we enable application trimming or Ahead-of-Time (AOT) compilation. The root issue is that we are using the reflection-based overloads of System.Text.Json.JsonSerializer. To fix this, I am planning to switch to using the source-generated overloads via a JsonSerializerContext. The last warning (IL2057) is similar, pointing out dynamic type loading (Type.GetType) which is also incompatible with trimming.

I noticed these NativeAot issues just minutes before releasing. Both fixes could have been merged, but doing so would introduce additional public APIs. This means anyone publishing with NativeAOT would need to manually use these APIs on every single external component, modifier, or analyzer they write.

But my philosophy remains: minimize what the developer needs to do and automate everything the library can handle. To do this properly--without creating extra developer work--I need to create a dedicated Source Generator package to automatically discover and handle these requirements. Because of this necessary work, I decided to postpone the full fix for a later release.

Another point that I am worried about is NativeAOT compatibility with Reflection. The new MIDI mapping system heavily depends on Reflection to control the mapped components/modifiers' properties and methods at runtime. Since I've never personally used NativeAOT and only noticed this compatibility challenge minutes before release (thanks to a report in the discussions), I haven't done any testing beyond basic playback on a NativeAOT-enabled project.

[theoractice]

I only use SoundFlow for audio playback, and I don't know how important NativeAOT is for other users. I can see Avalonia's choice to be cross-platform and to support NativeAOT has made it quite successful, and there may be greater demand for NativeAOT after it lands on Android/iOS, especially considering the significant improvements in app startup speed.

[LSXPrime]

I don't think anyone is going to create a DAW for Android/iOS, and MIDI doesn't seem to be supported on them by PortMidi either. Since the current situation is problematic only for the composition persistence features, I think it's safe to build for NativeAOT.

[theoractice]

I don't think anyone is going to create a DAW for Android/iOS, and MIDI doesn't seem to be supported on them by PortMidi either. Since the current situation is problematic only for the composition persistence features, I think it's safe to build for NativeAOT.

Haha, I haven't tried 1.3.0 yet and I even don't know the MIDI part is not cross-platform (actually I've never used MIDI). I think it's totally fine to keep the current design until there is actual need for NativeAOT.

[LSXPrime]

Good morning, In case you were the one to create a DAW on Android/iOS (PortMidi still doesn't support them yet), or if you're targeting any other platform that requires NativeAOT:

I just released V1.4.0, and it now has full NativeAOT support.

For composition persistence features (using SaveProjectAsync and LoadProjectAsync at CompositionProjectManager), you should be aware of a couple of requirements:

1. You will need to register custom modifiers (if they exist; built-in modifiers are already registered) in the TypeRegistry class.
2. You’ll need to create a custom JsonContext with your new custom modifiers.

Please refer to the Migration Guide of V1.3.0 to V1.4.0 for full details.