Skip to content

Conversation

@xenova
Copy link
Collaborator

@xenova xenova commented Mar 2, 2025

DAC:

import { DacModel, AutoFeatureExtractor } from '@huggingface/transformers';

const model_id = "onnx-community/dac_16khz-ONNX";
const model = await DacModel.from_pretrained(model_id);
const feature_extractor = await AutoFeatureExtractor.from_pretrained(model_id);

const audio_sample = new Float32Array(12000);

// pre-process the inputs
const inputs = await feature_extractor(audio_sample);
{
    // explicitly encode then decode the audio inputs
    const encoder_outputs = await model.encode(inputs);
    const { audio_values } = await model.decode(encoder_outputs);
    console.log(audio_values);
}

{
    // or the equivalent with a forward pass
    const { audio_values } = await model(inputs);
    console.log(audio_values);
}

Mimi:

import { MimiModel, AutoFeatureExtractor } from '@huggingface/transformers';

const model_id = "onnx-community/kyutai-mimi-ONNX";
const model = await MimiModel.from_pretrained(model_id);
const feature_extractor = await AutoFeatureExtractor.from_pretrained(model_id);

const audio_sample = new Float32Array(12000);

// pre-process the inputs
const inputs = await feature_extractor(audio_sample);
{
    // explicitly encode then decode the audio inputs
    const encoder_outputs = await model.encode(inputs);
    const { audio_values } = await model.decode(encoder_outputs);
    console.log(audio_values);
}

{
    // or the equivalent with a forward pass
    const { audio_values } = await model(inputs);
    console.log(audio_values);
}

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@xenova
Copy link
Collaborator Author

xenova commented Mar 5, 2025

In future, it could be cool to add support for Mimi w/ past key values to speed up streamed decoding. For now, this PR is okay.

@xenova xenova merged commit cdce6e8 into main Mar 5, 2025
4 checks passed
@xenova xenova deleted the add-mimi branch March 5, 2025 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants