Skip to content

Commit 4c9a278

Browse files
committed
Group inference processors
Signed-off-by: Sasha Meister <117230141+ssh-meister@users.noreply.github.com>
1 parent c53be5e commit 4c9a278

File tree

11 files changed

+960
-13
lines changed

11 files changed

+960
-13
lines changed

docs/src/sdp/api.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,12 @@ used in the downstream processing for additional enhancement or filtering.
181181
.. autodata:: sdp.processors.tts.metrics.BandwidthEstimationProcessor
182182
:annotation:
183183

184+
.. autodata:: sdp.processors.FasterWhisperInference
185+
:annotation:
186+
187+
.. autodata:: sdp.processors.vLLMInference
188+
:annotation:
189+
184190
Text-only processors
185191
####################
186192

@@ -325,6 +331,12 @@ Data filtering
325331
.. autodata:: sdp.processors.RejectIfBanned
326332
:annotation:
327333

334+
.. autodata:: sdp.processors.DetectWhisperHallucinationFeatures
335+
:annotation:
336+
337+
.. autodata:: sdp.processors.CleanQwenGeneration
338+
:annotation:
339+
328340
Miscellaneous
329341
#############
330342

sdp/processors/__init__.py

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -128,9 +128,14 @@
128128
from sdp.processors.modify_manifest.make_letters_uppercase_after_period import (
129129
MakeLettersUppercaseAfterPeriod,
130130
)
131-
from sdp.processors.nemo.asr_inference import ASRInference
131+
from sdp.processors.inference.asr.nemo.asr_inference import ASRInference
132+
from sdp.processors.inference.asr.faster_whisper.faster_whisper_inference import FasterWhisperInference
133+
from sdp.processors.inference.asr.transformers.speech_recognition import ASRTransformers
134+
from sdp.processors.inference.asr.post_processing.whisper_hallucinations import DetectWhisperHallucinationFeatures
135+
from sdp.processors.inference.nlp.pc_inference import PCInference
136+
from sdp.processors.inference.llm.vllm.vllm import vLLMInference
137+
from sdp.processors.inference.llm.post_processing.qwen_cleaning import CleanQwenGeneration
132138
from sdp.processors.nemo.estimate_bandwidth import EstimateBandwidth
133-
from sdp.processors.nemo.pc_inference import PCInference
134139
from sdp.processors.toloka.accept_if import AcceptIfWERLess
135140
from sdp.processors.toloka.create_pool import CreateTolokaPool
136141
from sdp.processors.toloka.create_project import CreateTolokaProject

0 commit comments

Comments
 (0)