ValueError in "gather_label_embeddings"

[matakanobu]

In "gather_label_embeddings", it could be "ValueError" in addition to "RuntimeError" when a segment it too small. Like this.

  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/pipelines/speech_turn_assignment.py", line 103, in apply
    X, assigned_labels, skipped_labels = gather_label_embeddings(
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/pipelines/utils.py", line 122, in gather_label_embeddings
    x = embeddings.crop(file, label_support)
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/core/inference.py", line 430, in crop
    return self.infer(waveform[None])[0]
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/core/inference.py", line 188, in infer
    outputs = self.model(chunks.to(self.device))
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/models/embedding/xvector.py", line 165, in forward
    outputs = self.sincnet(waveforms).squeeze(dim=1)
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/pyannote/audio/models/blocks/sincnet.py", line 93, in forward
    outputs = F.leaky_relu(norm1d(pool1d(outputs)))
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/modules/instancenorm.py", line 57, in forward
    return F.instance_norm(
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/functional.py", line 2184, in instance_norm
    _verify_batch_size(input.size())
  File "/home/vscode/.cache/pypoetry/virtualenvs/diarization-pyannote2-xS3fZVNL-py3.8/lib/python3.8/site-packages/torch/nn/functional.py", line 2114, in _verify_batch_size
    raise ValueError("Expected more than 1 value per channel when training, got input size {}".format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 60, 1])

I checked a duration of a segment where the exception occurred.

[duration of a segment when ValueError]
0.0675
0.06750000000010914
0.06750000000010914
0.06750000000010914
0.06749999999999545
0.03374999999994088
0.06749999999965439

[duration of a segment when RuntimeError]
0.13499999999976353
0.05062499999996817
0.05062500000030923
0.08437499999990905
0.20249999999987267
0.10125000000016371

It seems that ValueError occurres when a segment is too small.
So, I suggest that update into like the following.

try:
    x = embeddings.crop(file, label_support)
except RuntimeError:
    # skip labels so small that we cannot even extract embeddings
    skipped_labels.append(label)
    continue
except ValueError:
    # skip labels so small that we cannot even extract embeddings
    skipped_labels.append(label)
    continue

What do you think of this idea? Or I think that it needs to be processed somehow.

[hbredin]

This gather_label_embedding thing will most likely be removed when I finally rewrite the speaker diarization pipeline for pyannote 2.0. So I probably won't spend some time on this (buggy anyway) function...

See PR #666 for the current status of upcoming speaker diarization pipeline.