Fix ASR pipeline mono conversion for channels-last audio (fixes #47886)

[Kayvan-Zahiri]

What does this PR do?

Fixes #47886.

AutomaticSpeechRecognitionPipeline averaged multi-channel input over a hardcoded
axis=0, which assumes channels-first (channels, samples). soundfile.read,
librosa.load(..., mono=False) and scipy.io.wavfile.read all return channels-last
(samples, channels), so on that layout the mean ran across time: a 3-second stereo
clip collapsed to 2 numbers, was padded back to 30s of silence, and transcribed as
' you'.

The pipeline logged a warning saying the conversion had happened. It had, along the
wrong axis.

stereo, sr = sf.read(buf, dtype="float32")   # (48000, 2)
asr(mono)     # ' See you in the next video!'
asr(stereo)   # ' you'          <- before
asr(stereo)   # ' See you in the next video!'   <- after

The fix

Infer the channel axis rather than assume it. Real audio has far more samples than
channels, so the shorter axis is the channel axis. (2, N) keeps working exactly as
before; (N, 2) stops being destroyed.

Also report tuple(inputs.shape) in the warning instead of inputs.ndim. The old
message interpolated ndim, so it printed "got 2" for every 2-D input regardless of
channel count, and the number that would actually help you debug never appeared.

Why not raise instead

That is a reasonable alternative and I would switch if you prefer it. The two sibling
pipelines already reject multi-channel outright:

# pipelines/audio_classification.py:232
if len(inputs.shape) != 1:
    raise ValueError("We expect a single channel audio input for AudioClassificationPipeline")

# pipelines/zero_shot_audio_classification.py:119
if len(audio.shape) != 1:
    raise ValueError("We expect a single channel audio input for ZeroShotAudioClassificationPipeline")

So the same stereo array raises a clear ValueError in two pipelines and silently
transcribes as ' you' in the third. I kept the conversion because it is existing
documented behaviour and removing it would break callers who currently pass (2, N)
successfully, but making ASR raise like its siblings is a three-line change if that is
the preferred direction.

I left ndim > 2 alone deliberately; that is a separate question from the reported bug.

Tests

test_multichannel_mono_conversion_is_layout_agnostic covers (N, 2), (2, N), and
2-D mono in both orientations, asserting all four match the 1-D baseline.

It fails on main with AssertionError: choose a window size 400 that is [2, 2] from
torchaudio, which is the bug surfacing as a 2-sample waveform.

tests/pipelines/test_pipelines_automatic_speech_recognition.py
    before: 5 failed, 16 passed, 35 skipped
    after:  5 failed, 17 passed, 35 skipped

Identical failures before and after, all pre-existing in my environment
(test_return_timestamps_ctc_fast, test_pipeline_generation_kwargs and friends), so
no regressions introduced. ruff check and ruff format --check both clean.

Who can review?

@ylacombe @eustlb (audio pipelines)

[Rocketknight1]

The idea seems okay, and I agree that silently averaging a dim of 48000 is very suboptimal, but I think we should be more rigorous to make sure we don't accidentally select other dimensions. In particular, inputs.ndim != 1 seems like it's written in cases where ndim might be 3 or larger, in which case the argmin might catch just about anything. A better solution might be simply to keep the existing code but throw an error if the mean() dim has a size other than 1/2.

[Rocketknight1]

This fix runs after some steps like F.resample(). We might have to back up and do the fix earlier, or add a fix in that path too

[Rocketknight1]

This feels extremely dangerous to me! If we want to be robust to either (samples, channels) or (channels, samples) then we should be much more explicit; and probably only do this for dims of 1 or 2 - if you have 5.1 audio then you can handle downmixing yourself 😅

[Kayvan-Zahiri]

Thanks, that's better than what I had. I pushed a version with the guessing taken out.

If the array is 2-D, the first axis is treated as channels, so one or two channels get averaged down, and anything more raises an error that suggests transposing the array if it happens to be the other way round. Any other number of dimensions raises as well.

I also moved the downmix so that it happens before the resample. It turns out resample wasn't the problem, since it handles either layout fine. The step that actually needed mono input was the stride calculation below it.

The tests cover all of those cases.

[Kayvan-Zahiri]

CI is red here but I don't think this PR caused it.

The one failure is EvollaModelTest::test_batching_equivalence, a NaN in a batched protein model forward. This PR touches pipelines/automatic_speech_recognition.py and its test file, nothing else.

The commit history rules it out. The two commits carrying real code changes both passed. The red run is ec8cc803, which only rewords a comment, three lines in and three out, no executable change:

commit	change	tests_torch shard 1/8
85a7c97	original fix	5905 passed, 0 failed
1ea2eb6	the rewrite from your review	pass
ec8cc80	comment wording only	5904 passed, 1 failed

Same shard, same 5,905 tests both times, and tests/models/evolla/ was collected and green in the earlier run, so the test set didn't change, only the comment did.

Happy to rebase if a re-run is easier that way.

[Rocketknight1]

Yes, LGTM now!

[github-actions]

---

CI recap

Dashboard: View test results in Grafana
Latest run: 32249501407:2
Result: success | Jobs: 16 | Tests: 158,810 | Failures: 0 | Duration: 12h 47m

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.