decodeAudioData and corrupted files

[rtoy]

Describe the issue

If decodeAudioData is called on a corrupted encoded file, what should happen? There's some additional detail in https://crbug.com/1161118 with a test file, but in essence the middle of the file is badly formed. Chrome stops decoding at this point and just returns what's been decoded already. Firefox skips the bad part and continues decoding. For both browsers, the decoded audio is shorter than expected, with Chrome being much shorter and Firefox just a little bit shorter.

This was briefly discussed in today's teleconf, and the attendees leaned towards throwing an error if the file is badly formed in any way. Currently there's no way to know if the file is badly formed except to check if the decoded result has the expected length. (This is somewhat complicated if resampling is needed.)

In any case, we should clarify what should happen here.

[chrisguttandin]

Just for reference, the decode() method of the HTMLImageElement is meant to throw an EncodingError when decoding fails.

If decoding fails (for example due to invalid image data), reject promise with an "EncodingError" DOMException.

https://html.spec.whatwg.org/multipage/embedded-content.html#dom-img-decode

[haywirez]

I'm also in favor of throwing an error and not returning partial data. If that behavior is desired, it's always possible to use a custom decoder.

[bradisbell]

It is common to have corrupt files that are quite playable. I don't think it is desirable to throw a decoding error for something like an hour-long podcast if there is one MP3 frame in the middle that has an error. (Audio files like these are generally playable via an Audio element, but with a slight glitch/hiccup.)

There are also media files that are playable but may not have been properly closed at the end of their recording. I think that decoding as much of the audio as possible/reasonable is ideal. Some way of surfacing a decoding warning though would also be useful.

As a developer, I would expect that if it can be reasonably played through in an audio element, it should be able to be decoded for use in the Web Audio API, wherever practical.

@haywirez ... it's always possible to use a custom decoder.

This is not always the case. Licensing and performance issues around codecs often prevent usage of custom decoders.

[rtoy]

Teleconf: There was a difference of opinions here and some wanted an error for any kind of decoding problem. Others wanted to just skip over the bad packets.

But in any case, we should handle this in v1 and decide what should happen.

[padenot]

AudioWG resolution:
It would be best to reject the file (it's corrupted after all), but this is a breaking change: it would mean that code that used to not throw an exception now throws an exception, and this can (and will) break websites.

Implementation are encouraged to log an error in the developer console when this happens.

Web Codecs will make it easy to choose the best behaviour for a particular use-case.

[rtoy]

Just want to also add a note the @chrisguttandin had an interesting use case where he creates WAV files with the length set to maximum, and then adds data until done and saves the file. The resulting WAV file will be shorter than the length says. Is this an error in decodeAudioData?

[bradisbell]

"Live" streams for Mastroska/WebM use the same method.... setting the maximum duration to signify an undefined duration. I'd hope that this, and the WAV file issue, are not treated as errors.

[rtoy]

Teleconf: Reopening issue. We should add a non-normative note that browsers should log an error in the developer console when this happens. This API has been basically unchanged in the 10+ years it's been around (except for detaching the buffer and returning a promise). We don't want to break anything now. Text and conditions TBD.