What device and /e/ version?
I had a stall-then-fallback with audio once. The timeout when to fallback was predefined.
Device config advertised hardware offload support for decoding, but something was amiss (either device can’t do the codec in hw or the library that’s needed for the hw interface is missing). So it stalled maybe half a minute or less before falling back to decoding in software.