Hi Lars, FFmpeg is pretty well tested with the fate suite of tests, probably a couple of thousand test now. See (currently not loading)
http://fate.ffmpeg.org/I used to run it and pass all the tests. Unluckily our tool chain is just too buggy now. Problems with GCC, problems with libc, problems with git, even problems with the PMSHELL at least.
To pass I was using an early build of GCC 4.4.6, libc065 on Warp v4.
Problems included. At one point using the official nasm built with OpenWatcom, I had four H264 conformance tests failing, there were quite a few at the time. This was fixed by recompiling nasm with GCC. Weird subtle failure.
Upgrading GCC caused various transcoding failures as well filter errors.
Upgrading to eCS caused floating point crashes without patching to restore the FPU control word.
Upgrading to libc066 fixed the floating point errors but caused errors transcoding to MP2 and failures with other tests that depended on the MP2 file.
Currently running the testsuite causes git to fail and I believe output its help, at least it is actually less that hangs waiting for input. IIRC, this is a test to make sure all code has correct licensing and cats the output of git through grep or similar.
There are other tests where the ffmpeg binary hangs, killing it allows the test suite to continue.
Most platforms usually pass all the tests.