Bad video from h264 nal source file

[mondain]

I have an h.264 encoded file which looks good when I use ffmpeg to place it in an mp4 container, but when I use isoparser it doesnt look good at all. Heres the code that I used

H264TrackImpl h264Track = new H264TrackImpl(new FileDataSourceImpl("dump.h264"));
Movie m = new Movie();
m.addTrack(h264Track);
Container out = new DefaultMp4Builder().build(m);
FileOutputStream fos = new FileOutputStream(new File("output.mp4"));
FileChannel fc = fos.getChannel();
out.writeContainer(fc);
fos.close();

Here is the ffmpeg command line:

ffmpeg -i dump.h264 -an -threads 0 ffoutput.mp4

Bad image: https://dl.dropboxusercontent.com/u/7316897/projects/h264/bad.png
Good image: https://dl.dropboxusercontent.com/u/7316897/projects/h264/good.png

Dump file for verification: https://dl.dropboxusercontent.com/u/7316897/projects/h264/dump.h264

[sannies]

Hi, thank you for submitting such a complete, understandable and reproducable bug report!
The quick answer is: The stream contains more than 1 SPS/PPS and this causes problem/is not supported yet. This is your file ( NALType 7/8 is SPS/PPS NALType 5 is IDR Frame)

Got NAL @ 4 NALType = 7
Got NAL @ 31 NALType = 8
Got NAL @ 38 NALType = 6
Got NAL @ 664 NALType = 5
...
Got NAL @ 10783 NALType = 5
Got NAL @ 11582 NALType = 7
Got NAL @ 11609 NALType = 8
Got NAL @ 11616 NALType = 5

so you have one SPS/PPS pair then a few IDR NALs and then a new SPS/PPS. This is a problem for avc1 content in MP4 files as avc1 only allows on SPS/PPS combo (==one codec initialization) per file in the avcC Box and none in the actual payload. ffmpeg solves the problem by creating an invalid (according to specification) mp4

It is invalid cause the actual payload (samples in mdat) contains the SPS/PPS. Only the avc3 spec - a later revision of ISO/IEC 14496-15 - allows that.
So: ffmpeg muxing works but creates an invalid MP4 that most players will play anyways, the correct way would be to create an avc3 file. Let's see if I can mimick ffmpeg's (illegal but working) behavior and/or create a valid avc3. I got some time to spare today and it's an interesting issue but I'mnot sure if I can get it done.

[sannies]

Ok, more findings. The SPS/PPS is not even an issue. Even though the file contains SPS/PPS before each and every IDR frame they are all the same so just using the first one and filtering out the rest leaves us with a valid avc1 file.
The actual problem in this case is that the H264TrackImpl does not see that your IDR frames consist of more than one IDR NAL. Typically we have one IDR frame == one IDR NAL but in your case we have multiple IDR NAL (slices!?) belong to one frame.
I need to improve the frame boundary detection but nevertheless I was already able to improve a lot in the H264TrackImpl and in future I will detect the case where I need to use avc3. Stay tuned (this is really one of the more exciting bug reports).

[sannies]

LOL - just found the comment of my partner in crime Magnus from 2 years ago:

action = NALActions.STORE; // Will only work in single slice per frame mode!

[mondain]

I like getting helpful issue reports myself, so I try to provide the same. This data has been kicking my butt, since I get two different outputs for the same content; so I hope you'll have more insight on my issues.

[sannies]

Ok that should do it. Need to do very similar work for H265 anyways so this serves two causes: correct H264 muxing and training exercise for H265

[mondain]

Thanks for addressing this so quickly! Keep up the great work.