Update video_processor.py

[auesro]

Following issue #475 , I modified slightly video_processor.py .
Picking up FPS correctly from the original video is essential to obtain a labeled video of correct duration. Both openCV and ffmpeg implementations depend on OpenCV to read this tag from the header in the original video file.
At least one type of file (that with fourcc code 844516695 ) is read by OpenCV as to have 1000 fps (being original 30 fps), which gives rise to a labeled video 33.33 times shorter than the original.
If other file types were found to have the same problem, the if condition could be easily expanded to include them.

[AlexEMG]

Do you have any reason to believe that codec 844516695 is always misread?

[auesro]

Actually no, that's the only metadata I could find to tell apart my non-accurately readable videos from those that work flawlessly.
Indeed, I tried something: I re-encoded a perfectly OpenCV-readable file to the WMV2 codec (thats what the OpenCV fourcc number 844516695 means) with FFMPEG and OpenCV was able to read the metadata without issue so I can only think that the Logitech acquisition software is not good at encoding videos or does not fill the metadata correctly... but so far I am unable to find another piece of information in the metadata informative about these weird videos...

So for now, I would hold off and not apply this PR. But maybe leave a message somewhere about the possibility for some videos not to work properly in OpenCV even though you dont recieve any warning.

[MMathisLab]

I added a note to the troubleshooting tips wiki: https://github.com/AlexEMG/DeepLabCut/wiki/Troubleshooting-Tips

[catubc]

Hi. Are there any plans to reconcile this branch?
Thanks
Catubc

[AlexEMG]

I think this specific fix, is not generally useful for users. Does 844516695 happen to be your fourcc number?

[auesro]

@catubc could you check if your video fourcc code is the same one I was reporting?

[MMathisLab]

we did add a more robust frame reader for some issues actually;

DeepLabCut/deeplabcut/pose_estimation_tensorflow/predict_multianimal.py

Line 34 in e5c3384

robust_nframes=False,

You can set robust_nframes=True and test it out.

[AlexEMG]

Indeed, sometimes the metadata is incorrect with respect to the actual data. Then one solution, as linked by @MMathisLab is to first iterate over the frames (to count how many there are) and then use that. For the create labeled video steps one can also pass the target frame rate (if I am not mistaken)

[auesro]

@AlexEMG you can pass the argument outputframerate but I tried that and the result was the same, wrong FPS, hence wrong duration labeled video. As far as I remember, there were multiple places in the code (from training till creating the labeled videos) where FPS was read through OpenCV, so practically overwritting your input framerate.
But I must say I havent tried the latest version, and didnt know about the robust_nframes.

[catubc]

Hey everyone, so for me the issue was a 2-step issue:

1. Original .wmv files were too high resolution 1280 x 640. So I used ffmpeg to reduce to 640 x 320 and save as .mp4 files.

2. Then trained DLC on 640 x 320.

3. Then labeled on 640 x 320. But as I mentioned the pipeline reads the converted data as having 1000.0s FPS.

The solution that worked for me was to explicitly specify frame rate during the ffmpeg conversion process using -r 15 (in my case the vids had 15Hz).

ffmpeg -i input.wmv -c:v libx264 -crf 23 -q:a 100 -vf scale=640:360 -r 15 output.mp4

So now DLC reports the correct frame rates.

[MMathisLab]

@auesro no worries! - robust_nframes was just added last week - the list of actual updates would have been so long for the release note (it was 141 files changed and ~30K lines ... whelp!) Hence why, in part, we beta released it to be sure we didn't screw up everyone in one fell swoop 😄

[auesro]

Dear @MMathisLab ,
I am back to this. I was quickly getting back to speed again.
If I followed everything correctly here I think I found an unexpected behavior:
The option robust_nframes is not used for non multi-animal datasets even though you can explicit it in the call to the analyze_videos function, right? Because when dealing with non multi-animal datasets the code will go here and then opencv will be used for counting the number of frames, fps and duration, correct?
If thats the case, it would be great to be able to use the robust mode in non multi-animal datasets.