Topic: A/V desync with RIFE, duplicate frames removal, and h.265 @ 23.976 fps
In short: I get very noticable A/V desync in MPV under conditions noted in the topic title. Is this a known issue? Is there some configuration that will make it work correctly?
Before I get into my problems, I want to say that for anime, SVP with RIFE and especifically with "duplicate frames removal" set to "remove every other frame" is absolutely amazing! Praise be to the developers! When I first set it to "remove every other frame", it felt like going from 8 fps to 120 fps (hand-drawn animations within a scene often are 8 fps, but it varies). If only it worked properly...
The longer story:
Since switching to the RIFE profile and enabling "duplicate frames removal", I have found that the video and audio gradually desync in most of the videos I watch. Originally I was using FRC x5 ("To screen" when the TV is set to 120Hz or 119.88Hz), and I think the video was lagging. Now I'm testing with x2, and the visuals precede the corresponding audio by 1-2 seconds after 20 minutes or so. Seeking restores sync temporarily.
I have a Ryzen 9 9950X3D, RTX 4080 Super 16GB and 96GB RAM. With FRC x5 and Anime4K upscaling shaders on a 1080p 23.976 fps source, GPU usage is 90-95%. It is of course much lower at x3 or x2, which is what I've been testing with.
I found that the easiest way to test and reproduce the issue is to run a video file through SVPcode and then seek toward the end of the resultant video. The issue has been present in every video I have tested when the conditions described in the topic title are met. It could still be an issue specific to MPV, since SVPcode says it uses MPV.
h.264 videos do not desync. If I take one that works fine and convert it to h.265 as follows, interpolating the output file always results in desync.
ffmpeg -i INPUTFILE -c:v libx265 -acodec copy OUTPUTFILENormally everything I watch with SVP is 23.976 fps (1080p anime), but the issue affects other videos as well. For testing, I used the following live action video:
https://commons.wikimedia.org/wiki/File … sode).webm
This has 25 fps. If I convert it to h.265 and leave it at 25 fps, there is no desync. So I converted the framerate:
ffmpeg -i ATTENTION_Mr_Bean!_(Mr_Bean_Full_Episode).webm -r 24000/1001 -c:v libx265 -acodec copy bean.mkv After I run bean.mkv through SVPcode, the output file has considerable A/V desync.
I also tried various other video and audio encoding settings while trying to match either the good or bad videos, but it appears that only the video codec and framerate matter.
For comparing videos, I've been using the Tab key which I've bound in input.conf:
TAB set pause yes; seek 1200 absoluteI start up a video and press Tab (I know I could have used --start at the command line instead). Even just comparing the one frame at 20:00 and a few seconds after that clearly shows that the video is out of sync. If I change the seek to 2400 for Mr Bean, in the source video I see and hear him close the car door, but in the SVP video, I only hear it; the car door is already closed.
I searched the forum on multiple occasions and didn't find anything that seemed directly relevant, but some topics gave me ideas.
It was mentioned (in 2022, I think) that there could be issues with some VFR videos. I used ffmpeg's vfrdet filter as per https://superuser.com/a/1487417/94186, and it reports a value of about 0.58 (and min: 41 max: 42 avg: 41) for every 23.976 fps video I pass it, including the h.264 videos which don't desync. I'm guessing that it's not actually possible for a 23.976 fps video to be CFR with these codecs, and being "variable rate" is not the issue.
I think someone who had stuttering (which I don't have) and desync solved it by using a second GPU for output. I've tried using the integrated Radeon for output. I had to set MPV to prefer the RTX 4080, otherwise I couldn't get 60 fps let alone 120. I have a spare GTX 1070 I could install in a second slot, but haven't tried it because it doesn't support 4K 120Hz through HDMI.
Someone indicated inadequate hardware could be an issue. It shouldn't be in my case for a number of reasons. In particular, SVPcode interpolates at much faster than real-time with FRC x2, and still desyncs.
My mpv.conf is fairly minimal at the moment:
fullscreen
volume=60
title=${filename}
audio-exclusive=yes
audio-channels=empty
alang=jpn,ja
slang=en,eng,ass
sub=auto
video-sync=audio
hwdec=no
pause
# SVP
input-ipc-server=mpvpipe
hr-seek-framedrop=no
# because I'm using uosc
osd-bar=no
# Optimized shaders for lower-end GPU: Mode A (Fast)
# glsl-shaders="~~/shaders/Anime4K_Clamp_Highlights.glsl;~~/shaders/Anime4K_Restore_CNN_M.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_M.glsl;~~/shaders/Anime4K_AutoDownscalePre_x2.glsl;~~/shaders/Anime4K_AutoDownscalePre_x4.glsl;~~/shaders/Anime4K_Upscale_CNN_x2_S.glsl"I tried various video-sync modes and any other MPV settings I could find relating to A/V sync, and none helped. Hardware decoding was in use by default for h.265, but disabling it didn't help.
I have been testing without the shaders, but normally have them enabled while watching.
In SVP, my settings are:
Engine: TensorRT
GPU threads: 2
AI model: 4.16_lite (I have tried others)
Performance boost: Enabled
Scene change detection: Image comparison (the other methods don't appear to work at all)
Scene change threshold: 15% (seems to work well for me)
Processing of scene changes: Blend
Duplicate frames removal: Remove every other frame (of course)
VLC Player is able to play all of my videos with FRC x3 and duplicate frames removal enabled, without desync.
MPC-HC and PotPlayer basically do not work if I enable duplicate frames removal - video output freezes almost immediately and audio continues. Is this normal?
