201 (edited by UHD 14-04-2022 16:48:44)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RIFE Model 4.0 Benchmarks


720p - x2 interpolation - RIFE filter for VapourSynth (ncnn Vulkan):

70fps - NVIDIA GeForce RTX 2060 Mobile - Chainik
https://www.svp-team.com/forum/viewtopi … 158#p80158
92.956fps - NVIDIA GeForce RTX 2070 - blackmickey1007
https://www.svp-team.com/forum/viewtopi … 219#p80219
188.2fps - NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 163#p80163


720p - x2 interpolation - RIFE filter for VapourSynth (PyTorch CUDA):

54.115fps - NVIDIA GeForce RTX 2070 - blackmickey1007
https://www.svp-team.com/forum/viewtopi … 219#p80219
82.4fps - NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 691#p79691
91.3fps - NVIDIA GeForce RTX 3080 Ti - Quaternions
https://www.svp-team.com/forum/viewtopi … 727#p79727


1080p - x2 interpolation - RIFE filter for VapourSynth (PyTorch CUDA):

42.5fps - NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 699#p79699
45.0fps - NVIDIA GeForce RTX 3080 Ti - Quaternions
https://www.svp-team.com/forum/viewtopi … 719#p79719

-----
720p is original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing

1080p is an arbitrary file chosen

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

last ncnn optimizations are nuts yep
2.3x for RTX 3000 and 1.7x for RTX 2000
Seems like PyTorch is not needed anymore

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

SVP & RIFE updated: removed RIFE profile limitations; any target rate allowed (with model v4, obviously)

204 (edited by UHD 27-04-2022 12:44:18)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Huge thanks Chainik!!!

This is what I was waiting for! smile

I have another question: is there any chance to implement YUV444 format on the RIFE output, as requested by blackmickey1007 here: https://www.svp-team.com/forum/viewtopic.php?id=6582

What is your opinion Chainik on this?

205 (edited by UHD 27-04-2022 15:26:00)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I'd like to thank everyone again for the tests and explain again why I only ask and don't test anything myself.

The answer is simple: I don't have a suitable graphics card for testing, and my temporary PC is not suitable for the latest graphics cards anyway.

I've been saving money every month since 2019 to buy something really powerful. I had plans to upgrade my computer for 2020, but the situation at that time forced me to revise my plans. And maybe it's even better, because in the meantime the RIFE interpolation algorithm based on machine learning (artificial intelligence) appeared and it became clear that now the graphics card will be responsible for the best interpolation of frames in real time.

I'm now looking forward to COMPUTEX 2022, which will be held at the Taipei Nangang Exhibition Hall from May 24 to 27, 2022, where we may find out more about many new products that will be launched in the second half of this year.

It is in the second half of 2022 that I now intend to build my new PC completely from scratch. Then I will share very detailed test results using SVP & RIFE on the latest hardware that will be available then.

Until then, I would be grateful to everyone for sharing their tests here on the forum, so as to provide feedback to the SVP and RIFE developers, so that we can look forward to even better frame interpolation in the future, and to other testers, so that there are even more of us.

Special thanks to those involved in development:
https://github.com/hzwer/arXiv2021-RIFE
https://github.com/hzwer/Practical-RIFE
https://github.com/nihui/rife-ncnn-vulkan
https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan
https://www.svp-team.com/wiki/RIFE_AI_interpolation

and to all those testing the implementation of RIFE in SVP here on the forum.

206 (edited by UHD 27-04-2022 16:19:43)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Today is the big day!

Thanks to Chainik from today everyone can test x3, x4, x5...x10 RIFE frame interpolation in real time in the easy to use SVP.

We gain not only better smoothness, but above all better quality of interpolated frames, especially for x3 and x5 settings!

If anyone wants proof of the better quality of such settings they can find it here:  https://arxiv.org/pdf/2204.03513.pdf in Figure 6.

The graph (b) shows the quality of 7 interpolated frames for an interpolation factor of x8 for several algorithms, including RIFE. The worst quality for the RIFE algorithm is presented by Time Step 4. This is the same interpolated frame that we get with x2 interpolation.

So the best quality result will be obtained in RIFE when there is no need to interpolate the frame which will be exactly in the middle between two original frames, that is best to use interpolation factor x3, x5, x7 and so on. Of course, with the interpolation factor x4 we will still get a better effect than with x2, because the display time of this lowest quality interpolated frame will be half as long as with the interpolation factor x2.

By the way, has anyone noticed on the graph (a) in Figure 6, that we are getting a much faster algorithm for x4 and x8 interpolation factor than RIFE? Looking forward to the source code: https://github.com/feinanshan/M2M_VFI

Well, yes, I can not test, so I will paste such interesting tidbits. I hope that I'll be able to enrich this thread a bit before I build a new PC. wink

207

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

What am I missing? (MPV)

[E]: Playback [c95a15cf]: VS - Python exception: SmoothFps_RIFE: Function does not take argument(s) named multi

208 (edited by Blackfyre 28-04-2022 03:23:28)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Tried watching with RIFE did not work, video plays, SVP does not activate. Tried to transcode and getting this highlighted in the logs:

Code pasting upside/down.

Order in logs is from bottom to top.

11:16:48.871 [E]: Transcoder: VS - Script evaluation failed:
11:16:48.871 [E]: Transcoder: VS - Python exception: SmoothFps_RIFE: Function does not take argument(s) named multi
11:16:48.871 [E]: Transcoder: VS - Traceback (most recent call last):
11:16:48.871 [E]: Transcoder: VS - File 'src\cython\vapoursynth.pyx', line 2832, in vapoursynth._vpy_evaluate
11:16:48.871 [E]: Transcoder: VS - File 'src\cython\vapoursynth.pyx', line 2833, in vapoursynth._vpy_evaluate
11:16:48.871 [E]: Transcoder: VS - File 'C:\Users\name\AppData\Roaming\SVP4\scripts\ffff.py', line 75, in <module>
11:16:48.871 [E]: Transcoder: VS - smooth = interpolate(clip)
11:16:48.871 [E]: Transcoder: VS - File 'C:\Users\name\AppData\Roaming\SVP4\scripts\ffff.py', line 58, in interpolate
11:16:48.871 [E]: Transcoder: VS - smooth = core.svp2.SmoothFps_RIFE(smooth,smoothfps_params,src=input_um,multi=rife_num/rife_den,fps=src_fps)
11:16:48.871 [E]: Transcoder: VS - File 'src\cython\vapoursynth.pyx', line 2551, in vapoursynth.Function.__call__
11:16:48.871 [E]: Transcoder: VS - vapoursynth.Error: SmoothFps_RIFE: Function does not take argument(s) named multi
11:16:48.891 [E]: Transcoder: mpv failed with code 62097

209 (edited by XKCorp 28-04-2022 06:13:02)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Same error here, RIFE was working before the update (PMP/MPV).

17:02:16.653 [E]: Playback [658f65fa]: VS - Python exception: SmoothFps_RIFE: Function does not take argument(s) named multi

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

sorry, my fault big_smile
please update once again

211

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

please update once again

All good now.

From a quick test with RTX2070S + MPV with Generic (v4)

1920x1080 x2(48fps)
~100% GPU Load / Not fast enough

1920x800 x2(48fps)
~80% GPU Load

---

1280x720 x2(48fps)
~50% GPU Load

1280x528 x2(48fps)
~37% GPU Load

---

1280x720 x3(72fps)
~95% GPU Load / Not fast enough

1280x528 x3(72fps)
~70% GPU Load

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

=== RIFE / PyTorch installation ===

OUTDATED! DO NOT USE!

!!! Advanced users only, if you don't know what this is all about - you don't need it !!!

!!! Encoding only, not for a real-time playback !!!

0. Update SVP to the latest version

1. Dependencies installation:
--------------
download https://bootstrap.pypa.io/get-pip.py into SVP 4\mpv64

run cmd as Administrator
cd SVP 4\mpv64

run

python get-pip.py

run

python -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
python -m pip install vsrife --no-deps

will download 3+ GB and unpack it into 6+ GB of data big_smile

2. SVP modification
---------------
...

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Working great, thanks. Using x4 for 720p or less (old content) and x2 for 1080p

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

6900 XT checking in.
latest vapoursynth core 4.5.0.214
latest RIFE engine 3.0-1
display refresh rate set to 48Hz

RIFE Generic v4
1080p 24fps -> 48fps
~160W
1 second intervals, utilization fluctuates 30%-90%
1 second intervals, clock speed fluctuates 300MHz-2200Mhz
After a few minutes it starts dropping frames: anywhere from 1-5 frames dropped per second. Seeking a bit forward or behind resets and the video plays back smoothly for a few more minutes.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

=== RIFE / PyTorch installation ===

hello dev, thanks for the update.

I had a problem with this part. Do I need to install something else?
"python -m pip install vsrife"

https://cdn.discordapp.com/attachments/290709370600423424/969532234463842324/Untitled.png

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

they probably changed something...

try
> python -m pip install vsrife --no-deps

217 (edited by aloola 29-04-2022 12:40:56)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

they probably changed something...

try
> python -m pip install vsrife --no-deps

it works. I can install it now. but the ripe didn't work for me

https://media.discordapp.net/attachments/290709370600423424/969577797477601340/unknown.png

my spec. AMD RX 5900 + RTX 3070ti + Win 11 + nvidia driver 512.59

218 (edited by dlr5668 29-04-2022 13:12:54)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

my spec. AMD RX 5900 + RTX 3070ti + Win 11 + nvidia driver 512.59

RTX 2x/3x should skip pytorch. It actually slower

219 (edited by nerji 29-04-2022 14:42:36)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

it works. I can install it now. but the ripe didn't work for me

Check the event logs, might give you extra clues, mine said i was missing the `numpy` package


I got worse experience with it on though like dlr5668 mentioned (i'm using a 3090)

Post's attachments

Screenshot 2022-04-29 154044.jpg 61.47 kb, 83 downloads since 2022-04-29 

220 (edited by aloola 29-04-2022 15:19:14)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I uninstalled everything and just reinstalled svp. then actived profile "RIFE AI engine" and it works flawlessly.

my hardware couldn't go past 24fps x3 or 30fps x3. SVP index would below 0.9. also "anime" AI model profile is very lagging for me. V3/4 work best.

https://media.discordapp.net/attachments/290709370600423424/969616770170228746/unknown.png

221 (edited by RAGEdemon 01-05-2022 12:07:29)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Is there a way to downscale 1080p/4K content down to 720p BEFORE RIFE interpolates it live?

The objective is to have my high resolution videos downscaled to 720p so that I can use x3/x4/x5 RIFE.

With MPC/ffdshow, this was easily done by just rescaling the video in a step above AVISynth, in the FFDshow settings... I can't figure out a way to do it on mpv...


Thank you.

222 (edited by dlr5668 01-05-2022 12:12:52)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RAGEdemon wrote:

Is there a way to downscale 1080p/4K content down to 720p BEFORE RIFE interpolates it live?

The objective is to have my high resolution videos downscaled to 720p so that I can use x3/x4/x5 RIFE.

With MPC/ffdshow, this was easily done by just rescaling the video in a step above AVISynth, in the FFDshow settings... I can't figure out a way to do it on mpv...


Thank you.

https://i.imgur.com/rUKptFG.png

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Wow, thank you (especially to the devs!) I had no idea that option existed! SVP is a gift that keeps giving! big_smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

So, just tried uninstalling everything (Python 3.9, VapourSynth and SVP) and then only installing SVP, then following these instructions:

=== RIFE / PyTorch installation ===

!!! Advanced users only, if you don't know what this is all about - you don't need it !!!

!!! Encoding only, not for a real-time playback !!!

0. Update SVP to the latest version

1. Dependencies installation:
--------------
download https://bootstrap.pypa.io/get-pip.py into SVP 4\mpv64

run cmd as Administrator
cd SVP 4\mpv64

run

python get-pip.py
run

python -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
python -m pip install vsrife --no-deps
will download 3+ GB and unpack it into 6+ GB of data big_smile

2. SVP modification
---------------
---> no need to replace any files now <---

add custom profile option: go to Application settings -> User defined options, fill in:
- Title: RIFE via CUDA
- Script name: rife_cuda
- Options scope: FRC profile
- Allowed values: ON or OFF
Press "Add option"

in the "RIFE AI engine" video profile set "RIFE via CUDA" to ON
when it's ON then "AI model" and "GPU threads" change nothing

Previously, when I tried transcoding a video it would just flat out stop when looking for VapourSynth, but now that I have begun over, transcoding works but live playback seems unaffected.

I'm able to change the options in SVP itself and the program seems aware of the video and its current playback rate, but while the player itself reports SVP's stated one, madVR reports it as unaltered and I'm unable to see a difference.

Trying to switch back to the Automatic profile before starting over transitioned correctly and showed a difference in the video, but now it just freezes and I have to restart playback, while Python itself seems upset that I don't have the required components to make vsrife work as intended.

I'll attach logs and console output below:

23:39:32.946 [i]: Screens: screen 0 - 1920x1080 @360.000 Hz [using OS refresh rate], x1.3 [141 DPI]
23:39:32.946 [i]: Screens: primary screen is 0
23:41:48.668 [i]: Tube: video format id adaptive_hls-8112-0 selected
23:41:48.672 [i]: Tube: launching \C:\Program Files\MPC-HC\mpc-hc64.exe\ \%1\
23:41:53.892 [i]: AVSF: found new player instance
23:41:53.943 [i]: AVSF: filters in use: LAV Splitter Source (internal) -> LAV Video Decoder (internal) -> * -> madVR
23:41:53.949 [i]: AVSF: new video in mpc-hc64.exe (64-bit) [MPC-HC 1.9.18.0] on screen 0
23:41:53.951 [i]: Media: video looks like some kind of stream
23:41:53.951 [i]: Media: video 1920x1080 [PAR 1.000] at 23.975 fps (via video player)
23:41:53.951 [i]: Media: codec type is unknown, NV12
23:41:53.953 [i]: Playback: starting up...
23:41:53.962 [i]: Playback [e77a8ce3]: Frame server (64-bit) 3.5.0.0, AviSynth+ 3.5 (r3106, 3.5, x86_64), C:\WINDOWS\SYSTEM32\avisynth.dll
23:41:53.964 [i]: Playback [e77a8ce3]: resulting video frame 1920x1080
23:41:53.965 [i]: Playback [e77a8ce3]: 2 acceptible profiles, best is 'RIFE AI engine' [100]
23:41:53.968 [i]: Playback [e77a8ce3]: enabled while video is paused
23:41:53.974 [i]: Playback [e77a8ce3]: playing at 59.9375 [23.975 *5/2] 
23:42:14.328 [i]: Playback [e77a8ce3]: playing at 59.9375 [23.975 *5/2] 
23:42:24.115 [W]: Playback [e77a8ce3]: maximum FRC rate for RIFE v2-v3 is 2.0
23:42:24.118 [i]: Playback [e77a8ce3]: playing at 47.95 [23.975 *2/1] 
23:42:33.963 [i]: Playback [e77a8ce3]: playing at 47.95 [23.975 *2/1] 
23:42:41.252 [W]: Playback [e77a8ce3]: maximum FRC rate for RIFE v2-v3 is 2.0
23:42:41.254 [i]: Playback [e77a8ce3]: playing at 47.95 [23.975 *2/1] 
23:42:48.116 [W]: Playback [e77a8ce3]: maximum FRC rate for RIFE v2-v3 is 2.0
23:42:48.119 [i]: Playback [e77a8ce3]: playing at 47.95 [23.975 *2/1] 
23:42:58.025 [i]: Playback [e77a8ce3]: playing at 59.9375 [23.975 *5/2] 
23:43:01.733 [i]: Playback [e77a8ce3]: playing at 59.9375 [23.975 *5/2] 
23:43:04.059 [i]: Playback [e77a8ce3]: playing at 59.9375 [23.975 *5/2] 
 cd "C:\Program Files (x86)\SVP 4\mpv64"
ToasterDEV  C:/Program Files (x86)/SVP 4/mpv64  ﲍ                                                                     
 python get-pip.py
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
ToasterDEV  C:/Program Files (x86)/SVP 4/mpv64  ﲍ                                                                     
  .\python get-pip.py
Collecting pip
  Using cached pip-22.0.4-py3-none-any.whl (2.1 MB)
Collecting setuptools
  Using cached setuptools-62.1.0-py3-none-any.whl (1.1 MB)
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel, setuptools, pip
  WARNING: The script wheel.exe is installed in 'C:\Program Files (x86)\SVP 4\mpv64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip.exe, pip3.9.exe and pip3.exe are installed in 'C:\Program Files (x86)\SVP 4\mpv64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.0.4 setuptools-62.1.0 wheel-0.37.1
ToasterDEV  C:/Program Files (x86)/SVP 4/mpv64                                                                    
   .\python -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu113
Collecting torch
  Using cached https://download.pytorch.org/whl/cu113/torch-1.11.0%2Bcu113-cp39-cp39-win_amd64.whl (2186.0 MB)
Collecting typing-extensions
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Installing collected packages: typing-extensions, torch
  WARNING: The scripts convert-caffe2-to-onnx.exe, convert-onnx-to-caffe2.exe and torchrun.exe are installed in 'C:\Program Files (x86)\SVP 4\mpv64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed torch-1.11.0+cu113 typing-extensions-4.2.0
ToasterDEV  C:/Program Files (x86)/SVP 4/mpv64                                                                    
   .\python -m pip install vsrife --no-deps
Collecting vsrife
  Using cached vsrife-2.0.0-py3-none-any.whl (32.5 MB)
Installing collected packages: vsrife
Successfully installed vsrife-2.0.0
 ToasterDEV  C:/Program Files (x86)/SVP 4/mpv64                                                                   
   .\python -m pip install numpy
Collecting numpy
  Using cached numpy-1.22.3-cp39-cp39-win_amd64.whl (14.7 MB)
Installing collected packages: numpy
  WARNING: The script f2py.exe is installed in 'C:\Program Files (x86)\SVP 4\mpv64\Scripts' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
vsrife 2.0.0 requires VapourSynth>=55, which is not installed.
Successfully installed numpy-1.22.3

https://i.imgur.com/ed6CylG.png

Where should I begin to troubleshoot this issue?

Thanks in the meantime!

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

obviously, it doesn't work with Avisynth Filter