Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

sofakng wrote:

What GPU is needed for 4k (8-bit)?

Do we need an RTX 4090 or is a “slow” 3080 Ti good enough?

Evern 3090 is not enough. 4080 is probably fine with it but nobody tested it yet

377 (edited by aloola 29-12-2022 10:52:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

sofakng wrote:

What GPU is needed for 4k (8-bit)?

Do we need an RTX 4090 or is a “slow” 3080 Ti good enough?

my 3070ti can do x2 1440p at max.

2560*1440*24=88,473,600

3840*2160*24=199,065,600

so you might need a card that is stronger than 3070ti 2.5 times to x2 4k@24p

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:
sofakng wrote:

What GPU is needed for 4k (8-bit)?

Do we need an RTX 4090 or is a “slow” 3080 Ti good enough?

my 3070ti can do x2 1440p at max.

2560*1440*24=88,473,600

3840*2160*24=199,065,600

so you might need a card that is stronger than 3070ti 2.5 times to x2 4k@24p

looks about right. I run 2432x1376 x2 with 15% to spare

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I am new to SVP but due to RIFE implementation I ordered a 4090. If you guide me how to test / benchmark I will show all results

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

I am new to SVP but due to RIFE implementation I ordered a 4090. If you guide me how to test / benchmark I will show all results

Just install everything and run encode with rife. It will show max fps

381 (edited by grobalt 01-01-2023 20:54:23)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Trying (until the new card arrives) with my 6800 ... is this correct ?

https://i.ibb.co/LN1nLcW/rife-Radeon-6800-encode-1920.png#

If yes, a 4k is 17fps with 2 gpus threads and 9 fps with 1 gpu thread wink

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Any news on possible upcoming compability of the TensorRT version with SVP and MPC-HC?
Got the Rife-4.6 Ensemble True/False Versions to work by simply downloading them and place them in the models folder, but my 3080 struggles with real-time 1080p interpolation (720p works fine), so I am glad for every bit of additional performance optimization.
Are there any clear instructions/clever workarounds to get the new seemingly more performant version to work with SVP or is it outright not possible at the moment (and why)?
Thanks in advance!

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

4.6 true and false is not running with my 4090, mpv closes immediately . 4.4 works sometimes with low fps, most of the time black and white...

384

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

black and white colors you can get if you have AVS 3.5. You need to update it.

B/W image or any other issue playing via Avisynth Filter: ensure you have Avisynth+ 3.7.2 installed! Note that SVP installs AVS 3.5 by default.

https://www.svp-team.com/wiki/RIFE_AI_interpolation

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thanks, now colors but just 19fps wink

I have no clue how to install RifeRT Version 4.6 ...

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

=== RIFE / PyTorch+TensorRT installation ===

DO NOT DO THIS!
You DON'T NEED IT!
I'm SERIOUS! big_smile

============

!!! Advanced users only, if you don't know what this is all about - you don't need it !!!

0. Update SVP via online updater (this will update Python to ver. 3.10.9)

1. Dependencies installation:
--------------
Download TensorRT libs: https://github.com/HolyWu/vs-rife/relea … 2_win64.7z
unpack CUDA-blah-blah.7z to SVP 4\mpv64

run cmd as Administrator

cd SVP 4\mpv64

python get-pip.py

python -m pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/cu117
python -m pip install tensorrt-8.5.2.2-cp310-none-win_amd64.whl
python -m pip install torch-tensorrt-fx-only tqdm
python -m pip install -U --no-deps vsrife
python -m vsrife

A quick fix for vsrife's "slowmo" bug: replace __init__.py in SVP 4\mpv64\Lib\site-packages\vsrife\


2. SVP modification
---------------
Replace SVP 4\script\generate.js and base.py with the attached ones

menu -> Applications settings -> Additional options -> All settings, go to 'User defined options'
Title: TensortRT; Script name: rife_trt; Other values by default - "FRC profile", "ON or OFF"
Click 'Add option'

Go to the RIFE video profile, see the added TensortRT on/off swtich.
Make a copy of the RIFE video profile, one with TensorRT ON and another with OFF.

----------------
For a given video resolution the first-time start is super-slow (while it generates TRT net in a cache folder --> %APPDATA%\SVP4\cache\trt), so better do it in a "transcoding" mode.

Post's attachments

base.py 2.18 kb, 131 downloads since 2023-01-07 

generate.js 20.35 kb, 139 downloads since 2023-01-06 

__init__.py 13.29 kb, 126 downloads since 2023-01-07 

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Throws a CUDA device context error when try to run it in a real-time mpv. Transcoding does work though hmm

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

https://cdn.discordapp.com/attachments/290709370600423424/1060998958966640790/image.png
I tried vsrife trt before, but it doesn't work in real-time MPV, only when transcoding. I tried again with SVP and it still doesn't work in real-time and it's slower than TRT off LOL.


maybe you should try this? https://github.com/AmusementClub/vs-mlrt/wiki/RIFE
it works fine with mpc and mpv for me, 1080px3 in realtime.

389

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

=== RIFE / PyTorch+TensorRT installation ===

Huge thanks Chainik smile

390 (edited by UHD 06-01-2023 22:43:43)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thanks for the tests aloola smile

aloola wrote:

maybe you should try this? https://github.com/AmusementClub/vs-mlrt/wiki/RIFE
it works fine with mpc and mpv for me, 1080px3 in realtime.

I think we should first try to find the cause of the problems. Particularly since, with real-time interpolation, every frame matters.

vs-rife using TensorRT should be faster than vs-mlrt using TensorRT by at least 36% - https://github.com/HolyWu/vs-rife/discussions/19 :

45.91 fps NVIDIA GeForce RTX 3050 (1080p, FP16, model 4.6, vs-rife using TensorRT)
33.66 fps NVIDIA GeForce RTX 3050 (1080p, FP16, model 4.6, vs-mlrt using TensorRT)

Knowing the performance of graphics cards:

Fourth-generation Tensor Cores - Peak FP16 using the Sparsity feature:

660.6 TFLOPS - NVIDIA GeForce RTX 4090
https://images.nvidia.com/aem-dam/Solut … ecture.pdf

Third-Generation Tensor Cores - Peak FP16 using the Sparsity feature:

174 TFLOPS - NVIDIA GeForce RTX 3070 Ti
https://www.anandtech.com/show/17204/nv … more-money
72.8 TFLOPS - NVIDIA GeForce RTX 3050
https://www.computerbase.de/2022-01/nvi … 3050-test/

we should get the following results, at least in theory with scaling proportional to the increase in performance:

109,73 fps NVIDIA GeForce RTX 3070 Ti (1080p, FP16, model 4.6, vs-rife using TensorRT)
416,60 fps NVIDIA GeForce RTX 4090 (1080p, FP16, model 4.6, vs-rife using TensorRT)

391

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

I tried vsrife trt before, but it doesn't work in real-time MPV, only when transcoding.

Have you tried directly using this filter https://github.com/HolyWu/vs-rife in real time with mpv even on video with lower resolution, for example 720p?

392 (edited by aloola 06-01-2023 23:24:03)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:
aloola wrote:

I tried vsrife trt before, but it doesn't work in real-time MPV, only when transcoding.

Have you tried directly using this filter https://github.com/HolyWu/vs-rife in real time with mpv even on video with lower resolution, for example 720p?

yeah I did.
BTW the transcoding with TRT on in SVP didn't work for me. The output video still has the same old FPS and it's in slow motion and the video duration x2. I don't know what went wrong here. The output video with TRT off is working fine.

also, I have the cache file in trt cache folder: flownet_v4.6.pkl_NVIDIA GeForce RTX 3070 Ti_trt-8.5.2.2_1280x768_fp32_workspace-1073741824_scale-1.0_ensemble-False.pt

393 (edited by UHD 06-01-2023 23:37:01)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

If this vs-rife filter used directly with mpv with setting:

trt=False

allows real-time interpolation of 720p files

and with the setting

trt=True

does not allow real-time interpolation of 720p files, this means that we should report the issue to HolyWu.

Can you post the settings script you used with mpv?

I still don't have a graphics card that allows me to test, but if someone else confirms the same problem then we can report it to HolyWu.

The only point I can see in adding this filter to SVP is that it can interpolate in real time, and faster than using RIFE-ncnn-Vulkan.

394

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Can you post the settings script you used with mpv?

Something like here: https://github.com/HolyWu/vs-rife/issue … -967073164 but of course together with all the parameters you set for the vs-rife filter

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RTX4090:

Rife 4.4 with TensorRT on Scence change threshold 8%, GPU threads 2
Transcoding 1080p with 94fps
Transcoding 2160p with 19.7fps

Rife 4.6 ensembleFalse with TensorRT on Scence change threshold 10%, GPU threads 2
Transcoding 1080p with 100fps
Transcoding 2160p with 20fps

Rife 4.6 ensembleTrue with TensorRT on Scence change threshold 10%, GPU threads 2
Transcoding 2160p with 20.2fps

Rife 4.6 ensembleTrue with TensorRT on Scence change threshold 10%, GPU threads 4
Transcoding 2160p with 21.3fps

realtime mpv crashes or does not even start

the files in AppData\Roaming\SVP4\cache\trt look as:
flownet_v4.6.pkl_NVIDIA GeForce RTX 4090_trt-8.5.2.2_3840x2176_fp32_workspace-1073741824_scale-1.0_ensemble-False

i can offer teamviewer, discord .. whatever smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:
UHD wrote:

Can you post the settings script you used with mpv?

Something like here: https://github.com/HolyWu/vs-rife/issue … -967073164 but of course together with all the parameters you set for the vs-rife filter

no frame multi but it's eating alot of GPU usage. the same with trt SVP.
https://cdn.discordapp.com/attachments/290709370600423424/1061070311371579592/image.png

the code for mpv.

import vapoursynth as vs
core = vs.core
from vsrife import RIFE

clip = video_in
clip = clip.resize.Bicubic(format=vs.RGBS, matrix_in_s="709")
clip = RIFE(clip,trt=True,factor_num=5,factor_den=1)
clip = clip.resize.Bicubic(format=vs.YUV420P8, matrix_s="709")
clip.set_output()

397

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

the code for mpv.

import vapoursynth as vs
core = vs.core
from vsrife import RIFE

clip = video_in
clip = clip.resize.Bicubic(format=vs.RGBS, matrix_in_s="709")
clip = RIFE(clip,trt=True,factor_num=5,factor_den=1)
clip = clip.resize.Bicubic(format=vs.YUV420P8, matrix_s="709")
clip.set_output()

If I understood correctly, exactly the same settings, but without trt=True (or with trt=False) in your case allow for smooth real-time interpolation?

398

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

grobalt wrote:

I am new to SVP but due to RIFE implementation I ordered a 4090. If you guide me how to test / benchmark I will show all results

I have identical plans to buy a NVIDIA GeForce RTX 4090 also because of RIFE. I hope we can work something out together to make the fastest RIFE filter work in real time.

grobalt wrote:

RTX4090realtime mpv crashes or does not even start

You are already the third person to confirm this problem. Thanks for the tests smile

399

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

flownet_v4.6.pkl_NVIDIA GeForce RTX 3070 Ti_trt-8.5.2.2_1280x768_fp32_workspace-1073741824_scale-1.0_ensemble-False.pt


grobalt wrote:

flownet_v4.6.pkl_NVIDIA GeForce RTX 4090_trt-8.5.2.2_3840x2176_fp32_workspace-1073741824_scale-1.0_ensemble-False


aloola wrote:

clip = clip.resize.Bicubic(format=vs.RGBS, matrix_in_s="709")


I will be following this thread over the weekend and today at the end I propose to change it:

vs.RGBS

to

vs.RGBH

this should force FP16 precision in vs-rife and double the performance.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD
I propose to change it: vs.RGBS to vs.RGBH

great advice big_smile
updated the post above

aloola
The output video still has the same old FPS and it's in slow motion and the video duration x2.

looks like a vsrife's bug, fixed .py file attached above
https://github.com/HolyWu/vs-rife/issues/22