Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Since the last SVP update I can use TRT to watch 4k 24p movies at 2x!

My 3080Ti 12GB (undervolted 1750MHz/800mV) run CUDA at 95% but it's smooth, I have no lags smile

My config:
Engine = TensorRT
GPU threads = 2
AI model = 4.6
Performance Boost = yes

Player = Plex HTPC (mpv + video quality ultra)

Thank you Chainik.

627 (edited by Xenocyde 24-01-2023 13:12:47)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Piovasia wrote:

Could someone share me the generate.js and base.py file, please ?  (╯•﹏•╰)

Go to this page and find Chainik's post with red text warnings (towards the top of the page). It has those 2 files.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Xenocyde wrote:
Piovasia wrote:

Could someone share me the generate.js and base.py file, please ?  (╯•﹏•╰)

Go to this page and find Chainik's post with red text warnings (towards the top of the page). It has those 2 files.


Those 2 files doesn't work, I have tried many time, I find all 26 page in this topic and can't get them anywhere (╯•﹏•╰)

629 (edited by aloola 24-01-2023 13:39:04)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Piovasia wrote:

Could someone share me the generate.js and base.py file, please ?  (╯•﹏•╰)

just uninstall SVP, and install SVP again.


That's guide is outdated, you don't need it.

more infos here:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

630 (edited by Piovasia 24-01-2023 15:53:56)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

just uninstall SVP, and install SVP again.


That's guide is outdated, you don't need it.

more infos here:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

I know that's guide is outdated.
But for me, I think it's much better than version update in SVP now
I can choose to load model from: https://github.com/styler00dollar/Vapou … cnn-Vulkan, the assemble version pretty smooth and not only limited by version 4.4 and 4.6 now in SVP.
After update in SVP, I regret, I can't roll back.

631 (edited by aloola 24-01-2023 16:45:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Piovasia wrote:
aloola wrote:

just uninstall SVP, and install SVP again.


That's guide is outdated, you don't need it.

more infos here:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

I know that's guide is outdated.
But for me, I think it's much better than version update in SVP now
I can choose to load model from: https://github.com/styler00dollar/Vapou … cnn-Vulkan, the assemble version pretty smooth and not only limited by version 4.4 and 4.6 now in SVP.
After update in SVP, I regret, I can't roll back.

it's still there, RIFE NCNN/Vulkan (old version).

https://media.discordapp.net/attachments/290709370600423424/1067484032876687380/image.png

TRT model and NCNN model are different. (iirc)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:
Piovasia wrote:
aloola wrote:

just uninstall SVP, and install SVP again.


That's guide is outdated, you don't need it.

more infos here:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

I know that's guide is outdated.
But for me, I think it's much better than version update in SVP now
I can choose to load model from: https://github.com/styler00dollar/Vapou … cnn-Vulkan, the assemble version pretty smooth and not only limited by version 4.4 and 4.6 now in SVP.
After update in SVP, I regret, I can't roll back.

it's still there, RIFE NCNN/Vulkan (old version).

https://media.discordapp.net/attachments/290709370600423424/1067484032876687380/image.png

TRT model and NCNN model are different. (iirc)

Umm, If you had installed old method, you can load all Ncnn model with TensorRt.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Piovasia wrote:

Umm, If you had installed old method, you can load all Ncnn model with TensorRt.

Chainik wrote:

    That one - https://github.com/AmusementClub/vs-mlrt - is MUUUUCH better
    -------------
    Post updates:
    09/01: updated both libs and scripts, should improve 4K performance on 4080+
    -------------
    Let's try this:

    0. nothing to do with Python big_smile
    1. download --> https://www.svp-team.com/files/temp/rife-trt-0701.7z <--, unpack into SVP 4\rife (you should already have this folder), so vstrt.dll must be in the root, i.e. SVP 4\rife\vstrt.dll
    2. replace generate.js, base.py in SVP 4\script; restart SVP to be sure
    3. menu -> Applications settings -> Additional options -> All settings, go to 'User defined options'
    Title: TensortRT; Script name: rife_trt; Other values by default - "FRC profile", "ON or OFF"
    Click 'Add option'
    Go to the RIFE video profile, see the added TensortRT on/off swtich.
    Make a copy of the RIFE video profile, one with TensorRT ON and another with OFF.

    Pros:
    - no Pytorch!
    - works in a real-time
    - even faster (?) than vsrife [could be just because of some TensorRT options]
    Cons:
    - even slower (?) first-time initialization [could be just because of some TensorRT options]
    - a nasty command-line window will pop-up for every new video resolution

    It's OK for the command-line window to stay for 4-5 mins. Don't close it! Wait!


You followed this guide, right?

OK, I got what you want now. RIFE TRT in that guide always runs model 4.6 from the start. Changing models in that guide does nothing.


TRT model version use: rife_v4.x.onnx *.onnx file
ncnn model version use: flownet.bin, flownet.param file


the models from https://github.com/styler00dollar/Vapou … cnn-Vulkan, only contain flownet.bin, flownet.param file. It means it only runs with RIFE ncnn, not RIFE TRT.
the title also said VapourSynth-RIFE-ncnn-Vulkan.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:

You followed this guide, right?

OK, I got what you want now. RIFE TRT in that guide always runs model 4.6 from the start. Changing models in that guide does nothing.


TRT model version use: rife_v4.x.onnx *.onnx file
ncnn model version use: flownet.bin, flownet.param file


the models from https://github.com/styler00dollar/Vapou … cnn-Vulkan, only contain flownet.bin, flownet.param file. It means it only runs with RIFE ncnn, not RIFE TRT.
the title also said VapourSynth-RIFE-ncnn-Vulkan.

Oh, I see
Thanks you very much! (≧▽≦)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I'm using the TRT RIFE model and I think it's working so far, although I'm seeing the following warnings when launching mpv via command line:

vstrt: TensorRT version mismatch, built with 2135 but loaded with 2136; continue but fingers crossed...
CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars

Just wondering if this is expected or if I did something wrong during setup.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Hello

I have this issue when try to play 4K movies with RIFE:

An error has occurred. MPC-HC will close now.

Exception:
ACCESS VIOLATION
Crashing module:
c:\Program Files (x86)\SVP 4\rife\vstrt.dll

Offset: Ox74770AE9E, Codebase: Ox7FF7D2FCOOOO
The thread 9936 tried to read memory at address Ox8"

Can any one help me?

Post's attachments

Captura de tela 2023-01-30 105329.png, 132.8 kb, 441 x 340
Captura de tela 2023-01-30 105329.png 132.8 kb, 74 downloads since 2023-01-30 

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Isnt DLSS3.0 features like frame gen and nvidia reflex require HAGS to be enabled?
Or what is causing HAGS to make the real time decoding slower?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

earl088 wrote:

Isnt DLSS3.0 features like frame gen and nvidia reflex require HAGS to be enabled?
Or what is causing HAGS to make the real time decoding slower?

Reflex no, you can force it in most DX11/12 games for ages https://leonardo.osnova.io/ea1540f2-2fd … 84a5890f51 with GTX900 or newer

HAGS is only needed for frame generation feature of DLSS. In my experience HAGS cause problems with VR games and CUDa workload

639 (edited by beastsofty 14-02-2023 14:30:05)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

HolyWu is actively supporting the RIFE filter for VapourSynth based on PyTorch, maybe he will be able to help you over the implementation of this filter for real-time interpolation?  9apps
cartoon hd

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

https://upload.cc/i1/2023/02/09/LDSZE7.png
I get this error when I use RIFE, please help

Post's attachments

IMG_20230209_174733_344.jpg, 8.38 kb, 491 x 36
IMG_20230209_174733_344.jpg 8.38 kb, 64 downloads since 2023-02-09 

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

install avisynth 3.7.2

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Based on this post and my benchmark, hardware-accelerated GPU scheduling ON will reduce RIFE performance by ~30%.

https://github.com/AmusementClub/vs-mlr … nt-4957580

643 (edited by Tosskek 17-02-2023 23:48:51)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Hi, RTX 4070 Ti reporting in.
CPU i7 5950x, RAM 3200 DDR4
HAGS off
Hags on gives 50 instead of 60 fps.

https://i.imgur.com/SokMzlQ.jpg

Post's attachments

1080pp.JPG 346.48 kb, 37 downloads since 2023-02-17 

644 (edited by Ironclad 18-02-2023 08:07:17)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

22:33:32.546 [i]: AVSF: found new player instance
22:33:32.572 [i]: AVSF: filters in use: LAV Splitter Source -> LAV Video Decoder -> * -> madVR
22:33:32.573 [i]: AVSF: new video in mpc-hc64.exe (64-bit) [MPC-HC 2.0.0.0] on screen 0
22:33:32.614 [i]: Media: video 1920x1440 [PAR 1.000] at 23.976 fps
22:33:32.614 [i]: Media: codec type is AV1, YUV/4:2:0/10 bits
22:33:32.615 [i]: Playback: starting up...
22:33:32.619 [i]: Playback [73c0e3c8]: Frame server (64-bit) C:\Users\Sahil\AppData\Local\Programs\VapourSynth\core\vapoursynth.dll
22:33:32.620 [i]: Playback [73c0e3c8]: resulting video frame 1920x1440
22:33:32.621 [i]: Playback [73c0e3c8]: 4 acceptible profiles, best is 'RIFE AI engine' [5000]
22:33:32.623 [i]: Playback [73c0e3c8]: enabled while video is playing
22:33:32.626 [i]: Playback [73c0e3c8]: playing at 59.94 [23.976 *5/2] /10 bit
22:33:32.939 [E]: Playback [73c0e3c8]: VS - Python exception: [WinError 740] The requested operation requires elevation
22:33:32.939 [E]: Playback [73c0e3c8]: VS - Traceback (most recent call last):
22:33:32.939 [E]: Playback [73c0e3c8]: VS - File \src\cython\vapoursynth.pyx\, line 2866, in vapoursynth._vpy_evaluate
22:33:32.939 [E]: Playback [73c0e3c8]: VS - File \src\cython\vapoursynth.pyx\, line 2867, in vapoursynth._vpy_evaluate
22:33:32.939 [E]: Playback [73c0e3c8]: VS - File \C:\Users\Sahil\AppData\Roaming\SVP4\scripts\73c0e3c8.py\, line 77, in <module>
22:33:32.940 [E]: Playback [73c0e3c8]: VS - smooth = interpolate(clip)
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Users\Sahil\AppData\Roaming\SVP4\scripts\73c0e3c8.py\, line 56, in interpolate
22:33:32.940 [E]: Playback [73c0e3c8]: VS - smooth = RIFE(input_m,multi=rife_num,model=rife_mnum,backend=trt_backend)
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 936, in RIFE
22:33:32.940 [E]: Playback [73c0e3c8]: VS - output0 = RIFEMerge(
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 821, in RIFEMerge
22:33:32.940 [E]: Playback [73c0e3c8]: VS - return inference_with_fallback(
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 1420, in inference_with_fallback
22:33:32.940 [E]: Playback [73c0e3c8]: VS - raise e
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 1399, in inference_with_fallback
22:33:32.940 [E]: Playback [73c0e3c8]: VS - return _inference(
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 1339, in _inference
22:33:32.940 [E]: Playback [73c0e3c8]: VS - engine_path = trtexec(
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Program Files (x86)\SVP 4\rife\vsmlrt.py\, line 1172, in trtexec
22:33:32.940 [E]: Playback [73c0e3c8]: VS - completed_process = subprocess.run(args, env=env, check=False, stdout=sys.stderr)
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Users\Sahil\AppData\Local\Programs\Python\Python38\Lib\subprocess.py\, line 493, in run
22:33:32.940 [E]: Playback [73c0e3c8]: VS - with Popen(*popenargs, **kwargs) as process:
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Users\Sahil\AppData\Local\Programs\Python\Python38\Lib\subprocess.py\, line 858, in __init__
22:33:32.940 [E]: Playback [73c0e3c8]: VS - self._execute_child(args, executable, preexec_fn, close_fds,
22:33:32.940 [E]: Playback [73c0e3c8]: VS - File \C:\Users\Sahil\AppData\Local\Programs\Python\Python38\Lib\subprocess.py\, line 1311, in _execute_child
22:33:32.940 [E]: Playback [73c0e3c8]: VS - hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
22:33:32.940 [E]: Playback [73c0e3c8]: VS - OSError: [WinError 740] The requested operation requires elevation

Issue resolved by running mpc-hc in admin oddly.

Another issue. Trt inference model has to be built or cached for every new resolution. This caused me a lot of confusion for real time playback as it would hang and there's no indication how long that will take to finish building. Once that's done realtime playback works. It might help new users if the TensorRT version came with a benchmark tool that contained some reference footage at different resolutions and gave a notification when all the refence models for common resolutions were saved. Also going over the vsmlrt documentation, it's extremely important to set fp16, num_parameters, and previously workspace before the inference model is made. How is svp handling fp16 vs fp32 and does the gpu thread setting refer to the num_stream parameter in vsmlrt?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

> Python exception: [WinError 740] The requested operation requires elevation
> Issue resolved by running mpc-hc in admin oddly.

I can only think your user doesn't have write permissions for the temp dir for some reason

> Trt inference model has to be built or cached for every new resolution

Not true, unless you enable the "perf. boost" profile option.

646 (edited by Ironclad 18-02-2023 08:33:32)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> Python exception: [WinError 740] The requested operation requires elevation
> Issue resolved by running mpc-hc in admin oddly.

I can only think your user doesn't have write permissions for the temp dir for some reason

> Trt inference model has to be built or cached for every new resolution

Not true, unless you enable the "perf. boost" profile option.

Oh ok. The tooltip says 5%, but is this separate from the parameters mentioned in vsmlrt?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

https://www.svp-team.com/wiki/RIFE_AI_i … le_options

Performance boost - TensorRT only. Increase performance up to ~5% and lower memory usage in exchange for running a 5-10-minutes optimization pass in a command-line window for _every_ new video resolution. Previous runs are cached. For example, you're opening a 1080p video for the first time, you'll wait 5+ mins. Every other 1080p videos will start instantly, but when the frame size changes due to, for example, black bars cut off, you'll wait another 5+ mins.

648 (edited by Ironclad 21-02-2023 14:15:21)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

> Trt inference model has to be built or cached for every new resolution

Not true, unless you enable the "perf. boost" profile option.

Problem is if you disable it then the generalized models which are too large for my vram size are generated instead.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I've been noticing some drops in performance here and there, not sure what triggers this. The SVP index drops to half for several seconds and then it goes back up to full. Could this be a VRAM problem? I have an RTX 3070 mobile with 8 GB, performance boost enabled, but I did not deactivate GPU scheduling.

650

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Thanks for all the replies to my posts and the test results you have provided here. I have some very interesting news for you now....