Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Xenocyde wrote:
thericeking wrote:
reynbow wrote:

Where do you put these files?

Nvm. Am dumb. Found it.

C:\Program Files (x86)\SVP 4\rife\models\rife


I downloaded the fles on the link but they don't show up as an option in the drop down menu. How do I enable 4.9?

If you rename 4.9 to 4.4 and replace the existing 4.4, it will work with the 4.4 option in the dropdown.



This is the clear instructions I was looking for. I hope there's an update soon. I purchased two licenses plus one for my iPhone. So far I like SVP a lot.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

The simplest way to install a new Rife model is to copy the Rife onnx file to the SVP rife directory then modify the rife_trt_model property under the video profiles options settings in SVP.

I decided to compare Rife 4.9 v1 with Rife 4.9 v2. The good news is that as expected seek speed is almost instantaneous seeking and for me it uses about 4 percent less GPU than v1.  However, even though it technically uses a little less GPU power, my GPU temps went up about 4 degrees. BTW I don't watch 4K with Rife, but so far I haven't seen any resolution issues with v2.

Lol, a few weeks ago I was wondering why they hadn't released new versions after 4.6, and now we're at 4.9 V2 already.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I get the following message on the dos window when I used 4.9 for the first time. I'm driving a RTX 3080 "Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32."

929 (edited by dawkinscm 04-11-2023 11:26:55)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

thericeking wrote:

I get the following message on the dos window when I used 4.9 for the first time. I'm driving a RTX 3080 "Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32."

We all get that. It's a warning, not an error. It's mostly fine.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

maybe worth waiting for 4.10 and 4.11? big_smile

Oh don't tease me like that, if they're coming in the next couple of months, I can wait.

It does seem like updates are coming fast past few months.

https://media.giphy.com/media/DPznISmq0hLRQqG71g/giphy.gif

931 (edited by Blackfyre 04-11-2023 01:00:22)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

I decided to compare Rife 4.9 v1 with Rife 4.9 v2.

Interesting on Github under 4.9 v2 notes it says this:

This improvement does not work after onnx file renaming

The full quote:

RIFE v4.7-v4.9 models with v2 representation for TRT backend now has improved accuracy, contributed by @charlessuh (#66 (comment)). This has been backported to master. This improvement does not work after onnx file renaming.


EDIT:

Tried to test 4.9 v2 from 2 days ago, changed the name to 4.4 as usual, but it doesn't build. In the beginning of the building process, it breaks and stops.

EDIT 2 - adding end of crash info

[11/04/2023-08:48:47] [W] Could not read timing cache from: C:\Users\Username\AppData\Roaming\SVP4\cache\Program Files (x86)/SVP 4/rife\models\rife\rife_v4.4.onnx.3840x2144_fp16_trt-8502_cudnn_I-fp16_O-fp16_NVIDIA-GeForce-RTX-3090_cf15df92.engine.cache. A new timing cache will be generated and written.
[11/04/2023-08:48:47] [E] Error[4]: [network.cpp::nvinfer1::Network::validate::3100] Error Code 4: Internal Error (input: for dimension number 1 in profile 0 does not match network definition (got min=11, opt=11, max=11), expected min=opt=max=7).)
[11/04/2023-08:48:47] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[11/04/2023-08:48:47] [E] Engine could not be created from network
[11/04/2023-08:48:47] [E] Building engine failed
[11/04/2023-08:48:47] [E] Failed to create engine from model or file.
[11/04/2023-08:48:47] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # C:/Program Files (x86)/SVP 4/rife\vsmlrt-cuda\trtexec --onnx=C:/Program Files (x86)/SVP 4/rife\models\rife\rife_v4.4.onnx --timingCacheFile=C:\Users\Username\AppData\Roaming\SVP4\cache\Program Files (x86)/SVP 4/rife\models\rife\rife_v4.4.onnx.3840x2144_fp16_trt-8502_cudnn_I-fp16_O-fp16_NVIDIA-GeForce-RTX-3090_cf15df92.engine.cache --device=0 --saveEngine=C:\Users\Username\AppData\Roaming\SVP4\cache\Program Files (x86)/SVP 4/rife\models\rife\rife_v4.4.onnx.3840x2144_fp16_trt-8502_cudnn_I-fp16_O-fp16_NVIDIA-GeForce-RTX-3090_cf15df92.engine --shapes=input:1x11x2144x3840 --fp16 --tacticSources=-CUBLAS,-CUBLAS_LT --useCudaGraph --noDataTransfers --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw

932 (edited by aloola 04-11-2023 06:20:30)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Blackfyre wrote:
dawkinscm wrote:

I decided to compare Rife 4.9 v1 with Rife 4.9 v2.

Interesting on Github under 4.9 v2 notes it says this:

This improvement does not work after onnx file renaming

The full quote:

RIFE v4.7-v4.9 models with v2 representation for TRT backend now has improved accuracy, contributed by @charlessuh (#66 (comment)). This has been backported to master. This improvement does not work after onnx file renaming.

I tried the new V2 update but I cannot use the model above v4.7-v4.9 v2

I got the error when I ran V4.7>4.9 _v2
[E] Error[4]: [graphShapeAnalyzer.cpp::nvinfer1::builder::`anonymous-namespace'::ShapeAnalyzerImpl::analyzeShapes::2013] Error Code 4: Miscellaneous (IElementWiseLayer /Add: broadcast dimensions must be conformable)

933 (edited by cws 04-11-2023 02:54:00)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I think in `def RIFE()` of `vsmlrt.py`, you can set `_implementation` to 2, and it'll look for the model in the `C:\Program Files (x86)\SVP 4\rife\models\rife_v2` directory.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

From what I understand, 4.9 V2 is not really a major update over V1, so I'm not going to bother with it for now.

935 (edited by dawkinscm 05-11-2023 09:45:08)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Today I decided to compare Rife 4.6 v2 with 4.9 v2. After a full reboot, Rife 4.9 v2 GPU usage dips below 70% into the high 60s and averages around 74%. This is better than I reported yesterday. But Rife 4.6 v2 is below 50% smile Both models use about 8.6GB of RAM. Please note that my GPU stats are always a little higher than most because the GPU is doing multiple things.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

aloola wrote:
Blackfyre wrote:
dawkinscm wrote:

I decided to compare Rife 4.9 v1 with Rife 4.9 v2.

Interesting on Github under 4.9 v2 notes it says this:

This improvement does not work after onnx file renaming

The full quote:

RIFE v4.7-v4.9 models with v2 representation for TRT backend now has improved accuracy, contributed by @charlessuh (#66 (comment)). This has been backported to master. This improvement does not work after onnx file renaming.

I tried the new V2 update but I cannot use the model above v4.7-v4.9 v2

I got the error when I ran V4.7>4.9 _v2
[E] Error[4]: [graphShapeAnalyzer.cpp::nvinfer1::builder::`anonymous-namespace'::ShapeAnalyzerImpl::analyzeShapes::2013] Error Code 4: Miscellaneous (IElementWiseLayer /Add: broadcast dimensions must be conformable)

What about use this method instead of renaming

Post's attachments

Screenshot 2023-11-04 093905.png 139.22 kb, 22 downloads since 2023-11-04 

937 (edited by TechnoStone 04-11-2023 14:35:58)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01 wrote:

What about use this method instead of renaming

I use this method, works for v1 as should, but v2 models fail to build cache

[11/04/2023-07:20:20] [W] Could not read timing cache from: C:\Users\TECHNO~1\AppData\Local\Temp\Program Files (x86)/SVP 4/rife\models\rife\2eceb28c.engine.cache. A new timing cache will be generated and written.
[11/04/2023-07:20:20] [E] Error[4]: [network.cpp::nvinfer1::Network::validate::3100] Error Code 4: Internal Error (input: for dimension number 1 in profile 0 does not match network definition (got min=11, opt=11, max=11), expected min=opt=max=7).)
[11/04/2023-07:20:20] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[11/04/2023-07:20:20] [E] Engine could not be created from network
[11/04/2023-07:20:20] [E] Building engine failed
[11/04/2023-07:20:20] [E] Failed to create engine from model or file.
[11/04/2023-07:20:20] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8501] # C:/Program Files (x86)/SVP 4/rife\vsmlrt-cuda\trtexec --onnx=C:/Program Files (x86)/SVP 4/rife\models\rife\rife_v4.9.onnx --timingCacheFile=C:\Users\TECHNO~1\AppData\Local\Temp\Program Files (x86)/SVP 4/rife\models\rife\2eceb28c.engine.cache --device=0 --saveEngine=C:\Users\TECHNO~1\AppData\Local\Temp\Program Files (x86)/SVP 4/rife\models\rife\2eceb28c.engine --shapes=input:1x11x1088x1920 --fp16 --tacticSources=-CUBLAS,-CUBLAS_LT,-CUDNN,+EDGE_MASK_CONVOLUTIONS,+JIT_CONVOLUTIONS --useCudaGraph --noDataTransfers --noTF32 --inputIOFormats=fp16:chw --outputIOFormats=fp16:chw

cws wrote:

I think in `def RIFE()` of `vsmlrt.py`, you can set `_implementation` to 2, and it'll look for the model in the `C:\Program Files (x86)\SVP 4\rife\models\rife_v2` directory.

Tried this, rife just stops working after that. Tried using with newer vsmlrt.py (with 4.8 defined), same behavior (on both 4.8 and 4.9). Can you share your setup? The tree of C:\Program Files (x86)\SVP 4\rife\models and your vsmlrt.py. Or your command to build engine if you don't use SVP's variant



UPD. Also tried building manually with using 7 planes instead of 11, placed generated files to folder where others are, but that doesn't work

17:33:09.966 [i]: VideoPlayer: mpv [mpv.exe] connected, waiting for the video info...
17:33:10.500 [i]: VideoPlayer: mpv 0.35.1 /via 'mpvSockets/20840'
17:33:11.036 [i]: Media: video 1920x1080 [PAR 1.000] at 23.976 fps
17:33:11.036 [i]: Media: codec type is AVC, YUV/4:2:0/8 bits/BT.709 BT.709
17:33:11.045 [i]: Playback: starting up...
17:33:11.049 [i]: Playback [6baaeb05]: Frame server (64-bit) C:\Program Files (x86)\SVP 4\mpv64\vapoursynth.dll
17:33:11.050 [i]: Playback [6baaeb05]: resulting video frame 1920x1080
17:33:11.052 [i]: Playback [6baaeb05]: 3 acceptible profiles, best is 'Алгоритм RIFE' [600]
17:33:11.054 [i]: Playback [6baaeb05]: enabled while video is playing
17:33:11.058 [i]: Playback [6baaeb05]: playing at 71.928 [23.976 *3/1] 
17:33:11.620 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^
17:33:11.620 [E]: Playback [6baaeb05]: VS - File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 920, in RIFEMerge
17:33:11.620 [E]: Playback [6baaeb05]: VS - return inference_with_fallback(
17:33:11.620 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^^^^^^^^^^^^^^^
17:33:11.620 [E]: Playback [6baaeb05]: VS - File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 1631, in inference_with_fallback
17:33:11.620 [E]: Playback [6baaeb05]: VS - raise e
17:33:11.620 [E]: Playback [6baaeb05]: VS - File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 1610, in inference_with_fallback
17:33:11.620 [E]: Playback [6baaeb05]: VS - return _inference(
17:33:11.620 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^^
17:33:11.620 [E]: Playback [6baaeb05]: VS - File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 1574, in _inference
17:33:11.620 [E]: Playback [6baaeb05]: VS - clip = core.trt.Model(
17:33:11.620 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^^^^^^
17:33:11.620 [E]: Playback [6baaeb05]: VS - File 'src\\cython\\vapoursynth.pyx', line 2857, in vapoursynth.Function.__call__
17:33:11.620 [E]: Playback [6baaeb05]: VS - vapoursynth.Error: operator (): expects 7 input planes
17:33:11.624 [E]: Playback [6baaeb05]: VS - could not init VS
17:33:11.634 [E]: Playback [6baaeb05]: VS - Script evaluation failed:
17:33:11.634 [E]: Playback [6baaeb05]: VS - Python exception: operator (): expects 7 input planes
17:33:11.634 [E]: Playback [6baaeb05]: VS - Traceback (most recent call last):
17:33:11.635 [E]: Playback [6baaeb05]: VS - File 'src\\cython\\vapoursynth.pyx', line 3121, in vapoursynth._vpy_evaluate
17:33:11.635 [E]: Playback [6baaeb05]: VS - File 'src\\cython\\vapoursynth.pyx', line 3122, in vapoursynth._vpy_evaluate
17:33:11.635 [E]: Playback [6baaeb05]: VS - File 'C:\Users\TechnoStone\AppData\Roaming\SVP4\scripts\6baaeb05.py', line 78, in <module>
17:33:11.635 [E]: Playback [6baaeb05]: VS - smooth = interpolate(clip)
17:33:11.635 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^^^^^^^^
17:33:11.635 [E]: Playback [6baaeb05]: VS - File 'C:\Users\TechnoStone\AppData\Roaming\SVP4\scripts\6baaeb05.py', line 57, in interpolate
17:33:11.635 [E]: Playback [6baaeb05]: VS - smooth = RIFE(input_m,multi=rife_num,model=rife_mnum,backend=trt_backend)
17:33:11.635 [E]: Playback [6baaeb05]: VS - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17:33:11.635 [E]: Playback [6baaeb05]: VS - File 'C:\Program Files (x86)\SVP 4\rife\vsmlrt.py', line 1046, in RIFE
17:33:11.635 [E]: Playback [6baaeb05]: VS - output0 = RIFEMerge(

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

The simplest way to install a new Rife model is to copy the Rife onnx file to the SVP rife directory then modify the rife_trt_model property under the video profiles options settings in SVP.

I decided to compare Rife 4.9 v1 with Rife 4.9 v2. The good news is that as expected seek speed is almost instantaneous seeking and for me it uses about 4 percent less GPU than v1.  However, even though it technically uses a little less GPU power, my GPU temps went up about 4 degrees. BTW I don't watch 4K with Rife, but so far I haven't seen any resolution issues with v2.


Could you be kind and specify where could I get 4.9 v2, please ?

939 (edited by dawkinscm 04-11-2023 15:36:41)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

maorgr wrote:
dawkinscm wrote:

The simplest way to install a new Rife model is to copy the Rife onnx file to the SVP rife directory then modify the rife_trt_model property under the video profiles options settings in SVP.

I decided to compare Rife 4.9 v1 with Rife 4.9 v2. The good news is that as expected seek speed is almost instantaneous seeking and for me it uses about 4 percent less GPU than v1.  However, even though it technically uses a little less GPU power, my GPU temps went up about 4 degrees. BTW I don't watch 4K with Rife, but so far I haven't seen any resolution issues with v2.


Could you be kind and specify where could I get 4.9 v2, please ?

https://github.com/AmusementClub/vs-mlr … e_v2_v2.7z

This is the specific URL but the page itself has lots of other models too smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

4.9 v2 works just fine here

941 (edited by TechnoStone 04-11-2023 18:50:00)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:

4.9 v2 works just fine here

How to set up 4.9v2 here? Imagine you've just installed svp with rife from utilities, what steps will you do next?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

well,
in my crappy computer, 4.9v2 works with avisynth, not vapoursynth sad

with vapoursynth, I see the command prompt processing the movie, but when its gone, the movie is stuck and does not play (audio continues to play though)

943 (edited by aloola 04-11-2023 22:43:39)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

to use the new V2 47>49 model with bug fixes you will need the new vsmlrt file.
https://github.com/AmusementClub/vs-mlrt/issues/66

for now just stick with 49 V1, 49 V2 need more testing.

944 (edited by cws 05-11-2023 01:24:07)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

You can update `vsmlrt.py` by getting a copy from the `AmusementClub/vs-mlrt` repo, then modifying:

    alter_engine_path = os.path.join(
        tempfile.gettempdir(),
        os.path.splitdrive(engine_path)[1][1:]
    )

into

    alter_engine_path = os.path.join(
        os.path.expandvars("%APPDATA%\\SVP4\\cache\\"),
        os.path.splitdrive(engine_path)[1][1:]
    )

Then you can force the v2 implementation by either:

1) modifying `def RIFE()` in `vsmlrt.py` to make the implementation argument default to 2, or

2) modifying line 20 of `C:\Program Files (x86)\SVP 4\script\base.py` to pass in an additional implementation argument: `smooth = RIFE(input_m,multi=rife_num,model=rife_mnum,backend=trt_backend,_implementation=2)`

Note I have downloaded the v2 models to the folder `C:\Program Files (x86)\SVP 4\rife\models\rife_v2`. You should also set `rife_trt_model` to 49 in SVP's Advanced settings in the "Video profiles options" section that corresponds to the profile you're using, so that it uses rife_v4.9.onnx (without file renaming shenanigans.)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

is it just me or does 47-49 V2 require padding to make it work? I need to padding it to make it work.

4.0 - 46 v2 don't need padding to work.


RIFE v2 models which handle paddings internally and reduce memory transactions on heterogeneous devices.

946 (edited by dawkinscm 05-11-2023 17:36:51)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

cws wrote:
    alter_engine_path = os.path.join(
        os.path.expandvars("%APPDATA%\\SVP4\\cache\\"),
        os.path.splitdrive(engine_path)[1][1:]
    )

I always forget about this. It doesn't really matter because Rife still works without it. But it is neater and I might add it back for any future update.

947 (edited by maorgr 05-11-2023 15:17:41)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

it didn't change anything - i can only do rife 49 with avisynth... vapoursynth doesn't work and keeps spitting out errors.

i dont know python, but this is the error message i'm getting when using Vapoursyth:

[11/05/2023-17:16:04] [E] Error[4]: [network.cpp::nvinfer1::Network::validate::3100] Error Code 4: Internal Error (input: for dimension number 1 in profile 0 does not match network definition (got min=11, opt=11, max=11), expected min=opt=max=7).)
[11/05/2023-17:16:04] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[11/05/2023-17:16:04] [E] Engine could not be created from network
[11/05/2023-17:16:04] [E] Building engine failed
[11/05/2023-17:16:04] [E] Failed to create engine from model or file.
[11/05/2023-17:16:04] [E] Engine set up failed

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

maorgr wrote:

it didn't change anything - i can only do rife 49 with avisynth... vapoursynth doesn't work and keeps spitting out errors.

i dont know python, but this is the error message i'm getting when using Vapoursyth:

[11/05/2023-17:16:04] [E] Error[4]: [network.cpp::nvinfer1::Network::validate::3100] Error Code 4: Internal Error (input: for dimension number 1 in profile 0 does not match network definition (got min=11, opt=11, max=11), expected min=opt=max=7).)
[11/05/2023-17:16:04] [E] Error[2]: [builder.cpp::nvinfer1::builder::Builder::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[11/05/2023-17:16:04] [E] Engine could not be created from network
[11/05/2023-17:16:04] [E] Building engine failed
[11/05/2023-17:16:04] [E] Failed to create engine from model or file.
[11/05/2023-17:16:04] [E] Engine set up failed

Check that you don't have your own local copies of vapoursynth or avisynth installed. That will cause problems.

949 (edited by cws 05-11-2023 17:46:35)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Are you sure you followed all the steps?

You should not be replacing or renaming any files in `C:\Program Files (x86)\SVP 4\rife\models\rife`.

You should put the v2 models in the new directory `C:\Program Files (x86)\SVP 4\rife\models\rife_v2`.

After replacing `vsmlrt.py` with the latest you downloaded from GitHub, then modifying the engine path, then modifying it to use `_implementation=2`, you should set the "rife_trt_model" property in SVP to specify the file you want to load (like setting it to "49") instead of renaming files.

950 (edited by dawkinscm 06-11-2023 11:24:57)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

It's been a while since I properly looked at the Python wrapper code. Looking at your post reminded me of how I handled the v2 code previously and not the "hack" I did this time.

I also added the lines below to the Python script (replacing the temp dir conditional) but it didn't work.

cws wrote:

You can update `vsmlrt.py` by getting a copy from the `AmusementClub/vs-mlrt` repo, then modifying:

    alter_engine_path = os.path.join(
        os.path.expandvars("%APPDATA%\\SVP4\\cache\\"),
        os.path.splitdrive(engine_path)[1][1:]
    )

I had this issue before but I can't remember how I fixed it. But since it makes no obvious difference to how Rife works I'll probably just leave it as it is for now.