1,951 (edited by dawkinscm 22-09-2024 07:04:24)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

oriento wrote:

the onnx file is available now

Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.

1,952 (edited by dlr5668 22-09-2024 07:38:33)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RIFE 4.6 (very old model, fastest) - 112 fps
RIFE 4.15 lite (decent progress, exploring lite models) - 100 fps
RIFE 4.26 (finished trained 4.25) - 86 fps
RIFE 4.25 (very good breakthrough after 4.18) - 84 fps
RIFE 4.18 - 75 fps

I did speed test (with old SVP build)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

How does 4.26 work for you with no changes?

1,954

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:
oriento wrote:

the onnx file is available now

Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.

i suppose you must change 32 into 64

multiple_frac = 32 / Fraction(scale)

1,955 (edited by dawkinscm 22-09-2024 08:42:47)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

oriento wrote:
dawkinscm wrote:
oriento wrote:

the onnx file is available now

Yep. But it does not work properly with SVP. It might look like it is working but the engine parse crashes. Looking at the new vsmlrt script there are a lot of changes and I'm not sure which ones are relevant. Especially since SVP moved some functionality into helpers.py.

i suppose you must change 32 into 64

multiple_frac = 32 / Fraction(scale)

Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.

1,956

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

How does 4.26 work for you with no changes?

I didnt update yet

https://i.imgur.com/5bUIqJ4.png

1,957 (edited by dawkinscm 22-09-2024 09:18:24)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:
dawkinscm wrote:

How does 4.26 work for you with no changes?

I didnt update yet

https://i.imgur.com/5bUIqJ4.png

If the engine was created in less than 15 seconds then it crashed. It looks like it's working. SVP will tell you that it's working. But it is not coz there's an error log generated. When it works, no error logs are generated. BTW I did update but it makes no difference because the version number doesn't change.

1,958 (edited by flowreen91 22-09-2024 10:14:04)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:

Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.

It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:

[   1.419][f][vapoursynth] Script evaluation failed:
[   1.419][f][vapoursynth] Python exception: vsmlrt.RIFEMerge: tile size must be divisible by 64 (416, 256)
[   1.419][f][vapoursynth] 
[   1.419][f][vapoursynth] Traceback (most recent call last):
[   1.419][f][vapoursynth]   File "src\\cython\\vapoursynth.pyx", line 3121, in vapoursynth._vpy_evaluate
[   1.419][f][vapoursynth]   File "src\\cython\\vapoursynth.pyx", line 3122, in vapoursynth._vpy_evaluate
[   1.419][f][vapoursynth]   File "C:\Users\Flowr\AppData\Roaming\SVP4\scripts\1c2e810c.py", line 83, in <module>
[   1.419][f][vapoursynth]     smooth =  interpolate(clip)
[   1.419][f][vapoursynth]               ^^^^^^^^^^^^^^^^^
[   1.419][f][vapoursynth]   File "C:\Users\Flowr\AppData\Roaming\SVP4\scripts\1c2e810c.py", line 62, in interpolate
[   1.419][f][vapoursynth]     smooth = RIFE_imp(input_rife,multi=rife_num,model=rife_mnum,backend=trt_backend)
[   1.419][f][vapoursynth]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[   1.419][f][vapoursynth]   File "C:\Program Files (x86)\SVP 4\rife\helpers.py", line 23, in RIFE_imp
[   1.419][f][vapoursynth]     return RIFE(clip,multi,1.0,None,None,None,model_num,backend,ensemble,False,implementation)
[   1.419][f][vapoursynth]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[   1.419][f][vapoursynth]   File "C:\Program Files (x86)\SVP 4\rife\vsmlrt.py", line 1215, in RIFE
[   1.419][f][vapoursynth]     output0 = RIFEMerge(
[   1.419][f][vapoursynth]               ^^^^^^^^^^
[   1.419][f][vapoursynth]   File "C:\Program Files (x86)\SVP 4\rife\vsmlrt.py", line 1049, in RIFEMerge
[   1.419][f][vapoursynth]     raise ValueError(
[   1.419][f][vapoursynth] ValueError: vsmlrt.RIFEMerge: tile size must be divisible by 64 (416, 256)

So we also need to change the code from "C:\Program Files (x86)\SVP 4\script\generate.js" from:

    var pw = Math.floor((media.dst_w-1)/32+1)*32 - media.dst_w;
    var ph = Math.floor((media.dst_h-1)/32+1)*32 - media.dst_h;

to

    var pw = Math.floor((media.dst_w-1)/64+1)*64 - media.dst_w;
    var ph = Math.floor((media.dst_h-1)/64+1)*64 - media.dst_h;

Attaching files that load both v1 and v2 correctly below.

Post's attachments

generate.js 24.12 kb, 54 downloads since 2024-09-22 

helpers.py 717 b, 48 downloads since 2024-09-22 

vsmlrt.py 102.45 kb, 65 downloads since 2024-09-22 

1,959 (edited by dawkinscm 22-09-2024 10:25:41)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:
dawkinscm wrote:

Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.

It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:
...
So we also need to change the code from "C:\Program Files (x86)\SVP 4\script\generate.js" from:

    var pw = Math.floor((media.dst_w-1)/32+1)*32 - media.dst_w;
    var ph = Math.floor((media.dst_h-1)/32+1)*32 - media.dst_h;

to

    var pw = Math.floor((media.dst_w-1)/64+1)*64 - media.dst_w;
    var ph = Math.floor((media.dst_h-1)/64+1)*64 - media.dst_h;

Attaching files that load both v1 and v2 correctly below.

I got nowhere close in figuring this out lol. I will have to take a look at the changes but in the mean time: Damn good job buddy smile

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

flowreen91 wrote:
dawkinscm wrote:

Nope. Still crashes out. I've got the TRT error log and it says that broadcast dimensions must be comformable.

It seems 4.26 also crashes when the width or height of the resized video is not divisible by 64 with:

Attaching files that load both v1 and v2 correctly below.

Just a quick update. I don't like the way v4.26 and the new code is performing. v4.25 with the old code works better. I'm just double checking to make sure.

1,961

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Wow, 4.26 has almost no interference artifacts.

1,962

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Xenocyde wrote:

Wow, 4.26 has almost no interference artifacts.

And its faster than 4.25. Fucking magic

1,963 (edited by dawkinscm 22-09-2024 11:35:02)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

The artefacts I saw before v4.26 are still there but improved to the same level as v4.25. But when I tested DR Strange 2 with v4.26 with the new code, it makes IC behave identically to SVP and NVOF with intermediate Rife frames being created.

Updates:
So far using SVP motion detection for SC seems to work very well with v4.26 and the mod code. NVOF is very similar. What is a little annoying is that you can run the same scene multiple times and see varying degrees of the same artefact.

I can also forward and reverse some scenes when using SVP that previous got stuck.

1,964 (edited by Blackfyre 22-09-2024 13:30:02)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Firstly flowreen91, great work on the changed files.

I downloaded and changed the default ones in rife and script folders.

.

dlr5668 wrote:

RIFE 4.6 (very old model, fastest) - 112 fps
RIFE 4.15 lite (decent progress, exploring lite models) - 100 fps
RIFE 4.26 (finished trained 4.25) - 86 fps
RIFE 4.25 (very good breakthrough after 4.18) - 84 fps
RIFE 4.18 - 75 fps

I did speed test (with old SVP build)

Thanks for this, great work.

Xenocyde wrote:

Wow, 4.26 has almost no interference artifacts.

4.25 v2 remains the best one for artifacts with my testing.

4.26 v2 failed the No Time to Die test at around minute 3 with the sliding glass door (most difficult scene I test). In fact it was the worst I have seen during that specific scene, but in other scenes it performed better than others.

I have reverted back to 4.25 v2 personally. Interesting that after it finished training (became 4.25), artifacts became worse.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

From the vsmlrt script dev about the issue with SVP using v4.26:

It's not a problem with the official RIFE repository. I chose 32 to be the minimum requirement for the RIFE wrapper in vs-mlrt, and then the SVP developer adapted this requirement. The v4.26 model now requires mod64 input, and I have updated the code to explicit notify the change, and therefore it is the SVP developer that should update the code.

(this will only affect the "v1" variants, the "v2" variants handle this kinds of resolution requirement automatically

1,966 (edited by Drakko01 22-09-2024 16:58:05)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

For me 4.26 works properly without change any files, I use V1 models but  I test V2 only to make shure,and work properly too.

1,967 (edited by RickyAstle98 24-09-2024 09:32:33)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

1,968

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

I forgot that I used 2x2 all the time

How do you do that? I use only v2 models (for better rewind speed) and wanna try. How does it affect model performance?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.

1,970 (edited by RickyAstle98 24-09-2024 11:02:05)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dawkinscm wrote:
RickyAstle98 wrote:

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.

The 4.26 lite? I dont see 4.26 lite model, where is it? Youre wrong!

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:
dawkinscm wrote:
RickyAstle98 wrote:

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

Are you talking about 4.26 lite only because the lite models do behave differently to the normal ones.

I'm not exactly right or wrong. I was just suggesting that there is a difference between a lite model. But yes you are
The 4.26 lite? I dont see 4.26 lite model, where is it? Youre wrong!

I've seen this happen before but if there is no lite model then this is not the case here.

1,972

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

For testing what you stated, can you provide or indicate the exact scene.thanks

1,973 (edited by RickyAstle98 24-09-2024 17:11:31)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Drakko01 wrote:
RickyAstle98 wrote:

Whats the default align/valign values for SVP4 Pro? I forgot that I used 2x2 all the time, maybe thats why all models starting 4.16 looks fragmentary smooth!

Fragmentary smooth - when a small parts of one large object can be interpolated differently, even being a same large object! For example: faces, 4.4 to 4.16 faces looks buttery smooth, from 4.17 to 4.26 small parts like lips, cheeks, ears and other objects can be separated from the frame! There were cases when the lips moved apart alone, never happens for 4.4 to 4.16 before, also happens when the main object in the frame is interpolated correctly, but all the background moving both axis, looks like one step forward-two steps back frame!

The 4.26 new model is worse than 4.15 lite in Matrix rooftop 3 agents opening the doors, pattern errors on door textures, no door errors in 4.15v2 (both mod32 and mod64 input)!

Too noticeable door errors 4.4 (worst)
Less noticeable door errors 4.9 (okay)
No door errors 4.11 to 4.16 (perfect)

For testing what you stated, can you provide or indicate the exact scene.thanks

Nevermind, just say default align/valign values, thanks! The rendered results will look different on different PCs!

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

dlr5668 wrote:
RickyAstle98 wrote:

I forgot that I used 2x2 all the time

How do you do that? I use only v2 models (for better rewind speed) and wanna try. How does it affect model performance?

What do you mean?

1,975 (edited by flowreen91 24-09-2024 17:17:13)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

RickyAstle98 wrote:

Nevermind, just say default align/valign values, thanks!

Renaming the SVP folder from Roaming, generates a fresh config that looks like this:
https://gyazo.com/d8f75344daafabb8ecdcaeaecc2db2d3