76 (edited by UHD 23-10-2021 14:54:17)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I'm still looking for someone with an NVIDIA GPU and some time to test real-time interpolation using SVP with RIFE filter for VapourSynth (PyTorch version):

Chainik wrote:

in the RIFE video profile set
- TTA = disabled -> use RFIE/ncnn/Vulkan
- TTA = enabled -> use RIFE/torch/CUDA.

https://www.svp-team.com/forum/viewtopi … 023#p79023

I'm still looking, as so far only dlr5668 has confirmed that he has successfully done this, but unfortunately it freezes during rewind:
https://www.svp-team.com/forum/viewtopi … 319#p79319

However, I have not been able to find out if this is a permanent or temporary situation. Also, I was unable to find out if it freezes on pause and when resuming playback. The latter is particularly important to me, because if I want to watch something with the best real-time interpolation on PC that RIFE provides, without the possibility of pausing and resuming playback this solution would be doomed to failure for me.

If anyone is wondering why I don't check it myself, the matter is trivially simple: I don't have a modern NVIDIA GPU, because I didn't need it for anything. It is the RIFE interpolation algorithm based on machine learning (artificial intelligence) that could make me purchase such a card and a large expenditure would be justified.

This is why I keep bumping this thread - to find out if smooth real-time interpolation along with pause and resuming playback is possible using SVP with RIFE filter for VapourSynth (PyTorch version).

77 (edited by egandt 26-10-2021 01:18:23)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

When I try to use RIFE following the instructions that the previous posts references post I get:

20:55:20.530 [E]: Playback [3d8c3fef]: VS - Python exception: Ran out of input
20:55:20.530 [E]: Playback [3d8c3fef]: VS - Traceback (most recent call last):
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \src\cython\vapoursynth.pyx\, line 2244, in vapoursynth.vpy_evaluateScript
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \src\cython\vapoursynth.pyx\, line 2245, in vapoursynth.vpy_evaluateScript
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Users\htpc user\AppData\Roaming\SVP4\scripts\3d8c3fef.py\, line 70, in <module>
20:55:20.530 [E]: Playback [3d8c3fef]: VS - smooth = interpolate(clip)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Users\htpc user\AppData\Roaming\SVP4\scripts\3d8c3fef.py\, line 51, in interpolate
20:55:20.530 [E]: Playback [3d8c3fef]: VS - smooth = RIFE(input_m,model_ver=3.8,fp16=not rife_precision,scale=1.0 if rife_threads<=2 else 0.5,device_index=rife_gpu)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Program Files (x86)\SVP 4\mpv64\lib\site-packages\vsrife\__init__.py\, line 77, in RIFE
20:55:20.530 [E]: Playback [3d8c3fef]: VS - model.load_model(os.path.join(os.path.dirname(__file__), model_dir), -1)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Program Files (x86)\SVP 4\mpv64\lib\site-packages\vsrife\model38\RIFE_HDv3.py\, line 40, in load_model
20:55:20.530 [E]: Playback [3d8c3fef]: VS - self.flownet.load_state_dict(convert(torch.load('{}/flownet.pkl'.format(path))), False)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Program Files (x86)\SVP 4\mpv64\lib\site-packages\torch\serialization.py\, line 608, in load
20:55:20.530 [E]: Playback [3d8c3fef]: VS - return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - File \C:\Program Files (x86)\SVP 4\mpv64\lib\site-packages\torch\serialization.py\, line 777, in _legacy_load
20:55:20.530 [E]: Playback [3d8c3fef]: VS - magic_number = pickle_module.load(f, **pickle_load_args)
20:55:20.530 [E]: Playback [3d8c3fef]: VS - EOFError: Ran out of input

This then crashes the media player, after playing sound for say 10 second, video just hangs as soon as enabled.


I also tried with Vulkan, so set Test-Time Augmentaion: Disabled
This does not hang, but is unusable in real-time I get a 10 or so frames a second at best as is expected.

21:12:59.698 [i]: Playback [91b99dbe]: playing at 59.94 [23.976 *5/2] 
21:13:25.245 [i]: Playback [91b99dbe]: switching to profile 'RIFE AI engine'
21:13:25.247 [i]: Playback [91b99dbe]: playing at 47.952 [23.976 *2/1] 
21:13:39.714 [i]: Playback [91b99dbe]: source frame rate changed to 8.181
21:13:39.715 [i]: Playback [91b99dbe]: 2 acceptible profiles, best is 'Anime_SubsPlease' [100]
21:13:39.717 [i]: Playback [91b99dbe]: keep previously selected profile 'RIFE AI engine'
21:15:40.833 [i]: AVSF: player closed
21:15:40.836 [i]: Playback [91b99dbe]: disabled while video is stopped
21:15:40.836 [i]: Playback [91b99dbe]: deleted

ERIC

78

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

ERIC, thanks for the tests.

Could you elaborate on what GPU you have and try again to run the tests on a very light video file:

240p (320x240), 15FPS, 6 min 8 sec, 4:2:0 YUV, 8 bits, 298 kb/s (video)   
https://sample-videos.com/video123/mp4/ … p_30mb.mp4

I would also encourage others who might be able to conduct tests to start with the above test file. This file will allow you to test real-time interpolation using the RIFE algorithm based on machine learning (artificial intelligence) on both the PyTorch version and the ncnn version on most graphics cards.

79 (edited by egandt 27-10-2021 01:41:02)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Well I have not gotten very far as you can see I get some script failure using PyTorch based RIFE

As for specs: I'm trying on a AMD 3900x,  64GB of Memory and a 3080 TI card, so I should have the horse power to run RIFE if anything can.

I did try that video quickly using Vulkan; I had no issue with that GPU load is around 20%, will try to get some better data when I have some real  time to play more with RIFE.  Fully agree real-time SVP with RIFE would be a game changer, which is why I'm giving it a shot.

ERIC

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

So is there any way to get this on the linux version of SVP? when I go to add programs and features with svp4-manager (or whatever its called i forget) it doesn't show up. Is RIFE only for the paid versions at the moment?

81 (edited by UHD 28-10-2021 00:21:15)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

egandt wrote:

As for specs: I'm trying on a AMD 3900x, 64GB of Memory and a 3080 TI card

Great! With this graphics card theoretically it should be possible to interpolate 720p video in real time using SVP with RIFE filter for VapourSynth (PyTorch version) without any problem!


egandt wrote:

I did try that video quickly using Vulkan; I had no issue with that GPU load is around 20%

If we can find a solution for script failure using PyTorch based RIFE, this smaller 240p 15FPS file can be used to test x4 real-time interpolation with RIFE!


egandt wrote:

Fully agree real-time SVP with RIFE would be a game changer, which is why I'm giving it a shot.

That being said, I hope you don't give up and keep actively contributing to this thread until a solution can be found. Since dlr5668 was successful: https://www.svp-team.com/forum/viewtopi … 315#p79315 there must be some way to make it work on the 3080 TI too!

82

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

egandt wrote:

Well I have not gotten very far as you can see I get some script failure using PyTorch based RIFE

Chainik, can we please ask you to take a look at the script error https://www.svp-team.com/forum/viewtopi … 458#p79458 with your expert eye?

83 (edited by lwk7454 28-10-2021 02:59:07)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Hi guys, glad I found there are people working on real-time playback with RIFE. I've tried this method myself, hope I'm adding more valuable data to your development. Throwing my specs first:

CPU: Ryzen 7 5800X
GPU: NVIDIA GeForce RTX 3090 24GB
Memory: DDR4 4x16GB (64GB)
System: Windows 11

After some troubleshooting I got my VLC player to work with VS RIFE.
Straight to conclusion: it works perfectly in 720p, but laggy and complicated in 1080p.

Best result's configs I used:
Math precision: FP16
GPU threads: 6

Main take away here is that my GPU was poorly utilized.
In 720p, GPU utilization was about 28%. Using scale=1.0 (i.e. GPU thread=2) it goes up to 48%.
In 1080p GPU utilization was very unstable, when output is smooth it stays at around 60%, but drops to single digits quickly, and frames were droppepd. The result is a very jaggy GPU graph and laggy playback. I feel there are lots of space for optimization, or is my system not configured correctly? Anyway, in terms of raw performance I believe it's achievable to have a smooth playback in 1080p.

Another thing to note is that it ran very stable, I could do everything with the playback, like pause and skip forward/backward. Switching the profile was only a short freeze of 1s.

All video files I tried were 24fps anime videos. To my surprise the output image was not satisfactory to me, artifacts were terrible on texts, and interpolation did not seem much better than default SVP method.

Regarding to script errors:

UHD wrote:
egandt wrote:

Well I have not gotten very far as you can see I get some script failure using PyTorch based RIFE

Chainik, can we please ask you to take a look at the script error https://www.svp-team.com/forum/viewtopi … 458#p79458 with your expert eye?

I got the same error. I looked at the log and figured it was reading an empty file in vs-rife library, therefore the EOF error. Please bear with me that I don't know much about the technologies used here, it seems pytourch is reading this .pkl file below for AI inference perhaps?
SVP 4\mpv64\Lib\site-packages\vsrife\model38\flownet.pkl

I found vs-rife project is providing these files separately from the source code, so I downloaded them and put them into the Lib directory, then everything worked:
https://github.com/HolyWu/vs-rife/releases/tag/model38
Just download the source code zip and overwrite correponding folders and files.
I bet there is a better way to install them with pip, but it's not my profession here, guess Chainik could provide the command.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

I thought "pip install vsrife" is enough. And it used to be enough in August. Maybe they changed something in packaging...

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik, is there a way to get the rife beta for svp on linux using the add programs and features tool or is this only for paid versions of SVP?

86 (edited by UHD 28-10-2021 23:08:56)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

lwk7454 wrote:

Hi guys, glad I found there are people working on real-time playback with RIFE. I've tried this method myself, hope I'm adding more valuable data to your development.

I am also glad that another person took an interest in this topic and added something from himself, which I think will help many other people including me.

I have a request to you: could you please stay with us for a while on this thread and help us test real-time playback with RIFE?

I have read your post thoroughly and will be returning to individual parts in the near future. However, I don't want to write about everything at once as it will be a bit overwhelming, so I will do it gradually.

Firstly, a big thank you for finding a solution to the script problem. I hope this helps egandt and encourages others to test.

Right now I think the most important thing is to get as many people interested in the topic as possible to replicate your path to success with RIFE

My first question:

lwk7454 wrote:

After some troubleshooting I got my VLC player to work with VS RIFE.

Could you please elaborate on what these problems were and how you dealt with them? That might help a lot of people.

Now I will have a request for you to do some simple tests to help us determine the causes of the image freezing that have been described on this thread.

Test parameters:

SVP+VLC media player+RIFE filter for VapourSynth (PyTorch)
real time playback with x2 interpolation:
RIFE model: 3.8
scale=1.0
FP32 or FP16 (your preferred choice)
GPU threads: (your preferred choice)
Test file:
original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing
Monitor set to 50Hz or 100Hz or 150Hz or 200Hz

Test file only if image freezes more than 30s on 720p file:
240p (320x240), 15FPS, 6 min 8 sec, 4:2:0 YUV, 8 bits
https://sample-videos.com/video123/mp4/ … p_30mb.mp4

Test variables based on dlr5668 postings
https://www.svp-team.com/forum/viewtopi … 315#p79315
https://www.svp-team.com/forum/viewtopi … 321#p79321

1. disable black bar detection
2. enable black bar detection
3. switch profile in windowed mode
4. switch profile in full screen mode
5. gpu-api=vulkan
6. gpu-api=d3d11

Testing: does the image freeze and for how long (30s = permanent) during:

A. real-time playback with RIFE - first-time initialization
B. real-time playback with RIFE - after pause
C. real-time playback with RIFE - after rewind
B. real-time playback with RIFE - after fast forward

I would be very grateful if you could find some time to run these tests.

87 (edited by UHD 28-10-2021 23:19:37)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

I thought "pip install vsrife" is enough. And it used to be enough in August. Maybe they changed something in packaging...

Yes, HolyWu has made changes with version 1.3.0.
The problem and solution is described here: https://github.com/HolyWu/vs-rife/issues/6
And here the recently added 3 lines of code in the file __init__.py : https://github.com/HolyWu/vs-rife/commi … 86856aabc0

Can you please update the files for SVP so that the models download automatically?

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

there're no "files for SVP" regarding this

> run 'python -m vsrife' first

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

I have a request to you: could you please stay with us for a while on this thread and help us test real-time playback with RIFE?

Sure, I'm happy to test this out and follow this thread.

For the troubleshooting it was really just the script error, nothing else.

Not sure how to configure vulkan vs d3d11 in VLC, I don't have the config file mentioned in previous posts (SVP 4\mpv64\mpv conf). I suppose it's TTA on or off.
Otherwise I've tried all the parameters you mentioned. Some notes:
I edited base.py that fixed scale to 1.0 regardless of GPU threads.
Following parameters did not affect the results at all, I've tried all combinations with both 720p and 240p videos:
GPU threads
Screen refresh rate. I tried 50Hz and 60Hz
VLC windowed and fullscreen mode
Black bar detection. Switching to this tab causes a 2s freeze, but clicking any options in this tab did not have any effects to the video.

Other values:
RIFE model = 3.8
Scale = 1.0
Math precision = FP16

Following are results:

With the sample 720p file:
TTA on: GPU stables at round 46%, playback is smooth at 50Hz. Switching to this profile from automatic has 3s freeze. Resuming pause will cause a 1s frame drop + freeze. Other actions are fluid without freeze.
TTA off: Lags every second, about half of the frames are dropped, GPU jumps between 1% to 100% in a very consistent manner.

Sample 240p file:
TTA on: GPU stables at around 8%, playback is smooth at 30Hz.
TTA off: GPU stables at around 13%, playback is smooth at 30Hz.

Interestingly when TTA is on, FP32 never worked, it freezes video but playback continues with audio, GPU stays idle.

Hope above data gives you a clear idea how it performed.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Well if anyone gets the rife options to show up on a linux build I've got a 5950x and a 3090 ready to test on mpv.

91

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

seal wrote:

Well if anyone gets the rife options to show up on a linux build I've got a 5950x and a 3090 ready to test on mpv.

I don't use Linux myself, nor am I familiar with it, but I agree with your request. The more testers the better.

92

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Chainik wrote:

there're no "files for SVP" regarding this

> run 'python -m vsrife' first


The idea is that a random tester who comes in here https://www.svp-team.com/wiki/RIFE_AI_interpolation
doesn't have the problem egandt mentioned.

Someone who reads this thread will probably use the simple solution that lwk7454 presented, but probably not everyone will read all the pages of this thread.

93 (edited by UHD 29-10-2021 22:38:34)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

lwk7454 wrote:

Sure, I'm happy to test this out and follow this thread.

Thank you very much for the testing I requested and thank you very much for your willingness to continue to help on this thread.

I have read your post and the test results very carefully. I will come back to certain aspects of these results, but for now I am bothered by a few points:


1.

lwk7454 wrote:

I edited base.py that fixed scale to 1.0 regardless of GPU threads.

Did you make this change only before the tests I asked you to do, or before when you wrote in the previous post:

lwk7454 wrote:

Best result's configs I used:
Math precision: FP16
GPU threads: 6

Main take away here is that my GPU was poorly utilized.
In 720p, GPU utilization was about 28%. Using scale=1.0 (i.e. GPU thread=2) it goes up to 48%.

If I understand, by setting "GPU threads: 6" early on you were actually setting scale=0.5, according to this post: https://www.svp-team.com/forum/viewtopi … 183#p79183
which obviously doesn't do the quality any good: https://www.svp-team.com/forum/viewtopi … 250#p79250

2.

Can you share a fragment of the above code that you modified in base.py ?

3.

Request for a short test: to see if changing the GPU threads parameter changes anything in performance after the change in base.py you made.

lwk7454 wrote:

I edited base.py that fixed scale to 1.0 regardless of GPU threads.

Test A:

Test-Time Augmentation: Enabled [sets RIFE filter for VapourSynth (PyTorch)]
AI model: Generic [with above PyTorch version it doesn't matter, because it is set in the 2nd version of base.py by Chainik permanently to model 3.8]
Math precision: Half (FP16)
GPU threads: 2
Test file:
original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing

Test B:

Same, except for one parameter:
GPU threads: 6

Please present the performance results as Chainik did in this post: https://www.svp-team.com/forum/viewtopi … 465#p78465

I think such a scheme with 3 parameters will be the best to present these and future results:

Test A
GPU: XX% (XX% Compute_1)
VRAM: X.X GB
SVP index: X.X

Of course, this could be "Compute_1" "Compute_0", "Cuda" or another virtual node, loaded by RIFE.This most accurately shows the actual utilization of GPU computing power by RIFE.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

Still unable to do anything even using the 240p sample:

17:37:35.138 [i]: AVSF: player closed
17:37:35.150 [i]: Playback [dee418a3]: disabled while video is stopped
17:37:35.153 [i]: Playback [dee418a3]: deleted
17:37:50.068 [i]: AVSF: found new player instance
17:37:50.087 [i]: AVSF: filters in use: LAV Splitter Source -> LAV Video Decoder -> * -> Madshi Video Renderer
17:37:50.088 [i]: AVSF: new video in PotPlayerMini64.exe (64-bit) [PotPlayer 0.0.0.0] on screen 1
17:37:50.173 [i]: Media: video 320x240 [PAR 1.000] at 15.000 fps
17:37:50.173 [i]: Media: codec type is AVC, YUV/4:2:0/8 bits
17:37:50.174 [i]: Playback: starting up...
17:37:50.178 [i]: Playback [3d06215d]: Frame server (64-bit) C:\Program Files (x86)\SVP 4\mpv64\vapoursynth.dll
17:37:50.179 [i]: Playback [3d06215d]: resulting video frame 320x240
17:37:50.179 [i]: Playback [3d06215d]: 1 acceptible profiles, best is 'Automatic' [0]
17:37:50.181 [i]: Playback [3d06215d]: enabled while video is playing
17:37:50.182 [i]: Profile: using automatic profile #1 /film
17:37:50.184 [i]: Playback [3d06215d]: playing at 60 [15 *4/1] 
17:37:54.534 [i]: Playback [3d06215d]: switching to profile 'RIFE AI engine'
17:37:54.535 [i]: Playback [3d06215d]: playing at 30 [15 *2/1] 
17:38:16.983 [i]: AVSF: player closed

As soon as I select RIFE video stops GPU goes to 0% and Audio plays until I  kill the player.  So while the command above fixed the script error it has done nothing to allowing PyTorch to work. So maybe one step closer.

ERIC

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

egandt wrote:

Still unable to do anything even using the 240p sample

Hi Eric, could you share what parameters you used? It appears exactly the same behavior when I use FP32 with PyTorch. Please try adjust the parameters, you may find a working set.

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

Thank you very much for the testing I requested and thank you very much for your willingness to continue to help on this thread.

I have read your post and the test results very carefully. I will come back to certain aspects of these results, but for now I am bothered by a few points

No problem smile

I only fixed the scale to 1.0 since your request, i.e. after my first post. Here's the changed line 23 in base.py:

smooth = RIFE(input_m,model_ver=3.8,fp16=not rife_precision,scale=1.0,device_index=rife_gpu)

Though as I said before GPU threads was one of the parameters did not affect results, I tested them again, here are the results:

Video: 720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits

Test A (GPU threads = 2)
GPU: ~47%
VRAM: 4.57 GB
SVP index: 1.00

Test B (GPU threads = 6)
GPU: ~49%
VRAM: 4.59 GB
SVP index: 1.00

I'm not sure what are the virtual nodes you are referring to? I'm always using the same GPU device. Or could you suggest a tool that monitors those virtual nodes separately?
I know it's a bit odd but it seem this parameter doesn't matter...

97 (edited by UHD 30-10-2021 16:40:43)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

About virtual nodes:

GPU workloads are executed on GPU hardware engines. On Windows, these engines are represented by "nodes". With Hardware Scheduling disabled for Windows 10 May 2020 Update, some NVIDIA GPU engines are represented by virtual nodes, and multiple virtual nodes may represent more than one GPU hardware engine. This is done to achieve better parallel execution of workloads. Examples of these virtual nodes are "Cuda", "Compute_0", "Compute_1", and "Graphics_1" as shown in Windows Task Manager. These correspond to the same underlying hardware engines as the "3D" node in Windows Task Manager. With Hardware Scheduling enabled, the virtual nodes are no longer needed, and Task Manager shows only the "3D "node for the previous "3D" node and multiple virtual nodes shown before, combined. CUDA is still supported in this scenario.

NVIDIA CUDA TOOLKIT 11.0.3 https://docplayer.net/192620757-Nvidia- … olkit.html


In Windows Task Manager there should be 4 graphs and the content of each graph should be changeable from the drop down list:

https://static.nicehash.com/marketing%2FCompute_task_manager.png

In the quote below, Chainik has shown the load on his NVIDIA GeForce RTX 2060 Mobile graphics card with real-time interpolation using RIFE filter for VapourSynth (ncnn).

The first result shows how the graphics card handles the video file without issue (SVP index: 1.0) with a real-time load of 65% indicated by Compute_1.

The second result shows how the graphics card can't keep up with real-time interpolation (SVP index: 0.76) as indicated by Compute_1's full 100% saturation, despite the total load surprisingly dropping to single digits.

Chainik wrote:

GPU; VRAM; CPU utilization; SVP index

424*240 - 23% (65% compute_1); 0.9 GB; zero; 1.0
640*360 - 9% (100% compute_1); 0.9 GB; zero; 0.76


Notice how this is consistent with what you wrote earlier:

lwk7454 wrote:

In 1080p GPU utilization was very unstable, when output is smooth it stays at around 60%, but drops to single digits quickly, and frames were droppepd. The result is a very jaggy GPU graph and laggy playback. I feel there are lots of space for optimization, or is my system not configured correctly?

Of course with you NVIDIA GeForce RTX 3090 testing the RIFE filter for VapourSynth (PyTorch) it doesn't have to be Compute_1 at all, it could be a completely different node, or a combination of several. Likewise, changing FP32 to FP16 can also result in load on other nodes. No one has tested or described this. You may be the first and it may interest many people. Me for sure smile Especially since at the moment there is no more powerful graphics card than RTX 3090 for a mere mortal.

98 (edited by UHD 30-10-2021 22:11:10)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

lwk7454 wrote:

I know it's a bit odd but it seems this parameter doesn't matter...

Thanks for answering my questions and for the new tests.

Now everything is starting to come together logically and I'm slowly starting to understand almost everything.

I'll describe it, because someone might find it useful, although it's probably obvious to many.

I checked and the phrase "thread":
occurs only in the RIFE filter for VapourSynth (ncnn) code
https://github.com/HomeOfVapourSynthEvo … h?q=thread

does not appear in the RIFE filter for VapourSynth (PyTorch) code at all
https://github.com/HolyWu/vs-rife/search?q=thread

In other words, your change of code:

smooth = RIFE(input_m,model_ver=3.8,fp16=not rife_precision,scale=1.0,device_index=rife_gpu)

and the tests confirmed 2 things:
1. indeed the code change fixed scale=1 permanently
2. the "GPU threads" does not change anything for the RIFE filter for VapourSynth (PyTorch)

For people who will want to follow your path a small summary:

1. manual change of the code to the one shown above.
2. RIFE options:
Test-Time Augmentation: Enabled [sets RIFE filter for VapourSynth (PyTorch)]
Math precision: Half (FP16)

99 (edited by UHD 30-10-2021 23:57:00)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

lwk7454, I think now it would be worth to check again what is the actual GPU load (most loaded nodes) with the already tested reference 720p file at FP16 and if it reaches 100% nodes saturation for FP32.

Please check also 1080p file, which was already tested earlier on this thread if 100% nodes saturation is reached with it.

source.mkv
Video: 1080p (1920x1080), 23.976 FPS, 35 s 994 ms, 4:2:0 YUV, 8 bits
https://www.playbook.com/s/vadash/wcVXb … Ky5jaQMkLs

If you have a link to any other interesting test demo for interpolation with typical 720p or 1080p, 23.976 FPS, 8 bits that anyone today and in the future can download for comparison testing then let us know.

Fixed test parameters:

Test-Time Augmentation: Enabled [sets RIFE filter for VapourSynth (PyTorch)]
real time playback with x2 interpolation
RIFE model: 3.8
scale=1.0

Variable test parameters:

Math precision: FP16 and FP32
Video files: 720p and 1080p

Test results:
% GPU utilisation - most loaded GPU nodes

The results can be compared to other existing graphics cards, future graphics cards and future better and more efficient RIFE models!

100 (edited by lwk7454 31-10-2021 09:54:09)

Re: New RIFE filter - 3x faster AI interpolation possible in SVP!!!

UHD wrote:

About virtual nodes:

Thanks for explanation, I see how it works.
My windows 11 had hardware scheduling enabled by default, so all nodes were merged into 3D. I managed to turn it off so I could see Graphics_1, Compute_0, Compute_1 and CUDA virtual nodes.

I've tested all combinations of 720p, 1080p, FP16 and FP32:

FP16 720p: Cuda ~40%, SVP index 1.0
FP16 1080p: Cuda jumps between 35% - 51%, SVP index N/A

FP32 did not work at all, images were all frozen and all nodes were at 0%.

Turns out only Cuda node was used.