It looks like there is a problem and a serious one at that. I guess I can't help you any more than to advise you to install the GPU drivers cleanly and reinstall SVP from scratch as well.

3070 Ti:
https://www.svp-team.com/forum/viewtopi … 819#p81819

DragonicPrime wrote:

I just tried it myself and I can't do x3 either with a 4090. I only have DDR4 ram at 3800MHZ, but it doesn't seem to be the problem. On that test video, at 3x interpolation, my 4090 is maxed out at 100% usage. at 2x with the recommended settings, it's at around 70% usage

This is puzzling. 3.8x more power and the GPU load drops to only 70% compared to 90% for the 3070 Ti. If you've applied the tips above and still nothing, then I guess you'll have to compare step by step with those who have managed to interpolate x3 in real time.

Same video file - the LG demo I suggested and encoding to start with. Write how many FPS you have for RIFE interpolation. We will compare the FPS of the encoding first. In the next post I will give further, my ideas and we will see how they will affect the FPS.

Thanks for the test aloola smile

Please write what kind of CPU you have and at what frequency the RAM is running.

Knowing the performance of graphics cards:

Fourth-generation Tensor Cores - Peak FP16 using the Sparsity feature:

660.6 TFLOPS - NVIDIA GeForce RTX 4090
https://images.nvidia.com/aem-dam/Solut … ecture.pdf

Third-Generation Tensor Cores - Peak FP16 using the Sparsity feature:

174 TFLOPS - NVIDIA GeForce RTX 3070 Ti
https://www.anandtech.com/show/17204/nv … more-money


NVIDIA GeForce RTX 4090 has 3.80x more processing power!!!!!

This means that we should easily get x4 interpolation and be very close to x5 for 4K HDR. Clearly the bottleneck is not the performance of the Tensor Cores.

Mardon85 wrote:

I've filled in some clarifications above.

Thanks for the additional information.

175hz HDR
great, we have a big room to test

3440x1440
this probably shouldn't be a problem if scaling is done inside the monitor

Gsync
great, automatic adjustment

I NEED TO CHECK THIS AND REPORT BACK
OK

I use it on due to gaming on system also.
This means it would be worth checking if things improve with OFF. You can always easily go back to on

I scale everything down to by 21:9 screen so the black bars are completely gone and my entire screen is filled.
So the original file is full 4K. If any scaling is done in RAM then this can be an extra strain on RAM bandwidth. It would be good to have some 4K HDR file already in about 21:9 format for comparison that doesn't need to be scaled.

I NEED TO CHECK THIS AND REPORT BACK
OK

aloola wrote:

since we're using vs-mlrt maybe we should include RealESRGANv2 to upscale 480p~720p? or maybe for transcoding.
https://github.com/AmusementClub/vs-mlr … alESRGANv2

Soon we will have something better. Much better. Something that is much better suited to quality improvement for Video Frame Interpolation enthusiasts. I looked through literally hundreds of academic papers and found what I was looking for. It appeared literally a few months ago. As of yet there is no code for this paper, but the author assures that there will be. I will need the support of everyone on this forum to get this into vs-mlrt and then into SVP. The quality will be unattainable for RIFE even with RealESRGAN or any other SR method for live-action movies.

Give me a month until I sort out my repository on GitHub - I'll describe everything in detail based on these academic papers and this one most important one. For now, there is only a preprint and I think the author is waiting with publishing the code until the paper is accepted for publication. So we still have time.

Mardon85 wrote:

I have 6400mhz Dual Rank 64gb Sticks at netting just under 100Gb/s with a latency of 54Ns. Using MPV player i can't interpolate a 21:9 4k HDR film at 3x (13700k @ 5.6Ghz all core 4.5Ghz E cores).

You've been away from us for a month and a half. Nice to see you Mardon85 again smile

NVIDIA GeForce RTX 4090 graphics card and DDR5-6400 memory is an excellent set up to test the capabilities of RIFE. It's a bit puzzling that you can't achieve x3 interpolation, especially since if I understood one of your earlier posts correctly, for x2 interpolation the NVIDIA GeForce RTX 3090 graphics card was completely sufficient.

Mardon85 wrote:

BTW I was returning my watercooled 3090 back to stock today before selling on. I thought i'd give this latest version of Rife a go and it does run 4K HDR at 48FPS no issue.

So this isn't limited to 40 series cards. Power draw is low too, around 150w.

https://www.svp-team.com/forum/viewtopi … 686#p81686

Write more about the test parameters:

1 What is your maximum refresh rate of your monitor or TV set? 100Hz? 120Hz?
2 Are you using G-Sync or FreeSync automatic refresh rate change?
3. Do you use the same settings that earl088 used successfully for x3 interpolation?
https://www.svp-team.com/forum/viewtopi … 799#p81799
4. Do you use the "Hardware-accelerated GPU scheduling OFF" setting that aloola suggests for improved performance?
https://www.svp-team.com/forum/viewtopi … 819#p81819
5. Write something more about the 21:9 4k HDR movie you are testing. Is it a 3840x2160 23.976FPS file with black bars at the bottom and top or does it have a different resolution and FPS?
6. Using the file below, can't you interpolate in real time x3?
LG 4K HDR Demo - New York.ts.
File size: 448 MiB
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS.
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits.

Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

At the moment we need to find a way for you to be able to interpolate 4K HDR x3 in real time like others with NVIDIA GeForce RTX 4090 graphics cards.

If that works, then we'll try x4. I've got some new ideas for performance enhancements that haven't come up here yet for nowwink

Chainik wrote:

UHD

are you waiting for 5090 now?

I am waiting for Video Frame Interpolation enthusiasts who, like me, care about the highest possible interpolation quality. Those who believe that the growth of a community of people interested in Video Frame Interpolation based on Artificial Intelligence will also contribute to the development of interpolation methods and thus their quality.

When the first RIFE filter for VapourSynth appeared on 1 June 2021 the same day I asked about the possibility of implementing it into SVP: https://www.svp-team.com/forum/viewtopic.php?id=6196
Even your test that indicated 4fps for 1080p: https://www.svp-team.com/forum/viewtopi … 426#p78426 did not cool my enthusiasm. Even the lack of support at the time from others didn't faze me either. I know that these days most people expect ready-made solutions given on a platter.

4fps for 1080? Meh... next.

I, however, firmly believed at that time that real-time interpolation using Artificial Intelligence would be possible.

Today it is possible, even 4K HDR x3 in real time.

If the bottleneck in achieving 4K HDR x5 in real time is RAM then using quad-channel or octa-channel DDR5-6400 R-DIMM memory may help.

I am waiting for enthusiasts who has an NVIDIA GeForce RTX 4090 graphics card and at least DDR5-6000 memory who would like to help verify that the bottleneck is RAM.

One factor that might point to this are your own posts:

chainikdn wrote:

It's not about CPU load, it's about memory throughput. Increasing CPU threads may help.
Memory in a dual channel mode?

https://github.com/AmusementClub/vs-mlr … nt-4630846

chainikdn wrote:

Script-level / vapoursynth / mpv optimizations are out of scope here.

https://github.com/AmusementClub/vs-mlr … nt-4632772

chainikdn wrote:

Plus there're two more frames copying inside mpv - from mpv to vapoursynth and then back. Plus a copy-back video decoder, obviously.

https://github.com/AmusementClub/vs-mlr … nt-4633204

chainikdn wrote:

And I personally know how to make it even more gross and hacky big_smile and eliminate excessive frames copying back and forth, but I won't even make a PR for that, cause they will never accept it anyway....
But again, this is out of scope here.

https://github.com/AmusementClub/vs-mlr … nt-4634093

Chainik, if you could post here exactly how many times the video frames are copied back in RAM and what effect this might have on real-time 4K HDR x5 interpolation using RIFE then I would be very grateful.

If you have any idea how to reduce the number of copy operations in mpv and in vapoursynth then please post it in GitHub. I will support this idea and probably not only me. This is what I want to build a community of people interested in Artificial Intelligence in Video Frame Interpolation for.

This could eliminate the need for quad-channel or octa-channel DDR5-6400 R-DIMM memory or even NVIDIA GeForce RTX 5090!

4K 120Hz is standard on TV sets and monitors today so 4K HDR x5 interpolation could be very useful.

Who wants to interpolate 4K HDR videos x5 in real time with RIFE?
Perhaps a more appropriate question is who can afford it?

After reading all the posts on this thread I come to the conclusion that it is possible, and the limitation is not the NVIDIA GeForce RTX 4090 graphics card but the RAM bandwidth.

For someone with unlimited financial resources this is the solution:

ASUS Pro WS W790E-SAGE SE motherboard
Intel Xeon W9-3495X processor
G.SKILL DDR5-6400 CL32-39-39-102 octa-channel R-DIMM memory modules

https://www.gskill.com/img/pr/2023.02.23-zeta-r5-rdimm-ddr5-announce/06-zeta-r5-rdimm-spec-table-eng.png
Source: https://www.gskill.com/community/150223 … erformance

The result?

303.76GB/s read, 227.37 GB/s write, and 257.82 GB/s copy speed in the AIDA64 memory bandwidth benchmark, as seen in the screenshot below:

https://www.gskill.com/img/pr/2023.02.23-zeta-r5-rdimm-ddr5-announce/04-zeta-r5-rdimm-ddr5-6400-c32-16gbx8-bandwidth.png
Source: https://www.gskill.com/community/150223 … erformance

Of course, the Intel Xeon W9-3495X is completely out of my reach....

The cheapest unlocked Intel Xeon would be the W5-2455X at a suggested price of $1039: https://www.anandtech.com/show/18741/in … -5-0-lanes Should be enough. If the current dual-channel DDR5-6000 allows x3 interpolation then the quad-channel DDR5-6400 should be enough for x5 real-time interpolation.

I am looking for someone who has an NVIDIA GeForce RTX 4090 graphics card, at least DDR5-6000 memory and some spare time to test how RIFE real-time interpolation scales at different RAM speeds.

Alternatively, someone who would like to build an HTPC based on quad-channel or octa-channel DDR5-6400 R-DIMM memory. Octa-channel is for the 4K 240Hz screens that will soon go on sale big_smile

Thanks for all the replies to my posts and the test results you have provided here. I have some very interesting news for you now....

grobalt wrote:
UHD wrote:
Pezede wrote:

I've tried enabling and disabling and I'm not seeing any color differences.

For what it's worth even under 4K@25*3 I'm not seeing cuda use above 50% either...

Thank you!

Pezede, I think you have the perfect setup to test what really matters in the success of your interpolation.

I think with 4K HDR x3 in real time with RIFE, there is no point in playing around with encoding tests anymore.

What everyone is probably interested in is what the minimum configuration allows for:
1. 4K HDR x3 in real time with RIFE
2. 4K HDR x2 in real time with RIFE

My test proposal:

Your proposal to find the matching GPU does not work as the tensor cores are very different between the RTX cards ...

For me personally, there is no choice other than the RTX 4090. If only for one reason - in passive GPU cooling, die size is the most important factor:

GeForce RTX 4090 has 608.5 mm2
GeForce RTX 4080 has 378.6 mm2

The difference is significant. Here it is not possible, as in active cooling, to control the cooling by adjusting the fan speed.

I will be able to passively cool the GeForce RTX 4090 at around 250W TDP. I need the maximum contact area for heat transport away from the GPU. That is why I am particularly interested in this TDP value.

I suggested to Pezede this kind of testing as I assumed he would not have the option to change GPU, CPU and RAM for testing. This is the simplest test to see how the drop in performance of the various components in the setup affects performance. What can be saved on and what is not worth saving on.

DragonicPrime wrote:
UHD wrote:

Pezede, what size RAM do you have and how much did the RIFE use when interpolating x3 above 4K HDR video file?

DragonicPrime, what size RAM do you have and how much did the RIFE use when interpolating x2 above 4K HDR video file?

I have 32gb of ram. SVP and MPV combined only used around 4.2gb of ram

Thanks, that's very good news smile

Pezede, what size RAM do you have and how much did the RIFE use when interpolating x3 above 4K HDR video file?

DragonicPrime, what size RAM do you have and how much did the RIFE use when interpolating x2 above 4K HDR video file?

grobalt wrote:

I ordered a case and components to test and make some comparisons
I will have

Ryzen 5600x with 3600 MHz Memory
Ryzen 7700x with 7200 MHz Memory
Intel 13600 KF with 7200 MHz Memory

RTX4090
RTX4080
RTX4070ti

if someone wants to provide some good input or prepared scripts for comparison, i am happy to use them smile

Can someone integrate the script in MPC Player to add MadVR in the pipeline ?

I will be looking forward to the tests and will have lots of questions wink

If anyone else manages to pass the above tests without losing frames let us know.

If I have made any mistake in the above post also let me know.

UHD Video Frame Interpolation Benchmarks

SVP, mpv, vs-mlrt (vstrt: TensorRT-based GPU Runtime), RIFE Model 4.6
LG 4K HDR Demo - New York.ts
3840x2160, HDR10, 25FPS
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

4K HDR10 video playback with x3 RIFE interpolation in real time
PASS - RTX 4090, Ryzen 9 7950X, DDR5 6000 MHz – Pezede
https://www.svp-team.com/forum/viewtopi … 588#p81588

4K HDR10 video playback with x2 RIFE interpolation in real time
PASS - RTX 4090, Ryzen 9 5900X, DDR4 3800 MHz (OC) – DragonicPrime
https://www.svp-team.com/forum/viewtopi … 641#p81641

aloola wrote:

producing the same FPS with a lower GPU load is more efficient, right? big_smile

In fact, yes... That's why it's worth testing different configurations. I'm curious to see if anyone else gets a similar effect. However, checking why this happens is unlikely to be easy.

Pezede wrote:

Hey, I'm afraid that I don't really have the time to do those tests, it doesn't help that tinkering with ram on AM5 is far from a pleasant experience (I've tried)... sad

No problem. Thanks for the tests and I hope you will try to help other 4090 graphics card owners to replicate your real-time interpolation result smile

aloola wrote:
UHD wrote:
aloola wrote:

My MPC-MC+MPC Video Render + Vapoursynth Filter works perfectly fine with RIFE. Better results than MPV.

Better results???

well at least for me
1080p@24 x4
MPC+MPC render: ~75% GPU load.
SVP's MPV default: ~85%

maybe different between users' setups.

This is puzzling. Apparently it is the mpv that is more efficient....

Mardon85 wrote:

Ok I got it working with a 4K SDR video. Wow brilliant stuff. I'm not sure if its working with HDR though. Its x2 frame rate but the HDR colours don't seem to be coming through.

Have you tried the following file in real time?

video test:
LG 4K HDR Demo - New York.ts.
3840x2160, HDR10, 25FPS
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

Can you smoothly interpolate in real time with RIFE x2, without losing frames? How are the colours?

aloola wrote:

My MPC-MC+MPC Video Render + Vapoursynth Filter works perfectly fine with RIFE. Better results than MPV.

Better results???

Mardon85 wrote:

I've been a long time follower of the thread and now a very proud owner of a 4090. I can't wait to try out some 4K HDR RIFE!

Welcome another owner of a 4090 graphics card smile

DragonicPrime wrote:
Chainik wrote:

DragonicPrime

try this, will it improve a little bit? (replace in SVP 4\plugins64)

it helped. The LG HDR video now works. so 30fps x2 is now possible

So you are confirming that on your setup you are able to interpolate in real time x2 the file below?

LG 4K HDR Demo - New York.ts.

Recall what CPU you are using and whether the memory frequency is sufficient at 3600 MHz in your case or whether you need 3800 MHz for this test.

DragonicPrime wrote:

Edit 2: also seems like that LG HDR video we were testing on before still doesn't work for me. It's probably because it's a 30fps video. Testing with 24fps videos, I can do 2x with HDR at 4k without a single frame drop it seems. 30fps seems to still be too much though

It is 25fps, so only 4.2% more demanding for real-time interpolation than 24fps.

However, it's important to remember that it's full 4K: 3840x2160, so much more demanding than widescreen 4K movie.

LG 4K HDR Demo - New York.ts.
File size: 448 MiB.
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS.
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits.

Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

And how about the performance of these two methods?

Pezede wrote:

I've tried enabling and disabling and I'm not seeing any color differences.

For what it's worth even under 4K@25*3 I'm not seeing cuda use above 50% either...

Thank you!

Pezede, I think you have the perfect setup to test what really matters in the success of your interpolation.

I think with 4K HDR x3 in real time with RIFE, there is no point in playing around with encoding tests anymore.

What everyone is probably interested in is what the minimum configuration allows for:
1. 4K HDR x3 in real time with RIFE
2. 4K HDR x2 in real time with RIFE

My test proposal:

LG 4K HDR Demo - New York.ts
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/

1. Reduce only the TDP of GPU in steps of 10%

100%  4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
90% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
80% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
70% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
60% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
50% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

50% interests me personally, as I will be able to passively cool the graphics card to around 250W TDP

2. Reduce only the TDP of CPU in steps of 10%

100%  4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
90% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
80% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
70% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
60% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
50% 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

3. Reduce only the MHz of RAM in steps of 400 MHz

6000 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
5600 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
5200 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
4800 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
4400 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
4000 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time
3600 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

4800 MHz is of particular interest to me because this is currently the maximum with true ECC

I know this is probably too many tests, so maybe a simple 4 tests to start with and that will explain a lot from all of the above:

1.  GPU 50% TDP 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

2.  CPU 50% TDP (or 105W TDP or 65W TDP) 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

3. RAM 4800 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

4. RAM 3600 MHz 4K HDR x3 in real time pass or fail and if fail 4K HDR x2 in real time

Of course, it doesn't have to be today or tomorrow. It doesn't have to be anything at all. But I think you could help a lot of people to choose the optimal configuration for RIFE interpolation smile