Topic: Need help to find the best possible setting

First question i want to ask is which setting is responsible for fast paced scenes and the amount of artifacts created in those scenes?

Second question is which setting is responsible for the halo effect around moving objects and what is helpful other than x3 setting ?

Is it possible to nullify halo effect on moving objects but get bubble like artifacts instead ?

Is it possible to get a really good quality with the maxed out settings on fast paced anime and movie scenes with the least amount of artifacts possible?

I am using scripted settings that turn svp to dmitrirender like interpolation.

2 (edited by UHD 18-05-2022 21:27:18)

Re: Need help to find the best possible setting

If you want to get a really good quality with the least amount of artifacts possible use this: https://www.svp-team.com/wiki/RIFE_AI_interpolation

About halo effect around moving objects read here:
http://forum.doom9.net/showpost.php?p=1 … stcount=17
http://forum.doom9.net/showpost.php?p=1 … stcount=24

About anime read this: https://www.svp-team.com/forum/viewtopi … 308#p80308

Fast paced scenes are difficult for any model. Specifically created model for this is here:  https://github.com/google-research/frame-interpolation

If you want the best possible quality for anime, look for AI models with the best score in the ATD12K test.

If you want the best possible quality for fast paced scenes, look for AI models with the best score in the Xiph-2k test.

You can search for scores and compare them here among others:
https://arxiv.org/pdf/2204.03513.pdf
https://arxiv.org/pdf/2104.02495.pdf
https://arxiv.org/pdf/2202.04901.pdf

If you additionally want reasonably good speed and ease of use, then only SVP + RIFE filter for VapourSynth (ncnn Vulkan)

3 (edited by cemaydnlar 19-05-2022 14:15:41)

Re: Need help to find the best possible setting

UHD wrote:

If you want to get a really good quality with the least amount of artifacts possible use this: https://www.svp-team.com/wiki/RIFE_AI_interpolation

About halo effect around moving objects read here:
http://forum.doom9.net/showpost.php?p=1 … stcount=17
http://forum.doom9.net/showpost.php?p=1 … stcount=24

About anime read this: https://www.svp-team.com/forum/viewtopi … 308#p80308

Fast paced scenes are difficult for any model. Specifically created model for this is here:  https://github.com/google-research/frame-interpolation

If you want the best possible quality for anime, look for AI models with the best score in the ATD12K test.

If you want the best possible quality for fast paced scenes, look for AI models with the best score in the Xiph-2k test.

You can search for scores and compare them here among others:
https://arxiv.org/pdf/2204.03513.pdf
https://arxiv.org/pdf/2104.02495.pdf
https://arxiv.org/pdf/2202.04901.pdf

If you additionally want reasonably good speed and ease of use, then only SVP + RIFE filter for VapourSynth (ncnn Vulkan)

Thank you for your reply but isn't rife unusable for real time? Plus i like the way override scripts work but my intention is to get less halo and less artifacts on fast paced anime scenes. Therefore, i need svp options that are related with my issues. Plus which hardware acceleration setting is the fastest and the least artifact creating ?

4 (edited by UHD 19-05-2022 15:45:56)

Re: Need help to find the best possible setting

You have the same priorities as me: less halo and less artifacts during realtime interpolation.

Solution: x5 interpolation for 1920x1080 23.976 fps files in realtime using SVP + RIFE filter for VapourSynth (ncnn Vulkan)

Why x5? Better smoothess than x3 and both x3 and x5 give the best interpolation result with RIFE: https://www.svp-team.com/forum/viewtopi … 345#p80345

To do x5 interpolation for 1920x1080 23.976 fps files in realtime we need 116% more powerful graphics card than ZOTAC GAMING GeForce RTX 3070 Ti AMP Extreme Holo: https://www.svp-team.com/forum/viewtopi … 480#p80480

This will be possible soon with the NVIDIA GeForce RTX 4090.

5 (edited by UHD 19-05-2022 15:51:32)

Re: Need help to find the best possible setting

To-do List

Multi-frame input of the model

Frame interpolation at any time location (Done)

Eliminate artifacts as much as possible

Make the model applicable under any resolution input

Provide models with lower calculation consumption

https://github.com/hzwer/Practical-RIFE

6 (edited by UHD 19-05-2022 16:22:13)

Re: Need help to find the best possible setting

In general, RIFE is better with occlusions, object boundaries, y-axis rotational vectors than mvtools2 based tools.

http://forum.doom9.net/showpost.php?p=1 … tcount=312

https://i.postimg.cc/T2Zg6s1y/ezgif-2-85352ef164.gif
http://forum.doom9.net/showpost.php?p=1 … tcount=323

https://i.postimg.cc/zBBm9pbB/handwave-frc-rife.gif
http://forum.doom9.net/showpost.php?p=1 … tcount=324

Re: Need help to find the best possible setting

UHD wrote:

In general, RIFE is better with occlusions, object boundaries, y-axis rotational vectors than mvtools2 based tools.

http://forum.doom9.net/showpost.php?p=1 … tcount=312

https://i.postimg.cc/T2Zg6s1y/ezgif-2-85352ef164.gif
http://forum.doom9.net/showpost.php?p=1 … tcount=323

https://i.postimg.cc/zBBm9pbB/handwave-frc-rife.gif
http://forum.doom9.net/showpost.php?p=1 … tcount=324


I have a 3070 but i don't want to use rife since its x2 and too slow in processing. Can you please talk about the basic svp settings that are responsible for fast paced scene artifacts and halo around moving objects?

8 (edited by UHD 19-05-2022 18:40:32)

Re: Need help to find the best possible setting

You asked the same question almost 3 years ago and got plenty of answers: https://www.svp-team.com/forum/viewtopic.php?id=5500

You spoke up when others asked about similar problems: https://www.svp-team.com/forum/viewtopic.php?id=6139

Others have asked before you and someone even stated that he preferred FrameRateConverter over SVP for this reason: https://www.svp-team.com/forum/viewtopic.php?id=4382

I just showed You that RIFE beats both mvtools2 and FrameRateConverter in what you asked about.

It can be seen that You care a lot about this and over the years you have not found a solution that satisfies you.

Do you think there is some magical way to make the base SVP algorithm capable of approaching what artificial intelligence can achieve? 

Base SVP interpolation uses only a tiny fraction of the computing power that RIFE requires.

You can spend hours testing every possible setting in SVP yourself, or you can use that time to do extra work that will allow you to upgrade your GPU. You can also take a shortcut, although I don't recommend it: https://www.svp-team.com/forum/viewtopi … 388#p80388

Re: Need help to find the best possible setting

UHD wrote:

You asked the same question almost 3 years ago and got plenty of answers: https://www.svp-team.com/forum/viewtopic.php?id=5500

You spoke up when others asked about similar problems: https://www.svp-team.com/forum/viewtopic.php?id=6139

Others have asked before you and someone even stated that he preferred FrameRateConverter over SVP for this reason: https://www.svp-team.com/forum/viewtopic.php?id=4382

I just showed You that RIFE beats both mvtools2 and FrameRateConverter in what you asked about.

It can be seen that You care a lot about this and over the years you have not found a solution that satisfies you.

Do you think there is some magical way to make the base SVP algorithm capable of approaching what artificial intelligence can achieve? 

Base SVP interpolation uses only a tiny fraction of the computing power that RIFE requires.

You can spend hours testing every possible setting in SVP yourself, or you can use that time to do extra work that will allow you to upgrade your GPU. You can also take a shortcut, although I don't recommend it: https://www.svp-team.com/forum/viewtopi … 388#p80388

I also asked if there is a code or some kind of script to make svp better but they answerd no. A couple of month ago a rondom guy came and gave us a script that makes svp better instead of anyone working for svp. 3 years ago when i asked how we can get better result for haloing issue. The answer i got was x3. Anyway thx for your kind answers i'm sure someone will find a way to make it better instead of those who work for svp.

10 (edited by UHD 19-05-2022 22:08:02)

Re: Need help to find the best possible setting

You are probably writing about this: https://www.svp-team.com/forum/viewtopic.php?id=6488
Remember, there are probably 2 SVP developers and thousands of SVP users. The chance that some enthusiast finds a combination of settings that gives a slightly better visual effect is simply higher. However, this requires a great deal of time and a great deal of trying, and sometimes a little more luck.

Despite the fact that someone has found a slightly better setting than the others, you are still not completely happy. And you are not likely to be. The base algorithm has certain limitations that cannot be jumped over. It already has a huge number of setting combinations and this is a credit to the developers. Searching for optimal settings is the task of the users, who are also developers, but there are only two of them. And the search is an enormous amount of time.

That's why recently, thanks to the development of GPU computing power, a lot of machine learning-based frame interpolation models have started to appear. This makes it possible to speed up considerably to reach the ideal of artifact-free interpolation. There is no other way. Artificial intelligence is entering all areas of life.

If everyone supported AI models, we would show their developers that we care and that their work is not in vain. Thanks to them we are closer to the ideal of real-time artifact-free frame interpolation:

https://github.com/hzwer/arXiv2021-RIFE
https://github.com/hzwer/Practical-RIFE
https://github.com/nihui/rife-ncnn-vulkan
https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan
https://www.svp-team.com/wiki/RIFE_AI_interpolation

Of course I wish you and myself to find someone who will find even better settings for the basic SVP algorithm, because it is still irreplaceable for 4K 10bit HDR files. However progress cannot be stopped and the best quality frame interpolation will be provided only by artificial intelligence.

Re: Need help to find the best possible setting

UHD wrote:

You are probably writing about this: https://www.svp-team.com/forum/viewtopic.php?id=6488
Remember, there are probably 2 SVP developers and thousands of SVP users. The chance that some enthusiast finds a combination of settings that gives a slightly better visual effect is simply higher. However, this requires a great deal of time and a great deal of trying, and sometimes a little more luck.

Despite the fact that someone has found a slightly better setting than the others, you are still not completely happy. And you are not likely to be. The base algorithm has certain limitations that cannot be jumped over. It already has a huge number of setting combinations and this is a credit to the developers. Searching for optimal settings is the task of the users, who are also developers, but there are only two of them. And the search is an enormous amount of time.

That's why recently, thanks to the development of GPU computing power, a lot of machine learning-based frame interpolation models have started to appear. This makes it possible to speed up considerably to reach the ideal of artifact-free interpolation. There is no other way. Artificial intelligence is entering all areas of life.

If everyone supported AI models, we would show their developers that we care and that their work is not in vain. Thanks to them we are closer to the ideal of real-time artifact-free frame interpolation:

https://github.com/hzwer/arXiv2021-RIFE
https://github.com/hzwer/Practical-RIFE
https://github.com/nihui/rife-ncnn-vulkan
https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan
https://www.svp-team.com/wiki/RIFE_AI_interpolation

Of course I wish you and myself to find someone who will find even better settings for the basic SVP algorithm, because it is still irreplaceable for 4K 10bit HDR files. However progress cannot be stopped and the best quality frame interpolation will be provided only by artificial intelligence.

Ok i'll give rife a try. Is a rtx 3070 enough for x2 ? I am selecting rife interpolation but my fps stays at 24? What am i doing wrong ? Can you help me use rife?

12 (edited by UHD 20-05-2022 16:49:57)

Re: Need help to find the best possible setting

cemaydnlar wrote:

Ok i'll give rife a try. Is a rtx 3070 enough for x2 ?

Yes, as long as it is video with no more than 1080p resolution, 8-bit colour depth without HDR


cemaydnlar wrote:

I am selecting rife interpolation but my fps stays at 24 ? What am i doing wrong ? Can you help me use rife?

Read this first:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

Set the following values:

AI model: Generic (v4)
GPU threads: 2
GPU device: https://www.svp-team.com/forum/viewtopi … 170#p80170
RIFE via CUDA: Off

If you continue to have problems read what problems others had and how they dealt with them:

whole threads:
https://www.svp-team.com/forum/viewtopic.php?id=6580
https://www.svp-team.com/forum/viewtopic.php?id=6553

also read my thread from Chainik's post of April 28 to the end, everything should work from then on:
https://www.svp-team.com/forum/viewtopi … 360#p80360

skip it altogether: == RIFE / PyTorch installation == it's too slow now
https://www.svp-team.com/forum/viewtopi … 372#p80372

13 (edited by cemaydnlar 20-05-2022 18:58:43)

Re: Need help to find the best possible setting

UHD wrote:
cemaydnlar wrote:

Ok i'll give rife a try. Is a rtx 3070 enough for x2 ?

Yes, as long as it is video with no more than 1080p resolution, 8-bit colour depth without HDR


cemaydnlar wrote:

I am selecting rife interpolation but my fps stays at 24 ? What am i doing wrong ? Can you help me use rife?

Read this first:
https://www.svp-team.com/wiki/RIFE_AI_interpolation

Set the following values:

AI model: Generic (v4)
GPU threads: 2
GPU device: https://www.svp-team.com/forum/viewtopi … 170#p80170
RIFE via CUDA: Off

If you continue to have problems read what problems others had and how they dealt with them:

whole threads:
https://www.svp-team.com/forum/viewtopic.php?id=6580
https://www.svp-team.com/forum/viewtopic.php?id=6553

also read my thread from Chainik's post of April 28 to the end, everything should work from then on:
https://www.svp-team.com/forum/viewtopi … 360#p80360

skip it altogether: == RIFE / PyTorch installation == it's too slow now
https://www.svp-team.com/forum/viewtopi … 372#p80372

I'm doing everything as it says. I'm using mpv player. Still fps stays 24. Couldn't find the reason.

https://imgur.com/a/H85vrP8 this is the way it looks.

14 (edited by UHD 20-05-2022 21:55:42)

Re: Need help to find the best possible setting

cemaydnlar wrote:

I'm doing everything as it says.


UHD wrote:

Yes, as long as it is video with no more than 1080p resolution, 8-bit colour depth without HDR

Re: Need help to find the best possible setting

I recently helped friend setting up SVP. It was as easy as manual install + copy mpv setting folder to appdata (optional). RIFE worked fine out of the gate

Re: Need help to find the best possible setting

dlr5668 wrote:

I recently helped friend setting up SVP. It was as easy as manual install + copy mpv setting folder to appdata (optional). RIFE worked fine out of the gate

Did everything according to svp rife manuel. Fps stays the same as in the image i posted above.

17 (edited by UHD 21-05-2022 14:15:34)

Re: Need help to find the best possible setting

In the image you posted above:

Pixel Format: p010


Try to use 8-bit colour depth video.

Re: Need help to find the best possible setting

UHD wrote:

In the image you posted above:

Pixel Format: p010


Try to use 8-bit colour depth video.


I guess it works. Since estimated fps is 48. Can you check if it is the same on your mpv?

19 (edited by blackmickey1007 24-05-2022 16:07:14)

Re: Need help to find the best possible setting

The RGBS format is supported by RIFE.
YUV444P16 can be converted to RGBS format with almost no loss.

20 (edited by UHD 24-05-2022 20:15:22)

Re: Need help to find the best possible setting

blackmickey1007 wrote:

The RGBS format is supported by RIFE.
YUV444P16 can be converted to RGBS format with almost no loss.

At the moment I don't have a RIFE compliant graphics card to run the tests.
I am relying on what the creator of Flowframes wrote a long time ago:

HDR is currently not supported as the neural networks only work with 8bpp content.

https://github.com/n00mkrad/flowframes/issues/40

and on the fact that already several people had problems with 10bit colour depth videos.

Did you manage to interpolate 10-bit video with RIFE in SVP?

Re: Need help to find the best possible setting

UHD wrote:
blackmickey1007 wrote:

The RGBS format is supported by RIFE.
YUV444P16 can be converted to RGBS format with almost no loss.

At the moment I don't have a RIFE compliant graphics card to run the tests.
I am relying on what the creator of Flowframes wrote a long time ago:

HDR is currently not supported as the neural networks only work with 8bpp content.

https://github.com/n00mkrad/flowframes/issues/40

and on the fact that already several people had problems with 10bit colour depth videos.

Did you manage to interpolate 10-bit video with RIFE in SVP?


I have no idea if it works or not. It says estimated fps 48. When i try to do 60fps or above it gets laggy. I couldn't see a difference between x2 and normal video playback. x2 is just a little bit smoother. I get 0 arifacts with x2.

22

Re: Need help to find the best possible setting

cemaydnlar wrote:

I have no idea if it works or not. It says estimated fps 48. When i try to do 60fps or above it gets laggy. I couldn't see a difference between x2 and normal video playback. x2 is just a little bit smoother. I get 0 arifacts with x2.

If at x2 is a little bit smoother it means that interpolation is working and this should be the case for video with 8-bit colour depth. At the moment only one person with a GeForce 30 series graphics card has done a thorough test on a 1080p file: https://www.svp-team.com/forum/viewtopi … 477#p80477 and based on the fps we can assume that at the moment x2 for 1080p for your card is the maximum. If you want more: x3 or x4 then 720p files are left on the ground.

Perhaps there will be some optimisation of RIFE in the future. In any case, I estimate that I will need 116% more powerful graphics card than the ZOTAC GAMING GeForce RTX 3070 Ti AMP Extreme Holo to interpolate x5 1080p video files: https://www.svp-team.com/forum/viewtopi … 480#p80480 I suppose this will be more than possible for NVIDIA GeForce RTX 4090 graphics cards.

Oh well... artificial intelligence comes at a price.

Re: Need help to find the best possible setting

UHD wrote:
cemaydnlar wrote:

I have no idea if it works or not. It says estimated fps 48. When i try to do 60fps or above it gets laggy. I couldn't see a difference between x2 and normal video playback. x2 is just a little bit smoother. I get 0 arifacts with x2.

If at x2 is a little bit smoother it means that interpolation is working and this should be the case for video with 8-bit colour depth. At the moment only one person with a GeForce 30 series graphics card has done a thorough test on a 1080p file: https://www.svp-team.com/forum/viewtopi … 477#p80477 and based on the fps we can assume that at the moment x2 for 1080p for your card is the maximum. If you want more: x3 or x4 then 720p files are left on the ground.

Perhaps there will be some optimisation of RIFE in the future. In any case, I estimate that I will need 116% more powerful graphics card than the ZOTAC GAMING GeForce RTX 3070 Ti AMP Extreme Holo to interpolate x5 1080p video files: https://www.svp-team.com/forum/viewtopi … 480#p80480 I suppose this will be more than possible for NVIDIA GeForce RTX 4090 graphics cards.

Oh well... artificial intelligence comes at a price.


Well i live in TUrkey. A rtx 3090 ti costs 62.000 Try in here. If i go with the minimum a 4090 will cost maybe over 90k. You can buy a house with the same amount if you see that as even currency unit big_smile

24

Re: Need help to find the best possible setting

I understand you perfectly. I've been saving money every month since 2019 to buy something really powerful. I had plans to upgrade my computer for 2020, but the situation at that time forced me to revise my plans: https://www.svp-team.com/forum/viewtopi … 343#p80343

...a year later, graphics card prices shot into space, and this year has brought further problems and massive inflation in most countries around the world.

Fortunately, the performance of RIFE over the past year has improved enormously https://www.svp-team.com/forum/viewtopi … 426#p78426 and there is a chance that it will also improve in the future, allowing owners of also more accessible GPUs to enjoy RIFE interpolation in better quality.

I also want, by showing the potential performance of NVIDIA GeForce RTX 4090, to encourage as many people as possible to test RIFE. The more of us there are, the greater the chance that artificial intelligence in video frame interpolation will not only be the domain of researchers, but will also have real-world applications in real-time interpolation, which SVP is famous for.