You are not logged in. Please login or register.
SmoothVideo Project → Posts by UHD
NVIDIA GeForce RTX 4090 - first summary of graphics card data from various manufacturers:
https://docs.google.com/spreadsheets/d/ … AOQlHsRBY/
Source: https://twitter.com/harukaze5719
And here is what is very important to me in building a new HTPC completely passively cooled for real-time interpolation with RIFE:
.
Source: https://www.techpowerup.com/299096/nvid … management
This gives hope for the use of a passively cooled 700W PSU 
Only for 4090 4k@24fps x2
And how does real-time x2 RIFE interpolation of 1080p 10-bit HDR video files look like on your NVIDIA GeForce RTX 3070 Ti?
I found a publicly available demo that you can download for testing:
LG 4K HDR Demo - New York.ts
File size: 448 MiB
Duration: 1 min 13 sec
Overall bit rate: 51.4 Mbps
HDR format: SMPTE ST 2086, HDR10 compatible
Width: 3 840 pixels
Height: 2 160 pixels
Frame rate: 25.000 FPS
Color space: YUV
Chroma subsampling: 4:2:0
Bit depth: 10 bits
Direct link: https://drive.google.com/file/d/1dfR5TT … _bGfEXUvJ/
Source: http://hdr4k.blogspot.com/
The file of course needs to be re-encoded to 1080p 10-bit HDR for example as described in this thread:
https://www.reddit.com/r/ffmpeg/comment … intaining/
dlr5668 please write what you manage to do with this file. I am of course interested in real-time interpolation using the RIFE algorithm. Normal SVP interpolation is described here:
Colors - playing HDR video on HDR display
Only for Windows 10!1. AVSF + madVR is the best choice.
2. mpv player after 2019-10-31 can do direct HDR playback - use either SVP-provided build or shinchiro's build 20191102 or later.
3. VLC 3.0 can do the same but mpv's performance is better.
For mpv and VLC you have to turn on "Play HDR games and apps" or "HDR and WCG" (the name depends on the Windows version) switch in Windows Display settings first.
"Avisynth Filter" is a confusing naming cause there're _two_ different filters there, one for Avisynth and one for Vapoursynth
both supporting 10-bit and hdr
Thanks for your reply. I'm guessing you mean:
avisynth_filter: https://github.com/CrendKing/avisynth_filter
So if I understand correctly whichever combination:
AviSynthPlus-RIFE +avisynth_filter
or
VapourSynth-RIFE-ncnn-Vulkan +avisynth_filter
should be sufficient in SVP for real-time interpolation of 4K 10-bit HDR files using RIFE?
UHD wrote:The performance of the Tensor Cores of the new NVIDIA graphics cards is amazing! I'd be happy to help test as soon as I can buy one of these:
It is said that the performance of Optical Flow is also improved by about 2.5 times, because DLSS 3 uses motion interpolation.
Here is a great comparison of the Optical Multi Frame Generation:
00:22:57 DLSS Frame Generation vs Topaz Video Enhance AI & Adobe After Effects
https://www.youtube.com/watch?v=6pV93XhiC1Y&t=1377s
It is, of course, worth watching the entire video
...and so far Optical Multi Frame Generation will only be supported by GeForce RTX 40 Series graphics cards:
https://www.nvidia.com/en-us/geforce/ne … novations/
exactly the same as it work in vapoursynth-rife
Thank you very much for your answer, but unfortunately it doesn't explain anything to me.
I still don't have a suitable graphics card for vapoursynth-rife testing, so I have to rely on the knowledge of others here. Unfortunately, neither on this forum nor anywhere else have I found an answer as to how vapoursynth-rife handles 4K 10-bit HDR files.
My knowledge at this point is as follows:
1. Testing real-time interpolation for 4K 10-bit HDR files using the RIFE algorithm will only be possible after the release of the 4090 cards. Today, however, we can test 1080p 10-bit HDR files from re-encoded 4K 10-bit HDR files according to this thread: https://www.reddit.com/r/ffmpeg/comment … intaining/
2. SVP allows real-time interpolation for 4K 10-bit HDR files, but needs Avisynth Filter for this: https://www.svp-team.com/wiki/SVP:4K_and_HDR
3. Finally we have a port of the VapourSynth plugin RIFE to AviSynth+:
https://github.com/Asd-g/AviSynthPlus-RIFE hence my hope related to point 2 above that HDR can be pass through.
4. RIFE supports RGB format with float sample type of 32 bit depth: https://github.com/HomeOfVapourSynthEvo … cnn-Vulkan so I assume it could handle 10 bit colour depth.
5. This RIFE implementation: https://github.com/may-son/RIFE-FixDrop … ConvertFPS seems to confirm that handling 10-bit colour depth might be possible:
AI drop frame fixer and FPS converter
For image sequence from video. Not limited to 8 bits!Chainik, I know my knowledge is poor. If you could explain what else is needed for real-time interpolation using SVP and RIFE algorithm of 4K 10-bit HDR files then please write.
I have set up an account on GitHub: https://github.com/AIVFI and I can ask someone to adjust the RIFE filter to adapt to 4K 10-bit HDR files. Just please write what I should ask, because I have no testing capabilities myself and I'm not a programmer, so probably a lot of things that are obvious to you might be not understandable to me.
yeah, will do...
Thanks so much
but there's no noticeable difference with a "Vapoursynth filter". - load times and performance are exactly the same
This is actually very good news for me! I wasn't expecting a performance increase, quite the opposite: I was worried that there might be worse performance, and then no real-time 4K 10-bit HDR.
plus it requires an unreleased AVS+ build, crashes with the latest official release
It's probably only a matter of time before it becomes an official release.
Now the biggest concern still before me: will AviSynthPlus-RIFE work with 4K 10-bit HDR...?
Now this is my dream: real-time x2 interpolation of 4K 10-bit HDR video files using SVP with AviSynthPlus-RIFE.
I know it probably won't be that easy, it will require some time, but I think it might be of interest not only to people with the new 4090 card, because after all SVP can downscale source to, for example, 1080p while maintaining: 10-bit HDR: https://www.svp-team.com/wiki/Manual:Resizing
I also have a question if SVP working with AviSynthPlus-RIFE would be able to use RGB as the output format to a monitor? From what I understand AviSynth has more capabilities in this regard compared to VapourSynth.
RIFE requires RGB format with float sample type of 32 bit depth and the idea is not to go back to YUV at the end anymore, because any such conversion is lossy. The monitor needs RGB anyway.
AviSynthPlus-RIFE: https://github.com/Asd-g/AviSynthPlus-RIFE
Chainik, you've done an amazing thing by enabling real-time interpolation for 4K 10-bit HDR video files in SVP
SVP:4K and HDR
https://www.svp-team.com/wiki/SVP:4K_and_HDR
PUBLIC BETA TEST: AviSynth Filter - 10-bit/HDR in DirectShow
https://www.svp-team.com/forum/viewtopic.php?id=5877
As we can see, the essential element here is: AviSynth Filter.
Finally we have a port of the VapourSynth plugin RIFE to AviSynth+:
https://github.com/Asd-g/AviSynthPlus-RIFE
Chainik, would you be able to implement this new AviSynthPlus-RIFE into SVP so that it is capable of real-time interpolation of 4K 10-bit HDR files using RIFE?
The performance of the Tensor Cores of the new NVIDIA graphics cards is amazing! I'd be happy to help test as soon as I can buy one of these:
Source: TechPowerUp
Thanks Chainik for doing something new with RIFE
How about this:
Transcoding via SVP Manager with NVENC into H264 8Mb/s by the end it was working at 78 fps but it was climbing throughout the whole transcode. I am not sure if the fps counter in SVP Manager is accurate, is there any other way of keeping track of how fast my GPU is spitting out frames?
It is an LG Oled C1 48". It has a setting named TrueMotion and you can manually set de-judder and de-blur setting on a scale of 0 to 10. It seems like the video retains all of it's quality and the interpolation looks very smooth both in anime and movie footage. I would say it has less artifacts than RIFE x2 but on very few scenes (very choppy camera pans in anime) there is still some judder left. To be fair I believe on those scenes RIFE has problems as well but I haven't cross tested the same footage to compare them yet.
Today I found out that if I run a very simple script in Vapoursynth filter instead of SVP manager, I get a lot more consistent performance and without frame drops or gpu usage spikes. Now I can run 1536x864 resolution real time without any problems.
Thanks for sharing your results. It's a bit surprising that a card without Tensor Cores gives better results than a card with Tensor Cores. It's a shame that so few people share their results, we could then try to find the reasons for this.
It's good that you also shared a comparison of RIFE with TV capabilities. That's something we're missing a bit here too.
As you can see, for now there is no any better solution to your problem. The easiest thing to do, of course, is to buy a better graphics card. However, that's the way it is in life that you have to make the decision yourself. Money spent on a new card will save transcoding time and disk space. And vice versa. The time needed for transcoding and disk space will save money. All intermediate solutions come with the acceptance of lower quality.
Especially @UHD I want to thank you for your dedication to the thread. Your interest in motion interpolation burns with the white hot passion of a thousand burning suns. I further want to thank you for making me aware of Waifu2x-Extension-GUI by Aaron Feng, I had no idea this program existed, and the image upscaling on the REAL-ESRGAN-NCNN-VULKAN algorithm is just incredible. And above all, it's free!
Thanks. I'm glad you learned something new thanks to this thread. That was exactly my goal.
I've always been interested in watching videos in a way that reflects reality as much as possible. That's what brought me to SVP long before I started posting here on the forum. It was SVP that allowed me to experience the magic of real-time interpolation even on very archaic hardware.
It wasn't until 2018 that I saw for the first time that even better quality could be achieved with AI: https://developer.nvidia.com/blog/trans … n-with-ai/
A year later, I was admiring on YouTube what DAIN could do, but I saw no hope that this technology could ever be used in real time.
It wasn't until RIFE that I came to believe and to this day I am still fascinated by it and share this fascination with others. However, I am still raising funds for a PC that will allow me to fully enjoy the possibilities of AI.
So I invite anyone interested, like me who does not yet have the right hardware, but even more so those who have the hardware and can share the results of their tests, to join the discussion.
I, for the time being, can at most deal with the theoretical side and that is what I am trying to do. I will soon have some very interesting information on what the possibilities currently are in the field:
- video frame interpolation
- video deblurring
- video super-resolution
Or maybe even all 3 in 1 
I don't want to write too many things at once so as not to create too much confusion. However, it will be worth following this and my other thread dedicated to IFRNet: https://www.svp-team.com/forum/viewtopic.php?id=6666 
I understand. I will say that I'm a bit surprised that you are achieving such high speeds for GTX 1660 Super and 1080p x2 RIFE. I wonder if you achieve the same speeds for real life movies. Anime mostly have duplicate frames.
What is your result for Rife via Vulkan v4 for this file:
original demo video from the creator of RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing
I'm also curious to know what kind of TV you have and if you have compared the quality of TV interpolation with transcoded anime in full 1080p quality (RIFE) and good lossy or lossless compression.
And remember that you will always get a higher quality when interpolating with RIFE x3 than when interpolating with x2 - see Figure 6: https://arxiv.org/pdf/2204.03513.pdf
I hope someone can help you. I'm all for the highest quality, so for me it's either the right graphics card or encoding when it comes to RIFE. That's why I'm waiting patiently and raising funds for a suitable graphics card.
If you are happy with the interpolation on your TV, then why bother?
An hour ago nihui wrote:
This is yet another new video frame interpolation AI like RIFE
It claims to be better and faster than RIFE in the paperI ported IFRNet to ncnn-vulkan
https://twitter.com/nihui/status/1549763955396333568
For me, mainly interested in real-time interpolation, the IFRNet small interpolation model is particularly interesting:
RIFE 35.62 PSNR Vimeo90K vs. IFRNet small 35.59 PSNR Vimeo90K
Time:
RIFE 0.026 s vs. IFRNet small 0.019 s
Source: https://arxiv.org/pdf/2205.14620.pdf
Practically similar quality with a 27% reduction in interpolation time.
Great news!
nihui, who has made a very important optimisation of the AI RIFE algorithm for Tensor Cores and thanks to which we can already use this algorithm to interpolate 1080p files in real time....
an hour ago he published the full packages of the new AI interpolation algorithm optimised for Tensor Cores - IFRNet ncnn Vulkan
https://github.com/nihui/ifrnet-ncnn-vu … g/20220720
IFRNet algorithm source code:
https://github.com/ltkong218/IFRNet
Full description of the algorithm:
https://arxiv.org/pdf/2205.14620.pdf
RIFE Model 4.0 Benchmarks
720p - x2 interpolation - RIFE filter for VapourSynth (ncnn Vulkan):
70fps - NVIDIA GeForce RTX 2060 Mobile - Chainik
https://www.svp-team.com/forum/viewtopi … 158#p80158
92.956fps - NVIDIA GeForce RTX 2070 - blackmickey1007
https://www.svp-team.com/forum/viewtopi … 219#p80219
188.2fps - NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 163#p80163
1080p - x2 interpolation - RIFE filter for VapourSynth (ncnn Vulkan):
90fps - NVIDIA GeForce RTX 3070 Ti - dlr5668
https://www.svp-team.com/forum/viewtopi … 477#p80477
---------------------------------------------------------
720p is original demo video from the creator of RIFE at: https://github.com/hzwer/arXiv2020-RIFE
720p (1280x720), 25FPS, 53 s 680 ms, 4:2:0 YUV, 8 bits
direct link: https://drive.google.com/file/d/1i3xlKb … sp=sharing
1080p is an arbitrary file chosen
Fourth-generation Tensor Cores - Peak FP16 using the Sparsity feature:
2000 TFLOPS - NVIDIA H100 SXM5 (528 Tensor Cores, 700W TDP)
1600 TFLOPS - NVIDIA H100 PCIe (456 Tensor Cores, 350W TDP)
Third-Generation Tensor Cores - Peak FP16 using the Sparsity feature:
624 TFLOPS - NVIDIA A100 (432 Tensor Cores, 400W TDP)
320 TFLOPS - NVIDIA GeForce RTX 3090 Ti (336 Tensor Cores, 450W TDP)
284 TFLOPS - NVIDIA GeForce RTX 3090 FE (328 Tensor Cores, 350W TDP)
273 TFLOPS - NVIDIA GeForce RTX 3080 Ti (320 Tensor Cores, 350W TDP)
244 TFLOPS - NVIDIA GeForce RTX 3080 12GB (280 Tensor Cores, 350W TDP)
238 TFLOPS - NVIDIA GeForce RTX 3080 10GB FE (272 Tensor Cores, 320W TDP)
174 TFLOPS - NVIDIA GeForce RTX 3070 Ti (192 Tensor Cores, 290W TDP)
162.6 TFLOPS - NVIDIA GeForce RTX 3070 FE (184 Tensor Cores, 220W TDP)
129.6 TFLOPS - NVIDIA GeForce RTX 3060 Ti (152 Tensor Cores, 200W TDP)
102.4 TFLOPS - NVIDIA GeForce RTX 3060 (112 Tensor Cores, 170W TDP)
Second-Generation Tensor Cores - Peak FP16:
130.5 TFLOPS - NVIDIA TITAN RTX (576 Tensor Cores, 280W TDP)
113.8 TFLOPS - NVIDIA GeForce RTX 2080 Ti FE (544 Tensor Cores, 260W TDP)
107.6 TFLOPS - NVIDIA GeForce RTX 2080 Ti (544 Tensor Cores, 250W TDP)
89.2 TFLOPS - NVIDIA GeForce RTX 2080 Super FE (384 Tensor Cores, 250W TDP)
84.8 TFLOPS - NVIDIA GeForce RTX 2080 FE (368 Tensor Cores, 225W TDP)
80.5 TFLOPS - NVIDIA GeForce RTX 2080 (368 Tensor Cores, 215W TDP)
72.5 TFLOPS - NVIDIA GeForce RTX 2070 Super FE (320 Tensor Cores, 215W TDP)
63 TFLOPS - NVIDIA GeForce RTX 2070 FE (288 Tensor Cores, 185W TDP)
59.7 TFLOPS - NVIDIA GeForce RTX 2070 (288 Tensor Cores, 175W TDP)
57.4 TFLOPS - NVIDIA GeForce RTX 2060 Super (272 Tensor Cores, 175W TDP)
51.6 TFLOPS - NVIDIA GeForce RTX 2060 (240 Tensor Cores, 160W TDP)
First-generation Tensor Cores - Peak FP16:
125 TFLOPS - NVIDIA Tesla V100 (640 Tensor Cores, 300W TDP)
---------------------------------------------------------
H100
https://resources.nvidia.com/en-us-tensor-core
A100, V100
https://images.nvidia.com/aem-dam/en-zz … epaper.pdf
RTX 3090 FE, RTX 3080 10GB FE, RTX 3070 FE, TITAN RTX, RTX 2080 Super FE, RTX 2070 Super FE
https://images.nvidia.com/aem-dam/en-zz … per-V1.pdf
RTX 2080 Ti FE, RTX 2080 Ti, RTX 2080 FE, RTX 2080, RTX 2070 FE, RTX 2070
https://images.nvidia.com/aem-dam/en-zz … epaper.pdf
RTX 3090 Ti
https://www.anandtech.com/show/17334/nv … lpowerful-
https://www.techpowerup.com/gpu-specs/g … 0-ti.c3829
RTX 3080 Ti, RTX 3080 12GB, RTX 3070 Ti
https://www.anandtech.com/show/17204/nv … more-money
https://www.techpowerup.com/gpu-specs/g … 0-ti.c3735
https://www.techpowerup.com/gpu-specs/g … 2-gb.c3834
https://www.techpowerup.com/gpu-specs/g … 0-ti.c3675
RTX 3060 Ti, RTX 3060, RTX 2060
https://www.anandtech.com/show/16512/la … eam-at-329
https://www.techpowerup.com/gpu-specs/g … 0-ti.c3681
https://www.techpowerup.com/gpu-specs/g … 3060.c3682
https://www.techpowerup.com/gpu-specs/g … 2060.c3310
RTX 2060 Super
https://www.anandtech.com/show/16290/la … re-for-400
https://www.techpowerup.com/gpu-specs/g … uper.c3441
video choppyness/stuttering when played back at very slow speeds.
The best you can do is this (audio must be disabled or adjusted in some additional program):
1. It is best to use a monitor with FreeSync or G-Sync support and enable it.
2. If you have video: 29.97 fps transcode it only once and only with RIFE using multiplier x10, the resulting file will have 299.7 fps.
The maximum frame rate supported by HEVC is 300 fps
https://en.wikipedia.org/wiki/High_Effi … deo_Coding
3. Set your monitor to exactly 60.000Hz or rely on FreeSync or G-Sync. The latter is the best solution.
4. Change lossless fps from 299.7 fps to 60 fps if you want to watch 5x slower or 30 fps if you want to watch 10x slower. Use the raw bitstream method described in the following link:
https://trac.ffmpeg.org/wiki/How%20to%2 … 0a%20video
If you slow down in the player to some random value like 0.15 then don't be surprised if you get some stuttering. If you have a file encoded to 299.7 fps then choose either 0.2 (which gives 59.94 fps) or 0.1 (which gives 29.97 fps) and be sure to set the monitor to 59.94Hz. In any other case, you will have stuttering if you do not set your monitor exactly to the resulting fps or its exact double or triple. This is what FreeSync or G-Sync is for.
Today is the First Anniversary of the first RIFE filter for VapourSynth!
Exactly one year ago on June 1, 2021, this filter was created that made real-time interpolation with the RIFE algorithm possible: https://github.com/HomeOfVapourSynthEvo … ses/tag/r1
Also exactly one year ago on June 1, 2021, I asked the developers on this forum to integrate it into SVP: https://www.svp-team.com/forum/viewtopi … 419#p78419
The beginnings were difficult:
1080p @24->48 --> 4 fps
https://www.svp-team.com/forum/viewtopi … 426#p78426
but I believed that there is a huge optimization potential in this algorithm and today's version of RIFE filter for VapourSynth based on model 4.0 and optimized for Tensor Core makes it possible to believe that in August: https://videocardz.com/newz/rumor-nvidi … in-october it will be possible x5 interpolation for a 1920x1080 23.976 fps file using RIFE algorithm in real time: https://www.svp-team.com/forum/viewtopi … 480#p80480
If you want to use the RIFE algorithm for interpolation then only desktop!
Look at the difference in performance for:
GeForce 30 (30xx) series for desktops
and
GeForce 30 (30xx) series for laptops
in this link:
https://en.wikipedia.org/wiki/GeForce_30_series
compare the data in the columns named:
Tensor TFLOPS
It is the performance of the Tensor Cores that determines the speed of interpolation with RIFE.
Also consider waiting 3 months for NVIDIA GeForce RTX 4080: https://videocardz.com/newz/rumor-nvidi … in-october the price should be similar to RTX 3080, and the performance of the 4th generation Tensor Cores will increase dramatically looking at the performance of the first accelerator based on these cores - NVIDIA H100:
FP16 Tensor Core: 2000 teraFLOPS (with sparsity)
https://resources.nvidia.com/en-us-tensor-core
Graphics card prices are falling all the time, so it's worth waiting anyway:
https://www.3dcenter.org/news/news-des-2829-mai-2022
I understand you perfectly. I've been saving money every month since 2019 to buy something really powerful. I had plans to upgrade my computer for 2020, but the situation at that time forced me to revise my plans: https://www.svp-team.com/forum/viewtopi … 343#p80343
...a year later, graphics card prices shot into space, and this year has brought further problems and massive inflation in most countries around the world.
Fortunately, the performance of RIFE over the past year has improved enormously https://www.svp-team.com/forum/viewtopi … 426#p78426 and there is a chance that it will also improve in the future, allowing owners of also more accessible GPUs to enjoy RIFE interpolation in better quality.
I also want, by showing the potential performance of NVIDIA GeForce RTX 4090, to encourage as many people as possible to test RIFE. The more of us there are, the greater the chance that artificial intelligence in video frame interpolation will not only be the domain of researchers, but will also have real-world applications in real-time interpolation, which SVP is famous for.
I have no idea if it works or not. It says estimated fps 48. When i try to do 60fps or above it gets laggy. I couldn't see a difference between x2 and normal video playback. x2 is just a little bit smoother. I get 0 arifacts with x2.
If at x2 is a little bit smoother it means that interpolation is working and this should be the case for video with 8-bit colour depth. At the moment only one person with a GeForce 30 series graphics card has done a thorough test on a 1080p file: https://www.svp-team.com/forum/viewtopi … 477#p80477 and based on the fps we can assume that at the moment x2 for 1080p for your card is the maximum. If you want more: x3 or x4 then 720p files are left on the ground.
Perhaps there will be some optimisation of RIFE in the future. In any case, I estimate that I will need 116% more powerful graphics card than the ZOTAC GAMING GeForce RTX 3070 Ti AMP Extreme Holo to interpolate x5 1080p video files: https://www.svp-team.com/forum/viewtopi … 480#p80480 I suppose this will be more than possible for NVIDIA GeForce RTX 4090 graphics cards.
Oh well... artificial intelligence comes at a price.
The RGBS format is supported by RIFE.
YUV444P16 can be converted to RGBS format with almost no loss.
At the moment I don't have a RIFE compliant graphics card to run the tests.
I am relying on what the creator of Flowframes wrote a long time ago:
HDR is currently not supported as the neural networks only work with 8bpp content.
https://github.com/n00mkrad/flowframes/issues/40
and on the fact that already several people had problems with 10bit colour depth videos.
Did you manage to interpolate 10-bit video with RIFE in SVP?
In the image you posted above:
Pixel Format: p010
Try to use 8-bit colour depth video.
I'm doing everything as it says.
Yes, as long as it is video with no more than 1080p resolution, 8-bit colour depth without HDR
SmoothVideo Project → Posts by UHD
Powered by PunBB, supported by Informer Technologies, Inc.