You are not logged in. Please login or register.
SmoothVideo Project → Posts by dawkinscm
It is one year old project with no updates at github.
Yes I noticed that, but I thought, or maybe hoped that there was something I was missing.
Great news!
nihui, who has made a very important optimisation of the AI RIFE algorithm for Tensor Cores and thanks to which we can already use this algorithm to interpolate 1080p files in real time....
an hour ago he published the full packages of the new AI interpolation algorithm optimised for Tensor Cores - IFRNet ncnn Vulkan
https://github.com/nihui/ifrnet-ncnn-vu … g/20220720IFRNet algorithm source code:
https://github.com/ltkong218/IFRNetFull description of the algorithm:
https://arxiv.org/pdf/2205.14620.pdf
So the obvious question is how do we use this? Can we use it with SVP or do we have to wait to see if the DEVs can (or want to) incorporate it?
dawkinscm wrote:[
It's a very minor change which you can see if you search for ensemble in the wrapper code. If this doesn't make sense then unless you have a 3090ti or above just leave it for now.Is it difficult to explain how to do it? I would like to try it out and see the difference good or bad for myself and learn something new in the process. Is there any documentation on the forum how to do it? I find users saying they have used it, one asking to implement an on/off mode in svp, but no guide on how to do it.
Hence my initial question.
I did tell you. I've never coded in Python but I have written code so it was easy to work out. All I did was do a search for ensemble in vlsmrt.py. The rest is easy.
DragonicPrime wrote:Drakko01 wrote:Can someone remember me how to use ensemble model, in the model drop down I only see 4.4 and 4.6. I use Mpc-hc as a player.
copy the folder into SVP folder>rife>models. After that it should use it automatically. At least it does for me. If it worked correctly, it should pop up that command prompt window while it sets everything up for every resolution if you have performance mode enabled. Little warning. 4k content doesnt' seem to work very well with this version. The quality is noticeably worse for me and others in this thread if I remember correctly
Thanks for answering, but if understand correctly, by default use this model rife_v4.6.onnx from the v2 package. And my question is how to use this rife_v4.6_ensemble.onnx model.
It's a very minor change which you can see if you search for ensemble in the wrapper code. If this doesn't make sense then unless you have a 3090ti or above just leave it for now.
Blackfyre
To summarise:The best current model is:
rife_v4.6_ensemble.onnx from the v2 package: https://github.com/AmusementClub/vs-mlr … e_v2_v1.7zIf someone is not able to use the full capacity of their screen with this model, for example to interpolate 4K files, it is worth using a faster model, with a small loss of quality:
rife_v4.6.onnx from the v2 package: https://github.com/AmusementClub/vs-mlr … e_v2_v1.7zIf anyone is interested in much, much better quality than RIFE interpolation I will write about it soon, here on this forum. Please give me another week, maybe two weeks, until I update my repository on GitHub.
I would love to use Ensemble for my 1920x2160 files, but no matter what I try, my 3080 is basically stuck at 100% and the picture is very jerky.
A few months ago the direct show players weren't even close to the same quality as MPV. But as an experiment I've set up mpc-be with Nvidia super-resolution running Rife v2 upscaling to 4K. I'm only upscaling on the Y axis because the resolution of my files is 1920x2160p. I compared it to using MPV it is "almost" identical except for the areas where you have little control like colour, banding and aliasing. Also rife now works just as it does on MPV which again is another great improvement. I also tried Madvr for much which enables control over aliasing, banding and colour management and they were much better (but still not quite as good as MPV). Madvr can't be used with Nvidia super-resolution but it was the best option for picture quality in my use case.
Maybe it's always been this way and my issues were local, but Rife v2 is now very useable, even with Madvr upscaling to 4k. MPV still has the advantage, but the gap between the two isn't a great as it was before. Nice to have the choice 
the path is different in the original "Python wrapper script"
I see. The path code is different. Thanks I'll look into this further.
OK I see modified code for SVP. It works as expected now. I just have to remember to add this for future revisions. I don't have a temp directory set up so I guess it was using the current working directory. Can't find it though. I've never coded in Python so this is a fun way to learn bits and pieces of Python coding. Anyway thanks @Chainik.
not sure what you guys mean...
if there's a file for given resolution in the cache folder - it reads it
if the file isn't there, it generates it, end of storysimple as a AK-47
Could not read timing cache from: C:\Users\chain\AppData\Roaming\SVP4\cache\Program Files (x86)/SVP 4/rife\models\rife\rife_v4.6.onnx.1920x1088_fp16_no-tf32_trt-8502_cudnn_I-fp16_O-fp16_NVIDIA-GeForce-RTX-2060_3dcbe72f.engine.cache. A new timing cache will be generated and written.
Not for me. I can literally delete the models/rife folder and it does one of two things. It either keeps on working as if the cache is still there and nothing changes. Or it does generate a new timing cache but the rife folder does not reappear and if the empty cache folder is there then it stays empty. The only thing I can think of is that I am using a later version of the Python wrapper script (3.15.23) and rife v2.
I think it's specific to the GPU model and TensortRT version in use.
On a related question. I notice if I delete the Rife cache then even if new entries are created the cache folder stays empty and the rife folders also stay empty yet everything still works. What am I missing?
For questions below my message:
Chainik wrote:> How to re-generate the rife model engine in AppData in once a certain resolution is mistakenly generated(not that I want),
delete corresponding files?
nope, like I said, it will just play the video without rife re-generated like the default, only reinstall the whole program will somehow re-activate that procedure, seems like there's some registry info missing?
That's exactly what I found like I said above.
We haven't updated for some time, I saw that there are new versions of TensorRT OSS v8.6.1/v14. Latest TensorRT library, I don't know if you plan to upgrade or if it makes any sense/benefit to do so. Sometimes the updates mean big changes, other times, for example, drivers, big problems.Big fan of Rife option and looking forward to see it improve. Thanks for the work you already done.
No need to update. I've made some tweaks using the latest scripts but Rife is still Rife, and v2 still FF/REW instantly. I wouldn't bother.
Thank you @UHD for all that detailed analysis and what can be done for the future.
Loved reading it.
dawkinscm wrote:Rife is flawlessly smooth with minimal artefacts.
It has different artifacts, although minimal the vast majority of the time (compared to prior methods). However, when they are noticeable, like in the beginning of the movie AIR that came out recently (scenes with a lot of noise especially), they are very heavy and noticable. Or the HBO (static) background at the start of GoT/HoTD/etc
Yes that is true. The opening scenes in Blade Runner 2049 and the star fields in Gravity have a horrible artefacts but they don't last long. Also I don't think either film has the kind of slow pan that causes problems for Automatic mode so I could use that instead and the issues go away.
So I think the Video Deblurring and Video Frame Interpolation championed by @UHD should be in its own thread.
Once there is a working model, there should definitely be a new thread for it. But I feel as though that is a while away.
Wishing @UHD luck with his proposal.
Yes but this isn't the easiest thread to navigate already and the last page is effectively a large dump of info not related to the topic. Maybe a single post intro should be placed here which then links to the new thread.
This new stuff does seem extremely over complicated and Rife is flawlessly smooth with minimal artefacts. It just needs to be less resource hungry and reduce artefacts even further. But maybe Rife also seemed extremely complicated before members worked through it and got it to where it is now. So I think the Video Deblurring and Video Frame Interpolation championed by @UHD should be in its own thread.
Thank you for your answer.
Here is the weird thing:
I removed all the mpv.conf files I found, and restored the default one in SVP 4/mpv64.
Then I added a simple sub-font-size in the file, but nothing changes.
I'm totally lost.
Did the default config work?
just male a folder name "portable_config" on the same folder with your mpv.exe, then put the mpv.conf there
example: \MPV\portable_config\mpv.conf
if you want to add .lua scripts put them in
\MPV\portable_config\Scripts\ *.lua here.conf for Lua scripts
\MPV\portable_config\script-opts\ *.conf here
I remember having similar issues and this works but it's not an actual fix which is why I recommend the other approach.
Hello everyone,
I recently reinstalled windows and had to set up again my SVP pro app with mpv.
I had a nice mpv.conf that I saved, with proper subtitle size and font, but now I cannot custom anything : the MPV app doesn't seem to read my conf file.
I tried to change the file everywhere I found it, from the SVP/mpv64 folder to the roaming/MPV folder.I like MPV because its light on the cpu and powerfull at the same time.
Not an expert but either I find how to make this conf file work, or I will switch player since I was already lacking proper UI in MPV (And I would need suggestions cause VLC UI is good but SVP integration seems crap)
If you have a config in both the SVP and Appdata folder then it will use the one in the SVP folder. Usually the mpv.conf file is not read when there is a mistake in the config.
I would suggest that you remove your mpv.conf file from both places and use the default SVP conf file and make sure that is working first. Then start copying a line or two of config from your mpv.conf file to the default SVP conf file and see if it still works. That way you can see where the issue is.
Just bought this to enjoy 3D films in VR, while the basic interpolation is okay, is it possible to use Rife or TensorRT in a VR player?
Use Rife/TensorRT on a PC with Nvidia GPU then use software like Virtual Desktop to send the image to your headset. Once set up properly, especially using MPV player, it works well.
cemaydnlar wrote:Does decrease to hd setting have an effect on smoothness or is it just lowering the video quality ? I fell like original video feels smoother than the one with decrease to hd setting.
That button reduces the video to 720P. I can't imagine any situation where it would make sense to use it. It may be that the resolution of the video plays a role in how well the algorithms work, but as far as I know it just uses some kind of cheap scaler to resize the video to 1280x720 before SVP interpolates it.
Whatever scaler it uses it's unlikely to be better than the kind of scalers you can choose with MPV. I would rather choose my own scaler so that I know the quality I am getting.
Fortune424 wrote:The annoying thing about the performance boost for me has been movies that use multiple aspect ratios / resolutions. For example I'm watching Resident Evil (2002) BluRay right now and there's a 16:9 camera, an almost 16:9 camera, a 21:9 camera, 4:3 CGI for the computer vision/security camera stuff, etc. It has stopped to generate the training info a few times. For anime/web content it's not so bad - you've basically just got 21:9 and 16:9 versions of 480P through 4K and then the occasional portrait/4:3 video.
As an aside, I just realized that this is because of SVP's black bar detection. If you disable black bar detection, the number of possible resolutions you need to pre-cache the RIFE info for will decrease... but apparently the reason that exists in the first place is because SVP thinks the bars are part of the video to interpolate if they're NOT removed, so it sounds like you'd be giving up some quality there if you choose to leave it off.
The black bar detection was annoying even before Rife and caused me nothing but trouble. As for interpolating black bars, that makes no sense to me and if there was some kind of weird picture quality issue then they should mention it. But as I said, that would be weird.
dawkinscm wrote:Fortune424 wrote:I think I can confirm. It really is painfully obvious when you do a side by side comparison. I bet V2 looks worse than the 1080P BluRay would. I guess the free performance boost was too good to be true, for now.
Here's from the opening of the 4K BluRay of Avatar 2:
MPV with stock SVP settings, RIFE 4.6 w/ TensorRT on a 3090.
I don't think it does look worse than a 1080 BluRay. I use VR to watch 3D and as far as I can tell, I don't get the same issues. Maybe there's a resolution issue, but I checked and all the detail is there when compared to the original Blu-ray when using Rife v2 with no quality difference to Rife v1. I also checked a number of difficult scenes from other movies and the same applies there too. So maybe it's a 4K resolution issue but I'm not interested in 4K playback using SVP so I haven't tried that. Stock MPV settings uses gpu and I do notice a marked improvement in quality when using gpu-next with proper configuration so you may wish to try that as well.
You can see in our screenshots that SOMETHING is wrong. It looks a lot more jagged and low res with V2 enabled. It is entirely possible it's only at 4K though. I didn't notice a problem at 1080P, though there is less detail there to begin with so who knows. At this point I'm just going to stick with V1 personally, until the cause is revealed or it is fixed.
Yeah it's clear that there is an issue with aliasing which is one of those things that once you see it you can't unsee it. My MPV config is almost identical to my MadVR config and I've found that the Jinc algorithm works best for both in terms of anti-aliasing.
I think I can confirm. It really is painfully obvious when you do a side by side comparison. I bet V2 looks worse than the 1080P BluRay would. I guess the free performance boost was too good to be true, for now.
Here's from the opening of the 4K BluRay of Avatar 2:
MPV with stock SVP settings, RIFE 4.6 w/ TensorRT on a 3090.
I don't think it does look worse than a 1080 BluRay. I use VR to watch 3D and as far as I can tell, I don't get the same issues. Maybe there's a resolution issue, but I checked and all the detail is there when compared to the original Blu-ray when using Rife v2 with no quality difference to Rife v1. I also checked a number of difficult scenes from other movies and the same applies there too. So maybe it's a 4K resolution issue but I'm not interested in 4K playback using SVP so I haven't tried that. Stock MPV settings uses gpu and I do notice a marked improvement in quality when using gpu-next with proper configuration so you may wish to try that as well.
> You never know, I might get a response this time
how do you think, what devs might think about replying the same question for the 100th time?
if you're using your own vapoursynth installation you must be knowing what you're doing
First of all thank you @Chainik I searched and couldn't find any responses to my question about Rife v2 being automatically selected when using Tensor which is why I wasn't happy when I got no response. But in the end it was to my benefit because it made me go through the code. I am pretty sure I had VapourSynth working before so I'm not sure why I installed an external version which ended up breaking it so again I appreciate your feedback. But it did lead to me discovering MPV so the two issues turned into two positives. Apologies and thanks again smile
dawkinscm wrote:I moved away from MPC (with MadVR) because it was usable with Rife. With MPV I get better performance, almost perfect smoothness, less artefacts and much better interpolated picture quality than with MPC/MadVR so I can never go back. With a 3070 laptop GPU you are always going to have issues but maybe with MPC it will work better for you.
I installed MPC-HC and configured it to run through VapourSynth. Have you tested with this setup? I'm testing now, will report any improvements on the 3D load later.
L.E: Tried a 40 minute episode. Everything seems the same in MPC-HC quality-wise, except the 3D load is 10% lower on average. MPV hits 70%, while MPC-HC barely hits 60%. This could be good news, but need to test further with a long movie, as the frame drops rarely occur in 40 minute episodes with MPV as well.
Update since response from @Chainik
Using MPV is similar to using MPC with MadVR so if you are just using MPC-HC then that's probably why you are getting a lower 3D load. Now that I've go MPC+Rife working with VapourSynth, I see that is almost as smooth as with MPV and if I hadn't discovered MPV I would be pretty happy. But I can't see me going back to using MPC when MPV is basically MPC+MadVR but with better picture quality and fewer micro-stutters with Rife.
mpv can be too wild to config imo. Can you retest with MPC-HC ?
Both MadVR and MPV require a lot of reading but "by default" MadVR has a GUI while MPV does not. I'm new to MPV but I like the additional flexibility of the text file and the better picture quality. But MPV is easy to over-configure with lines of config that are not required. For the most part, leaving the basic commands at default works and then tweaking from there works best.
I switched last night to V2 as per the tutorial posted above. I can't really notice much of a difference in smoothness on my RTX 3070 laptop GPU for 1080p content. Artifact handling seems a bit better maybe.
However, I still get the framerate drops that last around 10 seconds. These usually happen after 1 h or 20-30 minutes during 2h+ movies, very rarely in TV series episodes of ~40 minutes. The SVP index drops from 1 to 0.8-0.7 usually, sometimes lower, and the GPU 3D utilization in task manager jumps from 70-80% to 100% while RAM utilization sits sub 2 GB most of the times. This is happening in MPV by the way. I need to test with MPC-HC.
I moved away from MPC (with MadVR) because it was unusable with Rife. With MPV I get better performance, almost perfect smoothness, less artefacts and much better interpolated picture quality than with MPC/MadVR so I can never go back. With a 3070 laptop GPU you are always going to have issues but maybe with MPC it will work better for you.
Note: Edited to say "unusable" -> I moved away from MPC (with MadVR) because it was "unusable" with Rife.
For the ones above, after testing yesterday, the quality with RIFE that comes with SVP is better than the v2 I updated to via the links above.
Current RIFE with SVP shows less artifacts for example on intros when names are showing up, and smoother overall.
rife_V2 is creating artifacts and the quality seems to drop as well on 4K content looking like 1080p, at least with my MPV settings.
Forgot to back up my vsmlrt.py and had to remove RIFE and add it back. Now picture quality and smoothness is back to being better.
By the way @Chainik, if it's possible to add a fourth option under maintenance called "Check File Integrity". So, it checks all the SVP files and replaces them with the default ones from SVP. Overriding any changes we make to RIFE or MPV folder, etc.
I think the v1/v2 debate may be dependant on local environment because I went back to v1 and didn't notice any improvements over v2. If anything it was slightly worse. Then I tried ensemble and the issues with the star field in Gravity became much worse. This scene seems to be Rife's Kryptonite. The same scene on Automatic has no issues.
I could have done with this info a few weeks ago But at least it made me go deeper and look into the code itself which has helped.
12. The thing here that confused me is that you don't have to select the V2 model specifically. It seems like SVP chooses the V2 model automatically if the folder structure exists like that. So you'll be using the non-ensemble V2 rife 4.6 model by default if you have your thing set up like mine.
Yes I did comment on this a few weeks ago but I got not reply and in general I haven't been getting any replies from the devs. There is a single piece of "wrapper" code for all models. I'm not a Python person but it looks like it defaults to Rife V2 for the Tensor backend if the Rife v2 folder and onnyx models exist. Otherwise it defaults back to Rife V1.
I don't need it to be any better for smoothness and as I said before, it is mostly way better for artefacts. But I just wish there was some way to adjust for artefacts because it does sometimes introduce it's own kind of artefacts.
UHD wrote:If anyone is interested in much, much better quality than RIFE interpolation I will write about it soon, here on this forum. Please give me another week, maybe two weeks, until I update my repository on GitHub.
This is very exciting by the way. I don't know how anything can be better than RIFE, basically RIFE with better Artifact Masking for me is the ultimate goal (just artifact masking getting better and better). Because smoothness wise, RIFE is already insane, so I can't imagine better. Maybe efficiency? Because RIFE is very demanding.
I agree. Smoothness is practically a non issue now because Rife is so smooth and SVP with Rife has much fewer artefacts. But artefacts are still noticeable and in some cases Rife introduces artefacts that weren't there before. Like the one in Gravity which looks like someone pulling a dark coat a cross the stars in the background.
Cleaned further (removed redundant settings for gpu-next), changed tone-mapping options to auto for better accuracy, removed all ewa_lanczossharp and changed them with ewa_lanczos as gpu-next makes the former useless and waste more energy for no reason.
Below configuration now runs better (performance improved compared to previous configurations):
In gpu-next, lanczos is basially the same as lanczossharp in GPU.
SmoothVideo Project → Posts by dawkinscm
Powered by PunBB, supported by Informer Technologies, Inc.
