You are not logged in. Please login or register.
SmoothVideo Project → Posts by madshi
Pages 1
Well, I could output YV12 to SVP, that wouldn't be such a big problem for me. But then of course madVR would have to upscale chroma *again* after SVP. So it's all not ideal. Anyway, let's not get ahead of ourselves. The first step would be to add a Vapoursynth frame server to madVR, which sits in front of madVR's GPU rendering.
Well, if you upscale before SVP, you do increase the amount of work SVP has to do. So while it would probably allow you to use higher settings in madVR, you might bring SVP into trouble. There's not free lunch, unfortunately. But theoretically it would be possible to offer both options.
does this even matter if madshi includes avisynth/vapoursynth in madVR? If we can use SVP inside of madVR, we can just let madVR deinterlace, which in my opinion has better deinterlacing than any single avisynth script out there.
Are you talking about telecined movies (IVTC / forced film mode) or about native interlaced content (sports, DXVA deinterlacing)?
I suppose my film mode is pretty good. But I always thought that something like QTGMC would beat the hell out of DXVA deinterlacing?
LAV decodes -> madVR deinterlaces -> madVR upscales -> SVP interpolates -> madVR does the last steps of rendering.
That would involve copyback. Not impossible, but I'm not sure how good that would perform.
About performance: SVP itself is optimized for real-time usage, right? The only other thing eating performance for motion compensated deinterlacing would be NNEDI3. I'm not sure how many neurons QTGMC uses by default for NNEDI3. Running maybe 32 NNEDI3 neurons via OpenCL is more than real time on a decent GPU today even for 1080i60 content. Ok, so copyback would have to be used, but LAV Video Decoder's latest copyback algorithm seems to be very fast and efficient. So I think this way doing all this in real time should be possible. Maybe some compromises on settings would have to be used. If all else fails, super-xbr could be used instead of NNEDI3, which is a magnitude faster and has very good edge interpolation quality, too.
Of course it's all your decision, Chainik, if you don't like the idea, I fully understand. You probably have enough work your hands as it stands. Just thought it might be something worth looking into because SVP already does most of the work necessary for this.
I'll be looking into Vapoursynth when I find some time. LGPL makes all the difference for me.
Pretty much all broadcasts here in Europe are 576i50 or 1080i50. I think also in the USA many broadcasts are still 480i60 or 1080i60. So while deinterlacing is a thing of the past for Blu-Rays, UHD Blu-Rays and custom encodes, it's still very very important for TV watching, especially live sport events (e.g. soccer and stuff). GPU HW deinterlace is decent quality, but can't really compete with good motion compensated deinterlacing.
I'm not sure how easy it will be for madVR to access motion vectors from "inside" of AviSynth. I'll see if I can get access to that. If that doesn't work, can we maybe talk about alternative APIs/interfaces?
P.S: Just to explain. Motion compensated deinterlacing done by SVP would work something like this:
-------
1)
Split the video into fields (e.g. 60 fields per second for NTSC). Field X now has the odd lines set and the even lines are missing. Field X+1 has the even lines set and the odd lines are missing.
2)
Use a good interpolation algorithm, e.g. NNEDI3, to interpolate the missing scanlines of each field, so each field is turned into a full frame.
3)
Now run SVP on 2) and recalculate every frame, using SVP. No new frames need to be added, just every existing frame needs to be recalculated, using the neighbor frames.
4)
Now create the final frames, by combining the known "good" scanlines from the original fields, and inserting the missing scanlines by using the results of step 3).
-------
There are 2 variants of this processing pipeline:
a) The simple variant would be for step 3) to use frames from step 2).
b) The more complicated, and probably slightly higher quality variant would be to run 3) and 4) "interleaved". Meaning that if step 3) wants to recalculate frame X, it would use frame X-1 from 4) and frame X+1 from 2).
-------
Variant a) might be possible to implement externally, if SVP could be used as a "toolbox". Variant b) would probably be hard to realize efficiently that way, so if we want to have Variant b), it would probably have to be implement inside of SVP.
Hi there,
a wild idea: SVP has to fully understand the whole motion situation from one frame to the next, in order to do its job, right? So I'm wondering if it would be feasible to make use of all the algorithms and knowledge in SVP to implement motion compensated deinterlacing as part of SVP?
How do you handle interlaced anime for example, anyway? Do you currently require deinterlacing to be performed before SVP? I think if you moved deinterlacing into SVP, that could produce a nice quality/performance improvement, even over the best currently available AviSynth deinterlacing scripts, or what do you think? I mean the best algos out there for deinterlacing use their own motion interpolation, but I suppose what SVP does it probably superior to those in terms of quality vs. performance ratio.
Just a wild thought, though. If you don't like the idea, just toss it. I'd love it, though. Have been wishing for high quality motion compensated deinterlacing in madVR for a long time.
Another thought: Would it be hard for SVP to "export" motion information for other filters (or e.g. madVR) to reuse? E.g. debanding, denoising, even sharpening etc algos might be tuned to benefit from knowing which part of the video moves where from one frame to the next. It might even be possible to "misuse" SVP to simply analyze a video sequence, without actually letting SVP modify it, just to use the motion vectors for other purposes. E.g. if you don't want to add deinterlacing, it might be possible to tell SVP to analyze a video sequence, then gather the detected motion information from SVP, and then use that to implement motion compensated deinterlacing externally. Basically you could extend SVP into an open motion toolbox which other software could use for all kinds of algorithms.
What do you think?
Sorry for the very late reply. Somehow the forum notification about the new posts didn't seem to reach me.
We could ask you to add Vapoursynth processing into madVR
You know, many people dream of SVP integrated into madVR...
I'm absolutely open to that. Not right now, too busy with other stuff, but sooner or later. Would you mind sending an email to my email account (madshi (at) gmail (dot) com), so that when I'm ready to look into that, we can talk via email, if that's alright with you? I think that would be more efficient than using the forum.
Yes, but if you *could* get access to the MVs from the stream, would you consider it useful to improve SVP performance? That's the key question. It might save some performance, or not?
I know that you currently don't have access to the video bitstream, but it's not impossible to get access. Here are some ways that could be achieved:
1) Obviously, you could integrate a decoder. Shouldn't be that hard, libav/ffmpeg does all the heavy lifting. But it's still somewhat ugly. So the other solutions might be better.
2) For DirectShow playback, you could ask nevcairiel to add an interface to LAV Video Decoder that allows you to get access to the bitstream / MVs.
3) For DirectShow playback, you could hook the decoder's input pin's "Receive" method. Not that difficult. madVR already does that to get some added information from the video bitstream.
I know, it's not an easy solution. Interpreting the video bitstream to extract the MVs might already require many hours of development, and that doesn't even include getting access to the bitstream in the first place. And when you do get access to the bitstream, there's still the problem of matching the bitstream to the decoder output. And maybe the bitstream MVs don't provide very good motion information. But I just thought I'd drop the idea here, in case it might help SVP to improve its quality/performance ratio.
Hey guys,
two ideas:
1) Have you thought about making use of the motion information from the video bitstream? Obviously this information is missing for key frames, and it's probably not completely reliable. But it might serve as a first guess, or as a low CPU variant.
2) I've seen Didée (doom9) comment that some of the AviSynth algos don't really track motion, but just try to find a block match. Is that the case for SVP? Or is SVP actively trying to find *continuous* motion vectors over multiple frames?
Maybe both 1) and 2) could be combined? E.g. as an ultra low CPU/GPU power algo, you could rely on the video bitstream motion information, and refine that by tracking motion vectors over multiple frames. E.g. let's say that block A moves 5 pixels right from frame X to frame X+1, and another 5 pixels right from frame X+1 to X+2. And let's say frame X+3 is a key frame. So motion information is missing for the key frame. So SVP could just "continue" all previous motion vectors in the same speed for the key frame. Maybe it would make sense to do some block comparisons just to make sure that continuing the motion vectors seem "likely" for the key frame, just to be safe. Or maybe SVP could run its usual algos just for key frames, but rely on the video stream motion vectors for the other frames? It might make sense to ignore "random" motion vectors which are not continuous, and just use those which stay similar over at least 3 frames or so.
Just some thoughts I had...
Pages 1
SmoothVideo Project → Posts by madshi
Powered by PunBB, supported by Informer Technologies, Inc.