1 Topic by Mystery (edited by Mystery 28-03-2017 19:35:45)

  • Thanks: 33

Topic: How does artifact masking work

We're investigating how to optimize frame interpolation for quality instead of for real-time processing in this thread.
https://forum.doom9.org/showthread.php? … ost1800439

This code does give significantly better results than SVP/Interframe ... *when* it behaves.

function jm_fps(clip source, float "fps")
{
fps = default(fps, 25.000)
fps_num = int(fps * 1000)
fps_den = 1000

prefiltered = RemoveGrain(source, 22)
super = MSuper(source, hpad = 16, vpad = 16, levels = 1) # one level is enough for MRecalculate
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16) # all levels for MAnalyse
backward = MAnalyse(superfilt, isb = true, blksize = 16, overlap = 4, search = 3, dct = 0)
forward = MAnalyse(superfilt, isb = false, blksize = 16, overlap = 4, search = 3, dct = 0)
forward_re = MRecalculate(super, forward, blksize = 8, overlap = 2, thSAD = 100)
backward_re = MRecalculate(super, backward, blksize = 8, overlap = 2, thSAD = 100)
out = MFlowFps(source, super, backward_re, forward_re, num = fps_num, den = fps_den, blend = false, ml = 200, mask = 2)

return out
}

This has no artifact masking, and produces considerable artifacts.

My question is: how does artifact masking work in SVP, and how could it be added to this script?

Then I'm thinking, artifact masking could fall-back on frame blending instead of disabling frame interpolation altogether.

Then, this algorithm most likely will still will result in some serious artifacts that must be removed manually (replacing these frames with a clip using frame blending).

Re: How does artifact masking work

I don't see anything special in your script except for feeding Analyse with RemveGrain-ed source.
Did you tried to do the same with SVP libs?

> My question is: how does artifact masking work in SVP, and how could it be added to this script?

http://avisynth.nl/index.php/YFRC

  • Thanks: 33

Re: How does artifact masking work

I did comparison here. Very notable quality improvement.
https://forum.doom9.org/showthread.php? … ost1802157

  • Thanks: 33

Re: How does artifact masking work

Chainik, back to that thread on Doom9, how would I go about applying the same settings? SVP and MvTools have a completely different syntax, I can't easily switch between one and the other.

Re: How does artifact masking work

I'm not saying it's easy big_smile

  • Thanks: 33

Re: How does artifact masking work

Does MVTools provide necessary features for artifact removal, or that's part of SVP's custom code?

Re: How does artifact masking work

have you read that "YFRC" sources? hmm

  • Thanks: 33

Re: How does artifact masking work

Oh sorry forgot you had sent me this already.

Is this the same method you use?

  • Thanks: 33

Re: How does artifact masking work

Chainik, you might want to investigate this. You might have overlooked a setting that is considerably lowering the quality.

.. but as you say, finding it might not be so easy.

Re: How does artifact masking work

It would be great if there's a little tweak somewhere that can improve the quality of masking. Would welcome such a change with open arms. I use "strong" artifact masking for all content, so I would love it if there's improvements that can be implemented.

Maybe add a new sub-menu in SVP Control Panel, under Artifact Masking, add "Enable Experimental Masking" with options (Enable/Disable). For us to test.

Re: How does artifact masking work

> Is this the same method you use?

this's the idea
mask resulting frame with two source frames in "bad" areas, where "bad area" means "area with high SAD value of the motion vectors"

"external" implementation requires 10 lines of avisynth code and is slow as hell
"internal" implementation requires just another multiply-add operation for each frame pixel

  • Thanks: 33

Re: How does artifact masking work

So, SVP implemented it internally (fast), and with MVTools, it has to be done with 10 lines of code (slow)?

Re: How does artifact masking work

yep

  • Thanks: 33

Re: How does artifact masking work

I haven't figured out what's wrong with SVP, but this script has HUGE improvements over SVP.
https://forum.doom9.org/showthread.php? … ost1804515

Re: How does artifact masking work

as I said before, may be (!) the major difference comes from

prefiltered = RemoveGrain(C, 22)

nothing can stop you from pre-process video before svpflow

=======

prefiltered = RemoveGrain(input, 22)
super = SVSuper(prefilter,"{gpu:1}")
vectors = SVAnalyse(super, "{refine:[{thsad:100}]}", src=prefilter)
smooth = SVSmoothFps(input, super, vectors, "{rate:{num:2,den:1}, algo:21, mask:{area:50}}", mt=threads)

=======

 BlkSize    = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)

you know you'll get significantly different results for the same video scaled to different sizes, right?

  • Thanks: 33

Re: How does artifact masking work

Chainik wrote:

as I said before, may be (!) the major difference comes from

prefiltered = RemoveGrain(C, 22)

No, removing that only did an extremely minor difference.

Chainik wrote:
 BlkSize    = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)

you know you'll get significantly different results for the same video scaled to different sizes, right?

What do you suggest?

17 Reply by Mystery (edited by Mystery 27-04-2017 22:41:57)

  • Thanks: 33

Re: How does artifact masking work

Here are screenshots of my latest script

Observations:

- The difference between Interframe and FrameRateConverter is HUGE!

- SVP/Interframe always appears softer, as if it was applying some level of frame blending on every frame (I had this issue with scene change mask being 16 for black and 235 for white, which caused unwanted frame blending)

- On dreaded frame 4477, Interframe shows a "hard blending", followed by that same image mixed with ugly artifacts, which brings an original frame followed by gradually increasing artifacts on a still image.

Prefiltering only has a very minor effect. Variance without would be less than 0.01%

Re: How does artifact masking work

Forget Interframe, just take plain SVPflow code  - the one I posted above - and add whatever you want/need.
In that case at least I'll be able to directly compare options in use.

If you want to do a fair comparison that is simple to reproduce and analyse, write two plain scripts, w/o any "interframe" or "FrameRateConverter" in use:

1. MSuper(...)
MAnalyse(...)
MRecalculate(...)
MSmoothFps(...)

2. SVSuper(...)
SVAnalyse(...)
SVSmoothFps(...)

Besides what's exactly the purpose of using those 20-yo-320x240-clips in the year 2017? it looks like shit in both cases...

Re: How does artifact masking work

Interframe:
- default mode: algo=13, mask.area=0
- smooth preset: algo=23, mask.area=150

BOTH modes are BAD, at least for this extremely low-quality source
this doesn't mean they're ALWAYS bad, I believe SubJunk made a lot of comparisons when he selected these values

set algo=21/23, mask.area <= 50 and you'll be fine

20 Reply by dlr5668 (edited by dlr5668 28-04-2017 18:11:47)

Re: How does artifact masking work

Mystery

Please repeat tests with 1080p 20+ mbit bitrate AND 2x or 3x frame rate

  • Thanks: 33

Re: How does artifact masking work

Sure

I don't really have 20+ mbit videos, but I have this

function SVP(clip Input) {
    SuperString = "{scale:{up:0,down:4},gpu:1,rc:false}"
    VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:2,coarse:{satd:false,distance:2}},penalty:{lambda:20.0}},refine:[{thsad:100}]}"
    SmoothString= "{gpuid:11,rate:{num:2,den:1,abs:false},algo:23}"
    Super   = SVSuper(Input, SuperString)
    Vectors = SVAnalyse(Super, VectorsString)
    smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, url="www.svp-team.com", mt=1)
    smooth_video
}

https://s22.postimg.org/8sazos73x/diamond-6965-svp.png

FrameRateConverter()
https://s22.postimg.org/yzc6kqpdp/diamond-6965-frc.png

22 Reply by nemoW (edited by nemoW 01-05-2017 08:27:15)

  • nemoW
  • Beta Tester
  • Offline
  • Thanks: 47

Re: How does artifact masking work

I like Mystery's script results. However it is worse than SVP on repetitive patterns:

AviSource("telnyashka.avi")
StackHorizontal(FrameRateConverter(),SVP())

https://s6.postimg.org/cjrp56srh/teln.png

Re: How does artifact masking work

Kind and number of artifacts are the same for both MVTools and SVPflow using comparable options, i.e. the same block sizes, search type and radius, with the only difference in shapes of those artifacts.
But I agree that in some cases (like in the example above) SVPflow produces more "blending", probably because of some nasty bug hmm and there's definitely a room for improvements here.