Re: Why does SVP sometimes get slow?

It happened again. This time, however,I also saw hat the CPU was only running at 50% while it was dropping frames. And then it would get back to normal. And then lag again.

Re: Why does SVP sometimes get slow?

Did toggling the GPU acceleration "fix" it?  If so, then that makes sense since that'd point to the poor performance being due to the GPU which won't show up as high CPU utilization.

Re: Why does SVP sometimes get slow?

This is really getting annoying. For now I'm back to AviSynth 2.5 because it happens less than with v2.6, and copy back v2.6 whenever I need to do other work with AviSynth. Switching back and forth like this isn't ideal.

Also, the performance improvements are kind of meaningless when it only applies for "some" content and it occasionally lags like this. The performance settings have to be adjusted to the lowest common denominator, which means that even if a single video doesn't benefit from the 40% improvements, none of the others of similar resolution can enjoy increased settings either.

Perhaps this issue is showing up more for me because I tweaked my settings to keep the CPU between 80-100%. If it was running at 50% (as default), then I wouldn't notice these lags as much.

Re: Why does SVP sometimes get slow?

Hmm that sounds really weird, since after I upgraded to 3.1.7 I can't seem to get more than 50% CPU utilization even if I completely max out all the normal menu options for 60Hz interpolation. Interpolating to 144Hz does "solve" the "issue" for me, but I wonder why you aren't seeing constantly less CPU usage than with 3.1.6.

Would you mind posting your full system specifications and your SVP settings?
You can also try to change the "Index&CPUControl" value from 1 to 0 (under "Hidden settings") to see if it makes any difference altough I wouldn't recommend keeping it disabled for normal usage.

With regards to your Avisynth version, the SVP developers have (as I understand it) explained in some other threads that 2.5.8 "SVP edition" does indeed provide increased performance in the majority of cases. Useing 2.6 could be makeing some existing problem worse?

30 (edited by Nintendo Maniac 64 18-07-2015 06:00:53)

Re: Why does SVP sometimes get slow?

xenonite wrote:

Hmm that sounds really weird, since after I upgraded to 3.1.7 I can't seem to get more than 50% CPU utilization even if I completely max out all the normal menu options for 60Hz interpolation.

But what is your exact CPU model?

Re: Why does SVP sometimes get slow?

Nintendo Maniac 64 wrote:

But what is your exact CPU model?

Intel 5960X @ 4.6GHz (I also have it in the "System Specs" area of my forum profile), why do you ask?

Do you think the performance improvement is maybe only for newer processor architectures?
Seems kinda unlikely to me as we are still stuck with only SSE2 optimizations...

32 (edited by Nintendo Maniac 64 18-07-2015 07:03:14)

Re: Why does SVP sometimes get slow?

xenonite wrote:

Intel 5960X @ 4.6GHz (I also have it in the "System Specs" area of my forum profile), why do you ask?

Uhhh... holy bananas, you literally have the fastest consumer CPU model in existence combined with a considerable overclock.  I'd like to point out that, if I recall correctly, Mystery "only" has a Nehalem i7 (though I obviously could be wrong).


Remember, SVP is very multi-thread friendly, so it's actually one of the cases where "moar cores!" actually gives quite the performance benefit, and an 8-core/16-thread Haswell E CPU is the current king of the crop when it comes to that kind of thing in the consumer marketplace.

Re: Why does SVP sometimes get slow?

Lol, that might be so, but I was actually trying to figure out why Mystery experiences SVP 3.1.7 as not being universally faster than 3.1.6.
I can understand my system being able to handle higher settings than his, but 3.1.7 should also be faster for him shouldn't it?

Re: Why does SVP sometimes get slow?

Well, I have come across one YouTube video that does not seem to be particularly faster on SVP 3.1.7 compared to 3.1.6 - particularly the first minute or so of this video when it's zooming in and out:
https://www.youtube.com/watch?v=wOP8XudHqr4

Re: Why does SVP sometimes get slow?

With 3.1.6, I was still using updated libraries so upgrading to the latest package didn't give improvements in itself.

And yes, I have "only" a quad-core i7 3630QM, which is still near the top of this list after 2 years with a score of 7638
https://www.cpubenchmark.net/laptop.html

Re: Why does SVP sometimes get slow?

Ok, so it's an Ivy Bridge i7 instead.  Nevertheless, that's still half the amount of cores and threads that Xenonite has and is around 25% slower per-core performance before factoring in the architectural improvements and larger cache of Haswell-E.

Re: Why does SVP sometimes get slow?

Mystery wrote:

With 3.1.6, I was still using updated libraries so upgrading to the latest package didn't give improvements in itself.

That actually explains it. If I change the libraries to the new ones then I also get identical performance between 3.1.6 and 3.1.7, which makes sense since I believe all performance inprovements should come from the dll libraries (the SVP manager should not use any substantial amount of resources). Also, if there were any architecture-specific improvements, I believe that Mystery's Ivy-Bridge based processor would also have benefitted from it as Ivy-Bridge also supports the AVX2 instruction set.

I also downloaded and tested that Youtube video after I disabled 4 CPU cores in the BIOS and backed off the "threads" setting to better aproximate more "mainstream" systems, and I still seem to get around a 10% performance improvement (lower total rendering time) with the new libraries.

I think that the reduced performance for some scenes can best be explained by SVP having to do more work to find suitable motion vectors when there are many different moving objects having large motion displacements, as compared to still or slowly moving frames.
If I change my SVP settings to those more suited to interpolating Anime content, SVP's performance and quality on that video also improves quite markedly.

Ps. You can tweak this scene-dependant adaptive behaviour by changeing the "badSAD" values and altering how far and with what algorithm SVP should search for replacement motion vectors; however the default values seem to be optimal for most mainstream 4-core machines and videos.

Re: Why does SVP sometimes get slow?

xenonite
what settings do you change for anime ?

Re: Why does SVP sometimes get slow?

Mystery
I tweaked my settings to keep the CPU between 80-100%

not a great idea

40 (edited by xenonite 19-07-2015 16:12:46)

Re: Why does SVP sometimes get slow?

dlr5668

TL;DR: skip to the end for my settings.

Well first I'd like to give the reasoning behind my settings. Anime content poses some unique challenges; specifically, only panning scenes or computer-generated content actually have a solid mathematical relationship (where motion is concerned) between past and future frames. The rest of the content, however, is normally hand-drawn at some fraction of the video's frame rate (i.e. only drawing 6 or 12 changed frames per every 24 frames of motion).
Since SVP uses a block-matching algorithm, (and artists re-drawing a character multiple times to create the illusion of motion rarely draw an exact 3D projection every time) very few blocks will match up close to perfectly (unlike the case with 'normal' video). Furthermore, the stop-start nature of the motion (caused by drawing fewer moving frames than the frame rate requires) also makes it hard for SVP to calculate the true motion since as far as it is concerned, that is how the motion is supposed to be. SVP has no way of knowing when two equal frames are the result of a production being too cheap to even draw 24fps motion, rather than simply signaling that the moving object has come to a halt. This leaves us with greatly increased artifacts (mostly in the form of solid lines becoming 'wavey' as shown in The SVP Wiki) or with the motion just not being any smoother, even for slowly moving or rotating objects.
Yes, in theory one can write some sort of algorithm to check for and remove the superfluous frames (while leaving legitimate motion-free portions alone) and then use SVP-framedoubling to interpolate the resulting variable frame rate video, but that is way beyond the scope of dlr5668's question...

As for the SVP settings, we have one of two choices:
1) We can only smooth reliable, global panning motion (produces fewer artifacts but does almost nothing to moving objects or characters)
2) Live with the increased artifacts and try to tweak SVP's settings as best we can for each specific anime's drawing style.

I prefer option 2 (but you can easily adapt my settings for 1 by using a very large 'motion vectors grid' setting, disabling 'decrease grid step' and setting motion vectors precision to one or two pixels).
These are my settings for MOST anime content:

Profile Settings:
Processing Threads: 24 (use less for more stability but less performance or for quad-core CPUs)
GPU-acceleration: Pick my second GPU (One not used by MadVR)
Frames Interpolation Mode: Adaptive (or "2m", depending on the severity of motion artifacts)
SVP shader: 2. Sharp
Target Frame Rate: Source multiplied by 3 (use "To screen refresh rate" for 60Hz monitors)
Motion vectors grid: 14px. Average 1 (for 720p: 8px. Small 9)
Decrease grid step: By two with global refinement
Search radius: Large
Motion vectors precision: One pixel (for 720p: Half pixel)
Wide search: Strongest
Artifacts masking: Weak
Processing of scene changes: Repeat frame
Decrease frame size: Disabled (this setting makes very little sense to me, I suppose it is for compatibility with very slow PCs?)

The following settings are best left to the profile defaults unless you have read through The svpflow dcumentation and The SVP MVTools2 dcumentation and understand what they do (can decrease both quality and performance).

Hidden settings: (SVPMgr.ini, override.js, etc. Very content dependent)
SetMemoryMax(768)
analyse.main.search.distance = -16 or -32
analyse.main.search.satd    = true (better quality but requires much more performance)
analyse.main.search.coarse.width = 1080 (set this >= to your video width)
analyse.main.search.coarse.distance    = -16 or -32
analyse.main.search.coarse.trymany    = (depends, can improve or worsen performance)
analyse.main.search.coarse.bad.sad    = 1000 (750~1250)
analyse.main.search.coarse.bad.range = -32 or -64
analyse.main.penalty.lsad    = 4000 (4000~8000)
analyse.main.penalty.pnew = 40;
analyse.main.penalty.pglobal = 40
analyse.main.penalty.pzero = 75;
analyse.refine[0].thsad = 300;
analyse.refine[0].penalty.lsad = 6000;

Re: Why does SVP sometimes get slow?

xenonite
The following settings are best left to the profile defaults unless you have read through The svpflow dcumentation and The SVP MVTools2 dcumentation and understand what they do (can decrease both quality and performance).

yep, I read. Thanks for detailed answer. This full post can be copy-pasted to wiki  smile

Re: Why does SVP sometimes get slow?

dlr5668
No problem, I'm glad I could contribute something.
And thanks for reading through all of it, I just wanted to ensure people know WHY we change certain settings as opposed to just "higher numbers must be better".  smile

Re: Why does SVP sometimes get slow?

Chainik wrote:

Mystery
I tweaked my settings to keep the CPU between 80-100%

not a great idea

Just for reference sake, what would you recommend?

Re: Why does SVP sometimes get slow?

xenonite

I only watch anime and from my observation, the default setting is better than setting described in this wiki

Frames Interpolation Mode: Adaptive (or "2m", depending on the severity of motion artifacts)
In this case, maybe you can tweak smooth.scene.limit options in override.js,

Target Frame Rate: Source multiplied by 3 (use "To screen refresh rate" for 60Hz monitors)
This certainly less artifact since you have exact integer multiple than the display with 60hz refresh rate


Nintendo Maniac 64

I think the most recommended is, the setting which give the smoothest video while keeping cpu usage downs as much as possible big_smile
for now it's the trade-off between the comfortable smoothness and cpu usage (I emphasized the comfortable smoothness smile )
higher setting which consumes cpu highly is not necessarily better smile

==========================================
Ok, back to topic,
Sometimes cpu is spiked when there's sudden change scene, it spikes really high but not very high to make the noticeable slowdown,
it's like normally ~40% to suddenly ~60-80%.
Since the TS, Mystery, keeps his setting about 80-100%, isn't just normal that he notices the slowdown?

Re: Why does SVP sometimes get slow?

SVP use "adaptive radius" for MVs search
this means that number of blocks comparisons depends on the block's contrast value
detailed and bright scenes will use considerably more CPU than night ones

Re: Why does SVP sometimes get slow?

Chainik
Please correct me if I misunderstood, but surely that search radius, combined with how far the block has moved, also has an impact on CPU usage?
For example, if a block moves further than the current search radius, then a suitable motion vector will not be found and SVP will have to do an additional "refined" search (if the "bad SAD" limit is exceeded), correct?

Ps. Does SVP recalculate the limit for what it considers to be "detailed and bright" areas when only dark scenes are displayed (i.e. as a fraction of the maximum brightness of a block in the current scene) for a number of consecutive frames, or does it continue to consider the entire frame's blocks to be 'unimportant' or 'less visible'?