Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79 wrote:

TCmullet
Your video is on my hard disk. Thank you for the link.
I will use it for debug GameDropFix_v5 wink
It will take some time.

Thank you for getting it and telling me here.  I can understand lots of reasons why it might take a lot of time (both internally to the work and externally in that you have a life).

I propose some further work in connection with this.  I had not considered suggesting this in the past as I did not anticipate delivering a full video file to you.  But now that you have one, may I suggest that we work together to address something that has bothered me from day 1?  There are many times that GDF4 corrects a dupped frame incorrectly.  The part I could do for you is to locate (using this big video) a number of instances where it is incorrect (along with brief description by me of why I believe it is incorrect).  I have overlooked it because (1) I didn't think we could ever correct it, and (2) even when it generates objects incorrectly, it (2a) moves the background correctly and (2b) enduring these errors is much better than having no correction at all.  But maybe we can now attempt correction, now that I have shared a full video.  We'd have to be using the same script as basis, so I will package a zip file containing all my .avs and .avsi that are involved in this video, and upload it here.  (When I first started using GDF4, I was operating on huge temp files that had to be deleted.  But now I'm working with more "compact" Mp4 files that I intend to keep.  For all the many output files I've created using GDF4, I *am* interested in recreating them if we fix the mem. leak AND get the logic anomalies (wrongly fixed frames) solved.

Are you willing to receive from me small groups of frame numbers that I use GDF4 debug mode to identify for you?  Also I'd need to give you my main script and the few .avsi involved.  (My script does lots of chopping out frames, so we'd want to interrupt it near the end, cutting off items that would hinder our search.  You'll understand when you see the script again.)

152 (edited by TCmullet 29-01-2016 17:52:58)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Here is my tiny zip file of the several zipfiles.  I realize you don't need Interframe, but as my script uses it, I include it.  For our tests though, I envision us chopping off all of the script after GameDropFix4.
http://www.tomsgoodfiles.com/DaveP/TCMu … cripts.zip

153

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

TCmullet
No file by your link: File not found (404)

I'd like to make GDF5 but had lack of time this weak. I think next weak will be more productive. wink

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79 wrote:

TCmullet
No file by your link: File not found (404)

I should have suspected there'd be a problem if I allowed an apostrophe to be in the file name.  When I tested the link, I didn't go as far as saving it, but believed if I got a save dialog, it would work.  No, it gave me a 404, too.  Had to rename it on the server to not have apostrophe, and respell the link.  It now woriks.  (Maybe bbcode has a way to allow, but didn't want to research it.)  Link now works.

155

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

*GameDropFix, version 5*

# GameDropFixV5.avsi
# ==================
# release date: 2016-02-10, MAG79
# ____________
# Requrements:
# - avisynth 2.6 MT, http://forum.doom9.org/showthread.php?t=148782
# - masktools2 b1, https://github.com/tp7/masktools
# - mvtools 2.5.11.3, http://avisynth.org.ru/mvtools/mvtools2.html
# - svpflow1/2 from SVP4, https://www.svp-team.com/wiki/Plugins:_SVPflow
# - MS VC++ 2012 Redistributable x86, https://www.microsoft.com/en-us/search/ … p;form=dlc

function GameDropFixV5(clip c, bool "myGPU", bool "myDebug", int "myErrSize", bool "DblFPS")
{
myGPU = default( myGPU, true )
myDebug = default( myDebug, false )
myErrSize = default( myErrSize, 12 )
dblFPS = default( dblFPS, false )

New in this version 5:
- modified to work with avisynth 2.6;
- speedup by using masktools2 b1 for avisynth 2.6 and svpflow1/2 (from last SVP4 version);
- internal modifications to get more stability and less memory use: no more memory leaks;
- new DblFPS parameter to fix dropped frame and double framerate with high quality in one pass;
- full length of clip are saved now without truncation at the end.

Usage:
- to fix dropped frames with GPU:

GameDropFixV5()

- to fix dropped frames with GPU and double framerate by insert interpolated frames:

GameDropFixV5(DblFPS=true)

- to fix dropped frames with debug info:

GameDropFixV5(myDebug=true)

Sample script:

SetMTMode(3,4)
DirectShowSource("video.mp4")
SetMTMode(2)
GameDropFixV5()
Distributor()

Download:
- GameDropFixV5_avsi.zip - only avsi-file
- GameDropFixV5_pluginsBundle.zip - whole plugins folder: avsi-file + masktools + mvtools + svpflow

Post's attachments

GameDropFixV5_avsi.zip 3.23 kb, 191 downloads since 2016-02-10 

GameDropFixV5_pluginsBundle.zip 994.7 kb, 263 downloads since 2016-02-10 

156

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

TCmullet
In your script I did 6 modifications:
mod1. added distributor() to the end of script. It lets script to use multi-thread processing. Significant speedup in AvsPmod and player. For x264 and AVSMeter it is not necessary
mod2. added SetMTMode(1) just after AudioDub(video,audio) to to less memory use
mod3. moved AddBorders(0,0,0,8) just after GameDropFix line. It gives more quality to interpolated frames
mod4. change global threads=4 to global threads=7 to use more CPU to get more speed
mod5. use GameDropFixV5 instead of GameDropFix4
mod6. change Interframe call to DblFPS=true parameter inside GameDropFixV5

Script before changes: GDFv4_speed.png - 59 fps
Script after changes: GDFv5_speed.png - 84 fps

Post's attachments

GDFv4_speed.png, 14.28 kb, 789 x 487
GDFv4_speed.png 14.28 kb, 327 downloads since 2016-02-10 

GDFv5_speed.png, 14.18 kb, 789 x 487
GDFv5_speed.png 14.18 kb, 332 downloads since 2016-02-10 

157 (edited by rainlys 23-02-2016 15:14:58)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

158

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

rainlys
Your sample is just telecined. 24-in-the-30 fps. You need to make decimation. IVTC.
You can use TIVTC plugin and simple script to restore 24 fps from 30 fps:

TDecimate(cycle=5)

159 (edited by rainlys 23-02-2016 15:14:51)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

160

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

rainlys
How many 24 fps cartoon scenes in that "hybrid" 30 fps video?

161 (edited by rainlys 23-02-2016 15:14:45)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Couldn't you cheat and just make a variable framerate video with both 24fps and 30fps without telecine?

Of course, I have absolutely no idea on how to achieve such a thing...

163 (edited by rainlys 23-02-2016 15:14:36)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

164

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Hi! I tried to read this thread, and if I understood correctly it detects dropped frames and replaces them with interpolated ones.

Would there be a way to tweak also frames that are not dropped? I'm getting incredibly smooth results, but then there are also "lagged" frames (captured with Open Broadcaster Software) that are just before or after the position where they actually should have been.
E.g. frame 28 should in fact have been frame 27 or the other way around. I suppose the way to do it would be to analyze the optical flow and detect sudden (unwanted) changes in the speed to find these bad frames.

Secondly, some feedback: My graphics driver is old so I had to disable GPU to prevent crash. I was also forced to set thread count to 1, otherwise I could see glitches in the output, the frames would be mixed up a bit about once every second.
It also appears to have some problems at the start of the video, lots of glitches there.

165

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

cores
You are right. GDFv5 script consist of analyse stage and frame-recovering stage. Separate frames from-the-future and frames from-the-past are detected and must be fixed correctly. You can control it by parameter myDebug=true.
You can see frames marked from '-fix50' to 'fix125'. It is mean -50% from source (back to the past) and +125% from source (go to the future).
Usual dropped frames must be marked 'fix50'. It is mean from the futire in the middle from current frame and next frame to avoid jump after dropped frame.

166 (edited by TCmullet 13-06-2016 17:08:01)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Thanks SO much for all this work, Mag!  I've just now been able to start to tackle usage.  I didn't even download more than the avsi (and change v4 to v5), and my script works!  (The script opens in Vdub and I can scrub forward a bit.)

The one item I am worried about is that when I first started using InterFrame, I was only able to get it to work with any degree of meaningfulness by using the Tuning="Film" setting.  I don't see how you are setting that in the case of DblFPS=true.  Could you please confirm that if I set DblFPS to true, that it will invoke the InterFrame logic with Tuning equal to "Film"?   (I will be very happy to now be able to leave out the InterFrame call in all cases where doubling the frame rate is exactly what I need.)

167 (edited by TCmullet 14-06-2016 17:46:36)

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79, in addition to my question above about the Tuning="Film" feature, I have to report some major problems.  These are on the very first file I'm tackling after the NH-v-FL file I supplied earlier.

My new file and script is here:
http://www.tomsgoodfiles.com/DaveP/2015 … 3.Xraw.mp4
http://www.tomsgoodfiles.com/DaveP/TCMullet's-GDF-test-scripts2.zip

1.  Vdub was blowing up for various strange reasons, an often one being, "An out-of-bounds memory access occurred in module 'svpflow1'...  ...reading address 17e8fced."  When I backed out the SetMTMode(1) just after AudioDub(), the problem cleared up.  (At least I was able to go forward.)

2.  The prior file was one of many of mine that did not need resizing.  This one is one of many that do.  (It is 1136x640.)  I use the SR() function as, SR(1280,720) after GameDropFix, as you can see in the script for the new video (WichSt-v-MoSt).  The SR is Super-Res, a moderately-priced licensed product that I've used as needed for several years, and is available here:
http://www.infognition.com/super_resolution_avisynth/
You can use it for free, but the result has visual artifacts, which are probably okay for these tests.

As I build the script, (adding functions, cutting out commercials), etc., I get down to where I experiment with GameDropFix.  Once that is working, I add (uncomment) my next item, which in this case is the SR() function.  The system HANGS.  The Vdub opens the file, but I cannot scroll forward even 1 frame before it hangs.  If I leave out the SR, it works.  If I leave out the GDFv5, it works.  If I include the SR and revert GDF to GDFv4, it works.  I have no idea why I cannot use GDFv5 and SR() together.  It just hangs with no error message of any kind, til I cancel it.  GDF5 must be doing something funky.  Would you please download my files (above) and try it out?

168

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

TCmullet
Can't download file TCMullet's-GDF-test-scripts2.zip.

File not found

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79 wrote:

TCmullet
Can't download file TCMullet's-GDF-test-scripts2.zip.

File not found

Rats.  I forgot that apostrophes don't come through.  It should work now.  (Welcome back, friend.)

170

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Thanks. It works. I got all your files. I will look them.
What Preset you using in InterFrame with "Film" tuning?

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79 wrote:

Thanks. It works. I got all your files. I will look them.
What Preset you using in InterFrame with "Film" tuning?

I don't know what you mean.  "Film" IS the preset, the value I give to "Tuning=".  See my Interframe line at the end of the script 2015-10-31.wv.WichSt-v-MoSt.x264v-q20.0720p60.pcm.avs.  Is there some place OTHER than function arguments where something with regard to Interframe gets set?

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

MAG79 wrote:

What Preset you using in InterFrame with "Film" tuning?

I repeat, your question doesn't make sense to me.  The line to call Interframe is:

Interframe(Cores=4, Tuning="Film", GPU=true)

So "Film" is the value I set for "Tuning".  The parameters to Interframe are the only way (to my knowledge) to send info *into* Interframe.  Those parameters (3 in this case) are what I "preset" Interframe to.

As to the absence of further action by either of us these several months, (1) I don't know why you've not responded.  And (2), my video circumstances have changed to where most of the files I deal with do NOT need the SR() tool to smart-resize to a larger size.  But I still do have many files where I must use both GDF and SR(), and GDFv5 is still crashing or hanging the system ONLY when SR() is included.  You can use SR() for free, as the artifacts shouldn't affect this test for crashing.

To get around this problem I have to do something very aggravating.  I have to revert to GDFv4.  But when I do that, I cannot run my whole video (1.5 to 2.5 hours long) through Avisynth/Vdub processing in one piece or it will crash due to the memory leak problem of GDFv4.  But it is very tedious to have to break up my video into 20 min. pieces, then after creating the several AVIs, concatenate them back together.  It would be so nice if we could find out why GDFv5 is crashing whenever SR() is present.

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

Uh, I think I should hasten to add that when this occurs, Interframe is not even being used, as I comment out everything after SR() for this testing.  Therefore Interframe is not being used.  (I constantly add and take away "__END__" lines during script building/testing.)

174

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

GameDropFix, version 5.01

# GameDropFixV5_01.avsi
# ==================
# release date: 2016-12-20, MAG79
# ____________
# Requrements:
# - avisynth 2.6 MT, http://forum.doom9.org/showthread.php?t=148782
# - masktools2 b1, https://github.com/tp7/masktools
# -- MS VC++ 2012 Redistributable x86, https://www.microsoft.com/en-us/search/ … p;form=dlc
# - mvtools 2.5.11.22, http://avisynth.org.ru/mvtools/mvtools2.html
# - svpflow1 4.0.0.132 from SVP4, https://www.svp-team.com/wiki/Download
# - svpflow2 4.0.0.136 from SVP4, https://www.svp-team.com/wiki/Download

function GameDropFixV5(clip c, bool "myGPU", bool "myDebug", int "myErrSize", bool "DblFPS")
{
myGPU = default( myGPU, true )
myDebug = default( myDebug, false )
myErrSize = default( myErrSize, 12 )
dblFPS = default( dblFPS, false )

New in this version 5.01:
- updated to last versions of plugins: MVTools 2.5.11.22, SVPFlow1.dll 4.0.0.132, SVPFlow2 4.0.0.136;
- no script code changes.

Usage:
- to fix dropped frames with GPU:

GameDropFixV5()

- to fix dropped frames with GPU and double framerate by insert interpolated frames:

GameDropFixV5(DblFPS=true)

- to fix dropped frames with debug info:

GameDropFixV5(myDebug=true)

Sample script:

SetMTMode(3,4)
DirectShowSource("video.mp4")
SetMTMode(2)
GameDropFixV5()
Distributor()

Download:
- GameDropFixV5_01_avsi.zip - only avsi-file
- GameDropFixV5_01_pluginsBundle.zip - whole plugins folder: avsi-file + masktools + mvtools + svpflow

It is recomended to install MS VC++ 2012 Redistributabe Update 4 (x86) for better work of MaskTools2 plugin.

Post's attachments

GameDropFixV5_01_avsi.zip 3.19 kb, 162 downloads since 2016-12-20 

GameDropFixV5_01_pluginsBundle.zip 1.01 mb, 206 downloads since 2016-12-20 

Re: Fixing dropped frames in action cam footage? (for stereoscopic 3D)

In another thread, I learned that there is a thing called Avisynth+ which handles all multithreading very differently than "setmtmode" calls did.  I was going to switch to the new + version of synth, but then discovered that I needed GameDropFixv5 in the script.  AND the system was very prone to alert me to the fact that GDF itself calls some setmtmode things.  The depth of setmtmode knowledge that I have (which is extremely small) makes me wish to ask you if, when you find it convenient, you could revise it to GameDropFixv6, to be used in Avisynth+ scripts.  Would you be willing to do that?  Is it even possible?