1 (edited by Nick 3DvB 14-11-2012 18:02:24)

Topic: Rift HMD, black borders lighting and stuff

ok, is the color back bars effect done by SVP internally, I coundn't find any scripts for it?

I can't work out if there is any way to "zoom out" (to add colored black side bars without cropping)

It would also be good to have some control of blending / dithering the hard screen edges somehow.

Hopefully you can added some features needed for the Rift in the a future version.

thanks

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
I coundn't find any scripts for it

really, it's not so hard to find

if there is any way to "zoom out"

by adding lighting on all sides of the frame you mean?  hmm
theoretically you can prcocess video through two SVSmoothFps instances - 1st for horizontal lights and 2nd for vertical (or vise versa), but you've to do something with the corners

3 (edited by Nick 3DvB 14-11-2012 19:33:13)

Re: Rift HMD, black borders lighting and stuff

Thanks, I thought maybe this effect was done with a different avisynth plugin, but now I can see the light: {   smile

I'm glad that it should be possible with two instances, I wonder what is the best way to deal with the missing corners problem?

We might not need colour bars on the sides if we can warp the image inwards (to look like a curved projector screen)

and then use head-tracking for active pan&scan, then maybe just color bar at the top / bottom will work ok,

it is hard to tell at the moment but I should have my Rift soon so I will be able to test it for real.

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
what is the best way to deal with the missing corners problem

well, not "missing"...
http://www.svp-team.com/forum/misc.php?item=1957&download=1

Post's attachments

double-light.png, 334.41 kb, 512 x 389
double-light.png 334.41 kb, 1141 downloads since 2012-11-14 

Re: Rift HMD, black borders lighting and stuff

Another one, different settings
http://www.svp-team.com/forum/misc.php?item=1958&download=1

Post's attachments

double-light-2.png, 192.67 kb, 416 x 379
double-light-2.png 192.67 kb, 1107 downloads since 2012-11-14 

6 (edited by Nick 3DvB 14-11-2012 20:32:10)

Re: Rift HMD, black borders lighting and stuff

Cool, that looks ok!  I was worried we might have 4 black squares!  If we can soften the frame edges a bit that should be fine, maybe we can mirror the bars effect back over the video frame itself, but the mirrored side would need more fading (get much lighter towards the center) but this still would not work in the corners right?  Can you post the usage syntax for the above?  Thanks again for testing it.

smile

Re: Rift HMD, black borders lighting and stuff

svp_scheduler=true
SetMemoryMax(1024)

LoadPlugin("C:\Program Files (x86)\SVP\plugins\svpflow1.dll")
LoadPlugin("C:\Program Files (x86)\SVP\plugins\svpflow2.dll")

threads=11
SetMTMode(3,threads)
ffdShow_source()
assert(width==640 && height==480,"Frame size "+string(width)+"x"+string(height)+" instead of 640x480")
SetMTMode(2)
Crop(88,8,-88,-8)

super_params="{scale:{up:0},gpu:1,rc:true}"
analyse_params="{block:{w:8,h:8},main:{search:{coarse:{distance:-10},bad:{sad:2000,range:24}}},refine:[{thsad:65000}]}"
smoothfps_params="{rate:{num:5,den:2},algo:2,cubic:1,scene:{},light:{aspect:1.6,lights:11,border:16,length:141,cell:4}}"

super=SVSuper(super_params)
vectors=SVAnalyse(super, analyse_params)
last=SVSmoothFps(super, vectors, smoothfps_params, mt=threads, url="www.svp-team.com")


super2=SVSuper("{}")
vectors2=SVAnalyse(super2,"{}")
smooth_video=SVSmoothFps(super2, vectors2, "{rate:{num:1},light:{aspect:1.1,lights:40,border:16,length:141,cell:4}}", mt=threads, url="www.svp-team.com")

smooth_video

Re: Rift HMD, black borders lighting and stuff

In fact you can try it yourself by modifying "Program files\SVP\AVS\MSmoothFps.avs" file:

super=SVSuper(super_params)
vectors=SVAnalyse(super, analyse_params)
last=SVSmoothFps(super, vectors, smoothfps_params, mt=threads, url="www.svp-team.com")

super2=SVSuper("{}")
vectors2=SVAnalyse(super2,"{}")
smooth_video=SVSmoothFps(super2, vectors2, "{rate:{num:1},light:{aspect:1.1,lights:40,border:16,length:141,cell:4}}", mt=threads, url="www.svp-team.com")

Re: Rift HMD, black borders lighting and stuff

Great, I am on a work PC at the moment but will try it out ASAP.  smile

What do you think about the blended mirror overlay idea to soften the edges?

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
What do you think about the blended mirror overlay idea to soften the edges?

i dunno why
try to paint it in photoshop / gimp / etc first smile

Re: Rift HMD, black borders lighting and stuff

Good idea, I will do some tests when I get home, thanks again for all your help so far.

12 (edited by Nick 3DvB 16-11-2012 05:34:04)

Re: Rift HMD, black borders lighting and stuff

Hi, I got it working!  smile   I could not get any side bars for a long time with the script you posted, I am not very experienced with avisynth but in the end I worked out it was an aspect ratio problem, then I had to find a way to over-ride super's smoothfps_params to set both aspect ratios.  It is hard to find the correct balance for both on different video / screen resolutions, maybe you could add some sliders to the GUI in a future release?  I am still doing some tests to find the best way to blur / soften the hard edges, but if you have any ideas on that it would be helpful.  smile

Also, to make the Rift HMD head-tracking work with SVP we will need an interface to control zoom/crop and pan & scan from SVP itself, because we only want to effect the video window, so we can not use the player's controls or this will also include the colored bars.  I posted some quick videos, but they are very low quality, and you will probably think it looks very strange!  hmm  The warping and video aspect ratio are not 100% correct yet but this is actually the kind of output we need to watch "normal" 2D/3D video through the Rift HMD optics:

http://www.youtube.com/watch?v=nFlRHRXdHqw
http://www.youtube.com/watch?v=FU-9D9ATQps

I will also be posting updates in the MTBS thread if you are interested:

http://www.mtbs3d.com/phpBB/viewtopic.p … 825#p86825

thanks
cool

13 (edited by cemnahit 17-11-2012 13:21:17)

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB wrote:

Hi, I got it working!  smile   I could not get any side bars for a long time with the script you posted, I am not very experienced with avisynth but in the end I worked out it was an aspect ratio problem, then I had to find a way to over-ride super's smoothfps_params to set both aspect ratios.  It is hard to find the correct balance for both on different video / screen resolutions, maybe you could add some sliders to the GUI in a future release?  I am still doing some tests to find the best way to blur / soften the hard edges, but if you have any ideas on that it would be helpful.  smile

Also, to make the Rift HMD head-tracking work with SVP we will need an interface to control zoom/crop and pan & scan from SVP itself, because we only want to effect the video window, so we can not use the player's controls or this will also include the colored bars.  I posted some quick videos, but they are very low quality, and you will probably think it looks very strange!  hmm  The warping and video aspect ratio are not 100% correct yet but this is actually the kind of output we need to watch "normal" 2D/3D video through the Rift HMD optics:

http://www.youtube.com/watch?v=nFlRHRXdHqw
http://www.youtube.com/watch?v=FU-9D9ATQps

I will also be posting updates in the MTBS thread if you are interested:

http://www.mtbs3d.com/phpBB/viewtopic.p … 825#p86825
cool


Hi Nick;

Is it Oculus Rift are you talking about, if it is how did you get your hands on with the device (Are you working as a developer) ? I am a backer at kickstarter and i don't know if i will receive it next month.

thanks

14 (edited by Nick 3DvB 18-11-2012 07:47:27)

Re: Rift HMD, black borders lighting and stuff

Yes the Oculus Rift.  I am in the same boat as you - I don't have mine yet either (don't worry from what I hear the delays will be worth the wait!).  No I'm not really a developer, I have played around a bit over the years but I'm more what John Carmack might call "one of the hack / maker crowd".  I've been working on some head-tracking ideas on MTBS3D and should be getting one of the first run of 100 DIY kits very soon.

In the mean time someone is helping me test SVP with a home-brew Rift design.  It is working quiet well already but what we really need is a custom "Decrease Frame Size" option because non of the current presets will fit and I can't work out how to add one to the SVP profile files, any ideas?  The fact that Stereoscopic player can't re-adjust the frame size automatically is also a big problem for the color bars even if the ffdshow thing is fixed, there doesn't seem to be a render delay command line option either so I've been trying to work out another solution. 

I have done some quick tests with head-tracking for pan & scan on my CRT by mapping MPC's shortcut keys but it is not very smooth, I'm hoping SVP can do a much better job of that.  The pan & scan effect I want is quiet hard to describe; when watching a 3D movie through SVP on the Rift with head-tracking enabled it should be just like looking through glasses that have a semi-transparent frosted border around the lenses, the 3D movie world appears stable because the color bars form a window that smoothly pans across the stereo scene as you move your head (even a few mm) I hope that makes sense...?  It should look awesome at 60fps but for your brain to buy it we HAVE to get the panning just as smooth as the SVP interpolation video framerate.  Lets make it happen!

smile

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
You ask me to think about your idea. I think it is good idea smile
As I understand You are waiting for device. You already have custom option to decrease frame size. Thanks to Chainik.
What you expect from me? wink

16 (edited by Nick 3DvB 19-11-2012 01:28:59)

Re: Rift HMD, black borders lighting and stuff

Thanks  smile   I found the thread about custom profile but I have not been able to get it working yet, I will keep trying, for the Rift we just need a profile for half the horizontal resolution reported by the display (because it is side by side).  But that is an easy fix, the main thing I need help with is the pan & scan idea, just changing the aspect ratio of the color bars in real time will not work, because the "screen" will follow your eyes when you move your head!  We need to "zoom in" the video about 20%, to overlay the color bars to make a frame that is 20% smaller than the video resolution, it is this frame that has to move over the video as you move you head.  I think we can do it using the crop feature but it has to be much faster to work with head-tracking, I really don't know where to start with this so any ideas you can give me would be very helpful.


EDIT: This is a photo taken through one "eye" of an early Rift prototype, it should give you an idea.

http://www.svp-team.com/forum/misc.php?item=1971

Note - It looks very bad for several reasons: the image is not filling the LCD (aspect ratio problem) the lens is too far from the screen, the camera is not in the correct eye position, and there is very strong moire pattern, don't worry the pixels will not look that bad in the final rift.

Post's attachments

DIY Rift.jpg, 143.4 kb, 612 x 816
DIY Rift.jpg 143.4 kb, 578 downloads since 2012-11-18 

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
Thank you for picture. You already have prototype? hmm

I think it is not easy to implement. Tracking data from glasses must be handled and translated to x,y coordinates for video shift. It is the hardest thing because we can't use SVP or avisynth for this. You must have 3D-simulator with virtual screen wall in it. And this 3D-simulator must be fast to display all 60 fps from SVP on its virtual screen. In this case Borderlight will be fixed around screen and will not moving with heads moves. Otherwise Borderlight must be implemented inside 3D-simulator, not in SVP or avisynth.

You need to find source or samples of such 3D-simulator that must be video player's renderer at once. Do you know about something like this?

18 (edited by Nick 3DvB 19-11-2012 02:21:54)

Re: Rift HMD, black borders lighting and stuff

I don't have a prototype yet, but someone is helping me test in the MTBS3D forum, I posted the link for you to look at before.  I was hoping we can still use SVPFlow together with a Direct3D renderer in "2D mode", maybe that will be fast enough to give smooth panning.  The base texture is just the movie zoomed by 20%, the overlay texture is the SVP color bars output (with an alpha channel "screen window" in the middle) then we just move the relative positions with the head-tracking input. Really it will be four textures (two for each eye) but lets not worry about that yet!  Sadly I am not a DirectX developer, but there are already a few people working on making "virtual cinemas" using a video texture map like this:

http://www.youtube.com/embed/9pIZZi0JRks

But watching a movie inside a whole virtual world is a very different type of experience, my idea is more like being IN the movie, well maybe not - more like being the camera man shooting the movie!   smile   The head-tracking interface is not so hard, the rift tracker will be perfect for this because its gyros are very fast, and we don't really want to map all head translations anyway, it might be cool to have a zoom effect but lateral head motions would produce an uncomfortable stereoscopic effect. It will be great to have the color bars to watch normal 3D movies on the Rift, if we can also have head-tracking to it would really be amazing, but as you said the hardest thing is making the frame pan smoothly, if it is jerky then the magic will be broken...

sad

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
Thank you for this link. Idea with screen in the forest is nice too smile

The base texture is just the movie zoomed by 20%
What must happen when user turn its head more than 20%?

overlay texture is the SVP color bars output (with an alpha channel "screen window" in the middle)
It is will be slow to compute color bars in SVP because processing on CPU and delays with bars recalculating while move will be very noticeable.

20 (edited by Nick 3DvB 19-11-2012 05:08:26)

Re: Rift HMD, black borders lighting and stuff

That video is very cool, I think it is actually running in the Skyrim game!

On an "IMAX size" flat screen you will only have very limited movement, because you don't want to zoom in too much, but even a small amount of movement is very important, normally when you watch a movie on a HMD (with no tracking) you must keep your head very still, if you move your head then the screen will move with your eyes and it makes you feel sick. This is because your balance sense tells your brain you are moving but your eyes have no motion reference, its a common problem and another good reason we need to get pan & scan working.

SVP colour bars give you a good reason to move your head, to see why they happened!  But when you hit the side of the screen it will just stop, but maybe we can add a special effect to this, like the iPhone menu bounce!  Another idea is to warp the screen outward so it "wraps around" your vision, like a projector cave, the head-tracking then pans your view around this curved screen, but on most normal movies you will miss important things, and your neck will get very tired!  lol

I don't actually think we need to recalculate the light bars in real time, I think it will be ok to always calculate them from the outside edge of the base video, the difference is so small you will not notice it.  This is true for the top and bottom bars anyway, which have to be very large because of the aspect-ratio of the Rift screen (4:5).  I think to create the side bars we could just apply a "frosted glass" effect to the base video because it will fill the whole horizontal axis anyway, but we can still use SVP for the sides if it looks better to match the other bars.

ps - Do you have an experience with the Direct3D video renderers?  I have only ever seen them used in media players to stop tearing problems.

Re: Rift HMD, black borders lighting and stuff

Nick 3DvB
Do you have an experience with the Direct3D video renderers?
Unfortunately no.

22 (edited by Nick 3DvB 19-11-2012 12:13:36)

Re: Rift HMD, black borders lighting and stuff

Same here...  sad   I will ask some devs on mtbs about it, hopefully we can still use SVP as the input.

@Chainik, thanks for splitting the thread, please explain what files need editing to add custom "decrease frame size" to 640x800 ?