How to create fluid slow motion video

[Lance Campeau]

Recently, I created a workflow for producing smooth looking slow motion video using my new Canon HV20...

(Sorry to all MAC users but this process was designed for PC...)
__________________________________________________ ___________

What you will be doing...

-Using an HDV video camera to produce fluid slow motion video footage that is of greater quality than a standard DVD

What I used to do this...

-Canon HV20 camcorder
-Sony Vegas 8.0
-AviSynth 2.5
-VirtualDub 1.7

__________________________________________________ ____________________________________________
How I did this...

STEP #1. Record good quality & well lit video footage of a subject in motion
-record in regular HDV using shutter priority @ 1/60 sec. This shutter speed provides the correct motion blur characteristic
-DO NOT USE THE 24P SETTING

STEP #2. Using Vegas 8.0, capture the footage in it's native .m2t format

STEP #3. Using Vegas 8.0, render the video into an HDV 1440x1080 60i .avi file (label this file "HIGHSPEED.avi", you may change this later if you need)

-since this entire process requires several renders, the best method for preserving quality is to render with any compression at all. This may not be practical for some users because of the large file sizes involved. If you decide to use file compression, choose the highest quality settings possible and experiment as some codecs will work better then others. For the demo video, I used the DIVX PRO codec set to 6000kbs/single pass. Despite the high bitrate, compression artifacts can be seen built up from each level of the rendering process. The source video of the demo can be downloaded here http://www.vimeo.com/download/video:37448227 (right click "Save As" etc... it's about 100mb)

STEP #4. Use *AviSynth 2.5 to separate the even & odd video fields into a 1/60p, reduced resolution video stream (just google "AviSynth", it's free)

-There are other, more simple methods for this step but this is the one I use. The resolution of HDV footage is 1440x1080i. For this process, it's only the 1080i that we are interested in. For those of you who are not firmiliar with *video interlacing* here is a brief description...

-Each HDV video frame is made from 1080 "interlaced" lines of resolution.
-From top to bottom, the camera scans the the odd lines first... 1,3,5,7 etc... up to 1079, this scan takes 1/60 sec
-Then the camera goes back to the top and scans the even lines 2,4,6,8 etc... up to 1080, this scan takes 1/60 sec
-Depending on the camera model used, this order may be reversed... this is known as "field order"
-Each 1/30 sec video frame is made by combining these two 1/60 sec "fields" into one complete frame 1,2,3,4,5,6,7 etc...

It is important to understand how video interlacing works because it's the process of "deinterlacing" the video stream fields that is the key to creating a fluid slow motion video. We use AviSynth to deinterlace the two video fields into a single progressive video stream (1/60p). Doing this reduces the vertical resolution from 1080 to 540 (-50%). A small price to pay considering you still have all 1440 pixels of the horizontal resolution (more on that later...)

*What is AviSynth?

-AviSynth is a scripting tool for video post-production. It provides a unique way of editing and processing videos. AviSynth works as a frameserver, providing instant editing without the need for temporary files. AviSynth itself does not provide a graphical user interface (GUI), but instead relies on a script system. While this may at first seem tedious and unintuitive, it is a good way to manage projects in a precise, consistent, and reproducible manner. Because the text-based scripts are human readable, the scripting language is simple and complex filters can be created from basic operations to develop a sophisticated palette of useful and unique effects.

AviSynth uses the .AVS file format for storing the scripts. You can easily edit .AVS files using Windows "Notepad". Here are some script samples you will need...

This one seems to work well with the Canon HV20
______________________

AVISource("HIGHSPEED.avi")
complementparity
separatefields
______________________


if the video is jumpy or you are using a different model of HDV camera try this one...

______________________

AVISource("HIGHSPEED.avi")
separatefields
______________________

-After you are finished editing the .AVS script with notepad name it "60FPS.avs" (this is just a sample name for the purpose this tutorial, you may change it)
-As I mentioned in section #3, the .AVI file name "HIGHSPEED.avi" can be changed to suit your project, just be sure to change it in the script too.

*THIS IS VERY IMPORTANT* you must now place your newly created "60FPS.avs" script file in the same folder as the source video, if you don't do this, the following step will not work.

STEP #5. Open the 60FPS.avs file with VirtualDub

-The .avs file you created in step #4 instructs VirtualDub to open the source video (which was 1440x1080 @ 60i) as 1440x540 @ 60p FPS (or 59.94 FPS to be more accurate but we will just round up to 60 to make it easier) Now, This is where we start to see the results. The video's aspect ratio will be distorted horizontally, this is normal and will be corrected.
-In VirtualDub press CTRL+R to open the "frame rate"panel (you can also find this under the "filter" menu in the edit bar)
-in the section "source rate adjustment" select "change to _____ frames per second" (add the frame rate you would like, 24 is optimal)
-Next, in the "frame rate conversion" section select "convert to fps_____ " (I use 24, you can use whatever your project requires)
-Now, Press CTRL+F to open up the "filter" section. This is the part where we correct the aspect ratio and resize the video.
-In the "filter" dialog box select "add" then select "resize" from the list of filters.
-In the "size options" section select "new size" as 960x540 (this is the mathematical optimized resolution, other sizes may reduce the quality)
-in the "aspect ratio" section select "disabled"
-in the "framing options" section select "do not letterbox or crop" then close window
-Save the video (press CTRL+P to select the compression codec of your choice) then F7 to open the "save as" window. Choose a name then save.

Your done!
__________________________________________________ _____

Here is a rundown of the benefits of the technique.
-The video is now 100% true progressive with a very "film like" quality. (the 1/60 sec shutter speed is critical for maximizing this effect)
-The video retains a quality level that significantly higher than standard DV footage (miniDV is 720x480i, this video is 960x540p)
-The video would still look fairly good resized to 1280x720p HD, a widely accepted standard for broadcasting and HD disc authoring
-The video can be slowed down even further in Vegas by using "frame interpolation/resampling" while still retaining a fluid look.
__________________________________________________ ______

I hope all this wasn't too hard to follow for most of you. Your comments are welcome, enjoy!

Lance Campeau
www.LanceCampeau.com

[jmalmsten]

I only want to add that for that true cinematic effect, one should use 180 degree shutter... meaning a double shutter compared to framerate... meaning, 1/120 for 60fps ntsc or 1/100 for 50fps pal

[racer-x]

You did such a great job on this, but I have to give some advise here.........

Sepparatefields is not a good option here. A better one is Bob(1), or beter yet, use Leakkernel plugin with a script like this:

AviSource("myvideo.avi")
LeakKernelBob(1) # (0=BFF, 1=TFF) 
AssumeFPS(29.97)#(gives 50% speed)

That's just great for 60 fps, but what if you wanted 120 or even 1000 fps? That can easily be done with mvtools plugin:

AviSource("myvideo.avi")
LeakKernelBob(1) # (0=BFF, 1=TFF) 
AssumeFPS(29.97)#(gives 50% speed)
smoothfps(120)
assumefps(29.97)
function smoothfps(clip source, float fps) {
fp=fps*100
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=int(fp),den=100, idx=1,idx2=2)
}

I did a Slow-Mo Replay effect on THIS CLIP with 120 fps

I did a Slow-Mo effect on THIS CLIP of 300 fps.

[Ian-T]

@ Lance...For some reason when I try to view at the site it says this video is private. Anyways..I'll echo what jmalmsten said...you need to raise the shutter speed to at least 1/120 (HV20 does not have 1/120 so the next best thing is 1/100) for smooth playback with no blur.

[Lance Campeau]

In my opinion, I feel that the natural blur added by using a 1/60 shutter is a closer approximation to the 1/48 sec used by a 180 degree shutter at 24fps. Also, The 1/60 shutter speed is an important for smoothing the overall illusion of "slow motion" if you plan to interpolate/resample/keyframe your video clips further in post. Others may not agree to these findings but I feel these settings work best for the "look" I am trying to achieve.

[Lance Campeau]

BTW... how do I use the "Leakkernel plugin" I cannot find any information on HOW to use this plugin..

[kbrafford]

I don't understand why you need the AVISynth steps at all. Doesn't vegas know how to treat a 1080i stream as fake-60p, if you select the settings correctly?

Check out this thread:

http://www.hv20.com/showthread.php?t=437

[Ian-T]

@ Lance...if that is what works for what you are trying to do then I'm not trying to step on your toes. That's cool. But just so you know filming at 60fps (I know..it's 60i...but an NLE like Sony Vegas can easily make 60i into 60fps)...anyways...at 60fps a 180 degree shutter is really 1/120. It's just basically double the frame rate. ex: 24fps = 1/48 shutter where 60fps = 1/120 shutter. If you approach it like your standard film camera..then 60fps will be 1/120 because it is not possible for a film camera to shoot 60fps @ 1/60 shutter speed. It will be possible for video cameras to do that...but then you are essentially filming at a 360 degree shutter. FYI

[racer-x]

I should have said LeakKernelDeint filter.........HERE is a link to some Avisynth filters. They are in alphabetical order. There youl find LeakKernelDeint and MVTools filters. Just copy the filter DLL's into Avisynth Plugins Folder. Then use the scripts I posted.

For all you Vegas gurus, Vegas does a better job than most NLE in regard to slow motion, but it simply is not in the same league as Avisynth...............

[Ian-T]

Hey what's up Racer-X.. from what I've learned about Avisynth in the past year, thanks to you and some other folks on this site, I would say you are right...and to top it off...it's free.

[racer-x]

Hey Ian, how are you doing?

I just stumbled across this thread. It was well done, but I thought it could use a little help. Judging by your post count, it looks like you're spending a lot of time here....... I'm always busy, but will post here from time to time when I find something interesting.

[Lance Campeau]

Hey Ian:

Now it makes sense... I have heard others discribe the different shutter speeds, I think I am mixing up my 24 FPS film "emulation" techniques with the 60p high speed video techniques....

[Guriken]

Originally Posted by racer-x

You did such a great job on this, but I have to give some advise here.........

Sepparatefields is not a good option here. A better one is Bob(1), or beter yet, use Leakkernel plugin with a script like this:

AviSource("myvideo.avi")
LeakKernelBob(1) # (0=BFF, 1=TFF) 
AssumeFPS(29.97)#(gives 50% speed)

Hi, what if I have a .m2t file? I created an .avs file with your code and modified the source location but it doesn't open the file in VirtualDub.. So what should i do?

Thanks in advance, it's my first try with slow motion.

[Lance Campeau]

Hello Guriken:

As far as I can tell, VirtualDub does not currently recognise .mt2 video files. You will first have to convert the .m2t video into a format that VirtualDub can open. A high quality AVI can be opened by VirtualDub.

[racer-x]

Guriken wrote:

Hi, what if I have a .m2t file? I created an .avs file with your code and modified the source location but it doesn't open the file in VirtualDub.. So what should i do?

Thanks in advance, it's my first try with slow motion.

Well if you just want test it out, then you can use:

DirectShowSource("myclip.m2t")


The best way to do it however, is to import the m2t into DGindex and save to project ".D2V" file that you open in Avisynth..........

[Lance Campeau]

Hey Racer X:

I looked at you demo clips. The dog is continuosly smooth. I have been playing around with the script you posted for mvtools.... I can't seem to dial in the right numbers in order to slow down my 24p footage smoothly... the plugin seems to have this sort of "stop-n-go" effect, where it's smooth for 4 or 5 frames then it holds for a couple of frames. do you have any ideas?

[Lance Campeau]

Also, VirtualDub just keeps crashing mid render... a problem I have never had before....

[racer-x]

Well the script I posted was intended for 60i / 29.97 fps. If you are using Canon HV20 - 24p as source, it will have some progressive frames and some interlaced frames. Really you should only shoot 60i for slow-mo as it gives the most possibilities. If you have to use 24p, then try to remove the pulldown before you process with mvtools, then skip lines 2 and 3 (same goes for 30p):

AviSource("myvideo.avi") # or use DirectShowSource("myclip.m2t") 
LeakKernelBob(1) # (0=BFF, 1=TFF) 
AssumeFPS(29.97)#(gives 50% speed)
smoothfps(120)  # (This is the fps you want to create)
assumefps(29.97)#(this is the playback fps, you can use 24 if you like)
function smoothfps(clip source, float fps) {
fp=fps*100
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=int(fp),den=100, idx=1,idx2=2)
}

[Alsone]

Dumb question but if you shrink the size of the video down on slo mo to 540 lines, how do you integrate it into your timeline with full HD or 720P footage? Surely there will be a problem with having different sizes on the same timeline, or even if you have a NLE that will handle this, how do you prevent the picture from switching size 1/2 way through in your final film?

[2Bdecided]

You don't, you use bob, or XXXdeint etc as described, to get 1080 lines.

Cheers,
David.

[Lance Campeau]

Hey Racer X:

Here is another script that I found from the following weblink...

http://nerds-central.blogspot.com/20...-avisynth.html

Here is the script...

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
source = AVISource("C:\Documents and Settings\ABC123\Desktop\untitled2.avi",false)
oSource=source
source=ConvertToYV12(source)
source=AssumeFPS(source,24)
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
fSource=source.MVFlowFps2(backward_vec,forward_vec ,backward_vec2,forward_vec2,num=96,idx=1,idx2=2)
fSource=AssumeFPS(fSource,24)
return fSource


You can see in line 5 that the source video is set at 24fps... this is because I have already deinterlaced and processed the footage. Have a look at the two different scripts, this script does not give me that stop-n-go effect I was discribing above. Even after deleting lines 2 and 3 from your script, I still had the stop-n-go effect. I assume that has something to do with pulldown removal or maybe your script is designed for interlaced footage only? I would like to understand these scripts a bit better so I could try playing around with the numbers. Also, I simply cannot complete a rendering while running this script... Does MVTOOLS have known bugs?

[Lance Campeau]

AH HA!

okay, your script works... I made a small error... okay, that's one problem down... now, why does VirtualDub always crash when rendering this script? I have tried absolutely everything I can think of but I keep getting the same "access violation" error on every attempt at rendering a file processed with this script... it will process and save a certain amount (the exact same amount every time). I have tried 3 different sources of footage with the same result. I have upgraded and downgraded AviSynth and VirtualDub several times, trying each configuration to no avail... the only thing I have not changed in any way is the MVtools plugin itself... I have come to the conclusion that this must be the source of the problem... Could this be it? Does anyone else have this issue? is there a newer version that I don't know about?

[racer-x]

I'm not sure what your problem might be. Perhaps not enough memory or CPU? I've never had that problem on my Desktop (AMD 3800 dual-core w/2 GB RAM) or my Laptop (AMD 3200 single-core w/1 GB RAM). Try it out on another PC if you can................

[racer-x]

Just to rule out a problem version of MVtools, I'm attaching a zip file that contains the version I'm using. I know mine works fine. I also included a few scripts of various fps speed. I also included an experimental script that takes 60i/29.97 fps footage and outputs true 24p/23.976 fps. It's sort of a poor man's Magic Bullet. I have no use for 24p, but you may want to play with it.........

The zip file is to big to upload here, so CLICK HERE for it.

[Lance Campeau]

Okay, Last night i may have figured out part of the problem... for a resolution of 960x540, the 540 is not divisable by 8 or 16... so, I changed it to 536 (divisable by 8) and I was able to render twice in a row without crashing.... Maybe the following line in the script is affected by the video's resolution...?

cropped = source.crop(4,4,-4,-4) # by half of block size 8 (I'm using the current version of MVtools again)

well, I'm going to do so more tests using different resolution & script settings to determine if the was really the cause. I will report back to everyone because I don't want anyone to go through the frustration that I did!