So for those who don’t know I currently work for a video production company, Beholder Productions as a full time Digital Media Artist outside of Philadelphia.  I create VFX and motion graphics animations, design websites and even shooting or editing footage when the need presents itself.  Tomorrow marks a big step in a positive direction for me and my company. We are traveling into the city to begin rolling on the first shoot for our new TV series that Beholder is producing.  I am very excited at the opportunity to collaborate with our graphic designer and come up with an awesome motion graphics and branding package for the show which he has already begun designing.  It will be the first time that my work will be available to be seen by a large scale audience and I am very proud/scared of what this has in store (those fears are for another post all together).

What is even more exciting is the workflow we plan to use in shooting the episodes for the show.  The show is going to be an informative talk show discussing personal finance and law.  It will be shot using two HD cameras and we will use both green screens at our disposal. I will be using brand new software I recently obtained to track and camera match our shots into an entirely 3d scene that I created for the set. (I am still working on a better res Philly skyline image)

I am very excited to see how this works out, granted as with anything I am not going to get my hopes up that the workflow will be 100% as this will be the first time I am implementing it, however I do feel that I have done a lot of the necessary prep work for it so I am excited for the potential it presents.

Besides purchasing an actual camera that can track and record position data right into a computer for thousands of dollars I think this way will be the best for what we are looking to create for the show.  My boss who comes from more of a tradition shooting background has his fears, as he has openly admitted to never being able to fully grasp the 3d concept.  Thus I find myself in a difficult situation.  A situation where I need to have the confidence to reassure my boss of how the final product is going to come together, but not be over confident and too near sighted to think that it is all going to work flawlessly.

The workflow I finally decided to use came about almost by chance.  Recently I had been seeing more and more references online being made to the new plugin by The Foundry called simply “Camera Tracker”.  From the descriptions and screen shots I had seen the plugin reminded me of a After Effects native Boujo tracker almost, but for a much lower price tag ($280 compared to the almost $10,000).  I guess the closest software for the price that Camera Tracker is going to compete with will be  SynthEyes which runs right around $300 but stands as its own application that forces you to export to After Effects.

Those of you who have used the simple tracker points in After Effects know, that even with the enhanced featured with Adobe CS5.5’s release, tracking inside of AE can simply not handle 3d camera movement very well if at all.  And Mocha, the software that ships with After Effects is great for what it does, planer tracking, but again it can not handle 3d scenes.  That is why I decided to go with the Foundry’s Camera Tracker plugin as my final decision, they have a ton of other great products that I have used in the past so I am comfortable and confident in their environment.

In preparation for the shoot tomorrow I already have my rolls of tape and sharpies ready to create tracking points to use all over the set, making sure I get every area covered so that when Camera Tracker generates its markers it will have an even amount of points to follow.  Tracking the background of the scene will not be so difficult, the issue that we are undoubtedly going to face is that fact that the host of our show is going to be behind and interacting with a desk that will be green screened as well.  This hopefully will not post to much of a problem as many of the shots will be still shots with no camera movement except for a few seconds at the beginning of each segment.  This also helps in rendering the scenes because on my quad core I7 at home a single frame is taking about 6 minutes at full HD resolution to render in Vray.

The only real thing that I can see that’s missing from the plugin is an option to export the tracking data to a 3d program, which by itself id a huge problem if your trying to match the scene to something other than object you create in AE.  This is where the other wonderful plugin MAX2AE comes in.  What is great about this plugin is that the name itself is deceiving, it actually is able to go from 3ds Max to After Effects and vice versa.  Using this plugin I will be able to take the 3d camera created in After Effects and then export it along with any reference objects or lights from AE into my 3ds Max file where the entire set design and models are located.  I cant wait to actually get the footage in and begin playing around with it.  I plan to key out the footage and composite from AE so I am going to create the camera according to the Camera Tracker plugin and use that to generate a 3d camera in 3ds Max, render out the scene from there and then use that as my background plate and make correction from inside AE.

I plan to have to make a few adjustments to the actual rendered frames from 3ds Max as anyone who has attempted to composite 3d object with 2d video can tell you that more often than not 3d objects are too perfect.  No camera is able to shoot in any resolution without some imperfections, so I will undoubtedly need to add in some grain and color correct the shot.  I also plan to render out a z-depth pass from 3dMax to use on the new enhanced “Camera Lens Blur” effect now in After Effects CS5.5 to generate a depth of field equivalent to that of the camera’s lens.  What is great about this plugin which used to be simply the Lens Blur Effect is that it has so many more settings like diffraction fringe, that allow me to tweak the specific options I need to match the actual footage.  Using a z-depth map and compositing the blur saves me tons and tons of time as opposed to rendering the z-depth information from straight inside after effects, plus it allows me to adjust the amount of blur without having to re-render the entire shot.

A great vide discussing the new Camera Lens Blur effect can be found on Adobe TV here.  There are also a ton of other great tutorials on this page so I encourage you all to have a look and find out some of the other new features shipping with CS5.5.  Also if you want to test out your own match moving techniques you can download a 15 day free trial of Camera Tracker from The Foundry as well as use free capture plates that they have included with their tutorials or head over to Hollywood Camera Work and download some of their free match moving plates that give you great examples to work with as well as provide some helpful tips as to why some of the examples work better than others.