I have been investigating the use of content aware rescaling for video. I recently corresponded with Michael Rubinstein of MIT on this topic. He has worked extensively with others on this difficult task. Here is a video tutorial explaining the work...