Announcing Hudl.FFmpeg — a C# framework to make FFmpeg interaction simple

We have used FFmpeg for years at Hudl as our go to source for video editing. Our FFmpeg inter­ac­tions were all relatively simple and were no more complex than a standard set of arguments defined by a string. This all changed when Campaign video automation came into the picture. We had to take a 4 hour manual process with Adobe After Effects and automate the production of these custom videos. These videos contained recorded audio from our coaches and a random selection of their best highlight film. Needless to say, our new task was far too complex for our current system. So we set out researching .NET wrappers for FFmpeg that would meet our needs. Here is what we were looking for; something that would be:

• Easy to implement.
• Easy to extend.
• An enhancement from our current system.
• Compatible with different FFmpeg versions.

After a lot of searching, with no luck, we set out to create a framework of our own: Hudl.FFmpeg. Hudl.FFmpeg is a .NET wrapper for FFmpeg that makes it easy to automate simple or complex video editing tasks.

So how do you create a full featured framework that works seamlessly with an open source project that constantly changes? You dissect the structure of that project and break it down into its base components. For FFmpeg, that meant we needed to account for these areas:

• Inputs
• Outputs
• Settings
• Filters

The framework achieves this by using custom objects that represent these areas of a FFmpeg command. These objects are translated, by the framework, into arguments that are fed to the FFmpeg executable.

Syntactic Sugar

Everything I mentioned before is very important, but, without a strong syntax, it doesn’t mean anything. So, we created the Sugar syntax. I’ll show you just how easy it is to use.

1. First we define our fil­ter­chains and settings that we want to apply to our video.

   
   

2. Next we would create our command, and add our video inputs.

   
   

3. After adding our inputs, we want to add our custom filters to the command.

   
   

4. Then it’s time to define our outputs, and kick off the render.

   
   

5. Finally if you put it all together, it looks just like this.

   
   

Creating Smarter Filters

Creating a wrapper for commands, settings, and filters wasn’t enough for us. We needed to go a step further. We needed to be up close and personal with these video and audio streams and pull their information at any time during our editing. This way, we could create smart automation routines that add custom effects based on the uniqueness of each video stream. So, we created a parser for the FFprobe executable, which allowed us to get to the good info that we needed. Or did it? Having info about the input streams before they are manipulated only got us half way there. What if we could get info about a stream that we are adding filters to before FFmpeg even renders it? So, we came up with the Metadata Calculator feature which makes this all possible.

The last example didn’t really require any smart technology. Concatenating two videos together is a fairly simple task that doesn’t merit the use of a special framework. Our Campaign videos contain a special effect that we call Crossfading. This effect will essentially fade from one input video directly into the next video. To achieve this, we needed to know about the streams we were working with. To achieve this in a single command required quite a few fil­ter­chains and some knowledge about the streams we are working with. Let’s take a look inside the smart filter and at how it works.

As you can imagine, it would have been pretty difficult to manage a process like this using strings.

Conclusion

Since we have built this automation framework, it has quickly become our go to source for any inter­ac­tions with FFmpeg. This framework is currently being used by the majority of our rendering agents. With it, we have saved ourselves many man hours developing new video effects. As mentioned in the title, we are now giving this project to the open source community. You can contribute to it here, or download our Hudl.FFmpeg NuGet package using your favorite package management utility or console.