Bohl's Blog

my digital life

building ffmpeg on Windows (1 of n)

Now it is time to set up Cygwin: go to http://cygwin.com/ and run the installer setup.exe. For our purposes, the following packages need to be installed:

  • subversion
  • wget
  • yasm
  • bison
  • flex
  • cvs
  • git
  • gcc-core
  • gcc-g++
  • rsync
  • bc
  • make

You will need to select all these packages by hand (at least I do not know about a way to script this).

Here are some screenshots of what you have to select: cygwin-selections.png (378.92 kb)

 

Now launch a Cygwin-shell, make a new folder (where you will build ffmpeg, say "cygwin"). Then get the Git-repository, and you are ready to configure and make ffmpeg - like so:

cd ~
mkdir cygwin
cd cygwin

git clone git://source.ffmpeg.org/ffmpeg.git ffmpeg

cd ffmpeg

./configure --enable-memalign-hack

make

 This should work without any flaw, and after a while you should find a newly-built ffmpeg.exe in the folder. If you happen to have more than one CPU on your machine, you might want to use 'make -j <number_of_parallel_jobs>' in order to speed up the build - where number_of_parallel_jobs should be around the number of cores you have. So, if you have 4 cores, then try 'make -j 4'.

Next, let's test-drive the binary and run the FATE-test-suite. This requires to download some test-data, around ~700MB. You may choose to download it to the ffmpeg-folder right away, or you may keep the samples in a separate folder (and re-use it for other builds). In the first case, with this command the samples will be downloaded to a folder "fate-suite" which is created in the ffmpeg-folder

make fate-rsync SAMPLES=fate-suite/

Or if you prefer to download it manually, try something like

rsync -vrltLW --timeout=60 --contimeout=60 rsync://fate-suite.ffmpeg.org/fate-suite/ fate-suite/

Now it is time to run the tests:

make fate  SAMPLES=fate-suite/

You need to tell the path where the FATE-samples are to found as an argument to the SAMPLES-variable.

All tests should work ok, and now have done it: you built the ffmpeg-binary yourself! Of course, this is just the start...

building ffmpeg on Windows (0 of n)

I will try to collect some step-by-step instructions how to build ffmpeg on a Windows-machine. I still find it tedious, and https://winffmpeg.codeplex.com/ is of course not really a viable solution. Above all, it is cumbersome to keep it up-to-date.

OK, my plot is:

  • Describe the setup of Cygwin in order to build ffmpeg with gcc
  • Make the ffmpeg-build work with icl

Stay tuned...

building ffmpeg with Visual Studio...

This is probably the first post of a long series. After working for quite some time on the plan of building ffmpeg with the Intel-C-Compiler with VisualStudio (cf. my previous post), I made some good progress.

  • ffmpeg builds ok (with all AT&T-inline assembly ported!)
  • so far all my tests work ok
  • I also ported x264 (used as external codec in ffmpeg)

In retrospect the changes are not too numerous or intrusive. I tried to reduce the changes to a minimum, that will be important in order to integrate updates of ffmpeg.

Porting x264 was easy, at least at first glance. The challenge here was that it relied on a feature that only gcc has to offer afaik (for x86-builds) - it can keep the stack pointer aligned on 16-byte boundaries. Or, as far as my understanding goes, the Intel compiler claims also to be capable of this - but I could not make it work, thus. Either it works differently than in gcc or I made some stupid mistakes. I ended up coding some assembly wrapper that tweaks the stack for the functions which require stack alignment. Maybe I will get around on elaborating on this in a future post.

My testing so far is by far not exhaustive. I ran some quick tests re-coding some footage to MPEG2, H.263, MPEG4 and H.264. For these preliminary tests I get the same results (byte-exact!) as with a "standard" MinGW-gcc-build. The next steps will probably involve porting the ffmpeg-test-suite FATE. I am not yet sure how to address this.

I am planning to post the project (after some polishing) on CodePlex. Meanwhile I found quite a few attempts at the same goal (e.g. Direct264 or here or of course ffdshow-tryouts see here) - but it seems I got a bit further in some respects as they did. E.g. Direct264 did not solve the stack-alignment issue, and ffdshow-tryouts did not port the ffmpeg-assembly code.

At this point I just have a screenshot to make you interested:

 

caught fire on ffmpeg

It has been a couple of days since I started to seriously dive into ffmpeg. I have to admit that I caught fire on it meanwhile. My goals now are: On the one hand I want to get the idea of "driving ffmpeg as an external application" working in a robust and versatile way (which seems to require some coding for ffmpeg) and on the other hand I want to build and debug it in a Windows-enviroment. The latter means: I want to be able to debug it with VisualStudio or WinDbg, which requires to have PDB files. I seems that quite a few people have already taken on this challenge, but no one has yet completed this port. My plan for compiling ffmpeg on Windows is to use the Intel C++ compiler (using version 12.1.1.258), since it is capable of producing PDBs, understands C99 and also the AT&T-inline assembly (at least to some degree, more on this latter).

So far I did the following:

  • I took the source of ffmpeg, put everything into a VisualStudio-project.
  • Made the source compile somehow. Problematic code (which did not compile) I either changed (if it was an easy fix) or commented out.

After a couple of days I arrived at a successful build of ffmpeg. "Successful" means: build went fine, and the executable was able to transcode a video (literally a single video at this point). Now I will be working trough all my changes so far and see if they were reasonable. Might be great if some of those changes could make its way back into the ffmpeg-trunk - but I have not yet reached the point where this makes sense.

So far, I faced two major issues:

  • trouble with the C99-standard-library function "lrint"
  • trouble with AT&T inline assembly

The first one is by far the easiest one. The story (as far as my understanding goes) is that lrint and its cousins are part of the C99-standard-library and are usually declared in <math.h>. However, with Intel C++ on the Windows platform, the compiler uses the MSVC-header, and there we do not find lrint. The nasty thing about it is, that it compiles and links ok (just warning about an implicit declaration of lrint, which is of course allowed in C), but it does not work. No idea why, but the lrint-function that gets linked just returns garbage. This is also reported here or here. My solution so far was to replace <math.h> by <mathimf.h>. It is important to replace all occurences of <math.h>, since one cannot have both in one compilation unit. Not the best solution I suppose, but it works fine.

The trouble with the AT&T inline assembly is a bigger one. I will come back in a latter post on this, but my goal still is to solve it (instead of just omitting the problematic code).

BTW - so far I am believing that gcc cannot produce PDBs. However - I am not definitely sure about this. If it were possible somehow I would probably go with "compile problematic code with GCC" in order to solve the AT&T inline assembly issue.

Driving ffmpeg III

So far I used the Windows-builds of ffmpeg from Zeranoe.com. But since I ran into a dead end with the piping of audio and video at the same time, I thought it was time to dig a bit deeper. Maybe I am doing something wrong, maybe the windows-port is somehow flawed. The first step is to have a look at the source. And this means - being able to build it myself. A quick web-search makes clear: this is not a task too easy... Luckily, these posts got me on the right track - here and here. I took quite a while, but finally I was able to set up a virtual machine with Linux, and in this machine I am able to cross-compile ffmpeg for Windows. I put together a little script which downloads the MingGW-build script, then downloads the sources for faac, x264, lame and of course ffmpeg, builds them and finally creates the Windows-binaries. So far only x86.

The virtual machine is available here. There is one user on the machine: build with the password "build".

In build's home, cd into ffmpeg-build and run "./buildffmpeg".

Now you can choose whether to run the MingGW-script (you only need to do this once, it will take some time...)

Here you need to choose 2 (Win32 x86) - I have not yet bothered with x64-builds: 

The process of downloading and building will take quite some time, and finally you should find the binaries in the folder ~/software/packages/win32.

Good luck!

Driving ffmpeg II

OK, the idea of using ffmpeg as a stand-alone executable and feeding images into via a named pipe seemed simple and straight forward. However, as usual, things are not that simple... I will try to walk through some of the obstacles that I have come across (but this for sure uncomplete).

My first idea was that I should be able to pipe anything that ffmpeg could deal with. However, it turned out, that the only format which worked (for me) is yuv4mpegpipe. All other attempts did not work, ffmpeg complained in various ways.

So, the command I am using to launch ffmpeg is:

ffmpeg -f yuv4mpegpipe -i {0} output.mov

For {0} the name of the pipe has to be filled in, something like \\.\pipe\mypipe.

Now, we need to find out what the format yuv4mpegpipe looks like. It starts with a header

YUV4MPEG2 W720 H576 F25:1 Ip A0:0 C420mpeg2 XYSCSS=420MPEG2\n

Here we have to give the width and height, the framerate (as a fraction), the aspect ratio (or 0:0 for unspecified). This line has to transmitted once. Then what follows is a magic string

FRAME\n

And now the image data follows (for one frame), in binary format: it is a 4:2:0 format, first the Y component, then followed by V and then U - usually called YV12 (see e.g. here). Creating this format is simple enough, especially if you have IPP at your hand - ippiBGRToYCbCr420 or one of its cousins will do the job. Now the following frame is again started with the magic "FRAME\n".

One caveat I ran across: make sure that you add the first two lines (i.e. the line starting with YUV4MPEG2 and the magic "FRAME\n") in one write call! Otherwise ffmpeg is complaining and it does not work.

So, basically all what has to be done is

serverStream = new NamedPipeServerStream(
                                 this.PipeName,
                                 PipeDirection.Out,
                                 1,
                                 PipeTransmissionMode.Byte, 
                                 PipeOptions.Asynchronous,
                                 65535,                      
                                 65535);

// now launch ffmpeg

serverStream.WaitForConnection();

// now pump data into the pipe with serverStream.Write

So far, so good - and this solves my task at hand. However - since I got this far, I also wanted to pipe audio into ffmpeg. To make a long story short - I did not get this to work properly so far. The trouble starts with the following: if we simply add a second input like so

ffmpeg -f yuv4mpegpipe  -i {0} -f s16le -ar 48000 -ac 2 -i {1} output.mov

then ffmpeg does not open the second file (or pipe) immediately, so we run into trouble with WaitForConnection. The file is opened after we have added some frames, so we can work around this. However - I did not manage to make it work reliably. 

driving ffmpeg

Ffmpeg is one of the open-source projects that have been fascinating me since its first days. I admire the project for what it has archieved, the quality of this piece of code is just marvelous. I had been in the video-editing business for quite some time, and maybe have a good grasp of what wonderful a tool those guys have put together. In particular, the approach to have all pieces in one binary has always attracted me - after having wasted countless hours in DLL- and codec-hell.

However, so far I did not have a chance to put ffmpeg to work in a serious (i.e. paid for) project. Now this chance seems to have arrived, and I am very excited of it. In short, the goal is to use ffmpeg to create movies from a series of images, no audio at this time. So far the implementation uses DirectShow (yes, I am a die-hard DirectShow-fan) - however, DirectShow right out of the box does not offer a lot of video formats. Sure, there are tons of codecs for every format one can think of - but going this route would inevitably mean to enter codec-hell once more. Then, there is MediaFoundation - imho not really better suited for the task at hand, because at least as complex as DirectShow and (again, right out of the box) does not deliver that many new codecs or containers. So, I decided to give ffmpeg a shot.

The first decision is about how to interface with ffmpeg. Three possible choices seem feasible:

  • First create an AVI with DirectShow, then let ffmpeg convert it.
  • Use the ffmpeg-API (i.e. link with libavcodec/libformat and use their API)
  • transfer the source images into ffmpeg, where ffmpeg runs as a stand-alone process, let it encode and output it into a file

The first approach is for sure the lamest - but the easiest. In fact, performance is not that much a concern, so it is not immediately ruled out. The second is probably the most efficient and most solid approach, but there are quite a few drawbacks: besides (potential) legal issues it is about the problem of integrating it into a build-environment, worries about the stability of the API (and problems with updates of ffmpeg) and of course, the complexity of the API itself and the inevitable learning curve (and some more worries). So, I decided to take the third approach.

I was hoping that someone else already tried this out - and was hoping to find a nice library or code snippets for this task. To my astonishment a web search did not bring up many hits, at least not what I was hoping for. And the documentation on the ffmpeg-site itself for this was not too enlighting as well.

The basic idea is to use a named pipe in order to transport the images over to ffmpeg. I got this part (basically) working after some meandering. More on this in one of the next posts...