Bohl's Blog

my digital life

building ffmpeg with Visual Studio...

This is probably the first post of a long series. After working for quite some time on the plan of building ffmpeg with the Intel-C-Compiler with VisualStudio (cf. my previous post), I made some good progress.

  • ffmpeg builds ok (with all AT&T-inline assembly ported!)
  • so far all my tests work ok
  • I also ported x264 (used as external codec in ffmpeg)

In retrospect the changes are not too numerous or intrusive. I tried to reduce the changes to a minimum, that will be important in order to integrate updates of ffmpeg.

Porting x264 was easy, at least at first glance. The challenge here was that it relied on a feature that only gcc has to offer afaik (for x86-builds) - it can keep the stack pointer aligned on 16-byte boundaries. Or, as far as my understanding goes, the Intel compiler claims also to be capable of this - but I could not make it work, thus. Either it works differently than in gcc or I made some stupid mistakes. I ended up coding some assembly wrapper that tweaks the stack for the functions which require stack alignment. Maybe I will get around on elaborating on this in a future post.

My testing so far is by far not exhaustive. I ran some quick tests re-coding some footage to MPEG2, H.263, MPEG4 and H.264. For these preliminary tests I get the same results (byte-exact!) as with a "standard" MinGW-gcc-build. The next steps will probably involve porting the ffmpeg-test-suite FATE. I am not yet sure how to address this.

I am planning to post the project (after some polishing) on CodePlex. Meanwhile I found quite a few attempts at the same goal (e.g. Direct264 or here or of course ffdshow-tryouts see here) - but it seems I got a bit further in some respects as they did. E.g. Direct264 did not solve the stack-alignment issue, and ffdshow-tryouts did not port the ffmpeg-assembly code.

At this point I just have a screenshot to make you interested:

 

Solve the "project is out of date issue"

If you ever ran across the issue that VisualStudio wants to re-build your project every time you hit F5

then the steps described here might be helpful.

At least in my case, it immediately spit out this line

[4924] Project 'S:\DEV\Projects\Video\ffmpeg\WINFFMPEG\Solution\ffmpeg\ffmpeg.vcxproj' not up to date because build input 'S:\DEV\PROJECTS\VIDEO\FFMPEG\WINFFMPEG\SOLUTION\FFMPEG\INTTYPES.H' is missing.

which pointed my directly to the culprit.

caught fire on ffmpeg

It has been a couple of days since I started to seriously dive into ffmpeg. I have to admit that I caught fire on it meanwhile. My goals now are: On the one hand I want to get the idea of "driving ffmpeg as an external application" working in a robust and versatile way (which seems to require some coding for ffmpeg) and on the other hand I want to build and debug it in a Windows-enviroment. The latter means: I want to be able to debug it with VisualStudio or WinDbg, which requires to have PDB files. I seems that quite a few people have already taken on this challenge, but no one has yet completed this port. My plan for compiling ffmpeg on Windows is to use the Intel C++ compiler (using version 12.1.1.258), since it is capable of producing PDBs, understands C99 and also the AT&T-inline assembly (at least to some degree, more on this latter).

So far I did the following:

  • I took the source of ffmpeg, put everything into a VisualStudio-project.
  • Made the source compile somehow. Problematic code (which did not compile) I either changed (if it was an easy fix) or commented out.

After a couple of days I arrived at a successful build of ffmpeg. "Successful" means: build went fine, and the executable was able to transcode a video (literally a single video at this point). Now I will be working trough all my changes so far and see if they were reasonable. Might be great if some of those changes could make its way back into the ffmpeg-trunk - but I have not yet reached the point where this makes sense.

So far, I faced two major issues:

  • trouble with the C99-standard-library function "lrint"
  • trouble with AT&T inline assembly

The first one is by far the easiest one. The story (as far as my understanding goes) is that lrint and its cousins are part of the C99-standard-library and are usually declared in <math.h>. However, with Intel C++ on the Windows platform, the compiler uses the MSVC-header, and there we do not find lrint. The nasty thing about it is, that it compiles and links ok (just warning about an implicit declaration of lrint, which is of course allowed in C), but it does not work. No idea why, but the lrint-function that gets linked just returns garbage. This is also reported here or here. My solution so far was to replace <math.h> by <mathimf.h>. It is important to replace all occurences of <math.h>, since one cannot have both in one compilation unit. Not the best solution I suppose, but it works fine.

The trouble with the AT&T inline assembly is a bigger one. I will come back in a latter post on this, but my goal still is to solve it (instead of just omitting the problematic code).

BTW - so far I am believing that gcc cannot produce PDBs. However - I am not definitely sure about this. If it were possible somehow I would probably go with "compile problematic code with GCC" in order to solve the AT&T inline assembly issue.

TrackMania Canyon likes X-BOX 360 controller

I a recent post I whined about problems with TrackMania Canyon and a Sidewinder Force Feedback 2 Joystick. Meanwhile a bought an X-BOX-360-controller (just the simple, wired one) - and this works perfectly well.

What makes me wonder is the CD that comes with it "Xbox 360 Accessories 1.2". Don't get me wrong, just plug in the controller and all is set and done, works without any driver installation, update or whatever. Nevertheless we have this CD, and the curious mind wants to know... Well, it installs about 100MB to your hard-disk, but after a while playing around with it I was unable to find any useful feature.

ConcurrentBag - suspicion of causing memory leaks

In a current project I was using the .NET-framework-class ConcurrentBag. I am quite confident that I was using this class in a context for which it was designed for: in a scenario where the same thread will both be producing and consuming data and where concurrent access is quite often to be expected.

However - there is reason to believe that ConcurrentBag causes (or at least: is likely to cause) a memory leak. A web-search gives quite a few hits where people come to (seemingly) the same conclusion - e. g. here or here or (possibly related) here. In all these discussion I have not yet seen a final and authoritative conclusion - so I am somewhat confused currently. Should it be possible that such a huge flaw goes unnoticed? Am I missing something?

Well, I hope I get around to investigate this case more thoroughly - and of course post the results here. For the time being I ended up replacing the ConcurrentBag by a simple List<>, spiced with some locking. We'll have to see if that fixes the issue.

Driving ffmpeg III

So far I used the Windows-builds of ffmpeg from Zeranoe.com. But since I ran into a dead end with the piping of audio and video at the same time, I thought it was time to dig a bit deeper. Maybe I am doing something wrong, maybe the windows-port is somehow flawed. The first step is to have a look at the source. And this means - being able to build it myself. A quick web-search makes clear: this is not a task too easy... Luckily, these posts got me on the right track - here and here. I took quite a while, but finally I was able to set up a virtual machine with Linux, and in this machine I am able to cross-compile ffmpeg for Windows. I put together a little script which downloads the MingGW-build script, then downloads the sources for faac, x264, lame and of course ffmpeg, builds them and finally creates the Windows-binaries. So far only x86.

The virtual machine is available here. There is one user on the machine: build with the password "build".

In build's home, cd into ffmpeg-build and run "./buildffmpeg".

Now you can choose whether to run the MingGW-script (you only need to do this once, it will take some time...)

Here you need to choose 2 (Win32 x86) - I have not yet bothered with x64-builds: 

The process of downloading and building will take quite some time, and finally you should find the binaries in the folder ~/software/packages/win32.

Good luck!

Driving ffmpeg II

OK, the idea of using ffmpeg as a stand-alone executable and feeding images into via a named pipe seemed simple and straight forward. However, as usual, things are not that simple... I will try to walk through some of the obstacles that I have come across (but this for sure uncomplete).

My first idea was that I should be able to pipe anything that ffmpeg could deal with. However, it turned out, that the only format which worked (for me) is yuv4mpegpipe. All other attempts did not work, ffmpeg complained in various ways.

So, the command I am using to launch ffmpeg is:

ffmpeg -f yuv4mpegpipe -i {0} output.mov

For {0} the name of the pipe has to be filled in, something like \\.\pipe\mypipe.

Now, we need to find out what the format yuv4mpegpipe looks like. It starts with a header

YUV4MPEG2 W720 H576 F25:1 Ip A0:0 C420mpeg2 XYSCSS=420MPEG2\n

Here we have to give the width and height, the framerate (as a fraction), the aspect ratio (or 0:0 for unspecified). This line has to transmitted once. Then what follows is a magic string

FRAME\n

And now the image data follows (for one frame), in binary format: it is a 4:2:0 format, first the Y component, then followed by V and then U - usually called YV12 (see e.g. here). Creating this format is simple enough, especially if you have IPP at your hand - ippiBGRToYCbCr420 or one of its cousins will do the job. Now the following frame is again started with the magic "FRAME\n".

One caveat I ran across: make sure that you add the first two lines (i.e. the line starting with YUV4MPEG2 and the magic "FRAME\n") in one write call! Otherwise ffmpeg is complaining and it does not work.

So, basically all what has to be done is

serverStream = new NamedPipeServerStream(
                                 this.PipeName,
                                 PipeDirection.Out,
                                 1,
                                 PipeTransmissionMode.Byte, 
                                 PipeOptions.Asynchronous,
                                 65535,                      
                                 65535);

// now launch ffmpeg

serverStream.WaitForConnection();

// now pump data into the pipe with serverStream.Write

So far, so good - and this solves my task at hand. However - since I got this far, I also wanted to pipe audio into ffmpeg. To make a long story short - I did not get this to work properly so far. The trouble starts with the following: if we simply add a second input like so

ffmpeg -f yuv4mpegpipe  -i {0} -f s16le -ar 48000 -ac 2 -i {1} output.mov

then ffmpeg does not open the second file (or pipe) immediately, so we run into trouble with WaitForConnection. The file is opened after we have added some frames, so we can work around this. However - I did not manage to make it work reliably.