I had the following Python code to invoke
ffmpeg and extract a frame from a movie:
from subprocess import Popen, PIPE ... args = [ 'ffmpeg', '-y', '-i', filein, '-r', '1', '-vframes', '1', '-f', 'image2', '-t', '00:00:01', '-ss', '20', '-',] p = Popen(args, stdout=PIPE, stderr=PIPE) ( out, err ) = p.communicate()
(At the end of this,
out contains the bytes of the frame.)
This was working fine from a standalone python test program, but when invoked as a celery task, ffmpeg would fail to decode the video. In the celery log (running
celeryd in verbose mode,
python manage.py celeryd -l info) the errors looked like this:
[mpeg4 @ 0x464d90]illegal MB_type [mpeg4 @ 0x464d90]Error at MB: 1338 [mpeg4 @ 0x464d90]concealing 1344 DC, 1344 AC, 1344 MV errors...
At first I thought perhaps the environment that the task ran in was different, so I tried a few things:
- hard-wiring the path to ffmpeg in the code (in case it was picking up a different ffmpeg from somewhere)
- examining the ffmpeg output, which is quite verbose about the details of how it was built, to establish the right ffmpeg was being run
- capturing the output of the env shell command and comparing it to the working environment, to look for salient differences
None of this led anywhere.
Since the code worked standalone, I figured it was something about how the subprocess was being invoked. Most likely, I thought, seeing as the working version was being run from an interactive shell, it was something to do with file descriptors and standard input/output.
My code wasn’t dealing with stdin explicitly, so I made it do so by opening a pipe and sending an empty string (the argument to
p = Popen(args, stdin=PIPE, stdout=PIPE, stderr=PIPE) ( out, err ) = p.communicate('')
Turns out my hunch was right, and this worked just fine. I’m not sure why it’s necessary, and at some point I’ll return and experiment a bit more.