[samworm]

It is interesting that YouTube isn't mentioned in this blogpost, despite there being good evidence that ffmpeg has been used there[1].

The fuzz testing they mention is based around constructing malformed (or at least "exotic") input files and then monitoring for failures... ie simulating exactly the kind of attack someone might use against YouTube's transcoding infrastructure.

[1] http://multimedia.cx/eggs/googles-youtube-uses-ffmpeg/

[chadillac]

I'm not proud of this legacy code, but... it exists because it was a real world issue that would cripple a conversion server in production envs when fed certain files with timing/syncing errors as part of an automated upload and conversion process. When it would crash it would consume 100% of the cores and eat up enough RAM to force swap.

This cron has been keeping ffmpeg in check for over 4 years (not 6, whoopsie) in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system.

  1 <?php
  2 /*
  3 **  Cron responsible for detecting failed/hung ffmpeg
  4 **  instances and killing them.
  5 */
  6 require_once '/lib/class/dbmysqli.php';
  7 
  8 $output = shell_exec('ps -aeo pid,etime,args | grep ffmpeg | grep -v grep');
  9 
 10
 11 
 12 
 13 preg_match_all("/^[ ]{0,}([0-9]*)[ ]{0,}(.*?) .*?([0-9]*?)\-[0-9]*?\.flv /m", $output, $preg_out, PREG_SET_ORDER);
 14 
 15 if (!empty($preg_out)) {
 16 
 17     $db = new DBmysqli('dbmaster');
 18 
 19     foreach ($preg_out as $process) {
 20         $pid = intval($process[1]);
 21         $etime = $process[2];
 22         $queue_id = intval($process[3]);
 23 
 24         $etime = intval(str_replace(':','',$etime));
 25 
 26         // elapsed time >= 60:00 (1 hour)
 27         if ($etime >= 6000) {
 28             $fail_sql = "DELETE FROM media_upload_queue WHERE queue_id = $queue_id";
 29             shell_exec("kill -9 ".$pid); // kill hung ffmpeg process
 30             $db->query($fail_sql); // remove file from DB
 31             // debate if life is worth living...
 32         }
 33     }
 34 }
 35 ?>

Yes I know there are better ways to do this now at the OS level, but it was a quick hack over a half decade ago and continues to work... ain't broke, don't fix it kinda deal.

[rip747]

"I'm not proud of this legacy code,"

"This cron has been keeping ffmpeg in check for over 6+ years in a production environment at this point... it processes thousands of videos a day using a custom queuing and reviewing system."

why do we programmers always feel we need to apologize for something that we did quickly, but has been running without incident for a number of years.

take a bow my friend. that was an awesome patch you came up with all those years ago.

[burntsushi]

> why do we programmers always feel we need to apologize for something that we did quickly

Because if we don't, our programming brethren will assume that we thought the fix was correct and optimal. And we'll get bashed for it.

Haven't you ever heard of the quip (summarized): "The quickest way to get help from a technical crowd is to claim what you're doing is correct. People will bend over backwards to prove you wrong."

[MBCook]

> why do we programmers always feel we need to apologize for something that we did quickly, but has been running without incident for a number of years.

The script is only treating the symptom, not the problem.

It would be better to detect the files before they waste an hour of time, so you could tell the user instead of having them silently disappear. Maybe there is something you could do to fix the files. The programmer part of me says there is a real fix that needs writing.

It's obviously a great script if it's worked that long. Who knows how long it would take to track down and fix the bug(s) causing the issue. If they haven't needed to fix the bug in all these years just writing that script was obviously a good decision.

[zaroth]

> If they haven't needed to fix the bug in all these years just writing that script was obviously a good decision.

Exactly this. There are an unbounded number of bugs which will cause this single symptom. I think detecting the symptom is exactly the right solution.

[kbenson]

In theory. In reality, after treating one or two causes, the occurrence rate may fall to once a month or year.

I agree that mitigating the problem is a good first step though.

[mastre_]

Tell the truth, it's only been 4 years.

[chadillac]

damn... should have checked svn logs first, busted.

r8645 | * | 2010-02-09 18:11:35 -0500 (Tue, 09 Feb 2010) | 2 lines

[felideon]

Did you really write that comment on line 31 four years ago? :)

[tttt1111]

Google has never officially acknowledged that YouTube uses ffmpeg. I seem to recall it was because of legal reasons they are not allowed to. So you will never hear a Google employee mention anything about YouTube and ffmpeg ;)

[dpe82]

We do at least know that they use x264: http://x264licensing.com/adopters