We found a division by zero bug in FFmpeg with a vibecoded fuzzer

(code.ffmpeg.org)

22 points by dclavijo1 hour ago

4 comments

dabinat56 minutes ago
It’s interesting how AI may both raise and lower the quality of software. It’s very easy to send an AI agent on an open-ended bug hunt, and if it wastes a bunch of time and effort and finds nothing, no big deal. Time is much more important for a human developer with a salary.
- dmix42 minutes ago
  Finding the bugs with LLMs is easy. Reviewing the output, cleaning it up, and making sure it doesn't break something else is the hard part.
  - bewareofscams16 minutes ago
    No, you frame it wrong. It's not reviewing output, it's (at the very least in this particular case) reviewing the outcome, - and without LLMs you ain't getting to review anything.<p>This ship has sailed, LLMs do find bugs, do save time, and do tasks dweebs considered art or craft (unable to do real either) with flying colors.
    - shevy-java14 minutes ago
      > LLMs do find bugs, do save time<p>They find bugs but whether they save time is nowhere near as clear as you try to insinuate here.
      - pixl9713 minutes ago
        They save time in finding bugs.
        lukan10 minutes ago
        And for me they also save time in fixing bugs.
- Supermancho46 minutes ago
  I don't care if you call it an over-engineered looping machine or what, there are concrete benefits to using LLMs for this. They work faster than developing your own looping algorithm and more often produce useful results than not.
- eviks42 minutes ago
  But what's your expectation of the net?
- shevy-java14 minutes ago
  I dislike AI, but if AI finds real bugs then this is in my opinion objectively a positive thing. Of course the question is what constitutes a real bug.
  - pixl9711 minutes ago
    Unfiltered models will help build exploits for the bugs they find, so there is some means of measuring their efficacy.
12j3afAv20 minutes ago
Generating an incorrect input file seems to be the easiest task of all for any fuzzer.<p>Generating correct input to get deep into the call stack and then finding something is the hard part.
Surac21 minutes ago
send patches
VCFundedGenYer28 minutes ago
The fruits of using LLMs to code. You'll waste far more time finding what it quietly and subtly wrecked than you would have if you just coded it yourself.
- jaggederest21 minutes ago
  Those sneaky LLMs going 7 years into the past and committing as a human:<p><a href="https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/8eda3c7f91e1a5b3917ef87ea3249968d5a9f263" rel="nofollow">https://code.ffmpeg.org/FFmpeg/FFmpeg/commit/8eda3c7f91e1a5b...</a>
- vegnus25 minutes ago
  You're not reading it right. The bug was found using a vibecoded fuzzer.
  - 12j3afAv19 minutes ago
    I wonder from where Claude stole this fuzzer.