Learn › Trusting AI-written code

Can AI fix bugs on its own?

Yes. An AI system can fix a bug on its own when the bug can be reproduced: when there is a test that fails because of it. With that test, the system can try a fix, see whether the test passes, and try again if it does not. Without it, the system is guessing, and so is anyone reviewing its work.

The difference between asking and not asking

Most people have used AI to fix a bug by pasting an error into an assistant. That works, and it has three steps a person does: noticing the bug, describing it, and checking the answer.

Fixing a bug “on its own” means the system does all three. It hears about the bug from your running app, works out what is wrong, and checks its own fix before showing you. That is what autonomous software maintenance means, and it is what Bugstack does.

How it works

  1. The bug announces itself. A crash in production, a failing build, a failed deploy.
  2. The system reproduces it. It writes a test that fails on the current code, for the bug’s own reason. This is the step everything else depends on.
  3. It writes a fix and runs the test. If the test still fails, the fix was wrong, and it tries again.
  4. It runs everything else. A fix that breaks another test is not a fix.
  5. It hands over a pull request with the cause, the change and the test.

How often it works

We measured the two halves separately.

StepMeasured onResult
Reproducing the bugSWT-Bench Verified: 433 real bugs from open-source projects91.5% reproduced, the top score on the public leaderboard
Fixing itOur own exam of planted bugs, sealed before tuning43 of 44 proposed fixes held

Reproduction is the half a public benchmark scores, and it is the one every fix depends on.

Which bugs it can fix

  • Crashes. A missing check, a wrong assumption about data, a value that can be empty.
  • Build and test failures. A renamed function, a changed type, a test that asserted old behaviour.
  • Deploy failures caused by a commit.
  • Known patterns, such as a package with a published flaw.

Which bugs still need a person

  • Bugs with no signal. The page loads and shows the wrong number. Nothing failed, so nothing started.
  • Bugs about intent. The code does what it was written to do, and that was the wrong thing.
  • Bugs outside the code. A database that is down, a setting on a host.
  • Bugs that will not reproduce. Timing problems and rare states. Bugstack says so in the pull request when it could not reproduce one.

Should you let it merge its own fixes?

Not at first. Bugstack starts every repository on Review: it opens the pull request and you merge. When you have seen its fixes hold on your code, you can set a repository to Autopilot, where a fix merges once your proof setting is met. The default asks for a sandbox proof and green CI.

Questions

Is an AI bug fix as good as a human one?

For the bugs it can reproduce, it is held to a strict standard: a test that failed before and passes after. For bugs that need judgment about what the code should do, a person is still better.

Can AI fix bugs it wrote itself?

Yes. A bug is a bug whoever wrote it. The fix depends on reproducing the failure, not on who introduced it.

What happens when the AI cannot fix a bug?

A system worth using tells you. Bugstack opens no pull request for a fix that does not hold, and when it cannot reproduce a bug, the pull request says so and waits for you.

See it work on your own code.

$100 a month

Includes $100 of usage credit every month. Start with a 14-day trial: $50 of credit and a first security audit on us. A card is required.