AI Code Review for Laravel, What It Catches and Misses
Published on — Last updated
AI code review means a language model reads your pull request and writes comments on it. It can find real bugs in the diff, but it also misses a lot and sometimes invents problems. I treat it as a cheap extra reader that comes after tests and static checks, never as a replacement for a human. This post shows how to add it to a Laravel repo, what the research says it catches and misses, and how to pick a tool.
What AI code review tools do
An AI code review tool runs on each pull request. It reads the diff, often looks at nearby files, and posts comments on the lines it dislikes. Many tools also write a short summary of the change.
There are three kinds you will meet. GitHub Copilot code review is hosted inside GitHub. PR-Agent is an open-source tool that you run in your own pipeline. Open Code Review is a newer open-source command line tool from Alibaba.
I write its name with capital letters, because open code review can also mean any open tool.
Qodo has its own AI code review platform. The PR-Agent README describes PR-Agent as a community-maintained legacy project of Qodo, and says it is not the Qodo offering for open-source projects.
I have not benchmarked these tools myself. The numbers below come from papers and from the projects' own pages. When a number comes from the vendor, I say so.
What AI code review catches
These tools do best on problems you can see inside the diff. The Open Code Review repo ships a rule file for PHP, and it shows what a reviewer is told to look for. The list includes loose comparisons that change types, SQL built from untrusted values, and mass assignment when the model's fillable or guarded settings do not limit the request fields. It also lists authorization that exists only in a template or hidden control, swallowed exceptions, and N+1 queries in loops that run at real scale.
The same file adds two good rules. The reviewer must check callers and framework settings before it makes a claim that goes beyond the diff. And it should skip anything that PHPStan, Psalm, or the code formatter already enforces. I would ask for both rules from any tool.
What AI code review misses
GitHub's own page says Copilot code review may not find all problems, especially when a change is large or complex. It also says the tool can point at problems that do not exist, and that feedback may lean toward some languages and styles.
Two studies from 2026 give numbers.
- Crupi, Tufano and Bavota compared 739 human review comments on 240 pull requests with ChatGPT-4 reviews. The model suggested about 2.4 times as many changes as humans. But it found only about 10 percent of the quality issues that humans reported, and about 40 percent of its extra comments pointed to meaningful quality issues. The authors say the two approaches complement each other, and that LLM review should not be considered a valid alternative to human review.
- Lin and colleagues studied 31,073 CodeRabbit comments, with developer reactions, on 10,191 pull requests in 239 repositories. Developers accepted 36.4 percent, 7.3 percent started a discussion, and 56.3 percent were rejected. Most rejected comments were invalid, repeated, or out of scope. This study has no human baseline.
The Open Code Review team also published a paper. These numbers are vendor-quoted, from the authors' own benchmark.
Their best setup reached 33.90 percent precision and 20.00 percent recall. Claude Code reached higher recall at 28.90 percent, but it wrote 4,580 comments on 200 pull requests, against 465 to 1,096 for Open Code Review. The README calls the lower recall a deliberate trade-off for fewer noisy comments.
These studies use different data and tools, so I do not compare their numbers with each other. They agree on one thing. No tool finds most issues, and many comments are wrong or not needed.
My view is that the misses come from missing context. A reviewer cannot see the ticket, the business rules, the real data, or how a policy is wired in a file outside the diff. A human who knows the system still has to check those parts.
Run cheaper checks first
Put the deterministic tools before the model.
Laravel Pint fixes style. Larastan, which is a PHPStan extension for Laravel, finds type and logic errors. Your tests show whether the code still works. These checks cost almost nothing and give the same answer every time.
Then add the AI step on top. It should only spend tokens on what is left, such as unclear logic or a missing check.
How to add it to a Laravel repo
Both open-source tools run as a GitHub Action. This is a trimmed version of the PR-Agent workflow from its docs.
1on: 2 pull_request: 3 types: [opened, reopened, ready_for_review, synchronize] 4jobs: 5 pr_agent_job: 6 if: ${{ github.event.sender.type != 'Bot' }} 7 runs-on: ubuntu-latest 8 permissions: 9 contents: read10 issues: write11 pull-requests: write12 steps:13 - uses: the-pr-agent/pr-agent@main14 env:15 OPENAI_KEY: ${{ secrets.OPENAI_KEY }}16 GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
The PR-Agent docs use the main branch in this example. I would pin to a release tag or a commit hash, so a change upstream cannot run new code with your API key. As of November 2026 the latest PR-Agent release is v0.50.0, published on 7 November 2026.
Open Code Review has its own action. It takes the model address, token, and model name as inputs, and the CLI needs Git 2.41 or newer.
Its example workflow uses the pull_request_target event, so pull requests from forks can read secrets. Read the comments in that example before you copy it. On a private repo with no outside contributors, the plain pull_request event is simpler.
Both tools can take project rules. Open Code Review rules are JSON entries that pair a file path with rule text, and its own repo has an example. For Laravel, a rule can say that every controller action must call a policy, or that request data must never reach a raw query.
How to pick a tool
I would check five things.
- Where my code goes. Your diff goes to the model provider you set up. PR-Agent works with OpenAI, Claude, Gemini, DeepSeek, Mistral, and any model reachable through LiteLLM, including a local Ollama model. Check your client contracts first.
- PHP knowledge. Open Code Review has built-in PHP and composer.json rules. For any other tool, run it on ten old pull requests with known bugs and see what it says.
- Custom rules. A tool that cannot learn your conventions will repeat the same noise.
- Maintainers and license. As of November 2026, the PR-Agent LICENSE file says MIT, and the README calls it a community-maintained legacy project of Qodo. Open Code Review uses Apache-2.0. It is young, because GitHub data shows the repo was created on 18 May 2026.
- Cost per pull request. The Open Code Review README says it used about one ninth of the tokens of Claude Code on the same model. That is vendor-quoted, so measure your own bill.
Start small and measure
Run the tool in comment-only mode for a month, and do not make it a required check.
For each comment, note whether it was useful. As a rough yardstick, CodeRabbit users accepted 36.4 percent of comments in the study above. If your useful rate is far lower, tighten the rules or change the tool.
This is the same idea I wrote about in agent evaluation. Do not trust the answer alone, check the outcome. And in your AI agent is not a coworker I wrote about research where people caught fewer errors when work was labeled as coming from an AI employee. So I call a review bot a tool, and a person still owns the merge.
If you want help setting up checks like these on a Laravel codebase, my page on hiring a Laravel engineer has the details. If an AI wrote most of your app, I also do a fixed-price vibe-coded app audit.