Start with the review comment you want to receive
A useful code review comment identifies a real problem, explains its impact, and points toward a fix. It does not merely announce that the code could be cleaner. Picking a model starts with defining that standard, rather than choosing the most impressive name on a model list.
For your first run, give the model a narrow job: find correctness problems introduced by a pull request. Leave formatting to your formatter and routine rule checks to your linter. This makes the model easier to evaluate and keeps its comments from becoming background noise.
Match the model to the review assignment
A small change to an isolated helper needs different context from a change to authentication, database transactions, or shared state. The harder assignment is not necessarily the longer diff. A short change can depend on behavior spread across several files.
- Local changes: Ask whether the changed function handles its inputs and failure paths correctly.
- Cross-file changes: Include callers, relevant types, and tests so the model can inspect assumptions across boundaries.
- High-risk changes: Treat the model as an additional reviewer, not the person responsible for approving the change.
You could start a trial with Claude Haiku 4.5 and Claude Sonnet 5.5. Give both the same review packet and judge the resulting comments. Their names alone do not establish which one will work best for your repository.
Build a small review packet before testing
The model needs more than a diff pasted without explanation. Include the purpose of the change, the changed code, relevant surrounding code, and any important constraints. For example, say whether an operation must be safe to retry or whether an empty result is expected.
Do not send the whole repository by default. Extra material increases token use and can bury the details that matter. Start with direct dependencies, then add context when the model cannot resolve a specific question. Remove secrets and check your organization's rules before sending private code to any service.
Use one prompt that discourages noisy comments
A consistent prompt makes model comparisons meaningful. Try this instruction:
Review this change for newly introduced correctness problems. Report only actionable findings supported by the supplied code. For each finding, name the file and relevant code, describe the triggering condition, explain the impact, and suggest a fix direction. Do not report style preferences. If evidence is missing, ask a question instead of asserting a bug. If there are no supported findings, say so.
This does not guarantee accurate results. It does give you a clear contract to test. In particular, accepting an empty review helps avoid rewarding a model simply for producing more comments.
Score findings against changes you already understand
Choose a small set of past pull requests containing known bugs, harmless changes, and tricky but correct code. Keep the original review findings out of the model's input. Then inspect each output yourself.
- Correctness: Does the claimed problem actually exist?
- Evidence: Does the explanation match the supplied code?
- Usefulness: Could the author act on the comment without guessing?
- Restraint: Does the model avoid inventing problems in correct changes?
Track missed bugs alongside false alarms. A model that catches one subtle defect but fills every clean review with speculation may create more work than it saves.
Check token cost before expanding the rollout
Compare total review cost, not just the advertised token rate. Input context, generated explanations, and repeated attempts all contribute. A practical measure is cost per useful, verified finding, considered alongside the effort spent dismissing incorrect comments.
SwiftRelay is a pay-as-you-go way to try supported models through one relay, including with an OpenAI-compatible SDK. Sign up for SwiftRelay to use the small gift credit for your first review trial, then check actual token charges before expanding it.
Keep a human at the comment gate
Begin with private draft reviews rather than automatic pull request comments. Let a developer accept, edit, or discard each finding. Choose the model that consistently produces evidence-backed comments your team would want to receive, and repeat the test when your codebase or review needs change.