We analyzed 10,643 AI code reviews.
Open-weight models showed comparable findings at 16x lower cost per token.
In late July, more than 230 organizations signed the Open Weights and American AI Leadership letter, asking Washington to protect the open-weight ecosystem rather than restrict it. Anaconda, our parent company, signed it.
Most of that debate runs on benchmarks and principle. We can add something narrower: what open-weight models actually did in a production workflow, measured on our own traffic.
We don’t make models, host them, or train on your data. Kilo is the routing and application layer, so we have no stake in which side of this wins. We see which models builders pick and what happens next.
The job we have the most data on is reviewing pull requests. Between June 22 and July 23, 2026, we classified 10,643 completed Kilo Code Reviewer runs and the 7,083 findings they produced. Every finding got a severity and a category. We normalized per review, so a model that ran 2,000 times doesn’t beat one that ran 300 times on volume alone.
Two of the top three models for surfacing critical issues were open weight. The closed-model security lead came almost entirely from one model.
Open weights took two of the top three spots
Kimi K2.7 Code led at 0.179 critical findings per review. Grok 4.5 came in at 0.176, Laguna M.1 at 0.171. Kimi and Laguna are open weight, Grok is closed, and the gap between first and third is under 5%.
It doesn’t mean open weights behave as a group. GLM 5.2’s rolling alias came 12th of 13 on the same measure. The spread inside the open-weight set was wider than the average difference between the open and closed sets, which suggests license category isn’t the variable doing the work.
Models don’t agree on what counts as critical
Models also differ in how they frame what they find.
Laguna M.1 escalates: 28% of its findings were critical. GLM 5.2’s rolling alias does the opposite, with 58% of its findings landing as suggestions. Both are open weight. GPT 5.6 Sol put 92% of its findings in the warning bucket.
Same data, different reviewer profiles. License doesn’t predict which one you get.
One model created most of the security gap
At the group level, closed models reported roughly twice as many security findings per review: 0.067 versus 0.034. That looks like a clear result until you break it out by model.
GPT 5.6 Sol reported 0.285 security findings per review across 274 reviews, far above every other model in the set. Take it out and the closed-model average drops to 0.042. Open weight stays at 0.034.
So if security coverage is what you’re buying, buy the model that demonstrates it. License category is a weak proxy: one model here carried the entire group difference.
Three quarters of the tokens, a sixth of the cost
Across broader Kilo traffic, open-weight models accounted for 75% of tokens and 16% of user-facing cost. Closed-weight models accounted for 25% of tokens and 84% of cost. Per token, open-weight traffic came in about 16x cheaper: (84% cost / 25% tokens) / (16% cost / 75% tokens) = 15.75.
That’s how our traffic is priced, not a same-task benchmark. Nobody ran the same pull request through both sets at matched settings.
The share is also still moving. In the week of July 20, 2026, open-weight models accounted for 79.1% of all token usage on Kilo, against 20.9% for proprietary models.
Use one model to write, another to review
Picking an AI coding model doesn’t have to mean picking one model for the whole workflow.
An author model should implement a correct change efficiently. A reviewer model should challenge it, look for failure modes, and catch what the author missed. Different jobs. Using one model for both can reproduce the same blind spots twice.
In the June traffic we could attribute, 32.3% of reviews already used a different model than the one that wrote the code. The most common pairing was Step 3.7 Flash authoring and Laguna M.1 reviewing.
Model freedom is what makes this practical. A cheap open-weight model can implement routine work while a model with stronger security behavior reviews it. Or a frontier model can author a hard change while an open-weight model with high critical intensity takes an independent pass.
No single model wins every task
A year ago, picking a model was a short conversation. Now a capable one lands most weeks, from Nemotron, Qwen, GLM, MiniMax, Kimi, Mistral, and the frontier labs. The review data says the same thing from the other direction: 13 routes, and the behavioral profiles don’t cluster by license.
That’s the argument for routing instead of standardizing on one model. Closed frontier models are strong on the hardest problems, and builders use them there. Across a lot of the remaining work, open weights compete on task accuracy, cost, privacy, and where you’re permitted to run them.
We also ran a head-to-head where Kimi K3 and Grok 4.5 built the same database. The open-model path cost significantly less for a comparable result.
One caveat on the routing layer itself: a router owned by a model vendor has a reason to prefer its own models. We partner with the labs rather than competing with them, so the only job is getting you to the right model.
Three rules for picking a reviewer
Buy a behavior, not a license category. Compare critical intensity, security emphasis, false positives, latency, and cost against your own repositories.
Separate authoring from review. Give the reviewer a different objective and, when it’s worth it, a different model.
Use open weights where they earn the work. In this sample they’re production options, not fallbacks.
What this data doesn’t prove
We measured what models flagged, not whether they were right. More findings can mean broader coverage, more false positives, or both. We don’t have a consistent accept-or-dismiss signal on every finding yet, so this isn’t an accuracy leaderboard.
Repository mix matters too. Models weren’t randomly assigned to identical pull requests. Some families show up twice in the charts because a versioned route stays pinned to one snapshot while a rolling alias can move to a newer one behind the same public name.
Acceptance rate is the measure we want next: which findings developers fix, dismiss, or argue with. That turns a behavioral comparison into a quality one.
Why this matters past code review
Open weights don’t have to beat closed models for the ecosystem argument to hold. They reduce how much of your stack depends on a single vendor, and they let you run inference where policy requires it. That’s what model freedom has meant at Kilo from the start: open-weight and local models alongside closed frontier APIs, your own keys if you already have vendor contracts, and an EU-first setup where data residency demands one.
Job laid out our full position on the letter in Open Weights Is All You Need. The per-model table, methodology, and sortable data behind this post are in the research writeup.