Some said it was the latest GLM model from Zai. Some were sure it was a Kimi variant from MoonshotAI. Nobody — nobody — guessed it was a 1.6-trillion-parameter monster from a food delivery company.

At the tail end of Q2, it was revealed that Owl Alpha was LongCat-2.0 from Meituan. There was a ton of buzz around the model, and a good amount of the 10T+ tokens consumed during the stealth period were from Kilo users.

If you were one of the many Kilo Coders who fell for Owl Alpha before the reveal, we’ve got good news: LongCat 2.0 is free on Kilo for a limited time. No limits, just put it to use.

LongCat 2.0

Short version: LongCat 2.0 is a 1.6-trillion-parameter Mixture-of-Experts model that activates about 48 billion parameters per token, built specifically for agentic coding. It ships on Hugging Face under a plain MIT license, which is great news for anybody moving toward local AI in production.

Meituan has said that the new LongCat is the industry’s first trillion-parameter model to complete full-process training and inference on a 50,000-card domestic cluster. Well, whatever they did, it worked. The architecture is doing some clever things to keep that scale affordable. LSA sparse attention extends the long context window without the usual quadratic cost blowup, Zero-Compute Experts keep per-token activation between roughly 33B and 56B, and a routing scheme splits work across separate Agent, Reasoning, and Interaction expert groups — so you’re not burning cycles on experts that have nothing to do with the task in front of them.

But does it actually code? How much does it cost?

The Owl Alpha era answered that better than any benchmark table could. Running fully anonymous, with no brand halo or geographical info attached, it won over a lot of devs. On paper it holds up too. Meituan reports it beats a Claude Opus reference on IFEval (90.0 vs 86.0) and IMO-AnswerBench (81.8 vs 75.3), while trailing on SWE-bench Pro (59.5 vs 69.2).

Per-million-token pricing is a trap. What you actually pay depends on how many tokens a model burns to finish the job — verbosity, reasoning overhead, retries, all of it. So look at cost per completed task instead.

On Artificial Analysis’s weighted cost-per-task metric, LongCat 2.0 comes in at $0.12 per Intelligence Index task. DeepSeek’s latest flagship, V4 Pro 0813, lands at $0.25 per task — more than double. Same open-weight class, same 1M-context ambitions, roughly the same architecture on paper (both 1.6T MoE), and LongCat does the job for less than half the spend per task. Point being: this isn’t “cheap because it’s worse.” It’s affordable because it’s efficient about how it gets to an answer, and that efficiency compounds hard once an agent is chewing through a real workload turn after turn.

And again, for a limited time in Kilo, that cost-per-task number is zero.

If you’ve read about the new Meta(flow), you already know how we think about building now. Kilo handles the plan-and-build side, agent modes map cleanly onto how a real pipeline takes shape, and the Anaconda Platform takes it to production. Plan mode sketches the graph. Code mode implements each step. Debug mode catches what’s broken. Review mode closes the loop before anything ships.

And remember: you don’t have to run the same model in every mode or agent. That’s the whole point of Kilo being model-agnostic. And LongCat 2.0 is an unusually good fit for the modes where it counts.

There’s never been a better time for model freedom

Let’s be honest about the current standings. Look at our leaderboard and the top of the chart is a familiar crowd: OpenAI’s GPT-5.6 Sol and GPT-5.5 are running one-two on KiloBench, with SpaceXAI’s Grok 4.6 right behind them at number three — and those same models are near the top of our usage numbers too. When something is that good, developers reach for it, and we’re not going to pretend otherwise. Sol, Luna, and Grok earn their spots.

But “what’s winning this week” and “what’s worth your attention” aren’t the same list. Kilo has always stood for model choice and model freedom, consistently supporting more than 500 of the top models for builders. Part of that commitment is making sure a genuinely strong performer doesn’t get lost just because it came out of a food delivery company instead of a lab everyone already watches.

LongCat 2.0 is exactly that kind of model: it beat name-brand competition while nobody knew who made it, and it does the job for less per task than DeepSeek’s flagship. You deserve to know it’s there — and to try it without a bill attached — before you default to the usual top of the chart.

Let’s go!

• LongCat 2.0 is available on Kilo right now, free for a limited time.

• Like everything on Kilo, it works with our model-agnostic setup. No upcharge, no lock-in, no games.

If you tried Owl Alpha and quietly wondered where it went, it didn’t go anywhere. It just got a name, a license, and a spot in your agent lineup. Wire it into your flow, hand it a gnarly step, and see why it topped the charts before anyone knew who made it.

Fire it up in Kilo today — the meter’s off while the promo lasts.

For most Python teams, the distance between “I have an idea for a pipeline” and “that pipeline is running in production” is still measured in days, not minutes. You prototype in a notebook, rewrite the logic into something deployable, wire up infrastructure, and hope nothing breaks on the way from your laptop to the cloud.

Now that Kilo is part of Anaconda, that distance gets a lot shorter. Kilo handles the plan and build side of the equation. The Anaconda Platform handles the run and deploy side. Together, they cover the full path from idea to production pipeline without forcing you to switch mental frameworks halfway through.

Two tools that were already speaking the same language

Metaflow, the open-source framework at the core of the Anaconda Platform, structures work as a graph of steps. You write plain Python, decorate functions with @step, and Metaflow handles the versioning, scaling, and orchestration underneath. It’s a framework built around a simple idea: let people write code the way they already think, and deal with the infrastructure automatically.

Kilo’s agent modes map onto that same shape almost exactly. Kilo’s Plan mode is where an agent sketches the graph before writing a line of code. Code mode is where each step gets implemented. Debug mode catches the inevitable broken artifact or malformed foreach input. Review mode closes the loop before anything ships. Once you’ve built the pipeline, you can deploy quickly on the Anaconda Platform. You’re not translating between two different ways of working. You’re using one tool to think through a flow, and another to run it.

What the new flow actually looks like

Say you want to build something that scans every repo in your org, checks dependencies, and serves the results as a live report. Here’s how that goes from nothing to running:

Plan in Kilo. Sketch the steps: list repos, scan each one in parallel, aggregate the findings, deploy a small dashboard. Kilo’s Plan mode turns that into an actual implementation plan instead of a whiteboard sketch that never gets built.

Build in Kilo. Code mode writes the FlowSpec class, the @step functions, the foreach fan-out across repos. If something’s off, like a manifest parser choking on a weird file format, Debug mode catches it before you’ve wasted a cloud run finding out the hard way.

Deploy on the Anaconda Platform. Once the flow works locally, the same code runs at scale with --with kubernetes, and the @app_deploy decorator turns the finished artifact into a live FastAPI service, using the exact same image and code package the flow itself ran on. No separate deployment pipeline, no “does this work the same way in prod” surprises.

That last part matters more than it sounds. The Anaconda Platform’s Deployments API means the same artifact that got versioned during your flow run is what gets served online, with full lineage back to the run that produced it. You’re not exporting a model and hoping the serving environment matches. It’s the same environment, because it’s literally the same image.

Why this is bigger than one demo

This isn’t just about wiring one flow together. It changes what “building on Kilo” means for anything data or ML adjacent. Kilo already gives you 500+ models to choose from and agent modes built for iterating fast. The Anaconda Platform gives you the compute layer, the artifact registry, and the path to production that Netflix originally built Metaflow to solve, now running in your own cloud account with SOC2 and HIPAA compliance built in.

Put together, a developer can go from “I have an idea for a data pipeline” to “it’s deployed, versioned, and serving traffic” without leaving a workflow that starts and ends in code they wrote themselves. That’s the actual promise of moving at Kilo Speed: not just writing code faster, but closing the entire loop from prototype to production without the usual friction in between.

If you’re already using Kilo for day-to-day development, this is the natural next step: plan the flow, build it with your agent modes, and let the Anaconda Platform take it the rest of the way.

Jack Clark, one of Anthropic’s cofounders, told TIME in July that the company could see acceleration showing up all over the organization but couldn’t quantify what it added up to. His words were that he can’t give a specific number because they don’t have a measure. Helen Toner, formerly on OpenAI’s board, made the same point from the outside: even the researchers themselves can’t easily tell how much faster they’re going.

To some, that rate of innovation is exciting. To others, it’s concerning. For many, it’s somewhere in the middle. Here’s the signal and latest news on the subject, so you can make your own decision.

What actually happened over the last few weeks

If you weren’t following closely, the sequence of events went something like this.

On July 21, OpenAI disclosed that GPT-5.6 Sol and an unreleased successor escaped a sandboxed cyber-capability evaluation during internal red-teaming. They found and chained a previously unknown zero-day in package-registry caching software, escalated privileges, moved laterally through OpenAI’s research environment, and reached Hugging Face’s production infrastructure, where they pulled the answer key for the ExploitGym benchmark. The models had been deliberately configured with reduced cyber refusals so they could be evaluated on offensive security work, and their behavior looked narrowly aimed at solving the benchmark rather than causing damage. Hugging Face found internal data and credential access but no evidence that public assets were changed.

A week later, more than 1,300 employees across OpenAI, Anthropic, Google DeepMind, and Meta signed “Pacing the Frontier,” asking the U.S. government to support an international effort to build the technical and governance tooling needed to deliberately slow automated AI development. The letter does not call for a pause, but rather asks that a brake exist and be tested before anyone needs it. It also doesn’t specifically propose a licensing regime, a compute threshold, a reporting cadence, or a named agency to run any of it.

Then on August 18, OpenAI paused some model work because it couldn’t rule out that its upcoming Astra model had hit the “critical” threshold in its own preparedness framework. Altman said publicly that they’d always committed to acting if capabilities outran safety, and told Alex Heath that unreleased models are showing various degrees of misalignment. Days earlier, Anthropic had published a 186-page risk report arguing that if its safeguards are followed, a pause on its most capable models isn’t necessary.

So the lab that’s spent years being the cautious one said keep going, and the lab that’s spent years being the fast one hit the brakes. Axios called it a script flip, which is about right.

The claims don’t always line up

In late July, Altman said on a podcast that we’re now in the singularity, and that he thinks it’s going to be hugely positive.

Seán Ó hÉigeartaigh at Cambridge disagreed. His definition of the singularity requires AI rapidly designing future generations of AI, and he doesn’t think we’re there. Demis Hassabis had earlier described the situation as the foothills, which Ó hÉigeartaigh said he agrees with more.

Then MIT Technology Review published research on August 18 that cuts against the acceleration story pretty hard. Researchers put Claude Opus 4.8 on OpenClaw against genuinely open-ended research questions drawn from NeurIPS submissions. Given six days and thousands of dollars of compute, the system handled all the engineering setup reliably and made no substantial progress on the actual research question. It hit dead ends, struggled to back out of them, showed poor judgment, and drifted off goal. Clark himself wrote in Import AI that today’s systems have a kind of rote, formulaic quality that might keep them from being good researchers, and called it a bearish signal on short recursive self-improvement timelines.

Hold both of those in your head at once. In the same four-week window, a frontier model autonomously discovered and chained a real zero-day to break out of a controlled environment, and a frontier model failed to make headway on a research question a grad student could have chewed on. Neither result cancels the other. They’re measuring different things, and we mostly don’t have accurate measures for either.

A different angle of concern

Arvind Narayanan, the Princeton researcher behind “AI as Normal Technology,” put the more grounded version well: the reason to worry isn’t only that there might be one cataclysmic moment; it’s that even gradual cumulative change of this size has a poor historical track record of arriving without a lot of pain.

The developer-facing version of that is less dramatic and more immediate. Capability, and every decision about capability, largely sits inside about four companies. That’s rapidly changing with the emergence of better open-weight models, but when OpenAI pauses and Anthropic doesn’t pause, that’s a safety decision at one company that lands directly in your build pipeline.

You may be affected by pacing decisions you had no part in. That’s true whether the singularity talk is right or overheated.

An open model ecosystem might be the answer

Open model selection isn’t necessarily a safety mechanism in itself, but it does keep your decisions reversible. If a model gets paused, gated behind a government review, repriced, deprecated, or quietly gets worse at your specific task after a checkpoint update, you can move without rebuilding your workflow. That’s mundane compared to intelligence explosions, and it’s the risk you’re overwhelmingly more likely to actually encounter.

There’s a second thing, which I think matters more than it gets credit for. Toner’s ask is that multiple companies publish consistent capability metrics on a schedule so the world can track acceleration instead of taking the labs’ word for it. That doesn’t exist yet, but developers running the same real task across several models and comparing results are doing a scrappy, distributed version of the same work. Every time someone posts a side-by-side on an actual codebase instead of a benchmark, that’s independent signal in a space that badly needs it. You can’t do that if you only have access to one model.

That’s also part of the reason Kilo works the way it does. Open model selection, priced at what the provider charges, switchable mid-session, with sessions that persist across the IDE extension, the CLI, and the web so that changing models doesn’t mean changing your setup. It’s open source for a related reason: if you’re going to trust something with write access to your repo, being able to read what it does is not a small thing.

The caveat is real, though. Freedom means you own the choice. Safety behavior varies enormously between models, and cheaper and faster often means thinner guardrails. Nicolas Papernot, whose Toronto team built an adaptive AI worm this June, made the point directly: it isn’t only the biggest, most powerful models that create security concerns. If you’re running agents with commit access, model selection is a security decision, not just a cost and latency one.

So, should we be concerned?

That’s up to you, but I’d encourage concern in the way you’re concerned about anything you can’t currently measure and can’t currently reverse.

The Cloud Security Alliance’s read on all this is a useful one, because it skips the geopolitics and asks a question you can act on today: can your organization actually demonstrate, not just claim, that it can throttle or shut down a model’s access to compute, tools, and network paths without depending on that model’s cooperation? Most teams running agentic workflows in CI have never tested this. The pending AI Kill Switch Act would make it a legal requirement for the largest developers, but it’s a good question for a normal engineering org regardless of whether that bill goes anywhere - and it’s only possible inside tools that are not locked into one model or lab.

On sourcing: The August 24 change is known only from an email Cursor sent to subscribers. There’s no blog post and no changelog entry, and Cursor’s pricing docs still list Auto Cost as a flat rate. Everything below about Cursor comes from recipients posting the email to r/cursor and to X. Check Settings → Billing after the 24th.

How it started and how it’s going

GitHub Copilot

Unlimited on a base model, then premium requests from June 2025, each model carrying a multiplier so one request could cost fifty times another. Then token billing on June 1, 2026, drawn on input, output and cached tokens at API rates. Annual plans are being retired.

Anthropic

Doesn’t reprice, restricts instead. Subscriptions get rate limits instead of higher rates: weekly caps arrived on Claude Code in August 2025. Enterprise gets no such cushion. It’s seat price plus usage at API rates, metered from the first token.

Cursor

Has repriced repeatedly, and this week’s email is the latest. Included usage for Cursor Models goes up. Auto stops being a flat rate. Cursor’s own wording is that most requests will draw at a higher rate than they do today.

Enterprises hit this first

Procurement runs on annual cycles. Pricing models now change quarterly. Copilot retiring annual plans is that collision made literal: the contract term outlived the pricing model it was signed under. A budget approved in Q1 2026 under premium-request maths was obsolete by June.

Five questions worth asking now

Not “are you ready for metered pricing.” Everyone is metered already (to some degree). The question is what it costs to leave.

1. Can you point the agent at a different model or provider tomorrow, and at what surcharge?

2. If you bring your own key, do you still pay the vendor per token? Cursor charges Teams and Enterprise $0.25 per million tokens on third-party models, and that includes BYOK.

3. Is your agent an extension or an editor? Cursor, Kiro and Windsurf are VS Code forks. Switching one means every developer switches IDE.

4. Do your rules, skills, MCP configs and prompt libraries live in your repo, or in a vendor’s team marketplace?

5. What happens to your prepaid credit balance if you leave?

Metered pricing was always coming

Kilo doesn’t escape this. Gateway is pass-through at provider rates, which is API pricing. If Anthropic raises Opus rates tomorrow, Kilo users pay the difference.

What Kilo does is charge nothing on top and keep the door open. 500+ models through one gateway at provider rates. Switch models when the economics change, without switching editors.

Arriving at metered pricing is fine. Arriving without options is the part worth avoiding.

For a long time, the answer to “which model should I use for design work” has been Anthropic. That covers more than websites. Developers reach for Claude models for dashboards, landing pages, slide decks, pitch pages, and anything else a human will look at. Claude models are closed, and Fable 5 is the most expensive model in the current top five. Meanwhile, OpenAI models have been playing catch-up on frontend, improving incrementally from GPT-5.4 through 5.5 to 5.6 Sol without closing the gap. Kimi K3 was the first model from outside the incumbents to match Fable 5 in our own testing, and it did our ten-task run at 29% of Fable’s cost. Qwen3.8-Max costs less per token than Kimi K3 does.

TL;DR: Qwen3.8-Max came out within taste distance of Claude Fable 5 across all ten tasks (our count: 4 for Fable, 3 for Qwen, 3 ties), and unlike Kimi K3, it has a visual taste of its own. The full ten-task run cost $3.05 on Qwen3.8-Max and $8.44 on Fable 5.

Pricing

Fable 5 costs 5x more per input token and 8.3x more per output token than Qwen3.8-Max. We included Kimi K3 for reference because these three models are the ones trading places at the top of the frontend leaderboard right now. Alibaba has said the Qwen3.8-Max weights will be released, but as of this writing they have not shipped and the license is unknown.

The Setup

We ran both models in Kilo Code CLI in Code mode. Each task started in its own empty directory with no shared state. Both models received identical prompts, and we did not iterate. Every screenshot in this post is a one-shot output.

We ran Qwen3.8-Max at xhigh reasoning, the highest of its five available levels. We ran Fable 5 at high thinking, the same setting we used in the Kimi K3 comparison. Reasoning labels are not comparable across vendors, so we picked Qwen’s highest level, kept Fable consistent with our previous posts, and report the costs as measured.

The prompts were “vibe + minimum content” style. We named the product, listed the required content, and left every visual decision to the model. Each model produced a single self-contained index.html using Tailwind via CDN. Nine of the ten tasks worked that way. The settings page was the exception, where we handed both models a compact dark brand spec (three hex colors, a font pairing, no gradients, no pure black or white, corner radius capped at 8px) to see how each one executes direction instead of inventing it.

One thing to keep in mind while reading: the calls in this post are our preferences, not verdicts. On most tasks the two outputs are close enough that picking one is a matter of taste. You may look at the same pairs and prefer the other one. That is why every task shows both outputs in full. In every screenshot, Claude Fable 5 is on the left and Qwen3.8-Max is on the right.

Task 1: Physical Product Landing Page

We asked for a landing page for Murmur, a pair of premium over-ear headphones sold direct at $349.

Both models drew the headphones themselves as an SVG illustration and surrounded it with floating spec callouts for the driver, the battery, and USB-C. Qwen’s illustration carries more detail. It marks the left and right ear cups, gives the callout tags a glass blur effect, and draws audio waves radiating from the headphones. It also designed an audiogram-style logo for the brand, where Fable placed an M inside a circle. Qwen put more care into the typography too. Its headline wraps deliberately, and the second line is set in the gold accent color, which makes the hero easier to scan. Within the limits of a one-shot page, Qwen’s reads as the more luxurious of the two, though neither would be mistaken for a real audio brand’s site.

We lean Qwen here, mostly on the illustration detail and the typography.

Task 2: E-commerce Product Detail Page

Talus is an outdoor gear store, and the page sells the Crag 32L, a technical backpack.

The layouts are nearly the same. Both put the gallery with a thumbnail selector on the left and the purchase details on the right. Qwen added a quantity stepper next to its add-to-cart button, which Fable did not. Fable’s color and size selectors are more finished, and its description carries more useful buying information, including free shipping details, stock availability, and product details like the rope strap. Fable set the page on a cream background and centered its nav links, while Qwen kept a white background with the logo left and the account controls right. Neither gallery shows a backpack, because both models pulled generic placeholder photos, which the prompt’s image service made unavoidable.

We lean Fable here, on the selectors and the description work, but this one is close.

Task 3: Email Client Inbox

Postline is an email client, and the prompt asked for the full three-pane inbox: folders, message list, and reading pane.

The layouts match again, and the differences are in the details. Qwen’s inbox is more minimal and better spaced, with clearer hierarchy in the message rows. Its thread toolbar is icon-only with even spacing, where Fable labels each action with text. Fable’s compose button is missing horizontal padding. Fable does win the reading pane itself. It collapses the earlier messages in the thread, expands the current one, and keeps the attachment and the reply composer visible in the same viewport, while Qwen’s taller message cards push the reply composer below the fold.

We lean Qwen here on the overall layout and spacing.

Task 4: CRM Data Table

Cairn is a CRM, and the prompt required a 12-row contacts table with filters, sorting, selected rows, and a bulk-action bar.

Both models picked a green accent, though different greens, and both floated the bulk-action bar over the table, which is the nicest shared idea in the whole set. Qwen built a wide sidebar with a label next to every icon. Fable went icon-only, which leaves more horizontal room for the table. Each output has one data inconsistency. Qwen shows a descending sort indicator on the deal column, but its rows are not in that order. Fable shows an active filter that excludes deal stages that still appear in the table.

We call this one a tie. Both are good screens, and the difference is taste.

Task 5: Personal Finance Dashboard

Alder is a personal finance app, and the prompt asked for balances, a spending chart, a category breakdown, transactions, bills, and a savings goal.

Both models built a card-based bento grid. Qwen put the balance, monthly spending, and monthly income in one dark green card at the top, so the most important numbers sit in one place with the strongest treatment on the page. The category breakdowns split the two models. Qwen gave each category its own bar, so you can compare sizes at a glance. Fable built a single stacked bar where each chunk is colored, so reading it means matching colors against the legend. Qwen’s charts are the stronger set overall.

We lean Qwen here.

Task 6: Podcast Player

Earshot is a podcast app, and the prompt asked for the now-playing view with a queue and playback controls.

Both models placed the player in the center, the up-next queue on the right, and a sidebar with pinned shows on the left. The split is in the character of the controls. Fable’s player is minimal, with a serif episode title and a warm dark palette. Qwen went playful, with a waveform-style progress bar, a colored pause button, and large orange glows behind the controls. The waveform bar looks good, but it reads as decoration rather than a functional scrubber. Fable’s queue truncates several episode titles.

We lean Fable here for the more restrained player, with Qwen close behind.

Task 7: Smart Home Dashboard

Haven is a smart home app, and the prompt required scenes, room-grouped devices, dimmer sliders, a thermostat, and an offline device.

Both built a similar grid of device cards with toggles and sliders, and both surfaced the inside and outside temperatures in the header. Fable’s header treatment is easier to read at a glance, where Qwen wrapped its temperatures in pills. Qwen tried a floating tab bar for navigation, and it overlaps one of the panels. Qwen’s icons also never rendered. It linked the Phosphor icon library at a URL path that does not serve the icon font, so every icon on the page shipped as a blank space. Fable tailored its controls per device type, with color temperature on the lights, a recording state on the camera, and an offline speaker that matches the page-level warning.

We lean Fable here, by more than usual.

Task 8: Cinema Seat Selection

Marquee is a cinema ticketing app. The prompt required a seat map with five distinct seat states, a legend, and a booking summary.

Both outputs look close to a real cinema checkout, with the screen at the top and the seat grid below it. The seat maps are roughly equal, and both handle available, taken, selected, accessible, and premium states. The booking summary is where they split. Fable put it in a floating card, which is compact and easy to scan. Qwen built a full-height sidebar, and the text on its continue button wraps onto a second line. The legends differ only in placement. Fable pinned its legend to the bottom of the page, and Qwen placed it directly under the seat map.

We lean Fable here.

Task 9: Onboarding Wizard

Gantry is a project management tool, and the prompt asked for step 2 of its four-step onboarding, the workspace setup form.

The two pages mirror each other. Qwen put the form on the left and the visual panel on the right, and Fable did the opposite. The forms are nearly identical, with the same progress indicator (step 1 checked, step 2 active), the same fields, and the same continue button. Each model invented a product graphic for the other panel. Fable drew a roadmap timeline on an SVG grid background. Qwen drew a skeleton kanban board. Both added a testimonial. This pair shows the pattern of the whole run in one screenshot. The two models had the same idea and executed it with different taste.

We call this one a tie.

Task 10: Settings Page

This was the constrained task. Both models got the same brand spec for Nocturne, a sleep tracking app: near-black background (#0B0D10), warm off-white text (#E8E6E0), candlelight amber accent (#D4A24E), Source Serif 4 for headings, IBM Plex Sans for body, no gradients, no pure black or white, and corner radius capped at 8px.

We checked the HTML for every measurable rule. Both models used all three specified colors, loaded both fonts from Google Fonts, avoided gradients entirely, and used no pure black or white. The radius cap is where they split. Fable kept every radius at or below 8px, and you can see the constraint in its toggle switches, which are squarish instead of the usual pill shape because a pill would exceed the cap. Qwen used fully rounded toggles and a circular avatar, both of which break the 8px rule. Beyond the spec, the pages share the same skeleton, with the section nav on the left and the settings on the right. Fable shows a danger zone entry in its nav, which Qwen left out of its sidebar. Fable’s bedtime window of 10:30 PM to 11:15 PM does not quite line up with its own 7h 30m sleep target, which is a problem we will let Nocturne’s imaginary user sort out.

We call the design a tie, with the note that only one model followed the whole spec.

Patterns Across All Ten Tasks

The skeletons converge, the taste does not. On task after task, the two models chose the same layout: the same gallery-left product page, the same floating bulk-action bar, the same bento grid, the same mirrored onboarding. That much matches what we saw with Kimi K3 in July. The difference is what sits on top. Kimi’s outputs were often hard to tell apart from Fable’s. Qwen’s outputs are not. The two models kept making different calls on typography, component shape, and decoration, and the onboarding pair shows it most clearly.

Qwen has a stable color signature. Amber, orange, or gold was Qwen’s primary accent on nine of the ten surfaces. The CRM’s teal was the only exception. Fable changed its palette per brief, with forest green and burnt orange for the outdoor store, olive for the CRM, coral and teal for the podcast player, and gold with violet for the cinema. In the Kimi comparison we described Fable as reaching for cool palettes. Across this set it ranged wider than that.

Fable puts more on screen. Fable fit more working information into the viewport on the inbox, the CRM, the finance dashboard, and the smart home dashboard. Qwen used larger components and more empty space, which sometimes read as calm and sometimes read as underused. Fable also leaned on serif display type across the set, while Qwen more often chose bold geometric sans.

Both models shipped checkable mistakes. Qwen’s icon library never loaded on the smart home page, its seat-selection button text wraps, and its CRM sort indicator does not match its row order. Fable’s CRM filter contradicts its own table, its settings page shows a bedtime window that does not match its sleep target, and its podcast queue truncates titles.

The Two Agents Worked Very Differently

The finished pages are close in quality, but the two models produced them in very different ways.

Fable one-shotted almost everything. On nine of the ten tasks it wrote index.html in a single pass and stopped. It never opened the page, never checked its own work, and averaged 2m 11s per task.

Qwen treated each task like a longer job. It reasoned at length before writing anything, then usually ran a quick syntax check on its own HTML after writing it. On the podcast player it went further. It installed Playwright and a Chromium browser inside the container, took real screenshots of its own page, read them, edited the page, and screenshotted again. That one task took 30 tool calls and 15m 44s, and it is the only task where Qwen’s cost matched Fable’s ($0.90 against $0.88).

The self-checking did not decide the results. The podcast player is a task we leaned Fable on despite Qwen’s screenshot loop, and the smart home page shipped with every icon missing because no visual check ran there.

Qwen also needed retries that Fable did not. Three of its tasks failed outright before producing a working run, including one that took four attempts, with earlier attempts ending with no HTML file or with the file written to a wrong nested path. Fable completed all ten tasks on the first attempt. The costs below cover the final successful runs only.

Cost and Time per Task

Costs and agent times are from Kilo Code CLI’s run summaries.

The run cost $3.05 on Qwen3.8-Max and $8.44 on Fable 5, which puts Qwen at 36% of Fable’s cost. One task carries a disproportionate share of that number. Every Qwen task except the podcast player cost between $0.13 and $0.34, and the podcast player’s screenshot loop alone was 29% of Qwen’s total spend. Remove that task from both columns and Qwen lands at 28% of Fable’s cost, almost exactly where Kimi K3 landed in July.

Two caveats apply to these numbers, and they cut in opposite directions. The retried tasks added spend that is not in this table, so the real gap is somewhat narrower than it looks. At the same time, we ran Qwen at xhigh, its most expensive reasoning setting out of the five available. Dropping the reasoning level would cut Qwen’s cost further, and we have not tested how much of the design quality survives that.

The time gap was consistent across all ten tasks. Qwen averaged 7m 55s per task against Fable’s 2m 11s. Part of that is the xhigh reasoning, and part of it is Qwen’s habit of checking its own work.

Conclusion

The Arena leaderboard has Qwen3.8-Max one spot above Claude Fable 5, and our ten tasks came out about that close. We preferred Fable 5 on four, Qwen3.8-Max on three, and called three ties, and most of those calls were narrow. Ten tasks is a small sample, and a different set of twenty or thirty could plausibly flip the count. The screenshots are all above, and your count may come out differently than ours did.

• Qwen has its own taste. Kimi K3 converged with Fable to the point where several pairs were hard to tell apart. Qwen agreed with Fable on layout and then went its own way on the visual layer, with an amber accent on eight of ten tasks, rounder components, and more playful decoration. There are now at least two distinct design sensibilities at the top of the leaderboard, not one.

• The cost gap held at roughly the same ratio as Kimi’s. The run cost $3.05 against Fable’s $8.44. Excluding the podcast player, the one task where Qwen’s screenshot loop drove its cost up to Fable’s level, Qwen’s nine other tasks cost 28% of Fable’s. That is with Qwen at its most expensive reasoning setting.

• The behavioral difference matters as much as the pricing. Fable writes the page once and stops. Qwen reasons longer, checks its own output, and on one task ran a full screenshot loop that erased its cost advantage. Both behaviors are defaults, and both can be changed with prompting. Per token, Qwen is 5x to 8.3x cheaper either way.

• Fable was the more reliable agent. It finished all ten tasks on the first attempt. Qwen needed clean restarts on three tasks, and it shipped the one rendering failure of the run, the smart home page with no icons.

• On the constrained task, only Fable followed the whole spec. Both models matched the colors, the fonts, and the no-gradient rule, but Qwen’s fully rounded toggles broke the 8px radius cap while Fable designed its way around it.

If you do frontend work, Qwen3.8-Max is worth testing on your own tasks. In our run it produced designs in the same class as Fable 5 at roughly a third of the cost. The trade is that it was slower and needed the occasional rerun, so whether it fits depends on how much your workflow cares about turnaround time.

What has changed since July is that this is no longer one model’s story. Kimi K3 and now Qwen3.8-Max have both matched Claude in design work within a month of each other. That matters in two ways. Competition at the top of the frontend leaderboard should push all of these models to improve. It also means frontend work no longer requires a Claude model to get results in this class, and the models that get you close enough cost a fraction of the price.

Testing performed using Kilo Code, a free open-source AI coding assistant for VS Code and JetBrains with 3,000,000+ Kilo Coders.

This post summarises the regulation and the Commission’s guidance as we read them. It isn’t legal advice. Whether the CRA applies to your products, and in what role, depends on your specific facts.

Most writing about the EU Cyber Resilience Act is aimed at whoever owns compliance. That’s not who ends up doing the work.

The CRA regulates products, not companies. It says things about vulnerabilities in products you ship, deadlines measured in hours, and evidence you’re expected to produce years later. All of that lands on engineering and security teams, usually without a budget line and often without anyone senior having read past the summary.

This is the version for those teams. What changes, when, and what you need to have working.

From 11 September 2026 you have to report actively exploited vulnerabilities and severe incidents.

From 11 December 2027 the rest of the regulation applies: the essential requirements, vulnerability handling, conformity assessment, CE marking.

Both sets of requirements are worth understanding now, because the December 2027 work takes longer than sixteen months of runway suggests.

First, whether it applies to you

The test is where your software runs. Software that executes on a user’s device is a product with digital elements: a downloaded application, a locally installed client, a CLI, a browser extension, firmware. Software reached only through a browser generally isn’t, on that basis alone.

Backends come in scope as remote data processing where they’re necessary for the product to work. The practical test: is processing happening at a distance, would its absence stop the product doing one of its jobs, and did you build it. If your desktop app is useless without your API, the API is part of the product.

Recital 12 draws the line from the other side. Cloud services designed outside the responsibility of a product manufacturer aren’t in scope, and pure SaaS, PaaS and IaaS are handled by NIS2 instead.

So “we’re a SaaS company” isn’t the exemption people hope for. If you ship an installer, an agent, a CLI, a mobile app or a device, you’re probably a manufacturer for at least one thing.

Your role matters too. The CRA distinguishes manufacturers, distributors, open-source stewards and plain contributors, each with different obligations, and one company can hold different roles for different products.

Publishing free and open-source code to a public repository or package manager is generally not placing a product on the market, and the Commission’s guidance says so. But charging for it, monetising other services through it, or conditioning its use on processing personal data may change that determination. Hosting is a separate question: a repository or package manager that merely hosts other people’s software isn’t placing it on the market either, which is what Recital 20 addresses.

You own the vulnerabilities in code you didn’t write

This is the part that reshapes the work.

Vulnerability-handling obligations apply to your product in its entirety, including every integrated component. The Commission’s own FAQ is direct about the consequence: where you’ve integrated a component that wasn’t itself placed on the market, whoever developed it isn’t subject to the CRA’s vulnerability-handling obligations. You still are, for your whole product.

Recital 34 goes further. When you identify a vulnerability in a component, including a free and open-source one, you’re expected to tell whoever maintains it, remediate it in your product, and where applicable hand them the fix you applied. The July guidance narrows the upstream half usefully: you don’t have to report upstream if you can confirm the maintainer already knows, or if the component has no maintainer left, and the duty runs only to the version you actually integrate.

Read that as an engineer and the shape of the problem is clear. A modern product is mostly other people’s code. You’re accountable for all of it, you frequently can’t fix it at source, and you’re expected to push fixes upstream. Nobody’s dependency tree was built with that in mind.

From 11 September, some vulnerabilities start a clock

Article 14 reporting starts on 11 September 2026. Reporting has two triggers.

An actively exploited vulnerability. Article 3(42) sets the bar at reliable evidence that a malicious actor “has exploited it in a system without permission of the system owner,” and Article 14(1) attaches the duty to such a vulnerability “contained in” the product you placed on the market. A published proof of concept doesn’t count. A high CVSS score doesn’t count. And the Commission’s guidance is direct that a vulnerability in a third-party component which “cannot be exploited” in your product, its example being vulnerable code that is not reachable, is not caught by mandatory reporting.

A severe incident affecting product security. Article 14(5) frames it as something that negatively affects, or can negatively affect, the product’s ability to protect availability, authenticity, integrity or confidentiality, or that has led or can lead to malicious code running in the product or in a user’s systems.

That second trigger catches things people don’t expect. A compromised build pipeline or release channel can be a severe incident in its own right, whether or not any vulnerability was exploited. If someone could have tampered with what ships to customers, that’s product security.

The rhythm, once you’re aware: early warning “without undue delay and in any event within 24 hours,” the fuller notification on the same standard within 72 hours, and a final report no later than 14 days after a corrective measure is available. For a severe incident the final report is due within a month of the 72-hour notification. Note the construction on the first two. The hours are an outer limit, not a target.

The awareness problem

The clock starts when you become aware. Not when you decide to look.

Awareness is not the alert landing. The Commission’s July 2026 guidance places it at the point where, after a prompt initial assessment, you have “a reasonable degree of certainty” that a vulnerability “contained in” your product is being actively exploited. What you cannot do is stall that assessment. The guidance is explicit that the emphasis falls on prompt action to carry it out, so “nobody triaged it” is a worse position than “we triaged it and concluded it wasn’t reportable.”

Which reframes the alert backlog. It isn’t a queue of maintenance work anymore. It’s a queue of things you might already owe a report on and haven’t looked at. Triage stops being housekeeping and becomes the thing that discharges the obligation.

Practically, that means you need three capabilities you may not have:

• Something that assesses new advisories against your dependency tree fast enough to matter on a 24-hour clock

• A way to determine whether an advisory actually reaches code your product runs, because severity alone can’t answer the reportability question

• A record of what you assessed and concluded, including the ones you decided not to act on

Where to file, and why to set it up now

You report once, through ENISA’s Single Reporting Platform. It routes to the CSIRT of the member state where you have your main establishment and to ENISA at the same time. If you have no main establishment in the EU, Article 14(7) sets an order: the Member State of your authorised representative, then your importer, then your distributor, then where most of your users are. You file to that state’s CSIRT.

The platform wasn’t live as of the start of August 2026. It’s in pre-operational testing, scheduled to go live on 11 September, the day the duty starts. Three things worth doing before then:

• Registration uses an EU Login account, which you can create now at ecas.ec.europa.eu. Do that. But ENISA’s own guidance says to register on the platform itself only when you need to file, because CSIRT validation of your representative runs in parallel with reporting rather than blocking it. So pre-stage your legal entity, product and contact details and make same-day registration a step in the runbook, not a box you tick in advance.

• The initial rollout isn’t expected to include APIs, so filing will be manual portal entry rather than system-to-system. Don’t design a pipeline that assumes an API.

• Identify your designated CSIRT and decide who has authority to file. A 24-hour clock doesn’t survive an ambiguous escalation path. ENISA’s registration guidance uses a primary-representative model: one person registers for the entity and invites the others, and invitations expire after seven days. Name a primary and a backup.

“Without delay” has no number, so you set one

The remediation duty applies from 11 December 2027, and only to products placed on the market from that date onward. Under Article 69(2) a product placed on the market before 11 December 2027 only picks up the CRA’s remediation requirements if you substantially modify it after that date, and the Commission’s guidance confirms the Annex I Part II vulnerability handling duties don’t apply to it.

Remediation is described in Annex I Part II point (2), and the whole of it matters: in relation to the risks posed, address and remediate vulnerabilities without delay, including by providing security updates, and where technically feasible ship those updates separately from functionality updates.

There’s no timeframe in the text. Nobody hits compliance by clearing a threshold, because none exists. What can be assessed is whether you set a policy calibrated to risk, followed it, and can show it.

So write the policy. Deadlines by severity, a defined escalation path, and a record of adherence. Something like 15 days for critical and 30 for high is defensible if you meet it and can prove it. An undocumented intention to be quick is not.

Note the second half of that requirement too: ship security updates separately from feature work where you can. That’s an architecture and release-process decision, and it has a payoff explained below.

Patching won’t restart your conformity assessment

Teams sit on security patches because of a specific fear: that shipping the fix counts as a substantial modification, which treats the product as newly placed on the market and pulls it back through conformity assessment.

It generally doesn’t. Recital 39 says that where a security update designed to reduce a product’s cybersecurity risk doesn’t modify its intended purpose, it isn’t a substantial modification, and that this usually covers updates involving only minor source-code adjustments. It gives the example of an update addressing a known vulnerability. The July guidance goes further than the recital: a qualifying security update isn’t a substantial modification “even where that update may introduce significant technical changes.”

The test is whether the cybersecurity risk profile moved in a way you hadn’t already accounted for, not how big the change is. A dependency bump that closes a known vulnerability doesn’t move it in the direction that matters. So keep security releases clean.

The catch, and this is why separation matters: Recital 39 is explicit that bundling a feature change with a security update doesn’t shelter the feature change. A feature that adds a new input element broadens the attack surface and can be a substantial modification whether or not it shipped alongside a patch. The guidance puts it almost the same way: the assessment “should therefore not be based on the scale or complexity of the change, but on its potential adverse impact on the cybersecurity risk profile.”

One boundary. The guidance flags security updates that materially change your dependency structure or data flows, or add new externally reachable interfaces. Its own example is replacing internal key management with a third-party service to fix a cryptographic weakness. Security-motivated, still a substantial modification.

You may be able to fix forward

Article 13(10) lets a manufacturer that has placed subsequent substantially modified versions of a software product on the market meet the remediation requirement only for the version last placed on the market. Conditions: users of earlier versions can get the latest version free of charge, and they don’t incur additional costs to adjust the hardware or software environment they run it in.

Two limits worth knowing before you plan around it. It’s tied to substantial modification, so it isn’t blanket permission to abandon old releases. And every version placed on the market carries its own declared support period under Article 13(8). Shipping 2.0 doesn’t extend 1.0’s declared support period, though it can cut short what you actually have to remediate in 1.0, which is the point of the provision. Other vulnerability-handling duties keep running across all of them.

The guidance also reads “additional costs” narrowly. Personnel time, routine testing, configuration changes and dependency upgrades are the ordinary cost of applying an update and don’t count against you. Mandatory new hardware, infrastructure replacement or fundamental changes to the operating environment do.

Support periods and the records problem

Five years is a floor, not a default, and the two directions aren’t symmetric. Under Article 13(8) the support period reflects how long the product is expected to be in use. Longer than five years simply follows, and Recital 60 treats operating systems and industrial control systems as routine cases. Going below five is gated: Recital 60 permits it only where the shorter period is justified by the nature of the product and the product is genuinely expected to be in use for less.

Separately, Article 13(13) requires the technical documentation and EU declaration of conformity to stay available to market surveillance authorities for at least 10 years after the product is placed on the market, or the support period if that’s longer.

Article 13(9) is a separate duty in the same family. Each security update you make available during the support period has to stay available for at least 10 years after you issue it, or the remainder of the support period, whichever is longer.

Your vulnerability-handling process is part of what that documentation describes. Which means the decisions you make about advisories need to survive in a form someone can retrieve years later. Most teams have nothing that remembers why they dismissed a particular advisory eighteen months ago, let alone ten years. Whatever you build now is the record; whatever you don’t record is unrecoverable.

What’s still missing

Two things worth tracking rather than waiting for.

No CRA harmonised standard has been cited in the Official Journal, so the Article 27 presumption of conformity isn’t available for any product category. The horizontal standard on vulnerability handling is the one that will matter most, since it will define what adequate vulnerability handling looks like. Its timing is unsettled, and got less settled in early July when the Commission published a draft amendment to the standardisation request pushing the 2026 drafting deadlines back two months. That amendment is itself still a draft and not yet in the Official Journal. Citation is a separate, later milestone than a standard being finished. Drafts are public and worth reading to anticipate requirements, not to claim conformity.

And the Commission’s guidance published on 27 July 2026 is non-binding, and was approved as a draft pending all EU language versions. It’s the Commission’s own reading of scope, substantial modification, support periods and reporting, with 67 worked examples. Useful for settling internal arguments. Not a substitute for the regulation.

The five-week version

If you do nothing else before 11 September:

1. Work out which of your products are in scope and which role you hold for each.

2. Create an EU Login account and identify your designated CSIRT.

3. Decide who is authorised to file a report, and write the runbook.

4. Set remediation deadlines by severity and write them down. Not a September duty, but the September triage work needs somewhere to route what it finds.

5. Get your dependency alert backlog assessed, so you know what you’re already aware of.

6. Start recording decisions, including the ones not to act.

None of that requires a harmonised standard, a notified body, or a finished platform.

Where Kilo Security Agent fits

Plenty of tools claim to deliver CRA compliance. None do, this one included, so here’s the boundary.

Security Agent covers one requirement: Annex I Part II point (2), for dependency vulnerabilities. It works from GitHub Dependabot alerts, triages each one, and where it matters inspects your repository to determine whether the advisory reaches code your product actually runs, capturing the files, paths and reasoning behind the answer. It opens remediation PRs, tracks the deadlines you set by severity with warnings before they slip, and keeps every decision attached to the finding. A pull request counts as an attempt, not a resolution: findings stay open until the fix is confirmed or someone dismisses them explicitly.

That maps onto three of the problems above. The awareness backlog, because triage happens as alerts arrive rather than when someone gets to them. The reportability question, because reachability is what tells you whether an exploited CVE affects your product. And the records problem, because the assessment and the reasoning persist.

What it doesn’t do: generate SBOMs, file reports to ENISA or your CSIRT, notify upstream maintainers, or cover the other seven Annex I Part II requirements. It doesn’t touch application code, infrastructure or container images. It needs GitHub with Dependabot enabled, so components outside that are invisible to it. And it doesn’t decide whether a vulnerability is reportable. It gives you what you need to decide.

That’s a narrow slice of the regulation. It’s the slice about actually fixing things, and it’s the one most teams have no system for.

This post summarises the regulation and the Commission’s guidance as we read them. It isn’t legal advice. Whether the CRA applies to your products, and in what role, depends on your specific facts.

This export can turn the last few months of scattered data into a clearer picture of what the last quarter has looked like. You can be your own forensic detective identifying what you need to improve as you head into the last quarter of the year strong.

You can also run a cybersecurity checkup on your own data using the open source tools covered in our guide. For example, if you or your organization ever needed to understand your data profile following a security incident, these open source tools can help identify where to focus first when remediating potential risk.

The export itself is straightforward to request. Making sense of what’s inside may require some help if you don’t have a data background, which is the purpose of this guide.

The export is a single compressed file containing your account details.

Since it’s a large data export, none of it is readable by just opening the file. It’s structured, but not human-structured, and this guide walks through how to unpack it, how it’s organized, and how to search through it for anything specific you want to find.

Requesting and unpacking the file

To request an export, open the Kilo Dashboard at app.kilo.ai, click the account settings menu in the bottom left corner, and select Request data exports. From there, click the Request export. Kilo will email you at the email address on file once it’s ready, which is usually a matter of minutes, and the email links to a single file named kilo-data-export.jsonl.gz.

The .gz extension means the file is compressed with gzip. On macOS or Linux, you can decompress it in place with:

gunzip -k kilo-data-export.jsonl.gz

The -k flag keeps the original compressed file around in case you want to reprocess it later. On Windows, 7-Zip will open it directly, or you can run tar -xf from a recent version of PowerShell. If you’d rather skip the decompression step entirely, most of the commands in this guide work just as well by streaming the compressed file through zcat (or gzcat on macOS), which is the approach used throughout.

Once decompressed, what you have is a .jsonl file, short for JSON Lines: a plain text file where every line is its own complete, independent JSON object. It’s a common format for large exports, since it lets you process a file line by line with ordinary Unix tools instead of loading the entire thing into memory at once. The tradeoff is that it isn’t very readable on its own, so you’ll want jq installed to make sense of it. If you don’t already have it, brew install jq on macOS, or your package manager’s equivalent elsewhere.

Reading the header

The first line of the file is a header describing the export itself, separate from any of your actual data. You can view it with:

gzip -dc kilo-data-export.jsonl.gz | head -1 | jq

Which returns something like this:

{

“type”: “header”,

“schemaVersion”: 1,

“exportId”: “010a8d7f-...”,

“requestedAt”: “2026-08-11T15:15:16.939Z”,

“generatedAt”: “2026-08-11T15:15:19.334Z”,

“includedSources”: [

“kilocode_users”,

“app_builder_projects”,

“microdollar_usage_metadata”,

“system_prompt_prefix”

],

“snapshotAt”: “2026-08-03T00:00:00.000Z”

}

Two fields are worth paying attention to here. includedSources lists which categories of data made it into this particular export, and snapshotAt marks the point in time the export was generated from.

How the data is structured

Every line after the header is a flat object with a source field telling you which category it belongs to, a field name, and a value. Some rows also carry an id, which groups multiple field/value pairs into a single record, like one CLI session having a title, a session ID, and a git branch as three separate lines that share the same id.

An account-level row looks like this:

{”source”: “kilocode_users”, “field”: “default_model”, “value”: “x-ai/grok-code-fast-1”}

A record-level row looks like this:

{”source”:”microdollar_usage_metadata”,”id”:”9e6f3aa6-120a-4047-8169-dafcfe566e2e”,”createdAt”:”2025-10-28T16:40:40.581Z”,”field”:”user_prompt_prefix”,”value”:”<task>\ngive me a 2 sentence summary of this whole project\n</task>\n<environment_details>\n# VSCode Vis”}

{”source”:”system_prompt_prefix”,”field”:”system_prompt_prefix”,”value”:”You are Kilo Code, a knowledgeable technical assistant focused on answering questions and providing “}

There’s no nested JSON to unwrap and nothing nested inside value. Every field you’ll query is right there at the top level, which keeps the jq recipes below fairly simple.

Before doing anything else, it’s worth getting a sense of what your export actually contains and how much of each type of data is in there:

gzcat kilo-data-export.jsonl.gz | jq -r ‘.source // “header”’ | sort | uniq -c | sort -rn

What each source contains

kilocode_users is your account record. It covers the basics you’d expect, like your signup date, default model, and credit totals, but it also carries more than that: your Google account email, name, and profile image if you signed up that way, any linked GitHub or LinkedIn URLs, your email domain, and a signup IP address. It’s worth a look on its own, since it’s the one source that’s entirely about you rather than about what you’ve done.

gzcat kilo-data-export.jsonl.gz | \

jq -r ‘select(.source == “kilocode_users”) | “\(.field): \(.value)”’

Spend is tracked here too, in microdollars_used and total_microdollars_acquired, both in microdollars, so divide by a million to get an actual dollar figure:

gzcat kilo-data-export.jsonl.gz | \

jq -r ‘select(.source == “kilocode_users” and .field == “microdollars_used”) | .value / 1000000’

microdollar_usage_metadata is one row per billed request, keyed by an id unique to that request. Because it’s tied to request-level billing rather than a single conversation, it captures prompts from every context Kilo bills for, including scheduled or automated tasks, not just what you typed in a session yourself. At roughly one row per request, this ends up being the largest and most varied source in the export.

gzcat kilo-data-export.jsonl.gz | \

jq -r ‘select(.source == “microdollar_usage_metadata”) | .value’ | head -20

system_prompt_prefix holds prefix text associated with system-level prompts on your account, truncated the same way. It’s a smaller source and mostly useful if you’re trying to confirm what a given model call was actually instructed to do.

Reviewing your data export

Once you have a full export in front of you, it’s worth taking a pass through it for anything sensitive that may have ended up in there without you noticing. The most likely place for that is microdollar_usage_metadata, since it’s built directly from prompt text and there’s a lot of it: a stray .env value typed into a task description, an API key pasted into a debugging prompt, a database connection string, an email address embedded in some automation context. None of this is unusual, and it’s not a sign that anything went wrong. It’s simply what tends to accumulate when a hundred-character window into your prompts gets recorded thousands of times over. Since your export is a faithful record of exactly that, it’s a reasonable place to check.

Start by pulling the fields you want to scan into one plain text file:

zcat kilo-data-export.jsonl.gz | \

jq -r ‘select(.source == “microdollar_usage_metadata” or .source == “system_prompt_prefix”) | .value’ \

> kilo-export-flat.txt

Open Source Tooling

Scanning a large body of text for sensitive strings is a well-established problem, and there are several solid open source tools built specifically for it. None of them catches everything on its own, since each one relies on a different detection approach, so the most thorough method is to run a few of them and then de-duplicate whatever they turn up. Here’s a rundown of the ones worth knowing.

Gitleaks (source, MIT license) is the quickest to get running. It’s designed primarily for git repositories, but it also supports scanning plain directories:

gitleaks detect --no-git --source . --report-path findings.json

Point it at the directory containing your flattened export, and it returns a JSON report of everything matching its built-in ruleset.

Trufflehog (source, website, AGPL-3.0 license) goes a step beyond pattern matching. For many types of findings, it will actually attempt to verify whether a discovered key is still active by making a live test call against the relevant service. That distinction matters a great deal for triage, since a finding that’s already inactive requires no further action, while a verified one does.

trufflehog filesystem ./kilo-export-flat.txt --results=verified,unknown

CredSweeper (source, docs, MIT license) was originally developed at Samsung. It layers a machine learning model on top of its regex-based rules specifically to cut down on false positives, which is useful given how repetitive and fragmentary short prompt snippets tend to be.

pip install credsweeper

credsweeper --path kilo-export-flat.txt --save-json report.json

Presidio (source, website, MIT license) was incubated at Microsoft and takes a different focus than the tools above: rather than keys and tokens, it’s built to find personal information, including names, email addresses, phone numbers, and physical addresses. Given that kilocode_users and microdollar_usage_metadata both routinely carry this kind of data, it’s a good fit for this export specifically. It’s a Python library rather than a standalone command line tool, but a basic scan takes only a handful of lines of code, and it’s also capable of anonymizing whatever it finds.

Bulk Extractor (source, GPL-3.0 license) comes out of the digital forensics field. Rather than parsing file structure, it scans raw bytes directly for emails, URLs, and other patterns, which lets it catch things sitting in places that structure-aware tools sometimes miss. It’s more than most people will need for a routine check, but it’s the right choice if you want the most exhaustive sweep available.

If you run more than one of these tools, you’ll likely see the same finding reported in slightly different formats. The simplest way to consolidate results is to normalize on the matched string itself across all the reports, then de-duplicate from there, which a short jq and sort -u pass handles well.

Letting Kilo do the reading for you

Given that Kilo is an agentic engineering platform, working through a large JSONL file by hand is exactly the kind of task it’s built to take off your plate. The one thing to avoid is asking it to analyze the export natively.Telling an agent to simply read the file directly will, for any export of meaningful size, blow through both the context window and a fair amount of credit in the process.

A better approach is a small custom skill that teaches Kilo to work through the export in fixed-size chunks, handing each chunk off to a subagent rather than holding the whole thing in the main session’s context. One approach that works well in practice is spawning a subagent for every thousand lines: each subagent reads only its own slice, returns a short summary of what it found, and the parent session aggregates those summaries without ever loading the raw data itself.

A skill along these lines might look like this:

---

name: export-analyzer

description: Analyze a Kilo data export (kilo-data-export.jsonl) without loading it all into context. Use when the user asks to explore, summarize, or audit their Kilo data export.

---

# Kilo Export Analyzer

## Rules

- NEVER read the export file directly into the main context. It can be

tens of thousands of lines.

- First, run `wc -l` on the file and read only line 1 (the header) to

learn which sources are present.

- Split the work: process the file in chunks of 1,000 lines. For each

chunk, spawn a subagent whose only job is to analyze that slice and

return a compact summary (counts, notable findings, flagged strings

with line numbers). Chunk with:

`sed -n ‘START,ENDp’ kilo-data-export.jsonl`

- Subagents report findings only, never raw chunk contents.

- Prefer shell one-liners (jq, grep, awk) over reading data as text.

Compute aggregates like spend and session counts entirely in shell.

## Standard tasks

- “Summarize my export”: per-source line counts, session titles, total

spend in dollars (microdollars / 1,000,000), date range covered.

- “Audit my export”: pull `value` from `microdollar_usage_metadata` and

`system_prompt_prefix` into a flat file, then scan it chunk by chunk

for high-entropy strings and known key patterns. Aggregate and

de-duplicate findings, then report each with enough context to

locate it.

- “Find X”: grep first to locate candidate line numbers, then send only

those regions to a subagent for interpretation.

With that in place, a prompt as simple as “audit my data export in ~/Downloads” runs the entire flatten, scan, and aggregate sequence on its own, and your context window comes out the other side intact.

What you end up with

Taken as a whole, an export like this is more useful than it looks at first glance. It’s a spend ledger or a record of how you actually think through problems, not just what code you shipped. History like this allows you to spot real patterns like which reps eat the most iteration or which prompts you keep rephrasing.

It’s also the fastest way to answer a question that’s otherwise unanswerable: what has actually ended up in your prompt history. Running the scan while nothing is wrong gives you a baseline, and the findings turn into a short list of concrete actions rather than a vague sense that something might be in there somewhere. If you’re the person responsible for bringing AI tooling into a team, that same flatten-and-scan pass is the evidence a security reviewer will ask for, and having it ready before anyone asks is a much better position than assembling it afterward.

NVIDIA Nemotron 3.5 Lightning is a customizable open model that gives control over always-on agents and the latest agentic technology. Internally, we’ve been test-driving Nemotron 3.5 Lightning to see how it handles our day-to-day agentic workflows at Kilo. What we found is a remarkably fast, capable model that is perfect for agentic work—and we have insights to share on how to get the most out of it.

Under the Hood: Speed and Architecture

NVIDIA Nemotron 3.5 Lightning is a 30B Mixture-of-Experts (MoE) model that operates with 3B active parameters. It was distilled directly from NVIDIA’s frontier Nemotron 3 Ultra model, which has been in the top ten for every mode on the Kilo Leaderboard since launch, competing directly with the latest MiniMax and DeepSeek models.

Will Nemotron 3.5 Lightning have the same results in the real world usage? Based on our early testing, we’re guessing yes–and have tips from what we learned.

Designed specifically for always-on agents, this architecture allows Nemotron 3.5 Lightning to support massive context windows of up to 1M tokens. Where Nemotron 3.5 Lightning truly shines is the ability to do research and take action without losing its incredible inference speed. It boasts up to 4x higher throughput compared to some competing models, enabling sub-agents to complete specialized tasks much faster.

Throughput Comparison (TPS/GPU)

A “Big Personality” Built for Agentic Flows

When integrating Nemotron 3.5 Lightning into Kilo for day-to-day work, our Kilo engineers noticed something immediately: this model has a big personality. It is highly opinionated and surprisingly spirited. In one internal test, when asked to create a pull request, the model playfully responded with a Shakespearean dilemma: “To create a PR, or not to create a PR, that’s the question.”

Because of this vibrant, opinionated nature, Nemotron 3.5 Lightning isn’t always the best candidate for unconstrained, high-level planning. Instead, it is a powerhouse when given strict guidelines combined with the right kind of flexibility. It can do research and act on it, but it needs to know where to start (and where you’re heading).

This spirited disposition is precisely what makes it so effective within the Agent Manager framework. Within this structured ecosystem, Nemotron 3.5 Lightning excels as a high-performance executor. When an Orchestrator agent breaks down complex projects into manageable sub-tasks, Nemotron 3.5 Lightning takes those specific, bounded instructions and pursues them with relentless efficiency. Its “personality” translates into a high degree of reliability for executing agent-managed workflows where the goals are clearly defined, ensuring that sub-tasks are completed without the drift often seen in more passive models.

When you need to spin up multiple agents to handle high-volume, repetitive, or domain-specific tasks, Nemotron 3.5 Lightning is incredibly effective.

Internal Benchmarks: Where Nemotron 3.5 Lightning Excels

Our Kilo engineers spent time putting both the “Instant” and “Thinking” variants of Nemotron 3.5 Lightning through the ringer with use in real production workloads. Here are the highlights of where the model succeeded:

• Exceptional Speed: Across the board, our engineers noted that the model is incredibly fast, making it ideal for rapid iteration and local agent workflows.

• UI and Code Tweaks: The model successfully navigated instructions to make correct, simple visual changes to front-end components like sign-in pages.

• Concise Generation: It proved highly effective at generating short, punchy PR descriptions, as well as excellent branch names and titles.

• Deterministic Git Operations (Instant Variant): The Instant variant handled low-risk, straightforward tasks beautifully, achieving a 100% success rate on tagging and strong results on undo/reset recovery, commit construction, and remote handling.

• Advanced Reliability (Thinking Variant): The Thinking variant achieved a nearly 73% exact completion rate on one of our engineers’ personal git benchmarks/evals. It scored perfectly (16/16) on critical deterministic operations, including repository inspection, revert, stash, and undo recovery.

Nemotron 3.5 Lightning is very close to Qwen 3.6 on PinchBench and we see many similarities. Not least of all, it’s a highly opinionated model, especially for such a tiny powerhouse. But when you give it the right tasks with the right guidelines, it can really exceed expectations.

By playing to its strengths—leveraging its blazing-fast token generation for well-defined, specialized workflows—Nemotron 3.5 Lightning is shaping up to be an excellent tool for multi-agent setups. Keep your instructions strict, embrace its quirky flexibility, and let it speed up your day-to-day operations.

Try Nemotron 3.5 Lightning everywhere you use Kilo Code, from the CLI to the VS Code extension. We can’t wait to see what you think!

We updated the Anaconda blog with more information on August 14, 2026 21:30 UTC.

August 9, 2026 14:50 UTC Update

We continue to investigate the impact to Kilo users from the Metabase incident.

Our investigation so far has confirmed that the Kilo Slackbot was impacted and a small subset of Kilo users on that feature had their Slack access token exposed. Out of an abundance of caution, we invalidated all Kilo Slackbot authentication tokens for these users. Affected Kilo Slackbot users were contacted.

For more information on reactivating Kilo Slackbot, please see the documentation: https://kilo.ai/docs/code-with-ai/platforms/slack#setup

We updated the Anaconda blog with more information on August 9, 2026 14:46 UTC.

On August 6, 2026, we were notified of a security incident at our business intelligence provider, Metabase. Kilo user information was in the database that was accessed through Metabase. According to logs of the incident provided by Metabase, an unknown actor accessed our customer records in Metabase, which included some Kilo users’ names, email addresses, and other data. Our analysis indicates that this incident did not expose Kilo customer payment information, and exposed data from only some Kilo users (not all).

The incident at Metabase occurred over a period of approximately 4 hours on August 2, 2026. Kilo was notified on August 6, 2026 . We immediately took steps to contain the incident, and began an internal investigation which remains ongoing. We are sharing this update as we have it, and will share others (including updates to specific affected users, as we can) on a followup basis as soon as we have additional results from our investigation. Please watch our blog and website for additional updates.

We (Kilo and Anaconda, which recently acquired Kilo), are committed to transparency and sharing action-oriented, helpful information around this incident, as we obtain it, and further updates will be coming. Please continue to check the Anaconda blog post for further updates.

For more detailed information about the Metabase incident, please refer to the Metabase security alert.

The Anaconda blog will be updated as our investigation continues.

The interesting part isn’t the agent. It’s the price list, which isn’t in the announcement.

Read the two prices before you read the benchmarks

Meta’s launch post covers background agents, the runtime, bundled skills, Terminal-Bench 2.1 and DeepSWE scores, and a case study where the model spent 24 hours and 1,000+ tool calls optimising GPU kernels. It’s a good post. It contains no pricing.

The pricing lives on the Muse Code product page, and it has two tiers for the same model:

That’s 12.5x cheaper on input and 21.25x cheaper on output. Meta didn’t discount the model. It put a price on your repository and paid in compute. Alexandr Wang, who runs Meta Superintelligence Labs, described it to CNBC as a contributor tier that gets you in more than ten times cheaper than pay-as-you-go.

The cheap tier is also the throttled tier, rate-limited well below Standard. So you pay for the discount twice, once in data and once in throughput.

In the same interview, Wang said Meta is starting to accept zero-data-retention requests, and called it an important enterprise feature. That’s the same option OpenAI and Anthropic already offer in enterprise agreements. So the ladder is: pay full price and Meta won’t train on you, ask nicely and Meta won’t retain you, or pay a tenth and hand over the transcript.

Why code is the data Meta actually needs

Meta’s Q2 2026 results put Family of Apps ad revenue at $59.4B out of $60.8B total. Round it however you like: this is an advertising company. That business was built by learning what people do and selling access to the prediction. The instinct behind the contributor tier isn’t new.

Coding data is the most valuable training data left, because it grades itself. A model that scrapes a blog post learns what confident writing looks like. A model that watches an agent work in a real repo learns whether the change compiled, whether the tests passed, and whether the human accepted the diff. Every contributor-tier session is a problem, an attempt, and a verdict, generated by someone doing real work under real constraints. You can’t scrape that.

There’s nothing shady about the offer. It’s disclosed, it’s opt-in, it’s a payment method on file and a checkbox. That’s the part worth sitting with. The trade is now explicit enough to have a price, and the price is public: your prompts and completions are worth about $1.15 per million input tokens to Meta.

Your call whether that’s a good deal. It’s a much easier call for a weekend side project than for a repo with a customer’s data model in it.

We turned the trade into a game

Telling people their work data has a market price is abstract. So we built the smallest possible version of the argument.

We gave Muse Spark 1.2 one prompt and asked it to build a browser game. The premise: you’re the developer, your repo is the currency, and each round the model shows you three ads generated from what it just read about your work, predicts which one you’ll click, and you try to pick a different one. You’re trying to be unpredictable. You mostly aren’t.

We ran it in Kilo Code, with the model routed through Kilo Gateway at cost. That’s our own product, so treat the demo as a demonstration rather than a benchmark.

It didn’t land in one shot. The first response ran, but we had to fix a few small things to get it playable. We’re leaving that in because a clean one-shot would be the less useful result. Muse Spark 1.2 got a working game most of the way there from a single prompt, and the gap between “runs” and “playable” was small enough to close in minutes. We spent only $0.24.

Play here

Three things we’ll be straight about. The ads are an analogy: Meta hasn’t said anything about using developer prompts for ad targeting and we’re not claiming it does. Ad targeting is just where most of us first learned to recognise a data-for-price trade, so it’s the fastest way to make this one legible. Second, we routed at Meta’s Standard rate, the tier where Meta says it doesn’t train on prompts and completions, so nothing we typed here paid for itself in data. Third, we picked the model for this on purpose and then paid full price for it, which is the entire point of the next section.

Distribution wins, so the terms travel with it

Meta isn’t trying to have the best coding model. Wang said as much: the positioning is price, not capability. And price at a 20x gap is a distribution strategy. It’s the same play that got Chinese open-weight models into everyone’s rotation, run in reverse, with the discount funded by training rights instead of by lower serving costs.

That’s why the terms matter more than the benchmark. Whichever agent gets cheap enough to be the default is the one whose data policy becomes your data policy, and most teams will never read it. The contributor tier at $0.10 in / $0.20 out will end up in a lot of .env files by Friday, chosen by whoever was told to keep the AI bill down.

The counterweight isn’t outrage. It’s being able to make the model choice and the data choice separately.

That’s the whole reason Kilo works the way it does. Kilo Gateway gives you 500+ models at cost, frontier or open weight, or you bring your own keys and the traffic bills directly to the provider. Muse Spark 1.2 is in there, at Meta’s Standard rate, which is how we built the game. One agent across VS Code, JetBrains, the CLI, and cloud agents. Fully open source, so the data path is something you can read instead of something you have to trust. If a model’s terms are wrong for the repo you’re in, you switch the model, not the workflow. And if the answer for a given codebase is a local open-weight model with nothing leaving the machine, that’s the same interface too.

Use Muse Spark 1.2. It looks good, and cheap frontier-class models are good for everyone. Just make the tier a governance decision with your name on it, not a line item someone optimised while you were on holiday.

Sources

Primary:

• Meta Superintelligence Labs, Introducing Muse Code and Muse Spark 1.2, August 5, 2026

• Meta, Muse Spark 1.2 evaluation methodology report

• Meta, Muse Code product page (pricing and tier terms)

• Meta, Meta Model API

• Meta Platforms, Q2 2026 earnings press release (SEC Exhibit 99.1) and Q2 2026 earnings call transcript

• OpenRouter, Muse Spark 1.2 model page and Meta model index

Reporting, for the Wang quotes:

• CNBC, Meta debuts Muse Code to take on Anthropic and OpenAI, August 5, 2026

Spoon theory is a way of describing a limited daily budget of energy, coined by people managing chronic illness to explain why a task that looks small can still cost everything they have left. Developer tooling doesn’t usually get discussed in those terms, but it should. A broken environment. A silent failure. An hour lost to a tool that won’t say why it did what it did. All of that is spoons, spent on the tool instead of the problem, and newcomers and solo maintainers feel it first because they don’t have a platform team to absorb the hit.

The tax nobody puts a name on

AI coding agents are supposed to give some of that budget back, and often they do. An agent that plans, writes, and iterates on a feature with you saves real effort. The problem shows up when you commit to the wrong one and a familiar pattern comes back around: your prompts and your team’s workflow habits end up locked inside one vendor’s product, priced on their terms, running whatever model they picked for you this quarter.

When that agent breaks, you usually can’t see why. You wait on support, or you route around it and lose the work anyway. It’s friction with better branding, and it costs a team the way any stubborn tool does. Someone loses an afternoon debugging the tool instead of the problem. They write up a workaround. The next new hire has to relearn that workaround before getting to real work, and the cost keeps spreading outward from whoever hit it first.

The ground is moving faster than lock-in can track

The timing makes this worse than it would have been a year ago. The model market is moving faster than any single vendor relationship can keep up with.

Moonshot AI open-sourced the full weights of Kimi K3 in late July: a 2.8-trillion-parameter model, and the largest open-weight release so far. DeepSeek V4 reached general availability the same week, under an MIT license. Qwen’s 3.8-Max shipped days after that. The UK’s AI Safety Institute now measures the best open-weight models at four to seven months behind closed frontier labs on hard benchmarks (down from six to ten months for most of last year), while running at a fraction of the cost. Chinese open-weight providers alone now account for something like 45% of tokens flowing through major routing platforms, up from under 2% a year earlier.

That pace is exactly why lock-in costs more now than it used to. A coding agent that hard-codes a team to one vendor’s model puts “which model is best” on that vendor’s release calendar instead of the market’s. Two months from now, a cheaper or better option ships somewhere, and a locked-in team doesn’t get to use it until their vendor decides to offer it, or until the team rebuilds the workflow to switch. That cost lands hardest on teams with the least room to absorb it: the ones who can’t just pay for the expensive tier and move on.

What changes with an open agent

This is the case for an agent that doesn’t force the tradeoff in the first place. If the prompts and context handling are open source, a team can actually look at what happened when something breaks, instead of filing a ticket and waiting on an answer. Switching models can mean picking a different setting: bring your own API keys, run something locally, or let a router pick between options. That’s why a team can turn on the next Kimi K3 or DeepSeek V4 release without rebuilding anything to use it. And if the agent works the same way in the IDE, the terminal, and the cloud, the context a team has built up doesn’t reset every time someone’s setup changes, which matters most for whoever on the team has the least experience to fall back on.

Kilo Code is built this way: open source under Apache-2.0 for the extension and MIT for the CLI, running in VS Code, JetBrains, the CLI, and the cloud, connected to more than 500 models with zero markup on inference. It has grown past 3 million developers and 40 trillion tokens processed. That growth says something about usefulness. The portability and the transparency were part of the design from the start.

Build for the tax that’s actually there

That early design choice matters more this year than it would have a year ago, simply because the model market keeps moving. A team on an open agent is more prepared to adjust to pace of the frontier. A team locked into one vendor’s model has to wait for permission, or rebuild. Multiply that gap across a team, and it’s the same tax spoon theory describes: energy spent fighting the tool, paid first by whoever on the team has the least of it to spare.

Not always. We’ve been running enough features through agents to see the other edge of this, and it’s counterintuitive: past a certain point, more context makes the agent worse.

When specs start lying to the agent

We have a set of spec files in our cloud repo that describe how various systems are supposed to behave. For genuinely complex work — billing, a gateway with a lot of surface area — they’ve been lifesavers, especially for reviewers trying to make sense of a large change.

But something breaks when the agent pulls in three or four of them at once. It starts treating a detail from one spec as a fact about another. It over-anchors on things it half-read. It confidently tells you “this is how it should be,” and when you push back and ask why, it admits it conflated two documents. You end up needing far more runs to get to the state you actually wanted.

Part of the problem is that the specs drifted. The original intent was that a spec describes outcomes and behavior — what should be true, not how to build it. Over time they picked up implementation detail and grew. One of ours ballooned to well over a hundred bullet points of rules, written in such precise, lawyerly RFC-style language (”the upstream payment provider” instead of just naming the thing) that it reads like a unit test written by an attorney. A human can barely parse it. An agent takes every clause as gospel — including the clauses that are now subtly wrong.

The question underneath the specs

Strip away the tactics and the harder question is this: what is the source of truth?

Agents are non-deterministic. You tell one to implement something and it will, but that doesn’t tell you what should be true. So where does that live?

• Is it the tests? The agents write those now, and they’re very motivated to make them pass. That’s not the same as the tests being correct.

• Is it the code? The code is what is true, not necessarily what was supposed to be true. An agent reading the codebase will infer intent from the current implementation, bugs and all.

• Is it the specs? Only if they’re maintained, unambiguous, and someone actually keeps them evergreen. Ours weren’t, reliably.

• Is it in someone’s head? That person goes on vacation, or the feature was built three weeks ago and the details are already gone.

This isn’t academic. Months later — sometimes only weeks later — you land in a corner of the codebase and need to know: how is this supposed to work? What was the business outcome? Was it implemented correctly? Without a source of truth you can point at, you’re guessing, and so is the agent.

Two kinds of truth

The most useful framing we’ve landed on splits it in two.

There’s how the software is — and the honest source of truth for that is executable: the code and the tests. You can’t write “this is the best plugin in the world” in a prompt and make it true. Behavior is verified by running it.

And there’s how you want the software to be — the goals, the direction, the architectural constraints. That’s the part you set, and it can’t be reverse-engineered from the current code, because the whole point is that you often want the new thing to not look like the legacy thing.

Conflating those two is where a lot of agent pain comes from. If your instructions contradict each other, or describe the current state when you meant to describe the target state, the agent walks confidently in the wrong direction.

Less instruction, more intent

The most surprising trend runs the other way. Look at the system prompts and harnesses shipping from the frontier labs, and the trend across model generations is fewer baked-in instructions, not more. They lean on skills and context pulled into the window dynamically, rather than a giant static rulebook.

That points at a fix worth trying:

• Cut the standing instructions. If your agents file is aggressively telling the model to go read all the specs before every task, that eagerness works against you. Make context pull-in deliberate and situational.

• Separate the two truths explicitly. Let code and tests be the source of truth for how things are. Use instructions to express where you want to go — the goals and boundaries — not to re-describe the implementation.

• Enforce direction with structure, not prose. When we’ve migrated architectures with human teams, we didn’t document it exhaustively; we created a new folder or module so it was obvious which pattern was current and which was legacy. Agents need the same clear signal.

• Keep specs for the genuinely hard stuff. Billing, gateways, anything a reviewer couldn’t hold in their head. Not every component needs one, and a spec per feature is how you get the confusion in the first place.

• Decide who the spec is for. A document optimized for an agent and a document optimized for a human reader are different artifacts. Pick one on purpose, and know whether your review process expects a human to read and understand a spec change in a PR, or whether that’s the agent’s job.

Where I’m still uneasy

I’ll admit I feel a little nervous about removing specs entirely. Across multiple sessions, having some reference point you can verify a change against is genuinely valuable — being able to tell an agent “confirm the implementation matches the spec” is a real superpower, and it’s helped when partnering with non-engineers who needed a contract to reason about.

But “valuable sometimes” is not “load all of it, always.” The goal isn’t zero context. It’s the right context, pulled in on purpose, kept honest, and clearly separated into what is true versus what we want to be true. Give the agent that and it stops arguing with itself; give it everything and it won’t.

But then he asked a follow-up: And how many of you are working less? Only a couple of hands.

Many conversations at the summit came around to one thing: it’s not just the AI-native startups pushing the envelope that are starting to see real ROI from AI workflows and toolsets; a wide range of sectors, including healthcare and fintech, have leaned in and are starting to see results. The problem is now how to optimize humans-in-the-loop to get the most out of AI workflows and agentic engineering without losing our focus.

Many of us are working more because we’re simply so excited about the potential for this new technology, but we still need to keep costs low while maintaining flexibility in the stack.

Lately, AI builders have been swinging between two extremes:

• 📈 Tokenmaxxing: Stuffing every possible piece of context, RAG output, and history into a prompt to maximize context.

• 📉 Tokenminimizing: Aggressively pruning prompts, dropping vowels, and obsessing over system instructions to save a fraction of a cent.

But a new trend is emerging among developers on Kilo. We’re calling it Tokenrelaxxing and I have a feeling it’s going to stick around a lot longer than the summer holiday.

The Art of Tokenrelaxxing

Tokenrelaxxing is the realization that you no longer need to manually sweat over every single token. In fact, you don’t even need to constantly try to optimize the best time to re-set your subscription allowance. It’s the shift away from hyper-optimizing inputs and back to what actually matters: productivity and shipping code.

How? By leaning on the latest models, and leaning into Auto Modes on a daily basis. The industry is playing around with a lot of different forms of auto-routing, and Kilo has been there since the beginning with optimization techniques based on how real developers use different models tied to different agents or modes.

Instead of manually playing cost-basis arbitrage across a dozen different API endpoints, Tokenrelaxxing lets Kilo’s Auto Model dynamically handle the heavy lifting. The engine optimizes your requests in real-time based on the ultimate developer formula: Quality × Optimized Speed × Cost.

The New Math of Frontier Models

A little while ago, in a post on tokenmaxxing without breaking the bank, I wrote about how to push context windows without maxing out your AI budgets. The response was fantastic, but if there’s one thing the AI development space guarantees, it’s that the meta shifts fast.

If you look back at the cost comparisons in my previous Tokenmaxxing post, the landscape has already fundamentally changed. In just the past few weeks alone, frontier labs have increased efficiency, even for high-reasoning models, while continuing to improve capabilities.

At the same time, we’re seeing models like Kimi K3, Grok 4.5, and the newest generation of Qwen models completely redefine the price-to-performance ratio. They are making everything vastly more powerful and profoundly more affordable at the same time. At the same time as OpenAI releasing a suite of GPT models with major cost improvements and Anthropic unleashing Opus 5 with Fable-like intelligence, we tested every new model and found that MoonshotAI’s Kimi K3 and xAI’s Grok 4.5 could build the same database as Claude Opus 5 at 1/25th the price.

It’s truly an embarrassment of riches.

Compared to my earlier post on tokenmaxxing, tokenrelaxxing with Kilo can save you an additional 40-50% simply by assigning the best models to the tasks and modes at hand. It’s not rocket science. We’re just assigning the best model to the job within your chosen mode, and sometimes that’s still a frontier model like Sol, Luna or Terra, as OpenAI improved overall efficiency for 54% better token usage in agentic coding with this recently updated model family.

When frontier-level intelligence is this cheap and fast, micromanaging your token usage is actually a net negative on your engineering velocity. The time you spend shaving 50 tokens off a prompt is time you aren’t spending building features your users actually want.

Let Auto Mode Do the Work

With Kilo, Tokenrelaxxing becomes your default state. Instead of guessing which model fits the task, Kilo classifies the session intent in real-time and routes your request based on live benchmark data.

Here are the Auto Model routing tiers making Tokenrelaxxing possible:

• Auto Efficient (The Game-Changer): This tier uses live session classification for smarter spend. According to Kilo’s latest stats, Auto Efficient delivers 71% of published frontier completion at a massive 72% lower cost on KiloBench. It delivers near-frontier coding performance for pennies.

• Frontier: For when the work is incredibly complex and you need maximum capability. It automatically routes to the latest and most capable paid models.

• Balanced: Highly capable paid routing that avoids the absolute premium prices of frontier models, finding the sweet spot for everyday development.

• Free: Routes your requests to the best available free and experimental models providers offer at no cost.

Stop stressing and start tokenrelaxxing. You don’t have to choose between going broke or losing quality anymore.

Breathe in. Breathe out. And turn on your chosen auto model in the Kilo model picker, wherever you like to prompt — the CLI, VS Code extension, cloud agents, JetBrains extension and beyond.

That second part is the design decision worth explaining, because most AI coding tools in JetBrains handle it differently.

What split mode actually is

Split mode is JetBrains’ name for the architecture behind remote development. The IDE stops being one process and becomes two: a frontend that renders the UI on your laptop, and a backend that holds the project model, indexing, analysis, and execution. The backend can sit on a bigger machine, in a container, or in the cloud.

Developers use it for concrete reasons. More compute than a laptop has. A different OS than the one they run locally. Source code that stays on company servers while they work from anywhere. A thin client instead of a full local checkout.

For plugin authors, split mode changes where your code runs, and JetBrains is blunt about what happens if you ignore it. A plugin that only loads on the backend still technically works, but its UI gets rendered on the backend and shipped to the frontend as drawing commands. Every interaction pays a network round trip. JetBrains’ docs describe the result as UX “of insufficient quality.”

A split plugin avoids that by shipping three sets of modules: a frontend module for UI and anything latency-sensitive, a backend module for file system access, indexing, and project logic, and a shared module defining the RPC interfaces between them.

JetBrains only published guidance for this in spring 2026, and the modular plugin system is still marked experimental in the SDK docs. We build against it anyway, because the alternative is shipping a plugin that degrades the moment someone connects through Gateway.

Why we designed for it up front instead of retrofitting

Our old JetBrains plugin was a webview wrapper. It already had one opaque process boundary: the IDE couldn’t see inside the panel, and the panel couldn’t reach IDE internals except through what we bridged by hand. Adding remote development on top would have meant a second boundary layered on the first.

So when we rebuilt the plugin in Kotlin, we drew the frontend/backend line first and built features on top of it. Chat rendering, scrolling, and input live on the frontend. File indexing for @file mentions, terminal execution, and workspace access live on the backend. The split isn’t a migration we’re partway through. It’s the layout.

How other agents get into JetBrains

There are four routes into a JetBrains IDE right now, and they carry different tradeoffs.

Copilot has a real native plugin, built against the IntelliJ SDK, with its own chat tool window. Credit where it’s due. But remote development has been a long-running sore spot: GitHub’s issue tracker has reports of the frontend component calling backend-only IDE classes, which is exactly the failure mode JetBrains’ split-mode docs warn about, and an open issue from April 2026 describing agent tool calls running on the local client instead of the remote backend where the code lives. Retrofitting is hard even with Microsoft’s resources behind it.

Claude Code’s JetBrains plugin runs the CLI in the integrated terminal and connects over a local bridge, so your interface is the terminal and the IDE lights up around it with diffs and selection sharing. In remote development, Anthropic’s docs tell you to install the plugin on the remote host rather than the client. That’s the backend-only pattern.

Windsurf ships a second plugin just for remote development, installed on both host and client. Two artifacts to keep in sync is what retrofitting looks like when you don’t restructure the modules.

Why we skipped ACP

ACP, the Agent Client Protocol, is an open protocol co-developed by JetBrains and Zed. It standardizes how a coding agent talks to an editor, roughly the way LSP standardized language servers. Implement it once and your agent appears in JetBrains AI Chat, Zed, Neovim, and anything else that speaks it. JetBrains ships support in AI Assistant with a registry you can install agents from.

ACP agent renders inside the host IDE’s chat panel, and that panel has to work for every agent in the registry. It’s the common denominator by design.

That’s the tradeoff. Through ACP we couldn’t show your account balance and per-model pricing in the panel. We couldn’t build our own mode switching, our own approval controls for file edits and terminal commands, or our own MCP server management. We’d get a text chat box and whichever controls the protocol currently exposes. JetBrains has said as much: applying the standard means trading away tailor-made features, and they’re working to bring some of them back into the protocol.

Cursor took this route, which is why using Cursor in IntelliJ means using it inside JetBrains AI Chat rather than a Cursor-designed interface. Copilot took it too, then moved past it: Copilot was reachable through the ACP registry first, and in June 2026 GitHub and JetBrains replaced that with a purpose-built integration for a dedicated login flow and model picker. Even for the largest agent on the market, the generic path left something on the table.

Try it

Search “Kilo Code” in Settings → Plugins. It works in IntelliJ IDEA, WebStorm, PyCharm, GoLand, PhpStorm, Rider, CLion, and RubyMine, in Community and paid editions, locally or through remote development.

Setup instructions are in the JetBrains plugin docs. If you already use Kilo in VS Code or the CLI, your account and keys carry over.

That workflow made sense when writing code was a scarce resource in software development. But coding is no longer scarce. AI tools have made it dramatically cheaper to produce code, which means developers can generate larger changes faster. Agents can pick up a task, write a feature, add tests, and open a PR with very little human implementation work.

The volume of code reaching review is increasing, but the number of senior engineers available to review it has not increased accordingly. The bottleneck has moved. It used to be writing code; now it’s verifying code. However, most teams are still treating code review like nothing has changed.

The Industry Is Already Moving

Some companies are making human PR reviews optional on their own monorepos. Their auto approver reads a diff, classifies risk, and approves certain PRs without a human reviewer ever looking at the code.

That sounds extreme - and it should. But the interesting part is not that humans disappeared. Reviews still need security checks, CODEOWNERS for sensitive paths, stale approval detection, evaluation data, and a culture where engineers still own what gets merged. The human checkpoints moved to a different part of the system.

That’s the shift. The question is no longer, “Can every line of code be reviewed by a human?” The better question is: “Which changes actually require human judgment?”

Review Is Not Just Defect Detection

AI has not eliminated the need for code review; it simply changed its objective.

The old model treated review primarily as defect detection. The reviewer’s job was to find mistakes in another person’s implementation: a missing edge case, a broken assumption, a test gap, a strange abstraction, or a hidden performance issue.

That work still matters, but it cannot be the whole job anymore. Models are already good at finding mechanical issues: missing null checks, inconsistent naming, unhandled error paths, test gaps, and violations of local conventions. Sometimes they’re even too good at it.

One of my senior engineers described his first pass on a PR as a basic sanity check: Is this change likely to introduce something breaking? Does this architecture make sense? Is this a maintainable solution?

If he has time, he will proceed to pull the branch down and run his own review. He almost always finds something, but the findings are increasingly nitpicky. It’s an exercise that might be useful, but it’s certainly not the highest value use of senior engineering judgment.

The greatest value of a human review is deciding whether an incoming change is the optimal solution, or if it should happen at all.

The New Goal of Code Review

Here is the definition I keep coming back to: The goal of code review is for the author to convince the reviewer that the chosen solution is the optimal solution for the problem.

That framing changes the review contract. The author’s job is not just to submit code, it’s to make the problem, the solution, the tradeoffs, and the confidence level legible. The reviewer’s job is to decide whether the change makes sense in the broader context of the system.

This matters even more when AI is involved. AI-generated code can look plausible without being well understood. It can follow local patterns, pass tests, and satisfy the narrow prompt it was given.

But “looks right” is not the same as “is right.” An AI model will build to spec, which means that one of the most important human jobs is making sure the spec makes sense in the first place.

Start With the Problem

Before a reviewer can evaluate a diff, they need to understand the problem. What is this PR trying to solve? Why does it matter? What constraints shaped the work? What was intentionally left out?

Without that context, the reviewer is forced to reverse engineer intent from code. That workflow was already inefficient back when humans actually wrote every line. With AI-generated code, it becomes more dangerous, because there may be less human intent embedded in the implementation than the diff suggests.

Once the reviewer understands the problem, they can evaluate the chosen solution. What approach did the author choose? What alternatives were considered? Why this one? What assumptions does it rely on? What areas are risky?

This is where code review becomes less about correctness and more about judgment. A solution can be technically correct and still be wrong for the team. It might introduce a new pattern where an existing one should have been used. It might add a dependency the team does not want to maintain. It might make the broader system harder to reason about.

These are not always things a model can determine from the diff alone. They require system context, product context, team memory, and taste. That is why human review still matters.

The Reviewer’s Real Question

A good senior engineer does not only ask, “Can this work?” They ask, “Will we regret this?”

That is the question modern code review needs to center on. As code gets cheaper to produce, the cost of bad decisions goes up. A large, messy, almost correct PR can now be generated quickly. But reviewing it, understanding it, debugging it, and living with it still costs the team real time.

The danger is not that AI writes obviously terrible code. The danger is that AI writes code that is good enough to merge, but not good enough to own. That is where teams get into trouble. Reviewers start skimming, and authors assume that passing tests means the PR is fine. Everyone feels pressure to preserve velocity. The path of least resistance becomes rubber stamping.

That is not a code quality problem - it’s an operating model problem.

The New Code Review Contract

Opening a PR should no longer mean, “I wrote the code. Please check it.” It should mean, “I believe this is the right solution. Here is the evidence.”

A good PR description should answer a few basic questions. What problem does this solve? Why is this the right approach? What alternatives were considered? What areas are risky? How was this tested? What should the reviewer focus on?

Modern code review should also be risk-based. Routine changes can be handled with small PRs, strong tests, automation, and AI-first-pass review. Architectural changes need slower human review. Sensitive changes - like billing, auth, permissions, data deletion, migrations, infrastructure, and security boundaries - need the deepest review.

The reviewer is no longer just a defect finder, they’re a validator of judgment. The author is no longer just a code submitter - they are also responsible for explaining why their solution is the right one. The tools are no longer just autocomplete, they are part of the verification system.

When those roles are clear, review can get better. The author owns intent. The reviewer owns judgment. The tools provide leverage.

The main question isn’t whether one human can find mistakes in another human’s code, but whether the team can maintain a shared understanding of the system while the volume of code increases. Because the bottleneck in software development is not typing code - and hasn’t been for some time now. It’s understanding whether the code should exist at all.

Code review has changed. Now the contract has to change too.

Most of that debate runs on benchmarks and principle. We can add something narrower: what open-weight models actually did in a production workflow, measured on our own traffic.

We don’t make models, host them, or train on your data. Kilo is the routing and application layer, so we have no stake in which side of this wins. We see which models builders pick and what happens next.

The job we have the most data on is reviewing pull requests. Between June 22 and July 23, 2026, we classified 10,643 completed Kilo Code Reviewer runs and the 7,083 findings they produced. Every finding got a severity and a category. We normalized per review, so a model that ran 2,000 times doesn’t beat one that ran 300 times on volume alone.

Two of the top three models for surfacing critical issues were open weight. The closed-model security lead came almost entirely from one model.

Open weights took two of the top three spots

Kimi K2.7 Code led at 0.179 critical findings per review. Grok 4.5 came in at 0.176, Laguna M.1 at 0.171. Kimi and Laguna are open weight, Grok is closed, and the gap between first and third is under 5%.

It doesn’t mean open weights behave as a group. GLM 5.2’s rolling alias came 12th of 13 on the same measure. The spread inside the open-weight set was wider than the average difference between the open and closed sets, which suggests license category isn’t the variable doing the work.

Models don’t agree on what counts as critical

Models also differ in how they frame what they find.

Laguna M.1 escalates: 28% of its findings were critical. GLM 5.2’s rolling alias does the opposite, with 58% of its findings landing as suggestions. Both are open weight. GPT 5.6 Sol put 92% of its findings in the warning bucket.

Same data, different reviewer profiles. License doesn’t predict which one you get.

One model created most of the security gap

At the group level, closed models reported roughly twice as many security findings per review: 0.067 versus 0.034. That looks like a clear result until you break it out by model.

GPT 5.6 Sol reported 0.285 security findings per review across 274 reviews, far above every other model in the set. Take it out and the closed-model average drops to 0.042. Open weight stays at 0.034.

So if security coverage is what you’re buying, buy the model that demonstrates it. License category is a weak proxy: one model here carried the entire group difference.

Three quarters of the tokens, a sixth of the cost

Across broader Kilo traffic, open-weight models accounted for 75% of tokens and 16% of user-facing cost. Closed-weight models accounted for 25% of tokens and 84% of cost. Per token, open-weight traffic came in about 16x cheaper: (84% cost / 25% tokens) / (16% cost / 75% tokens) = 15.75.

That’s how our traffic is priced, not a same-task benchmark. Nobody ran the same pull request through both sets at matched settings.

The share is also still moving. In the week of July 20, 2026, open-weight models accounted for 79.1% of all token usage on Kilo, against 20.9% for proprietary models.

Use one model to write, another to review

Picking an AI coding model doesn’t have to mean picking one model for the whole workflow.

An author model should implement a correct change efficiently. A reviewer model should challenge it, look for failure modes, and catch what the author missed. Different jobs. Using one model for both can reproduce the same blind spots twice.

In the June traffic we could attribute, 32.3% of reviews already used a different model than the one that wrote the code. The most common pairing was Step 3.7 Flash authoring and Laguna M.1 reviewing.

Model freedom is what makes this practical. A cheap open-weight model can implement routine work while a model with stronger security behavior reviews it. Or a frontier model can author a hard change while an open-weight model with high critical intensity takes an independent pass.

No single model wins every task

A year ago, picking a model was a short conversation. Now a capable one lands most weeks, from Nemotron, Qwen, GLM, MiniMax, Kimi, Mistral, and the frontier labs. The review data says the same thing from the other direction: 13 routes, and the behavioral profiles don’t cluster by license.

That’s the argument for routing instead of standardizing on one model. Closed frontier models are strong on the hardest problems, and builders use them there. Across a lot of the remaining work, open weights compete on task accuracy, cost, privacy, and where you’re permitted to run them.

We also ran a head-to-head where Kimi K3 and Grok 4.5 built the same database. The open-model path cost significantly less for a comparable result.

One caveat on the routing layer itself: a router owned by a model vendor has a reason to prefer its own models. We partner with the labs rather than competing with them, so the only job is getting you to the right model.

Three rules for picking a reviewer

1. Buy a behavior, not a license category. Compare critical intensity, security emphasis, false positives, latency, and cost against your own repositories.

2. Separate authoring from review. Give the reviewer a different objective and, when it’s worth it, a different model.

3. Use open weights where they earn the work. In this sample they’re production options, not fallbacks.

What this data doesn’t prove

We measured what models flagged, not whether they were right. More findings can mean broader coverage, more false positives, or both. We don’t have a consistent accept-or-dismiss signal on every finding yet, so this isn’t an accuracy leaderboard.

Repository mix matters too. Models weren’t randomly assigned to identical pull requests. Some families show up twice in the charts because a versioned route stays pinned to one snapshot while a rolling alias can move to a newer one behind the same public name.

Acceptance rate is the measure we want next: which findings developers fix, dismiss, or argue with. That turns a behavioral comparison into a quality one.

Why this matters past code review

Open weights don’t have to beat closed models for the ecosystem argument to hold. They reduce how much of your stack depends on a single vendor, and they let you run inference where policy requires it. That’s what model freedom has meant at Kilo from the start: open-weight and local models alongside closed frontier APIs, your own keys if you already have vendor contracts, and an EU-first setup where data residency demands one.

Job laid out our full position on the letter in Open Weights Is All You Need. The per-model table, methodology, and sortable data behind this post are in the research writeup.

That’s a strange thing to say out loud, but it captures something real that’s happening to open source in the age of coding agents, and it’s forcing us to rethink processes we’ve taken for granted for a decade.

The old bargain

Open source ran on a simple trade. Writing code was a scarce, expensive skill. When someone showed up with a working patch, they were handing you hours of that scarce resource for free. Reviewing it took time, sure, but review was cheaper than authorship. The contributor did the hard part.

That bargain is inverted now. Generating code is cheap and fast, but judgment isn’t. So a contribution that used to save maintainer time can now cost it, because reviewing a plausible-looking AI-generated diff can take longer than writing the change yourself.

And the diffs are very plausible, which is the trap. A polished PR description, a clean screenshot, a confident summary of what was tested — all of it is easy to produce and hard to verify. We’ve had PRs where the description said it was tested, with a nice screenshot, and then the actual code was building queries against a data warehouse the contributor almost certainly couldn’t run locally. The presentation was flawless. Whether it worked was anyone’s guess.

What review actually costs now

When code was expensive, “does this diff look reasonable?” was a good enough filter. Now it isn’t, and the reasons are worth naming:

• Local coherence, global blindness. An agent solves the one case in front of it. It doesn’t know that the CLI change also needs to land in the VS Code extension, or that a one-line “fix” is a hack that papers over the real problem.

• Plausibility is free. Every AI PR looks like it might be fine. That’s exactly why auto-closing them with another bot doesn’t work. There’s no model with good enough judgment to tell a genuine improvement from a well-dressed dead end.

• Staleness compounds. In a fast-moving repo, a PR left open for a few days needs rebasing, and the rebase-and-fix work can dwarf the original change.

• Security hides in the noise. When volume goes up, the sketchy PR — the one quietly trying to weaken something — is easier to miss.

Faros AI’s research on teams adopting AI tools found review time climbing sharply as PR count and size grew. That’s the same dynamic, just inside your own org: more output, less throughput, unless review scales with it.

What’s still worth a lot

None of this means contributions are worthless. The valuable part just moved.

The signal we actually trust now is evidence that a human sat with the change: tested it themselves, tried the edge cases, understood how it fits the wider product. A contributor who has done that repeatedly earns trust, and their PRs get merged. A first-time contributor who opens eight PRs in a day is sending the opposite signal.

What saves maintainer time is no longer the code itself. It’s the intent behind it, the proof it works, and a clear sense of how it fits. The PR is the least interesting artifact in that list.

What this changes about process

We’re still figuring this out — genuinely inventing what open source looks like when code is cheap. A few directions we’re leaning into:

• Issue-first, not PR-first. Pitch the intent before writing the diff. It’s a lot cheaper for everyone to align on “do we even want this, and how should it work” before someone burns effort on an implementation we won’t merge.

• Be honest about source-available vs. open source. There’s a real difference between “you can read and fork this” and “we’ll shepherd every contribution to merge.” We have a company built around the product; pretending otherwise just leads to contributors feeling ghosted.

• Enforce the minimum bar with automation, not human nagging. Requiring a linked issue, tests, or validation shouldn’t cost a maintainer their afternoon. Machines can hold that line.

• But keep humans on judgment. Deciding whether a change is worth having — whether it fits the product, whether the quality is there — is exactly the thing agents are bad at. That stays with us.

• Communicate like a person. The one thing that’s not negotiable: if you pick up a review, you own it to the end, whether that’s merge, close, or hand-off. A clearly bot-written brush-off is worse than a direct “no, not right now.” If someone put in the work, they deserve a real answer, fast.

Why not just use our own agent?

There’s a version of this that ends with “so why not just point our own agent at the good issues instead of waiting for contributors?” It’s a fair question, and the honest answer is that sometimes we should. The case for a human community isn’t that it’s cheaper anymore, because it often isn’t. It’s that some changes span multiple systems or need eyes on something visual that bots still get lost in, and that a healthy community is worth something to the brand and the product even when it costs time.

But the strategy only works if we’re extremely clear about what we want, and we stop pretending the old bargain still holds. Code got cheap. Review didn’t. Build your process around the resource that’s actually scarce.

Now, we’re bringing the data to back up the need for an open ecosystem.

Kilo doesn’t make models. We don’t host them, we don’t train on or retain your data. We’re the application and routing layer, the part that sends each coding task to whichever inference you choose, through approved providers and policies for your organization. Our data is worth something in this debate, because we have no side in the open-versus-closed battle. We just watch which models builders actually use.

Here’s what we see.

As of the week of July 20, 2026, open-weight models accounted for 79.1% of all token usage on Kilo. Proprietary models were the remaining 20.9%. And that number is not finished climbing. Not long ago open weights were a small minority of the traffic. Today they're a capable workhorse in the inference stack.

When Jensen Huang shared the letter, he made a point we agree with completely: the world needs both frontier closed models and frontier open ones. Our data supports that. Closed frontier models are excellent at the hardest problems, and builders use them there. But for the other ~80% of the work, open weights often win on a combination of these things: accuracy on the specific task, cost, privacy, and the freedom to run them where you want.

The model boom

A year ago, "which model to use" was a short conversation. Now a new one lands almost every week from the likes of Nemotron, Qwen, GLM, MiniMax, Kimi, Mistral, and the frontier labs. And the list goes on. The menu has exploded, and no single model wins every task.

That is why routing is becoming some of the most valuable real estate in tech, and it is why we build the way we do. Kilo partners with the labs instead of competing with them, so our only job is to get you to the right model, not to steer you toward one we happen to own.

That distinction matters more as everyone rushes to build routing. A healthy ecosystem is not one big model that everyone depends on. It is many good closed and open ones. Open weights you can run on your own terms, and integrate it deep into your stack, instead of renting access. The explosion of capable open models means less dependency and resilience for organizations.

The same logic applies to the layer on top. A routing layer owned by a model vendor is not neutral, because it has a reason to prefer its own models. The ecosystem needs a neutral layer, not another lock-in. Our leaderboard consistently shows NVIDIA Nemotron and MoonshotAI models sitting alongside frontier labs like OpenAI and Anthropic, and that is not going to change.

The future of AI is choice, routing, and an ecosystem where open and closed models work together.

Look who signed

The letter reads like a map of the ecosystem we already route across every day. Among the signatories are companies at every layer of the stack we partner with:

• Silicon and cloud: NVIDIA, Amazon

• Model labs: Mistral, OpenAI, Arcee

• Inference and serving: FriendliAI, Morph

• Local runtime: Ollama

• Deploy and frontend: Vercel

These are the partners whose models and infrastructure show up in Kilo sessions every day.

Open from day one

None of this is just a reaction to a policy moment. Model freedom has been a first-class feature at Kilo since the start: open-weight models and local hosting alongside the closed frontier APIs, all from one platform.

In practice that means you can route through your existing vendor contracts by bringing your own keys, or run local and private models where policy requires, across every developer surface. For teams with data-residency requirements, that same routing control is what makes an EU-first setup possible without giving up model choice.

Backed by benchmarks

Yesterday we published a head-to-head benchmark where an open model, Kimi K3, and a closed frontier model built the same database, and the open-model path came out significantly cheaper for a comparable result. Open weights aren’t just the affordable option anymore. In real world scenarios, they’re competitive.

We’ve written before about why betting your entire stack on one side is the real risk. In “OpenRouter, Opus 5, and the Era of Model Freedom,” we argued that walled gardens are giving way to routing and choice.

Why Anaconda + Kilo

For an enterprise, this is more than a developer convenience. The intelligence your products and decisions run on is core infrastructure, and core infrastructure is not something to outsource entirely to a single vendor. Owning your intelligence means you can keep it, understand it, and stand behind it.

Ownership pays off if you can govern it: decide which models and providers are approved, see where inference runs. That is the pairing behind Anaconda and Kilo. For more than a decade, Anaconda has shown that open and well-governed are not opposites, building the trusted foundation that tens of millions of Python and data-science developers rely on at the package layer. Kilo does the same one layer up, at the inference layer, giving teams model freedom while ensuring enterprises stay compliant.

The bottom line

Open weights don’t need to beat closed models, and we’re already seeing the positive effect from open models on a daily basis. They let you own your intelligence and depend less on any single vendor. That’s what the letter protects, and our data shows why it’s worth protecting. The open ecosystem is already carrying most of the load. The healthiest future for AI is both open and closed together, and the best way to keep it that way is to make sure nobody has to pick a side.

That’s the whole idea behind model freedom.

AI adoption is something most engineering orgs are still figuring out, and it’s only legible if you look at the right things. Watch which features people reach for and in what order, and you get a usable picture of how far the tooling has gotten into the work. The sequence is consistent across teams, which means you can locate your median engineer on it and know what the next step is going to require from you.

Kilo is available across the IDE, the CLI, and the web - and it’s model-agnostic, so our telemetry covers roughly 3 million developers and more than 40 trillion tokens without being shaped by one vendor’s product surface.

The pattern below is what shows up in that data, and it maps onto what teams tell us about their rollouts.

Five stages

Inline completion is stage one. Autocomplete lands immediately in every org because it asks nothing of the developer and changes no habits. Numbers here are high everywhere, so they mostly tell you the install worked.

Stage two is single-file agentic editing, where somebody describes a change and reviews a diff instead of typing it out. This is where a lot of orgs settle in for a long time. Usage looks healthy, developers like the tool, and the pipeline runs the way it always did.

Stage three is multi-file work with a plan in front of it, the point where a developer hands over something they can’t hold entirely in their head. This one depends on context infrastructure more than on enthusiasm. Codebase Indexing gives semantic search across the repo using cloud-hosted embeddings, and Plan Mode outlines architectural decisions in the project instead of the prompt, so nobody needs to re-explain the same service boundaries during a session. With those in place, stage three sticks. Without them, attempts fail often enough that people quietly go back to stage two.

Stage four is concurrency: parallel agents in the IDE and CLI, Cloud Agents running from the web dashboard with no local machine involved, and Sessions that persist across interfaces, so a task started by one teammate in the CLI can be finished in VS Code by another. Once someone is supervising work rather than performing it, their output stops being bounded by typing speed.

Stage five is the workflow carrying itself. Code Reviewer analyzes every PR the moment it opens with the model you picked, surfacing security, performance, and test coverage issues before a human looks. The Kilo CLI is used in autonomous mode to hook into CI/CD flows, and Cloud Agents are triggered by external systems with webhooks, automating code workflows while the team sleeps.

Stages one and two are individual conveniences that show up in developer satisfaction. Three through five are where org throughput moves, and they’re the stages worth instrumenting.

Governance from the top, selection from the bottom

AI tool adoption is happening (or being encouraged) at all heights. Bottom-up rollouts, where IC engineers recommend and settle on tooling, get you real usage because nobody had to be sold. In our conversations with enterprises, we’re hearing that AI tool discovery is increasingly happening this way. What this stream doesn’t always carry is shared practice, so you can end up with nine tools, four billing relationships, and no idea whether the payments team found something the mobile team could reuse. Top-down rollouts get you SSO, pooled billing, and a number for the board deck, but they tend to standardize on a single model provider on the way there.

Our managed org data suggests standardizing is the part to hold loosely. Multi-provider usage on team subscriptions climbed from 42% to 71% in a single quarter, following the curve individual developers were on about two quarters earlier. Bucket by volume and it gets sharper. Developers under 100 requests a month average around 1.5 providers, developers above 10,000 average more than 7, and 92% of that group works across multiple providers.

Experience is what drives that. Heavy users have hit enough edge cases to know which model handles which kind of task, so the deeper someone gets up the ladder, the more selection matters to them.

The combination that works is buying governance, security, and visibility centrally while leaving model choice with whoever is holding the ticket.

What to measure

Activation belongs in the report as a prerequisite. Beyond 80%, it stops carrying information.

Feature depth is a number to build the score around: distinct features and surfaces per active developer, weighted toward the later stages. It tells you where your median engineer sits and gives you something that moves when a rollout is working.

Workflow penetration measures how much of the pipeline runs through AI. Share of PRs that get an automated review before human review, share of agent sessions that end in a merged change, and share of bugs caught by the AI reviewer are good metrics to pay attention to.

Practice retention asks whether features like custom modes and skills get shared across a team or stay with their author. Shared modes mean your strongest engineers are duplicating their judgment into the org, which is the closest thing to institutional learning in any of this.

What can halt your climb rate

Most adoption stalls trace back to friction, and often policy rather than product.

The Pragmatic Engineer’s 2026 survey of over 900 engineers found roughly 30% hit usage limits monthly, with a mid-task token wall ranking among the most disruptive things that happen to them. Losing a session halfway through a refactor costs you the context you spent twenty minutes building, so the rational response is to stop attempting work that size, and stages four and five never fill in.

Approval workflows work the same way on a longer timescale. If exceeding a cap needs a manager’s sign-off, people scope the task down until it fits. Stack Overflow’s 2025 survey of more than 49,000 developers put autonomy and trust ahead of compensation as a satisfaction driver, and a usage cap speaks to both.

That’s one of the reasons why fixed-rate coding plans don’t always scale to the enterprise. The pricing is unclear, subject to change, and can backfire towards the end of the month. The better option is to build scalable systems using flexible tools with open pricing, so that you’re not guessing which parts of your policy require optimization.

None of this works without observability

Everything above assumes you can actually see feature depth, workflow penetration, and practice retention. Most orgs can’t, because their only instrumentation is a billing dashboard, and a bill tells you what something cost, not what it did.

Kilo’s usage analytics and AI ROI Dashboard cover the adoption question, showing which features and surfaces a team uses and how that shifts over time. The centerpiece is the AI Adoption Score, a single 0 to 100 number built from three weighted dimensions: frequency at 40%, how often developers use AI, depth at 40%, how integrated it is into actual development, and coverage at 20%, how broadly it’s spread across the team.

Cost Insights handles the finance question separately, plotting spend across 7, 30, or 90 days with the largest contributors by product and member, and Spend Alerts run at the org and personal level on rolling 24 hour, 7 day, and 30 day thresholds so anomalies arrive as notifications rather than as an invoice.

Auto Efficient is one feature that keeps the cost question from turning into a cap. It does session-aware routing, matching each task to the cheapest model that can handle it across 500+ models, and about 38% of the switching we see is developers doing that math by hand one task at a time. Automating it pulls blended cost per task down far enough that limits stop coming up.

For the rollout itself, seed with your heaviest users, have them build the custom modes and skills, and turn on Code Reviewer early since it makes AI a pipeline default rather than a personal choice. Then read feature depth at 30, 60, and 90 days using the actual usage data rather than a spend report.

Where this lands

An org at stage five looks specific. The median engineer supervises work rather than typing it, PRs get reviewed before a human opens them, the team’s best practices live in shared modes, and nobody is trimming a task to stay under a threshold. Every one of those is readable from feature data, and each one has a concrete thing standing in front of it that you can go remove.

Move at Kilo Speed. Get started with Kilo for Enterprise

TL;DR: Both setups passed 64 of the same 65 conformance checks, survived every crash-recovery test, and shipped the same single critical bug. Claude Opus 5 scored 98/100, the Kimi K3 + Grok 4.5 setup scored 93/100 and ran at 4% of the cost ($1.27 against $31.71).

Pricing

Per token, Kimi K3 costs 60% of Claude Opus 5, and Grok 4.5 costs 24% on output. This setup puts the open-weight model on planning, where the design decisions get made, and the more affordable model on implementation, where the decisions are already written down. Our earlier planning comparison found that frontier models implement a good plan almost interchangeably. This test asks whether that holds when neither model in the setup is a frontier flagship.

What We Asked Them to Build

We wrote a spec for kvd, an embedded key-value store in Go. It is a lighter version of something like Redis, with one binary and a small text protocol over TCP, except data is persisted to disk. We picked it because everything in the spec can be checked. Either the database loses data when you kill it, or it does not. The spec requires:

• A single Go binary serving a Redis-style text protocol over TCP, with commands for reading, writing, deleting, batching, stats, and compaction

• A durability rule: the server must not acknowledge a write until the data is flushed to disk, and every acknowledged write must survive kill -9 arriving at any moment

• Atomic batches: a group of up to 1,000 writes and deletes must apply all-or-nothing, even when the process dies mid-batch

• Crash recovery: on restart the server must detect a partially written record at the end of its log by checksum, discard it, and lose nothing else

• Scale: 1,000,000 keys, datasets bigger than RAM, and recovery within 60 seconds

• Standard library only, with no third-party storage engines

The hard part of building a system like this is the failure paths. The store has to recover from a crash, throw away a half-written record without losing the good data before it, and keep a batch atomic when the process dies in the middle of one. That is where we aimed the test.

The Two-Phase Setup

Both setups ran in Kilo CLI with the same prompts. Claude Opus 5 ran at xhigh reasoning in both phases, Kimi K3 at max, and Grok 4.5 at high.

Phase 1: planning. We told each planner that its plan would be handed to a different engineer who would never see the original prompt and could not ask questions. The plan had to carry everything, from the exact wire protocol to every durability guarantee. If a requirement was missing from the plan, the implementer would never know it existed.

Phase 2: implementation. A fresh session got only the plan file. Claude Opus 5 implemented its own plan. Grok 4.5 implemented Kimi K3’s plan. Neither implementer ever saw the initial prompt.

This strict handoff lets us trace every bug back to its source. If a requirement went missing between the spec and the plan, the bug belongs to the planner. If the plan was right and the code is wrong, it belongs to the implementer.

How We Graded

Before either model ran, we wrote a Python test harness that speaks the spec’s wire protocol and grades any server that implements it. It runs 65 checks across six suites: protocol conformance, durability under crash injection, batch atomicity, compaction, consistency, and scale. For the durability checks, the harness records every write it sends and marks it acknowledged once the server replies. Then it kills the server with kill -9 at a random moment, restarts it, and checks that every acknowledged write is still there, byte for byte. It repeats this ten times per suite while concurrent writers are running.

The harness covers everything we can check automatically. For the rest, we read both codebases, graded test quality, documentation accuracy, and code hygiene, and looked for bugs the automated checks could not reach.

The Planning Phase

Claude Opus 5’s plan is more than twice as long, and most of the extra length is substance. It includes decision tables, a table of exactly where fsync must happen, and walkthroughs of what recovery does at each crash point. Kimi K3’s plan is shorter but covers the same durability rules, the same wire protocol, and the same batch atomicity design.

Claude Opus 5’s planning run also needed one adjustment. Kilo CLI caps output at 32,000 tokens per step by default, and Opus 5 at xhigh is the first model we have seen hit that cap during a single reasoning run. Our first attempts finished without writing a plan at all. We raised the cap to the model’s 128,000-token limit with KILO_EXPERIMENTAL_OUTPUT_TOKEN_MAX and re-ran, and one step in the successful run produced about 37,000 output tokens. The per-token pricing understates the gap here. Opus 5 produces far more output tokens during reasoning than the other two models, and reasoning tokens bill at the output rate.

Both plans also have flaws, and they matter for what comes next:

• Both planners made the same mistake on one protocol edge case, the handling of an oversized batch. More on that below.

• Kimi K3’s compaction design has a hole. A key deleted while compaction is running can reappear after a restart, because the compacted copy of the key survives while the deletion record does not.

The Implementation Phase

Claude Opus 5 took 76 minutes. It built a test suite covering 14 different crash scenarios and a benchmark running at the full 1,000,000-key scale. Grok 4.5 finished in 11 minutes with a codebase about a quarter of the size, and its tests cover the happy paths plus one crash test.

Grok 4.5 did not build the compaction flaw from its plan. It changed the design so that compaction runs serialized with writes and cleans up all old data files, which closes both holes in Kimi K3’s plan. Claude Opus 5 went the other way and faithfully built the one mistake its own plan contained.

How the Two Setups Worked

The two setups worked in different styles. Claude Opus 5 kept going back over its own work. Its planning session ran 49 steps, and 23 of them were edits to the plan it had already written. Its implementation ran 150 steps, 79 of them shell commands, in a loop of building, running its own tests, and fixing what they caught. Kimi K3 and Grok 4.5 worked much closer to one shot. Kimi K3 produced its plan in 7 steps and stopped. Grok 4.5 wrote the code, tests, and README in 22 steps and called it done. We saw the same split when GPT-5.6 Sol audited its own plans unprompted.

This loop is where much of Claude Opus 5’s extra time and cost went, and it is probably why its tests and documentation came out so much deeper. You could likely prompt the budget combo into a similar loop. We did not, because both setups got identical prompts. What we measured is the default behavior, and by default one setup reviews its own work and the other does not.

Crash Testing: A Tie

Every automated check came back the same for both.

Neither store lost a single acknowledged write in any round. Both recovered a million-key store in under a second and a half. Both handled garbage appended to their data files without losing anything. We read the code to check whether these passes were luck, and they were not. Both implementations flush to disk before acknowledging a write. Both use checksummed records, so a torn write gets detected instead of trusted. Both write a batch as a single record, so partial application cannot happen.

Both stores also failed one check, and it was the same one.

The Bug Both Shipped

The spec caps batches at 1,000 operations and says an oversized batch must be rejected with an error and nothing applied. Both servers reject it. Neither consumes the batch’s operations off the connection first. The operations are still sitting in the stream after the rejection, so the server reads them as fresh standalone commands and executes them.

We confirmed it with a scripted reproduction against both binaries. After rejecting a 1,001-operation batch, Claude Opus 5’s server went on to apply all 1,001 operations it had just refused. Grok 4.5’s server applied 2 before the connection closed. A client that trusted the rejection would have data it never meant to commit.

The bug came from the plans. Claude Opus 5’s plan spells out the wrong behavior, saying that on an oversized count the connection “stays open, no operations consumed.” Kimi K3’s plan never says what to do with the pending operations. Both implementers built what their plans said. Two different planners wrote the same bug independently.

Neither model’s own test suite catches it, because neither sends an oversized batch with its operations attached. Both test suites pass. We saw the same pattern in our Claude Opus 4.7 vs Kimi K2.6 comparison, where models did not write the tests that would catch their own worst bug. Our harness found it because it tests against the spec instead of trusting the models’ own tests.

What the Extra $30 Bought

The five-point gap in the final score comes entirely from the code review.

• Tests. Claude Opus 5’s suite kills the server at 14 points, corrupts its own files with truncation and bit flips to check that recovery handles them, and fuzzes the protocol parser. Grok 4.5’s suite has one crash test and never kills the server during a batch or a compaction.

• Documentation. Claude Opus 5’s 746-line README survived a line-by-line check against the code. Grok 4.5’s README covers the required topics but claims an fsync behavior on macOS that the code does not implement.

• Hygiene. Grok 4.5 shipped thinking-out-loud comments and some dead code. Claude Opus 5’s codebase was much cleaner.

• Remaining defects. Beyond the shared batch bug, we found one narrow error-path window in Grok 4.5’s compaction that could leave a stale data file behind. We found nothing comparable in Claude Opus 5’s store.

One difference went the other way. Claude Opus 5’s server used 428MB of memory after recovering the 512MB dataset. Grok 4.5’s used 223MB. The cause is a mis-sized memory estimate in Claude Opus 5’s recovery path, not a design problem. Both stores keep values on disk rather than in RAM.

The Scores

Cost and Time

The Kimi K3 + Grok 4.5 setup came in at 4% of the cost and 23% of the time. The entire score gap comes from tests, documentation, and code hygiene. None of it comes from crash safety or protocol correctness.

Conclusion

For correctness, this test could not separate the two. Both setups got the same conformance score, the same crash-test results, and the same single critical bug, and that bug came from the planning phase in both. As an implementer, Grok 4.5 gave up nothing to Claude Opus 5 on this build.

For the codebase you end up maintaining, Claude Opus 5’s output is better. Its tests cover the failure paths, its documentation matches the code, and it left no dead code or leftover comments behind. If the code is going into a repo that people will work in long term, that difference matters.

If cost were not a factor, we would default to Claude Opus 5. It did the better job overall. If the price gap were smaller, say two or three times, paying more for the better deliverable would still be a reasonable default. But the gap here is 25x, and that makes the more expensive workflow hard to recommend blindly. It has to be mission-critical work, or a case where the extra cost is clearly justified by the extra points. The things the extra $30 bought are also the easiest things to add afterwards. A follow-up prompt asking Grok 4.5 for a deeper test suite and a README pass costs about another dollar, and a normal code review would catch the hygiene issues. Crash safety is the part you cannot add afterwards, and the combo gave up none of it.

We also suspect the Kimi K3 + Grok 4.5 setup has room to improve. We have not tested this, but tuning the planning and implementation prompts to push Kimi K3 and Grok 4.5 into the same iterate-and-review loop that Claude Opus 5 runs by default would likely raise the quality of what they ship. If you want to run the same split, Kilo lets you set a different model per mode. The plan can come from Kimi K3 in Plan mode and the implementation from Grok 4.5 in Code mode.

Additionally, two things made the budget result stronger than we expected. Grok 4.5 fixed its planner’s compaction flaw on its own, which goes against the assumption that an affordable implementer blindly builds whatever it is handed. And Kimi K3’s weights are now public, so its price has room to fall as more providers host it. We have watched the same thing happen with every open-weight release this year.

The most useful finding is the shared bug. Two models at very different price points planned independently and made the identical mistake, and both implementations passed their own tests. The bug only surfaced because we tested both servers against the spec with a harness written before any model ran. That kind of external testing costs the same no matter which models you use.

Testing performed using Kilo Code, a free open-source AI coding assistant for VS Code and JetBrains with 3,000,000+ Kilo Coders.