GPT-5.6 Sol vs Claude Fable 5: Which Model Writes Better Plans?
When we compared Claude Fable 5 against GPT-5.5 last month, planning was where the two models split. Fable 5’s plan outscored GPT-5.5’s 9.1 to 8.3 on our rubric, and once that plan existed, both models implemented it almost interchangeably. Both labs have had an eventful few weeks since. Fable 5 was briefly suspended under a US export-control directive and is available again, and OpenAI released GPT-5.6 Sol. With both flagships available at the same time, we ran the planning comparison again, this time on three backend systems instead of one.
TL;DR: GPT-5.6 Sol produced the higher-scoring plan on all three systems, averaging 9.10 to Fable 5’s 8.04. The main difference was in their default behavior. After writing each plan, GPT-5.6 Sol launched sub-agents to audit its own work, unprompted, and kept revising based on their findings, taking 3.4x longer than Fable 5 for 13% more money. Fable 5 produced leaner, more buildable plans in about 15 minutes each.
Why We Focused on Planning
Frontier models have become hard to tell apart when you hand them a detailed plan. We have now measured this twice. Fable 5 and GPT-5.5 implemented the same plan and passed the same 15 acceptance checks with identical rollout behavior, and GLM-5.2 and Kimi K2.7 built services from one plan that answered every user the same way. In our earlier tests, once the decisions were written down, the choice of implementation model mattered much less.
Planning is where models still differ. The model has to turn incomplete requirements into an architecture, make the calls the prompt leaves open, and catch the failure modes before they get built into everything downstream. Fable 5 won that phase against GPT-5.5. GPT-5.6 Sol is OpenAI’s first flagship since that result, so we wanted to see whether it closed the planning gap.
Test Setup
We gave both models three system-design assignments. Every run started in a fresh, empty project in Kilo Code CLI, with both models on Max reasoning, the highest effort setting Kilo exposes. Same prompt per task, no existing code, no feedback, no follow-ups. The only file each model had to produce was plan.md.
The last test used a single feature-flag service. This time we used three larger systems, each with requirements that depend on getting crashes, retries, and access revocation right:
A webhook delivery platform handling 10 million deliveries a day. An accepted event must reach every subscribed endpoint despite worker crashes, concurrent processing, rotated signing secrets, and customer-controlled URLs.
A usage metering and billing system ingesting 500 million events a day. The same event retried must never be billed twice, late events and corrections must land in the right invoice, and finance must be able to audit every number.
A collaborative project management backend with organizations, permissions, WebSockets, and search. A user removed from a project must lose access everywhere at once, including API reads, live updates, search results, notifications, and attachments.
Each prompt ended with the same instruction:
Please design this system and write the complete implementation plan to a Markdown file named
plan.mdin the current working directory.The plan should make concrete technical decisions rather than listing several options without choosing between them. Explain the reasoning and tradeoffs behind consequential decisions.
Do not implement the system. Do not create source-code files. Only create
plan.md.
We audited the plans against a weighted rubric written before any planning run, along with a task-specific audit checklist for each system.
How the Two Models Worked
Fable 5 reasoned about the task, wrote the plan, and finished. Its longest run took 18 minutes and 49 seconds.
GPT-5.6 Sol wrote a complete plan, then re-read it, launched sub-agents to audit different parts of it, and revised the plan based on their findings. It repeated this loop on every task until it decided the plan was done. On the billing system, the full run took one hour and 22 minutes.
We did not ask for any of this. Neither prompt mentioned sub-agents, reviews, or revision. Both models received the same instruction at the same setting; the single pass was Fable 5’s default behavior, and the audit loop was GPT-5.6 Sol’s.
We graded the final plan each model produced, because that is what a run hands you. GPT-5.6 Sol’s intermediate drafts were not saved as separate files, so this test does not show which model writes the better first draft. It compares the finished plans each model produces on its own.
The Scores
Both models covered the requirements and produced plans a team could build from. Fable 5’s lowest score was 7.75, and its plans were consistently clearer about libraries, file structure, and implementation order. GPT-5.6 Sol scored higher because when its plans made a promise, the details actually kept it. Fable 5’s sometimes did not.
Where GPT-5.6 Sol’s Plans Were Stronger
On all three tasks, Fable 5’s plans had places where a promise made in one section did not hold up in another.
The webhook plan said no accepted event would ever be lost. A few sections later, its overload rule capped fan-out at the first 1,000 matching subscriptions, so the publisher gets a success response and every subscriber past the cap silently gets nothing. GPT-5.6 Sol went the other way. If it cannot create every required delivery, it rejects the whole event before acknowledging it, so a success response keeps its meaning.
The billing plan said a usage event is identified by its tenant and event ID, and a duplicate must never be counted twice. But the ClickHouse schema it proposed deduplicates on four fields, and two of them, the meter and the event timestamp, can change between retries. Resend the same event with a shifted timestamp and it becomes a second row, and both get billed. That is the exact bug a billing system exists to prevent. GPT-5.6 Sol picked one identity at ingestion (tenant, producer, and event ID) and used it unchanged through storage, aggregation, replay, corrections, and invoicing.
The project-management plan promised that a removed user loses access right away. It then served search and reconnect reads from database replicas that lag behind the primary, and cleared cached permissions with a Redis message that can simply get lost. A removed user could keep receiving updates or finding project data until a periodic recheck caught it. GPT-5.6 Sol wrote the invalidations durably, tagged realtime events so delivery pauses the moment permissions change, and kept permission checks on the primary database.
Fable 5 usually got each piece right on its own and missed the contradiction between two of them. GPT-5.6 Sol took each promise and followed it through every part of the system that could break it. That is worth remembering when you review any AI-generated plan. Every section of Fable 5’s plans reads fine on its own, and the flaws sit between sections, exactly where someone skimming a 7,000-word document is least likely to look.
Where Fable 5’s Plans Were Stronger
Fable 5’s plans were the easiest to build from. They named specific libraries, drew out file and folder structures, included the exact SQL for claiming jobs safely across concurrent workers, and laid out milestones with sizes and acceptance criteria. Its project-management plan also had the best test idea in any of the six documents. Generate a test that puts another tenant’s IDs into every route parameter of every endpoint and check that each request gets rejected. GPT-5.6 Sol had nothing like it.
Fable 5’s plans were also smaller. Its billing plan ran 6,792 words to GPT-5.6 Sol’s 13,398, and most of the difference was extra machinery. GPT-5.6 Sol proposed 14 separately deployable processes, cross-region document checkpoints, and Merkle-style verification trees to prove a customer’s usage estimate matches the raw events behind it. Some of that would make sense for a mature payments platform. It is hard to justify for the first version of a metering service, and the rubric docked GPT-5.6 Sol for it on implementation burden and communication efficiency, which is why billing was its lowest score.
GPT-5.6 Sol still won all three tasks, and the reason is that the two kinds of problems cost very different amounts to fix. You can cut GPT-5.6 Sol’s extra services from a plan without touching the decisions that prevent double billing, incomplete delivery, and stale access. Fixing Fable 5’s deduplication key means redesigning the core of its billing plan.
Time and Cost
Fable 5 averaged 15 minutes and 27 seconds per plan. GPT-5.6 Sol averaged 52 minutes and 53 seconds, with nearly all of the extra time spent on the audit loop after it wrote the first draft.
GPT-5.6 Sol was cheaper than Fable 5 on two of the three tasks despite running two to three times longer. Only the billing task, where the full run took 82 minutes, cost more than Fable 5’s run, at slightly more than twice as much. Across all three tasks, GPT-5.6 Sol cost $13.57 to Fable 5’s $12.02, a 13% difference. Per plan, that works out to about $0.52 more, while the wait is about 37 minutes longer.
The dollar totals also do not buy the same thing. GPT-5.6 Sol’s price includes the full audit-and-revise loop, while Fable 5’s covers a single pass with no review. Reviewing Fable 5’s plans to the same depth, by prompting it to audit its own work or handing the plan to another model or an engineer, would cost more than the base $12.02 from our experiment.
How to Review Each Model’s Plans
For a Fable 5 plan, review the promises rather than the sections. Take each guarantee the plan makes (no accepted event is lost, nothing is billed twice, revoked access ends immediately) and trace it through every acknowledgement, retry, cache, and replica the plan describes. The individual sections will look right. The contradictions sit between them.
For a GPT-5.6 Sol plan, review the scope. Ask which components the first version actually needs, which ones answer a measured problem, and which can wait. A prompt that states a team size and delivery window would probably keep some of that machinery out of the plan in the first place.
You can also change both behaviors with your prompt, telling GPT-5.6 Sol to write a single pass and stop, or telling Fable 5 to audit its plan with sub-agents before finishing.
Conclusion
For the strongest final plan, GPT-5.6 Sol won this test, three for three. Its audit loop caught the double billing, incomplete delivery, and stale access problems that survived into Fable 5’s finished plans, and those problems sit at the center of the systems we asked for.
For speed and buildability, Fable 5 won just as clearly. Its plans were right-sized, easy to follow, and finished in less than a third of the time, with concrete guidance GPT-5.6 Sol never matched and none of the extra architecture.
On cost, there was no meaningful winner. GPT-5.6 Sol was cheaper on two tasks, more than twice as expensive on billing, and 13% more expensive in total. Its totals also include the review work, while Fable 5’s price covers a single unreviewed pass.
The clearest change from last month is that Fable 5’s planning lead over GPT-5.5 is gone. Both models wrote plans a team could build from, and neither made the kind of mistake that decides a comparison on its own. GPT-5.6 Sol scored higher mostly because of the audit loop it ran on its own work.
The audit loop gave GPT-5.6 Sol multiple fresh reads of its own plan, and that is where it caught the guarantee violations that survived in Fable 5’s one pass. Based on that, we recommend planning this way regardless of which model you use. Have the model write the plan, then have it review and revise the plan before you take it. The extra spend is easy to justify on plans specifically, because a flaw that makes it into a plan gets built, and unwinding double billing or silent delivery loss in production costs far more than a longer planning run.
For extra assurance, Kilo allows you to run the review across different models, for example writing the plan with GPT-5.6 Sol and having Fable 5 review it as a sub-agent, or the other way around, so the plan gets checked from two different perspectives. Then hand the finished plan to a cheaper model to implement, which is the part that differed less between models in our earlier tests.
Testing performed using Kilo Code, a free open-source AI coding assistant for VS Code and JetBrains with 3,000,000+ Kilo Coders.
Such a great article. Thanks for this!
Very helpful 👍