Change is the only constant in life. In data engineering, it usually arrives as a migration, with the modest expectation that moving everything should disrupt nothing.
After Anaconda acquired Kilo, the entire Kilo data platform had to move into Anaconda’s: the connections bringing data in, the code processing it, the syncs sending it back to business tools, and every business intelligence dashboard.
The challenge was coordinating dependencies, rebuilding permissions and integrations, keeping everything running during the move, and validating that the new environment produced the same results.
My previous two migrations took more than six months combined, though they involved more rewriting. With a similar stack, I’d have expected this migration to take three months without agents, assuming things went reasonably well. Instead, with agents doing most of the work, it took 12 business days from plan approval to finish.
As agents become capable of carrying longer sequences of work, more of my engineering effort goes into making that work reliably delegable. For this migration, I spent more time planning how the agents would execute the work so I could spend less time guiding them through it. That meant turning operational knowledge into procedures they could follow, giving them tools to act across the stack, and building harnesses that checked the results and enforced limits.
Spending more time planning to spend less time supervising
I gave the agent the planning work first: read Anaconda’s stack documentation in Confluence through the Atlassian MCP and inspect the connected systems to work out what the move would require. For the Kilo side, I gave it access through MCP to a knowledge base I’d built from Slack discussions, GitHub activity, meeting transcripts, and agent sessions. It could look up how our systems worked and the decisions behind them as questions came up, without depending on what I’d remembered to include in the prompt. From there, I had it draft the migration plan, including dependencies, validation checks, and rollback procedures, then worked through the result.
This let me spend the planning time reviewing concrete steps and resolving decisions, rather than writing the whole document myself. The aim was to leave fewer questions for execution, when an unanswered question could stop the agent or leave us waiting on another team. Total time in the document: roughly 48 hours.
Every step of the plan, all downstream dependencies that needed to be updated, every secret that had to be rotated or created fresh, the data validation scripts, and the migration scripts all existed on paper before leadership ever saw the document for approval.
What the document contained:
An overview of the proposed move and what each phase needed to accomplish
Architecture diagrams showing the starting point, intermediate states, and destination
A schedule showing dependencies and which work could happen in parallel
Decisions and approvals needed from the other teams before the affected work could begin
Numbered execution steps specifying what to create or change, what to validate before switching over, and which existing paths to keep available for rollback
A data contract defining the source tables and fields in scope and the outputs downstream systems expected
The architecture diagrams were agent-drawn
The agent needed every step spelled out, but I didn’t expect the same enthusiasm from leadership. To help people understand what would change and how the pieces fit together, the plan opened with diagrams, then moved through increasingly detailed summaries before getting into the tasks one by one. People could get their bearings before going deeper, and the agent could follow the procedure in the same document. Changes only had to be made in one place.
The four diagrams showed the starting architecture, phase 1, a bridge phase, and the final state. The agent drew them in Excalidraw, and I kept telling it what was wrong until they matched the system we actually needed. I never edited the drawings myself.
Design, then tickets, then runbooks
Progress lived in a migration tracking sheet. As I worked through a task, I’d ask the agent to write the ticket for it from the plan and add the ticket link back into the sheet, so three layers stayed connected: high-level design, detailed design, and a ticket per executable task. The Jira tickets split the execution along the same lines as the plan, with separate tickets for the jobs, for the tables and views, and for the historical backfill.
Anything that would repeat across sessions got a runbook. Migrating an Omni dashboard is the clearest example. I would do it once with the agent, have the agent write down the exact steps, and replay that file afterwards. When something went wrong, I’d correct the runbook on the spot, so a lesson from one session became a constraint in every session after it instead of dying with the context window.
The runbook also had to explain how to check the result. Otherwise, I’d save time on the migration and spend it checking the agent’s work instead.
Once the runbooks and tools were working, I could hand over a whole batch of dashboards without walking the agent through each one. The bulk of that migration happened in a day, with validation and cleanup continuing afterward.
Written steps were not enough. The agent needed tools that could act and checks that could stop unsafe moves.
A custom MCP for the dashboards
The dashboard procedure required the agent to recreate dashboards in the destination, preserving their layout and behavior. Omni’s existing MCP could read dashboards but couldn’t write them.
I already had a custom write MCP from an earlier move off Metabase. For this migration, I extended it to move dashboards between Omni environments while preserving their formatting and layout and updating references to the underlying data. The tool required a fresh export from the live source, rejected stale or modified snapshots, and checked the imported dashboard’s structure and ran its queries. The import returned failures the agent could act on and checks that compared the result.
Migrating the weekly active users dashboard exposed a missing capability: the agent couldn’t edit the semantic model safely on a separate branch. I extended the MCP, and it then produced a draft with all 41 tiles passing verification. I reviewed it before approving publication.
On the code side, the dbt MCP did the equivalent work.
The ClickOps went to a browser agent
Some steps in the runbooks required configuration through web interfaces that the available APIs didn’t cover. A browser agent made those steps executable too. This was the first time browser navigation genuinely worked for me: hand the agent a runbook, let it drive the UI, and stay out of it.
The Hightouch migration ran entirely that way. I also built a custom Hightouch MCP, but used it mostly for reading syncs, spotting failures, and getting fix suggestions rather than for the move itself.
Rebuilding the ad platform ingestion connections was the other case. Meta and Bing Ads each have their own token flow, and I don’t work in those platforms day to day, so the agent went and found where to generate each new token and what the steps were.
Keeping the agent in the right environment
Executing the runbooks also required the agent to choose the right tool and environment at each step. A question might require inspecting the running system rather than changing code, and a migration step might require reading from the source or acting on the destination.
I encoded that guidance in a custom Kilo mode I called Migration Assistant. During the migration, both Kilo’s and Anaconda’s warehouses were live behind near-identical MCPs. The mode encodes which one is the source and which is the destination, and when to query each. It also carries the validation procedure: after switching on a new ingestion, confirm the backfill completed, then check that the dbt models produce the same metrics on both sides. It knows when to reach for the knowledge base too, though this project didn’t need that often.
Deterministic harnesses did what instructions couldn’t
The agent had access to four versions of the same system at different stages of the migration: the live original in Kilo’s Snowflake, a copy I created, that copy exposed in Anaconda, and the final prod version in Anaconda. Choosing the right target depended on which step it was executing, while the targets themselves differed only by database name and account. Given how similar they were, the agent mixed them up.
Instructions didn’t fix it. I put the rule in AGENTS.md and it still targeted the wrong database from time to time, so I wrote a harness that checks the database against the environment and hard-fails when they don’t match. A wrong target should stop the run and bring the decision back to me. Letting the agent guess again would defeat the point.
The one I lean on most is dbt safe, a wrapper around the dbt CLI. The agent runs dbt safe run <model> instead of dbt run <model>, and the policy behind the wrapper refuses --full-refresh outright, refuses any command pointed at a production target, rejects commands it doesn’t recognise, and holds a migration apply until it gets approval. I wrote it after an agent full-refreshed a dev table with 7 billion rows in it, and after another overwrote a table I was working on. I wanted the CLI reachable as a fallback for when the MCP breaks, since the MCP holds my credentials, without leaving the destructive commands reachable with it.
Around those sit a script the agent runs when submitting a PR, which enforces that the description says what I want it to say, and a CI job that fails and sends the agent back to fix its own work. In many cases I also require the agent to write a unit test, so that a later agent that has forgotten the constraint hits a failing test instead of shipping a silent regression.
Since then, I’ve been replacing AGENTS.md instructions with scripts and tests. Long autonomous loops work better when there’s something deterministic to fail against. Most of these harnesses predate the migration, built during my regular work at Kilo and reused here.
What broke at cutover
The migration scripts ran through dbt and were tested in dev first. In prod, the dbt jobs failed.
The first cause was database targeting. A job would point at the wrong database, or build a malformed database name, because the macro constructing that name behaved differently between dev and prod. The fix loop was mechanical: the agent opened the failing job, read the Snowflake error message, traced it back into dbt, fixed it, and reran. Because the SQL scripts were written to be re-runnable, a failed step cost a rerun rather than a rebuild.
One failed job was reassuring: the row counts didn’t match, and the job refused to commit. On September 4, the migration job died on the PostHog window covering August 15 to September 5: 230,956,592 rows inserted against a target of 232,494,508. The macro doing the range replacement counts both sides before it commits and rolls back when they disagree, so the run failed with a comparison in the error message and left nothing half-written behind. The cause of the mismatch was never established. The retry split the same interval into four smaller windows and all four passed.
Issues ran through the first day or two. Since then, zero errors.
Models and cost
I used GPT-5.6 Sol on high for the early sessions to write the plan, then switched to GPT-5.6 Luna for execution.
I reconstructed usage from my local agent logs: 171 tasks and 19 subagent sessions between August 20 and September 6, which attributes roughly 9.41 billion tokens to the migration.
Cached input dominates because the same context gets processed again on every turn. The part that represents new text, fresh input plus output, comes to 291,836,281 tokens. Sol accounts for about 7.48 billion of the total and Luna for about 1.90 billion, with a long tail of a few tens of millions across other models.
Priced at standard published API rates on September 11 (Kilo charges provider rates with no markup), the attributed usage is worth $4,164.17. Sol carries $4,081.18 of that, on about four times Luna’s token count at roughly twenty times Luna’s published rate. For a platform migration, that is cheap.
The number needs three qualifications. It isn’t a billing export, it values recorded usage at current rates rather than reconstructing what was actually charged, and the records run from August 6 rather than covering only the twelve business days of execution. Which tasks count as migration work is also a judgment call I made by reading titles and prompts.
What I’d do differently
I’d batch harder. The plan walked through one data mart at a time, marketing, then finance, then product, migrating the dashboards, then the dbt jobs, then validating before moving on. With the runbooks written, the agent could have taken far more of that in one pass. After watching it work through that many dashboards in a day, I started wondering why I’d been handing them over one at a time.
I’d also stop earlier on the plan document. An agent will happily generate unlimited detail, and the document has two audiences: people who need to read and approve it, and an agent that needs enough precision to execute. Splitting it into two documents creates a drift problem, so one artifact has to serve both, and deciding how much detail is enough is a judgment call that took me too long each time.
The failure mode I actually hit wasn’t hallucinated schemas or invented SQL. It was over-engineering. The agents wrote things that were more complicated than the job required.
Twelve business days later, the platform was running in Anaconda, and the time spent preparing the work had paid for itself many times over. As agents get better at execution, the engineering has to move further upstream: making the context available, setting the boundaries, and defining what a correct result looks like before the work begins.
When an agent keeps coming back for guidance, I want to understand whether it needs my judgment or whether I’ve left something in my head that should have been in the procedure. The more capable the agents become, the more expensive it is to keep making myself a dependency.