AI agents in data migration – more business, less code

Picture a typical migration day. A finance controller and I are looking at a customer ledger from a 20-year-old ERP. "These 4,000 accounts marked blocked – are they dead, or just dormant?" That one question decides whether thousands of records cross into the new system or get archived. That conversation is the migration. The Python that reshapes the data once we've decided is just typing.

For most of my career the ratio was upside down: hours writing extract-transform-load code, minutes left for the questions that actually carried risk. AI agents flip it back – and that's why I've built them into how I work.

Less code, more business

In practice I describe the mappings and quality rules in plain business terms, and the agent writes the Python that extracts, transforms and produces the load files. A real instruction looks less like code and more like a brief:

"Map KUNDNR to CustomerNo, left-pad to 10 digits. Drop rows where the account is blocked and has had no transaction since 2021. Split NAMN into first/last on the last space. Flag any VAT number that fails the checksum rather than dropping it."

The agent turns that into tested, runnable Python and produces the deliverable I'd ship anyway: a CSV or parquet file ready to load into the pipeline or target system. What changes is where my hours go – into the requirements and the judgement calls, not into boilerplate I've written a hundred times before. When a rule changes (and it always does), I change a sentence instead of refactoring a script.

The tools I use: Hermes and Goclaw

Two open agent frameworks are particularly interesting here:

• Hermes by Nous Research (GitHub) – an MIT-licensed, fully self-hostable agent framework. It runs locally, supports many model providers, parallel subagents and the Model Context Protocol (MCP).
• Goclaw (GitHub) – a self-hostable multi-agent platform built in Go, with orchestration of multiple agents, built-in security (including prompt-injection detection) and a local "Lite" edition.

Both fit the AI workflow I've already described: several collaborating agents handling the repetitive work – mapping, profiling, transformation – while the business and I steer. In practice they divide the labour: a profiling agent reads the source and reports what's actually in it (null rates, distinct values, formats that don't match the spec), a transformation agent writes and runs the mapping code, and a validation agent checks the output before I ever sign off on it. Each one hands its result to the next, and I review at the seams.

An extra validation pass – before the business spends time

This is the point that makes the biggest difference. In my 8-step method, steps 6 and 7 – the test migrations – belong to the business. But before anyone in the business even logs into the target system – the ERP the data has been migrated to – I have a set of agents validate the prepared CSV/parquet files against the rules. Concretely, that pass checks things like:

• Completeness – row counts reconcile source to target; no mandatory field is silently empty.
• Formats & domains – dates, currencies and codes match the target's expected types; status values fall inside the allowed set.
• Referential integrity – every order points at a customer that exists; no orphaned foreign keys.
• Reconciliation – control totals (sums of balances, counts per category) match between source and target, so nothing was dropped or double-counted in transit.
• Business rules – the edge cases we agreed on actually hold: the blocked-and-dormant accounts really were excluded, the VAT checksums really were flagged.

The effect: the business starts from clean data instead of burning its review time flagging obvious errors. Their hours go to genuine business judgement – is this customer really still active? – not to catching things a machine should have caught first. It's more efficient for everyone, and trust in the data is built far faster, because the first thing they see holds up.

But what about sensitive data?

The obvious objection: migration data is sensitive – members, prices, salaries, personal data. Pasting that into a public chatbot is a real risk. It can end up in logs and debuggers, be absorbed into future training data, or be re-identified by aggregating it with other sources. Under GDPR that's an unlawful transfer waiting to happen, and the EU AI Act adds its own duties around transparency and oversight. For a regulated enterprise, "we pasted the customer file into a chatbot" is not a sentence anyone wants in an audit.

That's exactly why self-hosted, open agents matter. Hermes and Goclaw can run on your own infrastructure – your VPC, your on-prem box – so sensitive data never leaves your control. The key insight is that the agent doesn't need the data to do its job: it needs the schema and the rules to write the code, and the code is what touches the rows. Combine that with a few principles I always keep:

• Data minimization – give the agent the schema and the rules, not raw data with personal information. It writes a transformation against column names, not against your members' actual records.
• Anonymization – when the agent genuinely needs to see values to reason about a format, hand it masked or synthetic samples, not the live field.
• Opt-out & enterprise contracts – if a cloud model is ever used for a non-sensitive step, turn off training on your inputs and choose enterprise/API tiers with binding data-processing agreements.

The agent writes the code; the code runs on your data in your environment. Efficiency and data security don't have to be at odds – with the right setup they reinforce each other, because a self-hosted pipeline is also a repeatable, auditable one.

In summary

AI doesn't replace the migration lead – it removes the boilerplate so I can spend my time where it counts: the business conversation, the edge cases and the data-quality decisions. The agents profile, transform and validate; I bring the judgement about what the data means. Tools like Hermes and Goclaw make it possible to work faster and keep sensitive data under control – not as a trade-off, but because the same self-hosted setup that protects the data is also the one that makes the work fast and repeatable.

Curious how this would look at your organization? Get in touch – I'm happy to share more.