Blog
Insights on data migration
Practical lessons on data migration, data quality and master data management — from real projects.
The data object migration plan: what moves, and when
A migration plan isn't a field-mapping spreadsheet — it's a dated runbook of what data moves into the new system and on which day around go-live. Bulk loads versus delta loads, why master data and transactions freeze on different dates, and why the cleanest way to migrate the hardest objects is to clear them in the real world first.
Read the post
AI agents in data migration – more business, less code
With self-hosted AI agents like Hermes and Goclaw I spend my time on the business conversation and the requirements instead of on code. The agents write the transformations, produce CSV/parquet and validate the files before the business even goes into the target system to review the data – without sensitive data leaving my control.
Read the post
Polars vs. pandas: a speed comparison, through an Apple Silicon lens
Polars advertises gains of more than 30x over pandas. I sanity-checked that against five everyday data-engineering tasks – read, write, compute, filter and group-by – on small and large datasets, all measured on a modern Apple Silicon laptop (a 16 GB M4).
Read the post
CSV vs. Parquet: which file format should you choose for modern data platforms?
CSV is the universal exchange format; Parquet is the analytics workhorse. I measured both on the same MacBook Air (M4) – storage size, read, write, single-column reads and filtered scans – on a tiny Iris file and a real 5-million-row fraud dataset, then swept every Parquet compression codec. The crossover point is sharp, and the codec you pick matters more than you'd think.
Read the post
Book review: Managing Data Quality – A practical guide
Managing Data Quality by Tim King and Julian Schwarzenbach is a pragmatic guide to what decides most migrations: data quality. Here's my review and why I recommend it.
Read the post
GDPR and data migration: what you can't just copy over
On a recent membership-database project, the source data was about as sensitive as it gets in Sweden – personnummer, names, addresses, protected identities. None of that belongs in a test or migration-trial environment. Here's why, and the deterministic Python/Faker approach I used to replace real members with believable fakes while keeping the data usable for testing.
Read the post
Data migration at one of Sweden's largest retailers – PDM v3, an 8‑step method and AI agents
How I lead the data migration in a large-scale business transformation at one of Sweden's largest retailers – with PDM v3, my own refined 8‑step method per data object, and an AI workflow of several collaborating agents.
Read the postStreamlit in data migration – fast data visualization with parquet, free and open source
During a migration I constantly need to look at the data: volumes, anomalies, reconciliations. Streamlit lets me build interactive dashboards in pure Python in minutes – on top of parquet files, completely free and open source.
Read the post
The first 30 days of a successful data migration project
A migration's fate is often decided before a single row of data is moved. Here's how I spend the first 30 days – discovery workshops, source system analysis, data profiling and a sharp scope – so the rest of the project stays calm and predictable.
Read the post
A practical checklist for master data management
Master data management doesn't have to be a heavy framework. Here's a pragmatic checklist for getting started with governed, quality-assured business data – without getting stuck in theory.
Read the post
Migrating data into Business Central with RapidStart: built-in validation, and the Excel row limit
RapidStart configuration packages are the built-in way to load setup and master data into Dynamics 365 Business Central – and their best feature is that every record is validated with the same business logic as manual entry. But because the data rides through Excel, you hit the worksheet row ceiling, and large tables have to be split across several files. Here's how it works and where it fits.
Read the post
Book review: Practical Data Migration by Johny Morris
Practical Data Migration by Johny Morris is the book behind the PDM methodology I work to. Here's my review – why it's still one of the most valuable books for anyone leading a migration.
Read the post
Migrating data into Dynamics 365 Finance & Operations: which tool fits when?
Dynamics 365 Finance & Operations gives you several ways to load legacy data – the Data Management Framework, the Excel add-in and the dedicated upgrade tooling. Here's how they differ, which one fits which situation, and where the tool stops and your discipline takes over.
Read the post
Why data migrations fail – and how to avoid it
Most migration projects derail for the same reasons – and usually long before any data is moved: underestimated data quality, undefined quality expectations, unclear ownership, too much scope, no reconciliation strategy and testing that starts too late. Here are the most common pitfalls – and how I avoid them.
Read the post
MDM vs data migration – what's the difference?
Master data management and data migration are constantly confused – but they're different things with different goals and different lifespans. Here I clear up the misconception and show how they fit together.
Read the post
SAP S/4HANA Migration Cockpit: strengths and limitations
The SAP S/4HANA Migration Cockpit ships with two ways to load legacy data – staging tables and direct transfer from an SAP system. Here's how they differ, what the cockpit does well, and where you still need to bring your own discipline.
Read the post
Crow's Foot ERD – how I always draw data models for data migration
When I build information models at a client, a Crow's Foot entity-relationship diagram is almost always the first thing on the whiteboard. Here's a beginner-friendly walkthrough of entities, attributes and the cardinality symbols – and why this one diagram is so useful in migration and MDM work.
Read the post
Master data vs transactional data – the foundation under every ERP system
Whether you're implementing a new ERP, building a warehouse or planning a migration, one distinction decides how much pain you're in for: master data versus transactional data. Here's the difference, why master data carries the real risk, and the rule of thumb I use to tell them apart.
Read the post
Reconciliation: proving the migration was correct
A clean load is not a correct migration. The difference between the two is reconciliation – the discipline of proving, with evidence a CFO and an auditor will accept, that what arrived in the new system matches what left the old one. Here's how reconciliation actually works, layer by layer, and why it's the deliverable that earns sign-off.
Read the post
Big bang vs. phased migration: how to actually choose
Should you move everything in one cutover, or migrate in stages? It's one of the first big decisions on any migration, and it's usually argued about for the wrong reasons. Here's an honest comparison – what each approach really costs, the rollback question, and the factors that should actually decide it.
Read the post
Migrating data into Infor M3: the parts that bite at cutover
Most articles on M3 data migration tell you to involve your key users and clean your data early. True, and true of any migration ever. This is the other version – written from inside the cutover window. The M3-specific constraint that changes the whole approach, why master data and transactions are two different projects, and why the hardest data objects are usually best solved by the business, not by a script.
Read the post