How to get the best value from data migration

Lab Blog Article

  • Published — Oct 2019
  • Topic — Data Migration
  • Author — Clinton Abraham

Focusing on the future without understanding the past

Often businesses focus too much on the data of tomorrow — understandably, they're excited about the new solution. There's an assumption that migrating the data across from the old system would be a straightforward task. But that's not always the case.  

The pitfall here is not spending any time understanding your current data. And then finding you can't migrate it to the new system.    When you're starting a project, think about where data is going and where it's come from. You can call out issues early, so risk mitigation is easier, and you can assess if there is time and budget to fix them.   Understand explicitly what data the new business processes will need and that the migrated data will meet that need. The data model in legacy applications might not be conducive to transformation into the target solution data model. So, know what data are required for the target solution to operate successfully.    

Take it all!

The first reaction from leadership or within businesses is to migrate all the data from the legacy systems into the new system.    What's the problem with taking all our stuff? Well, there can be years or even decades worth of master and transactional data in legacy applications. An SMB that takes payment receipts for a million transactions a year, for 18 years? That's a lot.   Some cloud solutions have a complex migration method with obscure zip files — the resources involved in migrating even a relatively small amount of data can be prohibitive.    Regardless of the business' wishes, you might come across vendors who say no to a "migrate it all" approach — it can increase your project risk significantly. The impact on a business that needs to cleanse 18 years worth of data is simply not something they want to consider.   The trick is to know where to draw the line in the sand. This might mean your best course is to archive the old data and start again in the new system. Or only take a more recent subset of the legacy data.    

The push to shut down legacy systems

Often there's a push to migrate everything so that legacy systems can be shut down to realise benefits and get the infrastructure back. Without an adequate strategy to manage any legacy data left behind, and if legacy applications can’t be decommissioned on schedule, delays to the project benefits realisation are almost guaranteed. You can end up wasting resources trying to make migration happen for data that could be easily archived.    The best approach is to recognise what you need early on and take only that. Develop a post-project archive solution and opt to migrate some data post-go-live.    This will look different for each organisation. But it could be a data warehouse (where you store processed and refined data) or data lake (where you store unprocessed data in its native format). This way, your data can deliver all sorts of value to the business down the track.   

Data cleansing isn't always a good investment

Regardless of data quality, the complexity of the data migration will depend on how close the data model is to the new target data model. And whether the legacy system captured data at the granularity of the new system. These things determine the amount of transformation required for a single record to be moved into the new format.    There'll undoubtedly be duplicate entities across the organisation's data model that will be migrated into the same dataset. People with different addresses or name changes are a typical example. And there'll potentially be multiple versions of the same application in production environments to consider.   Cleansing old data for the new solution often sees no return on investment. Do you have the time and budget to add that to the project scope?     

Data archive options

Sometimes businesses are legislated to keep all of their data. Make sure you understand the requirements at the start of the migration — data doesn't necessarily have to be in your production system if it's not operational anymore. As long as it's archived, you can access it if needed and still comply with organisational or legislative access.   There are costs involved, so think about your storage options and understand your legislative obligations upfront.     

Lack of data governance

Often you'll find there's no strategy or framework for the whole organisation. They simply don't have a plan for how to deal with the data in their new ERP, or from the system before. This means the business will find it more difficult to get any value from their data or draw insights from any of it.    The Data Management Body of Knowledges suggests you tackle data governance over ten knowledge areas. It guides you to consider each knowledge area in context to your organisation’s data and each stage of the data lifespan.   You're not going to be able to take action on all ten steps at once. And you might never achieve the gold standard of data management because it's a continuous thing. By the time you've got your reference data and master data processes nailed, you're going to start again. But you've got a plan, and that's more than many organisations.  

Key data migration considerations

● Acknowledge early in the project that migration to cloud environments can be a bigger risk and more complex than migration to on-premise environments. Double down on that risk with brownfield cloud environments.

● Create a vision or strategic plan for the organisation's data management. Help the business to understand how to leverage their data and external datasets to deliver untapped value.

● A data warehouse or lake is likely to meet the needs of legislative obligations. It can remain ‘complete’ in its original data model and can be drawn upon or expanded for business intelligence, or archived and purged as required.

● Data migration for record keeping purposes only is not a valid reason for migration. 

● Data governance happens at all stages of the data lifespan.   

Data isn't sexy, but it is a hot topic

Let's face it, data is pretty dry. You're always talking about it from a cost perspective, not from funding revenue growth.    The thing is, there's huge opportunity and hidden value in data. Simply asking questions like "how can we use data to better customers' experience?" or "how can we drive process improvement and streamline our operations?" is a great start.   Your rockstar business analyst will look at your data as a business asset and manage and invest in it as such.    Get them involved early on in your migration goldmine, so you're not left wishing you'd taken a different approach ten years later. And make data part of your business' strategic plan now and in the long-term. If you need help with data migration or would like to understand your options, the team at the Project Lab would be happy to help you get started.

The Life of Data

Successful data migration starts with understanding what happens to your data during its often lengthy existence. It’s not a lifecycle – it’s a lifespan. It’s born, it’s used, it dies.

Download / 101.8 KB PDF