It is unlikely that two databases will have the same structure. Different databases put different emphasis on different functionality. One database might be extremely intricate, with myriad fields in each table allowing for extremely precise filtering and reporting, whilst another database may have a less complex data structure but be intuitive, user-friendly or powerful from a marketing perspective.
 
Data migrations can be like putting a square peg in a round hole (for example what if I have three fields in my old data structure but only one place to put this information in my new database – or conversely I have one field in my old database which now needs to be separated into two separate concepts in my new database)?
 
We can either keep hammering the peg until things start to break…… or we can examine the hole and whittle down the peg until it reaches a snug and seamless fit.
 
Sometimes a change in database requires a change in mind-set. What are the strengths of your new database – play to those when considering your data and how it will be migrated. There is no use trying to hold on to the exact structure/style of your data if it is still a square peg and is not making your new database efficient. It may be beneficial for you in the long term to let go of some unnecessary detail in your data or conversely require you to add greater detail in your data going forward.
 
So how do we maintain the essence and effectiveness of the previous data whilst ensuring the effective use of the new databases most powerful functionality – a good mapping:

• A good mapping does not just map data from one field in your old system to the field with the closest fieldname in your new system.
• A good mapping understands the underlying meaning of the information itself and the purpose for which that data will be used both in the past and going-forward.
• A good mapping will also translate the information from one structure or format into the most efficient structure or format in the new database.
• A good mapping should be done by people with a knowledge of the industry and/or purposes to which the data relates.
• A good mapping doesn’t make assumptions based on fieldnames or values (often a database user, faced with a lack of flexibility in their old system, might place an extra telephone number in a text field called reference number for example).

For this reason, communication between the client and the migrator is key. What can you do to help the process run seamlessly:

1. Plan
2. Audit
3. Clean and/or remove
4. Map and Consult
5. Transform
6. Test
7. Adapt 

• Plan a sensible time for your migration. Migrations are complex and can be time consuming.  Change can be challenging. Don’t put stress on your operations at the busiest time for your organisation.
• Take an audit of your current data. What is essential to keep? Is there any information that should actually be deleted for GDPR reasons? Separate the data into that which you use to continue to work towards the mission statement of your organisation (marketing, analysis, research) and that which is only there because it was necessary to record at the time but now is just eating up space. If someone bought tickets to an event 5 years ago, you may still need to know they attended the event but do you really need to still record the price or name or seating arrangement of each ticket they bought?
• Evaluate any data that is essential for you to keep but could be problematic to import. This is the whittling – make sure you retain the aspects of that data which will allow you to perform the tasks essential to your organisation but that still retain the efficiency of the new system. For example, in your old system you have distinct and searchable fields that indicate that someone in your organisation played in a football team, was captain of that sports team and won a particular award for that sports team. Your new system just has one possible field with a list of sports. Can you still perform the necessary marketing/event research strategy you need to if you just maintain a searchable and distinct ‘Football’ tag (perhaps keeping a note of the other details)? Is it better to concatenate the information to retain all of it in the same field (but perhaps make it less searchable and efficient) or to strip away the fat to maintain a clean and simple tag?
• Pinpoint the most important data that you use most frequently for filtering/reporting and ensure that this is migrated to fields which retain the same level of functionality. Don’t settle for a less filterable notes field where a distinct drop-down is available in another area of the data structure.
• Consult during the mapping process so that both parties understand the nature and relevance of each field. Understand the reasons behind any changes/transformations that will be made to your data.
• Following a test migration make rigorous tests to ensure data integrity. What reports can you run on both systems to compare figures and functionality.
• Forget the old ways of working with your old system. Rather than saying ‘but my old database used to do this!’, adapt and revel in your new data environment and spend time exploiting its strengths.

"ToucanTech have done a great job transferring our data from our old system - they worked closely with us to migrate 15,000 records, with information spread across 70 different main tables, carefully mapping an intricate structure of links and relationships to ensure that the data moved across to ToucanTech accurately. The resulting ToucanTech database is well structured and it's easy for us to run filters and reports and manage all of our communications and activities"