1 - Data Migrations Checklist

The following is a semi-profanity-ridden attempt at explaining one way to do data migrations while following best practices. It is rather long and laced with colorful language. If you have read it already, or if you want to avoid the profanity, you can consult the following checklist in the google doc below.

Note that all elements are considered mandatory.

As a quick note, and a reminder even if you've read the whole version:

DO NOT MODIFY SOURCE DATA FILES, EVER.

If you're doing data migrations, either use a script to modify the source files and save the edited version, or use excel workbooks that open the source file and then save the edited result elsewhere. Yes, even if the source is an excel file.

Why? Because sources change. People forget stuff, files aren't well formatted, shit gets broken, and people are human - meaning that one-time data import is actually going to be done multiple times. Edit the source file, and get to do everything all over again. Use scripts or workbooks to do the transformations ? Point that to the new source file and BAM Bob's your uncle.

Or, if you prefer excel, open a blank workbook, Import the source file via data from text/csv, then save it as a both the construction excel, and a NEW csv file after doing your changes in formula columns. That way when you change the source file you can just open the construction book again and resave.

 

Action	Completed?
DO YOU HAVE A BACKUP
Is it UTF-8 encoded
Did you check it is readable and well formatted
Does it have carriage returns stored as carriage returns, not as spaces
Is it up to date
Do you have a mapping for every object and field
Did you determine an ExternalID for each object
Did you determine source of truth (whether to overwrite or not) for reach field
Did the client sign off on the mapping
Do you have the source data
Is it in a format your tool can read
Are dates and date-times well formatted (yyyy-mm-dd || yyyy-mm-ddT00:00:00z) and are times exported in UTC
Are field lengths respected (emails not longer than 80 chars, Names not longer than 40, etc)
Do numbers have the right separators
Do all tables have the required data for loading (Account Name, Contact Last Name, etc etc etc)
Do all fields that have special characters or carriage returns have leading and trailing qualifiers (")
Do all records have an external Id
Did you do a dummy load with only one field mapped to make your sure tool can read the entire file
Are you doing transformations
Did you document them all
Did you automate them so you can run them again with a click
Did you read the LDV guide if you are loading more than 1M records
Did you activate validation rules bypass
Did you check all automations to deactivate any that should be, including email alerts
Did you warn the client about when you would do the data load
Did you warn the client about how long the data load would take
--------- run the migration -----------
Did you reactivate all automations
Did you remove validation rule bypass
Did you tell the client you were done and they could check
Did you check the quality of the data