1 - Data Migrations Checklist
The following is a semi-profanity-ridden attempt at explaining one way to do data migrations while following best practices. It is rather long and laced with colorful language. If you have read it already, or if you want to avoid the profanity, you can consult the following checklist in the google doc below.
Note that all elements are considered mandatory.
As a quick note, and a reminder even if you've read the whole version:
DO NOT MODIFY SOURCE DATA FILES, EVER.
If you're doing data migrations, either use thisa script developpedto by Geff (readmodify the readme):source https://github.com/SFXD/PSCSVfiles and save the edited version, or use excel workbooks that open the source file and then save the edited result elsewhere. Yes, even if the source is an excel file.
Why? Because sources change. People forget stuff, files aren't well formatted, shit gets broken, and people are human - meaning that one-time data import is actually going to be done multiple times. Edit the source file, and get to do everything all over again. Use scripts or workbooks to do the transformations ? Point that to the new source file and BAM Bob's your uncle.
Or, if you prefer excel, open a blank workbook, Import the source file via data from text/csv, then save it as a both the construction excel, and a NEW csv file after doing your changes in formula columns. That way when you change the source file you can just open the construction book again and resave.
| Action |
Completed? |
| DO YOU HAVE A BACKUP |
|
| Is it UTF-8 encoded |
|
| Did you check it is readable and well formatted |
|
| Does it have carriage returns stored as carriage returns, not as spaces |
|
| Is it up to date |
|
| Do you have a mapping for every object and field |
|
| Did you determine an ExternalID for each object |
|
| Did you determine source of truth (whether to overwrite or not) for reach field |
|
| Did the client sign off on the mapping |
|
| Do you have the source data |
|
| Is it in a format your tool can read |
|
| Are dates and date-times well formatted (yyyy-mm-dd || yyyy-mm-ddT00:00:00z) and are times exported in UTC |
|
| Are field lengths respected (emails not longer than 80 chars, Names not longer than 40, etc) |
|
| Do numbers have the right separators |
|
| Do all tables have the required data for loading (Account Name, Contact Last Name, etc etc etc) |
|
| Do all fields that have special characters or carriage returns have leading and trailing qualifiers (") |
|
| Do all records have an external Id |
|
| Did you do a dummy load with only one field mapped to make your sure tool can read the entire file |
|
| Are you doing transformations |
|
| Did you document them all |
|
| Did you automate them so you can run them again with a click |
|
| Did you read the LDV guide if you are loading more than 1M records |
|
| Did you activate validation rules bypass |
|
| Did you check all automations to deactivate any that should be, including email alerts |
|
| Did you warn the client about when you would do the data load |
|
| Did you warn the client about how long the data load would take |
|
| --------- run the migration ----------- |
|
| Did you reactivate all automations |
|
| Did you remove validation rule bypass |
|
| Did you tell the client you were done and they could check |
|
| Did you check the quality of the data |
|
