2. TABLE OF CONTENTS
1. Introduction
2. Data Migration- Process Flow
• System Discovery-Source, Target
• Field Mappings –Source, Target
• Data Extraction
• Data Transformation
• Staging System
• Data Load
• Data Verification
3. Data Migration Tools
4. Data Migration Checklist
3. What is Data Migration
Typically part of a large
program and is often
triggered by a merger or
Acquisition, a business
decision to standardise
systems or modernization
of an organisation’s system .
Why it’s required
Data
Migration
Data migration is the
process of
transferring data between
storage types, formats, or
computer systems.
Introduction
4. Process Flow
Source and Target System Discovery
Source and Target
Field mapping
Data Transformation
Data Load
Data Extraction
5. System
Discovery
(Positive forces for change) (Obstacles to change)
System Discovery
Migration Hardware and software
environment details collected.
Number of Source Systems involved.
Size of the legacy system data.
More critical data elements with
respect to target application system.
Whether aim is to convert all the
legacy application data at once or in
Phases.
Data Cleansing involvement in the
migration of data from legacy system
to target system with/without data.
Redundant data during migration.
Level of Data cleansing required to
maximize the benefit of conversion
and migration.
6. Field Mappings- Source, Target
Mapping performed with the information derived from source, target and business rules.
Document maintained containing the following :
• Change Description (Indicated mapping changes)
• Key Indicator(Indicates whether the field is primary key or not
• Source Field Name
• Source Table/File Name
• Source Field Data Type
• Source Field Length
• Source Field Description
• Business Rule
• Target Table Name
• Target Field Name
• Target Data Type
• Target Field Length
• Comments
7. Export Features of the source
systems used.
Data Extraction
Identified data extracted in
System Discovery phase.
Scripts generated to export the data.
• Systems provide the feasibility to export data
into CSV formats
Salesforce,Siebel CRM, Microsoft Dynamics.
Microsoft Access provides .accdb formats.
SQL provides the .ldf , .mdf and .bak files.
8. Transformations achieved
with scripting
Transact -SQL stored
procedures,functions,SQL
blocks
Data type conversion
Transaction Rules
Implemented
Data Accuracy
Data Validation
Data Transformation
9. Data Cleansing
• Detects incomplete parts of data.
• Modifies/Eliminates inaccurate records.
• Detects Anomalies
Data Sampling- (*) of the rows for a department column needs
to be counted
• Column Property Enforcement
Null values in required columns.
Numeric values that fall outside of expected high and lows.
Columns whose lengths are exceptionally short/long.
Business Rule Application.
Data Transformation
10. In Staging, data is dumped to a location (Staging Area )
1
Minimizes ETL activity on both Source and
Target systems
Used by the transformation phase
2
3
4
Staging System
Can be used for next processing phase
Transformation performed on rational database
server separate from the source databases
and target database.
11. • Target system loaded with data.
• Cumulative data may overwrite the existing data.
Data Load
Load
Update
Schedule
• The extracted data is updated.
• Bulk load facility is available in most DBMS.
• Schedule the load/import process.
• The speed of loading can be influenced by many factors
such as table size, proportion of updates and inserts.
12. Involves testing
for datatype
conversions.
Check the
Table Row
Count.
Monitor Database
Integrity.
Has an impact
on the
operational
performance.Testing of
Translational
rules
implementation.
Tester should
fulfil these
tests against
the subset of
data.
Black Box
Testing- Business
Cases,
Data samplings
use cases Use
cases.
Data Verification
13. Data Migration Tools
Pentaho Data Integration Dataloader
Jitterbit Dataloader Ab Initio
Cognos Decisionstream Talent Open Studio
SAS Open Studio Microsoft SQL Server
Integration Services
Informatica PowerCenter Oracle ETL
14. Migration Planning
Datamap Document
Source System Information
Target System Information
Efficient Scripting
Migration execution
Test cases development for Data
Verification
Data Migration Checklist