More Related Content Similar to Rapid Data Integration: Tools & Methodology Similar to Rapid Data Integration: Tools & Methodology (20) 1. 1 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Rapid Data Integration
Tools and Methodology
2. 2 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Agenda
What is Data Science?
Why Data Integration matters
Why Iterative Methodology is critical
How do we get there?
What is the payoff?
3. 3 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
What Is Data Science?
Data Science is about analytics
– Different from Business Intelligence
– Looks forward
– Statistical analysis, predictive
analysis, trending, etc.
4. 4 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Data Science
Analytical tools
– Slicing and dicing, not
reporting
Requirements
– By definition, you don’t know
what you need up front
– Traditional data
management tools fall short
Data profiling, modeling,
ETL, etc., all depend upon
defined requirements
5. 5 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Why Data Integration Matters
To be successful, data
scientists need good data
– Key question is “why?”
– Are we seeing a trend, or is
the data bad?
One question will lead to
another
Experimentation
New requirements, more
data, new subject areas
How to keep up?
6. 6 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
The Importance Of Iterative Methodology
Change
– Market driven change – quarterly
– Business driven change – monthly
– Analysis driven change – daily
To deal with change, iterative methodologies succeed
– Absorb
– Implement
– Deploy
– Test
– Repeat
7. 7 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Iterative Data Integration
Integration model
– Map sources to analytical model
– Revise physical schema as model evolves
– Automation
Reduce error, speed deployment and track changes
Usability
– Make the data available – now!
8. 8 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Business Focused Model
Start with a model
– Design logically, deploy physically
– Support iterations
9. 9 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Automation Using Best Practices
Master Data Governance and Stewardship
Schema Management
Model and Metadata Management
Hierarchy Management Workflow and SecurityData Profiling and Validation Data Authoring
Controlled PublicationIdentity Management
Auto-generated Application
Browse and Search Full History and Audit TrailsAuto Match and Merge
OperationsData Integration
Data Validation
Suspense and Exception Handling
Data Sourcing and Field Mapping
Delta Detection
Surrogate Key Management
Code Management and Lookup
Currency and UoM
Graphical Modeling
Model FederationMulti-GranularitySub-typing and Inheritance
Composite EntitiesRagged Hierarchies Change ManagementKPI Management
Business Metadata Classification Hierarchies
Star and Snowflake Schema
Physical Schema Management
Slowly Changing Dimensions
Data Mart and Aggregates
Data Load and Index Management
Rollup Path Awareness
Incremental Summary Generation
Process Automation
Task Execution and Monitoring
Deployment and Migration
Archiving
Restore for Model and Data
Undo Loads
Audit and Logging
Presentation
Metadata Management for BOBJ
Native XLS Pivot Table Generation
Native QlikView Generation
Metadata Management for MSAS
Metadata Management for COGN
MDM Consumer Interface
Report-Time Formula Management
Automated
10. 10 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
What Is The Payoff?
Better quality data
– Better analysis
– Better decisions
– Better business results
More timely data
– Reduce guesswork and
estimations
– Act with confidence
More efficient
– Conserve your resources
– Keep pace with change
11. 11 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Summary
Data Science offers new insights into business
Data Science demands current accurate data
Requirements difficult to pre-determine
Flexible, agile data platform is a must
Kalido Information Engine supports…
– Business focused modeling
– Rapid integration
– Process automation
– Accurate data – fast!
12. 12 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
What Are Your Next Steps?
Attend the next how-to webinar on how to use the
Business Information Model for rapid data integration
Schedule a 1:1 Discovery Session with one of our
experts and get hands on time with the tool
Schedule a more in depth session on rapid data
integration for you and your colleagues
13. 13 May 28, 2013© Kalido I Kalido Confidential I May 28, 2013
Polling
I have a clear understanding of how data integration
affects my ability to improve insights
I would like to attend a “how-to” webinar on the Business
Information Modeler and how it is used
I would like to schedule a Discovery Session with a Kalido
Expert and get some experience with the tool
I am unsure as to how rapid data integration affects my
ability to improve insights
14. 14 May 28, 2013© Kalido I Kalido Confidential I May 28, 201314
Thank you
Editor's Notes Recall that I said there are many steps in the process of turning data into information. This is just a partial list of the tasks involved in a best practices data warehouse operation. These tasks don’t go away in a model-driven approach. But, being well-governed requires repeatable processes that deliver consistent results. There is no way to achieve that if these remain manual tasks or are driven from different perspectives. Manual processes are often prone to human error, are inconsistent, and take too much time – especially when requirements change frequently. In a model-driven approach, information policies that support business requirements emerge directly from the business information model. These information policies form a reference against which all of these tasks can be executed. Because of this, you can automate a very large number of these tasks and achieve consistent results. Anybody who’s built data warehouses using Kalido would never go back to what one of our customers call “stick build”. Because you’d never want to have to design and write code that does all this from scratch again. We handle ragged and variable depth hierarchies. We handle data exceptions. We have a built-in match engine for identity resolution. We build slowly changing dimensions without you having to do a single thing. We generate data marts. And all of this is driven by the business information model.
