The document discusses optimizing regulatory compliance through next-generation data management and visualization. It outlines trends like increasing data volumes, sources, and regulatory requirements that are challenging traditional compliance architectures. A modern approach is proposed using big data platforms to ingest diverse data sources, perform automated preparation and analysis, and enable flexible reporting and visualization. This can help reduce costs, speed reporting, and improve auditability versus manual spreadsheet-based processes. Examples show how data preparation platforms combined with data storage, analytics, and visualization tools help financial firms more efficiently meet regulatory obligations like the SEC's Form PF.
Software Project Health Check: Best Practices and Techniques for Your Product...
Optimizing Regulatory Compliance with Big Data
1. Optimizing compliance with next-generation
data management and visualization
Optimizing Regulatory
Compliance with Big Data
2. Agenda
Has Regulatory Reporting been left behind?
Perspective from a user
Paul Van Siclen
Director of Market Development
Financial Services
Qlik
Mihaela Risca
Sr. Solutions Marketing Manager
Financial Services
Cloudera
Trends and Challenges in today’s Regulatory environment
Bertrand Cariou
Sr. Director of Solutions and Partner
Marketing
Trifacta
Bringing it all together, the Power of 3
3. Paul Van Siclen
Director of Market Development
Financial Services
Qlik
Has Regulatory Reporting been left behind?
Perspective from a user
5. Let’s get real
Business Unit
(1)
Risk
(2)
Internal Audit
(3)
Senior Management / Corporate
Board, Audit & Risk Committee
ExternalAudit
Regulators
Originations/
Transaction
systems
Balance Sheet/
Portfolio
Sub-ledger
Market Data
Economic/
Regulatory
Capital
….the ugly truth
Traditional static BI, spreadsheet driven
Modern Regulatory Compliance Architecture
Purpose built Applications; self service BI; Rapid
deployment
Business Unit
(1)
Risk
(2)
Internal Audit
(3)
Senior Management
Board, Audit & Risk Committee
ExternalAudit
Regulators
Originations/
Transaction
Systems
Balance
Sheet/
Portfolio
Sub-ledger
Market Data
Economic/
Regulatory
Capital
External Data
Sources
Next generation Big Data & visual analytics
9. Challenges with current state of Compliance
15-20% of the “run the bank” budget is
spent on regulatory compliance.
Data silos lead to duplication
Proprietary architectures are resource
intensive and expensive to maintain.
• Financial data grows 100% per year. 70%
of this data is unstructured, which is
challenging to handle with traditional
systems.
• Too many data sources lead to
inconsistency and hard to reconcile data.
• Challenges with finer granularity (BCBS239
mandates transaction level)
• Challenges with data lineage and audit
High Cost of Compliance Risk of Non-Compliance Long Time to Reports
Manual processes
Slow to implement changes.
Preparing the data for analysis takes
too long.
Report processing and validation
exceed the window for reporting.
10. Challenges with Traditional Data Architecture
● Data silos
● Difficult to add
unstructured data
sources
● Proprietary
architecture
● Sensitive data spread
across systems
● Batch reporting
processes
● Rapid data collection
is challenging
● Issues with data
quality
● Lacking agility and
scalability to address
future regulatory
requirements
11. Managing Data for Comprehensive Regulatory Compliance
Handle real-time
data ingest from
diverse sources
Governance and
Security
Data Streams
Deployment Flexibility
Machine Learning
Capabilities
Diverse Analytical
Options
Enterprise Data Hub
Scale easily & Cost
effectively
Batch or Real- time
Data Streams
Data Sources
A comprehensive data management platform to support more accurate and efficient compliance
Data Sources
Data Storage &
Processing
Reporting, Analytics &
Auditing
Data Ingest
Other
Data Governance (Data Lineage, Data Protection)
Data workflows and policies
Data Preparation
14. To drive more
value here
Ingestion
Before Data Can be Reported, It Has to Be Prepared
15. To drive more
value here
You have to make this more efficient
Optimizing &
Publishing
Enriching &
Blending
CleaningStructuringDiscovery
Ingestion
Before Data Can be Reported, It Has to Be Prepared
16. To drive more
value here
You have to make this more efficient
Optimizing &
Publishing
Enriching &
Blending
Cleaning
80% of the time spent 20%
StructuringDiscovery
Ingestion
Before Data Can be Reported, It Has to Be Prepared
35. Private Fund Adviser SEC's Form PF: Portfolio Transparency
Data Storage & Processing Platform
Data Preparation
SEC Form PF Regulatory Reporting
GENERAL LEDGER DIVIDENDS
Reporting Total Balance per Account
and Credit Default Swap Interest
Business Problem:
- Inefficiency done in Excel, requiring loads of manual
intervention hence risks and high cost maintenance
- Inflexible to catch up with growing analyst community
requirements
- Highly visible throughout organization and to the
Regulators
Solution:
- Stored in Hadoop, managed with Cloudera, wrangled
with Trifacta and visualized with Qlik
- Exponentially improve the “work paper” process with
confidence of sound master data management
Value Achieved:
- x5 times faster to report both in design and processing
- Standardized process across 200 analysts
- Lineage tracking for audits
- More agility in turning around compliance/reg. reports
- Reduce overhead on IT and promote data access to LOB
users.