Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection , April 17, 2007 (First Draft), London, Tim Bass, CISSP, Director, Principal Global Architect
Emerging Technologies Group
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection
1. Tim Bass, CISSP Director, Principal Global Architect Emerging Technologies Group Complex Event Processing (CEP) for Next-Generation Security Event Management, Fraud and Intrusion Detection April 17, 2007 (First Draft) London
2.
3. Who We Are and What We Do We help our customers… Improve operational visibility, collaboration and ability to be proactive Increase operational efficiency and effectiveness Accelerate projects, initiatives and go-to-market cycles A leading provider of business integration and process management software.
4. How TIBCO Delivers for Customers Accelerate projects, initiatives, and go-to-market cycles Increase operational efficiency and effectiveness. Improve operational visibility, security, collaboration and responsiveness
5.
6.
7.
8. Revenue Numbers FY 2004 – 2006 (in thousands of dollars) 15.8% $61,060 $73,715 $387,220 FY2004 16.4% $73,127 $67,081 $445,910 FY2005 16.6% $85,923 $90,558 $517,279 FY 2006 R&D SPEND AS A % OF REVENUE R&D SPEND PRE-TAX PROFIT REVENUE
11. Complex Event Processing " Events in several forms, from simple events to complex events, will become very widely used in business applications during 2004 through 2008 " --- Gartner July 2003
12. What is Complex Event Processing? Detecting Threats and Opportunities with PredictiveBusiness®
13.
14. Bloor Report on Event Processing Event Processing and Decision Making Automated Operational Decisions Automated Predictive Decisions Human Predictive Decisions Human Operational Decisions Decision Latency Event Complexity Process Complexity Pattern Matching and Inferencing Anti-Money Laundering Credit-Card Fraud Exchange Compliance Database Monitoring Algorithmic Trading Trade Desk Monitoring Customer Interaction Order Routing RFID Tariff Look-Up Rail Networks Search & Rescue Baggage Handling Liquidity Management
15.
16.
17.
18. Classification of Intrusion and Fraud Detection Systems Traditional View Before Data Fusion Approach to FDS and IDS Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Security “Stovepipes” Centralized
19. Intrusion Detection and Data Fusion (2000) Next-Generation Intrusion Detection Systems Source: Bass, T., CACM, 2000
22. Emerging Event-Decision Architecture Customer Profiles Purpose-Built Analytics Secure, Distributed Messaging Backbone Internet/Extranet Sensors Human Sensors Edge/POC Sensors Operations Center Other Reference Data Rule-Based Event Processors
23. Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
24. CEP – Situation Detection Hierarchy 22 Adapted from: Waltz, E. & Llinas, J., Multisensor Data Fusion, 1990 Impact Assessment Situational Assessment Relationship of Events Identify Events Location, Times and Rates of Events of Interest Existence of Possible Event of Interest Data/Event Cloud Analysis of Situation & Plans Contextual and Causal Analysis, Rules Causal Analysis, Bayesian Belief Networks, Rules, NNs, Correlation, State Estimation, Classification Use of Distributed Sensors for Estimations Raw Sensor Data (Passive and Active) HIGH LOW MED
25. CEP High Level Architecture 22 Adapted from: Engelmore, R. S., Morgan, A.J., & and Nii, H. P., Blackboard Systems, 1988 & Luckham, D., The Power of Events, 2002 EVENT CLOUD (DISTRIBUTED DATA SET) KS KS KS KS KS KS KS KS KS KS KS KS KS KS
26.
27. Complex Event Processing Reference Architecture Next-Generation Functional Architecture for Fraud and Intrusion Detection 24 EVENT PRE-PROCESSING EVENT SOURCES EXTERNAL . . . LEVEL ONE EVENT TRACKING Visualization, BAM, User Interaction CEP Reference Architecture DB MANAGEMENT Historical Data Profiles & Patterns DISTRIBUTED LOCAL EVENT SERVICES . . EVENT PROFILES . . DATA BASES . . OTHER DATA LEVEL TWO SITUATION DETECTION LEVEL THREE PREDICTIVE ANALYSIS LEVEL FOUR ADAPTIVE BPM
28.
29.
30.
31.
32.
33.
34.
35.
36. Business Optimization Summary A Simplified View of the CEP Reference Architecture Flexible SOA and Event-Driven Architecture
37.
38. TIBCO’s Real-Time Agent-Based SEM Approach A Multisensor Data Fusion Approach to Security Event Management Distributed Fraud and Intrusion Detection Systems, Logs Detection Approach Systems Protected Architecture Data Sources Analysis Timing Detection Actions HIDS NIDS Hybrid Audit Logs Net Traffic System Stats Real Time Data Mining Anomaly Detection Signature Detection Centralized Distributed Active Passive Agent Based Enterprise Correlation of Security Events
39. Security Event Management High Level Event-Driven Architecture (EDA) for SEM (CEP and BPM) JAVA MESSAGING SERVICE (JMS) DISTRIBUTED EVENTS (TIBCO EMS) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) HIGH PERFORMANCE RULES-ENGINE (TIBCO BE) SENSOR NETWORK RULES NETWORK FDS BW JMS LOGFILE JMS BW LOGFILE JMS BW LOGFILE JMS BW IDS JMS BW FDS JMS BW SQL DB BW JMS ADB SQL DB BW JMS ADB MESSAGING NETWORK SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM SYSTEM BPM Compliance Workflow (TIBCO iProcess)
43. TIBCO BusinessEvents™ Awards 2006 Best Complex Event Processing Software Winner: TIBCO 2006 Event Processing General Purpose Gold Award Winner
44.
45. On-Line Fraud Detection Use Case Architecture and Capacity Planning Approx. 12,000 Hits Per Second During Peak Period Across the Three Sites – One Instance Of TIBCO BusinessEvents™ Capable of Handling Maximum Hits Overall 100 Million Hits Handled Between 3PM – 4 PM Peak Approx. 250 Million Hits Per Day Across the Three Sites TIBCO EMS™ TIBCO Business Events™ Session Info Three Server Farms ~600-700 Application Servers