Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

8

Share

Big Data Day LA 2015 - Event Driven Architecture for Web Analytics by Peyman Mohajerian of Teradata

As integrated web analytics evolves to both a service oriented and event based model, there will be higher emphasis on moving toward event based analytics. Business analytics is moving from purely counts of analytics to time-series, relationship and usage analytics. Examples of web analytics that can take advantage of this architecture are conversions analytics or cross channel marketing.
The advantage of storing raw event data is that you have maximum flexibility for analysis. For example, you can trace the sequence of pages that one person visited over the course of their session. You can’t do that if you’ve squashed all the events into e.g. counters. That sort of analysis is really important for some offline processing tasks, such as training a recommender system (“people who bought X also bought Y”, that sort of thing). For such use cases, it’s best to simply keep all the raw events, so that you can later feed them all into your shiny new machine learning system.
In this session we are going to elaborate on using Kafka, an Event Processing framework (e.g. Storm or Spark Streaming) and either Hadoop or EDW for building an Event Driven Architecture.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Big Data Day LA 2015 - Event Driven Architecture for Web Analytics by Peyman Mohajerian of Teradata

  1. 1. Event Driven Architecture for Web Analytics Peyman Mohajerian June 2015 Big Data Day LA
  2. 2. 2 Why Event Analytics? Understanding the Customer Experience Building Your Business Around Your Customer The Customer Thread Solution Framework Use Cases Teradata Listener Contents © 2014 Teradata2
  3. 3. 3 The questions analytics are intended to answer… © 2014 Teradata Event Analytics answer “Why” and “How” Traditional BI answers…
  4. 4. 4 Mobile Event analytics focuses on how the business looks to the customer © 2014 Teradata Turning the Analytic View of the Customer 180o What segment are you in?When to you last visit? Traditional BI focuses on how the customer looks to the business What did you buy? Why did they make me that offer? Why do they keep sending me emails? How do I make that selection? Why does the agent keep asking for the same information?
  5. 5. 5 © 2014 Teradata Applications Deliver the Company’s Brand and Customer Experience Social Media The Customer Marketing Channels Mobile Apps Devices & Form-factors Entirety of applications combine to deliver the full customer experience Today they are mostly designed in a silo’d manner Applications are not designed to solicit and extract customer experience data well At the core of application design should be the considerations for obtaining and delivering information about the customer experience
  6. 6. 6 © 2014 Teradata The Customer Experience Universe Day 1 Day 3 Day 7 Day 17 Day 21 Day 25 IM Campaign Fragment Email Campaign Fragment Customers Services Fragment PaidSearch LandingPage CreateAccount TXN AttachedCC EmailSent EmailOpened EmailLinkClicked EmailClicked AccountLogin BannerAd1Impression BannerAd2Impression AddBank EmailSent EmailSent TXN AccountLogin HelpCenter EnterDispute C.S.EmailSent EmailOpened EmailLinkClicked HelpCenterHP DisputePage VirtualAgent CallsIntoIVR IVR:DisputeWorkflow TransferredtoAgent DisputeResolved C.S.SurveyEmailed Social Media The Customer Marketing Channels Mobile Apps Devices & Form-factors A universe of customer experience data: • Create threads • Build graphs • Identify patterns
  7. 7. 7 © 2014 Teradata Event Analytics Ecosystem Social Media Email Marketing Display Marketing Website Activity Customer Account Products Transactions Customer Care Event Repository EAP Metadata Dictionary & Library Core Event Dictionary, Library & Data Source Adapters Custom Business Event Dictionary & Library Machine Learning Customer Experience Best Offers Digital Marketing Applications ReportingHigh Speed Query & Reporting APIs Guided UI Driven Analytics Funnel Path Graph Guided UI Funnel & Path Processing Functions Graph Engine & Functions Business Analyst Business Analyst
  8. 8. 8 Event Analytics Ecosystem EAP Metadata Dictionary & Library Core Event Dictionary, Library & Data Source Adapters Custom Business Event Dictionary & Library Event Repository Offers Best Offers Machine Learning A/B Testing Reporting High Speed Query & Reporting APIs Guided UI Driven Analytics Funnel Path Graph Guided UI Funnel & Path Processing Functions Graph Engine & Functions Business Analyst Business Analyst Product, Customer and Transaction Data Mobile Apps Web Site Activity Social Media Display & Search Marketing Customer State eComm Customer Care 3rd Party TrackingBatch Ingest Data Dictionary Event Pattern Matching & Scoring Decisioning Buffer Serve LWIftp Aster Analytic Engine Event Metadata Dictionary Guided UI Funnel Reporting UI Processing Engine Dashboard Engine Dashboard API R-T Events for Decisioning Dashboard API Data WarehouseProduct, Customer, Transaction Event Processing & Event Repository Event Processing Engine HDFS (Time) Event Repository (HBase) Event Repository (Hive) Stream Ingest Spark
  9. 9. 9 • Funnel and pathing analytics are a class of analytics used across the company to analyze user behavior, conversion and product experience • Funnel analytics are complex due to: – Source categorization – Visitor identification – Pathing – Attribution – Conversion • Can be built using a single Guided UI without needing to write-code or move data; allowing analytics to scale at the speed of business © 2014 Teradata Funnel Analytics Use Case Account Transaction Actions
  10. 10. 10 • Frequently Used Aster SQL-MapReduce Functions can be run without knowing SQL. • Forms build dynamically to display necessary parameters based on the analytics being run. • Results can be visualized, published and shared with others for refresh and reuse. © 2014 Teradata Execute Advanced Analytics with Ease 321
  11. 11. 11 • Machine learning is a collection of algorithms to: – Detect hidden patterns in data – Create useful predictions about unseen data – Decision making under uncertainty • Event Repository provides the universe of customer events; a trusted set of events • Machine Learning algorithms can continuously search through the Event Repository looking for complex patterns of interesting behavior; triggering actions © 2014 Teradata Machine Learning on Event Repository Event Repository
  12. 12. 12 © 2014 Teradata Machine Learning Use Cases • Production recommendations • Market Basket Analysis • Event/Activity/Behavioral Analysis • Campaign management and optimization • Market and consumer segmentations Day 1 Day 3 Day 7Day 17 Day 21 Day 25 IM Campaign Fragment Email Campaign Fragment Customers Services Fragment PaidSearch LandingPage CreateAccount TXN AttachedCC EmailSent EmailOpened EmailLinkClicked EmailClicked AccountLogin BannerAd1Impression BannerAd2Impression AddBank EmailSent EmailSent TXN AccountLogin HelpCenter EnterDispute C.S.EmailSent EmailOpened EmailLinkClicked HelpCenterHP DisputePage VirtualAgent CallsIntoIVR IVR:DisputeWorkflow TransferredtoAgent DisputeResolved C.S.SurveyEmailed Time-series Classification Clustering
  13. 13. 13 UNIFIED DATA ARCHITECTURE Security, Workload Management Applications INTEGRATED DATA WAREHOUSE DATA PLATFORM INTEGRATED DISCOVERY PLATFORM Security, Workload ManagementREAL TIME PROCESSING TERADATA PORTFOLIO FOR HADOOP TERADATA DATABASE TERADATA ASTER DATABASE RESTFULAPI LISTENINGFRAMEWORK RESTFULAPI APPFRAMEWORK
  14. 14. Listener Framework
  15. 15. 15 Teradata Listener Common Data Integration Platform for Streaming Data Simplifies data integration across the enterprise Provides a platform for (near) real-time applications Scalable and reliable to support the entire enterprise Open and API based to encourage use Teradata Confid
  16. 16. 16 Listener Data Flow Data flow sequence from sources to target systems 1 6 Multiple sources Write to firehose Read mini-batches INGEST FIREHOSE Write to streams Read mini-batches Write tuples SOURCES ROUTER STREAMS WRITERS SYSTEMS Teradata Confid
  17. 17. 17 INGEST SERVICES Ingesting Data Visualizing the flow through Ingest Services { "uuid":"79f3325f-c75c-4f98-b01e- c4845f69f58c", "source":"6fde3548-65ed-4fa5- 927c-dfc06f1691c6", "data":{ "foo":"bar" }, "time":"2015-01-27T16:17:57Z", "hour":"16", "minute":"17", } { foo":"ba r" } { "uuid":"79f3325f-c75c-4f98-b01e- c4845f69f58c" } Teradata Confid
  18. 18. 18 FIREHOSE Regulating Pressure Distributed write-ahead logging allows bursts of data without impacting systems 1 8 Apache Kafka Resilient and durable; Horizontally scalable; Built for maximum throughput. Asynchronous Reads & W Producers append to the log at their own pace; Consumers read at their own pace; Latest data is always in memory. Teradata Confid
  19. 19. 19 {…}, {…} {…}, {…} {…}, {…} CONSUM E FIREHOSE Routing Data Demuxing data into streams based on rules ROUTER STREAM 1 STREAM … STREAM n Built on Apache Spark Resilient and durable; Horizontally scalable. Rule Based Initially based on API key (per registered data source); Eventually enable combined streams. Teradata Confid
  20. 20. 20 {…}, {…} {…}, {…} Writing Data Write to target systems through Spark jobs {…}, {…} TERADATA WRITER STREAM 1 STREAM … STREAM n ASTER WRITER HDFS WRITER Built on Apache Spark Resilient and durable; Horizontally scalable. Writer Options Initial support for Teradata, Aster and HDFS; Initially leverage JDBC batch writes for Teradata; Exploring rate limiting writers and other systems. Teradata Confid
  21. 21. 2121 © 2014 Teradata
  • calvin0812

    Dec. 9, 2019
  • olaayo

    Nov. 10, 2017
  • ChinaOpen

    Jan. 23, 2017
  • abehere

    Nov. 23, 2016
  • johnzhao6

    Feb. 8, 2016
  • PeerasakWangsom

    Jan. 3, 2016
  • akhendup

    Nov. 11, 2015
  • mail2ramakrsna

    Jul. 29, 2015

As integrated web analytics evolves to both a service oriented and event based model, there will be higher emphasis on moving toward event based analytics. Business analytics is moving from purely counts of analytics to time-series, relationship and usage analytics. Examples of web analytics that can take advantage of this architecture are conversions analytics or cross channel marketing. The advantage of storing raw event data is that you have maximum flexibility for analysis. For example, you can trace the sequence of pages that one person visited over the course of their session. You can’t do that if you’ve squashed all the events into e.g. counters. That sort of analysis is really important for some offline processing tasks, such as training a recommender system (“people who bought X also bought Y”, that sort of thing). For such use cases, it’s best to simply keep all the raw events, so that you can later feed them all into your shiny new machine learning system. In this session we are going to elaborate on using Kafka, an Event Processing framework (e.g. Storm or Spark Streaming) and either Hadoop or EDW for building an Event Driven Architecture.

Views

Total views

1,457

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

0

Shares

0

Comments

0

Likes

8

×