Oracle GoldenGate - Real-time Access to Real-time Information

1. Oracle GoldenGate Real-time Access to Real-time Information Gavin Soorma Senior Principal Consultant, OnCall DBA

3. A private company, founded in 1995 and based in San Francisco, CA

4. Recognized by industry analysts as a leader in real-time data integration

5. Fortune 500 customers across several key industries including Financial Services,Communications, Healthcare, Public Sector, Retail and Utilities

6. Extends Oracle’s data integration capabilities with the fastest, most scalable realtime Heterogeneous data integration solution

7. Another Oracle High Availability solution

10. Complements existing Oracle products

11. Continuous Availability for heterogeneous systems

12. Real-time data access for Reporting & BI/EPM

13. Zero-downtime migrations / upgrades to Oracle Database and Applications

14. Over 500 customers with 4,000+ implementations across Fortune 500 companies: Financial Services, Communications, Healthcare, Public Sector, Retail & Utilities industries

15. Top 3 of 5 largest commercial banks

16. Top 3 of 3 busiest ATM networks

17. Top 7 of 10 financial data services companies

18. Top 4 of 5 telecommunications providers

20. Source– Oracle 10g | Target – MySQL

22. High Performance Extensibility & Flexibility Reliability What is Oracle GoldenGate? Oracle GoldenGate provides low-impact capture, routing, transformation, and delivery of transactional dataacross heterogeneous environments in real-time Why Oracle GoldenGate? Non-intrusive, low-impact, sub-second latency Open, modular architecture - Supports heterogeneous sources and targets Maintains transactional integrity - Resilient against interruptions and failures 6

23. Oracle GoldenGate Heterogeneity Supports Applications running on ….

25. GoldenGate Management Pack

26. Golden Gate Veridata

27. Available for download on Oracle E-Delivery web site.

29. Oracle GoldenGate Topologies Bi-Directional Live Standby or Active-Active for HA Peer-to-Peer Load Balancing, Multi-Master Unidirectional Query Offloading Broadcast Data Distribution Integration/Consolidation Data Warehouse Cascading Data Marts

30. Integration with Data Warehousing Integration via: Staging tables Flat files Messaging

32. Capture – Extract process

33. Delivery – Replicat process

34. Data Pump

35. Trail files

36. Checkpoints

37. Data definition files

40. One manager per GoldenGate installation

41. Supports multiple Extract and Replicat processes

42. Starts other GoldenGate processes

43. Performs trail file management – purge old extracts

45. Capture mechanism of GoldenGate

46. Configure extract for

47. Initial Data Load

48. Online Change Synchronization

50. Optional

51. When not used, Extract sends data over network to remote trail on destination

52. When used, Extract process writes to local trail. Data Pump reads this trail and writes to the remote trail

53. Data pump isolates the primary extract process from TCP/IP activity

54. Performs data filtering, mapping and conversion

56. Reads and extracts data changes (and DDL changes when configured)

57. Configured for initial load or online change synchronization.

58. Multiple replicat processes to increase throughput

60. Can exist on the source (local trail), target (remote trail) or on intermediary systems

61. By default 10 MB in size

62. Files are named at configuration time with a two letter identifier and are appended with an unique 6 digit serial number

63. GoldenGate writes to trail files in Universal Data Format – a proprietary format

64. This proprietary format enables the heterogeneous data integration

65. Checkpoints provide known position in trail from where to start in case of unexpected failure

68. Support for ASM

69. Support for index-organized tables (IOTs) with overflow

70. Support for clustered tables

71. Support for object tables

72. Support for object types (UDTs)

73. Support for DDL operations

74. Archived log only (ALO) mode

75. Off-platform capture (LOGSOURCE)

76. Multi-threaded captureCapture Online Log Archived Log OCI/Oracle Net

78. Coexistence with SQL Server Replication

79. SQL Server Replication components are not required

80. Support tables with no unique key (not supported by SQL Server Replication)

81. Support for computed columns

82. Support for identity columnsCapture Transaction Log Backups ODBC

84. Database Migrations (e.g. DB2 to Oracle)

85. Database Server Upgrades

86. Database Maintenance (e.g. de-fragmentation, table re-orgs, de-chaining rows, etc.)

88. Minimal impact and continuous availability during the project

89. Technology solution for eliminating database downtime

90. Improved capabilities for data integrity, accuracy

91. Fail-back solution in the event of unexpected issues/downtime

93. Cross Platform Migration 1 2 3,4 5 6,7 9i Clone  10g Solaris 9i Solaris 10g Linux Start GoldenGate TDM’s Capture process Set up Clone database, then Upgrade to 10g Cross platform transportable tablespaces metadata export Use a full database NOROWS export (Views, Packages, etc) Set up a new 10g vanilla target Cross platform transportable tablespaces metadata import Full import with IGNORE option

94. Cross Platform Migration 1 2 5 8 3,4 6,7 9i Clone  10g Solaris 9i Solaris 10g Linux Start GoldenGate TDM’s Capture process Set up Clone database, then Upgrade to 10g Cross platform transportable tablespaces metadata export Use a full database NOROWS export (Views, Packages, etc) Set up a new 10g vanilla target Cross platform transportable tablespaces metadata import Full import with IGNORE option Start GoldenGate TDM Apply process at target **SWITCHOVER** (not depicted)

95. Cross Platform Migration 1 5 8 10 11 2 3,4 6,7 9i Clone  10g Solaris 9i Solaris Failback contingency 10g Linux Start GoldenGate TDM’s Capture process Set up Clone database, then Upgrade to 10g Cross platform transportable tablespaces metadata export Use a full database NOROWS export (Views, Packages, etc) Set up a new 10g vanilla target Cross platform transportable tablespaces metadata import Full import with IGNORE option Start GoldenGate TDM Apply process at target Verify data (optional) **SWITCHOVER** (not depicted) Start TDM Capture at target (new source) Start TDM Apply at target (old source)

97. GoldenGate Director Client

99. Identifies and reports on data discrepancies between heterogeneous databases with minimal impact on infrastructure

100. Companies can audit and verify large volumes of data across different business applications with certainty

102. GoldenGate Veridata: How it Works The user chooses tables or files on the source and target databases The comparison is initiated from the GUI, command line or batch As the databases continue to change, GoldenGate Veridata reports: Persistent discrepancies In-flight data discrepancies (user configurable)

103. GoldenGate Director

104. GoldenGate Director

105. GoldenGate Director Web

106. GoldenGate Demo

107. Create the Manager, Extract and Replicat Process ./ggsci GGSCI (linux01.oncalldba.com) 1> edit params mgr PORT 7809 USERID ggs_owner, PASSWORD ggs_owner Source – Extract Process GGSCI (linux01.oncalldba.com) 1> ADD EXTRACT ext1, TRANLOG, BEGIN NOW EXTRACT added. GGSCI (linux01.oncalldba.com) 2> ADD RMTTRAIL /home/oracle/goldengate/dirdat/rt, EXTRACT ext1 RMTTRAIL added. Target – Replicat Process GGSCI (linux03.oncalldba.com) 1> ADD REPLICAT rep1, EXTTRAIL /home/oracle/goldengate/dirdat/rt REPLICAT added.

108. Start Manager (Source) [oracle@linux01 goldengate]$ ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 10.4.0.19 Build 002 Linux, x86, 32bit (optimized), Oracle 10 on Sep 17 2009 23:49:42 Copyright (C) 1995, 2009, Oracle and/or its affiliates. All rights reserved. GGSCI (linux01.oncalldba.com) 1> start manager Manager started. GGSCI (linux01.oncalldba.com) 2> info all Program Status Group Lag Time Since Chkpt MANAGER RUNNING

109. Start Manager (Target) [oracle@linux03 goldengate]$ ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 10.4.0.19 Build 002 Linux, x86, 32bit (optimized), Oracle 11 on Sep 29 2009 08:50:50 Copyright (C) 1995, 2009, Oracle and/or its affiliates. All rights reserved. GGSCI (linux03.oncalldba.com) 1> start manager Manager started. GGSCI (linux03.oncalldba.com) 2> info all Program Status Group Lag Time Since Chkpt MANAGER RUNNING

110. Source – Start Extract Process GGSCI (linux01.oncalldba.com) 4> view params ext1 EXTRACT ext1 USERID ggs_owner@oncalldb.world,passwordggs_owner RMTHOST linux03, MGRPORT 7809 RMTTRAIL /home/oracle/goldengate/dirdat/rt DDL INCLUDE ALL TABLE ggs_owner.myobjects; ; GGSCI (linux01.oncalldba.com) 5> start extract ext1 Sending START request to MANAGER ... EXTRACT EXT1 starting GGSCI (linux01.oncalldba.com) 6> info extract ext1 EXTRACT EXT1 Last Started 2010-08-13 13:12 Status RUNNING Checkpoint Lag 03:00:32 (updated 00:00:00 ago) Log Read Checkpoint Oracle Redo Logs 2010-08-13 10:12:23 Seqno 4, RBA 19823616

111. Target – Start Replicat Process GGSCI (linux03.oncalldba.com) 3> view params rep1 REPLICAT rep1 ASSUMETARGETDEFS USERID ggs_owner, PASSWORD ggs_owner discardfile /home/oracle/goldengate/dirout/discard.txt,append DDL INCLUDE MAPPED MAP ggs_owner.myobjects, TARGET ggs_owner.myobjects; GGSCI (linux03.oncalldba.com) 4> start replicat rep1 Sending START request to MANAGER ... REPLICAT REP1 starting GGSCI (linux03.oncalldba.com) 5> info replicat rep1 REPLICAT REP1 Last Started 2010-08-13 13:15 Status RUNNING Checkpoint Lag 00:00:00 (updated 00:00:04 ago) Log Read Checkpoint File /home/oracle/goldengate/dirdat/rt000010 2010-08-13 13:12:55.105247 RBA 1030

112. Test DDL Replication Create a table and index on source database [oracle@linux01 ~]$ sqlplusggs_owner/ggs_owner SQL*Plus: Release 10.2.0.4.0 - Production on Fri Aug 13 13:09:38 2010 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL> create table myobjects as select * from dba_objects where 1=2; Table created. SQL> alter table myobjects add constraint pk_myobjects 2 primary key (object_id); Table altered.

113. Test DDL Replication On Target database MYOBJECTS table and index created SQL> descmyobjects Name Null? Type ----------------------------------------- -------- ---------------------------- OWNER VARCHAR2(30) OBJECT_NAME VARCHAR2(128) SUBOBJECT_NAME VARCHAR2(30) OBJECT_ID NOT NULL NUMBER DATA_OBJECT_ID NUMBER OBJECT_TYPE VARCHAR2(19) CREATED DATE LAST_DDL_TIME DATE TIMESTAMP VARCHAR2(19) STATUS VARCHAR2(7) TEMPORARY VARCHAR2(1) SQL> select index_name from user_indexes where table_name='MYOBJECTS'; INDEX_NAME ------------------------------ PK_MYOBJECTS

114. Test Online Change Synchronization INSERT rows on source tables SQL> !date Fri Aug 13 13:24:14 WST 2010 SQL> insert into myobjects select * from dba_objects where owner='SYSMAN'; 1341 rows created. SQL> commit; Commit complete. SQL> !date Fri Aug 13 13:27:01 WST 2010 SQL> insert into myobjects select * from dba_objects where owner='SYS'; 22937 rows created. SQL> commit; Commit complete.

115. Test Online Change Synchronization SQL> !date Fri Aug 13 13:24:44 WST 2010 SQL> select count(*) from myobjects; COUNT(*) ---------- 1341 SQL> !date Fri Aug 13 13:27:14 WST 2010 SQL> select count(*) from myobjects; COUNT(*) ---------- 24278 Check target tables to see that all the rows have been inserted

116. Monitor Replicat Progress GGSCI (linux03.oncalldba.com) 8> stats replicat rep1 Start of Statistics at 2010-08-13 13:35:39. DDL replication statistics: *** Total statistics since replicat started *** Operations 2.00 Mapped operations 2.00 Unmapped operations 0.00 Other operations 0.00 Excluded operations 0.00 Errors 0.00 Retried errors 0.00 Discarded errors 0.00 Ignored errors 0.00 Replicating from GGS_OWNER.MYOBJECTS to GGS_OWNER.MYOBJECTS: *** Total statistics since 2010-08-13 13:24:33 *** Total inserts 24278.00 Total updates 0.00 Total deletes 0.00 Total discards 0.00 Total operations 24278.00

117. Cleanup – some more DDL Replication SOURCE SQL> truncate table myobjects; Table truncated. TARGET SQL> select count(*) from myobjects; COUNT(*) ---------- 0 SOURCE SQL> drop table myobjects; Table dropped. TARGET SQL> select count(*) from myobjects; select count(*) from myobjects * ERROR at line 1: ORA-00942: table or view does not exist

118. Q & A gavin.soorma@oncalldba.com.au http://www.gavinsoorma.com http://www.oncalldba.com.au