Migration to Oracle Multitenant

Upgrade to Oracle Database 19c
Virtual Classroom Series
Mike Dietrich
Master Product Manager
Database Upgrade
and MigrationsMike Dietrich
Database Upgrade and Migration
Roy Swonger
Vice President
Database Upgrade,
Utilities and Patching

Vice President
Database Upgrade,
Utilities & Patching
Roy F. Swonger
@RoyFSwonger

Database Upgrade
and Migrations
Mike Dietrich
https://MikeDietrichDE.com
@MikeDietrichDE
mikedietrich

Slides | https://MikeDietrichDE.com

Content Overview
Migration to
Multitenant
4
Ensure
Performance Stability
3
Upgrade to Oracle
Database 19c
2Release and
Patching Strategy
1

AutoUpgrade to Oracle 19c
Migration to
Oracle Multitenant
PhotobyMathiasJensenonUnsplash

Oracle Multitenant | Oracle 19c
• Oracle 18c and Oracle 19c belong to the Oracle Database 12.2 release family
• For further details see MOS Note: 742060.1

And by the way …
Since Oracle 19c you can have 3 user-created PDBs without need for a Multitenant license!!!

Download | Always download the latest autoupgrade.jar
MOS Note: 2485457.1

Upgrade | AutoUpgrade and Plugin
upg>
Job Console
11.2.0.4
12.1.0.2
12.2.0.1
18c
autoupgrade.jar
MOS Note: 2485457.1
2MB
config.cfg
autoupgrade.jar
12.2.0.1
18.5.0
19c

Upgrade to 19c | Migration to Multitenant
AutoUpgrade One-Command Orchestration
Oracle 11.2.0.4
AutoUpgrade
Oracle 19.7.0
Oracle 19.7.0
CDB
PDB$
SEED
Oracle 19.7.0
CDB
PDB$
SEED
1️⃣
2️⃣ Plug into CDB
3️⃣
Conversion with
noncdb_to_pdb.sql

Upgrade | AutoUpgrade Demo Video
https://mikedietrichde.com/2020/05/20/autoupgrade-and-plug-in-to-a-cdb-with-a-single-command/

AutoUpgrade | Migration to Multitenant
• Use these parameters in your config file
• The receiving CDB must be precreated already
# Specify the SID of the target CDB to plug into after upgrade
# Mandatory if you'd like to plugin
upg2.target_cdb=CDB2
# Give the non-CDB a new name when it becomes a PDB
# Optional
upg2.target_pdb_name=PDB1
# Define whether you'd like to use the COPY option
# Optional - if not specified, NOCOPY will be used
upg2.target_pdb_copy_option=file_name_convert('ORCL','PDB1')

Oracle Multitenant | Deployments 19c
• Non-CDB
• No change to
previous releases
• CDB: Three Tenants
• No extra license
• Available with SE2 / EE
• Three user-created
PDBs included
• SPFILE: max_pdbs=3
• CDB: Multitenant
• Multitenant
license required
• Available with EE only
• Up to 252 PDBs
• Up to 4096 PDBs in the Cloud
and on Engineered Systems
PDB$
SEED
PDB$
SEED
…

Oracle Multitenant | Concept
CDB1
PDB$
SEED
CDB2
PDB$
SEED
CDB3
PDB$
SEED
PDB$
SEED

Oracle Multitenant | Resource Sharing
Redo Control Flashback
PDB$
SEED
spfile

Oracle Multitenant | Connecting
CDB$ROOT
• $> sqlplus / as sysdba
PDB
• $> sqlplus "sys/oracle@PDB1 as sysdba"
• $> sqlplus "sys/oracle@//localhost/pdb1 as sysdba"
• $> export TWO_TASK=pdb1
$> sqlplus "sys/oracle as sysdba"
Switch between containers
• SQL> alter session set container=pdb1;
• SQL> alter session set container=cdb$root;
PDB$
SEED
CDB
PDB$
SEED
CDB
PDB$
SEED
CDB

Oracle Multitenant | Connecting - New since Oracle 18.5.0
• PDB
• $> export ORACLE_PDB_SID=pdb1
$> sqlplus / as sysdba
PDB$
SEED
CDB

Oracle Multitenant | Resource Manager
Memory Management
• SGA_MIN_SIZE - minimum SGA size
IO Resource Management [IORM]
• MAX_IOPS - maximum I/O requests per second
• MAX_MBPS - maximum MB per second of I/O
• Can't be set in the CDB$ROOT and on Exadata
• Can be dynamically altered
CPU Consumption
• CPU_COUNT - number of CPU cores
Can be set
on PDB level

Oracle Multitenant | Components/Options
Create CDBs with fewer options: DBCA's "CUSTOM" mode creation
• https://mikedietrichde.com/2018/08/08/creating-cdbs-non-cdbs-with-less-options/
• https://mikedietrichde.com/2017/07/11/always-create-custom-database/

Only "Advanced Configuration" + "Custom Database" gives you:
Be really aware!
Be aware!

Remove or add options
• https://mikedietrichde.com/2017/07/26/remove-clean-components-oracle-11-2-12-2/

Oracle Multitenant | catcon.pl
Execute scripts with catcon.pl
• MOS Note: 1932340.1 - How to execute sql scripts in Multitenant environment (catcon.pl)
• Example: Run preupgrade_fixups.sql in all containers
$> $ORACLE_HOME/perl/bin/perl
$ORACLE_HOME/rdbms/admin/catcon.pl
-n 1
-d $ORACLE_BASE/cfgtoollogs/UPGR/preupgrade
-l /home/oracle/upgrade
-b preupfix
preupgrade_fixups.sql

Oracle Multitenant | _ORACLE_SCRIPT
Parameter _ORACLE_SCRIPT is used to allow actions in PDBs and especially in PDB$SEED
• Don't use _ORACLE_SCRIPT=TRUE by yourself, at least not as a standard
- https://mikedietrichde.com/2020/02/10/be-aware-when-you-use-_oracle_script-in-scripts/
- MOS Note:2378735.1 – “_ORACLE_SCRIPT”=TRUE PARAMETER Should not be Invoked by Users
• Example:
- alter session set "_ORACLE_SCRIPT"=TRUE;
- create table hugo.test01(col1 number);
- select object_name, object_type, ORACLE_MAINTAINED from DBA_OBJECTS where
object_name='TEST01;
- OBJECT_NAME OBJECT_TYPE ORACLE_MAINTAINED
------------- -------------- -------------------
TEST01 TABLE Y

Oracle Multitenant | Local Undo
Local Undo is default since Oracle 12.2.0.1
• Requirement for key features such as Flashback Pluggable Database, Hot Cloning, and more
• In STARTUP UPGRADE mode: SQL> alter database local undo ON;
PDB$
SEED
CDB
SYSTEM
SYSAUX
TEMP
UNDO
SYSTEM
SYSAUX
TEMP
UNDO
DATA SYSTEM
SYSAUX
TEMP
UNDO
DATA

Oracle Multitenant | Creation of a PDB
Fast provisioning:
• Admin user can be deleted afterwards
• PDB is ready to use within seconds
PDB$
SEED
CDB1
create pluggable database PDB1
admin user adm identified by pwd
file_name_convert=('pdbseed',’pdb1');
PDB$
SEED
Tip:
You can specify just the path difference in the
file_name_convert string.
Instead of:
=('/data/CDB1/pdbseed','/data/CDB1/pdb1')
you can shorten this to;
=('pdbseed','pdb1')
as the rest of the expression is identical.

Oracle Multitenant | Cloning of a PDB
Local cloning:
• parallel can speed up the process
• Default: cpu_count
• Monitoring in
v$session_longops
• kpdbfCopyTaskCbk (datafile copy)
• kcrfremnoc (redo file copy)
Remote cloning via Database Link:
create pluggable database PDB2 from
PDB1 file_name_convert=('pdb1','pdb2')
parallel 8;
create pluggable database PDB1 from
PDB1@dblink_to_cdb1
file_name_convert=('CDB1','CDB2');
PDB$
SEED
CDB1
PDB$
SEED
CDB1
PDB$
SEED
CDB2

Oracle Multitenant | Hot Cloning insights
P P P P P
P P P P
P P P
PP P
DataFile2DataFile1 Undo
• Source PDB remains open for read and write
• Read and copy in parallel
• On-going operations imply a “dirty read”
• Some data changes not included in initial file copy
• Ship and apply redo to catch up with source
• Apply undo to rollback uncommitted transactions
DataFile1 DataFile2Undo
Redo Log
P P P P P
P P P P
P P P
PP P
P P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P P
P
P P
P
P
P
P
P
P P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
P
Legend
Uncommitted block
changed in interval
Block changed in interval
Committed in interval
Uncommitted redo
Undo written in interval
Committed in interval
P
P

Oracle Multitenant | Relocate a PDB
create pluggable database PDB1 from PDB1@dblink_to_cdb1
relocate availability max file_name_convert=('CDB1','CDB2');
PDB$
SEED
CDB1
PDB$
SEED
CDB2
Redo
Redo

Oracle Multitenant | Upgrade and Plugin
Upgrade non-CDB to CDB version
• Start database read only
• Create XML manifest file
• Shutdown database
• Plug into CDB
• Sanity script
• https://mikedietrichde.com/2017/03/08/converting-an-12-1-non-cdb-and-plug-it-into-an-12-2-cdb/
PDB$
SEED
CDB1
exec DBMS_PDB.DESCRIBE('PDB1.xml');
create pluggable database PDB1
using ('PDB1.xml') nocopy tempfile reuse;
start ?/rdbms/admin/noncdb_to_pdb.sql
noncdb_to_pdb.sql
Read Only
PDB1
xml

Oracle Multitenant | noncdb_to_pdb.sql
Sanity script when plugging in a stand-alone database
• Irreversible
• Runs only once in the life of a database
• Rerunnable starting with 12.2
• Runtime depends ...
• One-off patch 25809128 may decrease downtime significantly for non-Oracle-owned objects
- Most likely included in 19.9.0 (RU Oct 2020)
• See also:
https://mikedietrichde.com/2017/06/08/noncdb_to_pdb-sql-take-long/

Oracle Multitenant | Full Transportable Export/Import
• Create a new PDB
• Create a database link
• Set tablespaces read only in source
• Copy data files
• Run impdp:
PDB$
SEED
CDB1
Tablespaces
Read Only
impdp
impdp oow/passwd@PDB1
NETWORK_LINK=DB1 VERSION=12 FULL=Y
TRANSPORTABLE=ALWAYS METRICS=Y
LOGFILE=oow_dir:src112fullimp.log
TRANSPORT_DATAFILES='/oradata/ts1.dbf' …
DatabaseLink

Oracle Multitenant | Upgrade
Everything at once Unplug / Plug / Upgrade
PDB$
SEED
CDB1
PDB$
SEED
CDB1
Upgrade
Upgrade
Upgrade
Upgrade
PDB$
SEED
CDB1
PDB$
SEED
CDB2
Upgrade

Oracle Multitenant | PDB Archive
Unplug the PDB including all files
• Issue with PDB archives in Oracle 12.2.0.1:
https://mikedietrichde.com/2017/05/22/issue-with-pdb-archives-in-oracle-12-2-0-1-in-asm/
ALTER PLUGGABLE DATABASE pdb1
UNPLUG INTO '/home/oracle/pdb1.pdb'
$ unzip pdb1.pdb
Archive: pdb1.pdb
inflating: system01.dbf
inflating: sysaux01.dbf
inflating: undotbs01.dbf
warning: stripped absolute path spec from /home/oracle/pdb1.xml
inflating: home/oracle/pdb1.xml

Oracle Multitenant | Save State
You need to startup a PDB
• ALTER PLUGGABLE DATABASE pdb1 OPEN;
By default PDBs need to be started manually
• ALTER PLUGGABLE DATABASE pdb1 SAVE STATE;
- This preserves the last state of a PDB
• ALTER PLUGGABLE DATABASE pdb1 DISCARD STATE;
- This removes any state preservation
• This is not necessary in a cluster environment!

Oracle Multitenant | CON_ID
Views:
• USER_...
• ALL_...
• DBA_...
• CDB_... - CDB_views have CON_ID column
• Plus some very creative names: PDB_...
CON_ID
• 0: Pertains to the entire database
• 1: Object belongs to CDB$ROOT container
• 2: Object belongs to PDB$SEED
Set _exclude_seed_cdb_view=FALSE to see PDB$SEED's objects
• 3 - n: Object belongs to PDB with this CON_ID

Oracle Multitenant | Character Sets
Since Oracle 12.2.0.1:
• Mix of different character sets is possible
• CDB$ROOT must be AL32UTF8 to allow mixtures

Oracle Multitenant | Flashback PDB
Requirement: Local Undo must be configured
• How to?
• SQL> alter session set container=PDB1;
• SQL> create restore point GRP guarantee flashback database;
• <…do something …>
• SQL> shutdown immediate
• SQL> flashback pluggable database PDB1 to restore point GRP;
• SQL> alter pluggable database PDB1 open resetlogs; *
• SQL> drop restore point GRP;

Oracle Multitenant | Lockdown Profiles
CDB
• SQL> alter session set container=CDB$ROOT;
• SQL> create lockdown profile P1;
• SQL> alter lockdown profile P1 disable
statement=('ALTER SYSTEM') clause=('SET')
OPTION ALL EXCEPT=('optimizer_mode','cursor_sharing');
• SQL> alter system set PDB_LOCKDOWN=P1;
PDB
• SQL> alter session set container=PDB2;
• SQL> alter system set sql_trace=TRUE;
*
ERROR at line 1:
ORA-01031: insufficient privileges
• SQL> alter system set cursor_sharing='FORCE';
System altered.

Multitenant | Silent Compatible Change
Be aware when you unplug/plug between
different environments
• Default in Oracle 19c: COMPATIBLE=19.0.0
• PDB's COMPATIBLE = CDB$ROOT's COMPATIBLE
→ No issue
• PDB's COMPATIBLE < CDB$ROOT's COMPATIBLE
→ PDB's COMPATIBLE will be increased automatically
→ No PDB downgrade will be possible in this case!
• PDB's COMPATIBLE > CDB$ROOT's COMPATIBLE
→ PDB can't be plugged in
PDB$
SEED
CDB1
PDB$
SEED
CDB2
COMPATIBLE=12.2.0
COMPATIBLE=19.0.0
COMPATIBLE=19.0.0
Automatically

Oracle Multitenant | Downgrade
Downgrade works for CDB/PDB entirely as well as for single/multiple PDBs
• Manual tasks
• catdwgrd.sql in current (after upgrade) environment
• catrelod.sql in previous (before upgrade) environment
• Don't change COMPATIBLE
• datapatch must roll back SPUs/PSUs/BPs manually
• MOS Note: 2172185.1
How to Downgrade a Single Pluggable Oracle Database ( PDB ) to previous release

Migration Strategies
Migration to and Fallback
with Oracle Multitenant
PhotobyJackmanChiuonUnsplash

Move to Multitenant | High Level Overview
Two approaches
• Upgrade to matching version of CDB, then plugin as new PDB, then run sanity script
• Migrate directly into PDB with Data Pump, TTS or FTEX
Every approach requires unpredictable, often significant downtime
• Can be reduced with Oracle GoldenGate
Proven upgrade fallback strategies don't work
• Export/import with Data Pump necessary as fallback
• Oracle GoldenGate can be used on top

Move to Multitenant | Typical Pitfalls
Summary on the Upgrade Blog
https://mikedietrichde.com/2019/07/29/database-migration-from-non-cdb-to-pdb-typical-plugin-issues-and-workarounds/
• DB components (e.g. Spatial, Label Security, Java etc.) not matching
• Patch levels
• noncdb_to_pdb.sql requires downtime
• Some features require attention
- Unified Auditing
- Standby implementation, especially in ASM
- Performance monitoring, e.g. AWR
• Typical Plugin Issues and Workarounds
• The Compatible Pitfall
• The Time Zone Pitfall
• The Component Pitfall
• The Patch Level Pitfall
• Various Pitfalls
• The Fallback Challenge
• The Minimal Downtime Challenge

Move to Multitenant | Your "best" friend
PDB_PLUG_IN_VIOLATIONS
• select con_id, type, message, status from PDB_PLUG_IN_VIOLATIONS
where status<>'RESOLVED' order by time;

Non-CDB to CDB | Migrate with any technique
Includes Data Pump, Transportable Tablespaces, FTEX etc.
PDB$
SEED
CDB
Oracle 11.2.0.3
Oracle 19.7.0
2⃣ Migrate
Provision
new PDB1⃣

Move to Multitenant | TTS Migration
Views
Code
Grants
Trigger
DATA
TS
Alter User
Create User
Set tablespaces read-only
Copy tablespace files DATA
TS
Export/import tablespace content info
Set tablespaces read-write
Rebuild logic (code, views, grants, roles, synonyms etc)
Views
Code
Grants
Trigger
Create new CDB/PDB
PDB$
SEED
PDB
Source - non-CDB Target - PDB

Non-CDB to CDB | Data Pump with Dumpfile
Oracle 11.2.0.4
Provision
new PDB
Oracle 19.7.0
CDB
PDB$
SEED
1⃣
2⃣ expdp
DUMP FILE DUMP FILE
3⃣ impdp

Non-CDB to CDB | Data Pump over NETWORK_LINK
Oracle 11.2.0.4
Provision
new PDB
Oracle 19.7.0
CDB
PDB$
SEED
1⃣
2⃣ Create DB Link
3⃣
impdp with
NETWORK_LINK

Non-CDB to CDB | Fallback: Data Pump w. dumpfile
Oracle 11.2.0.4
expdp with
VERSION=11.2.0.4
Oracle 19.7.0
CDB
PDB$
SEED
1⃣2⃣ impdp
DUMP FILE DUMP FILE

Non-CDB to CDB | Upgrade
Oracle 11.2.0.4
AutoUpgrade
toOracle19c
Oracle 19.7.0
Oracle 19.7.0
CDB
PDB$
SEED
Oracle 19.7.0
CDB
PDB$
SEED
Plug into CDB
noncdb_to_pdb.sql
3
2
1

Non-CDB to CDB | Upgrade - Fallback
3
2
1
Oracle 11.2.0.4
AutoUpgrade
toOracle19c
Oracle 19.7.0
Oracle 19.7.0
CDB
PDB$
SEED
Oracle 19.7.0
CDB
PDB$
SEED
Plug into CDB
noncdb_to_pdb.sql
Flashback
to GRP

Non-CDB to CDB | Upgrade – Fallback: Alternative?
3
2
1
Oracle 12.1.0.2
AutoUpgrade
toOracle19c
Oracle 19.7.0
COMPATIBLE=12.1.0
Oracle 19.7.0
COMPATIBLE=12.1.0
CDB
PDB$
SEED
Oracle 19.7.0
COMPATIBLE=12.1.0
CDB
PDB$
SEED
Plug into CDB
noncdb_to_pdb.sql
Oracle 12.1.0.2
COMPATIBLE=12.1.0
CDB
PDB$
SEED
???
Downgrade /
Unplug /
Plugin
4⃣

Move to Multitenant | Fallback for upgrades?
Move to Multitenant is always a "Migration"
• Usual fallback techniques don't exist
• No downgrade
• No flashback to GRP
• Only possible fallback options
• Data Pump export/import
• Oracle GoldenGate

A typical example from the Real World
Migrating to Oracle Multitenant with a
distant physical standby database in place

Plug in your Standby | Initial Setup
Oracle 12.1.0.2
PROD
10 TB size
Oracle 12.1.0.2
STBDY
10 TB size
100 km distance Maximum Protection

Plug in your Standby | AutoUpgrade
Oracle 12.1.0.2
PROD
Oracle 12.1.0.02
STBDY
Disable Broker - Defer Log Transport
Oracle 19.7.0
PROD
>> DOWNTIME <<

Plug in your Standby | Re-Establish
Oracle 19.7.0
PROD
Oracle 12.1.0.2
STBDY
Enable Log Transport
Enable Broker
Oracle 19.7.0
PROD

Plug in your Standby | Implicit Standby Upgrade
Oracle 19.7.0
PROD
Oracle 12.1.0.2
STBDY
Upgrade
via Redo
Apply
Oracle 19.7.0
STBDY

Plug in your Standby | Create CDBs
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED
standby_pdb_source_file_dblink

Plug in your Standby | Read Only Phase
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED
PDB1
xml
Read Only

Plug in your Standby | ASM Alias File
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED
PDB1
xml
Read Only
ASM
Alias
List
Execute in standby ASM instance

Plug in your Standby | Plugin Production
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED
PDB1
xml
Read Only

Plug in your Standby | Conversion to PDB
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED
noncdb_to_pdb.sql
(runtime varies)

Plug in your Standby | Plugin Production Completed
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED

Plug in your Standby | Plugin the Standby
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED

Plug in your Standby | Mission Accomplished!
Oracle 19.7.0
PROD
Oracle 19.7.0
STBDY
CDB
PDB$
SEED
CDB
PDB$
SEED

Non-CDB to CDB with Standby | Plugin of STBY
Read more …
• Data Guard PM - Pieter van Puymbroeck:
https://vanpupi.stepi.net/2019/06/to-cdb-or-not-to-cdb-thats-the-question/
• MAA Team:
MOS Note: 2273304.1
Reusing the Source Standby Database Files When Plugging a non-CDB as a PDB into the Primary Database of a
Data Guard Configuration

Oracle Multitenant | Standby Databases
• Only on per-CDB basis
• PDBs can be excluded [disrecommended]
• standby_pdb_source_file_dblink
• Used to replicate remote cloning operations to the standby
• Set it to the DB-Link used for remote cloning
• standby_pdb_source_file_directory
• Used to replicate plugin operations to the standby
• Set it to the directory where the files of the PDB are located
• MOS Note: 2274735.1
Using standby_pdb_source_file_dblink and standby_pdb_source_file_directory to maintain standby
databases when performing PDB remote clones or plugins

Move to Multitenant | Summary
• Every migration is an architectural change
• Every migration requires downtime
• Incremental backup procedures can
lower migration downtime
• Oracle GoldenGate can help decrease
the downtime
PhotobyKSKYUNGonUnsplash

Move to Multitenant | Refreshable PDBs for Upgrade Testing
Oracle 12.2.0.1
CDB
PDB$
SEED
Oracle 19.7.0
CDB
PDB$
SEED
Clone & Refresh
12.2.0.1
Oracle 19.7.0
CDB
PDB$
SEED
Refresh
12.2.0.1
Oracle 19.7.0
CDB
PDB$
SEED
Upgrade PDB
19.7.0

Move to Multitenant | It's powerful
• Use Refreshable PDBs for upgrade testing
• https://mikedietrichde.com/2020/05/05/upgrade
-testing-with-a-refreshable-pdb-does-this-work/
• You determine the synchronization point

https://mikedietrichde.com/2019/07/22/database-migration-from-non-cdb-to-pdb-overview/

Getting ready
for the future
Upgrade / Migrate and
Consolidate to Multitenant
at La Mobilière, Switzerland

Customer Case | La Mobilière
• Swiss Mobiliar
• Switzerland's most personal insurer
• Founded 1826 in Bern, oldest Swiss insurance
• Legal form:
• Cooperative association (mutual company
Customer
Project
Constraints
Preparation
Upgrade
Success?
Remarks

Customer
Project
Constraints
Preparation
Upgrade
Success?
Remarks

• Upgrade 337 databases
• Oracle Database 12.1.0.2 to Oracle Database 12.2.0.1
• 82 production databases
• 18 container databases
• 350 PDB’s
• Max of 50 PDB’s in one CDB in dev
• Move from schema-based consolidation to PDBs
• PDB-only architecture with Oracle 12.2
• Except 3rd party app restrictions
Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks

Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
• Motivation
• Developers want Oracle 12.2 features
• Cost savings with Multitenant
• Reduce admin costs by automation

Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
• Regression tests
• Done during the testing phase of the Mobiliar Software Release
• Database RELEASE UPDATE (RU): 12.2.0.1.170718
• Dual Oracle Home strategy
• Upgrade to Oracle Database 12.2
• With catctl.pl embedded into home-built shell script
• Performance tests performed by application owner

Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
Performance
Warehouse
• Performance tracking with
Mobiliar’s own AWR Warehouse
• Compare performance before/after
• 7 TB of historical performance data
• Covers entire database lifetime

Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks
Upgrade Steps (parallel degree: -n 32) Duration
Execution of preupgrade.jar
2h 46 min
Execution of pre upgrade fixup scripts
All in one Upgrade of CDB$ROOT and all PDB’s
Recompile of CDB$ROOT and all PDB’s after upgrade to 12.2
Enable local undo mode for container database
Immediate create level 0 backup of container database
Create guarantee restore point and change oracle home to 12.2
Drop guarantee restore point after successful upgrade
PRE TASKS (online)
Upgrade TASKS (offline)
POST TASKS (online)
5 min
32 min
6 min
8 min
13 min
5 min
3h 29 min
10 min
1 min
11 min
Total Upgrade Time 3h 53 min
50PDBs-Upgradeinonepass

• Parallel upgrade catctl.pl unfolds its full power when upgrading many
PDBs at the same time
• 50 PDBs upgraded in less than 4 hours
• We feel very well prepared for the annual Oracle releases
• When we encounter issues, we fix them before going live
• Follow their projects on: https://mobiliar.ch/db-blog
• 100% Multitenant Consolidation reached in Oct 1, 2019
Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks

• High increase in number of PDBs, often driven by Microservices
• Self-developed DBaaS interface to privision, alter and drop PDBs
• Used to create >1000 PDBs
Customer
Project 2017
Constraints
Preparation
Upgrade
Success?
Remarks

Migrating to the
Future
Exadata Migration

• Move many PDBs to a new Exadata
• 2 Exadata X8
• 3 DB Nodes each
Customer
Project 2019
Constraints
Preparation
Migration
Success?
Remarks

• Downtime <24 hours
Customer
Project 2019
Constraints
Preparation
Migration
Success?
Remarks

Customer
Project 2020
Constraints
Preparation
Migration
Success?
Remarks
Move to Exadata
Pre-Actions
• Lock the app user on source PDB
• Deactivate the app service on the PDB
• Create DB Link for remote clone
• Remove PDB from Cloud Control
Post actions
• Perform datapatch - newer RU
• Perform backup of each PDB
• Unlock the app user on target PDB
• Create the new app service
• Update Cloud Control with new PDB
• Delete Clone DB Link

Customer
Project 2020
Constraints
Preparation
Migration
Success?
Remarks
• Move to Exadata
• Complete script automation
• Including error handling

Customer
Project 2020
Constraints
Preparation
Migration
Success?
Remarks
• Automated script
• Kick off: Friday, 22:00h
• 8 parallel script loops
• Monitored first clone loop - looked good!
• Went to sleep …

Customer
Project 2020
Constraints
Preparation
Migration
Success?
Remarks
• 352 PDBs
• 346 moved to the new CDBs fully automatically without errors
• 8 PDB’s aborted with errors
• Got identified quickly and moved manually
• Loop scripts needed between 3 - 6 hours to move the 150 PDBs
• Including pre and post tasks

Customer
Project 2020
Constraints
Preparation
Migration
Success?
Remarks
• Read the complete story at:
• https://mobiliardbblog.wordpress.com/2019/12/16/consolidating-350-pdbs-
in-less-than-6-hours/

Upgrade NOW!
Upgrade to Oracle 19c
PhotobyRicardoGomezAngelonUnsplash

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
• Upgrade 2000 PDBs
• Oracle Database 12.2.0.1 to 19c
• Up to 50 PDBs per CDB in PROD
• Up to 150 PDBs per CDB in DEV/TEST

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
• CPU resources limited
• Solution: Sequential upgrades
• One DBA covers 1-2 CDB upgrades
• Once done, next DBA steps in

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
• Adopt AutoUpgrade
• Download newest version from MOS: 2485457.1
• Phase 1: 735 PDBs on a single weekend
• CDB1 144 PDBs
• CDB2 148 PDBs
• CDB3 148 PDBs
• CDB4 147 PDBs
• CDB5 148 PDBs

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
* Logs of CDB5 are lost
Upgrade timings *

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
• Upgrade timings
• Average: 6 - 7.5 hours
• Including recompilation
• Only 10 oCPUs used per CDB

Customer
Project 2020
Constraints
Preparation
Upgrade
Success?
Remarks
• Post-fixups issue
• Solution: Use the newest OPatch
• Processes parameter
• Upgrade in PDBs spawns a lot of processes
• Solution: Increase it for the upgrade
• Recompile PDBs before upgrade
• ORACLE_HOME/perl/bin/perl $ORACLE_HOME/rdbms/admin/catcon.pl
-n 32 -e -d $ORACLE_HOME/rdbms/admin -l /tmp -b utlrp_log
utlrp.sql
• Read the full story here:
• https://mobiliardbblog.wordpress.com/2020/05/05/what-does-it-take-to-migrate-
735-databases-to-19c-in-a-weekend-the-multitenant-architecture-and-great-dbas/

"We migrated 735 databases to
19c, and the task was mostly
relatively relaxed. Start the
AutoUpgrade tool and monitor
the progress from time to time.
Sitting in front of the screen the
whole time is not necessary."
Alain Fuhrer
Head IT Database Services
La Mobilière
Bern, Switzerland

Further Information
Finally …

HOL | https://MikeDietrichDE.com

Migration to Oracle Multitenant

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Migration to Oracle Multitenant

Similaire à Migration to Oracle Multitenant (20)

Dernier

Dernier (20)

Migration to Oracle Multitenant