SlideShare une entreprise Scribd logo
1  sur  79
Télécharger pour lire hors ligne
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js and Oracle Database
Christopher Jones
Data Access Development
Oracle Database
22 May 2019
christopher.jones@oracle.com
@ghrd
New Development Techniques
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5
About Me
Christopher Jones
Product Manger for Oracle Database,
Scripting Language Drivers (and related
technology)
Contact information
christopher.jones@oracle.com
@ghrd
blogs.oracle.com/opal
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing, and pricing of any
features or functionality described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
8
Introduction
It Starts and Ends With Connections
Getting Data Out of the Database
Putting Data Into the Database
JSON and Simple Oracle Document Access
What’s coming in node-oracledb 4
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
9
Introduction1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Node.js
• Middle-tier JavaScript runtime
– Built on Chrome’s V8 JavaScript engine
– Lightweight and efficient
– Event-driven, non-blocking I/O model
• Allows one language, front-end and back-end
• npm package ecosystem
– World’s largest repo of open-source libraries
• Open Source
– Under Node.js Foundation
10
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Node.js module for accessing Oracle Database
• Development is under Apache 2.0 license
– GitHub repository for source code
– Ongoing features and maintenance by Oracle
• Installable from npm registry
Users can contribute under the Oracle Contributor Agreement.
Thanks to all who have contributed code, documentation and ideas
11
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12
Under the Covers
app.js
node-oracledb
ODPI-C
Oracle Call Interface
in Oracle Client
libraries
Oracle Net Provides:
• Connectivity, Failover, Encryption,
and more
• Configure with tnsnames.ora,
sqlnet.ora
Oracle Client Provides:
• Data Access, Performance, High
Availability, and more
• Configure with oraaccess.xml
Node.js
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb Features
• Async/Await, Promises, Callbacks and Streams
• SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs
• Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution
• Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams
• Smart mapping between JavaScript and Oracle types with manual mapping also available
• Query results as JavaScript objects or arrays
• Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining
• External Authentication / Privileged Connections / Proxy Connections / Password Changing
• Row Prefetching / Statement Caching / Client Result Caching
• End-to-End Tracing / Edition-Based Redefinition
• Continuous Query Notification
• Simple Oracle Document Access (SODA)
• Oracle Database High Availability Features and Oracle Net Features
13
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14
node-oracledb Classes
Base class
• Get connections or create pools
• Set configuration parameters
Connection Pooling
• Dramatically increases performance
• Built-in pool cache for convenience
SQL and PL/SQL Execution
• Transaction support w/data type conversion
• Bind using JavaScript objects or arrays
Read-consistent, pageable cursor
• Used for large result sets
• Callbacks, Promises or Node.js streams
Large object support
• Stream large LOBs with this class
• Can fetch smaller LOBs as string/buffer
Oracledb
Pool
Connection
ResultSet
Lob
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Starting with node-oracledb
1. Install Node.js
2. Install Oracle Instant Client
3. Create package.json
4. Run npm install
Confidential – Oracle Internal/Restricted/Highly Restricted 16
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb fine print
• Prebuilt node-oracledb 3.1 binary modules exist for:
– Windows, Linux and macOS
– Node.js 6, 8, 10, 11
• Upcoming node-oracledb 4.0 will be for Node.js 8+
• You can also build node-oracledb from source
• LTS strategy
Confidential – Oracle Internal/Restricted/Highly Restricted 17
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
oracledb.getConnection(
{user:"hr", password:"welcome", connectString:"localhost/orclpdb1"},
function(err, connection) {
if (err) {
console.error(err.message);
return;
}
connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`,
[110], { outFormat: oracledb.ARRAY },
function(err, result) {
if (err) {
console.error(err.message);
return;
}
console.log(result.rows);
connection.close(function(err) {
if (err) {
console.error(err.message);
}
});
});
});
18
node-oracledb – Node.js Callback Style
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19
node-oracledb - Node.js 8’s Async/Await
async function getEmployee(empid) {
let conn;
try {
connection = await oracledb.getConnection(
{user: "hr", password: "welcome", connectString: "localhost/orclpdb1" });
const result = await connection.execute(
`SELECT department_id, department_name
FROM departments
WHERE manager_id < :id`, [empid] );
console.log(result.rows);
} catch (err) {
console.error(err);
} finally {
if (connection) {
try {
await connection.close();
} catch (err) {
console.error(err);
}
}
}
}
$ node select.js
[ [ 60, 'IT' ],
[ 90, 'Executive' ],
[ 100, 'Finance' ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip:
• Using a fixed Oracle time zone helps avoid machine and deployment
differences
process.env.ORA_SDTZ = 'UTC';
Confidential – Oracle Internal/Restricted/Highly Restricted 20
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
21
It Starts and Ends With Connections
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Standalone Connections
const oracledb = require("oracledb");
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"};
let connection = await oracledb.getConnection(dbConfig);
// ... Use connection
await connection.close();
22
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pooled Connections
// Application initialization
let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60};
let pool = await oracledb.createPool(dbConfig);
// General runtime
let connection = await pool.getConnection();
// ... Use connection
await connection.close();
// Application termination
await pool.close();
23
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24
Node.js Architecture (not entirely accurate)
Timers
TCP/UDP
Evented
File I/O
DNS
User Code
Thread Pool
Async API CallsMain Thread
Event / Callback Queue
Callback
Functions
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• oracledb.createPool({ . . .,
poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . .
– 10 users access the app
• Result: slower than expected throughput. May see deadlocks
– In-use connections wait for DB responses so they hold a thread in use
– Node.js has maximum 4 worker threads by default (upper limit is 128)
• Solution: increase Node.js thread limit before Node.js threadpool starts:
export UV_THREADPOOL_SIZE=10
– May want more threads than connections, to do non-database work
25
Scenario 1: Want to Allow 10 Concurrent DB Users
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– UV_THREADPOOL_SIZE=4
– App opens one connection
– promise.all on 1 x long running SELECT and 3 x INSERTs
• Result
– Each execute() gets thread from worker thread pool
– But the connection can only do one thing
• So the query will block the inserts, blocking the threads from doing other work
• Solution
– Keep control in the JavaScript layer
26
Scenario 2: promise.all
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Threads vs Connections
• Scenario:
– Each iteration of promise.all gets its own connection
– Each connection used to insert some data
• Can the database even handle (lots of) multiple connections?
– You have to balance workload expectations with actual resources available
• Transactional consistency not possible
• Data load solution shown in later slides!
27
Scenario 3: Parallelizing Data Load
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid Connection Storms on DB Server
• Set poolMin = poolMax:
oracledb.createPool(
{ user: "hr", password: "welcome", connectString: "localhost/XE”,
poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 })
• Or use Oracle Net Listener rate limit settings
– CONNECTION_RATE_listener name
– RATE_LIMIT
28
Node.js DB
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Basic High Availability
• Configure operating system network TCP timeouts
• Configure Oracle Net options on the DB and/or node-oracledb side e.g.
– SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT,
SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc.
• connection.break()
– may need: DISABLE_OOB=on if firewall blocks out-of-band breaks
• connection.callTimeout with 18c Oracle Client libraries
29
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Immediately Usable Connections
• Connections in pools can become unusable due to network dropouts etc
• Use 19, 18, or 12.2 Oracle Client libraries
– these have an always-on, network check during pool.getConnection()
• Tune createPool()’s pingInterval setting
– Round-trip ping for detecting session issues
Confidential – Oracle Internal/Restricted/Highly Restricted 31
c = p.getConnection()
c.execute() // error
c = p.getConnection()
c.execute() // error
X
X
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
poolPingInterval
poolPingInterval Behavior at pool.getConnection()
< 0 Never check for connection aliveness
= 0 Always check for each pool.getConnection()
> 0
(default is 60)
Checks aliveness with round-trip ping if the connection has been
idle in the pool (not "checked out" to the application by
getConnection()) for at least poolPingInterval seconds.
Confidential – Oracle Internal/Restricted/Highly Restricted 32
If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Usable Connections
• Always do error checking and recovery after execute()
– network failures could occur anytime after getConnection()
– Could disable poolPingInterval to get ultimate scalability
• Avoid firewall timeouts and DBAs killing “idle” sessions
– node-oracledb ping features may mask these problems
• scalability impact
– Use AWR to check connection rate
Confidential – Oracle Internal/Restricted/Highly Restricted 33
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34
Use the pool cache to access pool by name
Pro Tip: No need to pass a pool around
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"};
/* pool = */ await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection("mypool");
dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”,
poolMin:2, poolMax:4, poolIncrement:1};
await oracledb.createPool(dbConfig);
connection = await oracledb.getConnection();
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection State
• Many applications set connection state
– e.g. using ALTER SESSION commands to set the date format, or a time zone
– For all practical purposes, a 'session' is the same as a 'connection’
• Pooled connections retain state after being released back to the pool
– Next user of the connection will see the same state
– But will the next user get the same connection or a brand new one?
Confidential – Oracle Internal/Restricted/Highly Restricted 35
Node.js DB
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Avoid unnecessary ALTER SESSIONs
connection = await oracledb.getConnection(dbConfig);
connection.execute(`ALTER SESSION . . . `);
Use a connection callback function to set session state
Confidential – Oracle Internal/Restricted/Highly Restricted 36
TIP
X
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
await oracledb.createPool(
{user: un, password: pw, connectString: cs, sessionCallback: initSession});
// Called only when the pool selects a brand new, never-before used connection.
// Called before pool.getConnection() returns.
function initSession(connection, requestedTag, cb) {
connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb);
}
. . .
connection = await oracledb.getConnection(); // get connection in UTC
let result = await connection.execute(sql, binds, options);
await connection.close();
See examples/sessionfixup.js
Confidential – Oracle Internal/Restricted/Highly Restricted 37
Scenario 1: when all connections should have the same state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool sessionCallback
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
Confidential – Oracle Internal/Restricted/Highly Restricted 38
Scenario 1: when all connections should have the same state
Micro-service where each service invocation does one query
poolMax
Service
Invoked
getConnection()
calls
ALTER SESSION
calls
SELECT
calls
Total SQL
Calls
Without a callback 4 1000 1000 1000 1000 2000
With a callback 4 1000 1000 4 1000 1004
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Connection Pool sessionCallback
Avoid round-trips
function initSession(connection, requestedTag, cb) {
connection.execute(
`begin
execute immediate
'alter session set nls_date_format = ''YYYY-MM-DD''
nls_language = AMERICAN';
execute immediate
' . . . '; -- other SQL
end;`,
cb);
}
Confidential – Oracle Internal/Restricted/Highly Restricted 39
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Set a connection tag to represent connection state
connection = await pool.getConnection();
// Set state and update the tag
connection.execute(`ALTER SESSION . . . `, cb);
connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’;
await connection.close();
Confidential – Oracle Internal/Restricted/Highly Restricted 40
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Requesting an already tagged pooled connection
conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'});
When using a sessionCallback this will:
– Select an existing connection in the pool that has the tag USER_TZ=UTC
• sessionCallback is NOT called
– Or, select a new, previously unused connection in the pool (which will have no tag)
• sessionCallback is called
– Or, select a previously used connection with a different tag
• The existing session state and tag are cleared
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 41
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
conn = await oracledb.getConnection({poolAlias:'default’,
tag:'USER_TZ=UTC', matchAnyTag: true});
• This will:
– Select an existing connection in the pool that has the requested tag
• sessionCallback is NOT called
– Or, select another connection in the pool
• Any existing connection state and tag are retained
• sessionCallback is called
Confidential – Oracle Internal/Restricted/Highly Restricted 42
Scenario 2: when connections need differing state
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Tagging
Confidential – Oracle Internal/Restricted/Highly Restricted 43
Scenario 2: when connections need differing state
function initSession(connection, requestedTag, cb) {
console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`);
// Parse requestedTag and connection.tag to decide what state to set
. . .
// Set the state
connection.execute(`ALTER SESSION SET . . .`,
(err) => {
connection.tag = requestedTag; // Update to match the new state
cb(err); // Call cb() last
});
}
See examples/sessiontagging1.js and examples/sessiontagging2.js
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
End Connections When No Longer Needed
• Releasing connections is:
– Required for pooled connections
– Best practice for standalone connections
• Don’t release connections until you are done with them, otherwise:
– “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537:
TNS:connection closed”, etc.
– Example scenario: promise.all error handler releasing the connection
• Handler is called on first error
– Example scenario: streaming from a Lob instance
45
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Connection Pool Draining and Force Closing
• Closing the pool closes connections cleanly
• May need: DISABLE_OOB=on if firewall blocks out-of-band breaks
46
// close pool only if no connections in use
await pool.close();
// close pool when no connections are in use, or force it closed after 10 seconds
// No new connections can be created but existing connections can be used for 10 seconds
await pool.close(10);
// force the pool closed immediately
await pool.close(0);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
47
Getting Data Out of the Database
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Query Methods
• Direct Fetches: execute("SELECT . . .", . . . )
– All rows are returned in one big memory-limited array, or limited to maxRows
• ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . )
– getRow(): Returns one row on each call until all rows are returned
– getRows(numRows): Returns batches of rows in each call until all rows are returned
• Stream: queryStream("SELECT . . .", . . . )
– Streams rows until all rows are returned
48
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Direct Fetches
• Default query behavior
– Easy to use
• Tune network transfer performance with fetchArraySize
– Memory can incrementally grow when the number of query rows is unknown, or
varies from execution to execution
– A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst
case' of a large number of rows
• Drawbacks of direct fetches:
– One big array of results is needed
– Concatenation of record batches can use more memory than the final array requires
49
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50
ResultSet Fetches
const result = await connection.execute(
`SELECT * FROM bigtable`,
[], // no bind variables
{ resultSet: true }
);
const rs = result.resultSet;
let row;
while ((row = await rs.getRow())) {
console.log(row);
}
await rs.close(); // always close the ResultSet
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: Querying Data
• Use row-limiting SQL clauses:
SELECT last_name FROM employees ORDER BY last_name
OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY
• Use Direct Fetches for known small number of rows
– set fetchArraySize to number of rows, if known
– E.g for single row fetches set fetchArraySize to 1
• Use ResultSets or queryStream() for data sets of unknown or big size
– Always close ResultSets
• Cast date and timestamp bind variables in WHERE clauses:
– . . . WHERE cast(:ts as timestamp) < mytimestampcol
– . . . WHERE cast(:dt as date) < mydatecol
51
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Fetching LOBs
• Fetch and bind CLOB and BLOB as String and Buffer for performance
– Only use Lob Stream class for huge LOBs or if memory is limited
const result = await connection.execute(
`SELECT c FROM mylobs WHERE id = :idbv`,
[1],
{ fetchInfo: {"C": {type: oracledb.STRING}} }
);
const clob = result.rows[0][0];
console.log(clob);
52
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
53
Putting Data Into the Database
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Single Row DML is Easy
SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40));
result = await connection.execute(
"INSERT INTO mytable VALUES (:k, :f)",
{ k: 1, f: 'apple' }, // Bind values
{ autoCommit: true }); // Or use connection.commit() later
console.log("Rows inserted: " + result.rowsAffected); // 1
54
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tip: Use Batch Execution for DML
• Executes one DML statement (e.g INSERT, UPDATE) with many data values
• Reduces round trips: "A server round-trip is defined as the trip from the
client to the server and back to the client."
55
executeMany()
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56
Basic Array DML
sql = "INSERT INTO mytable VALUES (:k, :f)";
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ];
options = { autoCommit: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
{ rowsAffected: 2 }
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57
data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" },
{ k: null, f: "cherry" }, { k: 3, f: "pear" },
{ k: null, f: "damson" } ];
options = { batchErrors: true,
bindDefs: {
k: { type: oracledb.NUMBER },
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
BatchErrors
{ rowsAffected: 3,
batchErrors:
[ { Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 },
{ Error: ORA-01400: cannot insert NULL into
("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Batch Errors
• Batch errors are in the result, not the error
– Some classes of error will always trigger real errors
• Any autoCommit will be ignored if there are DML errors
– Valid rows will have been inserted and can be explicitly committed
• Attribute rowsAffected shows 3 rows were inserted
• Array of errors, one for each problematic record
– The offset is the data array position (0-based) of the problem record
58
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
{rowsAffected: 7, dmlRowCounts: [ 5, 2 ]}
59
DML Row Counts
// assume 5 apples and 2 bananas exist in the table
sql = "DELETE FROM mytable WHERE fruit = :f";
data = [ { f: "apple" }, { f: " banana" } ];
options = { dmlRowCounts: true,
bindDefs: {
f: { type: oracledb.STRING, maxSize: 25 } } };
result = await connection.executeMany(sql, data, options);
console.log(result);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60
sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2";
binds = [ ["One"], ["Two"], ["Three"] ];
options = {
bindDefs: [
{ type: oracledb.STRING, maxSize: 5 },
{ type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT }
]
};
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
DMLRETURNING
[ [ [ 'AAAmWkAAMAAAAnWAAA' ] ],
[ [ 'AAAmWkAAMAAAAnWAAB' ] ],
[ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark
• Data: for (var i = 0; i < numRows; i++)
data.push([i, "Test " + i]);
• SQL: sql = "INSERT INTO mytable VALUES (:1, :2)";
• Multiple inserts:
for (var i = 0; i < numRows; i++) {
await connection.execute(sql, data[i],
{autoCommit: (i < numRows-1 ? false : true)}); }
• Single insert:
await connection.executeMany(sql, data, {autoCommit: true});
61
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
executeMany() Benchmark – Scenario 1
• 1 number,
1 short string
• Local DB
• YMMV
1 10 100 1000 10000 100000
execute() 10 38 265 2,323 23,914 227,727
executeMany() 16 9 20 16 39 361
62
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63
executeMany() Benchmark – Scenario 2
• 1 x NUMBER,
• 3 x VARCHAR2(1000))
• Remote DB
• YMMV
1 10 100 1,000
execute() 57 510 5,032 50,949
executeMany() 57 155 368 2,537
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Pro Tips: executeMany()
• For huge data sets may need to call executeMany() multiple times with
batches of records
– autoCommit: false for first batches
– autoCommit: true for the last batch
• Use SQL*Loader or Data Pump instead, where appropriate
– These were added to Instant Client 12.2
64
TIP
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 66
PL/SQL
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Calling PL/SQL is easy
const result = await connection.execute(
`BEGIN
myproc(:i, :io, :o);
END;`,
{
i: 'Chris', // type found from the data. Default dir is BIND_IN
io: { val: 'Jones', dir: oracledb.BIND_INOUT },
o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
});
console.log(result.outBinds);
Confidential – Oracle Internal/Restricted/Highly Restricted 67
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68
SQL> CREATE OR REPLACE PROCEDURE testproc
(p_in IN NUMBER, p_out OUT NUMBER) AS
BEGIN
p_out := p_in * 2;
END;
plsql = "BEGIN testproc(:1, :2); END;";
binds = [ [1], [2], [3] ];
options = {
bindDefs: [
{ type: oracledb.NUMBER },
{ type: oracledb.NUMBER, dir: oracledb.BIND_OUT }
] };
result = await connection.executeMany(sql, data, options);
console.log(result.outBinds);
CallingPL/SQL
[ [ 2 ], [ 4 ], [ 6 ] ]
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
PL/SQL Collection Associative Array (Index-by) Binds
TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;
PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN
FORALL i IN INDICES OF p_vals
INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i));
END;
await connection.execute(
"BEGIN myinproc(:idbv, :valbv); END;",
{ idbv: 1234,
valbv: { type: oracledb.NUMBER,
dir: oracledb.BIND_IN,
val: [1, 2, 23, 4, 10] } };
69
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
70
JSON and Simple Oracle Document Access
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
SQL> CREATE TABLE myjsontab (
c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE);
myContent = {name: "Sally", address: {city: "Melbourne"}};
json = JSON.stringify(myContent);
result = await connection.execute(
'insert into myjsontab (c) values (:cbv)',
{ cbv: json }
);
71
Inserting
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
JSON with Oracle Database 12.1.0.2
result = await connection.execute(
`select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax
{ cbv: 'Sally' },
{ fetchInfo: {"C": {type: oracledb.STRING } }});
js = JSON.parse(result.rows[0]);
console.log('Name is: ' + js.name);
console.log('City is: ' + js.address.city);
//sql = "select c FROM myjsontab where json_exists(c, '$.address.city')";
72
Fetching
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Simple Oracle Document Access (SODA)
• Set of NoSQL-style APIs to create and access documents
– Documents are often JSON
– Oracle maps SODA calls to managed tables, indexes and sequences
– Query-by-example access makes data operations easy
• SQL is not necessary
– But could be used for advanced analysis
• SODA APIs also exist in Python, PL/SQL, C and Java
73
node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74
node-oracledb SODA Classes
Base SODA class
• Get SODA database
• Manipulate SODA collections
Set of Documents
• Create and find documents
Operation Builder
• Filters and QBE
• Retrieve, replace or remove documents
Document Cursor
• Iterate over retrieved documents
SODA Document
• Document content
• Access as JSON, object or buffer
SodaDatabase
SodaCollection
SodaOperation
SodaDocumentCursor
SodaDocument
Oracledb
Pool
Connection
ResultSet
Lob
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75
Creating Collections
Using SODA in node-oracledb
SQL> grant SODA_APP to cj;
conn = await oracledb.getConnection({user:"cj",....});
soda = await conn.getSodaDatabase();
collection = await soda.createCollection("mycollection");
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76
Inserting Documents
Using SODA in node-oracledb
content = {name: "Anthony", address: {city: "Edmonton"}};
await collection.insertOne(content);
doc = await collection.insertOneAndGet(content);
console.log("Key is:", doc.key);
// JSON (or Object or Buffer)
doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}');
await collection.insertOne(doc2);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Using SODA in node-oracledb
• Recommendation is to turn on oracle.autocommit
– Like with SQL, avoid auto commit when inserting lots of documents
• SODA’s DDL-like operations occur in autonomous transactions
77
Commit Behavior
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78
Operation Builder – find()
Using SODA in node-oracledb
doc = await collection.find().key(key).getOne(); // A SodaDocument
content = doc.getContent(); // A JavaScript object
console.log('Retrieved SODA document as an object:');
console.log(content);
content = doc.getContentAsString(); // A JSON string
console.log('Retrieved SODA document as a string:');
console.log(content);
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79
Operation Builder – filter() Query-by-example (QBE)
Using SODA in node-oracledb
// Find all documents with city names starting with 'S'
console.log('Cities starting with S');
documents = await collection.find()
.filter({"address.city": {"$like": "S%"}})
.getDocuments();
for (let i = 0; i < documents.length; i++) {
content = documents[i].getContent();
console.log(' city is: ', content.address.city);
}
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
80
What’s coming in node-oracledb 4
1
2
3
4
5
6
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
What’s coming in node-oracledb 4.0
• Install (compile) current code snapshot with:
npm install oracle/node-oracledb#dev-4.0
• Already landed:
– N-NAPI code refactor improving Node.js version compatibility
– Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers
– Implicit Results
– SODA bulk insert
• Being investigated (no promises!):
– SQL and PL/SQL object binding & queries
• When? Planned CY2019
Confidential – Oracle Internal/Restricted/Highly Restricted 81
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82
AQ ‘RAW’ Queue Demo
Oracle Advanced Queuing
Dequeue:
queue = connection.queue('MYQUEUE');
msg = await queue.deqOne();
await connection.commit();
console.log(msg.payload.toString());
Enqueue:
queue = connection.queue('MYQUEUE');
messageString = 'This is my message';
await queue.enqOne(messageString);
await connection.commit();
BEGIN
dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW');
dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB');
dbms_aqadm.start_queue('MYQUEUE');
END;
/
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83
Wrap up
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
node-oracledb
• Connection management is the basis for good scalability
• Utilize the best query method for the task, and tune it
• Use executeMany() to insert/update/delete multiple records efficiently
• Use SODA’s NoSQL-style data access to simplify applications
84
Copyright © 2019, Oracle and/or its affiliates. All rights reserved. |
Resources
• Mail: christopher.jones@oracle.com
• Twitter: @ghrd, @AnthonyTuininga, @dmcghan
• Blog: https://blogs.oracle.com/opal
• Office Hours: https://tinyurl.com/NodeHours
• Facebook: https://www.facebook.com/groups/oraclescripting/
• Home: https://oracle.github.io/node-oracledb/
• Code: https://github.com/oracle/node-oracledb
85
Node.js and Oracle Database: New Development Techniques

Contenu connexe

Tendances

Zoho & Dim Dim Collaboration
Zoho & Dim Dim CollaborationZoho & Dim Dim Collaboration
Zoho & Dim Dim Collaborationaliniaz3
 
Software requirement specification for online examination system
Software requirement specification for online examination systemSoftware requirement specification for online examination system
Software requirement specification for online examination systemkarthik venkatesh
 
Rich internet application (ria)
Rich internet application (ria)Rich internet application (ria)
Rich internet application (ria)TAInteractive
 
Cloud computing and software engineering
Cloud computing and software engineeringCloud computing and software engineering
Cloud computing and software engineeringRavindra Dastikop
 
College management
College managementCollege management
College managementanandhan30
 
Client side &amp; Server side Scripting
Client side &amp; Server side Scripting Client side &amp; Server side Scripting
Client side &amp; Server side Scripting Webtech Learning
 
University Management System
University Management SystemUniversity Management System
University Management SystemNurul Amin Muhit
 
Online Quiz System Project PPT
Online Quiz System Project PPTOnline Quiz System Project PPT
Online Quiz System Project PPTShanthan Reddy
 
Online Examination System Project report
Online Examination System Project report Online Examination System Project report
Online Examination System Project report SARASWATENDRA SINGH
 
Design patterns difference between interview questions
Design patterns   difference between interview questionsDesign patterns   difference between interview questions
Design patterns difference between interview questionsUmar Ali
 
Cloud Computing and Virtualization
Cloud Computing and Virtualization Cloud Computing and Virtualization
Cloud Computing and Virtualization Mahbub Noor Bappy
 
Mobile dbms
Mobile dbmsMobile dbms
Mobile dbmsTech_MX
 
Online examination system of open and distance education kunti
Online examination system of open and distance education kuntiOnline examination system of open and distance education kunti
Online examination system of open and distance education kuntiNawaraj Ghimire
 

Tendances (20)

Zoho & Dim Dim Collaboration
Zoho & Dim Dim CollaborationZoho & Dim Dim Collaboration
Zoho & Dim Dim Collaboration
 
Software requirement specification for online examination system
Software requirement specification for online examination systemSoftware requirement specification for online examination system
Software requirement specification for online examination system
 
Rich internet application (ria)
Rich internet application (ria)Rich internet application (ria)
Rich internet application (ria)
 
Cloud computing and software engineering
Cloud computing and software engineeringCloud computing and software engineering
Cloud computing and software engineering
 
Introduction to CloudStack
Introduction to CloudStack Introduction to CloudStack
Introduction to CloudStack
 
College management
College managementCollege management
College management
 
Online quiz system
Online quiz systemOnline quiz system
Online quiz system
 
Aneka platform
Aneka platformAneka platform
Aneka platform
 
Client side &amp; Server side Scripting
Client side &amp; Server side Scripting Client side &amp; Server side Scripting
Client side &amp; Server side Scripting
 
University Management System
University Management SystemUniversity Management System
University Management System
 
Open Source Cloud
Open Source CloudOpen Source Cloud
Open Source Cloud
 
Low Level Design
Low Level DesignLow Level Design
Low Level Design
 
Paas
PaasPaas
Paas
 
Online Quiz System Project PPT
Online Quiz System Project PPTOnline Quiz System Project PPT
Online Quiz System Project PPT
 
Virtualization- Cloud Computing
Virtualization- Cloud ComputingVirtualization- Cloud Computing
Virtualization- Cloud Computing
 
Online Examination System Project report
Online Examination System Project report Online Examination System Project report
Online Examination System Project report
 
Design patterns difference between interview questions
Design patterns   difference between interview questionsDesign patterns   difference between interview questions
Design patterns difference between interview questions
 
Cloud Computing and Virtualization
Cloud Computing and Virtualization Cloud Computing and Virtualization
Cloud Computing and Virtualization
 
Mobile dbms
Mobile dbmsMobile dbms
Mobile dbms
 
Online examination system of open and distance education kunti
Online examination system of open and distance education kuntiOnline examination system of open and distance education kunti
Online examination system of open and distance education kunti
 

Similaire à Node.js and Oracle Database: New Development Techniques

Confoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetConfoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetDave Stokes
 
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareMulti-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareBruno Borges
 
Why to Use an Oracle Database?
Why to Use an Oracle Database? Why to Use an Oracle Database?
Why to Use an Oracle Database? Markus Michalewicz
 
Serverless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsServerless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsDavid Delabassee
 
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...Trivadis
 
Oracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesOracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesChris Muir
 
Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Oracle Developers
 
Serverless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsServerless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsDavid Delabassee
 
Java EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudJava EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudBruno Borges
 
Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Shaun Smith
 
Nonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SENonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SEDmitry Kornilov
 
UNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New FeaturesUNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New Featuresmsewtz
 
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...Miguel Araújo
 
Presentation oracle exalogic elastic cloud
Presentation   oracle exalogic elastic cloudPresentation   oracle exalogic elastic cloud
Presentation oracle exalogic elastic cloudsolarisyougood
 
Oracle Autonomous Database - introducción técnica y hands on lab
Oracle Autonomous Database  - introducción técnica y hands on labOracle Autonomous Database  - introducción técnica y hands on lab
Oracle Autonomous Database - introducción técnica y hands on lab"Diego \"Perico\"" Sanchez
 
Consolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesConsolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesDLT Solutions
 
SmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsSmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsKoppelaars
 

Similaire à Node.js and Oracle Database: New Development Techniques (20)

Confoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSetConfoo 202 - MySQL Group Replication and ReplicaSet
Confoo 202 - MySQL Group Replication and ReplicaSet
 
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao MiddlewareMulti-Tenancy: Da Teoria à Prática, do DB ao Middleware
Multi-Tenancy: Da Teoria à Prática, do DB ao Middleware
 
Why to Use an Oracle Database?
Why to Use an Oracle Database? Why to Use an Oracle Database?
Why to Use an Oracle Database?
 
Cloud based database
Cloud based databaseCloud based database
Cloud based database
 
Serverless Java - Challenges and Triumphs
Serverless Java - Challenges and TriumphsServerless Java - Challenges and Triumphs
Serverless Java - Challenges and Triumphs
 
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
TechEvent 2019: Create a Private Database Cloud in the Public Cloud using the...
 
Oracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best PracticesOracle ADF Architecture TV - Development - Programming Best Practices
Oracle ADF Architecture TV - Development - Programming Best Practices
 
Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data Java Library for High Speed Streaming Data
Java Library for High Speed Streaming Data
 
Oracle NoSQL
Oracle NoSQLOracle NoSQL
Oracle NoSQL
 
Serverless Java Challenges & Triumphs
Serverless Java Challenges & TriumphsServerless Java Challenges & Triumphs
Serverless Java Challenges & Triumphs
 
Java EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The CloudJava EE Arquillian Testing with Docker & The Cloud
Java EE Arquillian Testing with Docker & The Cloud
 
Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019
 
con8832-cloudha-2811114.pdf
con8832-cloudha-2811114.pdfcon8832-cloudha-2811114.pdf
con8832-cloudha-2811114.pdf
 
Nonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SENonblocking Database Access in Helidon SE
Nonblocking Database Access in Helidon SE
 
UNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New FeaturesUNYOUG - APEX 19.2 New Features
UNYOUG - APEX 19.2 New Features
 
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
MySQL Shell - A DevOps-engineer day with MySQL’s development and administrati...
 
Presentation oracle exalogic elastic cloud
Presentation   oracle exalogic elastic cloudPresentation   oracle exalogic elastic cloud
Presentation oracle exalogic elastic cloud
 
Oracle Autonomous Database - introducción técnica y hands on lab
Oracle Autonomous Database  - introducción técnica y hands on labOracle Autonomous Database  - introducción técnica y hands on lab
Oracle Autonomous Database - introducción técnica y hands on lab
 
Consolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficienciesConsolidate and prepare for cloud efficiencies
Consolidate and prepare for cloud efficiencies
 
SmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing ConceptsSmartDB Office Hours: Connection Pool Sizing Concepts
SmartDB Office Hours: Connection Pool Sizing Concepts
 

Dernier

Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsJean Silva
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxRTS corp
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesVictoriaMetrics
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsChristian Birchler
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprisepreethippts
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsSafe Software
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?Alexandre Beguel
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorTier1 app
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationBradBedford3
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonApplitools
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfDrew Moseley
 

Dernier (20)

Strategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero resultsStrategies for using alternative queries to mitigate zero results
Strategies for using alternative queries to mitigate zero results
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptxThe Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
The Role of IoT and Sensor Technology in Cargo Cloud Solutions.pptx
 
What’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 UpdatesWhat’s New in VictoriaMetrics: Q1 2024 Updates
What’s New in VictoriaMetrics: Q1 2024 Updates
 
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving CarsSensoDat: Simulation-based Sensor Dataset of Self-driving Cars
SensoDat: Simulation-based Sensor Dataset of Self-driving Cars
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
Odoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 EnterpriseOdoo 14 - eLearning Module In Odoo 14 Enterprise
Odoo 14 - eLearning Module In Odoo 14 Enterprise
 
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News UpdateVictoriaMetrics Q1 Meet Up '24 - Community & News Update
VictoriaMetrics Q1 Meet Up '24 - Community & News Update
 
Powering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data StreamsPowering Real-Time Decisions with Continuous Data Streams
Powering Real-Time Decisions with Continuous Data Streams
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?SAM Training Session - How to use EXCEL ?
SAM Training Session - How to use EXCEL ?
 
Effectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryErrorEffectively Troubleshoot 9 Types of OutOfMemoryError
Effectively Troubleshoot 9 Types of OutOfMemoryError
 
How to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion ApplicationHow to submit a standout Adobe Champion Application
How to submit a standout Adobe Champion Application
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 
VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024VictoriaMetrics Anomaly Detection Updates: Q1 2024
VictoriaMetrics Anomaly Detection Updates: Q1 2024
 
Comparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdfComparing Linux OS Image Update Models - EOSS 2024.pdf
Comparing Linux OS Image Update Models - EOSS 2024.pdf
 

Node.js and Oracle Database: New Development Techniques

  • 1.
  • 2. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js and Oracle Database Christopher Jones Data Access Development Oracle Database 22 May 2019 christopher.jones@oracle.com @ghrd New Development Techniques
  • 3. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5 About Me Christopher Jones Product Manger for Oracle Database, Scripting Language Drivers (and related technology) Contact information christopher.jones@oracle.com @ghrd blogs.oracle.com/opal
  • 4. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. 6
  • 5. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 8 Introduction It Starts and Ends With Connections Getting Data Out of the Database Putting Data Into the Database JSON and Simple Oracle Document Access What’s coming in node-oracledb 4 1 2 3 4 5 6
  • 6. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 9 Introduction1 2 3 4 5 6
  • 7. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Node.js • Middle-tier JavaScript runtime – Built on Chrome’s V8 JavaScript engine – Lightweight and efficient – Event-driven, non-blocking I/O model • Allows one language, front-end and back-end • npm package ecosystem – World’s largest repo of open-source libraries • Open Source – Under Node.js Foundation 10
  • 8. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Node.js module for accessing Oracle Database • Development is under Apache 2.0 license – GitHub repository for source code – Ongoing features and maintenance by Oracle • Installable from npm registry Users can contribute under the Oracle Contributor Agreement. Thanks to all who have contributed code, documentation and ideas 11
  • 9. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12 Under the Covers app.js node-oracledb ODPI-C Oracle Call Interface in Oracle Client libraries Oracle Net Provides: • Connectivity, Failover, Encryption, and more • Configure with tnsnames.ora, sqlnet.ora Oracle Client Provides: • Data Access, Performance, High Availability, and more • Configure with oraaccess.xml Node.js
  • 10. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb Features • Async/Await, Promises, Callbacks and Streams • SQL and PL/SQL Execution / Fetching of large result sets / REF CURSORs • Binding using JavaScript objects or arrays / DML RETURNING / Batch Statement Execution • Large Objects: CLOBs and BLOBs and Strings/Buffers or Streams • Smart mapping between JavaScript and Oracle types with manual mapping also available • Query results as JavaScript objects or arrays • Connection Pooling (Hetero & Homogeneous) with Aliasing, Queuing, Caching, Tagging, Draining • External Authentication / Privileged Connections / Proxy Connections / Password Changing • Row Prefetching / Statement Caching / Client Result Caching • End-to-End Tracing / Edition-Based Redefinition • Continuous Query Notification • Simple Oracle Document Access (SODA) • Oracle Database High Availability Features and Oracle Net Features 13
  • 11. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14 node-oracledb Classes Base class • Get connections or create pools • Set configuration parameters Connection Pooling • Dramatically increases performance • Built-in pool cache for convenience SQL and PL/SQL Execution • Transaction support w/data type conversion • Bind using JavaScript objects or arrays Read-consistent, pageable cursor • Used for large result sets • Callbacks, Promises or Node.js streams Large object support • Stream large LOBs with this class • Can fetch smaller LOBs as string/buffer Oracledb Pool Connection ResultSet Lob
  • 12. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Starting with node-oracledb 1. Install Node.js 2. Install Oracle Instant Client 3. Create package.json 4. Run npm install Confidential – Oracle Internal/Restricted/Highly Restricted 16
  • 13. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb fine print • Prebuilt node-oracledb 3.1 binary modules exist for: – Windows, Linux and macOS – Node.js 6, 8, 10, 11 • Upcoming node-oracledb 4.0 will be for Node.js 8+ • You can also build node-oracledb from source • LTS strategy Confidential – Oracle Internal/Restricted/Highly Restricted 17
  • 14. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | oracledb.getConnection( {user:"hr", password:"welcome", connectString:"localhost/orclpdb1"}, function(err, connection) { if (err) { console.error(err.message); return; } connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [110], { outFormat: oracledb.ARRAY }, function(err, result) { if (err) { console.error(err.message); return; } console.log(result.rows); connection.close(function(err) { if (err) { console.error(err.message); } }); }); }); 18 node-oracledb – Node.js Callback Style
  • 15. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19 node-oracledb - Node.js 8’s Async/Await async function getEmployee(empid) { let conn; try { connection = await oracledb.getConnection( {user: "hr", password: "welcome", connectString: "localhost/orclpdb1" }); const result = await connection.execute( `SELECT department_id, department_name FROM departments WHERE manager_id < :id`, [empid] ); console.log(result.rows); } catch (err) { console.error(err); } finally { if (connection) { try { await connection.close(); } catch (err) { console.error(err); } } } } $ node select.js [ [ 60, 'IT' ], [ 90, 'Executive' ], [ 100, 'Finance' ] ]
  • 16. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: • Using a fixed Oracle time zone helps avoid machine and deployment differences process.env.ORA_SDTZ = 'UTC'; Confidential – Oracle Internal/Restricted/Highly Restricted 20 TIP
  • 17. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 21 It Starts and Ends With Connections 1 2 3 4 5 6
  • 18. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Standalone Connections const oracledb = require("oracledb"); let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE"}; let connection = await oracledb.getConnection(dbConfig); // ... Use connection await connection.close(); 22 Node.js DB
  • 19. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pooled Connections // Application initialization let dbConfig = {user:"hr", password:"***", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, poolTimeout:60}; let pool = await oracledb.createPool(dbConfig); // General runtime let connection = await pool.getConnection(); // ... Use connection await connection.close(); // Application termination await pool.close(); 23 Node.js DB
  • 20. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24 Node.js Architecture (not entirely accurate) Timers TCP/UDP Evented File I/O DNS User Code Thread Pool Async API CallsMain Thread Event / Callback Queue Callback Functions
  • 21. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • oracledb.createPool({ . . ., poolMin:0, poolMax:10, poolIncrement:1, poolTimeout:60 }, . . . – 10 users access the app • Result: slower than expected throughput. May see deadlocks – In-use connections wait for DB responses so they hold a thread in use – Node.js has maximum 4 worker threads by default (upper limit is 128) • Solution: increase Node.js thread limit before Node.js threadpool starts: export UV_THREADPOOL_SIZE=10 – May want more threads than connections, to do non-database work 25 Scenario 1: Want to Allow 10 Concurrent DB Users
  • 22. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – UV_THREADPOOL_SIZE=4 – App opens one connection – promise.all on 1 x long running SELECT and 3 x INSERTs • Result – Each execute() gets thread from worker thread pool – But the connection can only do one thing • So the query will block the inserts, blocking the threads from doing other work • Solution – Keep control in the JavaScript layer 26 Scenario 2: promise.all
  • 23. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Threads vs Connections • Scenario: – Each iteration of promise.all gets its own connection – Each connection used to insert some data • Can the database even handle (lots of) multiple connections? – You have to balance workload expectations with actual resources available • Transactional consistency not possible • Data load solution shown in later slides! 27 Scenario 3: Parallelizing Data Load
  • 24. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid Connection Storms on DB Server • Set poolMin = poolMax: oracledb.createPool( { user: "hr", password: "welcome", connectString: "localhost/XE”, poolMin: 10, poolMax: 10, poolIncrement: 0, poolTimeout: 60 }) • Or use Oracle Net Listener rate limit settings – CONNECTION_RATE_listener name – RATE_LIMIT 28 Node.js DB TIP
  • 25. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Basic High Availability • Configure operating system network TCP timeouts • Configure Oracle Net options on the DB and/or node-oracledb side e.g. – SQLNET.OUTBOUND_CONNECT_TIMEOUT, SQLNET.RECV_TIMEOUT, SQLNET.SEND_TIMEOUT, EXPIRE_TIME,(ENABLE=BROKEN)etc. • connection.break() – may need: DISABLE_OOB=on if firewall blocks out-of-band breaks • connection.callTimeout with 18c Oracle Client libraries 29 TIP
  • 26. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Immediately Usable Connections • Connections in pools can become unusable due to network dropouts etc • Use 19, 18, or 12.2 Oracle Client libraries – these have an always-on, network check during pool.getConnection() • Tune createPool()’s pingInterval setting – Round-trip ping for detecting session issues Confidential – Oracle Internal/Restricted/Highly Restricted 31 c = p.getConnection() c.execute() // error c = p.getConnection() c.execute() // error X X
  • 27. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | poolPingInterval poolPingInterval Behavior at pool.getConnection() < 0 Never check for connection aliveness = 0 Always check for each pool.getConnection() > 0 (default is 60) Checks aliveness with round-trip ping if the connection has been idle in the pool (not "checked out" to the application by getConnection()) for at least poolPingInterval seconds. Confidential – Oracle Internal/Restricted/Highly Restricted 32 If a ping detects an unusable connection, it is dropped and a new one created before p.getConnection() returns
  • 28. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Usable Connections • Always do error checking and recovery after execute() – network failures could occur anytime after getConnection() – Could disable poolPingInterval to get ultimate scalability • Avoid firewall timeouts and DBAs killing “idle” sessions – node-oracledb ping features may mask these problems • scalability impact – Use AWR to check connection rate Confidential – Oracle Internal/Restricted/Highly Restricted 33 TIP
  • 29. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34 Use the pool cache to access pool by name Pro Tip: No need to pass a pool around dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1, alias:"mypool"}; /* pool = */ await oracledb.createPool(dbConfig); connection = await oracledb.getConnection("mypool"); dbConfig = {user:"hr", password:"XXX", connectString:"localhost/XE”, poolMin:2, poolMax:4, poolIncrement:1}; await oracledb.createPool(dbConfig); connection = await oracledb.getConnection(); TIP
  • 30. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection State • Many applications set connection state – e.g. using ALTER SESSION commands to set the date format, or a time zone – For all practical purposes, a 'session' is the same as a 'connection’ • Pooled connections retain state after being released back to the pool – Next user of the connection will see the same state – But will the next user get the same connection or a brand new one? Confidential – Oracle Internal/Restricted/Highly Restricted 35 Node.js DB
  • 31. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Avoid unnecessary ALTER SESSIONs connection = await oracledb.getConnection(dbConfig); connection.execute(`ALTER SESSION . . . `); Use a connection callback function to set session state Confidential – Oracle Internal/Restricted/Highly Restricted 36 TIP X
  • 32. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback await oracledb.createPool( {user: un, password: pw, connectString: cs, sessionCallback: initSession}); // Called only when the pool selects a brand new, never-before used connection. // Called before pool.getConnection() returns. function initSession(connection, requestedTag, cb) { connection.execute(`ALTER SESSION SET TIME_ZONE = 'UTC'`, cb); } . . . connection = await oracledb.getConnection(); // get connection in UTC let result = await connection.execute(sql, binds, options); await connection.close(); See examples/sessionfixup.js Confidential – Oracle Internal/Restricted/Highly Restricted 37 Scenario 1: when all connections should have the same state
  • 33. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool sessionCallback poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 Confidential – Oracle Internal/Restricted/Highly Restricted 38 Scenario 1: when all connections should have the same state Micro-service where each service invocation does one query poolMax Service Invoked getConnection() calls ALTER SESSION calls SELECT calls Total SQL Calls Without a callback 4 1000 1000 1000 1000 2000 With a callback 4 1000 1000 4 1000 1004
  • 34. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Connection Pool sessionCallback Avoid round-trips function initSession(connection, requestedTag, cb) { connection.execute( `begin execute immediate 'alter session set nls_date_format = ''YYYY-MM-DD'' nls_language = AMERICAN'; execute immediate ' . . . '; -- other SQL end;`, cb); } Confidential – Oracle Internal/Restricted/Highly Restricted 39 TIP
  • 35. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Set a connection tag to represent connection state connection = await pool.getConnection(); // Set state and update the tag connection.execute(`ALTER SESSION . . . `, cb); connection.tag = 'NAME1=VALUE1;NAME2=VALUE2’; await connection.close(); Confidential – Oracle Internal/Restricted/Highly Restricted 40 Scenario 2: when connections need differing state
  • 36. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Requesting an already tagged pooled connection conn = await oracledb.getConnection({poolAlias:'default', tag:'USER_TZ=UTC'}); When using a sessionCallback this will: – Select an existing connection in the pool that has the tag USER_TZ=UTC • sessionCallback is NOT called – Or, select a new, previously unused connection in the pool (which will have no tag) • sessionCallback is called – Or, select a previously used connection with a different tag • The existing session state and tag are cleared • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 41 Scenario 2: when connections need differing state
  • 37. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging conn = await oracledb.getConnection({poolAlias:'default’, tag:'USER_TZ=UTC', matchAnyTag: true}); • This will: – Select an existing connection in the pool that has the requested tag • sessionCallback is NOT called – Or, select another connection in the pool • Any existing connection state and tag are retained • sessionCallback is called Confidential – Oracle Internal/Restricted/Highly Restricted 42 Scenario 2: when connections need differing state
  • 38. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Tagging Confidential – Oracle Internal/Restricted/Highly Restricted 43 Scenario 2: when connections need differing state function initSession(connection, requestedTag, cb) { console.log(`requested tag: ${requestedTag}, actual tag: ${connection.tag}`); // Parse requestedTag and connection.tag to decide what state to set . . . // Set the state connection.execute(`ALTER SESSION SET . . .`, (err) => { connection.tag = requestedTag; // Update to match the new state cb(err); // Call cb() last }); } See examples/sessiontagging1.js and examples/sessiontagging2.js
  • 39. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | End Connections When No Longer Needed • Releasing connections is: – Required for pooled connections – Best practice for standalone connections • Don’t release connections until you are done with them, otherwise: – “NJS-003: invalid connection”, “DPI-1010: not connected”, “ORA-12537: TNS:connection closed”, etc. – Example scenario: promise.all error handler releasing the connection • Handler is called on first error – Example scenario: streaming from a Lob instance 45
  • 40. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Connection Pool Draining and Force Closing • Closing the pool closes connections cleanly • May need: DISABLE_OOB=on if firewall blocks out-of-band breaks 46 // close pool only if no connections in use await pool.close(); // close pool when no connections are in use, or force it closed after 10 seconds // No new connections can be created but existing connections can be used for 10 seconds await pool.close(10); // force the pool closed immediately await pool.close(0);
  • 41. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 47 Getting Data Out of the Database 1 2 3 4 5 6
  • 42. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Query Methods • Direct Fetches: execute("SELECT . . .", . . . ) – All rows are returned in one big memory-limited array, or limited to maxRows • ResultSet : execute("SELECT . . .", . . ., { resultSet: true }, . . . ) – getRow(): Returns one row on each call until all rows are returned – getRows(numRows): Returns batches of rows in each call until all rows are returned • Stream: queryStream("SELECT . . .", . . . ) – Streams rows until all rows are returned 48
  • 43. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Direct Fetches • Default query behavior – Easy to use • Tune network transfer performance with fetchArraySize – Memory can incrementally grow when the number of query rows is unknown, or varies from execution to execution – A single large chunk of memory doesn't need to be pre-allocated to handle the 'worst case' of a large number of rows • Drawbacks of direct fetches: – One big array of results is needed – Concatenation of record batches can use more memory than the final array requires 49
  • 44. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 50 ResultSet Fetches const result = await connection.execute( `SELECT * FROM bigtable`, [], // no bind variables { resultSet: true } ); const rs = result.resultSet; let row; while ((row = await rs.getRow())) { console.log(row); } await rs.close(); // always close the ResultSet
  • 45. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: Querying Data • Use row-limiting SQL clauses: SELECT last_name FROM employees ORDER BY last_name OFFSET :offset ROWS FETCH NEXT :maxnumrows ROWS ONLY • Use Direct Fetches for known small number of rows – set fetchArraySize to number of rows, if known – E.g for single row fetches set fetchArraySize to 1 • Use ResultSets or queryStream() for data sets of unknown or big size – Always close ResultSets • Cast date and timestamp bind variables in WHERE clauses: – . . . WHERE cast(:ts as timestamp) < mytimestampcol – . . . WHERE cast(:dt as date) < mydatecol 51 TIP
  • 46. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Fetching LOBs • Fetch and bind CLOB and BLOB as String and Buffer for performance – Only use Lob Stream class for huge LOBs or if memory is limited const result = await connection.execute( `SELECT c FROM mylobs WHERE id = :idbv`, [1], { fetchInfo: {"C": {type: oracledb.STRING}} } ); const clob = result.rows[0][0]; console.log(clob); 52
  • 47. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 53 Putting Data Into the Database 1 2 3 4 5 6
  • 48. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Single Row DML is Easy SQL> CREATE TABLE mytable (key NUMBER(9) NOT NULL, fruit VARCHAR2(40)); result = await connection.execute( "INSERT INTO mytable VALUES (:k, :f)", { k: 1, f: 'apple' }, // Bind values { autoCommit: true }); // Or use connection.commit() later console.log("Rows inserted: " + result.rowsAffected); // 1 54
  • 49. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tip: Use Batch Execution for DML • Executes one DML statement (e.g INSERT, UPDATE) with many data values • Reduces round trips: "A server round-trip is defined as the trip from the client to the server and back to the client." 55 executeMany() TIP
  • 50. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 56 Basic Array DML sql = "INSERT INTO mytable VALUES (:k, :f)"; data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" } ]; options = { autoCommit: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); { rowsAffected: 2 }
  • 51. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 57 data = [ { k: 1, f: "apple" }, { k: 2, f: "banana" }, { k: null, f: "cherry" }, { k: 3, f: "pear" }, { k: null, f: "damson" } ]; options = { batchErrors: true, bindDefs: { k: { type: oracledb.NUMBER }, f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result); BatchErrors { rowsAffected: 3, batchErrors: [ { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 2 }, { Error: ORA-01400: cannot insert NULL into ("CJ"."MYTABLE"."KEY") errorNum: 1400, offset: 4 } ] }
  • 52. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Batch Errors • Batch errors are in the result, not the error – Some classes of error will always trigger real errors • Any autoCommit will be ignored if there are DML errors – Valid rows will have been inserted and can be explicitly committed • Attribute rowsAffected shows 3 rows were inserted • Array of errors, one for each problematic record – The offset is the data array position (0-based) of the problem record 58
  • 53. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | {rowsAffected: 7, dmlRowCounts: [ 5, 2 ]} 59 DML Row Counts // assume 5 apples and 2 bananas exist in the table sql = "DELETE FROM mytable WHERE fruit = :f"; data = [ { f: "apple" }, { f: " banana" } ]; options = { dmlRowCounts: true, bindDefs: { f: { type: oracledb.STRING, maxSize: 25 } } }; result = await connection.executeMany(sql, data, options); console.log(result);
  • 54. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 60 sql = "INSERT INTO tab VALUES (:1) RETURNING ROWID INTO :2"; binds = [ ["One"], ["Two"], ["Three"] ]; options = { bindDefs: [ { type: oracledb.STRING, maxSize: 5 }, { type: oracledb.STRING, maxSize: 18, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); DMLRETURNING [ [ [ 'AAAmWkAAMAAAAnWAAA' ] ], [ [ 'AAAmWkAAMAAAAnWAAB' ] ], [ [ 'AAAmWkAAMAAAAnWAAC' ] ] ]
  • 55. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark • Data: for (var i = 0; i < numRows; i++) data.push([i, "Test " + i]); • SQL: sql = "INSERT INTO mytable VALUES (:1, :2)"; • Multiple inserts: for (var i = 0; i < numRows; i++) { await connection.execute(sql, data[i], {autoCommit: (i < numRows-1 ? false : true)}); } • Single insert: await connection.executeMany(sql, data, {autoCommit: true}); 61
  • 56. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | executeMany() Benchmark – Scenario 1 • 1 number, 1 short string • Local DB • YMMV 1 10 100 1000 10000 100000 execute() 10 38 265 2,323 23,914 227,727 executeMany() 16 9 20 16 39 361 62
  • 57. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 63 executeMany() Benchmark – Scenario 2 • 1 x NUMBER, • 3 x VARCHAR2(1000)) • Remote DB • YMMV 1 10 100 1,000 execute() 57 510 5,032 50,949 executeMany() 57 155 368 2,537
  • 58. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Pro Tips: executeMany() • For huge data sets may need to call executeMany() multiple times with batches of records – autoCommit: false for first batches – autoCommit: true for the last batch • Use SQL*Loader or Data Pump instead, where appropriate – These were added to Instant Client 12.2 64 TIP
  • 59. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 66 PL/SQL
  • 60. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Calling PL/SQL is easy const result = await connection.execute( `BEGIN myproc(:i, :io, :o); END;`, { i: 'Chris', // type found from the data. Default dir is BIND_IN io: { val: 'Jones', dir: oracledb.BIND_INOUT }, o: { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } }); console.log(result.outBinds); Confidential – Oracle Internal/Restricted/Highly Restricted 67
  • 61. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 68 SQL> CREATE OR REPLACE PROCEDURE testproc (p_in IN NUMBER, p_out OUT NUMBER) AS BEGIN p_out := p_in * 2; END; plsql = "BEGIN testproc(:1, :2); END;"; binds = [ [1], [2], [3] ]; options = { bindDefs: [ { type: oracledb.NUMBER }, { type: oracledb.NUMBER, dir: oracledb.BIND_OUT } ] }; result = await connection.executeMany(sql, data, options); console.log(result.outBinds); CallingPL/SQL [ [ 2 ], [ 4 ], [ 6 ] ]
  • 62. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | PL/SQL Collection Associative Array (Index-by) Binds TYPE numtype IS TABLE OF NUMBER INDEX BY BINARY_INTEGER; PROCEDURE myinproc(p_id IN NUMBER, p_vals IN numtype) IS BEGIN FORALL i IN INDICES OF p_vals INSERT INTO sometable (id, numcol) VALUES (p_id, p_vals(i)); END; await connection.execute( "BEGIN myinproc(:idbv, :valbv); END;", { idbv: 1234, valbv: { type: oracledb.NUMBER, dir: oracledb.BIND_IN, val: [1, 2, 23, 4, 10] } }; 69
  • 63. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 70 JSON and Simple Oracle Document Access 1 2 3 4 5 6
  • 64. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 SQL> CREATE TABLE myjsontab ( c CLOB CHECK (c IS JSON)) LOB (c) STORE AS (CACHE); myContent = {name: "Sally", address: {city: "Melbourne"}}; json = JSON.stringify(myContent); result = await connection.execute( 'insert into myjsontab (c) values (:cbv)', { cbv: json } ); 71 Inserting
  • 65. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | JSON with Oracle Database 12.1.0.2 result = await connection.execute( `select c from myjsontab t where t.c.name = :cbv`, // 12.2 syntax { cbv: 'Sally' }, { fetchInfo: {"C": {type: oracledb.STRING } }}); js = JSON.parse(result.rows[0]); console.log('Name is: ' + js.name); console.log('City is: ' + js.address.city); //sql = "select c FROM myjsontab where json_exists(c, '$.address.city')"; 72 Fetching
  • 66. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Simple Oracle Document Access (SODA) • Set of NoSQL-style APIs to create and access documents – Documents are often JSON – Oracle maps SODA calls to managed tables, indexes and sequences – Query-by-example access makes data operations easy • SQL is not necessary – But could be used for advanced analysis • SODA APIs also exist in Python, PL/SQL, C and Java 73 node-oracledb support requires DB 18.3 and Oracle Client 18.5/19.3
  • 67. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 74 node-oracledb SODA Classes Base SODA class • Get SODA database • Manipulate SODA collections Set of Documents • Create and find documents Operation Builder • Filters and QBE • Retrieve, replace or remove documents Document Cursor • Iterate over retrieved documents SODA Document • Document content • Access as JSON, object or buffer SodaDatabase SodaCollection SodaOperation SodaDocumentCursor SodaDocument Oracledb Pool Connection ResultSet Lob
  • 68. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 75 Creating Collections Using SODA in node-oracledb SQL> grant SODA_APP to cj; conn = await oracledb.getConnection({user:"cj",....}); soda = await conn.getSodaDatabase(); collection = await soda.createCollection("mycollection");
  • 69. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 76 Inserting Documents Using SODA in node-oracledb content = {name: "Anthony", address: {city: "Edmonton"}}; await collection.insertOne(content); doc = await collection.insertOneAndGet(content); console.log("Key is:", doc.key); // JSON (or Object or Buffer) doc2 = soda.createDocument('{"name":"Venkat","city":"Bengaluru"}'); await collection.insertOne(doc2);
  • 70. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Using SODA in node-oracledb • Recommendation is to turn on oracle.autocommit – Like with SQL, avoid auto commit when inserting lots of documents • SODA’s DDL-like operations occur in autonomous transactions 77 Commit Behavior
  • 71. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 78 Operation Builder – find() Using SODA in node-oracledb doc = await collection.find().key(key).getOne(); // A SodaDocument content = doc.getContent(); // A JavaScript object console.log('Retrieved SODA document as an object:'); console.log(content); content = doc.getContentAsString(); // A JSON string console.log('Retrieved SODA document as a string:'); console.log(content);
  • 72. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 79 Operation Builder – filter() Query-by-example (QBE) Using SODA in node-oracledb // Find all documents with city names starting with 'S' console.log('Cities starting with S'); documents = await collection.find() .filter({"address.city": {"$like": "S%"}}) .getDocuments(); for (let i = 0; i < documents.length; i++) { content = documents[i].getContent(); console.log(' city is: ', content.address.city); }
  • 73. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Program Agenda 80 What’s coming in node-oracledb 4 1 2 3 4 5 6
  • 74. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | What’s coming in node-oracledb 4.0 • Install (compile) current code snapshot with: npm install oracle/node-oracledb#dev-4.0 • Already landed: – N-NAPI code refactor improving Node.js version compatibility – Advanced Queuing for “RAW” queues, ie. Node.js String and Buffers – Implicit Results – SODA bulk insert • Being investigated (no promises!): – SQL and PL/SQL object binding & queries • When? Planned CY2019 Confidential – Oracle Internal/Restricted/Highly Restricted 81
  • 75. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 82 AQ ‘RAW’ Queue Demo Oracle Advanced Queuing Dequeue: queue = connection.queue('MYQUEUE'); msg = await queue.deqOne(); await connection.commit(); console.log(msg.payload.toString()); Enqueue: queue = connection.queue('MYQUEUE'); messageString = 'This is my message'; await queue.enqOne(messageString); await connection.commit(); BEGIN dbms_aqadm.create_queue_table('DEMO_RAW_QUEUE_TAB', 'RAW'); dbms_aqadm.create_queue('MYQUEUE', 'DEMO_RAW_QUEUE_TAB'); dbms_aqadm.start_queue('MYQUEUE'); END; /
  • 76. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 83 Wrap up
  • 77. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | node-oracledb • Connection management is the basis for good scalability • Utilize the best query method for the task, and tune it • Use executeMany() to insert/update/delete multiple records efficiently • Use SODA’s NoSQL-style data access to simplify applications 84
  • 78. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Resources • Mail: christopher.jones@oracle.com • Twitter: @ghrd, @AnthonyTuininga, @dmcghan • Blog: https://blogs.oracle.com/opal • Office Hours: https://tinyurl.com/NodeHours • Facebook: https://www.facebook.com/groups/oraclescripting/ • Home: https://oracle.github.io/node-oracledb/ • Code: https://github.com/oracle/node-oracledb 85