In recent years, technological developments have undergone in-depth analysis among banks, but we are still far from attaining mature levels both at the methodological and at the credit granting, monitoring and control process levels. Banks should equip themselves with new and more structured Model Risk frameworks to manage new Machine Learning model validation paradigms. Learn more from Accenture Finance & Risk: https://accntu.re/2qGUUMx

1. MEASURING AND MANAGING
CREDIT RISK WITH
MACHINE LEARNING AND
ARTIFICIAL INTELLIGENCE:
A NEW ERA?
STEFANO BONINI, ACCENTURE FINANCE & RISK
GIULIANA CAIVANO, ACCENTURE FINANCE & RISK

2. TOPICS
ARTIFICIAL INTELLIGENCE AND MACHINE
LEARNING: A JOURNEY THROUGH TIME1
MACHINE LEARNING IN RISK MANAGEMENT2
EVOLUTION OF CREDIT RISK3
4 CONCLUSIONS AND NEXT STEPS
Copyright © 2019 Accenture. All rights reserved. 2

3. ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING
A JOURNEY THROUGH TIME
“Are there imaginable digital computers which would do well in the imitation game?“1
1950-1960 1990-2000
1970-1980 TODAY…
Copyright © 2019 Accenture. All rights reserved. 3
Alan Turing
proposes Turing Test
as a measure of
machine
intelligence7
1950 1966
The
MIT Computer
Science & Artificial
Intelligence Lab
creates Eliza - the
Chatbot8
20142001
UBS AG uses Sqreem
Technologies Pte.
Ltd. Artificial
Intelligence to
provide
financial
advice11
2019
Amazon Alexa™ is a
cloud-based voice
service developed by
Amazon.com Inc., and
used in Amazon Echo™
devices12
Robots beat humans in a
simulated financial trading
competition; the Robots made
7% more cash than
the humans10
1955 1987
The term "Artificial Intelligence“
is first coined by computer scientist
John McCarthy for the
Dartmouth College AI
conference2
Security Pacific National
Bank introduces fraud
prevention task based on
artificial neural network3
2013
“KENSHO,” the financial
answer machine combines
latest big data and machine
learning techniques to
analyze how real-world
events affect markets4
Google Duplex™
assistive
technology, a service
to allow an AI
assistant to book an
appointment by
phone6
20182017
A top tier investment bank adopts
COiN (Contract Intelligence), an
Artificial Intelligence tool that
analyzes legal documents and
contracts using image recognition5
1997
The Deep Blue
chess machine
defeats world chess
champion, Garry
Kasparov9

4. MACHINE LEARNING IN RISK MANAGEMENT
GLOBAL VIEW
Degree of banks’ maturity with respect to the application of Machine Learning in credit risk
Copyright © 2019 Accenture. All rights reserved. 4
MATURITY OF MACHINE LEARNING BASED ON COMPANY
SIZE (TOTAL ASSETS, USD) APPLICATIONS AREAS OF MACHINE LEARNING IN CREDIT RISK
$1t plus
$500b-$1t
$150b-$500b
Under $150b
0 20 40 60 80 100 0 10 20 30 40 50 60 70 80 90
Collections
Credit Monitoring
Credit Scoring and decisioning
Provisioning
Economic Capital
Stress Testing
Regulatory Capital
Mature Intermediate BeginnerMature Intermediate Beginner None
KEY BENEFITS OF APPLYING MACHINE LEARNING
MORE
PRECISE
MODELS
BETTER
DATA
USAGE
MORE EFFICIENCY
IN MODEL
DEVELOPMENT
DATA
DEFICIENCIES
ADDRESSED
Source: Institute of International Finance – Machine Learning in Credit Risk Summary Report – Nov 2018
Regulatory capital
Stress Testing
Economic capital
Provisioning
Credit classification and
decisioning
Credit Monitoring
Collections
Mature Intermediate

5. LEGENDA
 Si
 No, ma: No, ma lo reputo potenzialmente utile/applicabile
 No, non: No, e non lo reputo potenzialmente utile/applicabile
 N/A: Non sa/Non Risponde
 There is a willingness to use machine learning in model estimation, given the high amount of data available to banks; despite the
advantages, only 25% of banking players adopt it for internal model estimation and 19% for rating scales enhancement
 A Few (8%) banks use Machine Learning techniques to do stress test and to manage non-performing loans
 Over 50% of banks, while not applying them, consider Machine Learning techniques as potentially useful and applicable in each area
that was previously analyzed
Accenture Analysis based on the evidence from the survey pool
MACHINE LEARNING IN RISK MANAGEMENT
ITALIAN VIEW
Copyright © 2019 Accenture. All rights reserved. 5
CREDIT
RISK
MODEL ESTIMATION /
DEVELOPMENT
RATING SCALE
STRESS TEST
DATA QUALITY
EARLY WARNING
CUSTOMER SEGMENTATION
NON-PERFORMING LOANS
Yes– 25.0% No, but– 52.4%
Yes – 19.3% No, but – 59.1%
Yes– 8.6% No, but – 75.0%
Yes – 10.0% No, but – 76.2%
Yes– 7.0% No, but – 71.4%
Yes– 8.6% No, but – 68.7%
No, but – 61.9%Yes– 7.5%
No, not /NA – 22.7%
No, not /NA – 21.6%
No, not /NA – 13.8%
No, not /NA – 30.6%
No, not /NA – 21.6%
No, not/NA – 23.6%
No, not /NA – 16.4%
No, but: No, but I consider it potentially useful/applicable *No, not/NA: No, and I don't think it's useful/ Don’t know or don’t answerLEGEND
EVIDENCES
Source: INTELLIGENZA ARTIFICIALE: L’APPLICAZIONE DI MACHINE LEARNING E PREDICTIVE ANALYTICS NEL RISK MANAGEMENT, AIFRIM position paper,
March 13, 2019. Access at: http://www.aifirm.it/presentazione-position-paper-aifirm-intelligenza-artificiale-lapplicazione-di-machine-learning-e-predictive-analytics-nel-
risk-management/

6. MAIN CHALLENGES
SHOULD PROBE ALGORITHMS TO PRODUCE
INTERMEDIATE RESULTS THAT EXPLAIN WHAT, HOW
AND WHY
COMPLEX ALGORITHMS FOLLOW A LOGIC IN WHICH
THE ROUTES DEVELOP DYNAMICALLY AND ARE MORE
DIFFICULT TO EXPLAIN
SHOULD HAVE AN INTELLIGENT AND PERSPECTIVE
VIEW OF RESULTS - CONSIDERING THE COMPLEXITY
AND AMOUNT OF POSSIBLE OUTPUTS
MACHINE LEARNING IN RISK MANAGEMENT
CHALLENGES AND TESTING
“…it is inevitable that the more AI enters our lives, the more we are not going to be willing to interact with black boxes that just tell us what
to do without ever telling us why.”13
Copyright © 2019 Accenture. All rights reserved. 6
 The new European Banking Authority Guidelines on loan origination and monitoring require banks to perform sensitivity
analysis to test the sustainability of counterparties, simulating adverse conditions, considering both market and idiosyncratic
variables
 Banks that use advanced technologies for credit supply processes should take into consideration the risk deriving from these
technologies (e.g. bias deriving from Machine Learning models) in their risk management frameworks and be able to adequately
govern the outcomes of the models for strength
THE NEW REGULATORY GUIDANCE14
INTERPRETABILITY
ACCURACYINESTIMATION
NEURAL NETWORK: DEEP
LEARNING
RANDOM FOREST
SVM
DECISION TREES
LOGISTIC
REGRESSION

7. EVOLUTION OF CREDIT RISK
USE CASES EXAMPLES
Copyright © 2019 Accenture. All rights reserved. 7
ESTIMATE&
VALIDATIONOF
INTERNAL MODELS
Inclusion of new types of
data for model estimation
Standardization and
efficiency of repetitive tasks
by dedicating internal
resources to challenge
activities
STRESS
TESTING
 Development of multiple
scenarios in an
automated and objective
way
 Increased objectivity in
defining scenarios
2nd LEVEL
CONTROLS ON CREDIT
 Reduction of control
execution times
 Error reduction and
performance increase
EARLY
WARNING
 Automation of a large
number of indicators,
enhancing the predictive
performance of the
model
 Use of social data for the
preventive interception
of anomalies
STRESS
TESTING
EARLY
WARNING
VALIDAZI
ONE
CONTROL
LI DI II
LIVELLO

8. SFIDEOpportunity to use different data
(e.g. data on real estate values of external
companies)
Adding information through deep learning techniques (e.g.
reading the financial statements’ notes)
Role of Open Banking (real-time knowledge of all system
customer information, not just liabilities as a risk center)
BACKTESTING JUST IN TIME
EVOLUTION OF CREDIT RISK
ESTIMATE AND VALIDATION OF INTERNAL MODELS
OUTPUT
Δ ≥ X%
Y% ≥ Δ ≥ X%
Δ ≤ Y%
TEST OK
TEST KO
TEST OK
APPLICATION
MODEL
APPLICATION
BACKTESTING
SAMPLE
ANALYSIS
MACHINE LEARNING
& ROBOTICS
Automatic quantitative
analysis of model
performance based on
adaptive Machine Learning-
based tools
BENCHMARK
PROBABILITY OF
DEFAULT MODEL
SUPERVISED MACHINE LEARNING
BENCHMARK MODEL
1 2 3 4 5 6 7
% DR PD
PROBABILITY
OF DEFAULT
MODEL
BENCHMARK SCALE
UNSUPERVISED MACHINE LEARNING
BENCHMARK RATING SCALE
DEVELOPMENT DATA
BENCHMARKING
TRAINED
MACHINE
Analysis of model documentation based on what has been
learned from historical / regulatory documentation
ANALYSIS OF MODEL REPORTS
FINAL REPORT
ANALYSIS OUTCOME
 COMPLETENESS OF THE TOPICS COVERED
 Description of the Section Template
 Input Date
 Defining Default Section
 …
 COMPLIANCE OF THE DOCUMENT STRUCTURE WITH
THE LEGISLATION
 …
Copyright © 2019 Accenture. All rights reserved.
DOCUMENT ANALYSIS
USE OF NEW INFORMATION
8

9. The introduction of automation and Machine Learning methodologies can lead to important efficiency in the credit monitoring process,
significantly improving the predictive performance of Early Warning models
EVOLUTION OF CREDIT RISK
EARLY WARNING MODELS
9
 High number of experientially
defined indicators
 Different levels of severity
assigned by experts
 High % of false positives in the
face of few correct ignitions
INITIAL SITUATION ADVANTAGES
 Significant decrease in the number
of early warning indicators used
 Affiliation of indicators to the
appropriate level of severity
 Significant decrease in false
positives
 Significant increase in correct
indicator switch on
SERIOUSVERY SERIOUS
LIGHT STANDARD
METHODOLOGY
MACHINE EARNING
Application of supervised techniques
(e.g. decision trees) to identify the correct
severity level of each indicator
HEURISTIC APPROACH
Automate the indicator selection
process through a heuristic approach
SOCIAL DATA USAGE
Copyright © 2019 Accenture. All rights reserved.
Inclusion of new data sources
via social media - indicative of
reviews and customer trends
• Deep learning techniques that through analysis and
reading are able to extrapolate information from large
amounts of text
• Predictive analytics techniques to assess the
correlation between web info and customer
creditworthiness
HOW?
NEW TREND

10. Machine Learning and Robotic Process Automation techniques find numerous applications in the perimeter of 2nd level controls, from
Key Risk Indicators identification to control development and automation
AUTOMATED
COLLECTION OF
INFORMATION FROM
APPLICATIONS
AUTOMATED FILL IN
OF THE CONTROL
REPORT
AUTOMATED
PRODUCTION OF
THE SUMMARY
CONTROL REPORT
MACHINE LEARNING ROBOTIC PROCESS AUTOMATION
 ENHANCING THE PREDICTIVE EFFECTIVENESS OF
ADOPTED STATISTICAL TECHNIQUES
 ENHANCING THE COMPUTATIONAL / ANALYSIS
CAPACITY
 ENHANCING THE INFORMATION CONTRIBUTION OF
EACH VARIABLE
 REDUCTION OF TIME / COSTS OF THE CONTROL
PROCESS
 DISPLACEMENT OF RESOURCES FROM «REPETITIVE»
ACTIVITIES TO ACTIVITIES THAT ENHANCE “IT” AND
INCREASE SKILLS
 REDUCTION OF OPERATIONAL ERRORS
ADVANTAGES
ADVANTAGES
Recourse to supervised / unsupervised
Machine Learning techniques for:
IDENTIFICATION AND
ANALYSIS OF VARIABLES
AUTOMATIC SAMPLE
EXTRACTION
EVOLUTION OF CREDIT RISK
2ND LEVEL CONTROLS ON CREDIT
Copyright © 2019 Accenture. All rights reserved. 10

11. EVOLUTION OF CREDIT RISK
STRESS TESTING
 Current stress testing approaches are grounded in models based on a priori assumptions about the relationships between market variables and
plausible idiosyncratic variables
 A Machine Learning solution could allow greater effectiveness and robustness of stress testing approaches
SCENARIOS DEFINED
FROM THE LIMITS OF
CURRENT APPROACHES
TO THE NEW APPROACHES OF STRESS TESTING
MACHINE LEARNING
Copyright © 2019 Accenture. All rights reserved. 11
 MODEL HYPOTHESES
The robustness of current
approaches is based on formalized
hypotheses for the purposes of the
models
 NON-LINEAR RELATIONSHIPS
They are usually not properly
captured by models
 HISTORICAL EVENTS
The past is not always applicable to
the present
 IDENTIFYING BIASES
The distortions of current models
are not easily identifiable
MODEL TRANSLATION
Supervised machine learning + expert judgment
MODEL VALIDATION
Historical
data
backtesting
DATA GRANULARITY& GOVERNANCE
METHODOLOGICAL FLEXIBILITY
Data management
Data quality
Data ownership
Data flows
 METHODOLOGIC FLEXIBILITY
Machine learning approaches allow firms to independently identify
the causal structure of the relationships between input variables
(even non-linear) without formal a priori hypotheses
 DATA CENTRALITY
Machine learning approaches require a large amount of granular
(idiosyncratic and market) data input, so data management is crucial
 INTERPRETATION OF RESULTS
The results of the machine learning models should be interpreted by
experts to identify plausible scenarios
 VALIDATION OF MODELS
Validation plays a crucial role in maintaining the robustness of the
model framework and outcomes
THE COMPONENTS OF
THE MACHINE LEARNING
APPROACH
Credit portfolio
Client ID
Stipulation
Payment

12. CONCLUSIONS AND NEXT STEPS
A NEW ERA FOR CREDIT RISK ?
Copyright © 2019 Accenture. All rights reserved. 12
Regulators are starting to incorporate these innovations into new legislation by outlining new features aimed at
improving and enhancing credit analysis through the use of more advanced statistical methods and enhancement
techniques that can evaluate both new information sources and data in «real time»
In recent years, technological developments have undergone in-depth analysis among banks, but we are still far
from attaining mature levels both at the methodological and at the credit granting, monitoring and control process
levels
All banking functions that participate in the credit cycle (Lending, Risk, Workout) and Control functions
should be equipped with tools and capabilities to:
 exploit the methodological / modeling potential even in more managerial areas, not just the regulatory space
 rethink operational practices through the application of various levels of Machine Learning and Artificial
Intelligence sophistication to improve operational efficiency
With the increase in models devoted to bank management, the impact of model risk should increase and banks
should equip themselves with new and more structured Model Risk frameworks to manage new Machine Learning
model validation paradigms

13. Stefano Bonini, PhD, CStat, PStat®
stefano.bonini@accenture.com
Giuliana Caivano, PhD
giuliana.caivano@accenture.com
CONTACTS
Copyright © 2019 Accenture. All rights reserved.
13

14. REFERENCES
1. “Computing Machinery and Intelligence,” A.M. Turing, UMBC, 1950. Access at: https://www.csee.umbc.edu/courses/471/papers/turing.pdf
2. “A proposal for the Dartmouth summer research project on Artificial Intelligence,” J. McCarthy, August 31, 1955. Access at: http://www-
formal.stanford.edu/jmc/history/dartmouth/dartmouth.html
3. “The Fraud Examiner – AI in the fight against fraud,” M. Wilder, Association of Certified Fraud Examiners, (1990). Access at:
https://www.acfe.com/fraud-examiner.aspx?id=4294999437
4. “Kensho: The Financial Answer Machine,” L. Shin, Forbes, December 9, 2015. Access at: https://www.forbes.com/sites/laurashin/2015/12/09/kensho-
the-financial-answer-machine/#7fe18e4ef310
5. “An AI Completed 360,000 Hours of Finance Work in Just Seconds,” D. Galeon, Futurism.com, March 8, 2017. Access at: https://futurism.com/an-ai-
completed-360000-hours-of-finance-work-in-just-seconds
6. “What is Google Duplex? The smartest chatbot ever explained,” E. Rawes, Digital Trends, April 3, 2019. Access at:
https://www.digitaltrends.com/home/what-is-google-duplex/
7. “Computing Machinery and Intelligence,” A.M. Turing, UMBC, 1950. Access at: https://www.csee.umbc.edu/courses/471/papers/turing.pdf
8. “Story of ELIZA, the first chatbot developed in 1966,” M. Salecha, Analytics India Magazine, October 5, 2016. Access at:
https://www.analyticsindiamag.com/story-eliza-first-chatbot-developed-1966/
9. “Garry Kasparov and the game of artificial intelligence,” M. Sollinger, PRI, January 5, 2018. Access at: https://www.pri.org/stories/2018-01-05/garry-
kasparov-and-game-artificial-intelligence
10. “Robots beat humans in trading battle,” BBC News, August 8, 2001. Access at: http://news.bbc.co.uk/2/hi/business/1481339.stm
11. “UBS Turns to Artificial Intelligence to Advise Clients,” J. Vögeli, Bloomberg, December 7, 2014. Access at:
https://www.bloomberg.com/news/articles/2014-12-07/ubs-turns-to-artificial-intelligence-to-advise-wealthy-clients
12. “Why Amazon Alexa Is Always Listening To Your Conversations: Analysis” J. Su, Forbes, May 16, 2019. Access at:
https://www.forbes.com/sites/jeanbaptiste/2019/05/16/why-amazon-alexa-is-always-listening-to-your-conversations-analysis/#36291ac72378
13. “Machine learning hits explainability barrier,” D. DeFrancesco, Risk.net, November 6, 2018. Access at: https://www.risk.net/risk-
management/6008221/machine-learning-hits-explainability-barrier
14. “Draft Guidelines on loan origination and monitoring,” European Banking Authority, Consultation Paper, June 19, 2019. Access at:
https://eba.europa.eu/documents/10180/2831176/CP+on+GLs+on+loan+origination+and+monitoring.pdf
Copyright © 2019 Accenture. All rights reserved. 14

15. MEASURING AND MANAGING CREDIT RISK
WITH MACHINE LEARNING & ARTIFICIAL
INTELLIGENCE: A NEW ERA?
About Accenture
Accenture is a leading global professional
services company, providing a broad range
of services and solutions in strategy,
consulting, digital, technology and
operations. Combining unmatched
experience and specialized skills across
more than 40 industries and all business
functions—underpinned by the world’s
largest delivery network —Accenture works
at the intersection of business and
technology to help clients improve their
performance and create sustainable value
for their stakeholders. With more than
482,000 people serving clients in more than
120 countries, Accenture drives innovation
to improve the way the world works and
lives. Visit us at www.accenture.com
Disclaimer
This presentation is intended for general
informational purposes only and does not
take into account the reader’s specific
circumstances, and may not reflect the
most current developments. Accenture
disclaims, to the fullest extent permitted
by applicable law, any and all liability for
the accuracy and completeness of the
information in this presentation and for
any acts or omissions made based on
such information. Accenture does not
provide legal, regulatory, audit, or tax
advice. Readers are responsible for
obtaining such advice from their own
legal counsel or other licensed
professionals.