Be aers-fara-modellinginsolvency-nov2010

Modelling in Solvency II:
from regulation to practice

November 2010

© 2010 Deloitte

Solvency II Processes: What are we talking about?
As depicted hereunder, the Solvency II value chain is built upon
complex processes and information flows:
Data
Preparation

Lif e
Liabilities
Model Point

Asset Data
External Market
Data
Accounting Data

Counterparty
inf ormation
Operational
Losses

Assets
Model Point

Experience
analysis and
assumptions
setting

Reporting

MVL & Tech
Provisions Lif e
Lif e Insurance
Risk
Cash Flow
projection Lif e
Coherence control, validation and sign of f

Non lif e / Health:
Contracts and
claims

Coherence control, validation and sign of f

Economic
Scenario
Generation

Extract Transf orm and Load

Lif e:
Contracts and
claims

Conso. and
aggreg.

Calculation

Cash Flow
projection
Assets
Cash Flow
projection
Non-lif e /
Health

Asset
Pricing
Market
Risk

Non-lif e and
Health Risk
MVL & Tech
Provisions Non
lif e and Health
Counterparty
Def ault Risk

SFCR &
RTS Group

Validation and sign of f

Data
Collection

Risk Class aggregator and consolidation

Data sources

SFCR &
RTS Solo

Internal
Reports

Operational
Risk

Results:
SII Balance
sheet, Capital
Requirements

Monitor and Control
2

Modelling in Solvency II: from regulation to practice

© 2010 Deloitte

What are the Solvency II options for the calculation of
the SCR?
5 options can be envisaged which will influence the level of complexity and the
Complexity,
accuracy of the implemented SCR calculations.
5
Accuracy
4
3
2
Complexity,
Accuracy

1

Standard
Formula

Simplified
calculation in
the Standard
formula

Standard
Formula
using own
data to
calibrate the
parameters
in the risk
modules

Internal
Model

Partial
Internal
Model

And of course, don’t forget Technical Provisions …
3


© 2010 Deloitte

How effective do you think your organization is in managing
each of the following types of risks?
Percent responding extremely or very effective
0%

10%

20%

Liquidity
Regulatory/compliance
Market
Credit
Legal
Budgeting/financial
Tax
Mortality
Liability management
Privacy
Morbidity
Business continuity/IT security
Country/sovereign risk
Fraud
Property and casualty
Reputation
Strategic
Operational
Lapse
Data integrity
Vendor/service provider
Human resource
Model
Systemic
Catastrophe
Geopolitical

30%

40%

50%

60%

70%

80%

90%

77%
76%
74%
71%
71%
71%
71%
64%
64%
62%
60%
56%
54%
54%
53%
49%
48%
47%
45%
44%
44%
43%
41%

37%
37%
36%

7th Edition Global Risk Management Survey Review, Draft Report

4


© 2010 Deloitte

Avatars of model risk
There are many aspects to model risk. For instance:
•

The model design, and the choices made in representing reality, do not correspond
to the objectives of the model. I.e. some essential features of the economic reality or
of the contracts modelled, are not captured.
‒

•

Somewhere along the lines of the mathematical developments, a mistake was
made. Describing the setting correctly is no guarantee for a correct solution …
‒

•

Correlations, length of the liquid market curve, unlisted equity, historical loss data, mortality trends,…

The model’s implementation in a software environment, does not correspond to the
specifications
‒
‒

5

A model developed for valuing a complex financial instrument for accounting purposes, where there is a
materiality concept and where timeliness is key, may not be the best guide in setting up the hedge, since
it is probably a simplified model.

Absence of accurate and adequate input data leads to non-sensical output
‒

•

Using results that only hold for a normal distribution in credit or operational risk modelling

A model is used for a purpose it was not designed for. For instance, a model may
only work for a specific range of market parameters, but is applied outside that range
‒

•

Pricing model for mortgages that ignores prepayment risk; valuing a life insurance company while ignoring
the cost of guarantees, correlation effects in defaults,…

Numerical instability e.g. Excel has its limits
Software bugs, version control, use of libraries from vendors,…


© 2010 Deloitte

Agenda
1
2

Some ideas on model governance

3

Adding value through model validation

4

6

Solvency II Internal model requirements

Model documentation


© 2010 Deloitte

How will internal models be approved?
Internal model tests (principles-based approach)
To gain sign-off an internal model needs to pass six tests which need to be repeated when
the model is changed to enable on-going appropriateness of the model. Where the
application for that approval relates to a partial internal model, the requirements shall be
adapted to take account of the limited scope of the application of the model.
The tests explicitly refers to economic capital under Pillar I.
Use test
(Article 120)

Demonstrate that the internal model is widely used and plays an important role in system governance , in particular in the
risk-management system en decision-making processes, the economic and Solvency Capital assessment and allocation
processes, including ORSA

(Article 121)

Demonstrate that internal model complies with adequate actuarial and statistical techniques and data quality requirements.
Verify it assesses all material risks the company is exposed to and the mitigation actions for these risks taking into account
policyholders and management actions using realistic assumptions.

Calibration
standards

Demonstrate calibration details of the internal model and verify the reconciliation to regulatory standard i.e. the level of
protection within 1 year being at 99.5% confidence interval.

Statistical quality
standards

(Article 122)

Profit and loss
attribution

Demonstrate that the causes and sources of profit and losses for each major business unit are reviewed at least annually
and verify how categorisation of risks chosen in the model explains the causes of profits and losses.

(Article 123)

Validation
standards
(Article 124)

Documentation
standards
(Article 125)

7

Demonstrate that there is a regular cycle of model validation that includes monitoring performance, appropriateness of
specification and testing results against experience.
Document the design and operational details of the internal model to provide a detailed outline of theory, assumptions and
mathematically and empirical basis underlying the model and indicate any circumstances where the model does not work
effectively.


© 2010 Deloitte

Internal Model

Insurers have the choice of:
• Standard model
• Partial internal model
• Full internal model

• Interaction and
integration
‒ Reserving
‒ Pricing
‒ Planning
‒ Management information
• Parameters
‒ Frequency
‒ Interaction and
consistency
‒ Ownership
‒ Sign-off

8

Risk Strategy
• Risk appetite
• Understanding risk impacts
• Accountability

Decision Making
• Pricing
• Capital allocation
• Reinsurance
• Planning


Internal
Model

Information
• Data
• Assumptions
• Technology

Management
• Decision Structure
• Remuneration Policy
• Responsibilities

© 2010 Deloitte

Internal model
Requirements for Approval

Is the internal model widely accepted within
the organisation and does it capture the
fundamental risks?

Is the internal model
adequately documented?

Is there an ongoing feedback
loop between the administrative
and management board (AMB)
and the risk management
function?

Use test

Documentation
standards

Does the internal model
produce the proper
results?

Model
Governance

Are the quality of the data,
assumptions and methods
used sufficient?

Approval
criteria

Statistical
quality
standards

Validation
standards

Profit and loss
statement

Can the realised gains and losses be explained
using the internal model?
9


Calibration
standards

produce the same level
of prudence as the
standard formula?

Solvency II Directive, Articles 120 - 126
CEIOPS Advice on Level 2 Implementing measures (former CP 56)

© 2010 Deloitte

Use Test

Is the internal model widely accepted within
the organisation and does it capture the
fundamental risks?

General Principles

… is designed to
facilitate the
analysis of
business
decisions

… is well
understood by
senior
management

… is used to
improve the risk
management
system

… is appropriate
in the business
model

The
internal
model
…

… is widely
integrated into
the risk
management
system

… is used to
calculate SCR

… is used to
support and
verify decision
making

10

… covers risk
sufficiently to be
useful in risk
management and
decision making


... is integrated
into the risk
management
system on a
consistent basis
for all uses

Solvency II Directive, Articles 120
CEIOPS Advice on Level 2 Implementing
measures (former CP 56)
© 2010 Deloitte

Use Test
Some examples on the use of the internal model
An important aspect of the use test is the interaction between strategy, capital
and risk appetite
• Business strategy
should be formulated
and implemented in
daily operations.
• The risk strategy has a
direct impact on capital
requirements

Strategy

Capital

Risk
appetite

• Volatility of capital is
required to withstand
unexpected losses. The
volatility depends on the
adopted risk management
strategy.
• A high risk profile requires
more capital, with the
possibility of a higher return
on capital. An insurer should
understand their risk
appetite and the resulting
capital requirements.

• The risk appetite should be determined and be related to the strategy and solvency position. The
appetite depends on available capital and the willingness to take more risks in exchange for a
higher expected return.
• The risk appetite depends on the risk-taking capacity in terms of both capital and organisation
11


© 2010 Deloitte

In which areas do you plan to use your internal model as
part of the decision-making process for Solvency II?
Risk-based performance reporting

87%

Capital management and planning

87%

Management information, e.g., providing information on how the risk position compares…

80%

Decisions on asset mix strategy and the possible effects of investment decisions

67%

Strategy and planning, e.g., as an input to planning and strategy by providing an …

67%

Analysis, design or purchase of reinsurance

60%

Pricing of business

60%

Assessment of the risks, value and impact to the business of potential mergers,…

53%

Product development

47%

Prioritization of risk management activity

40%

Purchase of hedging assets or changes to existing hedges

33%

Excess surplus investigations (with profit funds)

33%

Executive compensation

20%
0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


12


© 2010 Deloitte

Use Test
Some examples on the use of the internal model

Governance system
• Connection between
the internal model
and technical
provisions
the internal model
output and internal
and external
reporting
the internal model
and the technical
implementation of
management actions

Risk management
system
• Measurement of
material risks
• ALM
• External risk
reporting

Decision Making

Capital Assessment
and Allocation

• Investment Decisions

• ORSA

• Identifying targets for
return on capital and
rewards

• Calculation of SCR

• Internal risk reporting

• Product development
and pricing

• Draft reinsurance
program

• Business planning
and strategy

• Development of risk
strategies

• Company policy for
taking on risk

• Efficient use of
Capital

• Economic capital
allocation for each
entity, business line,
risk or business unit

• Capital management

• Solvency capital
allocation by entity,
business line, risk or
business unit

13


© 2010 Deloitte

Statistical Quality Standards

Are the quality of the
data, assumptions and
methods used sufficient?

The Statistical Quality Standards philosophy: an internal model is not just a
black box or an expert with good predictive power for the probability
distribution forecast.
Potential discussion areas:
• Probability distribution forecast (does a probability distribution forecast always has to
consist of a full distribution, i.e. whether every quantile must be known?)
• The use of expert judgement related to data.
• Materiality (circumstances under which risks which are in the scope of the internal
model must actually be modelled in the internal model?)
• Aggregation in internal models
The methods used to calculate the probability distribution forecast shall be based
on:
• Adequate actuarial and statistical techniques
• Consistency of calculation methods used for the probability distribution forecast and
technical provisions
• Current and credible information
• Justification of underlying assumptions
14


© 2010 Deloitte

Statistical Quality Standards

Are the quality of the
data, assumptions and
methods used sufficient?

• Risk ranking & model coverage
‒ Relative importance of risks, comparables, relative classification
‒ Link with use test: coverage, resolution, consistency,...

• Recognition of diversification effects
‒ What drives dependence? Is there diversification gain?
‒ Account for extreme scenarios and tail dependence
‒ Part of business decisions?

• Recognition of risk mitigation
‒ Economic view, but legal effective/enforceable?
‒ How incorporated in model?

• Financial Guarantees and options
‒ Identify and document
‒ Influencing factors
‒ Coherence with TP

• Future management actions

15


© 2010 Deloitte

Calibration Standards

produce the same level of
prudence as the standard
formula?

The Calibration Standards aim to assess whether the SCR derived from the
internal model has the appropriate level of prudence.
Article 122.1
Insurance and reinsurance undertakings may use a different time period or risk
measure than that set out in Article 101(3) for internal modelling purposes as long as the
outputs of the internal model can be used by those undertakings to calculate the
Solvency Capital Requirement in a manner that provides policy holders and
beneficiaries with a level of protection equivalent to that set out in Article 101.
Article 101(3)
The Solvency Capital Requirement shall be calibrated so as to ensure that all
quantifiable risks to which an insurance or reinsurance undertaking is exposed are taken
into account. It shall cover existing business, as well as the new business expected to be
written over the following 12 months. With respect to existing business, it shall cover only
unexpected losses.
It shall correspond to the Value-at-Risk of the basic own funds of an insurance or
reinsurance undertaking subject to a confidence level of 99,5 % over a one-year period.
16


© 2010 Deloitte

Calibration Standards

produce the same level of
prudence as the standard
formula?

Different time period and risk measure:
• If the time period used is different from the one set out in Article 101:
‒ Demonstrate that the time effects of the risks are taken into account
‒ Demonstrate that all significant risks over a one-year-period are properly managed
‒ Give special attention to the choice of the data used
‒ Justify the choice of time horizon in view of the average duration of the liabilities of the
undertaking, of the business model and of the uncertainties associated with too far time horizons
• Consider the practical implications if the undertaking uses different time periods and/or risk
measures within the same internal model, especially when aggregating the capital for the different
risks.
• When taking this approach, the undertaking shall show that the level of protection provided by
the SCR is equivalent to that set out in Article 101(3), and specifically that the approach taken to
aggregate the risks is appropriate.
Equivalent protection of policy holders:
• If the SCR cannot be derived directly from the probability distribution:
‒ Explain how risks are rescaled and justify that the bias introduced when doing so is immaterial.
‒ Explain the shortcuts used to reconcile the outputs of its internal model with the distribution of
the Basic Own Funds, if any.
‒ If considering a longer time horizon than that set out in Article 101(3), show due consideration of
the solvency position at the earlier time horizons.
‒ If considering a different time horizon than that set out in Article 101(3), justify the particular
assumptions made in order to properly take into account the dependencies between
consecutive time steps.
17


© 2010 Deloitte

Profit and loss attribution

Can the realised gains and
losses be explained using the
internal model?

The Profit and Loss attribution aims to demonstrate how the categorisation
of risk chosen in the model explains the causes and sources of profits and
losses.
Profit and loss attribution and the Use test
• The P&L attribution will give the undertaking information relating to the risk profile of the
undertaking, and therefore this information is also used in the ORSA.
• The P&L attribution may also provide an unbiased view on the risks of the portfolio to better
understand the portfolio exposures and assess whether the risk management framework is
appropriate.
• The results of the P&L attribution can also be used for other internal purposes such as
budgeting, forecasting, reinsurance-program testing.
Categorisation of risks
• The economic capital requirements resulting from the internal model shall lead directly to a
categorisation of all material risks. The qualitative assessment of non-material risks or nonquantifiable risks completes the categorisation of risks based upon the internal model results.
Form of profit to be taken
• The variable may differ from basic own funds, because a different internal definition may be used
for economic capital resources.
• Be aware what causes differences in the profits and losses used in the Profit and loss attribution
and the profits and losses reported in the accounting systems.
18


© 2010 Deloitte

Agenda
1
2


3


4

19


Model documentation


© 2010 Deloitte

Ongoing feedback loop

Internal Model Governance under SOLVENCY II regulation
The administrative or management body (high level internal model governance):
• Approving the application for approval to use the internal model to calculate the SCR, and the application for approval
for major changes or extensions to the model.
• Deciding roles and responsibilities for the internal model governance.
The administrative,
• Deciding on the strategic direction of the model and hence any changes to the model.
management or
• Agreeing major changes in advance of the change being made.
supervisory body may
• Aligning the model design and operations with the undertaking’s risk profile and operations.
set up an internal
control committee
• Ensuring there are sufficient resources to develop, monitor and maintain the model.
• Monitoring on-going compliance with the requirements for internal model approval, and informing the
supervisory authorities if the model ceases to comply.
• Ensuring there are adequate independent review procedures in place around the internal model design, operation and
validation.
• Ensuring that outputs are aligned with use – i.e. that the management information produced by the model assists in
decisions made at Board level.
• If the internal model ceases to comply with the requirements for approval, the administrative or management body
must ensure that a plan to restore compliance is developed or assess the non-compliance as immaterial.
Undertakings may use external validation
The risk management function (detailed internal model governance):
review, however the ultimate responsibility
• Design and implementation of the internal model.
for signing off the appropriate validation
• Testing and validation of the internal model.
processes shall fall on the board, and may
not be delegated to any third party.
• Documentation of the internal model and any changes to it.
• Analysing the performance of the internal model, and reporting on the performance to the high-level
governance, including compliance with the internal model approval requirements.
• Suggesting areas for improvement and reporting on the status of efforts to improve previously identified weaknesses
to the high level governance.
• Liaise closely with users of the outputs of the internal model.
• Develop a communication loop with the actuarial function to pass the detailed actuarial perspective to the risk
management function and in return receive the insights on the internal model.

20


© 2010 Deloitte

Conceptual considerations
•

Model Governance should support the
achievement of at least two objectives:
‒
‒

•

The orderly development and usage of models
The containment of model risk in the organisation

Some criteria that can be used to prioritise the
need for model governance in the universe of
models discussed before:
‒

Regulatory expectations (explicit and implied)

‒

Stakeholder expectations
•
•

Credit rating agencies

•
‒

Counterparts like reinsurers, dealing rooms,...
Clients (e.g. Investment models)

Risks for the group
•

Financial risks; e.g. due to mispricing,
mishedging

•

Reputational risks

•

Materiality concept to be incorporated!

‒
‒

21

The assessment of model risk, as hinted at in the
previous section
Cost of installing a rigorous process vs. the
expected benefits


FDIC, Supervisory Insights, Winter 2005, Vol. 2, Issue 2

.

© 2010 Deloitte

A bit more pragmatic thinking
•

Any fair value or risk model is composed of three
ingredients:
1.

A mathematical recipe, the model

2.

Position data/contract data

3.

Model data (e.g. market data such as interest rates; lapse
rates; correlations, behavioural parameters,...)

Market & behavioural
data

Contract data

Reasonably
easily
verifiable

Fair Value
Or
Risk Measure

Ease of control
depends on
market or risk
type

Similar set-up seems possible for most models.

•

•

Of these three ingredients, only the mathematical part
can be legitimately be claimed to require dedicated
expert skills.

A mathematical recipe

Least
accessible to
broad user
base

A tentative role for Finance Department:

‒

Risk department often is a knowledge centre for
mathematical modelling within the organisation (so, it can
act as internal consultant, even if it is not the
“owner”/”responsible” for the model)

•

‒

Under Solvency II, the risk department gets the model
validation role explicitly

In Risk Models
‒ Ensuring accurate and complete
capture of exposure data
‒ Back-testing any performance
predictions coming from the risk
models

‒

Setting of all sorts of parameters is a management decision,
and should remain their ultimate responsibility

•

‒

22

Different departments have different skills/roles relevant
for each of the above ingredients:

The gold copy of any contractual data will mostly be the
financial accounts (since audited, and rigorous internal
control procedures mostly apply to them)

In Fair Value models
‒ Validate consistency of parameters
across the group
‒ Understand sensitivity w.r.t. to
parameters


© 2010 Deloitte

Conceptual considerations
•

It would be best practice to ensure that there is
always a 4-eyes principle applied. The rigour with
which this is applied should be function of the
regulatory expectations, the risk the model presents
to the company,...

•

In terms of controlling model risk, there are at least
two control layers:
‒

‒

•

23

MODEL VALIDATION: model validation is a formal
process, that challenges the assumptions, mathematical
algorithm, the implementation in a tool of a model.
MODEL REVIEW: a set of periodic activities, not
necessarily requiring in-depth mathematical skills, but
that are confronting the “predictions” of the model with
reality. E.g. realised losses or earnings over the first
quarter, prices realised in actual transactions, excess
losses beyond the VaR amount,... Such controls can
provide indications that the model is not reliable.

Requires in-depth technical modelling
skills, so mostly concentrated in an expert
centre, often within the risk department

Often performed by professionals close to
the business, like performance
controllers,….

The question of scope. Which models will you include
in such a formal process?


© 2010 Deloitte

An example: ALM model
Potential allocation of responsibilities related to ALM modelling and ALM
management
What

Who Performs

Who Controls (second line of defense)

Development of the model

Risk department

Independent validation by a model validation team or
external third party

Running the model (including
setting of model data and
uploading contract data)

Balance sheet management/ALM
department

• Model data subject to challenging validation of the
Risk department (formal sign off)
• Position data subject to formal reconciliation
process (documented checklist) at each run,
possibly in collaboration with finance department
Periodically reviewed by internal audit

Analysis of the results and
suggestions for management
action

department

• Formally reviewed by Risk department before
being presented to ALCO/MT Finance

Analysis of new balance sheet
management
proposals/instruments

department

Validation and challenge by Risk department

Back testing of earnings
forecasts by reconciliation with
realised

Finance department (financial
controlling)

NA

24


© 2010 Deloitte

Agenda
1
2


3


4

25


Model documentation


© 2010 Deloitte

Introduction to the validation cycle
In designing the internal model, the Level 1 Text allows a large amount of freedom. Two examples of
this are given below:
a.
b.

Article 121(4) sets out that “No particular method for the calculation of the probability distribution forecast shall be
prescribed.”
Article 122(1) sets out that “Insurance and reinsurance undertakings may use a different time period or risk
measure than that set out in Article 101(3) for internal modelling purposes as long as the outputs of the internal
model can be used by those undertakings to calculate the Solvency Capital Requirement in a manner that provides
policy holders and beneficiaries with a level of protection equivalent to that set out in Article 101.”

In addition to being used for the undertaking’s own risk management purposes, the internal model is
also used to calculate the regulatory capital. Materially misstating the regulatory capital, especially
holding regulatory capital that is not high enough, will result in a decrease in the level of the policy holder
protection provided by the undertaking.
In addition, as the internal model should also be widely used in and play an important role in the system
of governance, any material misestimation within the internal model will affect not only policy holder
protection, but also the whole risk management and decision making processes of the undertaking.

Therefore, supervisory authorities will require undertakings to take appropriate steps to validate
that the internal model is appropriate for the calculation of regulatory capital to ensure that the
level of regulatory capital is not materially misstated so as to decrease the level of the policy
holder protection, for use within the undertaking’s risk management and decision making processes.
It is the undertaking which has the primary role in this validation process, not the supervisory
authority. The validation process ≠ the approval process, in respect of which the supervisory authority
needs to take a decision. As part of the approval process, the supervisor will need to evaluate the
validation processes which the undertaking has in place.
26


© 2010 Deloitte

Model validation function
Model validation is a key activity to help make sure that models function as
intended, both when they are implemented and over time. Ongoing monitoring
and validation of risk management models are important in order to assess a
model’s sensitivity to structural changes and to changes in parameters and
assumptions. Fifty-nine percent of institutions reported having a model
validation function, an increase from 53 percent in 2008 . Larger
institutions were more likely to have a model validation function, with 79
percent institutions with more than $100 billion in assets have such a
function, up from 66% in 2008.
Model validation is most often placed in an independent risk management
function. Among institutions with a model validation function, 65 percent reported
that this model validation resides in independent risk management, while 19
percent placed it in Internal Audit and 8 percent in Actuarial. Larger institutions
were even more likely to have risk management handle model validation. Among
institutions with more than $100 billion in assets, 77 percent said that model
validation was placed in independent risk management.
27


© 2010 Deloitte

The validation policy
The validation policy which sets out the way in which undertakings will validate
their own internal model and why that way is appropriate.
How to demonstrate compliance:
The validation policy shall set out at least the following items:
• Purpose and Scope of Validation: the areas of the internal model that need to be validated shall
include at least:
‒
‒
‒
‒
‒
‒
‒
‒

Data
Methods
Assumptions
Expert judgement
Documentation
Systems/IT
Model governance
Use test

• Tools used: may be a mathematically well defined test, a qualitative judgement or any other
process designed to gain comfort that the internal model is appropriate and reliable.
‒ Some validation tools will need to be used by all undertakings using an internal model to calculate the SCR:
•
•
•
•

Testing results against experience
Testing the robustness of the internal model (shall include at least sensitivity testing)
Stress and scenario testing
Profit and loss attribution

‒ Examples of what further validation tools undertakings may want to use:
•
•
•
•
28

Benchmarking
Analysis of change
Hypothetical portfolio
Qualitative reviews


© 2010 Deloitte

The validation policy (cont.)
• Frequency of validation process
• Governance of validation results
‒ The risk management function shall be tasked with the validation of the internal model.
‒ Nevertheless, certain parts of the validation process may be carried out by other parts of the undertaking, as
long as there are clear lines of reporting and the risk management function retains overall responsibility for the
validation process.
‒ Set out how the senior management is involved in the validation processes
‒ Consideration needs to be given to what is validated at group level and what is validated at the related
undertaking level

• Limitations and future developments: expect that the model and its associated validation
processes will constantly develop as the risk profile of the undertaking develops and as new
validation tools become available
• Documentation of the validation policy
• Independent review: set out how independent review, external or internal, is being used within the
validation process

29


© 2010 Deloitte

Internal Model Change Policy

Model Change Policy: The development of a model change policy is a requirement on
all undertakings applying for approval to use the internal model to calculate the SCR.
This is part of the application and must be approved by the supervisory authorities.
Model changes
• Major changes to the internal model are subject to prior supervisory approval
• The amended internal model should not be used to calculate the SCR until approval is granted
• Undertakings may further assign model changes to the following subcategories:
‒
‒
‒
‒
‒

The calculation kernel of the internal model
The risk management
The internal model governance
The existing and regulatory approved internal model change policy of the undertaking
Other aspects of the internal model

The policy for model changes shall contain an indication of the undertaking internal governance for
changes to the model (e.g. internal approval of changes, escalation path, internal communication,
documentation and validation of changes, etc.).

30


© 2010 Deloitte

Model risk and an extra capital buffer under ORSA?
CEIOPS Level 2: Technical Provisions – Article 86 h Simplified methods and techniques to
calculate technical provisions
3.109 Therefore the undertaking shall assess the model error that results from the use of a given
valuation method, having regard to the nature, scale and complexity of the underlying risks The
valuation method should be regarded as proportionate if the model error is expected to be nonmaterial.
3.110 For this purpose the undertaking should define a concept on materiality which should lay down
the criteria on basis of which a decision on the materiality of a potential misstatement of technical
provisions is made. This materiality concept and should be reflected in the undertaking’s own risk and
solvency assessment (ORSA).
3.112 An assessment of the model error may be carried out, by:
• Sensitivity analysis in the framework of the applied model: this means to vary the parameters
and/or the data thereby observing the range where a best estimate might be located.
• Comparison with the results of other methods: applying different methods gives insight in
potential model errors. These methods would not necessarily need to be more complex.
• Descriptive statistics: in some cases the applied model allows the derivation of descriptive
statistics on the estimation error contained in the estimation. Such information may assist in
quantitatively describing the sources of uncertainty.
• Back-testing: comparing the results of the estimation against experience may help to identify
systemic deviations which are due to deficiencies in the modelling.
31


© 2010 Deloitte

Making model validation concrete and tangible
• The following areas related to an internal model need to be
considered and addressed in the context of a model validation:
Data

Documentation

Methods

Systems/IT

Assumptions

Model
Governance

Expert judgment

Use Test

• The level II CEIOPS also suggests the following validation tools:
MUST CONSIDER
Testing against experience

Benchmarking

Robustness

Analysis of change

Stress & Scenario Testing

Hypothetical portfolios

P&L attribution
32

MAY CONSIDER

Qualitative review


© 2010 Deloitte

ESG Validation Workplan along the above lines (1/4)
Model Review
Questions to address
Reading documentation of the • Is the objective of what needs to be modelled clearly stated?
ESG, and assessment of some • Does the documentation justify the choice of approach?
content questions
• Are the strengths and weaknesses of the chosen approach explained?
• Is part of the documentation readable by an interested, non-technical user,
and will it allow that person to assess strengths and weaknesses of the
approach? I.e. roughly speaking, there would be three levels of
documentation, one for non-technical users, one providing the full
mathematical detail, and one describing the implementation in a tool and the
process to be used to run the model.
• Is the calibration process well-described (market data to be used,
formulae,...) both theoretically and operationally?
• How is the choice for other asset classes (real estate, shares) modelled
based on the interest rate model, justified? How are correlations accounted
for/incorporated?
Review the mathematical
model

Acceptability of the model for
the stated aim

33


• Check all formulae, either by reference to the literature (include in review file)
or by recomputation (document in file). This includes the stochastic
simulation engine, the closed form formulae e.g. used for calibration, the
numerical discretisation schemes,...
• Benchmarking with approach/model chosen in the literature, by peers or in
vendor packages. Collect and store benchmark information.
• How was the choice of model argued in the past? Who approved its use?
Has it been subject to review/challenge by (senior) users?
• Where is the ESG used in practice in the organisation (as compared to
stated objectives)? Is this acceptable given its characteristics? (USE test)

© 2010 Deloitte


Input Testing
Input Data

Stability of the calibration

34


• Review of the procedures used to extract input parameters (e.g. implied or
historical volatilities, zero coupon curve, risk premia) from market data.
• Review of interpretation and acceptance controls of such data when
received from vendors.
• Hull-White type models are calibrated using a minimization on a set of
chosen derivatives for which the price is known (swaptions, caplets).
Impact of choosing a different set of instruments? Impact of choosing a
different seed point for the optimisation?
• How is correlation between stochastic processes linked to observable
parameter of correlation between asset classes?

© 2010 Deloitte


Computational Process
Review of the code

Assessment of the numerical
procedures

35


• Does the code correspond to what was documented in the model?
• Have pre-coded subroutines been tested/assessed or is there general
documentation available? (e.g. random number generation for a given
distribution, numerical minimizers)
• Impact of the discretisation, no creation of fictitious value ("1=1" test,
martingale test)?
• Is the starting yield curve found back as E[Product(exp(-r.dt))]?
• Are the original parameters such as volatilities and correlations that one
calibrated again, found back in the runs?
• Stability for extreme values of parameters (inverted curve, high volatility,
no mean reversion/strong mean reversion, …)?
• Sensitivity relative to the parametrisation/interpolation/extrapolation for the
yield curve to which the mode is calibrated?
• Comparison of the simulated value of an instrument with the closed form
formula (in the chose model) if available?
• Convergence enhancing tricks, such as control variates or anti-thetic
variables.
• Numerical stability of the simulated variables in particular for longer time
horizons or extreme environments (high or low rates, ...)?
• Estimation error in function of the number of simulation runs (e.g. for
known instruments)?

© 2010 Deloitte


Outcome Analysis
Back Test

• Review the implemented back testing procedures. If incomplete, design
and possibly perform some back testing procedures: shape of the
distributions for rates that are generated e.g. the 10y swap rate in 20years;
standard deviation of these distributions,…

Sensitivity analysis

• Controlled assessment of the impact on the output of changing input
parameters. The output here should primarily be interpreted as being the
value or risk measures that one wants to compute with the ESG. This step
may therefore require considering the ESG and the applications that rely
on it, jointly. The purpose is to indentify and separate the important from
the less important input parameters, in order to focus the calibration and
expert judgement controls.
• How are the generated scenarios stored? Are the labels of the data stored
aligned with their mathematical meaning and definition?

Use of scenarios

36


© 2010 Deloitte

Documentation
standards

The Documentation standards aim to instill a
culture of good documentation praxis leading to
all model developments being documented.

The documentation shall demonstrate compliance with the internal model tests.
• Detailed outline of the theory, assumptions, and mathematical and empirical basis underlying the internal model.
• Describe the drawbacks and weaknesses of the model, including the circumstances under which the model does
not work effectively, such as:
‒

‒

Any specific features of the internal model or circumstances or limitations that present potential concerns or would significantly
increase the uncertainty of the results of the internal model beyond what would reasonably be expected. For instance:
• Limitations in risk modelling and the cover of risk captured
• The nature, degree and sources of uncertainty surrounding the results of the internal model and sensitivity to key assumptions
• Shortcoming and/or deficiencies in input data
Insufficiencies in IT-systems, governance and related controls surrounding the internal model.

• Tailored for key bodies and key personnel
• Include evidence such as training that all levels and functions of management, for example the board, senior
management, and the internal audit, of the undertaking understand the relevant aspects of the internal model
• Include a list of all documents that the undertaking considers relevant to the internal model, and where and how
these documents can be accessed.
• Identify those responsible for pulling together and/or updating documents.

The documentation does not have to be one single document, provided there is a list or a mapping
process that brings it all together.
The documentation of an internal model shall be thorough, sufficiently detailed and sufficiently complete
to satisfy the criterion that an independent knowledgeable third party could form a sound judgment as
to the reliability of the internal model and the compliance with Articles 120 to 126 and could understand
the reasoning and the underlying design and operational details of the internal model.

38


© 2010 Deloitte

Documentation: Qualitative Questions from the QIS 5
1. Model description and overview
General Model Governance:
2. Policies, controls and procedures for the management of the internal model
3. Model change policy
4. Evidence of Use Test
5. Training Manual for Management and staff training in use and understanding of
Internal Model
Model Changes:
6. Record of Major and Minor changes of the model
Technological Specifications:
7. Description of the Information Technology platform(s) used in the internal model

8. Description of Contingency plans relating to the technology platform(s) used

9. User guide
10. Source code

39


© 2010 Deloitte

Data:
11. Data policy
12. Documentation evidencing or justifying the accuracy, completeness and
appropriateness of the data
13. Data directory
Statistical Quality Standards:
14. Detailed description of Internal Model Methodology

15. Description of underlying assumptions
Expert Judgement:
16. Description of where Expert Judgement is applied in the model

17. Validation of Expert Judgement as applied in the model
Calibration:

18. If your model output uses a risk measure other than 1-year VaR at 99.5%, do you
have documentation evidencing or verifying that the chosen risk measure is at least as
strong as 1 year VaR at 99.5%?

40


© 2010 Deloitte

Profit and Loss Attribution:
19. Results of Profit and Loss Attribution

Validation:
20. Description and report/results of Validation Tests

Scope:
21. Partial Internal Model Scope
22. Qualitative and Quantitative indicators for the coverage of risk
Other:
23. Description of risk mitigation techniques accounted for in the internal model

24. Description of Future Management Actions accounted for in the internal model

25. Description of known internal model shortcomings / weaknesses, including
circumstances under which the internal model does not work effectively

41


© 2010 Deloitte

Our contact details
Partner, Bedrijfsrevisoren
Yves Dehogne
Audit Partner

Email: ydehogne@deloitte.com
Phone: +32 2 800 20 45

Director, Bedrijfsrevisoren
Dirk Vlaminckx
Audit Director

Email: dvlaminckx@deloitte.com
Phone: + 32 2 800 21 46

42


Director, Actuarial & Financial Risk
Advisory
Arno De Groote
Director, Enterprise Risk Services

Email: adegroote@deloitte.com
Phone: + 32 2 800 24 73

Manager, Actuarial & Financial Risk
Advisory
Christophe Vandeweghe
Manager, Enterprise Risk
Services

Email: cvandeweghe@deloitte.com
Phone: + 32 2 800 24 94

© 2010 Deloitte

About Deloitte
Deloitte refers to one or more of Deloitte Touche Tohmatsu Limited, a UK private
company limited by guarantee, and its network of member firms, each of which is a
legally separate and independent entity. Please see www.deloitte.com/about
detailed description of the legal structure of Deloitte Touche Tohmatsu Limited and
its member firms.
Deloitte provides audit, tax, consulting, and financial advisory services to public
and private clients spanning multiple industries. With a globally connected network
of member firms in more than 140 countries, Deloitte brings world-class
capabilities and deep local expertise to help clients succeed wherever they
operate. Deloitte's approximately 170,000 professionals are committed to
becoming the standard of excellence.

Deloitte in Belgium
A leading audit and consulting practice in Belgium, Deloitte offers value added
services in audit, accounting, tax, consulting and financial advisory.
In Belgium, Deloitte has more than 2,240 employees in over 12 offices across the
country, serving national and international companies, from small and middle-sized
enterprises, to public sector and non-profit organisations. The turnover reached 317
million euros in the financial year 2009.
The Belgian practice is part of the international firm Deloitte Touche Tohmatsu
Limited. For the fiscal year ended 31 May 2009, DDTL realised a turnover of
US$26.1 billion worldwide, with approximately 170,000 members of staff in more
than 140 countries.

This document is confidential and prepared solely for your information.
Therefore you should not, without our prior written consent, refer to or use our
name or this document for any other purpose, disclose them or refer to them
in any prospectus or other document, or make them available or communicate
them to any other party. No other party is entitled to rely on our document for
any purpose whatsoever and thus we accept no liability to any other party who
is shown or gains access to this document.

Be aers-fara-modellinginsolvency-nov2010

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Be aers-fara-modellinginsolvency-nov2010

Similar to Be aers-fara-modellinginsolvency-nov2010 (20)

Recently uploaded

Recently uploaded (20)

Be aers-fara-modellinginsolvency-nov2010