General
1. In order to assess an institution’s compliance with the requirements on the design, management and documentation of rating systems, as referred to in Article 144(1)(e) of Regulation (EU) No 575/2013, competent authorities shall verify all of the following:
(a)
the adequacy of the documentation on the rationale, design, and operational details of the rating systems, as set out in Article 175 of Regulation (EU) No 575/2013, in accordance with Articles 31 and 32;
(b)
the adequacy of the structure of the rating systems, as referred to in Article 170 of Regulation (EU) No 575/2013, in accordance with Articles 33 to 36;
(c)
the application by the institution of the specific requirements for statistical models or other mechanical methods, as referred to in Article 174 of Regulation (EU) No 575/2013, in accordance with Articles 37 to 40.
2. For the purposes of the verification under paragraph 1, competent authorities shall apply all of the following methods:
(a)
review the institution’s relevant internal policies;
(b)
review the institution’s technical documentation on the methodology and the process of the rating systems development;
(c)
review the development manuals, methodologies and processes on which the rating systems are based;
(d)
review the minutes of the institution’s internal bodies responsible for approving the rating systems, including the management body or committees designated by it;
(e)
review the reports on the performance of the rating systems and the recommendations of the credit risk control unit, validation function, internal audit function or any other control function of the institution;
(f)
review the progress reports on the efforts made by the institution to correct shortcomings and mitigate risks detected during monitoring, validations and relevant audits;
(g)
obtain written statements from or interview the relevant staff and senior management of the institution.
3. For the purposes of the verification under paragraph 1, competent authorities may apply any of the following additional methods:
(a)
request and analyse data used in the process of developing the rating systems;
(b)
conduct their own estimations or replicate those of the institution performed during the development and monitoring of the rating systems using relevant data supplied by the institution;
(c)
request additional documentation from the institution or request that it provides analysis related to the choice of methodology for designing the rating system and provides information about the results obtained;
(d)
review the functional documentation of the IT systems relevant to the scope of the assessment of the rating systems design, operational details and documentation;
(e)
perform the competent authority’s own tests on the data of the institution or request the institution to perform tests proposed by the competent authority;
(f)
review other relevant documents of the institution.
SECTION 2 — Methodology for assessing the documentation on the rationale, design and operational details of rating systems
Completeness of the documentation of rating systems
1. When assessing the completeness of the documentation on the design, operational details and rationale of the rating systems as referred to in Article 144(1)(e) and set out in Article 175 of Regulation (EU) No 575/2013, competent authorities shall verify that the documentation is complete and includes the following:
(a)
the adequacy of the rating system and the models used within the rating system taking into account the portfolio characteristics;
(b)
a description of data sources and data cleansing practices;
(c)
definitions of default and loss;
(d)
methodological choices;
(e)
technical specification of the models;
(f)
the weaknesses and limitations of the models and possible mitigating factors thereof;
(g)
the results of the implementation tests of the models in the IT systems, in particular information on whether the implementation was successful and error-free;
(h)
a self-assessment of compliance with regulatory requirements for the Internal Ratings Based Approach as referred to in Articles 169 to 191 of Regulation (EU) No 575/2013.
2. For the purposes of the verification under paragraph 1(a), competent authorities shall verify that:
(a)
the documentation clearly outlines the purpose of the rating system and the models;
(b)
the documentation includes a description of the range of application of the rating system and the scope of application of the models used within the rating system, i.e. a specification of the type of exposures covered by each model within the rating system, both in a qualitative and in a quantitative manner, the type of outputs of each model and the use made of the outputs;
(c)
the documentation includes an explanation about how the information obtained by means of the rating system and the results of the models is taken into account for the purposes of risk management, decision-making and credit approval processes, as referred to in Article 19.
3. For the purposes of the verification under paragraph 1(b), competent authorities shall verify that the documentation includes:
(a)
detailed information regarding all data used for the model development, including a precise definition of the content of the model, its source, format and coding and, where applicable, exclusions of data from it;
(b)
any data cleansing procedures including procedures for data exclusions, outlier detection and treatment and data adaptations, as well as an explicit justification for their use and an evaluation of their impact.
4. For the purposes of the verification under paragraph 1(c), competent authorities shall verify whether the definitions of default and loss used in the development of the model are adequately documented, in particular where other definitions of default are used for the purpose of model specification than those which are used by the institution in accordance with Article 178 of Regulation (EU) No 575/2013.
5. For the purposes of the verification under paragraph 1(d), competent authorities shall verify that the documentation includes:
(a)
details on the design, theory, assumptions, and logic underlying the model;
(b)
detailed descriptions of the model methodologies and their rationale, statistical techniques and approximations and, where appropriate, the rationale and details on segmentation methods, the outputs of statistical processes and the diagnostics and measures of predictive power of the models;
(c)
the role of experts from the relevant business areas in developing the rating system and models, including a detailed description of the consultation process with experts from the relevant business areas in the design of the rating system and models as well as outputs and rationale provided by those experts from the relevant business areas;
(d)
an explanation of how the statistical model and human judgement are combined to derive the final model output;
(e)
an explanation of how the institution takes into account unsatisfactory quality of data, lack of homogeneous pools of exposures, changes in business processes, economic or legal environment and other factors relating to quality of data that may affect the performance of the rating system or model;
(f)
a description of the analyses performed for the purposes of statistical models or other mechanical methods, as applicable:
(i)
the univariate analysis of the variables considered and respective criteria for variable selection;
(ii)
the multivariate analysis of the variables selected and respective criteria for variable selection;
(iii)
the procedure for the design of the final model, including:
—
the final selection of variables,
—
adjustments based on human judgement to the variables resulting from the multivariate analysis,
—
transformations of the variables,
—
assignment of weights to the variables,
—
the method of composition of model components, in particular where the contribution of qualitative and quantitative components is joined.
6. For the purposes of the verification under paragraph 1(e), competent authorities shall verify that the documentation includes:
(a)
the technical specification of the final model structure including final model specifications, input components including type and format of selected variables, weights applied for variables and output components including type and format of output data;
(b)
references to the computer codes and tools used in terms of IT languages and programs allowing a third party to reproduce the final results.
For the purposes of point (b), the third party may be the vendor in the case of vendor models.
7. For the purposes of the verification under paragraph 1(f), competent authorities shall verify that the documentation includes a description of the weaknesses and limitations of the model, an assessment of whether the key assumptions of the model are met and an anticipation of situations where the model may perform below expectations or become inadequate, as well as an assessment of the significance of model weaknesses and possible mitigating factors thereof.
8. For the purposes of the verification under paragraph 1(g), competent authorities shall verify that:
(a)
the documentation specifies the process to be followed when a new or changed model is implemented in the production environment;
(b)
the documentation covers the results of the tests of the implementation of the rating models in the IT systems, including the confirmation that the rating model implemented in the production system is the same as the one described in the documentation and is operating as intended.
9. For the purposes of the verification under paragraph 1(h), competent authorities shall verify that the institution’s self-assessment of compliance with regulatory requirements for the IRB Approach is performed separately for each rating system and is reviewed by the internal audit or another comparable independent auditing unit.
Register of rating systems
1. When assessing the documentation system and procedures for gathering and storing the information on the rating systems as referred to in Articles 144(1)(e) and 175 of Regulation (EU) No 575/2013, competent authorities shall verify that the institution has implemented and maintains a register of all current and past versions of the rating systems for at least the last three years (‘register of rating systems’).
2. For the purposes of paragraph 1, competent authorities shall verify that the procedures for maintaining the register of rating systems include a recording of the following information in respect of each version:
(a)
the range of application of the rating system, specifying which type of exposures is to be rated by each rating model;
(b)
the management responsible for the approval and date of internal approval, the date of notifying to the competent authorities, the date of the approval by the competent authorities, where applicable, and the date of implementation of the version;
(c)
a brief description of any changes relative to the previous version that has been considered in the register, including a description of the aspects of the rating system which have been changed and a reference to the model documentation;
(d)
the change category assigned in accordance with Delegated Regulation (EU) No 529/2014 and a reference to the criteria for assignment to a change category.
SECTION 3 — Methodology for assessing the structure of rating systems
Risk drivers and rating criteria
1. When assessing the risk drivers and rating criteria used in the rating system for the purposes of Article 170(1), point (a), (c) and (e), (3), point (a), and (4) of Regulation (EU) No 575/2013, competent authorities shall verify all of the following:
(a)
the selection process of the relevant risk drivers and rating criteria, including the definition of potential risk drivers, criteria for selection of risk drivers and decisions taken on the relevant risk drivers;
(b)
the consistency of the selected risk drivers and rating criteria and their contribution to the risk assessment with the expectations of the business users of the rating system;
(c)
the consistency of the risk drivers and rating criteria selected on the basis of statistical methods with the statistical evidence on risk differentiation associated with each grade or pool.
2. The potential risk drivers and rating criteria to be analysed in accordance with paragraph 1(a) shall include the following, where available for the type of exposures:
(a)
obligor risk characteristics, including:
(i)
for exposures to corporates and institutions: financial statements, qualitative information, industry risk, country risk, support from parent entity;
(ii)
for retail exposures: financial statements or personal income information, qualitative information, behavioural information, socio- demographic information;
(b)
transaction risk characteristics, including type of product, type of collateral, seniority, loan-to-value ratio;
(c)
information on delinquency: internal information or information derived from external sources, such as credit bureaus.
Distribution of obligors and exposures in the grades or pools
1. When assessing the distribution of obligors and exposures within the grades or pools of each rating system for the purposes of Article 170(1), points (b), (d) and (f), (2) and (3)(c) of Regulation (EU) No 575/2013, competent authorities shall verify that:
(a)
the number of rating grades and pools is adequate to ensure a meaningful risk differentiation and a quantification of the loss characteristics at the grade or pool level and that:
(i)
for exposures to corporates, institutions, central governments and central banks and specialised lending exposures, the obligor rating scale has at least the number of grades set out in Article 170(1)(b) and (2) of Regulation (EU) No 575/2013, respectively;
(ii)
for purchased receivables classified as retail exposures, that the grouping reflects the seller’s underwriting practices and the heterogeneity of its customers;
(b)
the concentration of numbers of exposures or obligors is not excessive in any grade or pool, unless such distribution is supported by convincing empirical evidence of homogeneity of risk of those exposures or obligors;
(c)
the rating and facility grades or pools for retail exposures have a sufficient number of exposures or obligors in a single grade or pool, unless such distribution is supported by convincing empirical evidence that the grouping of those exposures or obligors is adequate, or that direct estimates of risk parameters for individual obligors or exposures are used as referred to in Article 169(3) of Regulation (EU) No 575/2013;
(d)
the rating and facility grades or pools for exposures to corporates, institutions, central governments and central banks, where sufficient data is available, do not have too few exposures or obligors in a single grade or pool, unless the distribution of exposures or obligors is supported by convincing empirical evidence that the grouping of those exposures or obligors is adequate, or that direct estimates of risk parameters for individual obligors or exposures are used as referred to in Article 169(3) of Regulation (EU) No 575/2013.
2. In addition to the verification laid down in paragraph 1, competent authorities shall assess, where appropriate, the criteria applied by the institution when determining:
(a)
the maximum and the minimum overall number of grades or pools;
(b)
the proportion of exposures and obligors assigned to each grade or pool.
3. For the purposes of paragraphs 1 and 2, competent authorities shall take into account the current and past observed distributions of the number of exposures and obligors and of the exposure values, including the migration of exposures and obligors between different grades or pools.
Risk differentiation
1. When assessing the risk differentiation of each rating system for the purposes of points (b) and (c) of paragraph 3 of Article 170 of Regulation (EU) No 575/2013 for retail exposures, competent authorities shall verify all of the following:
(a)
that the tools used to assess risk differentiation are sound and adequate considering the available data and that the adequate risk differentiation is evidenced with records of time series of realised default rates or loss rates for grades or pools under various economic conditions;
(b)
that the expected performance of the rating system as regards risk differentiation is defined by the institution by means of clearly established fixed targets and tolerances for defined metrics and tools as well as actions to rectify deviations from these targets or tolerances; separate targets and tolerances may be defined for the initial development and the ongoing performance;
(c)
that the targets and tolerances for defined metrics and tools and mechanisms applied to meet those targets and tolerances ensure sufficient differentiation of risk.
2. The competent authorities shall also apply paragraph 1 to the assessment of risk differentiation for exposures other than retail exposures pursuant to Article 170(1) of Regulation (EU) No 575/2013 if a sufficient quantity of data is available for this to be possible.
Homogeneity
1. When assessing the homogeneity of obligors or exposures assigned to the same grade or pool for the purposes of Article 170(1) and (3)(c) of Regulation (EU) No 575/2013, competent authorities shall assess the similarity of the obligors and transaction loss characteristics included in each grade or pool with regard to all of the following factors:
(a)
internal ratings;
(b)
estimates of PD;
(c)
where applicable, own estimates of LGD;
(d)
where applicable, own estimates of conversion factors;
(e)
where applicable, own estimates of total losses.
For retail exposures competent authorities shall assess those factors for each rating system. For exposures other than retail exposures competent authorities shall assess them only for those rating systems in respect of which a sufficient quantity of data is available.
2. For the purposes of the assessment under paragraph 1, competent authorities shall assess the range of values and the distributions of the obligor and transaction loss characteristics included within each grade or pool.
SECTION 4 — Methodology for assessing specific requirements for statistical models or other mechanical methods
Data requirements
1. When assessing the process for vetting data inputs into the model in accordance with Article 174(b) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the reliability and quality of the internal and external data sources and the range of data obtained from those sources, as well as the time period the sources cover;
(b)
the process of data merging, where the model is fed with data from multiple data sources;
(c)
the rationale and scale of data exclusions broken down by reason for exclusion, using statistics on the share of total data which each exclusion covers where certain data were excluded from the model development sample;
(d)
the procedures for dealing with erroneous and missing data and treatment of outliers and categorical data, and verify that, where there has been a change in the type of categorisation, this does not lead to decreased data quality or structural breaks in the data;
(e)
the processes for data transformation, including standardization and other functional transformations, and the appropriateness of those transformations having regard to the risk of model overfitting.
2. When assessing the representativeness of the data used to build the model as referred to in Article 174(c) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the comparability of risk characteristics of the obligors or facilities reflected in the data used to build the model with those of the exposures covered by a particular rating model;
(b)
the comparability of the current underwriting and recovery standards with the ones applied at the time to which the reference data set used for the modelling relates;
(c)
the consistency of default definition over time in the data used for the modelling and verify:
(i)
that adjustments have been made to achieve consistency with the current default definition where the default definition has been changed during the observation period;
(ii)
that adequate measures ensuring the representativeness of data have been adopted by the institution where the institution operates in several jurisdictions having different default definitions;
(iii)
that the default definition used for the purposes of model specification does not have a negative impact on the structure and performance of the rating model where this definition is different from the definition of default laid down in Article 178 of Regulation (EU) No 575/2013;
(d)
where external data or data pooled across institutions is used in the model development, the relevance and adequacy of such data for the institution’s exposures, products and risk profile.
Model design
When assessing the rating model design for the purposes of Article 174(a) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the adequacy of the model having regard to its specific application;
(b)
the institution’s analysis of alternative assumptions or alternative approaches to those chosen in the model;
(c)
the institution’s methodology for model development;
(d)
that relevant staff of the institution fully understands the model’s capabilities and limitations, in particular that the model documentation of the institution:
(i)
describes which of the model limitations are related to the model inputs, uncertain assumptions, the processing component of the model, and whether the model output is performed manually or in the IT system;
(ii)
identifies situations where the model can perform below expectations or become inadequate and contains an assessment of the materiality of model weaknesses and possible mitigating factors thereof.
Human judgement
When assessing whether the statistical model or another mechanical method is complemented by human judgement in accordance with Article 174(e) of Regulation (EU) No 575/2013 and whether human judgement is applied in a proportionate and adequate manner in the development of the rating model and in the process of assigning exposures to grades or pools, competent authorities shall verify that:
(a)
the manner in which human judgement is applied is justified and fully documented and that the impact of human judgement on the rating system is assessed, if possible also by means of a computation of the marginal contribution of human judgement to the performance of the rating system;
(b)
all relevant information not considered in the model is taken into account and an adequate level of conservatism is applied;
(c)
where the process of assignment of exposures to grades or pools in a rating system requires the application of human judgement in the form of subjective input data or where the credit policy allows for overrides of inputs or outputs of the model, all of the following applies:
(i)
the manual for model users clearly defines the input data and the situations where the input data can be adjusted by human judgement;
(ii)
the situations where the input data have actually been adjusted are limited;
(iii)
the manual for model users clearly defines the situations where the input or output of rating models may be overridden and the procedures for overriding the input or output of the models;
(iv)
all data regarding the application of human judgement and the situations where the inputs or outputs of the rating models have been overridden are stored and analysed periodically by the credit risk control unit or by the validation function in order to ascertain its impact on the rating model;
(d)
the application of human judgement is appropriately managed and proportionate to the type of exposures for each rating system.
Model performance
When assessing the predictive power of the model required under Article 174(a) of Regulation (EU) No 575/2013, competent authorities shall verify that the institution’s internal standards:
(a)
provide an outline of the assumptions and theory underlying the metrics chosen by the institution for the purpose of the assessment of the model’s performance;
(b)
specify the application of the metrics, indicate whether the use of each metric is compulsory or discretionary and when it is to be used and ensure that the metrics are used coherently;
(c)
specify the conditions of the applicability and acceptable thresholds and accepted deviations for the metrics and set out whether and, if so, how statistical errors relating to the values of those metrics are taken into account in the assessment process, and, where more than one metric is calculated, establishes the methods of aggregating several test results to one single assessment;
(d)
determine a process for ensuring that events of model performance deterioration leading to the breach of the thresholds referred to in point (c) are communicated to the appropriate members of the senior management in charge of it and that clear guidance on how the outcomes of the metrics are considered is provided by the members of the management responsible for taking final decision as regards implementation of the necessary changes to the model.
Source: EUR-Lex (Publications Office of the EU), © European Union, reuse permitted under Commission Decision 2011/833/EU.
