Data requirements
1. When assessing the process for vetting data inputs into the model in accordance with Article 174(b) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the reliability and quality of the internal and external data sources and the range of data obtained from those sources, as well as the time period the sources cover;
(b)
the process of data merging, where the model is fed with data from multiple data sources;
(c)
the rationale and scale of data exclusions broken down by reason for exclusion, using statistics on the share of total data which each exclusion covers where certain data were excluded from the model development sample;
(d)
the procedures for dealing with erroneous and missing data and treatment of outliers and categorical data, and verify that, where there has been a change in the type of categorisation, this does not lead to decreased data quality or structural breaks in the data;
(e)
the processes for data transformation, including standardization and other functional transformations, and the appropriateness of those transformations having regard to the risk of model overfitting.
2. When assessing the representativeness of the data used to build the model as referred to in Article 174(c) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the comparability of risk characteristics of the obligors or facilities reflected in the data used to build the model with those of the exposures covered by a particular rating model;
(b)
the comparability of the current underwriting and recovery standards with the ones applied at the time to which the reference data set used for the modelling relates;
(c)
the consistency of default definition over time in the data used for the modelling and verify:
(i)
that adjustments have been made to achieve consistency with the current default definition where the default definition has been changed during the observation period;
(ii)
that adequate measures ensuring the representativeness of data have been adopted by the institution where the institution operates in several jurisdictions having different default definitions;
(iii)
that the default definition used for the purposes of model specification does not have a negative impact on the structure and performance of the rating model where this definition is different from the definition of default laid down in Article 178 of Regulation (EU) No 575/2013;
(d)
where external data or data pooled across institutions is used in the model development, the relevance and adequacy of such data for the institution’s exposures, products and risk profile.
Model design
When assessing the rating model design for the purposes of Article 174(a) of Regulation (EU) No 575/2013, competent authorities shall verify:
(a)
the adequacy of the model having regard to its specific application;
(b)
the institution’s analysis of alternative assumptions or alternative approaches to those chosen in the model;
(c)
the institution’s methodology for model development;
(d)
that relevant staff of the institution fully understands the model’s capabilities and limitations, in particular that the model documentation of the institution:
(i)
describes which of the model limitations are related to the model inputs, uncertain assumptions, the processing component of the model, and whether the model output is performed manually or in the IT system;
(ii)
identifies situations where the model can perform below expectations or become inadequate and contains an assessment of the materiality of model weaknesses and possible mitigating factors thereof.
Human judgement
When assessing whether the statistical model or another mechanical method is complemented by human judgement in accordance with Article 174(e) of Regulation (EU) No 575/2013 and whether human judgement is applied in a proportionate and adequate manner in the development of the rating model and in the process of assigning exposures to grades or pools, competent authorities shall verify that:
(a)
the manner in which human judgement is applied is justified and fully documented and that the impact of human judgement on the rating system is assessed, if possible also by means of a computation of the marginal contribution of human judgement to the performance of the rating system;
(b)
all relevant information not considered in the model is taken into account and an adequate level of conservatism is applied;
(c)
where the process of assignment of exposures to grades or pools in a rating system requires the application of human judgement in the form of subjective input data or where the credit policy allows for overrides of inputs or outputs of the model, all of the following applies:
(i)
the manual for model users clearly defines the input data and the situations where the input data can be adjusted by human judgement;
(ii)
the situations where the input data have actually been adjusted are limited;
(iii)
the manual for model users clearly defines the situations where the input or output of rating models may be overridden and the procedures for overriding the input or output of the models;
(iv)
all data regarding the application of human judgement and the situations where the inputs or outputs of the rating models have been overridden are stored and analysed periodically by the credit risk control unit or by the validation function in order to ascertain its impact on the rating model;
(d)
the application of human judgement is appropriately managed and proportionate to the type of exposures for each rating system.
Model performance
When assessing the predictive power of the model required under Article 174(a) of Regulation (EU) No 575/2013, competent authorities shall verify that the institution’s internal standards:
(a)
provide an outline of the assumptions and theory underlying the metrics chosen by the institution for the purpose of the assessment of the model’s performance;
(b)
specify the application of the metrics, indicate whether the use of each metric is compulsory or discretionary and when it is to be used and ensure that the metrics are used coherently;
(c)
specify the conditions of the applicability and acceptable thresholds and accepted deviations for the metrics and set out whether and, if so, how statistical errors relating to the values of those metrics are taken into account in the assessment process, and, where more than one metric is calculated, establishes the methods of aggregating several test results to one single assessment;
(d)
determine a process for ensuring that events of model performance deterioration leading to the breach of the thresholds referred to in point (c) are communicated to the appropriate members of the senior management in charge of it and that clear guidance on how the outcomes of the metrics are considered is provided by the members of the management responsible for taking final decision as regards implementation of the necessary changes to the model.
Source: EUR-Lex (Publications Office of the EU), © European Union, reuse permitted under Commission Decision 2011/833/EU.
