Check against delivery.

Ladies and Gentlemen,

In May 1978, the band “Kraftwerk” released the album “The Man Machine”. With this album and its predecessors, Kraftwerk set a benchmark, and even shaped later music styles such as techno, perhaps more than any other German band. “The Man Machine” was inspired by Fritz Lang’s classic silent film “Metropolis”, released in 1927. This film, from German production company UFA, influenced both the album’s title and its theme of a world dominated by technology, in which free men and women are reduced to being the servants of a complex machine.

In Kraftwerk’s early years, and in Fritz Lang’s time, the idea of a “man machine” might have been science fiction. Today, artificial intelligence and machine learning are hot topics in the financial sector, as in many other sectors. We supervisors are focussing primarily on the question of how much artificial intelligence and machine learning will change the financial industry and what the implications are for financial stability, market and firm supervision, and collective consumer protection.

Almost exactly a year ago, in order to find answers to these questions, BaFin published the report “Big data meets artificial intelligence”¹ and subsequently launched a public consultation on this report.

Machine learning (ML) means using appropriate algorithms to give computers the ability to learn from data and thereby put them in a position to find better solutions to the tasks they are intended to solve. Big data, meanwhile, is a key driver in making this machine learning economically usable. And this is becoming easier and easier, because not only is the quantity of data constantly expanding, but better methods are also being developed that can be used to analyse data that it was previously impossible to analyse. The increasing power of hardware also helps to expand the potential value of machine learning based on big data. And what role does artificial intelligence play? To put it bluntly, big data plus computing resources plus machine learning = (equals) artificial intelligence.

But realistically, it is necessary to point out here that approaches for the general simulation of human intelligence – strong AI – are still just visions for the future.

Ladies and Gentlemen,

Let me reflect on four key issues associated with the application of artificial intelligence, two of them being relevant from a supervisory angle and two of them leading to a broader set of questions.

My first key issue is a seemingly simple question: Who and what are we actually supervising? Due to the increased use of BDAI in financial companies, many processes will become faster, more efficient, and more automated. To pick an example: even today, insurers are able to carry out procedures such as risk assessments and claims processing without involving a single human being. However, the management board must not just shift responsibility to machines and algorithms as they can with certain work processes. The ultimate responsibility has to remain with the management board – with people. For this reason, we will not accept models that are presented to us as a black box.

But how do supervisory benchmarks and requirements need to be designed in order to allow us to carry out a proper legal and technical examination of the models? The buzzwords that come up time and again in this context are explainability and transparency. Transparency means that it is possible to fully understand the behaviour of an entire system. But many of the algorithms are too complex for that. It is therefore necessary to try to at least ensure explainability: this involves being able to identify the key factors that influence the decision a machine makes. But what level of explainability is ultimately enough? These are the questions that we are concerned with at the moment. But these are issues that are too important and too complex to be hurried. Rather than rushing in head first, we should strengthen the dialogue with academia and industry, which we can then use as a basis to develop best practices.

This leads me to another key issue: how long is supervisory approval valid for in times of BDAI and machine learning? Here, as you have probably realised, I am referring to the internal models used by banks and insurance companies that require supervisory approval. The short answer is: their validity might be reduced.

This is primarily because processes and models, driven by self-learning elements, are constantly being refined and developed. A model that is approved by supervisors can change literally by the day, hour or even minute. We will therefore need to carefully reconsider which modifications constitute a model change in the supervisory sense, requiring the bank or the insurer to get straight back in touch with the supervisor. And quite fundamentally, we need to decide how much dynamic adjustment we can even permit in such models.

The third key issue is another question: where does sensible and desirable risk adequacy and differentiation stop and where does undesirable discrimination aimed only at maximising profits begin? Now, differentiation – even using personal data – is normal and sensible; indeed, in some cases the supervisors actually demand it. For example, if a customer wishes to purchase new insurance for their car, the insurer is explicitly required to request a risk-adequate price under the applicable supervisory law. For the time being, big data and machine learning do not change this. However, BaFin also has a collective consumer protection mandate. It needs to keep in mind that discrimination-free access to financial products must still be maintained in the age of digitalisation.

Moreover, consumers need to be alerted to the fact that these new technologies allow companies to gain insights into their private life and obtain extensive information about their personal preferences, wishes, personal solvency and even their health. If these data are used in the interests of the customer, for instance to offer them suitable products and services, this is, at least from a supervisory point of view, acceptable. However, if they are used, intentionally or unintentionally, against the consumer in a discriminatory way, this is not acceptable.

Both industry and the supervisors need to be clear that excessive invasions into customers’ private lives, or actual misuse of data, can very quickly cause customers to lose their trust in the companies to which they have provided sensitive financial data. Machine learning and big data can only achieve their full potential if companies are able to build and retain customers’ trust by using their data properly, legally and transparently. For the companies, it is in their own best interests to be open about the extent to which they use machine learning and big data to increase returns by monetising personal data. There is still a great deal of work ahead of the financial industry, supervisors and politicians to find appropriate solutions to the difficult questions that the next few years will present.

My fourth and final question is: who monetises the value of data? That means: who secures the ultimate prize – customers’ data – and wins the customer interface battle? The traditional financial institutions or rather global technology giants, the bigtech companies? One thing we already know for sure: the use of machine learning and big data will bring about significant changes to the traditional value chains. It will become rarer and rarer for services to be provided by a single institution. Instead, processes will be increasingly outsourced to third parties, and services will be offered that are not exclusively provided by the company itself. For bigtech companies, this does not present a particular challenge: they move like fish in the waters of platformised markets.

We can expect increased competition for market share and a trend towards monopolies on data and data analysis. Now, we are a supervisory authority and not the legislature or the Federal Cartel Office. It is not part of our mandate to decide on a desirable market structure. What we are interested in, above all else, is avoiding undesired consequences for the stability of the financial markets.
Furthermore, we will need to ask ourselves who or what should be subject to what kind of supervision in the world of machine learning and big data. For example, will providers that offer structural expertise and information on the financial market need to be supervised although they are not providing financial services themselves? A lot of work needs to be done.

Ladies and Gentlemen,

If you were to ask me, in conclusion, how machine learning and big data will change the day-to-day life of financial companies and supervisors, my answer would be: a great deal. But not so much that customers will only encounter “man machines” in banks and insurers in the future. Technology will undoubtedly have far greater prominence than it does today, but it will certainly offer benefits for customers too. Nevertheless, it is vital that we do not ignore the risks.

We need to identify them, and set to work on finding ways to deal with them from a supervisory or a regulatory perspective.

If we manage that, we can ensure that the reality of machine learning is very different from the dystopian fantasies of filmmakers and musicians.

Thank you for your attention. I look forward to further insights and predictions in the following panel discussion.

Footnote: