State Secretary, the Federal Government adopted the national AI strategy at the Digital Cabinet meeting on 15 November 2018. What is it about?

This strategy marks the first time that the Federal Government has established an end-to-end political framework for developing and applying Artificial Intelligence (AI) in Germany. It will pursue three main objectives. First, it is designed to make Germany and Europe a leading location for developing and applying AI technologies – including to secure Germany’s future competitive position. Second, it is designed to ensure the responsible development and deployment of AI in the public interest, and third, it is designed to embed AI ethically, legally, culturally and institutionally in society through a broad social dialogue and by actively shaping the political framework. The government is earmarking a great deal of money to achieve these objectives, with planned investments totalling three billion euros.

The strategy also includes essential aspects for the financial industry and thus incorporates recommendations by the Fintech Council at the Federal Ministry of Finance. AI and Big Data enable innovations that are transforming the financial market. All this requires technical and professional expertise, and supervisors, too, must respond accordingly. BaFin’s study on Big Data and AI¹ is an important first step in this direction.

Where do you see the greatest opportunities and potential risks in using Big Data and AI in the financial market?

The challenge to government, society, business, administration and academia is to seize the opportunities AI presents, but also to face up to the risks it poses. With its AI strategy, the Federal Government has established a basis for creating the conditions for leveraging the potential AI has to offer. BDAI must be developed and deployed to ensure human-centric application, in the public interest, for the economy and society, rooted in fundamental democratic principles.

The interplay between AI and Big Data is leading to autonomous systems that are currently fuelling the rapid pace of digitalisation in all sectors of the economy. In the near future, these developments could also lead to changes in our financial system. We can already observe how BDAI is being used in the capital markets. And BDAI is also increasingly being recognised as an opportunity by banks and insurers.
BDAI applications could, for example, be used in institutions’ risk assessment models or to facilitate claims processing in insurance undertakings.

It is important when deploying BDAI technologies to ensure that financial stability and integrity are not compromised. Another factor is that consumers must not be left behind, meaning that they must not lose sovereignty over their personal data in a world of BDAI.

Do you think it is possible that the increased use of Big Data and AI will lead to new types of systemic risk?

No systemic risks can be identified at the present time because the use of Big Data and AI in the financial industry is still too low. But this can change quickly. That is why it is important for supervisors to keep an eye on possible risks to financial stability so that they can address them quickly if necessary.

The application of Big Data and AI also frequently raises data protection issues. How do you think financial supervision should approach this?

When companies analyse huge volumes of data in a self-learning process, they must ensure that their customers stay in control of their data. The GDPR², which came into force last year, established important standards for this across Europe, i.e. customers must consent to their data being processed for pricing purposes. At the same time, customers must be sensitised to an even greater extent to the fact that they generally pay for faster and better service with their data. But the companies, too, have to ensure that customer data is handled responsibly.

The Federal Government has therefore set itself the goal in its AI strategy of convening a round table with data protection supervisors and industry associations that will devise joint guidelines for developing and applying AI systems that comply with data protection rules, and to prepare examples of best practice applications.

Compliance with data protection rules is a matter for the responsible authorities of the Länder and the Federal Commissioner for Data Protection. If the relevant requirements are met, BaFin may also intervene as part of its supervision of violations of consumer protection law. In particularly extreme cases, the suitability of management board members would also have to be called into question.

In the first edition of BaFinPerspectives, BaFin President Felix Hufeld described the problem of unintentional discrimination by Big Data and AI. How can this problem be solved?

Algorithms are not underpinned by a certain world view; they draw their conclusions from the data records. However, depending on how these records are selected or what the analysis will be used for, this can result in discrimination. The Fintech Council drew attention to this problem in its recommendations for action at the end of 2017. Part of a responsible approach to AI that bears the public interest in mind is also for companies using AI to ensure that discrimination is prevented – if necessary by using human control mechanisms. For the financial industry, this means that using AI technology does not absolve the management board from its overall responsibility for ensuring a proper business organisation.

Big Data and AI make it possible to link heterogeneous data. This may also allow companies to explore customers’ maximum willingness to pay – and, given corresponding market power, to exploit it. What is your view on this?

Pricing is linked to numerous issues that go far beyond supervisory law. As in other areas, the bounds of what is acceptable are reached when the company exploits its market power at the expense of consumers. If pricing is based on personal data, the protection afforded by the GDPR also applies here.

The Federal Government has set itself ambitious goals with its AI strategy. What do you expect from BaFin?

The Federal Government wants to take the lead in the further deployment of AI in public administration and hence contribute to improving the efficiency, quality and reliability of administrative services. BaFin can also play an important role here. It laid the foundations for this with its BDAI study, and it is now time to build on them. BDAI applications can add value especially in supervisory areas where BaFin has structured data in sufficient quality and quantity. One area that sprints to mind, for example, is the reporting system for securities supervision. BaFin should be ambitious and examine whether the analysis of large structured data volumes can be managed more often by AI in the future. BaFin should also aim to enhance its digitalisation strategy in this context so as to further strengthen financial supervision in the long term.

Dr Kukies, thank you for the interview.

Footnotes: