BSI AIC4: when the AI catalogue fits and what it does not cover

This page explains what AIC4 is for, where it stops and how it relates to C5. How an AI SaaS provider combines C5 and AIC4 in a project is shown on C5 for AI SaaS; AI software in healthcare has its own page, AI in healthcare.

What is AIC4?

AIC4 is the BSI’s “AI Cloud Service Compliance Criteria Catalogue”. The BSI describes it as a catalogue that sets minimum requirements for the secure use of machine learning methods in cloud services, designed as an extension of C5. Its aim is a transparent account of the information security of an AI cloud service, based on a standardised audit, which customers can use in their own risk analysis.

Four key facts to know before you plan anything:

• Date: The PDF carries the notice “© Federal Office for Information Security 2021”. The BSI states on its website that it has decided not to update the current version directly and not to translate it, because the international development of audit criteria is still ongoing.
• Language: English only.
• No certificate: According to the BSI, there is currently no certification scheme for AIC4. The result of an audit is an audit report, as with C5.
• Reference to C5:2020: The catalogue refers to the 2020 version of C5 and uses its criteria numbers (see below).

Which services is AIC4 meant for?

For cloud services that use machine learning methods and improve their performance with training data. The catalogue names gradient boosting algorithms, random forests and deep neural networks as its focus; typical applications include image classification and segmentation, time series prediction, natural language processing, voice recognition and scoring (AIC4, chapter 2). It explicitly does not cover federated learning or reinforcement learning.

Use is voluntary; the BSI stresses that the criteria are not regulation. As a guide it poses two questions: Are ML methods used particularly intensively, or in areas that can affect the information security of the service? Do customers need information on the use of the ML methods to assess the suitability of the service in their risk analysis? If either answer is yes, the BSI recommends considering AIC4.

Our assessment for services that only call a third party’s large language model through an interface: the catalogue does not mention such models specifically, and many of its criteria assume that the provider controls training, validation and data itself. Whether AIC4 serves a purpose then is a case-by-case question; with your own training or fine-tuning it is a closer fit.

Is there a duty to use AIC4?

None follows from the catalogue or the BSI website. Section 393 SGB V (Book V of the German Social Code) requires a current C5 attestation for the basic criteria for cloud services in healthcare, or instead, under certain conditions, evidence under the C5 Equivalence Ordinance (C5GleichwV), and does not mention AIC4 (Section 393 SGB V, C5 Equivalence Ordinance). A duty can, however, arise from your customers’ contracts or tenders. The BSI itself writes on its English AIC4 page that, to assess the information security of a cloud service, a customer “usually needs a C5 audit report” as well as an AIC4 audit report. That is a recommendation to customers; whether your customers adopt it is something to clarify in sales.

The eight areas of AIC4

The catalogue is divided into eight areas with 41 criteria in total (our count). The names in the table are the catalogue’s English names (the graphic below uses the same names), the description is our summary.

In addition, AIC4 requires its own information on the general conditions of the AI service in the system description (BC-01 to BC-06), for example on how often the model is trained, the degree of automated decision-making and the limits of robustness. AIC4 has no additional criteria in the C5 sense.

How do AIC4 and C5 relate?

C5 is the prerequisite. Criterion PC-01 requires that the AI service meets the C5 criteria and that a C5 report shows this; on its website the BSI calls C5 the overarching framework. AIC4 does not repeat the C5 criteria. It adds AI-specific requirements and extends individual C5 criteria, for example risk management to cover AI-specific attacks (SR-02). PC-01 sets six requirements for the C5 report, which the catalogue headings call “Scope”, “Coverage Period”, “Report Type”, “Qualified Service Auditor”, “Opinion” and “Timing”. The left column summarises them in our own words and in short form; the English wording of PC-01 is authoritative.

According to chapter 4, the audit follows the C5:2020 methodology: ISAE 3000 (Revised) or IDW PS 860, reasonable assurance, type 1 only for the first audit; because AI services change quickly, the catalogue recommends auditing at least annually. New compared with C5 is a requirement for the audit team: at least one person who tests the AI service should have three years of professional experience as a data scientist or as a developer of ML models. Section 4.4.9 phrases this as a recommendation, the overview in chapter 7.1 as a requirement; ask your auditor how they handle it.

Reading references to C5:2020 correctly

Because AIC4 dates from 2021, it cites criteria of C5:2020. With C5:2026 many of them have been renumbered, and the same number can refer to a different criterion. The mapping is in the BSI’s change history for C5:2026. Some examples:

Check every reference in AIC4 this way before you map controls. Otherwise you link an AIC4 requirement to the wrong C5:2026 criterion. Which C5 version applies to your next audit is explained on C5:2026.

What AIC4 does not do

• No statement of suitability: According to the catalogue, a successful report does not mean that robustness, performance, reliability, data quality, explainability or the absence of unwanted bias is sufficient for a particular application. Customers assess that themselves; according to the BSI, they also bear the residual risks themselves.
• No data protection audit: According to the BSI, an AIC4 audit says nothing about whether personal data, such as training or input data, is used lawfully.
• No ethics audit: On bias, AIC4 only considers aspects that concern information security.
• No substitute for C5: Without a C5 report under PC-01, AIC4 cannot be met.
• No ongoing updates: The BSI is currently not updating the catalogue directly. For auditing AI systems, the BSI is developing a separate, modular audit framework, A5; its status and structure are described on BSI A5.

Example: two AI services, two answers

Fictitious examples, not clients.

• Damage photos for insurers: A provider trains its own image classification model that pre-sorts photos of vehicle damage. The insurers want to know how the model is trained, tested and monitored. The answer to both BSI questions is yes. The provider first plans a C5 type 2 report with the AI service in scope and sets the AIC4 period to the same period.
• Text assistant with a third-party language model: Another provider summarises e-mails and calls a third party’s language model for this, without training its own. Its customers ask for the C5 attestation and about the model provider. It focuses on C5, treats the model provider as a subservice organisation and considers AIC4 only when customers ask for it.