European Bank for Reconstruction and Development

Solve how AI incidents would managed in traditional companies. Create a best practice of AI RACI that involves IT, AI, Business, data management and Risk team. Address the implications of new data engineering tools requiring business to be a Data/AI engineer.

• AI capacity-building
• Safe, secure and trustworthy AI
• Interoperability of governance approaches
• Open-source software, open data and open AI models

a

One point relates to how errors in the AI bot would be handled. In many organisations, issues or knowledge errors identified in the AI bot (for example, when the bot produces incorrect answers and users report this to IT or the AI team) would currently be addressed through the Software Development Life Cycle (SDLC) rather than through the Incident management framework. - If AI incidents go through SDLC, resolution may take longer than expected, as SDLC processes are typically designed for development changes rather than operational fixes. - If AI incidents go through incident management framework, I am not sure whether the current global framework is designed to address them effectively. Even if such issues were logged in ServiceNow, they may be treated as relatively low-priority operational incidents compared with critical system incidents (e.g. a payment error versus a bot producing an incorrect answer). à Over time, this could erode user trust in the AI bot, reduce adoption, and ultimately undermine the value of the investment.. Another area concerns the ownership of the content generated by the AI bot. The underlying data and information used by the bot are owned and maintained by business teams. - For example, within EBRD's data governance framework, business units act as Data SMEs and manage specific domains in EBX (e.g. Impact data stewards maintain the TI section and has read/write rights). à In practice, this means the business maintains both the data and the information that feed into these systems (from the data àinformation àknowledge nexus) However, when we move into the knowledge layer represented by the AI bot (i.e. the data → information → knowledge chain), the RACI between IT, AI teams, business owners and data management is not entirely clear. For example: • Change management: Who is responsible for updating models or knowledge sources used by the AI bot? What is the process? • Incident management: When the bot produces incorrect outputs, should the issue be addressed by the AI team, IT, or the business owner of the content? (In EBRD, the business typically addresses content-related issues rather than IT.) • Release management: If the AI team updates code or models and these are embedded in Monarch platform, how are these handled? • Problem management: How are recurring issues with the bot analysed and addressed? There is also a related technical dimension as the Bank expands self-service analytics and data engineering capabilities, particularly with the introduction of Microsoft Fabric. Here, business teams may increasingly participate in activities traditionally associated with data warehouse team (e.g. ETL processes). For the AI bot specifically, the data pipeline may look roughly as follows: Extract data from Monarch → store TI data in an EBRD relational database → transform and move this data into a vector database → use the vector database to support RAG-based queries for the AI bot. This implies shared responsibilities across IT, AI, business and data management for: 1. Extracting and storing data 2. Maintaining the relational database 3. Managing the vector database 4. Ensuring that the data feeding the AI bot remains accurate and up to date It may also have implications for access rights and potentially operational risk oversight.

If the operational governance of AI (incident managment, data engineering, RACI between IT, AI, business, data mgmt, and risk), there will be huge waste of AI resources.

a

a

Involve ISACA (who built COBIT Framework), and ISO who developed ISO 27001. Also involve stakeholders from EU who developed AI ACT, DORA ACT.

The global organisations who developed standards should be more involved.

a

a