The Luxembourg Institute of Science and Technology (LIST) and Banque Internationale à Luxembourg (BIL) have completed their joint research project on evaluating conversational AI systems in a banking context.

Launched in June 2025, the project set out to develop and apply a rigorous, science-based evaluation framework to assess AI systems used in customer interactions, with a focus on fairness, robustness, and service quality. The collaboration combined LIST’s expertise in AI testing using the LIST AI Sandbox with BIL’s real-world operational requirements.

The project delivered actionable insights under strict confidentiality conditions, with no access to customer conversations or data by LIST researchers, and all testing performed within BIL’s secure IT infrastructure.

Results showed that no significant discriminatory behaviour was identified. Some minor variations were observed, suggesting opportunities for refinement but without affecting the overall fairness of the system. The analysis of user experience revealed that frustration was primarily driven by situational constraints or unmet expectations, with the system’s tone and language remaining consistently professional and reassuring. In addition, adversarial testing confirmed strong safeguards, maintaining compliance and security across multiple scenarios and languages

Enabling responsible AI with the LIST AI Sandbox

A key enabler of the collaboration was the LIST AI Sandbox, a platform specifically designed to test, validate, and improve the reliability of artificial intelligence in a controlled environment. The Sandbox allows companies, researchers, and public institutions to evaluate their AI models on real-world cases, while minimising technical, regulatory, and ethical risks. By leveraging this platform, the project was able to conduct comprehensive assessments without exposing sensitive systems or data, providing actionable insights while maintaining security and compliance.

Toward a replicable framework for trustworthy AI

Beyond the individual evaluation results, the collaboration contributes to a broader objective: establishing a replicable and structured approach to AI evaluation and governance.

The methodologies and tools developed during the project support a continuous learning cycle from assessment to reporting and improvement helping organisations better understand and manage the behaviour of AI systems over time.

Francesco Ferrero, Leader of the Flagship Initiative on Artificial Intelligence and the Human-Centred AI, Data and Software Research Unit at LIST, stated:

This joint project with BIL demonstrates how financial institutions can combine scientific methods with secure infrastructure to ensure AI systems are trustworthy, reliable, and aligned with regulatory and ethical standards.

Olivier Gorin, Head of Digital Banking at BIL, concluded:

At BIL, we’re always looking for ways to make our services genuinely easier and more helpful for our clients. This project has helped us improve how we use AI so that Berry, our AI assistant, can provide support that is clearer, quicker and reassuring. Since its launch in December 2025, Berry has already made everyday banking smoother for many of our clients. We’re grateful to the LIST teams for their excellent work and the constructive collaboration throughout the project.

About LIST

The Luxembourg Institute for Science and Technology (LIST) is a leading European research centre focusing on natural and engineering sciences in Luxembourg and the Greater Region. With 840 employees - nearly 80% of them are researchers - LIST drives transformative science across natural, built, and industrial environments and makes the most of its expertise in space, artificial intelligence, as well as security and defence. LIST bridges fundamental and applied research and develops prototypes and market-ready solutions that empower industry, government and society. Through strong partnerships with public and private stakeholders, LIST accelerates technology transfers, fosters collaboration, strengthens Luxembourg’s global competitiveness and contributes to a sustainable and resilient future.