ELISE’s 2021 Strategic Research Agenda set out the research challenges that need to be addressed to strengthen the technical capabilities of AI; improve its performance in deployment; and align AI development with societal interests. This Agenda sought to bridge between the frontiers of technology development and the EU’s AI policy agendas, recognising that the success of those policy agendas would depend on Europe’s ability to pursue excellent research that both advances foundational AI technologies and applies those technologies to areas of critical social and scientific need.
The research themes and areas of research interest explored in 2021’s Strategic Research Agenda are considered in the following sections. As technical capabilities continue to shift, the intention here is not to provide a comprehensive review of the state of the art under each theme, but to convey a sense of key issues and how the field has progressed in recent years. To illustrate how the research topics explored relate to practical applications, a selection of use cases introduces how ELISE industrial collaborators have deployed machine learning to enhance their work.
Over the two years since the publication of the initial ELISE Strategic Research Agenda, research, practice, and policy have changed at pace. Headline-grabbing advances in technologies such as Large Language Models have re-ignited debates about the opportunities and risks associated with AI, highlighting the potential for rapid advances in technical capabilities. Understandings of how to deploy both known and state-of-the-art technologies have continued to evolve. Legislative and regulatory proposals have grappled with the challenge of stewarding a technology that is dynamic, pervasive across sectors, and associated with both beneficial and harmful uses. From this shifting landscape emerge ten trends in technology and regulation that are shaping the development of AI today.
The research pursued across the ELISE network is at the forefront of these trends. ELISE’s 14 Research Programmes continue to advance the capabilities of AI technologies – across theory, methods, and application – in turn, creating new questions and research challenges. Progress can be seen in the shifting contours of research and practice across the five cross-cutting themes set out in the first ELISE Strategic Research Agenda.
The development of trustworthy AI that can be safely and effectively deployed to deliver real-world benefits continues to be a cornerstone of AI research and AI policy. Delivering the EU’s seven characteristics of trustworthy AI – human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental wellbeing; and accountability – requires theoretical, methodological, and practical advances in AI technologies. Current policy debates highlight the importance of bolstering these technical characteristics with standards, certifications, and guarantees that ensure trustworthiness is not only a principle for development, but is also delivered in practice. Both the technical capabilities to underpin trustworthy AI and the certifications that can prove whether an AI system is trustworthy have been areas of focus for ELISE research.
A variety of methods to deliver on each of the seven characteristics of trustworthy AI now exist, including methods for working with real-world data, explainability, robustness, privacy preservation, and bias-reduction. AI systems can perform well where they are trained on large datasets, where there is a defined task, and where there is a clear understanding of how to validate performance at that task. There remains, however, a gap between these controlled conditions and the real-world environments in which many AI systems are developed and deployed.
The size of the AI models underpinning Large Language Models and their equivalents poses a further challenge to implementation of methods for trustworthiness by design. Bridging this gap requires progress in AI robustness, and new understandings of verification and validation. This in turn requires close collaboration between technologists, practitioners, and policymakers to identify appropriate benchmarks, certifications, and validation procedures that connect technical capabilities to effective risk evaluation in deployment.
Certifying or guaranteeing the performance of AI systems
Verifying and validating machine learning
Improving the robustness of machine learning in deployment
AI has been both a catalyst for fresh concerns about security and privacy and a source of new tools to tackle these concerns. ELISE’s 2021 Strategic Research Agenda highlighted the importance of further progress in the development of principled methods for achieving privacy and security by design, and of integrating user needs and societal expectations around the safety and reliability of AI systems into the research and practice that underpins their deployment. Research progress has yielded a collection of methods for enhancing the security of AI systems and ensuring they respect fundamental rights in relation to privacy. These include privacy-enhancing technologies, such as differential privacy and federated learning, and advanced methods for attack detection and mitigation, such as adversarial learning.
With new threat vectors emerging and amidst continuing concerns about data privacy – for example, as users submit new data types to Large Language Models – the challenge for the next wave of AI progress is to make these state-of-the-art methods workable in practice and to establish best practices in data stewardship. This requires further work to test that the assumptions on which theoretical developments are based are valid in practice;
to resolve design trade-offs in design that affect overall system performance, such as the ability to deploy advanced methods in differential privacy and cryptography in systems that are computationally efficient; and to better understand what type of security threat is likely to emerge from advances in AI. Across these areas, continued dialogue is needed to understand society’s expectations in relation to data use, privacy, and security.
Privacy and security by design
Working with practitioners to translate state-of-the-art technology to practice
Better understanding future threats
Concerns about AI as a ‘black box’ in decision-making – with technical and non-technical communities alike unable to understand its workings – have received widespread attention. In the research and policy debates that follow, a collection of ideas about the features needed from AI to ensure its trustworthiness collide.
Attempts to respond to these questions through innovations in technology and governance have resulted in explainability becoming a popular research topic. Many methods for explainable AI have been developed, in particular methods for increasing model interpretability and for creating counterfactual explanations. These innovations are valuable. However, they become more difficult to deploy in complex systems, and they do not necessarily align with user expectations around explainability, or how humans might approach the task of data interpretation.
In response, further technical developments are needed to ground these methods to theory – both in terms of modelling and human understanding – and to connect these methods to user needs. Aligned progress in fields such as causal AI could also help drive progress, by allowing users to interrogate the relationships or causal factors that result in a particular output or decision. The result should be explainable AI methods that function well at scale; that are human-centric, based on an understanding of what sort of explanation is needed for whom in what contexts; and that deliver interpretability-by-design, by aligning user perspectives with technical functionality.
Advancing explainable AI tools and methods
Grounding new methods to theory and practice
Supporting practitioners to implement explainable AI methods that meet stakeholder needs
Recognising that the difficulties of integrating innovative technologies within existing systems can cause promising AI tools to fail in practice, the field of MLOps focuses on the action required to deploy and maintain machine learning models in production. Effective deployment requires careful consideration of technical, organisational, and regulatory aspects of AI integration, and best practices are emerging across each of these areas. As technologies like Large Language Models offer to increase the accessibility of AI tools to organisations across sectors, the need for effective integration of AI into existing systems – taking into account concerns about safety, explainability, security, and other aspects of trustworthy AI – will become more pressing.
Research in robustness has created methods and approaches that seek to increase the safety and reliability of AI in deployment, creating methods that are better able to respond to real-world challenges. These challenges often fall outside the scope of the data on which a model was trained, and require the ability to adapt to different domains, detect and respond to dataset shift, or transfer learning between domains. A collection of approaches to increasing robustness have been developed, which now need translating into practice. A continuing challenge is how to scale these methods across complex AI systems, comprised of interacting components that mix automated and human elements, taking into account human needs and maintaining overall system functioning. AutoAI offers a route to enhancing the safe and effective deployment of such AI systems through AI-enabled performance monitoring and management. Alongside these technical considerations,
Alongside these technical considerations, understanding of the organisational and regulatory elements of AI integration are also evolving. Organisations deploying AI need to consider the skills needed by those working alongside AI systems, the experiences of users affected by those systems, and the regulatory requirements associated with AI in their sector. With the field developing fast, dynamic ways of responding to issues in deployment are needed, allowing organisations to trial new technologies without causing harm. Proposals for controlled testing environments such as regulatory sandboxes have already been created as part of current European policy developments and offer a way of stewarding new AI tools into deployment.
Improving performance in deployment through more robust AI methods and AI-enhanced monitoring
Connecting research and practice
Designing effective simulators and emulators
Understanding interactions with human users
Combining data-driven and structural insights
Increasing data availability
AI is a disruptive technology, bringing with it both opportunities to enhance societal wellbeing and potential harms. As interest in the field of AI ethics continues to increase, progress in the technical capabilities of AI systems has created AI tools that can help reduce the risks of those harms emerging. The field of human-centric AI has driven research advances across a variety of areas of societal concern, including algorithmic fairness, explainability and transparency, privacy preservation, and the detection of AI-generated ‘deepfakes’.
The wider challenge associated with AI ethics is how to connect technology development to societal interests, leveraging AI’s capabilities to help tackle areas of critical need while embedding human interests and concerns in its development.
With technology moving at pace, new applications emerging across sectors, and understanding of the risks and benefits of technology use in flux, continued dialogue between research and practice is needed to influence technology development towards beneficial outcomes.
Advancing foundational research to create human-centric AI
Putting ethical AI principles into practice
Designing governance frameworks for trustworthy AI
Developing AI applications in areas of societal interest
Progress in AI has historically come in waves. Over the last ten years, this has yielded AI technologies that are capable of delivering impressive performance when trained on tightly defined tasks. A new wave of technical advances, based on a trend towards large models, is extending these capabilities, creating AI systems that are broader purpose. These advances can deliver plausible results across a wider range of tasks than previously, and innovations in the coming years could extend their capabilities further.
The challenge that faces the field is how to translate this performance into real-world benefits for individuals, organisations, and society. Rapid progress in trustworthiness, security and privacy, explainability, AI integration, and AI ethics has provided a constellation of theoretical, methodological, and operational tools that can help deliver safe and effective AI systems. Continued support to drive further advances in these areas – advancing the technical capabilities or AI, overcoming current limitations, and understanding what works in practice – can help translate this progress into economic and social benefits.
ELISE is driving a new wave of research and development to deliver AI ‘made in Europe’. Together, its 14 research programmes create AI methods, techniques, and toolkits that are technically innovative, safe, and effective in deployment, while being aligned with social needs. By combining our research agenda with initiatives to attract top talent to Europe, train the next generation of AI researchers, and enhance local start-up and innovation networks, ELISE is creating a European AI ecosystem of excellence and trust.
Methods and tools to analyse real-world, multi-modal data; Strengthen core machine learning capabilities, through methodological and theoretical advances, such as techniques to bridge between data-driven and domain knowledge; Interrogate workings of complex systems through advances in simulation, emulation, and causality.
AI that is robust under dynamic or uncertain conditions; Human-centric tools that are effective as decision-support; Methods to enhance explainability in decision-making.
Techniques for trustworthy AI; Deployed AI that is integrated into areas of critical need; AI research and development that engages stakeholder perspectives.
