AI Procurement in a Box: Lessons in best practice from Brazil

Artificial intelligence (AI) technologies can transform the deployment of government operations by expanding services, improving efficiency, and implementing cost-reduction improvements.

However, the inherent risks of using these technologies can be challenging to predict, especially in governmental entities that do not yet have adequate institutional capacities to implement AI systems.

Despite some relevant initiatives in this area, governments often suffer the consequences of a lack of resources and qualified staff to accompany the hiring and development of AI solutions, which delays the diffusion of this technology in the public sector.

With these considerations in mind, the World Economic Forum published "Unlocking Public Sector AI: AI Procurement in a Box" in 2020, a toolkit with 10 guidelines designed to assist governments in properly assessing their technological and contextual needs and the risks associated with contracting AI solutions.

A product of collaboration between public sector actors, the toolkit guidance drew from pilots conducted in the United Kingdom, and findings derived from workshops in Bahrain, India and the United Arab Emirates.

To build upon this work and bring these practices to Brazil, the Centre for the Fourth Industrial Revolution Brazil (C4IR Brazil) implemented the AI Procurement in a Box toolkit in partnership with São Paulo Metrô and the Hospital das Clínicas.

We intended to enter the existing debate on government innovation procurement and benefit the public sector's ethical use of artificial AI and institutional preparation for Fourth Industrial Revolution (4IR) technologies.

In addition, we wanted to look at how we can adapt the toolkit for the Brazilian context to demonstrate how to scale these best practices around the world.

As a result, we published a toolkit on the former with specific recommendations for the Brazilian context, illustrating the two case studies mentioned above and connecting them to particular challenges in procuring AI in Brazil, that we identified through conversations with relevant stakeholders.

One of our significant challenges is that the public procurement framework in Brazil is complex and relatively restrictive, which results in officials following formulaic procedures that are not tailored to incentivize innovation and the holistic evaluation of project proposals. Many public servants are not familiar with the innovation or AI procurement practices and rely on reverse-bidding processes that might not adequately assess complex projects.

Through our pilot project with the São Paulo Metrô ­– one of the largest railway networks in Latin America with 3.7 million passengers per day – we were able to test out the guidelines on a concrete procurement process.

Due to the high traffic volume the subway system deals with daily, many maintenance interruptions arise throughout the day, delaying service for approximately 1,840 minutes of interference between 2016 and 2020.

To increase efficiencies, the Metro is preparing to buy a real-time online predictive modelling tool that can identify segments of rail that need maintenance and minimize service interruptions to benefit both passengers and the subway system.

By applying the AI Procurement in a Box guidelines, the Metro was able to leverage Brazilian innovation procurement legislation to engage essential stakeholders and effectively communicate project specifics to potential suppliers.

The Metro also applied Brazil's first-ever Algorithmic Impact Assessment to evaluate the risks of using AI and help formulate effective mitigation strategies.

The case study on the Hospital das Clínicas of São Paulo looked at one institution's journey into technical and data access maturity, mainly to ethically use AI in an area of public scrutiny.

Due to its multifaceted attributions, the institution envisioned organizing its more than 60 systems and protocols into a cohesive apparatus that would enable it to efficiently use the enormous volume of data it collects in its other fields of work – teaching, research, and innovation.

To overcome a myriad of obstacles in achieving this goal, the hospital focused on integrating data registries both within and outside the hospital by creating a data lake; technical training for the workforce and optimization of initiatives; data architecture and management, and data harmonization between different areas.

Through these actions, the Hospital das Clínicas was able to create a roadmap for leveraging its data to adopt AI and other innovations better, safely and securely.

The lessons from these two case studies illustrated the importance of the AI Procurement in a Box project and how the World Economic Forum guidance can continue to evolve and benefit public sector officials, particularly in the Global South and low resource areas.

As an extension to the original project, the Forum and C4IR Brazil have published 'Unpacking AI Procurement in a Box: Insights from Implementation'. This white paper discusses various aspects of procurement and public sector adoption, such as AI maturity for public sector organizations – from basic infrastructure, to data governance, to people and culture.

We also focused on looking beyond risk assessments and towards a policy mix governing these technologies. What kinds of oversight tools, beyond human in the loop, are necessary to ensure safety and efficacy?

To facilitate the continued global implementation of AI Procurement in a Box, we have included a set of indicators for understanding the success of an AI procurement project to guide any implementing partners in the future.

We hope that this project will continue to be implemented, not only through the affiliated C4IR centres but also by governments worldwide. Through this, AI Procurement in a Box will grow and always share necessary best practices for responsible, innovative public sector AI globally.