On-device AI would democratise GenAI and ensure inclusive participation in the economy

The generative AI (GenAI) revolution is here. The pace of innovation and development in GenAI, as well as its adoption, is unprecedented and its impact will be transformative as GenAI applications become indispensable companions and business enablers.

Over 93% of employers expect to use GenAI within the next five years to increase innovation and creativity, automate repetitive tasks and boost learning. According to McKinsey, the technology’s total economic benefit could add $2.6 to $4.4 trillion annually across more than 60 use cases – by comparison, the United Kingdom’s gross domestic product in 2021 was $3.1 trillion.

While the majority of GenAI development has been focused in the cloud – and the cloud will continue to play an indispensable role – it is quickly evolving to run directly on devices, including smartphones, personal computers (PCs), vehicles, mixed reality and internet of things (IoT) devices, Wi-Fi access points and more. This will be key to realizing GenAI’s potential to accelerate digital transformation.

Starting this year, we expect dramatic growth in a new generation of affordable devices that can run GenAI models locally, including smartphones, PCs, vehicles, mixed reality, IoT devices and network equipment. Devices have a unique role, as executing GenAI on device allows for increased responsiveness, more precise personalization, greater reliability and enhanced privacy.

The devices’ performance and efficient AI capabilities will also allow GenAI to run pervasively and operate proactively. Users can have digital assistants anticipating their needs rather than just reacting to requests through clicks and taps. Applications are on the way to use these capabilities – enabling entirely new experiences and applications focused on productivity, content creation, education, research and development, enterprise applications and more.

With on-device and edge AI, applications can run continuously, enabling them to learn about the user, their preferences and behaviours and utilize complementary external data. This invaluable context and content can enable more relevant, specialized and individualized responses to users, including for key topics such as education and healthcare.

Also, since computations are performed locally, on-device AI avoids the potential for latency while increasing reliability by being able to execute a query anywhere and anytime. This quicker response is crucial for applications requiring fast decision-making, such as voice assistants, augmented reality and gaming.

As generative AI is adopted, it will be vital for confidential and personal information to remain private. A key benefit of on-device and edge AI is that queries and personal and proprietary information can remain on device or on premise using private edge clouds. This enhanced privacy and security is essential for wide-scale trust and adoption across consumer and enterprise applications.

It also helps address the requirement to comply with privacy regulations, such as the European Union’s General Data Protection Regulation. Certainly, careful implementation will be required to balance benefits with protecting user data.

GenAI is one of the biggest transformations in computing – from cloud to devices. The cloud and devices will work together to elevate human capacity.

In a hybrid AI approach, workloads are distributed and coordinated among cloud and edge devices to provide optimal performance and efficiency across use cases. When the cloud and device use the same GenAI model, the device can give the cloud a head start. The data on the device also enables the AI application to be more precise because it has real-time context about the user.

Making use of distributed computing and processing more AI on device or in a hybrid approach will help manage costs associated with data centres.

On-device and edge AI reduces the burden of the data centres’ infrastructure and environmental costs. The annual global AI data centre cost could top $76 billion by 2028. However, according to Tirias Research, if 20% of GenAI processing workloads could be offloaded by running on the device or through hybrid processing, the global AI data centre cost would decline by $15 billion.

Further, a study found that one AI-generated image created in the cloud can require as much power as charging a smartphone. To illustrate the energy efficiency of running AI on mobile devices, we tested a commercial smartphone and were able to generate more than 400 images on a single battery charge using an optimized AI model.

As a new generation of always connected, smarter and more capable devices scale at the edge, working collaboratively with the cloud, they will help communities drive sustainable growth and innovation, unlock efficiencies, increase productivity and enable new business models.

The widespread availability of smartphones, PCs, smart vehicles and other devices represents a significant opportunity for individuals, enterprises and nations to take part in the benefits of GenAI. Enabling intelligent computing everywhere fosters greater opportunities for participation in the digital economy.