How on-device AI can help us cut AI's energy demand

The age of IT devices driven by artificial intelligence (AI) has arrived, much like the revolutions brought by personal computers and mobile devices. Each has brought undeniable and lasting impacts on society.

The evolution of computing followed three key phases:

AI is now following this same trajectory, with rapid advancements in AI chips driving a new technological wave.

AI chip power consumption has surged, particularly in data centres, which require vast energy resources to process large-scale data in real time. AI-driven data centres, primarily powered by graphics processing units (GPUs), now consume more electricity than entire nations, including South Africa and Indonesia.

Projections suggest this energy usage will more than double from 260 terawatt-hours in 2024 to 500 terawatt-hours in 2027. A single query in OpenAI's ChatGPT, for example, consumes 2.9Wh of electricity — roughly 10 times that of a Google search.

This massive power demand has raised environmental concerns, emphasizing the need for sustainable industry practices. The manufacturing sector alone accounts for over 40% of global power consumption, making it a focal point for energy efficiency improvements.

Governments are taking action. For example, Singapore has introduced regulations limiting data centre capacity due to energy shortages. The country currently operates more than 70 data centres, accounting for 60% of Southeast Asia’s total data centre capacity. It stopped further approvals between 2019 and 2022, with those under review subject to capacity restraints.

AI leaders acknowledge this challenge. At the 2024 World Economic Forum in Davos, Switzerland OpenAI CEO Sam Altman stated: “An energy breakthrough is necessary for future artificial intelligence, which will consume vastly more power than people have expected.”

AI is essential to the future, automating repetitive tasks and driving progress. However, for AI to be widely adopted across industries, its energy consumption must be drastically reduced.

Delivering sustainable AI requires both a refinement of the technology, which is already underway, as well as creative and ambitious policy-making.

Startups such as Groq, DeepSeek and DeepX are pioneering energy-efficient AI technologies that could shape an AI-driven, super-intelligent society.

Software optimization can help reduce power usage in servers, but on-device AI – where AI processing happens directly on the device rather than in a cloud data centre – is the most promising solution to this challenge.

Currently, AI systems rely heavily on cloud computing, which consumes significant energy to transmit data between edge devices and data centres. Additionally, high-performance GPUs and central processing units (CPUs) in data centres require substantial power.

By contrast, on-device AI processes data locally, eliminating the need for energy-intensive data transmission. AI chips designed for on-device processing prioritize energy efficiency over sheer computing power, resulting in a 100 to 1,000-fold reduction in energy consumption per AI task compared to cloud-based AI.

As a result, on-device AI is emerging as a game-changing technology that maximizes AI adoption while minimizing environmental impact. Governments must act quickly to create policies that accelerate its adoption.

To promote energy-efficient AI, a global “energy credit trading system” could provide financial incentives for companies that adopt low-power AI solutions. Under this system, businesses implementing energy-saving AI could trade energy usage credits, financially benefiting while reducing their environmental footprint.

DEEPX presented this innovation at the 2024 World Economic Forum Annual Meeting in Dalian, China.

A similar precedent exists in the electric vehicle (EV) industry. In the 2010s, government subsidies and tax incentives drove rapid EV adoption, leading to battery technology and charging infrastructure growth.

AI energy credits could play a similar role in managing the power consumption of AI models, ensuring a sustainable future.

The AI era is here to stay — but its success depends on making AI technology energy-efficient. Governments, businesses and innovators must work together to ensure that AI’s rapid growth does not come at an unsustainable cost.

With the right policies and technologies, we can create an AI-powered future that benefits society and the planet.