# On-device AI in 2026: what NPUs and unified memory actually do

- **Published:** 2026-04-22
- **Updated:** 2026-10-12
- **Category:** Hardware & Infrastructure
- **Author:** Sinthetix
- **URL:** https://sinthetix.com/blog/on-device-ai-npus-and-local-inference

> NPU ratings from Qualcomm, AMD and Intel compared, plus the memory constraint that decides which models can run locally on a laptop.

Laptop makers now advertise "AI PCs" by a single number: NPU TOPS. The figure is real, but it is not the one that decides whether a business can run a useful language model on an employee's machine. This post compares the current NPU ratings from the three main laptop chip vendors and explains the constraint that matters more.

**Key takeaways**

- Vendor-rated NPU peaks range from 50 to 80 TOPS across current Qualcomm, AMD and Intel laptop chips, and the ratings are not directly comparable.
- Memory capacity and bandwidth, not TOPS, set the practical limit on local language models.
- Google's Gemma 4 (April 2) ships open models sized for phones, laptops and a single workstation GPU under Apache 2.0.
- Local inference suits privacy-sensitive, latency-sensitive and offline tasks; it does not replace large hosted models.

## The NPU ratings

An NPU (neural processing unit) is a low-power block designed to run neural-network math efficiently. TOPS means trillions of operations per second, usually quoted as a peak for 8-bit integer math. These are the current vendor figures:

**Peak NPU rating by vendor, current laptop generation**

Top-end figures as published at launch

Unit: TOPS

| Category | NPU TOPS |
| --- | --- |
| Qualcomm Snapdragon X2 Elite | 80 |
| AMD Ryzen AI 400 (top SKU) | 60 |
| Intel Core Ultra Series 3 | 50 |

*Qualcomm 80 TOPS (INT8) per GSMArena \[3\]; AMD 'up to 60' per AMD \[1\]; Intel 50 per PCPer \[2\]. Peak vendor ratings, not measured performance. Source: [3], GSMArena*

- **Qualcomm Snapdragon X2 Elite:** 80 TOPS (INT8) on TSMC 3 nm, with first laptops expected in spring 2026. The Extreme model has a 192-bit memory bus at 228 GB/s; the other two models are 128-bit at 152 GB/s, all with LPDDR5X [3].
- **AMD Ryzen AI 400:** up to 60 TOPS on the top SKU, with most others at 50 or 55. Laptops began shipping from Q1 2026 from Acer, ASUS, Dell, HP, GIGABYTE and Lenovo, and desktop parts are planned for Q2 [1].
- **Intel Core Ultra Series 3 (Panther Lake):** 50 TOPS from the NPU, plus 120 TOPS from the Arc GPU, for a quoted platform total of 180 TOPS. Memory support includes DDR5 and LPDDR5 up to 9,600 MT/s, with laptops available from January 27 [2].

Two cautions. First, Intel's 180 TOPS headline adds NPU and GPU, so it is not comparable with the other vendors' NPU-only numbers [2]. Second, these are peak ratings from the vendors. Sustained performance depends on cooling, power limits and software support, and the numbers say nothing about whether the applications your staff use take advantage of the NPU.

## The constraint that matters: memory

A language model's weights have to sit in memory while it runs, and generating each word means reading a large share of them. Two consequences follow, which is why the memory column in the specs deserves as much attention as TOPS.

**Capacity decides what fits.** Google's Gemma 4 family shows the range. The E2B and E4B models (roughly 2 and 4 billion effective parameters) target phones, Raspberry Pi and Jetson Orin Nano devices. The 26B mixture-of-experts model activates 3.8 billion parameters per inference, which speeds up token generation. The 31B dense model is aimed at maximum quality, and its unquantized weights fit on a single 80 GB NVIDIA H100 [4]. All four support offline operation and image and video input [4].

**Bandwidth decides how fast it feels.** That is why Qualcomm's 228 GB/s versus 152 GB/s memory bus figures are relevant to local models in a way the NPU rating alone is not [3]. AMD's own announcement makes the point about scale: its Ryzen AI Halo developer platform, with up to 128 GB of unified memory, is described as capable of running models of up to 200 billion parameters locally, and is planned for Q2 2026; the release makes no such claim for the Ryzen AI 400 laptop chips [1].

On quality, Google reports Arena AI text scores of 1452 for Gemma 4 31B and 1441 for the 26B model, against 1365 for the previous Gemma 3 27B [4]. These are vendor-cited rankings among open models, and Arena scores measure general chat preference, not performance on your documents.

## Where local inference fits

| Good fit | Poor fit |
| --- | --- |
| Summarizing or classifying text that must not leave the device | Long, complex analysis where the best available model matters |
| Offline or low-connectivity field work | Workloads needing very large context or many users at once |
| Latency-sensitive assistance such as dictation and local search | Anything your IT team cannot patch, monitor or update centrally |

## What to check before buying hardware

1. **Memory size and type** in the configuration you will actually purchase, not the maximum the chip supports.
2. **Which of your applications use the NPU** today. If none do, the TOPS rating has no effect yet.
3. **The model you intend to run**, and whether it fits quantized in available memory with room left for the operating system.
4. **Manageability.** AMD offers PRO variants aimed at enterprise security and manageability [1]; confirm equivalents for any fleet you deploy.

For businesses, treat NPU TOPS as a minimum threshold and size laptop purchases for local AI by memory instead. Pilot one concrete task, such as private document summarization, on a single device before refreshing a fleet.

## Sources

1. [AMD Expands AI Leadership Across Client, Graphics and Software with New Ryzen, Ryzen AI and AMD ROCm Announcements at CES 2026](https://ir.amd.com/news-events/press-releases/detail/1270/amd-expands-ai-leadership-across-client-graphics-and-software-with-new-ryzen-ryzen-ai-and-amd-rocm-announcements-at-ces-2026), AMD, 2026-01-05
2. [CES 2026: Intel Launches Core Ultra 3 Series Panther Lake Processors](https://pcper.com/2026/01/ces-2026-intel-launches-core-ultra-3-series-panther-lake-processors/), PCPer, 2026-01-07
3. [Snapdragon X2 Elite and X2 Elite Extreme announced with sizeable performance and efficiency gains](https://www.gsmarena.com/snapdragon_x2_elite_and_x2_elite_extreme_announced_with_sizeable_performance_and_efficiency_gains-news-69658.php), GSMArena, 2025-09-25
4. [Google's Gemma 4 is now available with Apache 2.0 licensing for the first time](https://the-decoder.com/googles-gemma-4-is-now-available-with-apache-2-0-licensing-for-the-first-time/), The Decoder, 2026-04-02

## How Sinthetix helps

An engineering studio for AI agents, automation, and internal tooling. We build the systems that make operators scale.

[Our services](https://sinthetix.com/services) | [Talk to us](https://sinthetix.com/contact)

Sinthetix is an AI engineering studio. We ship agents, automation, and internal tools for operators who need results, not reports. [About Sinthetix](https://sinthetix.com/about)