On-device AI in 2026: what NPUs and unified memory actually do

Laptop makers now advertise "AI PCs" by a single number: NPU TOPS. The figure is real, but it is not the one that decides whether a business can run a useful language model on an employee's machine. This post compares the current NPU ratings from the three main laptop chip vendors and explains the constraint that matters more.

The NPU ratings

An NPU (neural processing unit) is a low-power block designed to run neural-network math efficiently. TOPS means trillions of operations per second, usually quoted as a peak for 8-bit integer math. These are the current vendor figures:

• Qualcomm Snapdragon X2 Elite: 80 TOPS (INT8) on TSMC 3 nm, with first laptops expected in spring 2026. The Extreme model has a 192-bit memory bus at 228 GB/s; the other two models are 128-bit at 152 GB/s, all with LPDDR5X ^[3].

• AMD Ryzen AI 400: up to 60 TOPS on the top SKU, with most others at 50 or 55. Laptops began shipping from Q1 2026 from Acer, ASUS, Dell, HP, GIGABYTE and Lenovo, and desktop parts are planned for Q2 ^[1].

• Intel Core Ultra Series 3 (Panther Lake): 50 TOPS from the NPU, plus 120 TOPS from the Arc GPU, for a quoted platform total of 180 TOPS. Memory support includes DDR5 and LPDDR5 up to 9,600 MT/s, with laptops available from January 27 ^[2].

Two cautions. First, Intel's 180 TOPS headline adds NPU and GPU, so it is not comparable with the other vendors' NPU-only numbers ^[2]. Second, these are peak ratings from the vendors. Sustained performance depends on cooling, power limits and software support, and the numbers say nothing about whether the applications your staff use take advantage of the NPU.

The constraint that matters: memory

A language model's weights have to sit in memory while it runs, and generating each word means reading a large share of them. Two consequences follow, which is why the memory column in the specs deserves as much attention as TOPS.

Capacity decides what fits. Google's Gemma 4 family shows the range. The E2B and E4B models (roughly 2 and 4 billion effective parameters) target phones, Raspberry Pi and Jetson Orin Nano devices. The 26B mixture-of-experts model activates 3.8 billion parameters per inference, which speeds up token generation. The 31B dense model is aimed at maximum quality, and its unquantized weights fit on a single 80 GB NVIDIA H100 ^[4]. All four support offline operation and image and video input ^[4].

Bandwidth decides how fast it feels. That is why Qualcomm's 228 GB/s versus 152 GB/s memory bus figures are relevant to local models in a way the NPU rating alone is not ^[3]. AMD's own announcement makes the point about scale: its Ryzen AI Halo developer platform, with up to 128 GB of unified memory, is described as capable of running models of up to 200 billion parameters locally, and is planned for Q2 2026; the release makes no such claim for the Ryzen AI 400 laptop chips ^[1].

On quality, Google reports Arena AI text scores of 1452 for Gemma 4 31B and 1441 for the 26B model, against 1365 for the previous Gemma 3 27B ^[4]. These are vendor-cited rankings among open models, and Arena scores measure general chat preference, not performance on your documents.

Where local inference fits

What to check before buying hardware

1. Memory size and type in the configuration you will actually purchase, not the maximum the chip supports.

2. Which of your applications use the NPU today. If none do, the TOPS rating has no effect yet.

3. The model you intend to run, and whether it fits quantized in available memory with room left for the operating system.

4. Manageability. AMD offers PRO variants aimed at enterprise security and manageability ^[1]; confirm equivalents for any fleet you deploy.

For businesses, treat NPU TOPS as a minimum threshold and size laptop purchases for local AI by memory instead. Pilot one concrete task, such as private document summarization, on a single device before refreshing a fleet.