On-device AI is a deployment shift, not the end of cloud AI
The first large wave of generative AI investment was concentrated in cloud training and data-centre inference. The next wave is increasingly about where AI is used: PCs, vehicles, cameras, industrial controllers, robots and other devices that must perceive and respond in a real operating environment.
That makes on-device AI strategically important, but it does not mean that the cloud disappears. Training, fleet-level learning and the largest models remain natural cloud workloads. The device handles the work that benefits from low latency, local data, offline operation, predictable service cost or direct control of sensors and actuators. Many successful products will therefore use a cloud-edge-device architecture rather than choose only one location.
| Workload location | Best suited to | Main advantage | Main constraint |
|---|---|---|---|
| Cloud | Training, very large models, shared enterprise services | Scale and model capacity | Network dependency, latency, service and data-governance cost |
| Edge server or gateway | Factory cells, stores, campuses and multi-camera systems | Local aggregation and manageable power envelope | Infrastructure and maintenance |
| Device | Robots, cameras, vehicles, PCs and offline assistants | Fast response, privacy and availability | Tight memory, power, thermal and software limits |
| Hybrid | Products that need local response plus cloud intelligence | Flexible workload placement | More complex system and lifecycle management |
The policy direction supports this shift. China's 2025 *AI Plus* action plan set a target for the adoption rate of next-generation intelligent terminals and agents to exceed 70% by 2027. The multi-ministry *AI Plus Manufacturing* action plan published in January 2026 also calls out real-time operation, on-device deployment and reliability as industrial-model requirements. These are adoption targets and policy signals, not proof that every AI terminal project will reach commercial scale.
Why the silicon selection rule changes at the device
Peak TOPS is useful, but it is not a complete purchasing specification. An edge platform succeeds only when the selected model fits its memory, supported operators, precision modes, latency target and power envelope. Camera inputs, video codecs, ISP capability, safety islands, real-time control and operating system support may matter more than a headline accelerator number.
For engineers, the real benchmark is an approved model running with the target input size, context length, batch size, precision and thermal condition. For buyers, the corresponding question is whether the exact processor, memory, PMIC and module configuration can remain available through qualification and production.
China-based edge AI platforms: five different positions
The following companies should not be forced into one ranking. They address different power envelopes, qualification regimes and software environments. Specifications are vendor-published figures and should be validated on the customer's own model before design approval.
| Platform route | Publicly stated position | Strong-fit projects | What must be verified |
|---|---|---|---|
| Rockchip RK3588 and RK182X | RK3588 integrates a 6 TOPS NPU; RK1820/RK1828 add dedicated model acceleration alongside a host SoC | AIoT, vision, industrial terminals, service robots and compact local assistants | RKNN model conversion, host pairing, memory capacity, sustained thermals and exact SDK version |
| Horizon Robotics Journey and Starry | Journey is a scaled automotive-compute family; the 5 nm Starry chip is specified at up to 650 TOPS BPU compute and 273 GB/s bandwidth | Production ADAS, cockpit and vehicle-agent platforms | Functional-safety scope, OEM software baseline, sensor interfaces, automotive lifecycle and approved module design |
| Black Sesame Huashan A2000 | A2000 family targets physical AI and assisted-driving tiers, with the top family member stated at up to 1,000 TOPS | Cockpit AI, urban NOA and higher-level vehicle compute programmes | Which member is sampled or production-ready, model toolchain, safety evidence, memory and programme schedule |
| Moore Threads MTT E300 / “Yangtze” SoC | E300 module provides up to 50 TOPS dense INT8 compute with CPU, GPU, NPU and media engines | Industrial vision, edge servers, local agents and multimodal terminals | Module power, operating-temperature version, MUSA software compatibility, memory option and model throughput |
| Hygon CPU/DCU ecosystem | General-purpose CPU and accelerator route extending from data centres toward edge and industry systems | Industrial PCs, secure industry infrastructure and edge servers with existing software stacks | Exact processor/module availability, workload portability, system-vendor support and deployment-level benchmarks |
Rockchip: host SoC plus AI coprocessor
Rockchip's route is attractive when a product needs both embedded interfaces and local AI. The RK3588 combines Arm CPU cores, video functions and a 6 TOPS NPU. The newer RK182X approach pairs an RK1820 or RK1828 accelerator with a host such as RK3588 or RK3576. Rockchip's March 2026 RKNN3 release reports a Qwen2.5-3B decode rate of 102.01 tokens per second on RK182X under its stated test conditions.
That number demonstrates a vendor-tested configuration; it should not be used as a universal LLM speed. Prompt length, quantisation, memory, sampling settings and cooling can materially change latency and sustained throughput. An RFQ should therefore identify the host SoC, accelerator, memory build and software baseline together.
Horizon Robotics: automotive edge AI at production scale
Horizon Robotics reported that cumulative Journey-family production shipments passed ten million units in August 2025. Its April 2026 Starry announcement positions a 5 nm vehicle-agent processor with up to 650 TOPS of BPU compute and 273 GB/s memory bandwidth, integrating cockpit and assisted-driving workloads with physical isolation.
Automotive procurement cannot stop at the processor name. The approved compute board, safety concept, memory and PMIC combination, software release and vehicle programme timing are part of the design. “650 TOPS” is not a drop-in substitution argument for another automotive SoC.
Black Sesame Intelligence: a physical-AI automotive family
Black Sesame Intelligence introduced the Huashan A2000N, A2000L, A2000U and A2000X as a tiered family for cockpit agents, urban NOA, L3 functions and Robotaxi-class applications. The company states a maximum of 1,000 TOPS for the family and support for multi-chip cooperation.
This is a roadmap and family-level positioning statement. Before a purchase, engineers should confirm the exact orderable device, silicon and software maturity, supported precision, memory subsystem, functional-safety deliverables and the milestone that applies to the customer's programme. Comparing only the largest TOPS number with another vendor's chip is not a valid performance test.
Moore Threads: one heterogeneous SoC from edge module to AI PC
The MTT E300 uses Moore Threads' “Yangtze” intelligent SoC and combines CPU, GPU, NPU, video and other engines. Official specifications state up to 50 TOPS dense INT8 compute and a -20°C to +65°C operating range for the module. The same SoC strategy also appears in the company's AIBOOK and AICUBE products.
The purchasing advantage of a module can be faster system integration, but the module itself becomes a controlled BOM item. Memory capacity, connector and mechanical revision, cooling, operating system, driver and MUSA toolkit version must all be frozen for production.
Hygon: an edge-system route rather than a low-power terminal SoC
Hygon is better treated as an industrial and edge-computing platform route than as a direct alternative to a low-power camera SoC. CPU compatibility, accelerator support and domestic computing ecosystems can be valuable where existing industrial applications, security requirements and system integration matter.
Projects should compare complete systems and validated workloads. Broad claims about near-zero migration cost or compatibility with almost every model are not a substitute for application testing, driver qualification and support commitments from the system supplier.
What on-device AI does to the component BOM
The AI processor is only the most visible line item. Moving inference into a device can raise requirements across the surrounding BOM.
| BOM area | Why demand changes | Engineering and sourcing check |
|---|---|---|
| LPDDR/DDR memory | Model weights, KV cache, vision buffers and shared CPU/NPU workloads | Capacity, bandwidth, package, temperature grade and approved vendor list |
| eMMC/UFS/NAND/NOR | Operating system, models, logs and over-the-air images | Endurance, retention, managed-NAND firmware, lifecycle and secure boot support |
| PMIC and DC/DC | Accelerators create fast, high-current load transients | Rail sequence, peak current, efficiency, thermals and reference-design approval |
| Ethernet, PCIe and SerDes | Cameras, sensors, storage and coprocessors move more data | Lane count, data rate, protocol, signal integrity and connector design |
| Clocking and timing | High-speed compute and sensors need stable references | Jitter, output format, temperature grade and clock-tree approval |
| Thermal and mechanical parts | Peak and sustained inference are different loads | Sustained benchmark, airflow, heatsink, enclosure and ambient temperature |
This is why a processor shortage is not the only supply risk. A project can secure the AI SoC and still be blocked by the exact LPDDR device, managed NAND, PMIC, connector or production-approved module revision.
Engineering selection checklist
1. Freeze the model, framework, input dimensions, context length and required precision before comparing accelerators. 2. Measure time to first token, sustained token rate or frame rate, end-to-end latency, memory use and wall power on the target hardware. 3. Confirm that every required operator is supported without excessive CPU fallback or an unmaintainable custom implementation. 4. Test performance after thermal stabilisation, not only during a short demo. 5. Include ISP, codec, sensor, real-time control, security and safety needs in the platform comparison. 6. Freeze the OS, driver, compiler, model-conversion toolkit and firmware versions used for production qualification.
RFQ checklist for an edge AI platform
- Exact processor, accelerator or module orderable code—not only the family name.
- Target quantity, prototype and mass-production dates, and expected programme life.
- Required operating-temperature range and any automotive or industrial qualification.
- Memory density, organisation, speed grade and approved manufacturers.
- Storage type, capacity, endurance and managed-NAND firmware restrictions.
- PMIC and power-stage references approved by the platform vendor or customer.
- Packing, date-code window, lot consistency, traceability and storage condition.
- SDK, BSP, driver and model-toolchain version tied to the qualified hardware.
Current stock, date code, lot condition and delivery must be confirmed before purchase. For roadmap devices, request the vendor's current product-status and programme documentation rather than treating a launch announcement as evidence of unrestricted volume availability.
Conclusion: the opportunity is local intelligence, not a TOPS race
On-device AI is likely to be one of the most important deployment directions after the cloud-training build-out, because real products need local perception, fast response, privacy and offline operation. The winning architecture will often be hybrid: the device performs immediate inference and control while the cloud supplies larger models, updates and fleet-level intelligence.
For engineers, success comes from model-to-hardware fit and a stable software stack. For procurement teams, it comes from controlling the complete platform BOM and lifecycle—not buying the processor with the largest headline number.
Primary references
- China State Council: AI Plus action plan
- MIIT: AI Plus Manufacturing action plan
- Rockchip RK3588 product page
- Rockchip RKNN3 for RK1820/RK1828
- Horizon Robotics Starry announcement
- Horizon Robotics Journey shipment milestone
- Black Sesame Intelligence A2000 family
- Moore Threads MTT E300
Need stock, date-code or package confirmation?
Send the part number, quantity, target date code and packaging requirements. LimChip will check available lots and RFQ details before you place the order.
Send RFQ