In Android 17 and higher, the Neural Processing Unit
(NPU) Manager coordinates model admission control and workload priority across
apps and system services. Rather than allowing background jobs to starve
latency-sensitive foreground tasks, the npu system service tracks app
lifecycle states from ActivityManager and relays effective scheduling rules to
the vendor NPU driver through the android.hardware.npu Android Interface
Definition Language (AIDL) hardware abstraction layer (HAL).
Framework API for model load arbitration
Before calling low-level neural network libraries or loading large model files,
framework clients interact with android.npumanager.NpuManager to reserve model
capacity. NpuManager is a system API (@SystemApi), not a public app SDK API,
and callers must hold the android.permission.ACCESS_NPU_MODEL_MANAGER_API
permission. Clients first build a model load request and then run the
asynchronous request and approval flow.
Build a model load request
A model load reservation is represented by ModelLoadRequest. This object
contains the following parameters:
- Unique request identifier
- Estimated model size class:
NPU_MODEL_SIZE_LESS_THAN_1GB,NPU_MODEL_SIZE_BETWEEN_1GB_AND_2GB, orNPU_MODEL_SIZE_GREATER_THAN_2G - Intended workload priority, such as
NPU_MODEL_PRIORITY_BACKGROUND,NPU_MODEL_PRIORITY_NORMAL, orNPU_MODEL_PRIORITY_OPPORTUNISTIC
The following Java example builds a ModelLoadRequest for a model larger than
2 GB with normal execution priority:
ModelLoadRequest request = new ModelLoadRequest.Builder(requestId)
.setSize(NPU_MODEL_SIZE_GREATER_THAN_2G)
.setPriority(NPU_MODEL_PRIORITY_NORMAL)
.build();
Execute the request and approval flow
Clients submit the reservation asynchronously using the requestCanLoadModel
method:
npuManager.requestCanLoadModel(request, callback, executor);
When NPU capacity is evaluated or system pressure changes, the framework
notifies the client through ModelLoadRequestCallback with the following
events:
onCanLoadModel(request, status, listener): Fired when the request is approved. The client receives anNpuManager.ModelLoadStatusListenertoken. After loading the model into NPU memory, the client must calllistener.notifyModelLoaded(request).onRequestUnloadModel(request)oronRequestUnloadModel(request, reason): Fired when the system experiences resource contention, such as an incoming foreground workload or thermal spike, and requires the client to release its model. After releasing NPU resources, the client must calllistener.notifyModelUnloaded(request).onModelLoadRequestComplete(request, status): Notifies the client of terminal request lifecycle transitions, such as cancellation.
Clients can cancel pending requests at any time by calling
cancelModelLoad(request).
Scheduling HAL integration
Device-specific vendor implementations integrate with the platform scheduler by
implementing the IScheduling.aidl service interface in
the android.hardware.npu package.
Configure UID priorities and access control
The npu system service pushes per-UID priority values and access rules to the
vendor HAL using setSchedulingConfigs and updateSchedulingConfigs with the
SchedulingConfig.aidl parcelable:
package android.hardware.npu;
@VintfStability
parcelable SchedulingConfig {
const int MIN_PRIORITY = 0;
const int MAX_PRIORITY = 1000;
int uid;
int priority;
boolean hasDirectAccess;
boolean canAttributeOtherUid;
}
Each field in SchedulingConfig controls how the vendor NPU driver arbitrates
work submitted by a user identifier (UID):
| Field or constant | Description |
|---|---|
MIN_PRIORITY and MAX_PRIORITY |
Valid priority bounds (0 to 1000), where
MIN_PRIORITY (0) represents the highest
scheduling priority and MAX_PRIORITY (1000)
represents the lowest scheduling priority.
|
uid |
Linux UID of the client app or system service. |
priority |
Platform importance score in the range 0 to
1000, where 0 is highest priority. Derived
from ActivityManager process state so foreground apps
outrank cached or background processes.
|
hasDirectAccess |
Whether uid is permitted to open direct NPU
execution sessions. If false, direct session creation or
execution fails with EX_SECURITY.
|
canAttributeOtherUid |
Whether a privileged intermediary service, such as AICore, is
authorized to attribute NPU work and power usage to a calling client
originalUid.
|
Vendor NPU schedulers must compute an effective priority in the range 0 to
2000 (where 0 is highest priority) by adding SchedulingConfig.priority
(0 to 1000) to the client-requested job priority normalized to 0 to
1000, and schedule hardware execution queues according to that effective
priority. For example, if a background app submits a high-priority job, its
higher priority score lowers its effective priority (producing a higher
numeric value) so it doesn't delay foreground camera or UI workloads.
Report task status and attribution
Vendor drivers report the lifecycle of NPU execution groups back to the platform
using WorkInfo.aidl:
package android.hardware.npu;
import android.hardware.npu.Uuid;
@VintfStability
parcelable WorkInfo {
int id;
@nullable Uuid groupId;
int uid;
int debugPid;
int originalUid;
@nullable String debugFeatureId;
int jobPriority;
int effectivePriority;
long timestampMs;
int deviceNumber;
}
When an intermediary service with canAttributeOtherUid enabled executes a
model on behalf of a client app, originalUid carries the originating app UID
while uid identifies the submitting service. This separation allows platform
telemetry and power accounting to bill work to the requesting app instead of the
intermediary daemon.
Configure event debouncing
When an ISchedulingCallback listener is registered with
IScheduling.setCallback, the HAL debounces related work notifications across
at least ISchedulingCallback.DEBOUNCE_DURATION_MS (50 ms). Work items are
considered related when they share the same uid, originalUid, and
debugFeatureId, or share the same groupId. Debouncing suppresses rapid
callback storms during streaming or multi-step inference runs, reducing Binder
overhead and log flooding.
Callback lifecycle states are reported through three methods on
ISchedulingCallback:
onWorkRequested: Workload is received by the vendor service or driver.onWorkStarted: Hardware execution begins, tagged with aStartReason:StartReason.INITIAL: Initial execution run for a new app request.StartReason.RESUMED: Execution resumed after being paused (PAUSED).
onWorkEnded: Hardware execution stops afterDEBOUNCE_DURATION_MSelapses without related follow-up work, tagged with anEndReason:EndReason.COMPLETED: Execution completed normally.EndReason.CANCELLED_USER: Execution cancelled by the calling app.EndReason.CANCELLED_SYSTEM: Execution preempted and cancelled by higher-priority work.EndReason.PAUSED: Execution temporarily paused to run higher-priority work and eligible to resume withStartReason.RESUMED.EndReason.FAILED: Execution failed due to a driver or hardware error.
Validation
Validate NPU scheduling implementations with the following test suites:
- Vendor Test Suite (VTS): Run
VtsHalNpuSchedulingTargetTest(packaged inVtsHalNpuTargetTest) to verifyISchedulingconfiguration updates, effective priority ordering, UID attribution rules, andISchedulingCallbackdebouncing. - Compatibility Test Suite (CTS): Run
CtsNpuManagerMoblyTestCasesto verify end-to-end foreground versus background NPU workload prioritization. For test setup instructions, see Run NPU Manager Mobly tests.