Docs / Concepts / NPU, DSP & DMA abstraction

NPU, DSP & DMA abstractionv0.2.0

Everything above the HAL is hardware-agnostic. The syn_hal_* interfaces abstract the three accelerators an inference pipeline touches — the NPU that runs the model, the DSP that runs pre/post-processing math, and the DMA engine that moves data — so the inference engine, memory manager, and your application never see silicon-specific code.

One interface, multiple backends

Each HAL is a fixed C function interface declared once in include/synaptic/ and implemented by more than one backend. The build system selects the backend at configure time: on MCXN947-series SoCs it links the hardware drivers from src/hal/mcxn947/, and on every other target (QEMU included) it links the software fallbacks from src/hal/stub/. Both backends export the same symbols, so swapping them requires zero changes to application code — the same hello_inference sample binary logic runs against the Neutron backend on the board and the stub on QEMU.

The abstraction is deliberately sized so that porting to a different accelerator — an Arm Ethos-U55, say, or a RISC-V NPU — means implementing one backend file, with no changes to the inference engine above.

The NPU state machine

The NPU HAL models the accelerator as an explicit four-state machine plus a load/set/invoke/get execution flow:

typedef enum {
    SYN_NPU_STATE_IDLE,
    SYN_NPU_STATE_BUSY,
    SYN_NPU_STATE_ERROR,
    SYN_NPU_STATE_SUSPENDED,
} syn_npu_state_t;

/* Lifecycle */
int  syn_hal_npu_init(void);
int  syn_hal_npu_get_caps(syn_npu_caps_t *caps);
syn_npu_state_t syn_hal_npu_get_state(void);

/* Execution */
int  syn_hal_npu_load_model(const uint8_t *model_data, size_t model_size);
int  syn_hal_npu_set_input(uint8_t index, const void *data, size_t size);
int  syn_hal_npu_invoke(void);                              /* Blocking     */
int  syn_hal_npu_invoke_async(syn_npu_done_cb_t cb, void *user_data);
int  syn_hal_npu_get_output(uint8_t index, void *data, size_t *size);

/* Power management */
int  syn_hal_npu_suspend(void);
int  syn_hal_npu_resume(void);

Explicit state matters because the NPU is a shared, single-owner resource: without it, a second job invoking mid-inference or a resume on a never-suspended device would be silent corruption. Instead, every entry point checks state and fails deterministically — operations on a BUSY NPU return -EBUSY, calls before init or before a model is loaded return -EPERM, and resume from any state other than SUSPENDED returns -EINVAL. These state guards are exactly what the scheduler will lean on for layer-boundary preemption in a later phase (the Phase 2 scheduler dispatches whole pipelines and runs them to completion). syn_hal_npu_get_caps() reports the backend's name, throughput, scratch requirement, supported dtypes, and async capability, so upper layers can adapt without #ifdefs.

The deterministic stub and hardware-free CI

The stub backend is not a mock that returns canned success — it simulates the full NPU lifecycle, including every state transition and error path, with a deterministic fake inference. Its output is a 10-class classification computed from the input itself:

/* Simple hash: sum input bytes to pick "winner" class */
uint32_t sum = 0;

for (size_t i = 0; i < stub.input_size; i++) {
        sum += stub.input_buf[i];
}

uint8_t winner = sum % stub.output_size;

stub.output_buf[winner] = 127; /* Max INT8 confidence */

Because the winning class is the sum of the input bytes modulo 10, a test can compute the expected prediction from its input and assert on it — inference results are repeatable, byte-for-byte, on any machine. This is what makes hardware-free CI possible: the entire 133-test suite runs on qemu_cortex_m3 with no board attached, exercising real state machines and real error paths rather than mocks. The stub accepts models up to 256 KB and inputs up to 1024 bytes (the Neutron driver sizes its input buffer at 96×96×3 for image workloads and, since v0.4.0, accepts models up to the 440 KB flash slot capacity), and simulates inference latency so profiling code has something real to measure.

DSP operations

The DSP HAL covers the vector math that brackets an NPU invocation — normalization on the way in, softmax and argmax on the way out, FFT for audio front-ends, and matrix multiply:

The DMA HAL

The DMA HAL exists for zero-copy data movement: streaming a camera frame directly into a tensor buffer, or a tensor out to a display, without the CPU touching each byte. The interface defines channel-based transfers between camera, SPI, and memory endpoints, with a circular flag for auto-restarting streaming transfers and a completion callback per channel:

typedef struct {
    syn_dma_periph_t src_periph;    /* CAMERA, SPI, or MEMORY      */
    syn_dma_periph_t dst_periph;
    void            *src_addr;
    void            *dst_addr;
    size_t           transfer_size;
    bool             circular;      /* Auto-restart for streaming  */
} syn_dma_config_t;

int  syn_hal_dma_init(void);
int  syn_hal_dma_configure(int channel, const syn_dma_config_t *config);
int  syn_hal_dma_start(int channel, syn_dma_cb_t callback, void *user_data);
int  syn_hal_dma_stop(int channel);
int  syn_hal_dma_get_remaining(int channel, size_t *remaining);