Docs / API reference / syn_infer.h

syn_infer.hv0.5.0

<synaptic/syn_infer.h> is the inference engine and pipeline scheduler API. A pipeline chains pre-processing stages, a registered model, and post-processing stages into a single unit; jobs are submitted against a pipeline with a priority, an optional deadline, and a completion callback, then awaited, cancelled, or polled for results. The header also declares a synchronous convenience call, a scheduler concurrency knob, and a set of built-in pre- and post-processor stages. The design is discussed in Inference pipelines & scheduler and Pre- & post-processing.

Types & constants

Functions

Usage

Build a classify pipeline from built-in stages and a registered model, then submit a prioritized job:

/* resize -> quantize -> model -> softmax */
syn_resize_config_t resize_cfg = { .w = 96, .h = 96 };

syn_pipeline_t *pipe = syn_pipeline_create("classify");
if (pipe == NULL) {
    return -ENOMEM;
}

syn_pipeline_add_preprocess(pipe, syn_preprocess_image_resize, &resize_cfg);
syn_pipeline_add_preprocess(pipe, syn_preprocess_quantize_int8, NULL);
syn_pipeline_add_model(pipe, model_handle);   /* from syn_model_register() */
syn_pipeline_add_postprocess(pipe, syn_postprocess_softmax, NULL);

int ret = syn_pipeline_build(pipe);
if (ret != 0) {
    syn_pipeline_destroy(pipe);
    return ret;
}

/* Submit asynchronously with a 5 ms deadline, then wait */
syn_infer_params_t params = {
    .priority    = SYN_PRIORITY_REALTIME,
    .deadline_us = 5000,
    .preemptible = false,
    .callback    = NULL,
    .user_data   = NULL,
};

syn_job_id_t job = syn_infer_submit(pipe, input, &params);
if (job == SYN_JOB_INVALID) {
    return -EIO;
}

ret = syn_infer_wait(job, 100);
if (ret == 0) {
    syn_tensor_t output;
    syn_infer_get_result(job, &output);
}

Note that syn_preprocess_quantize_int8 and syn_postprocess_softmax take config structures from syn_process.h (softmax's is optional). Pipelines are drawn from a static pool of 4; stages must be added in canonical order — pre-processors, then exactly one model, then post-processors — and syn_pipeline_build() validates the chain and computes a worst-case memory estimate. The direct NPU HAL path — syn_hal_npu_set_input(), syn_hal_npu_invoke(), syn_hal_npu_get_output() — remains available, as shown in Hello Inference in 20 lines.

Notes

• Implementation status: implemented in Phase 2 (v0.2.0, released) — pipeline engine, priority scheduler (REALTIME > NORMAL > BEST_EFFORT, FIFO within a class, dedicated scheduler thread, per-job completion semaphores), and all built-in stages, verified on the FRDM-MCXN947. The deadline_us and preemptible parameters are recorded but not acted on yet; deadline-aware dispatch and layer-granular preemption come later (see the implementation notes in syn_infer.c).
• Tensors: inputs and outputs are syn_tensor_t descriptors from syn_mem.h; models are handles from syn_model.h.
• Ownership: a pipeline created with syn_pipeline_create() is owned by the caller and must be released with syn_pipeline_destroy(). Stage config pointers are stored, not copied — they must stay valid for the pipeline's lifetime.
• Callbacks: syn_infer_cb_t completion callbacks run in scheduler context; keep them short and defer heavy work to an application thread.
• Custom stages: any function matching syn_preprocess_fn_t / syn_postprocess_fn_t (return 0 on success, negative errno on failure) can be added alongside the built-ins.
• Results: job outputs are allocated from the ephemeral arena; they stay valid until syn_mem_reset_ephemeral() is called. Reset between jobs once results are consumed.
• Related: syn_process.h for the stage config structures, syn_hal_npu.h for the execution backend, syn_prof.h for per-stage timing, Inference pipelines & scheduler for the design.