Docs / Concepts / Pre- & post-processing

Pre- & post-processingv0.2.0

Real inference rarely starts with a model-ready tensor. Sensor data has to be resized, normalized and quantized on the way in, and raw model output has to be turned into class labels or bounding boxes on the way out. SynapticOS treats these transformations as first-class pipeline stages with a stable function signature, a set of built-in processors declared in syn_infer.h, and a DSP HAL underneath for hardware acceleration.

Processing stages in the pipeline

A pipeline is an ordered chain: zero or more pre-processing stages, one model, zero or more post-processing stages. Every stage — built-in or your own — implements the same signature: it reads one tensor, writes another, and takes an optional per-stage configuration pointer.

/* syn_infer.h — pre/post-processing function signatures */
typedef int (*syn_preprocess_fn_t)(const syn_tensor_t *in,
                                   syn_tensor_t *out, const void *config);
typedef int (*syn_postprocess_fn_t)(const syn_tensor_t *in,
                                    syn_tensor_t *out, const void *config);

Stages are attached with syn_pipeline_add_preprocess() and syn_pipeline_add_postprocess() during pipeline construction. A typical image-classification pipeline chains resize, INT8 quantization, the model, then softmax:

syn_pipeline_add_preprocess(pipe, syn_preprocess_image_resize,
                            &(syn_resize_config_t){.w=96, .h=96});
syn_pipeline_add_preprocess(pipe, syn_preprocess_quantize_int8, NULL);
syn_pipeline_add_model(pipe, model);
syn_pipeline_add_postprocess(pipe, syn_postprocess_softmax, NULL);

See Inference pipelines & scheduler for how the built pipeline is submitted and executed.

Built-in pre-processors

Four built-in pre-processors are declared in syn_infer.h and implemented (as of Phase 2) across src/preprocess/:

The resize configuration struct lives in syn_infer.h; every other config type is defined in the Phase 2 header syn_process.h:

typedef struct { uint16_t w; uint16_t h; } syn_resize_config_t;

Built-in post-processors

Five built-in post-processors, all implemented in Phase 2, turn raw model output into application-level results:

The result-record layouts (syn_classification_t, syn_bbox_t) and all the config structures are documented field-by-field in the syn_process.h reference.

DSP acceleration via the DSP HAL

Pre- and post-processing is exactly the workload the MCXN947's PowerQuad co-processor exists for. The built-in stages are designed to call through syn_hal_dsp.h rather than doing the math on the CPU:

/* syn_hal_dsp.h — DSP HAL (PowerQuad) */
int syn_hal_dsp_normalize_int8(const uint8_t *in, int8_t *out,
                               size_t len, float scale, int32_t zero_point);
int syn_hal_dsp_softmax_f32(const float *in, float *out, size_t len);
int syn_hal_dsp_argmax(const int8_t *data, size_t len, uint32_t *index);
int syn_hal_dsp_fft_f32(const float *in, float *out, size_t len);
int syn_hal_dsp_mat_mult_q15(const int16_t *a, const int16_t *b,
                             int16_t *out, uint16_t rows, uint16_t cols);

The mapping is direct: softmax uses syn_hal_dsp_softmax_f32(), argmax uses syn_hal_dsp_argmax(), and the MFCC front-end builds on syn_hal_dsp_fft_f32(). Because everything goes through the HAL, the same pipeline code runs on QEMU (software fallback) and on the FRDM board.

Implementation status (Phase 2)

Honest status: all five processing source files, placeholders in v0.1.0, are fully implemented as of v0.2.0: