The Memory Wall on Edge Microcontrollers

Standard neural networks store weights and compute activations using single-precision floating point (32-bit float). On hardware with limited SRAM, storing thousands of 32-bit parameters causes memory overflow.

INT8 quantization converts 32-bit floating-point values into 8-bit signed integers. This delivers three practical advantages:

  • Reduces parameter storage (Flash) by up to 75%.
  • Cuts intermediate activation buffer requirements (SRAM) significantly.
  • Allows microcontrollers lacking hardware floating-point units (FPUs) to execute inference through single-cycle integer arithmetic.

Using MATLAB's Deep Learning Quantizer

MATLAB provides the dlquantizer tool to calibrate, validate, and convert neural networks without retraining from scratch.

Step 1: Set Up Calibration and Validation Data

Quantization requires a representative dataset to measure the dynamic range of activations across every layer. This prevents numerical saturation and clipping.

% Load floating point network
net = load('audio_keyword_net.mat').net;

% Prepare a small calibration datastore (e.g., 50-100 sample inputs)
calData = imageDatastore('data/calibration', 'IncludeSubfolders', true);
valData = imageDatastore('data/validation', 'IncludeSubfolders', true);

Step 2: Quantize with dlquantizer

Run the quantization workflow via the command-line API:

% Create the quantizer object
dq = dlquantizer(net, 'ExecutionEnvironment', 'arm-cortex');

% Run calibration to determine scaling factors and dynamic ranges
calResults = calibrate(dq, calData);

% Evaluate validation accuracy before finalizing
valResults = validate(dq, valData);
disp(valResults.MetricComparison);

Step 3: Save the Quantized Network

Inspect the accuracy difference between the original floating-point model and the INT8 quantized model. In most feedforward networks, the accuracy drop is below 1%.

% Export the INT8 quantized model
quantizedNet = quantize(dq);
save('quantized_keyword_net.mat', 'quantizedNet');

Step 4: Generate INT8 C Code

Once quantized, feed the new network directly into Embedded Coder. The generated C code uses int8_t arrays and CMSIS-NN integer SIMD instructions, drastically reducing execution time per inference cycle.

Experiencing accuracy loss after quantization? Our engineers at MATLABSolutions specialize in mixed-precision calibration and post-quantization recovery for complex signal and control models.