<p>Image preprocessing transforms are fundamental to embedded vision pipelines, yet existing approaches operate exclusively in either the spatial or frequency domain, failing to exploit their complementary strengths. This paper presents the Adaptive Frequency-Spatial Transform (AFST), a lightweight image transform that combines block-wise Discrete Cosine Transform (DCT) coefficients and local spatial statistics through a content-adaptive gating mechanism. The gating module, parameterized by only four tunable scalar weights (16&#xa0;bytes) optimized via gradient descent, or deployed as a compact <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(16 \times 16\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>16</mn> <mo>×</mo> <mn>16</mn> </mrow> </math></EquationSource> </InlineEquation> lookup table (1&#xa0;KB), dynamically balances frequency and spatial emphasis on a per-patch basis. AFST is designed explicitly for MCU-class edge devices, such as the ESP32 (240&#xa0;MHz, 520&#xa0;KB SRAM). Hardware measurement on an ESP32-WROOM-32 demonstrates that AFST processes a <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(48 \times 48\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>48</mn> <mo>×</mo> <mn>48</mn> </mrow> </math></EquationSource> </InlineEquation> image in 3.27&#xa0;ms (305&#xa0;FPS) using 6.04&#xa0;KB static memory (1.16% of SRAM), with near-zero timing jitter (&lt;0.2%), confirming deterministic real-time operation well above the 25&#xa0;FPS video threshold. On the GTSRB traffic sign benchmark, AFST with optimized multi-scale gating achieves 88.73% accuracy, competitive with HOG (90.38%) and CENSUS (89.61%) while using integer-only arithmetic. On the BTSD benchmark, AFST achieves 93.93%, within 0.6% of HOG (94.52%). Evaluation on CIFAR-10 confirms generalization beyond traffic sign recognition, with multi-scale AFST (54.65%) outperforming frequency-only (44.52%), spatial-only (49.09%), and static fusion (50.42%) baselines. At block size <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(B=4\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi>B</mi> <mo>=</mo> <mn>4</mn> </mrow> </math></EquationSource> </InlineEquation>, joint optimization reveals that the gating mechanism converges to completely suppress the high-frequency energy input (<InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(w_2 \rightarrow 0\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <msub> <mi>w</mi> <mn>2</mn> </msub> <mo stretchy="false">→</mo> <mn>0</mn> </mrow> </math></EquationSource> </InlineEquation>) across all three datasets, providing interpretable evidence that spatial features dominate at fine block granularity. The complete transform requires no floating-point coprocessor, involves zero convolutional filters, and is fully implementable in fixed-point arithmetic.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AFST: Adaptive Frequency-Spatial Transform for real-time edge vision on resource-constrained microcontrollers

  • Geevarghese Titus

摘要

Image preprocessing transforms are fundamental to embedded vision pipelines, yet existing approaches operate exclusively in either the spatial or frequency domain, failing to exploit their complementary strengths. This paper presents the Adaptive Frequency-Spatial Transform (AFST), a lightweight image transform that combines block-wise Discrete Cosine Transform (DCT) coefficients and local spatial statistics through a content-adaptive gating mechanism. The gating module, parameterized by only four tunable scalar weights (16 bytes) optimized via gradient descent, or deployed as a compact \(16 \times 16\) 16 × 16 lookup table (1 KB), dynamically balances frequency and spatial emphasis on a per-patch basis. AFST is designed explicitly for MCU-class edge devices, such as the ESP32 (240 MHz, 520 KB SRAM). Hardware measurement on an ESP32-WROOM-32 demonstrates that AFST processes a \(48 \times 48\) 48 × 48 image in 3.27 ms (305 FPS) using 6.04 KB static memory (1.16% of SRAM), with near-zero timing jitter (<0.2%), confirming deterministic real-time operation well above the 25 FPS video threshold. On the GTSRB traffic sign benchmark, AFST with optimized multi-scale gating achieves 88.73% accuracy, competitive with HOG (90.38%) and CENSUS (89.61%) while using integer-only arithmetic. On the BTSD benchmark, AFST achieves 93.93%, within 0.6% of HOG (94.52%). Evaluation on CIFAR-10 confirms generalization beyond traffic sign recognition, with multi-scale AFST (54.65%) outperforming frequency-only (44.52%), spatial-only (49.09%), and static fusion (50.42%) baselines. At block size \(B=4\) B = 4 , joint optimization reveals that the gating mechanism converges to completely suppress the high-frequency energy input ( \(w_2 \rightarrow 0\) w 2 0 ) across all three datasets, providing interpretable evidence that spatial features dominate at fine block granularity. The complete transform requires no floating-point coprocessor, involves zero convolutional filters, and is fully implementable in fixed-point arithmetic.