Scholarlink

<p>Robotic perception often fails at the moment it matters most during contact because frame-clocked models waste compute when nothing changes and drift under sensor latency. We introduce the event-clocked tactile transformer (ECTT), a unique perception algorithm that advances computation only on physically meaningful tactile micro-events (first contact, incipient slip, stick–slip, release). Each event opens a bounded window for extracting compact visual and acoustic cues and updates a causal, event horizon transformer with a fixed FLOP budget, yielding deterministic P95/P99 latency on edge hardware. ECTT’s event-based clock eliminates idle time drift, provides a natural causal ordering for contact reasoning, and enables contract-calibrated confidence outputs for safety controllers. We offer a complete implementation path for micro-event detection at 1&#xa0;kHz, PPS-synchronized time stamps, event-conditioned projections for camera and microphone inputs, and a fail-silent diagnostic channel, and evaluate on contact-rich tasks (bin picking of near-identical parts, tool use on composite panels, belt drive anomaly triage, and human-aware navigation with incidental contact). Under illumination changes, acoustic clutter, and induced tactile drift, ECTT reduces contact pose error variance and contract violations per hour relative to frame-clocked baselines at equal compute, while maintaining &lt; 25&#xa0;ms P99 end-to-end latency on Jetson-class devices. The results demonstrate that event-clocked perception is a viable path to reliable, auditable contact reasoning in production robotics.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Application of multimodal learning in robotic perception: an intelligent perception framework integrating vision, sound, and touch

  • 期刊论文

  • 发表日期：

    2026年2月6日

查看全文
  • Jiajun He

摘要

Robotic perception often fails at the moment it matters most during contact because frame-clocked models waste compute when nothing changes and drift under sensor latency. We introduce the event-clocked tactile transformer (ECTT), a unique perception algorithm that advances computation only on physically meaningful tactile micro-events (first contact, incipient slip, stick–slip, release). Each event opens a bounded window for extracting compact visual and acoustic cues and updates a causal, event horizon transformer with a fixed FLOP budget, yielding deterministic P95/P99 latency on edge hardware. ECTT’s event-based clock eliminates idle time drift, provides a natural causal ordering for contact reasoning, and enables contract-calibrated confidence outputs for safety controllers. We offer a complete implementation path for micro-event detection at 1 kHz, PPS-synchronized time stamps, event-conditioned projections for camera and microphone inputs, and a fail-silent diagnostic channel, and evaluate on contact-rich tasks (bin picking of near-identical parts, tool use on composite panels, belt drive anomaly triage, and human-aware navigation with incidental contact). Under illumination changes, acoustic clutter, and induced tactile drift, ECTT reduces contact pose error variance and contract violations per hour relative to frame-clocked baselines at equal compute, while maintaining < 25 ms P99 end-to-end latency on Jetson-class devices. The results demonstrate that event-clocked perception is a viable path to reliable, auditable contact reasoning in production robotics.