Overview
Vision–Language–Action (VLA) models exhibit strong semantic understanding and policy generalization, yet their predominantly vision-centric perception struggles to characterize key contact states such as contact establishment, in-hand slip, and normal and shear contact interactions, which limits their performance in contact-rich manipulation. In this paper, we propose DT-TacVLA (Dual-Timescale Tactile-Enhanced VLA), a dual-timescale tactile fusion framework built on top of SmolVLA that processes tactile inputs along two complementary slow and fast pathways.
The slow semantic pathway operates at a lower update rate and adaptively fuses tactile image sequences with VLM features through a semantic-guided gating mechanism, producing a task–contact context that can be reused across multiple fast reflex updates. The fast reflex pathway is updated multiple times within each slow-context window. Under the fixed slow-context conditioning, it encodes the current tactile marker displacements into compact tokens and injects them into the action expert through a Tactile-Marker Adapter, modulating the action chunk with real-time contact feedback.
Across three contact-rich real-world manipulation tasks, including power plug insertion, USB connector insertion, and test tube placement, DT-TacVLA consistently achieves higher success rates than vision-only VLA baselines and its ablation variants.
Given expert demonstration trajectories, DT-TacVLA learns a Vision-Tactile-Language-Action (VTLA) policy that maps visual observation, language instruction, robot proprioceptive state, a recent tactile image sequence, and current tactile marker displacements to a continuous action chunk.
(a) Slow semantic pathway: The VLM processes vision and language inputs at a low rate and outputs multi-layer hidden states aligned with the action expert. TacVideoMAE extracts temporal features from the recent tactile image history, and the semantic-guided gating mechanism fuses the layer-wise semantic summary, proprioceptive encoding, and tactile sequence features to produce slow-context conditions. These conditions are written into a reusable K/V cache and remain fixed across several subsequent fast reflex updates.
(b) Fast reflex pathway: The action expert reuses the cached slow-context conditions. Meanwhile, the marker extractor tracks the tactile marker array in the most recent frame, encodes the marker displacements into compact tactile tokens, and injects them into the action tokens as an additive residual through the Tactile-Marker Adapter.
(c) Fast–slow coupling: The slow pathway determines the task–contact semantics that the action expert should attend to, while the fast pathway applies instantaneous tactile corrections within this context. During inference, the slow K/V cache remains unchanged inside a slow window; only the tactile marker tokens are updated with the current tactile frame.
To evaluate the effectiveness of DT-TacVLA in contact-rich manipulation, we conduct experiments on three real-robot tasks:
Each task contains two phases—free-space motion and contact interaction—making it possible to test the policy's ability to switch between the vision-dominated non-contact phase and the touch-dominated contact phase and to fuse multimodal information.
Hardware platform: The experimental platform consists of a UR5 robotic arm and an Inspire EG2-4C2 gripper, with one Vitai GF255 vision-based tactile sensor mounted on each fingertip. Visual observations are provided by two RGB-D cameras: an Intel RealSense L515 for the global scene view and an Intel RealSense D435i mounted on the wrist for a close-range end-effector view.
Experimental protocol: We collect 100 expert demonstration trajectories per task and evaluate each method over 20 real-robot trials per task.
Note: The video is published on a newly created channel, set to unlisted, and does not contain any author information.
DT-TacVLA Avg. Success
Average success rate across 3 tasks
Power Plug Insertion
Perfect success rate
Success rates are measured with 20 real-robot trials per task per model.
| Method | Power Plug | USB Connector | Test Tube | Average |
|---|---|---|---|---|
| Diffusion Policy | 35% | 30% | 45% | 36.7% |
| π0.5 | 65% | 55% | 75% | 65.0% |
| SmolVLA | 50% | 50% | 65% | 55.0% |
| TacVLA* | 85% | 65% | 85% | 78.3% |
| DT-TacVLA (Ours) | 100% | 80% | 95% | 91.7% |
This paper proposes DT-TacVLA, a dual-timescale tactile-enhanced Vision–Language–Action framework tailored to contact-rich manipulation. Compared with VLA methods that are primarily vision-driven or introduce tactile sensing at a single rate, DT-TacVLA models the task–contact context through the slow semantic pathway and applies instantaneous modulation to the action chunk via the fast reflex pathway using marker displacements, thereby capturing fine contact interactions more effectively.
Real-robot experiments show that DT-TacVLA outperforms vision-only VLA baselines on power plug insertion, USB connector insertion, and test tube placement, validating the effectiveness of dual-timescale tactile fusion for high-precision contact manipulation.
Although DT-TacVLA demonstrates effective contact adaptation, it is trained on offline demonstrations, which provide limited coverage of recovery behaviors and long-tailed contact failures. Future work will investigate interactive policy refinement, including reinforcement learning, to improve fine contact adjustment and error recovery, as well as robustness under complex contact perturbations, bimanual coordination, and deformable-object manipulation.