Deform360: A Massive Multi-view Visuotactile Dataset for Deformable World Models
198 daily-life deformable objects. 1,980 interactions. 41 surround-view cameras and bimanual tactile grippers — a foundation for benchmarking 2D and 3D world models on real-world deformable dynamics.
Watch it move. Could you predict the next second?
Deformable motion is hard to predict: state is high-dimensional, and the contacts that drive motion are often occluded. Watch the cloth, then explore the data and models behind its next second.
Why deformable dynamics challenge current world models
Rope, cloth, and plush have high-dimensional state, while grippers and folds hide the contacts that drive motion. Deform360 enables controlled 2D-vs.-3D comparison with synchronized views, dense geometry, and touch.
Internet-scale pre-training captures rich appearance, yet long-horizon rollouts suffer 3D and temporal inconsistency.
Explicit geometry and structural priors support data-efficient prediction, but current learned 3D models lack comparable massive pre-training.
No data could settle it — so we built Deform360
Deform360 pairs 215.7 cumulative multi-view hours from 1,980 interactions across 198 objects with 41 synchronized views, bimanual touch, and dense markerless 3D particles.
A significant increase in scale and sensory richness
Object taxonomy — graded by material response
Varying stiffness and thickness.
Diverse textiles, airbags and thin shells.
Objects that exhibit large shape change.
Explore the captured objects directly.
Fully interactive 3D reconstructions
Click any object to load it live — orbit, zoom and pan in real 3D.
per-frame 3DGS · full set released with the dataset
From raw video & touch to dense particle motion
Per-frame 3DGS recovers geometry; multi-view tracks are lifted into 3D; touch constrains motion through occlusion.
Multi-view video + tactile → per-frame 3DGS → markerless 2D tracking → 3D lifting → physics-informed optimization.
{{ p.body }}
Tactile signals through occlusion
Synchronized tactile sensors measure normal-pressure contact cues that help constrain particle motion where cameras are occluded. Tangential micro-slip remains unobserved.
Explore the reconstructions in 4D
Choose a sequence, then orbit, scrub, and inspect synchronized 3DGS and particle dynamics.
Browser preview: 3 views; centroids and motion trails are subsampled for speed. {{ seqReady }} · full set of 1,980 in the release.
Real-world planning
Deform360-trained PhysTwin plans goal-conditioned cloth and rope manipulation on an unseen xArm setup in a second lab, without fine-tuning.
Predicted futures, side by side
Toggle unseen episodes and unseen objects, then compare ground truth with three model rollouts.
Qualitative future rollouts for the {{ qualName }} under the {{ qualSettingCaption }} setting.
{{ qualSettingFindingTitle }}. {{ qualSettingFinding }} Full metrics: paper {{ qualSettingPaperTable }} ↗
Fully open — plug and play
The dataset and modular capture-to-model pipeline are open-source. Swap components, rerun the benchmark, and contribute improvements.
BibTeX
{{ bibtex }}