Tactile Diffusion-Forcing world model · tac_s5_hf (final)

Where the tactile world model works, and where it fails

The same model evaluated across eight conditions — train vs held-out, short (1.3 s) vs long (16 s) horizon, and high vs low contact — with three examples each. Every clip is ground truth on the left, model prediction on the right. The point of the grid is that a single headline number hides three very different regimes.

The short answer. Short-horizon prediction genuinely works and generalizes. Long-horizon rollout on held-out data is worse than doing nothing — a baseline that simply freezes the last context frame beats it. And in every high-contact condition the model erases fine texture that was already present in its own context.

The comparison baseline

Every cell is scored against frozen context: take the last ground-truth context frame and hold it for the whole prediction horizon. It is the "predict no change" null model. A world model that cannot beat it is not contributing anything on that condition.

PSNR and texture are measured on predicted frames only (decoded frames 5 onward; frames 0–4 come from the ground-truth context latents). Texture = mean Sobel gradient energy of the prediction divided by that of the ground truth, so 1.0 means the prediction is as detailed as reality and lower means blur.

All eight conditions

conditioncontactmodel PSNR frozen PSNRΔ model texturefrozen texturebeats frozen
Read the contact column before comparing splits. The held-out episodes never reach the contact levels the training set does — "high contact" means ~0.22–0.38 on test but up to 0.68 on train. Only one episode in the whole 36-episode corpus (0511_episode_017, contact 0.435 vs a 0.160 median) is genuinely high-contact, and it is in the training split. Train-vs-test rows in the high-contact conditions are therefore not matched comparisons.

What the three regimes mean

Short horizon — real, generalizing skill. The model beats frozen context on essentially every short window, on held-out data as much as on training data. Given 8 frames of context it predicts the next 8 frames of gel deformation better than assuming nothing moves. This is the part that works.
Long horizon on held-out data — net harmful. Frozen context wins on every held-out 16 s segment. Notably the deficit is not accumulating drift: the model loses about 10 dB at the very first predicted frame and is then flat, decaying only 1 dB across 16 s while frozen context decays 7.7 dB as the scene genuinely changes. A constant offset that does not compound points at a systematic first-step mismatch rather than error accumulation.
High contact — texture is destroyed, not merely missed. On contacting sensors the model's texture energy falls below frozen context. Freezing a stale frame preserves more fine structure than the model predicts. Ridges and pin-dot patterns that are clearly visible in the model's own context frames are gone within about five predicted frames. Separately, contact onset — a new object entering the gel — is never predicted, which is likely not learnable from proprioception alone and is the case for conditioning on vision.

Caveats