Do Motion Tokenizers for Co-Speech Gesture Generation Encode Gesture Semantics?

Excerpt

arXiv:2610.03765v1 Announce Type: cross Abstract: Discrete motion tokenizers encode motion as atomic units and are widely used for co-speech gesture generation. It remains unclear which motion properties, especially those relevant to gesture semantics, are recoverable from these codebooks. We probe a reconstruction-trained codebook using 19 co-speech gesture descriptors spanning from raw motion to abstract communicative function. Results show that geometry and handedness are readily decodable fr