TopoCompress: Topology Aware Token Compression Algorithm for Distributed Edge MoE Inference

Excerpt

arXiv:2609.26061v1 Announce Type: cross Abstract: Mixture-of-experts (MoE) models improve capacity with moderate overhead by sparsely activating experts per token. However, deploying MoE across resource-constrained edge servers incurs substantial cross-server communication as experts are distributed across heterogeneous servers. Existing placement methods optimize for raw token traffic, while conventional compression considers semantics but ignores topology-dependent routing costs. Consequently,