Supervising Sound Localization by In-the-wild Egomotion

Excerpt

arXiv:2610.01388v1 Announce Type: cross Abstract: We present a method for learning binaural sound localization using egomotion as a supervisory signal. Over the course of a video, the cameras direction to a sound source will change as the camera moves. We train an audio model to predict sound directions that are consistent with visual estimates of camera motion, which we obtain using traditional methods from multi-view geometry. This provides a weak but plentiful form of supervision that we comb