Intent-Driven Dynamic Chunking: Segmenting Documents to Reflect Predicted Information Needs

Excerpt

arXiv:2602.14784v1 Announce Type: cross Abstract: Breaking long documents into smaller segments is a fundamental challenge in information retrieval. Whether for search engines, question-answering systems, or retrieval-augmented generation (RAG), effective segmentation determines how well systems can locate and return relevant information. However, traditional methods, such as fixed-length or coherence-based segmentation, ignore user intent, leading to chunks that split answers or contain irrelev