FACT: Examining the Effectiveness of Iterative Context Rewriting for Multi-fact Retrieval¶
组织: DeepWisdom, Université de Montréal & Mila, Google
大型语言模型（LLMs）擅长从扩展上下文中检索单个事实，但它们在需要同时检索多个事实的任务中遇到困难，尤其是在生成过程中。本文提出了一种新颖的“迷失在中间(lost-in-the-middle)”现象(phenomenon)，即 LLMs 在整个生成过程中逐渐失去关键信息的踪迹，导致检索不完整或不准确。为了应对这一挑战，我们引入了(FACT, Find All Crucial Texts)，这是一种迭代检索方法，可通过连续几轮重写来细化上下文。这种方法使模型能够增量地捕获基本事实，而这些事实在单遍检索中经常被忽视(overlook)。实验表明，FACT 显着增强了跨各种任务的多事实检索性能，尽管在通用 QA 场景中改进不太明显。我们的研究结果揭示(shed light)了 LLMs 在多事实检索中的局限性，并强调了对更具弹性的长上下文检索策略的需求。
Large Language Models (LLMs) are proficient at retrieving single facts from extended contexts, yet they struggle with tasks requiring the simultaneous retrieval of multiple facts, especially during generation. This paper identifies a novel “lost-in-the-middle” phenomenon, where LLMs progressively lose track of critical information throughout the generation process, resulting in incomplete or inaccurate retrieval. To address this challenge, we introduce Find All Crucial Texts (FACT), an iterative retrieval method that refines context through successive rounds of rewriting. This approach enables models to capture essential facts incrementally, which are often overlooked in single-pass retrieval. Experiments demonstrate that FACT substantially enhances multi-fact retrieval performance across various tasks, though improvements are less notable in general-purpose QA scenarios. Our findings shed light on the limitations of LLMs in multi-fact retrieval and underscore the need for more resilient long-context retrieval strategies.
备注
快速简介：提高LLM在多事实任务检索中的性能，把一次检索忽视的事实重新重视起来。
Introduce¶
引入了（FACT, Find All Crucial Texts），这是一种专为多事实检索量身定制的迭代方法。
单遍检索通常无法捕获多个事实，因为模型的注意力往往主要集中在排名靠前的事实上。
迭代过程通过逐步从上下文中删除已识别的事实来解决此限制，使模型能够在后续轮次中专注于其他关键事实。