LLM¶
LLMs are used at multiple different stages of your pipeline:
During Indexing you may use an LLM to:
determine the relevance of data (whether to index it at all)
or
summarize the raw data and index the summaries instead.
During Querying LLMs can be used in two ways:
During Retrieval (fetching data from your index)
LLMs can be given an array of options (such as multiple different indices)
and make decisions about where best to find the information you're looking for.
An agentic LLM can also use tools at this stage to query different data sources.
During Response Synthesis (turning the retrieved data into an answer)
an LLM can combine answers to multiple sub-queries into a single coherent answer,
or it can transform data, such as from unstructured text to JSON or another programmatic output format.