Anthropic’s Contextual Retrieval

One of the problems I have encountered in trying to do Retrieval Augmented Generation (RAG) where a complete single document was not uploaded for queries on that document alone is that chunks created from documents for embedding often lose context. I have often considered what would be the outcome if one were to put some limited but useful context along with each chunk. Now I know. Introducing Anthropic's contextual retrieval concept.

One of my concerns with placing some additional context for each chunk is that you’d probably need to pass the entire document being chunked as context along with each individual chunk. That would be very expensive and slow. Now, with Anthropic’s Claude prompt caching, the cost and latency is reduced significantly and it seems actually doable for chunk contexting (is contexting a word?). An initial prompt including the full document could be done with that prompt cached for future prompt reference.

I plan to try this out.

Check out the Anthropic “Introducing Contextual Retrieval” post for greater details.