In this talk Carly talked about how the context of an AI Agent works and what to pay attention to. She has also made a Blog post on the same topic you can find here: What is context engineering? Components, techniques, and best practices - Elasticsearch Labs
- A context window should only have the information it needs to work on the task that it was given.
- It will hallucinate if the required information is not part of the context.
- A context contains several parts
Context Engineering
- The prompt itself
- Instruction on how the output should look like
- RAG: Retrieval Augmentation Generation, meaning sending additional documents to the context window.
- Additional information on RAG Retrieval Augmented Generation: Refine LLM Responses with RAG - Elasticsearch Labs
- Short and Long term memor
- A short term memory is for example the whole current chat window that you have open
- Long term memory on the other hand is the whole history from the past, that is stored in a vector database or similar
- Available Tools, essentially a set of functions for the AI to use
- Depending on the given task, the LLM will decide itself when to use a tool or not. A tool could be the integration of getting flights information.
- Structured Output, we describe to the AI the schema of what we expect