Your codebase, your rules: Customizing Copilot with context engineering

Key Concepts

Context Engineering: The practice of optimizing and making context available for AI agents to use effectively.
Agentic Flow: A process where an AI agent performs tasks, makes decisions, and interacts with its environment.
Copilot: An AI pair programmer that understands code context and can assist with coding tasks.
Vector-based Semantic Search: A method for finding code that understands the meaning and intent behind queries, not just keywords.
Language Intelligence: The in-depth understanding of code structure and syntax provided by language servers and extensions in VS Code.
Tasks (VS Code): Background processes like builds and linting that agents can monitor and react to.
Symbol and Pattern Search: The ability to find specific code elements or recurring patterns within a codebase.
Problems (VS Code): Linting errors and other issues flagged by language intelligence, visible to agents.
Embeddings Model: A custom model used for code search, offering improved retrieval quality and faster throughput compared to previous GPT models.
Copilot Instructions: A mechanism to provide agents with a "mini-map" of a codebase, guiding them on where things are, how to run code, and which tools to use.
Domain-Specific Instructions: Micro-concepts shared across a codebase, used to guide agents on specific patterns or APIs.
Prompts: Reusable, one-shot commands designed to automate specific tasks or workflows.
Custom Agents: Rich personas and workflows that allow for multi-step, tool-constrained interactions.
Plan Mode: A feature in VS Code that allows users to visualize and manage agent plans, including tool access and handoffs.
Sub-agent: An isolated agent that performs specific discovery tasks and returns only the necessary data to the parent process, reducing context buildup.
Test-Driven Development (TDD) Flow: A workflow involving distinct modes for writing failing tests (TDD read), passing tests (TDD green), and refactoring.
Composable Primitives: Prompts and modes that can be combined to create different entry points and workflows.

Context Engineering in VS Code

This video introduces the concept of Context Engineering, focusing on how to effectively leverage AI agents, particularly within the VS Code environment. The core idea is to provide agents with the right context to enable longer, more complex tasks, rather than just offloading simple chores.

Understanding Agent Capabilities in VS Code

The presentation begins by outlining the out-of-the-box capabilities of AI agents in VS Code, primarily focusing on Copilot:

Codebase Understanding: Copilot understands the entire repository context.
Vector-based Semantic Search: It can find code using semantic search, allowing for broader queries beyond specific file or function names. This means it can find "authentication" even if it's named "login." This is a key principle for effective agentic flows.
Terminal Command Execution: Agents can run terminal commands and observe their final execution.
Error Detection in Agent Flows: Agents can identify errors within their own execution flows.
Task Awareness: VS Code agents are aware of background tasks like build or linting tasks. They can monitor these tasks and react to their outcomes. For example, if a code-watching task rebuilds and encounters errors, the Copilot Agent loop can be aware of this and check after each turn. This is actively used within the VS Code team.
Symbol and Pattern Search: This is powered by VS Code's language intelligence. The speaker emphasizes the importance of having proper extensions and language servers set up for every language used in a project. Poorly configured linting or missing language support can confuse both humans and agents.
Test Execution: VS Code has a built-in test runner exposed to agents. Agents can run entire test suites, partial tests, and access information about failed tests. Investing in proper test setup benefits both humans and agents.
Problem Reporting: Linting errors and other issues flagged by language intelligence are exposed to agents. This feedback loop is crucial; if an agent makes an edit that causes a linting error, the agent is immediately aware. This is a significant advantage over CLI-based interactions. The speaker recommends enabling accessibility linting as an example of expanding context beyond just language errors.

Code Search Updates and Embeddings

Recent updates to code search include a new embeddings model. This custom model, replacing previous GPT models, offers:

Better Retrieval Quality: Leading to higher acceptance rates and faster throughput.
Real-world Benefits: Faster and more reliable code discovery.
Agentic Flow Improvement: Agents require fewer turns to find the right code.
User Benefit: Allows for more ambiguous requests from users, with the agent still able to discover the correct code sections.

Context Engineering in the VS Code Repository

The speaker then dives into the VS Code GitHub repository (github.com/microsoft/vscode) to illustrate context engineering in practice.

No Hard and Fast Rules: Context engineering is described as a technique for optimizing and making context available, rather than a set of specific files to add.
Copilot Instructions: This is presented as the best starting point. It acts as a "mini-map" of the codebase, guiding the agent on:
- General location of components.
- How to run the code.
- How to find related code.
- Which tools work best.
- General rules for validation.
- Specific linting tools and how to run them.
- How to run tests, pointing to task output for validation after changes. These instructions are included by default in the VS Code repo, benefiting all contributors.
Domain-Specific Instructions: These are included by the agent based on descriptions and are pulled in as needed. They describe "micro-concepts" shared across the codebase, often addressing areas where the agent has made mistakes. An example given is the "observable pattern" specific to the VS Code codebase, guiding the agent to use observables correctly and avoid introducing new, incompatible APIs.
Prompts (Data Example): The speaker highlights a "data prompt" for democratizing access to VS Code telemetry data. This prompt allows users to query telemetry without needing to learn complex data tools like Kusto and KQL. It describes the flow for writing Kusto queries based on the data and the typical process. The prompt leverages an MCP (likely a model or service) and access to GitHub repos.

Plan Mode and Sub-agents

Plan Mode: Now built into VS Code, Plan Mode allows users to visualize and manage agent plans. Key UI elements include descriptions and argument hints. A crucial context engineering pattern here is restricting the LLM's access to tools and managing handoffs.
Run Sub-agent: This pattern is used within Plan Mode. By mentioning tools in the description, an isolated sub-agent performs discovery. This UI pattern signifies that the agent is running in its own loop, returning only the necessary data to the parent process. The speaker uses this for research, directing a sub-agent to explore deeply and return only what's needed to answer a question, enabling extremely long-running agentic conversations by reducing context buildup.

Prompts and Custom Instructions

Custom Instructions: These are described as "rules of engagement" that are always on, focusing on coding standards. The advice is to focus on how to write good code, what to avoid, and which tools to use, rather than role-playing (e.g., "TypeScript expert"). A general chat instruction button allows for rerunning prompts and generating instructions for the codebase, which should be reviewed and updated as a living document.
Prompts (Reusable Commands): These are one-shot, reusable commands for automating tasks. Examples include:
- A prompt to commit all changes and push.
- A prompt to commit and create a pull request. The recommendation is to start by putting personal prompts into reusable commands for tasks that are tedious or require research.
Custom Agents: These are more complex personas and workflows for multi-step processes, constrained to specific tools and systems.

Specific Workflows and Examples

Test-Driven Development (TDD) Flow: This workflow is composed of three modes:
- TDD Read: Writes failing tests.
- TDD Green: Writes passing tests by adding implementation.
- Refactor: Improves and polishes the implementation. These modes are connected with handoffs. The TDD prompt is the entry point, initiating the process by describing the feature. The agent identifies the lack of a TDD artifact, creates a TDD document (listing tests, edge cases, design notes), and then proceeds to pick the next test and prompt the user for implementation.
- Human-Agent Cooperation: The TDD flow highlights the value of human-agent cooperation, allowing for review of tests as a contract before moving to implementation. This contrasts with more orchestrated agents that proceed without human oversight.
Visual Flows (Plan Mode): The speaker reiterates the flexibility of Plan Mode, encouraging users to scope plans. An example is a "plan spike problem" focused on architectural proof-of-concept, specifically reasoning about backend or infrastructure. Another example is a plan to visualize UI from a markdown document.
UI Interaction: The ability to move into a "designer mode" for UI work, allowing the agent to experiment and implement live changes.

Conclusion and Next Steps

The presentation concludes with a suggested starting point for context engineering:

Custom Instructions: Bootstrap minimal instructions.
Fix the Agent: Automate chores.
Move on to Workflows: Build more complex processes.

The speaker directs viewers to the VS Code documentation for more information on context engineering, written by themselves, and encourages feedback through issues.

Key Arguments and Perspectives

Context is King: The central argument is that effective AI agent performance hinges on providing them with the right, optimized context.
VS Code as a Platform: VS Code's integrated features (language intelligence, tasks, test runner, problem reporting) are crucial enablers for context engineering.
Empowering Developers: Context engineering aims to empower developers by enabling AI agents to handle more complex tasks, leading to increased productivity and efficiency.
Iterative Improvement: Context engineering is an ongoing process, requiring continuous refinement of instructions, prompts, and workflows based on agent performance and user needs.
Human-AI Collaboration: The TDD example emphasizes the importance of human oversight and review in agentic workflows, fostering collaboration rather than full automation.
Democratizing Data Access: Prompts can be used to make complex data accessible to a wider audience, reducing the need for specialized knowledge.

Notable Quotes

"And that's not just for you to assign more of your daily chores to the AI, but to generally getting these things right means you can run longer, more complex tasks with the LM."
"It's more the technique of how do you optimize context and compress it enough and make it available enough for the agent to pull in as needed."
"So you can think about prompts and modes as composable primitives that give you different kind of entry points."
"You want to have that control." (Referring to human oversight in agent workflows).

Technical Terms and Concepts Explained

LM (Large Language Model): The underlying AI model that powers agents like Copilot.
Agentic Flow: A sequence of actions and decisions made by an AI agent.
Vector-based Semantic Search: A search technique that understands the meaning of words and phrases, not just exact matches.
Language Intelligence: The ability of VS Code to understand and process code in various programming languages, providing features like syntax highlighting, autocompletion, and error detection.
Embeddings Model: A type of machine learning model that represents data (like code) as numerical vectors, enabling semantic understanding and similarity comparisons.
Kusto and KQL: Kusto is a data exploration service, and KQL (Kusto Query Language) is its query language, used for analyzing large datasets.
MCP: Likely refers to a Model or Machine Learning Compute Platform, used for processing and running AI models.
Sub-agent: A smaller, specialized agent that performs a specific task within a larger agentic workflow.

Logical Connections Between Sections

The video progresses logically from introducing the fundamental concepts of context engineering and agent capabilities to demonstrating practical applications within the VS Code ecosystem.

The initial explanation of agent capabilities sets the stage for why context engineering is necessary.
The discussion of code search updates highlights advancements that directly improve agent performance.
The deep dive into the VS Code repository provides concrete examples of how context engineering is implemented through Copilot Instructions, Domain-Specific Instructions, and Prompts.
The introduction of Plan Mode and Sub-agents showcases advanced techniques for managing complex workflows and reducing context overhead.
The detailed explanation of Custom Instructions and Prompts offers actionable advice for users to start implementing context engineering.
The specific workflows like TDD and visual flows demonstrate how these concepts translate into practical, multi-step processes.
The conclusion ties everything together with a clear roadmap for getting started.

Data, Research Findings, or Statistics

The video mentions a new embeddings model for code search that offers "better retrieval quality" and "higher acceptance rates and faster throughput." While specific figures aren't provided, the qualitative benefits are emphasized.
Telemetry data in VS Code is mentioned as a source for learning and improvement, though no specific statistics are presented in this segment.

Synthesis/Conclusion

Context engineering is presented as a critical discipline for unlocking the full potential of AI agents in software development. By strategically providing agents with relevant information, guidance, and constraints, developers can enable them to perform more complex, nuanced tasks. VS Code, with its rich set of integrated tools and features, serves as a powerful platform for implementing these techniques. The key takeaways are to leverage VS Code's language intelligence, tasks, and testing capabilities, and to start building context through instructions, domain-specific guidance, and reusable prompts. The ultimate goal is to foster more effective human-AI collaboration, leading to increased productivity and better software development outcomes.