Rubber Duck Thursdays: Building Agents with Copilot

Key Concepts

• AI Agents: Autonomous or semi-autonomous software entities that use LLMs to perform tasks.
• Middleware: A filtering layer placed between the user prompt and the LLM to enforce safety, compliance, and guardrails.
• MCP (Model Context Protocol): A standard for connecting AI assistants to systems, data, and tools (e.g., a cupcake ordering server).
• Guardrails: Mechanisms to prevent AI from generating harmful, insecure, or non-compliant content.
• Context Engineering: Managing the amount of information sent to an LLM to prevent "context overload" and maintain performance.
• Rubber Ducking: A development practice where a secondary AI model reviews code changes to provide a critique or identify vulnerabilities.

---

1. Building AI Agents: Frameworks and Methodology

The speaker emphasizes that the industry is shifting from experimental AI agent building to production-ready deployments.

• Frameworks vs. Scratch: While building from scratch is possible, the speaker advocates for using established frameworks (like the Microsoft Agent Framework or LangChain) to speed up development and easily integrate pre-built tools like MCP servers.
• The Workshop Approach: The speaker highlights a collaborative workshop with the Claude team (Anthropic) and Microsoft, which demonstrates how to deploy models (e.g., Claude Sonnet 3.5) via the Azure Marketplace and integrate them into agentic workflows.
• Niche Focus: A key argument presented is that the more "niche" an agent’s purpose (e.g., a compliance app for pest control), the more effective and valuable it becomes.

2. Security and Compliance

A significant portion of the discussion addresses the "Zero Trust" security context in AI development.

• The Challenge: Developers are concerned about AI agents generating repetitive vulnerabilities (e.g., insecure file uploads).
• Middleware as a Solution: The speaker explains that middleware acts as a "pre-sandbox" filter. Before a prompt reaches the LLM, the middleware checks it against defined categories and strictness thresholds. If the prompt violates safety policies, it is blocked before the LLM ever processes it.
• Enterprise Safety: For enterprise applications, the speaker recommends using platforms like Azure, which provide built-in content safety and guardrails, rather than relying solely on custom-built security layers.

3. Model Context Protocol (MCP)

MCP is presented as a vital tool for providing agents with external data.

• Real-World Application: The speaker describes an MCP server designed to order cupcakes. The server provides the agent with specific tools and pre-defined prompts, allowing the agent to interact with the store's backend.
• Context Management: To avoid performance degradation, developers must manage context carefully. Techniques like "compacting" context are necessary because LLMs perform better when not overwhelmed by excessive data.

4. Key Arguments and Perspectives

• Trusting Automated Tools: Regarding "rubber ducking" (using a second model to review code), the speaker suggests trusting the automated triggers in tools like the Copilot CLI for large changes, as they are designed to recognize when a second model family is needed for a security review.
• Developer Well-being: The speaker candidly notes that building robust AI agents requires a clear, rested mind, emphasizing that technical success is tied to the developer's ability to manage their own workflow and environment.

5. Notable Quotes

• "I think the more niche your agent is going to be, the better it is going to be."
• "Middleware is like a filter... it's going to take in that prompt, it's going to filter any bad things... and that query is not even going to reach the LLM."
• "I think frameworks are there just to speed you up... I think it's easier just to plug in pre-built code."

6. Synthesis and Conclusion

The session highlights that the future of AI development lies in moving beyond simple experimentation toward secure, production-grade agents. The primary takeaways are:

1. Security is paramount: Use middleware and platform-level guardrails to ensure agents remain within safe operational boundaries.
2. Leverage existing protocols: Utilize frameworks and standards like MCP to connect agents to real-world tools efficiently.
3. Optimize for performance: Practice effective context engineering to ensure agents remain fast and accurate.
4. Use automation for quality: Rely on automated code review tools (like rubber ducking) to maintain security standards during the development lifecycle.