Build apps & agents that scale with VS Code, GitHub Copilot, and Agent Framework

Key Concepts

• AI Pair Programmer: A tool that assists developers by suggesting code completions and generating code.
• Agent Mode: A mode of operation for AI tools where they can perform tasks autonomously based on instructions.
• AI Native Code Editor: A code editor designed to deeply integrate AI capabilities into the development workflow.
• Spec Kit: An open-source project for spec-driven development, enabling developers to define desired outcomes for applications and features.
• Constitution: A file within Spec Kit that defines governing principles and best practices for app development.
• MCP (Model-driven Code Production): A framework or toolset that leverages AI models for code generation and production.
• Azure MCP: A specific implementation of MCP for Azure services, enabling secure connection between Azure services and AI tools.
• Agent Framework: An open-source toolkit for building, orchestrating, and managing AI agents.
• AI Foundry: A service for deploying and managing AI models.
• Model Catalog: A repository of available AI models.
• Model Playground: An environment for testing and experimenting with AI models.
• Tracing: A mechanism for observing and debugging the execution flow of AI agents.
• Evaluations: A process for assessing the performance and quality of AI agent responses, akin to unit tests for stochastic processes.
• SRE (Site Reliability Engineering) Agent: An AI agent designed to automate incident management and resource optimization for applications.
• Spec-Driven Development: A development methodology where specifications (specs) define, generate, verify, and operate systems, serving as a single source of truth for automation.
• Inner Loop: The rapid cycle of coding, testing, and debugging that developers perform locally.
• Platform Engineering: The practice of designing and building the toolchains and workflows that enable product development teams to deliver software more efficiently.

Building an Intelligent App from Scratch and Deploying to Production

This presentation outlines a modern approach to application development, emphasizing cloud-native, agentic, and self-improving applications managed through specifications. The core idea is to shift from traditional coding to spec-driven development, where AI agents execute tasks based on defined principles and requirements.

GitHub Copilot and the Evolution of AI in Development

• Copilot's Genesis: Launched in 2021 as the world's first AI pair programmer.
• Advancements: Introduction of Model Choice and the concept of "agents" in the past year.
• Impact: Agent mode with GitHub Copilot Coding agent is significantly increasing engagement and productivity.
• Adoption: 80% of new developers on GitHub use Copilot within their first week, highlighting its essential role in the developer experience.
• Vision: To make Visual Studio Code an "AI native code editor."
• Recent Developments: Open-sourcing of the Copilot chat extension and the introduction of Agent Session views, custom local/remote agents, Codex agents, and planning mode.
• Essential for Cloud Native: These tools collectively make GitHub Copilot crucial for building cloud-native applications.

Spec-Driven Development with Spec Kit

The presentation introduces "spec kit driven development" as an alternative to "vibe coding." Spec Kit is an open-source project that facilitates defining application or feature requirements.

• Core Functionality:

  • Defining Requirements: Developers define what they want to build.
  • Generating User Stories: The specify command, when run with a prompt, generates user stories based on the defined requirements.
  • Planning: The plan command generates a plan based on the prompt and specified tools or principles.
  • Task Generation: The task mode breaks down the plan and research into individual, actionable steps.

• Constitution File:

  • Purpose: Acts as governing principles and guiding guidelines for app development.
  • Functionality: Similar to best practices but shareable with teams. Serves as a template that sets principles without requiring extensive prompt engineering.
  • Example: Specifying the use of Azure CLI and Azure MCP for codegen best practices, or the Azure AI toolkit and MCP tools for agent-driven features.
  • Flexibility: Not specific to Azure; can be adapted for other clouds or custom tools.
  • Invocation: Tools and principles defined in the Constitution are invoked when running prompts with Spec Kit.

• Example: Airbnb-style App for Octets

  • Prompt: "Give me an Airbnb style app for our pet."
  • User Story Generation: Spec Kit expands the single prompt into multiple user stories.
  • Planning Output:
    • Generates a plan summary including feature overview and technical content.
    • Identifies key decisions needed.
    • Generates necessary files for the plan and research.
  • Research File Insights:
    • Invokes MCP tools (e.g., Azure MCP tools).
    • Provides model recommendations (e.g., GPT-4 for many use cases).
    • Suggests deployment to Azure AI Foundry for production.
    • Offers alternative considerations.
  • Task Generation:
    • Breaks down the plan into individual steps.
    • Generates tasks for frontend and other components.
    • Automation with GitHub MCP: The GitHub MCP is used to create GitHub Issues for each task, streamlining the workflow.
    • Agent Assignment: Tasks can be assigned to coding agents for execution (e.g., "create back end").

Building AI Agents with the AI Toolkit

The presentation shifts to building AI agents, emphasizing that they operate on different principles than traditional applications.

• Agent Characteristics:

  • Probabilistic Models: Work on probabilistic models, not static control flow.
  • Reasoning and Adaptation: Agents reason, plan, and adapt dynamically.
  • Intent-Focused: Developers focus on the intent, building signal loops and guardrails.
  • Contrast with Traditional Apps: Traditional apps have fixed logic, manual updates, and rigid control flow.

• AI Toolkit for VS Code:

  • Purpose: Facilitates generative AI development by embedding models and workflows into the developer's inner loop.
  • Capabilities:
    • Discover and explore local or remote models.
    • Author and evaluate agents and multi-agent workflows within the IDE.
    • Seamless integration into AI applications.
    • Deployment to Azure AI Foundry.
    • Unified VS Code and GitHub Copilot experience.

• Building a Single Agent (Pet Sitter Suggestion):

  • Model Selection:
    • Utilize the AI Toolkit's Model Catalog to explore and deploy models to AI Foundry or download local models.
    • Test models in the Model Playground.
    • Copilot Assistance: Ask Copilot for model recommendations for a specific agent (e.g., "which foundry models do you recommend for an agent that can suggest pet sitters in the area?").
    • Copilot analyzes agent requirements and provides detailed model information (cost, context window, use cases).
    • Example Recommendation: GPT-4o is recommended.
  • Agent Creation:
    • Prompt Copilot to create an agent (e.g., "create an agent to suggest pet sitters").
    • Microsoft Agent Framework: Utilized for writing single agents or orchestrating multi-agents.
    • Agent Definition: Includes system instructions and a list of tools.
    • Tracing: Copilot can enable tracing for local runs to observe agent performance.
  • Agent Execution and Tracing:
    • Run the agent locally.
    • Query the agent (e.g., "look for a dog sitter in San Francisco area").
    • Observe model calls and tool calls in the trace collector.
    • Example Trace: Shows a model call to GPT-4o and a tool call.

• Orchestrating Multi-Agent Workflows:

  • Scenario: Creating a second agent for recommending pet venues and orchestrating both agents.
  • Copilot Assistance: Prompt Copilot to orchestrate multiple agents into a workflow.
  • Workflow Builder: The Microsoft Agent Framework's workflow builder is used to orchestrate multiple agents.
  • Testing: User queries are tested, and agents (sometimes both) are invoked.

• Agent Evaluation:

  • Purpose: To assess the quality of agent responses, similar to unit tests for stochastic processes.
  • Copilot Assistance: Prompt Copilot to "add evaluation to my agent."
  • Automated Guidance: Copilot suggests metrics and generates test data (e.g., user queries) using Data Wrangler.
  • Execution: Evaluations can be run offline locally or integrated with GitHub Actions for CI/CD pipelines.
  • Azure AI SDK: Powers the evaluation scripts.

• Real-World Application Example:

  • Scenario: Asking for a cafe recommendation in San Francisco with a budget limit.
  • Agent Workflow: A team of agents is invoked, calling model endpoints.
  • Response: The agent provides pet-friendly cafe recommendations and notes the unavailability of sitters within the budget.
  • Budget Adjustment: The budget is increased, and the agent finds a sitter.

Azure SRE Agent for Incident Management and Optimization

The presentation introduces the Azure SRE Agent, designed to automate incident management and resource optimization.

• Problem Addressed: Traditional incident resolution is slow, labor-intensive, and leads to frequent on-call alerts.

• SRE Agent Capabilities:

  • 24/7 Availability: Acts as a knowledgeable IT person available around the clock.
  • Data Analysis: Instantly analyzes thousands of system metrics and logs.
  • Automated Remediation: Automatically fixes common problems, sometimes without human intervention.
  • Reduced Toil: Minimizes repetitive tasks for SREs.
  • Proactive Diagnosis: Identifies and mitigates issues before they impact users.

• Implementation and Usage:

  • Codified Guidance: The agent is built by codifying the guidance for Azure SREs.
  • Chat Interface: Works like a Copilot Agent, allowing users to chat and set up monitoring.
  • Integration: Connects with incident management platforms like PagerDuty, Azure Monitor, and ServiceNow.
  • Response Plans: Configurable response plans for different incident types.
  • Execution Plan: Based on runbooks, outlining steps for investigation and resolution.
  • Resource Knowledge: The agent understands deployed resources, Azure resource groups, and code locations.

• Demonstration:

  • Incident Trigger: The agent detects issues like 500 errors.
  • Investigation: Checks CPU, memory, scaling, and network configurations.
  • Automated Resolution Attempts: Attempts automated resolutions, such as increasing CPU.
  • GitHub Issue Creation: Logs an issue in GitHub for manual review if automated resolution fails or is not possible.
  • Detailed Findings: Provides detailed error information, proposed code fixes, and Infrastructure as Code (IaC) drift.
  • Assignment to Copilot: The generated issue can be assigned to Copilot for further action.
  • Pull Request Generation: Copilot can create pull requests to address identified issues.
  • Example Fix: Adjusting min/max replicas for scaling and fixing memory allocation problems.
  • Session View: Shows Copilot's actions, including configuration changes, testing with Playwright, and back-channeling reports via Azure MCP Server and GitHub MCP Server.

Synthesis and Conclusion

The presentation highlights a significant shift in software development driven by AI agents.

• Productivity Boost: In 30 minutes, natural language specs and MCP were used to enhance productivity.
• Coordinating Agents: Multiple AI agents were incorporated to handle complex tasks.
• Monitoring and Guardrails: AI systems in production were monitored and guarded.
• AI for Production Management: AI was leveraged to manage applications in production.
• Rise of AI Agents: This marks a fundamental shift in developer mindset and workflow.
• From Prompt Engineering to Spec-Driven Development: The emphasis is moving from crafting prompts to defining clear, testable specifications that guide agent behavior.
• Valuable Developers: Developers who can effectively articulate intent and enable agents to execute tasks with precision will be highly valued.
• Focus on Higher-Level Problem Solving: As agents handle routine coding tasks, developers can focus on strategy, problem-solving, and the more enjoyable aspects of their jobs.
• Orchestrators of Intelligent Systems: Developers are transitioning from mere coders to orchestrators of these intelligent systems.
• Present Reality: The presented technologies and methodologies are not future concepts but are available and in use today.
• Call to Action: Developers are encouraged to explore and deploy new agents to enhance their building experience.