Rubber Duck Thursdays - Let's build with custom agents (again!)

Key Concepts

• Rubber Duck Debugging: A method of debugging code by explaining the problem to an inanimate object (like a rubber duck) to help articulate and solve the issue.
• Custom Agents (GitHub Copilot): Specialized AI agents that can be configured with specific expertise and instructions to assist with various tasks within the software development lifecycle (SDLC).
• Custom Instructions (GitHub Copilot): User-defined configurations that guide Copilot's behavior, patterns, and practices, influencing how it generates code or responds to prompts.
• Software Development Lifecycle (SDLC): The entire process of developing software, from planning and design to deployment and maintenance.
• Model Context Protocol (MCP) Server: A server that orchestrates interactions with AI models, allowing for tool calls and context management.
• GitHub Actions: A CI/CD platform that automates software workflows, including building, testing, and deploying code.
• VS Code Copilot Extension: The integration of GitHub Copilot within Visual Studio Code, enabling features like custom agents and plan mode.
• Plan Mode (VS Code Copilot): A feature within the VS Code Copilot extension that allows users to plan tasks and workflows with AI assistance, often involving a series of clarifying questions.
• Agent Mode (VS Code Copilot): A general mode for interacting with Copilot agents, allowing for tool usage and file editing.
• Ask Mode (VS Code Copilot): A conversational mode for Copilot, similar to traditional chat interfaces, without direct file editing capabilities.
• Edit Mode (VS Code Copilot): A mode that allows Copilot to edit files directly.
• Retrieval Augmented Generation (RAG): A technique that enhances language models by retrieving relevant information from external sources to ground their responses.
• Context Engineering: The process of carefully crafting the input and context provided to AI models to elicit desired outputs.

Rubber Duck Thursdays: Building Custom Agents for SDLC

This session of Rubber Duck Thursdays focused on extending the concept of custom agents, building upon the previous week's work on game-playing agents. The primary goal was to integrate custom agents into the software development lifecycle (SDLC) using GitHub Copilot and its associated tools.

1. Recap and Project Context

The stream began with a recap of the "Rubber Duck Thursdays" concept, emphasizing its informal, community-driven approach to live coding and learning. The project context was established: a Next.js web app and API, an MCP server, and shared TypeScript libraries. The presenter shared the repository link (g.io/timlog) for viewers to follow along.

2. GitHub Updates and Announcements

Several recent GitHub updates were highlighted:

• Blocking Repository Administrators from Installing GitHub Apps: This new general availability feature allows organization owners to prevent repository administrators from installing GitHub apps, enhancing governance.
• Copilot Spaces Updates: Public spaces, individual sharing, and code view support were introduced, enabling easier collaboration and sharing of development environments.
• Secret Scanning Updates (November 2025): Enhancements include new provider patterns, improved private key detection, and support for secrets in unlisted GitHub gists.
• GitHub Enterprise Server 3.19 Release Candidate: This release includes improvements in deployment efficiency, monitoring, code security, policy management, and more intuitive repository creation.
• Assigning Issues to Copilot via API: The ability to assign issues to Copilot using GraphQL and REST APIs was announced, facilitating automation in issue management.
• Claude Opus 4.5 Availability: The latest version of Claude Opus is now available in various IDEs, including Visual Studio, JetBrains IDEs, Xcode, and Eclipse.

3. Custom Agents and Their Role in SDLC

The core of the session revolved around custom agents and their application within the SDLC.

• Definition of Custom Agents: Custom agents are specialized AI agents designed with tailored expertise for specific tasks. They differ from custom instructions, which define broader patterns and practices.
• Inspiration from GitHub Blog and Awesome Copilot: The presenter referenced a GitHub blog post on custom agents and the github/awesome-copilot repository as valuable resources for inspiration and examples. This repository showcases various custom agents built by partners, covering areas like data analysis, API development, and documentation.
• Examples of Potential Custom Agents for SDLC:
  • Testing Specialist: Focuses on test coverage, quality, and best practices.
  • Implementation Planner: Assists in planning and outlining development tasks (similar to VS Code's "plan mode").
  • Tech Debt Analyzer: Identifies and suggests solutions for technical debt.
  • Security Reviewer: Analyzes code for security vulnerabilities.
  • Documentation Writer/Generator: Automates the creation of documentation.
  • PR Review Assistant: Helps in reviewing pull requests.
  • Onboarding Guide: Explains the repository to new team members.
  • GitHub Actions Author: Assists in writing and updating GitHub Actions workflows.

4. Building a GitHub Actions Author Agent

A significant portion of the stream was dedicated to building a custom agent specifically for authoring and updating GitHub Actions workflows.

• Methodology: The process involved using VS Code's Copilot extension in "plan mode" to define the agent's requirements and capabilities.
• Agent Definition:
  • File Naming: The agent file was named actions-author.md.
  • Roles and Capabilities: The agent was defined to assist with authoring and updating GitHub Actions workflows, following existing repository patterns and understanding GitHub Actions syntax, mono CI patterns, and repository-specific conventions.
  • Workflow Reference: The agent was instructed to cover trigger types, job syntax, and matrix strategies.
  • Repository Context: The agent was to consider the mono-repo build order.
  • Critical Rules: The agent was to adhere to critical rules, including using commit SHAs for pinning third-party actions and avoiding inline scripts for sensitive data.
• Custom Instructions for the Agent: A custom-instructions.md file was created to guide the agent's behavior, including:
  • Build Order: Specifying the correct build order for shared packages, web, and MCP server.
  • Permissions: Recommending minimal permissions per job.
  • Secrets Handling: Emphasizing the use of secret context and avoiding inline secrets.
  • Action Pinning: Mandating pinning to commit SHAs for security.
  • Version Verification: Instructing the agent to verify action existence using tools.
  • Fail Fast vs. Debugging: Discussing the trade-offs between fail-fast strategies and allowing all jobs to complete for better debugging.
• Iterative Development and Refinement: The process involved multiple iterations of prompting the agent, reviewing its output, and refining the custom instructions and agent definition. This included:
  • The agent suggesting improvements to the existing workflows, such as adding fail-fast to matrix jobs and pinning actions to commit SHAs.
  • The agent identifying missing elements, like the need to compile the shared package before other builds.
  • The presenter manually adjusting the agent's output and instructions based on their understanding and desired outcomes.
  • The agent successfully creating a pull request with the proposed changes.
• Observing Agent Behavior: The stream demonstrated how the agent, when prompted, would:
  • Read existing workflows and custom instructions.
  • Use tools (like fetching release information) to verify action versions.
  • Suggest code modifications based on best practices and defined rules.
  • Generate a pull request with the proposed changes.
• Discussion on Agent Limitations and Human Oversight: The presenter emphasized the importance of human oversight in reviewing and validating the agent's suggestions. The example of code coverage thresholds failing highlighted that while the agent can automate tasks, human judgment is still crucial for configuration and interpretation.

5. Key Arguments and Perspectives

• Custom Agents as SDLC Accelerators: The core argument is that custom agents can significantly accelerate and improve various stages of the SDLC by automating repetitive tasks, enforcing best practices, and providing specialized expertise.
• Human-AI Collaboration: The session underscored the collaborative nature of AI tools like GitHub Copilot. The presenter actively guided and refined the agent's output, demonstrating that AI is a powerful assistant rather than a complete replacement for human developers.
• Context Engineering is Key: The effectiveness of custom agents and instructions relies heavily on providing the right context and clearly defining the desired behavior. This involves meticulous prompt engineering and the creation of well-defined instruction files.
• Iterative Improvement: Building effective custom agents is an iterative process. It requires experimentation, feedback, and continuous refinement of instructions and agent definitions.

6. Notable Quotes and Significant Statements

• "Rubber duck debugging is surprisingly effective. Forces you to articulate the problem clearly." (Chat user)
• "GitHub Copilot isn't just here to help you write code, it's here to help you manage the entire software development life cycle." (Presenter)
• "Custom agents allow you to create specialized agents with tailored expertise for specific tasks." (Presenter, referencing documentation)
• "You need that human in the loop to kind of update." (Presenter, regarding continuous learning and avoiding drift)
• "Everything you've seen me do here is nothing about model training. We're using... the models that are available... All I'm doing is effectively using a bit of retrieval augmented generation and kind of that context engineering." (Presenter, explaining domain adaptation)

7. Technical Terms and Concepts Explained

• MCP Server: Orchestrates AI model interactions and tool calls.
• Custom Agent Definition File (.md): A markdown file that defines the role, capabilities, and instructions for a custom agent.
• Custom Instructions File (.md): A markdown file that provides general guidance, patterns, and best practices for Copilot.
• Commit SHA Pinning: A security practice where specific commits of an action are used instead of just tags, ensuring stability and preventing unexpected changes.
• Matrix Strategy (GitHub Actions): Allows for running jobs with different configurations (e.g., Node.js versions) in parallel.
• Fail Fast: A strategy where a workflow stops immediately upon encountering an error, allowing for quicker identification of issues.
• RAG (Retrieval Augmented Generation): Enhances AI responses by retrieving relevant external data.

8. Data, Research Findings, or Statistics

While no specific research findings or statistics were presented, the session implicitly demonstrated the effectiveness of custom agents and instructions through practical application and the successful generation of a pull request for workflow improvements. The presenter's reliance on GitHub documentation and best practices suggests an underlying data-driven approach to AI development.

9. Logical Connections Between Sections

The stream flowed logically from an introduction to the concept and project context, through an overview of relevant GitHub updates, to the core topic of custom agents for SDLC. The practical demonstration of building a GitHub Actions author agent served as a concrete example of how these concepts are applied. The session concluded with a discussion of the agent's output, limitations, and future possibilities, reinforcing the iterative and collaborative nature of AI-assisted development.

10. Synthesis and Conclusion

This Rubber Duck Thursdays session effectively showcased the power of custom agents and custom instructions within GitHub Copilot for enhancing the software development lifecycle. By building a specialized agent for GitHub Actions workflows, the presenter demonstrated how to guide AI to adhere to specific best practices, security guidelines, and project conventions. The iterative process of defining the agent, refining instructions, and reviewing its output highlighted the crucial role of human oversight and context engineering in achieving desired outcomes. The session concluded with a strong emphasis on the collaborative potential of AI tools, encouraging viewers to explore and experiment with custom agents to streamline their own development workflows. The presenter indicated a continuation of this theme in future sessions, potentially focusing on technical debt or testing agents.