Microsoft's VS Code AI Toolkit 2.0: This Extenstion is AWESOME for AI Coding! (w/ FREE GPT-5!)

Key Concepts:

• Microsoft AI Toolkit (VS Code extension)
• Local and cloud models
• Olama, ENX, Azure Foundry
• GitHub models (GPT5, Claude)
• Agent Builder (custom AI agents)
• MCP Server (integration for tools)
• Bulk Run (batch prompt testing)
• Model Evaluation (dataset-based scoring)
• Fine-tuning
• Tracing (model behavior insights)
• Ninja Chat (AI platform with multiple models)

1. Introduction and Overview of Microsoft AI Toolkit

• The video revisits Microsoft's AI Toolkit, a VS Code extension, highlighting significant upgrades since its initial coverage 10 months prior.
• The toolkit facilitates the use of both local and cloud-based AI models directly within VS Code.
• It allows users to provide custom instructions to models and easily download/install models locally.
• The extension supports running models locally via runtimes like Olama and ENX.
• Users can access a wide range of models, including GPT5 and Claude, through the GitHub models option.

2. New Features and Upgrades

• Agent Builder: A new feature that enables users to build custom AI agents within a graphical interface.
  • Agents can be given access to custom tools, which can be created by the user or integrated via an MCP server.
  • Examples of tools include web scrapers and code interpreters.
• Bulk Run: A feature for executing batch prompt testing across multiple models simultaneously.
  • Useful for comparing model performance across different models.
• Model Evaluation: Allows users to build datasets with questions and expected answers to evaluate model responses.
  • The system checks for similarity in responses and assigns scores.
• Fine-tuning: Models can now be fine-tuned directly from VS Code with a few clicks.
• Tracing: Collects and visualizes trace data to provide insights into model behavior and performance.

3. Demonstration and Usage of the AI Toolkit

• Installation and Setup:
  • The AI Toolkit must be installed from the VS Code extension marketplace and updated to the latest version.
  • The toolkit is accessible via a sidebar in VS Code.
• Models:
  • Models can be added via multiple runtimes like ENX or Azure Foundry.
  • Any Open AI compatible API model can be added.
  • The playground section allows users to interact with the models.
  • Users can access models directly through their GitHub account, including GPT4.1 and GPT5 for free.
  • Inference parameters can be set, and images, documents, and code can be attached.
  • The toolkit supports web searches.
  • Code generation includes "add" and "copy" buttons for easy integration.
  • A "get code" option provides an Open AI SDK script for projects.
• Agent Builder (Detailed):
  • Users select a model and provide system instructions for the agent.
  • Dynamic variables can be included in the instructions using curly braces.
  • Tools can be added via an MCP server or defined as custom tools.
  • MCP servers must be added from the left sidebar.
  • Examples include a web scraper.
  • The presenter uses it with Deep Wiki and web search tools as a context engine.
  • Playwright web scraper is useful for fetching and compressing documentation.
• Bulk Run and Evaluation:
  • The bulk run option allows users to create a table of prompts and run them all at once.
  • The evaluation option allows users to build evaluations for AI models by selecting question types.
  • It can measure tool calling accuracy.
  • Users enter the question, ground truth, and check the response to see the calculated accuracy.
• Tracing:
  • The tracing option allows users to see logs of AI apps and agents.

4. Ninja Chat Advertisement

• Ninja Chat is presented as an all-in-one AI platform for $11 per month.
• It provides access to top AI models like GPT40, Claude 4 Sonnet, and Gemini 2.5 Pro.
• The AI playground allows users to compare responses from different models side by side.
• The mind map generator is highlighted for organizing complex ideas.
• The basic plan includes 1,000 messages, 30 images, and five videos monthly.
• The presenter offers discount codes: "king25" for 25% off any plan and "king40yearly" for 40% off annual subscriptions.

5. Key Arguments and Perspectives

• The AI Toolkit is particularly useful for those who like to experiment with local models.
• The chat interface runs plain models without system prompts, which can slow down local models.
• The toolkit allows for fine-tuning through a conversion option.
• The presenter emphasizes the value of having a chat interface directly within the code editor.

6. Conclusion

• The Microsoft AI Toolkit is a valuable tool for VS Code users who want to experiment with local models, test multiple models, and have a chat interface within their code editor.
• The new features, such as the Agent Builder, Bulk Run, Model Evaluation, Fine-tuning, and Tracing, significantly enhance the toolkit's capabilities.
• The presenter encourages viewers to share their thoughts and subscribe to the channel.