Live Q&A - Let's Chat About All Things AI & AI Agents!

Key Concepts

• AI Agents: Software entities designed to perform tasks autonomously, often leveraging large language models (LLMs).
• LLMs (Large Language Models): AI models trained on vast amounts of text data, capable of generating human-like text, translating languages, and answering questions.
• RAG (Retrieval-Augmented Generation): A technique that enhances LLMs by retrieving relevant information from external knowledge sources before generating a response.
• MCP (Model Context Protocol): A protocol that enables AI coding assistants to access and utilize external tools and information sources.
• AI Coding Assistants: Tools that assist developers in writing code, often leveraging AI to suggest code completions, identify errors, and generate code snippets.
• Open Source: Software with publicly available source code, allowing for modification and distribution.
• Local AI: Running AI models and applications on a local machine rather than relying on cloud-based services.
• Agentic Design Patterns: Architectures and strategies for building and combining AI agents to achieve complex tasks.
• Guardrails: Mechanisms to ensure that AI agents behave safely and reliably, preventing them from generating harmful or inaccurate outputs.
• Dynamis.ai: A community and platform focused on mastering the building of AI agents.

Main Topics and Key Points

Introduction

• The live Q&A session is a recurring event, held approximately monthly, following the positive reception of the previous session.
• The focus is on answering questions related to AI, without showcasing specific projects or live coding.

Clarverse

• Clarverse is a local AI package that functions as a complete application, similar to Open Web UI but with additional features.
• It allows users to build agents, chat with local LLMs (e.g., Gemma 327B via Olama), and attach images and files.
• Clarverse integrates NATON workflows directly within its UI and offers various workflow templates.
• It features an agent builder with tools, long-term memory, and advanced system prompts.
• The application also includes an app builder resembling Lang flow or Voice flow.
• Clarverse is open source and has a promising tech stack.

Video Editing and Live Streaming

• OBS is used for recording and live streaming.
• Cap Cut is used for video editing due to its simplicity and sufficient features.
• The focus is on providing value through open-source projects and code examples rather than flashy production.

Chatterbox

• Chatterbox is a production-grade open-source text-to-speech model from resemble.ai, licensed under MIT.
• It is based on a 0.5 billion parameter llama backbone trained on 0.5 million hours of clean data.
• Chatterbox claims to outperform 11 Labs in voice conversion.

Productivity

• Passion for AI and automation drives productivity.
• Time is primarily invested in YouTube and Dynamis, creating content, building projects, and engaging with the community.

Process Map for Learning AI Agents

• A roadmap for learning how to build AI agents, starting with foundational knowledge and progressing to advanced architectures and deployment.
• The process emphasizes learning by doing, starting with no-code tools like NAN and then transitioning to coding with AI coding assistants.
• Key phases include:
  • Learning the foundations.
  • Mastering AI coding assistants.
  • Coding AI agents with AI assistance.
  • Implementing advanced architectures (guardrails, fallback systems).
  • Deploying and monitoring AI agents in production.
• The roadmap highlights the importance of mastering AI with others, leading to the creation of Dynamis.ai.

Dynamis.ai

• Dynamis.ai is a community and platform for mastering AI agent building.
• It offers a course on building AI agents from scratch, daily events (office hours, workshops), templates, and resources.
• The AI agent mastery course includes modules on planning, prototyping, coding, and building full applications around agents.
• A recent module release covers turning agents into APIs and building front ends with tools like lovable and superbase.

Pantic AI vs. Crew AI

• Pantic AI is preferred for its greater control and customization options compared to Crew AI.
• While Crew AI is easier to use, Pantic AI allows for more flexibility in formatting outputs and accessing underlying agent responses.
• Pantic AI's graph-based implementation and streaming structured outputs are also highlighted as advantages.

Future of AI Agents

• Websites will be designed more for AI agents, allowing them to guide users and summarize information.
• Enterprise-level adoption of AI coding assistants will increase.
• Anthropic already has 70% of its code written by AI.

Light Rag and Rag Strategies

• Light rag is liked but can be slow.
• Combining multiple rag strategies is recommended.
• Light rag combines a knowledge graph and a vector database.
• Other strategies include agentic rag, contextual embeddings, hybrid searching, reranking, and query expansion.

Claraara as an NAN Killer

• Claraara is not an NAN killer; it solves a different problem.
• NAN is integrated into Claraara.

Most Difficult Thing in Making and Maintaining AI Agents

• The most difficult thing is making the agent go from good to great.
• This is achieved through effective evaluation and iterative improvement.

AI Agent for Gathering Data Across Socials

• There are ethical and technical challenges in scraping social media data.
• Tools like bright data and crawl for AI can be used for this purpose.
• Building such an agent live in a future stream is considered.

Archon and Local AI Package Plans

• Archon is an open-source AI agent builder that functions as an MCP server and standalone application.
• The local AI package is a collection of open-source tools for running local AI agents (N8N, Superbase, Olama, Quadrant, Neo4j, CRXNG, Langfuse).
• The local AI package is in a good state, with a user interface being developed to manage environment variables and customize the stack.
• Archon's vision is evolving to focus more on knowledge-based building for rag, using crawl for AAI.
• A new MCP server called crawl for AAI rag is being developed to scrape websites and create knowledge bases.
• Archon will be overhauled to incorporate this knowledge backbone and potentially become a taskmaster for AI coding assistants.

Last Mile MCP Agent

• The Last Mile MCP Agent is based on Anthropic's patterns for building effective agents.
• It aims to provide an MCP server that embodies best practices for agent architecture.

Agentic Design Pattern

• The Anthropic article "Building Effective Agents" is a favorite resource for agentic design patterns.
• Key patterns include:
  • Chaining: Combining LLMs in a workflow.
  • Routing: Directing requests to specialized agents.
  • Parallelization: Running multiple agents concurrently.
  • Orchestrator-Worker: Combining task preparation, distribution, and synthesis.
  • Evaluator-Optimizer: Using an LLM to evaluate the output of another.
• Incrementally improving tasks involves evaluator-optimizer patterns, long-term memory, and code modifications.

Cloud Code Parallelization

• Cloud code parallelization is a promising technique for AI coding assistance.
• It allows for concurrent task execution and AB testing of code solutions.
• A community member in Dynamis is hosting a workshop on this topic.

GBR (Gain Beyond Rag)

• GBR is a technique that seems similar to agentic rag, giving agents the ability to look at knowledge bases in different ways.

Unsloth AI

• Unsloth AI is a tool for fine-tuning LLMs.

AI Voice Customer Support Agents Cost

• To minimize costs, use the cheapest LLM provider that meets the use case requirements.
• Consider speed as well as cost.

A2A Protocol

• A2A is a protocol developed by Google for connecting AI agents, allowing them to learn each other's capabilities.

Convex

• Convex is a reactive database for app developers, potentially replacing superbase and integrating directly into the front end.
• It is open source and can be self-hosted.

Search R1 GBR

• Search R1 GBR is a recently released technique that gains beyond rag.

Taskmaster for AI Coding

• Taskmaster is a tool for orchestrating AI coding tasks.
• The speaker plans to build similar functionality into Archon.

MongoDB for Agents

• MongoDB hasn't been used for AI agents recently.
• SQL databases are preferred because LLMs understand SQL well, making it easier to analyze trends and patterns.

Ice Cream Flavor

• Favorite ice cream flavor is peanut butter cup, especially from Dairy Queen or Culver's.

Relational Database for AI Agents

• Relational databases are great for AI agents.

Volt Agent

• Volt Agent is an open-source TypeScript AI agent framework with a workflow builder.

N8N in Production

• N8N is great for fast prototyping and internal automations.
• For production-grade agents that need to scale, coding the agent is preferred.

Keeping Coding Agents on Guardrails

• Use global rules to dictate the agent's behavior.
• Provide examples to guide the agent.
• Use MCP servers to guide reasoning.

AGUI Protocol with HTMX

• No experience with AGUI with HTMX.

Collaboration or Partnerships

• Open to collaborations or partnerships.

Overarching Principles

• Focus on capabilities, not tools.
• Develop skills that translate across different tools and frameworks.

Prompt Engineering

• Check out the speaker's video on using AI coding assistants.
• Use promptingguide.ai as a resource.

Relational Tables for Knowledge AI

• The number of tables depends on the rag strategy.
• A basic setup requires a single table with documents and embeddings.
• Agentic rag may use a second table for document metadata.

AGI in 5-10 Years

• There is a realistic possibility of achieving AGI in the next 5-10 years.

Coding Assistant Recommendation

• Windsurf is recommended for simplicity.
• Ruko or Claude Code are recommended for complex, agentic coding processes.

Agent Flow Framework

• Use agentic design patterns from the "Building Effective Agents" article.
• Consider the specific use case and whether a multi-agent workflow is necessary.

Open Source, Free, and Local

• These words may be daunting to some, but local AI is becoming easier to use.
• The gap between cloud AI and local AI is closing.

Agent Zero

• Agent Zero seems cool.

Revenue Stream in Publishing and Selling Workflows on the NAN Website

• Use templates as a lead magnet for consulting services.

Conclusion

• The live Q&A session covered a wide range of topics related to AI agents, LLMs, rag, and AI coding assistants.
• The speaker shared insights, recommendations, and resources based on their experience and expertise.
• The importance of open source, local AI, and focusing on capabilities over tools was emphasized.
• The session provided valuable information and inspiration for those interested in mastering the building of AI agents.