Building Agents with Amazon Nova Act and MCP - Du'An Lightfoot, Amazon (Full Workshop)

Key Concepts

• Agentic AI: AI systems that can plan, act, and reason to achieve objectives.
• Amazon Nova ACT: A research preview model for browser automation.
• Modern Context Protocol (MCP): A protocol for communication between AI agents and tools.
• Strands Agents: An open-source, lightweight framework for building agentic systems on AWS.
• Amazon Bedrock: A serverless API to access different foundation models.
• Amazon Q: An AI assistant for AWS console, IDE, and CLI.

Building Agents with Amazon Nova ACT and MCP

1. Introduction

• The presentation focuses on building intelligent, autonomous AI systems using Amazon Nova ACT and MCP to enhance applications and businesses.
• The current era is highlighted as the most exciting time for AI engineers due to advancements in agentic AI.

2. Agentic AI Fundamentals

• Planning: An agent receives a prompt and determines the necessary actions to achieve the objective.
• Acting: The agent executes the planned actions using tools.
• Reasoning: The agent evaluates the results of its actions and updates the plan as needed until the objective is met.

3. Agentic System Architecture

• User Input: The initial prompt or request from the user.
• Agentic System: The core AI agent responsible for planning, acting, and reasoning.
• Human in the Loop (Optional): Allows for human intervention and guidance.
• Generated Response: The final output or result produced by the agent.

3.1. Components of the Agentic System

• LLM (Large Language Model): Provides the reasoning and decision-making capabilities.
• Knowledge Base: External information used to ground the model and ensure accuracy.
• Guardrails: Rules and constraints to prevent undesirable actions or outputs.
• Tools: External resources or APIs that the agent can use to perform actions.
• Memory: Storage for the agent to remember past interactions and information.
• Additional Agents/LLMs: The ability to communicate with other agents or LLMs, such as Amazon Nova ACT via MCP.

4. Continuous Evaluation Framework

• LLM Selection: Determining the most suitable LLM for the task.
• Prompt Optimization: Ensuring prompts are consistent, accurate, and optimized for performance.
• System Evaluation: Judging the system's effectiveness in solving the intended problems.
• Logging: Recording all relevant information for analysis and improvement.
• Subject Matter Expert Review: Human review to identify areas for improvement.
• Iterative Approach: Continuously improving and optimizing the agentic system based on feedback.

5. Use Cases for Agentic Systems

• Agentic systems are best suited for complex tasks where the required tools and steps are not known in advance.
• Traditional "if this then that" approaches are more appropriate for simple, single-step tasks.

6. Agents on AWS: Three Approaches

• Specialized (Amazon Q): Using Amazon Q in the AWS console, IDE, or CLI to solve problems and increase productivity.
  • Example: Using Amazon Q CLI agent to analyze code, identify API mismatches, and save time.
• Fully Managed (Amazon Bedrock Agents): Building and managing agents within Amazon Bedrock.
• Do-It-Yourself (Strands Agents): Using Strands Agents to leverage Amazon Bedrock and other model providers via Light LLM.

7. Strands Agents

• Announced recently, Strands Agents is an open-source, lightweight framework for building agentic systems.
• It simplifies agent creation with a few lines of code.
• Components:
  • Prompt: Instructions for the agent.
  • LLM: The language model used for reasoning.
  • Tools: Functions or APIs that the agent can use.
• Example: Creating a "get weather" tool and defining an agent with a prompt.

8. Amazon Nova ACT

• Amazon Nova ACT is a research preview model with capabilities for complex tasks like browsing the internet and interacting with websites.
• It can be used via the SDK or through MCP.
• Example: Using Nova ACT to browse Amazon.com, find the top widgets, add them to a cart.

9. Modern Context Protocol (MCP)

• MCP has significantly accelerated the development of agents.
• MCP allows agents to communicate with various tools and services.
• Examples:
  • Building an Obsidian MCP server for documentation management.
  • Creating a bookmark manager MCP server to save and retrieve bookmarks with descriptions and notes.

10. Hands-on Workshop Overview

• The workshop provides a provisioned AWS account for participants to build agents using Nova ACT.
• The workshop consists of three modules:
  • Getting started with Nova ACT.
  • Creating an MCP server that leverages Nova ACT.
  • Using Strands Agents to integrate everything.

11. Module 1: Getting Started with Nova ACT

• Enabling Amazon Bedrock models (Claude 3 3.5 IQ and 3.5 Sonnet).
• Accessing a pre-configured VS Code server.
• Logging into Amazon Q.
• Generating a Nova ACT API key from Nova Amazon.com.
• Running a Python script to open amazon.com and search for a coffee maker.
• The code demonstrates how Nova ACT can interact with a website using natural language commands instead of manually inspecting HTML elements.
• Example code:
  • Specifies the page to go to (amazon.com).
  • Instructs the agent to search for a coffee maker.
  • Selects the first result.
  • Retrieves the title of the product page.
• The agent logs its actions, including screenshots and HTML content.
• Nova ACT can also perform parallel execution, such as searching for multiple monitors simultaneously.

12. Module 2: MCP Server for Nova ACT

• Creating an MCP server that can leverage Nova ACT.
• Using Amazon Q to explain the functionality of the MCP server.
• The MCP server includes tools for browsing sessions, executing browser actions, performing parallel tasks, taking screenshots, and closing browsers.
• The MCP client uses a large language model (Claude 3.5 Sonnet) to understand user queries and determine which tools to use.
• Example: Asking the MCP server to find a website to fix Wi-Fi problems.
• The agent opens google.com, searches for "how to fix Wi-Fi problems," and returns a list of websites with titles and descriptions.
• The agent encounters a reCAPTCHA page and gets stuck, demonstrating a limitation of Nova ACT.

13. Module 3: Strands Agents Integration

• Using Strands Agents to build agentic workflows.
• Strands Agents is a lightweight framework that simplifies agent creation.
• Example: Creating a solutions architect agent that can leverage AWS documentation and diagramming tools.
• The agent uses two MCP servers: AWS documentation and AWS diagrams.
• The agent is given a prompt to get the documentation for AWS Lambda and create a diagram of a website that uses Lambda.
• The agent searches the AWS Lambda documentation, reads the documentation, and generates a diagram illustrating a static site.
• Strands Agents supports native MCP integration and can use other models like Light LLM and O llama.
• Strands Agents also supports multi-agent collaboration, where different agents with specific roles and tools work together to achieve a common goal.
• Example: Creating a cost analysis agent, a solutions architect agent, and a presentation agent to generate a cloud migration plan and a PowerPoint presentation.

14. Key Takeaways

• Agentic AI is a powerful approach for building intelligent, autonomous systems.
• Amazon Nova ACT and MCP provide the tools and infrastructure for building these systems on AWS.
• Strands Agents simplifies agent creation and enables multi-agent collaboration.
• Continuous evaluation and iterative improvement are essential for optimizing agentic systems.

15. Additional Points from Q&A

• Nova ACT is currently only available in the US.
• Nova ACT does not bypass CAPTCHAs.
• Nova ACT can be used with a custom browser instance in the cloud.
• Nova ACT can be used for automated UI testing.
• Responsible AI is a priority, and measures are in place to prevent misuse of the technology.
• The more specific and prescriptive the instructions given to Nova ACT, the better the results.
• Strands Agents allows for defining which tools are passed on to each agent in a multi-agent system.
• Cloud 4 has interleaving thinking which allows it to handle multiple tools processing much better than most models today.