How to Connect an AWS Lambda Function to RDS Database

Key Concepts

• Lambda Function in VPC: Placing a Lambda function within a Virtual Private Cloud (VPC) for enhanced security and access to private resources.
• RDS Instance in Private Subnet: Hosting a Relational Database Service (RDS) instance within a private subnet to restrict public access and improve security.
• Security Groups: Virtual firewalls that control inbound and outbound traffic to AWS resources.
• NAT Gateway: A Network Address Translation (NAT) service that enables instances in a private subnet to connect to the internet without being directly exposed.
• Public vs. Private Subnets: Public subnets have a route to the internet gateway, allowing internet access. Private subnets do not and require a NAT gateway for outbound internet access.
• Route Tables: Define the routes for network traffic within a VPC.

Connecting Lambda to RDS in a Private Subnet and Enabling Internet Access

The Problem: Default Lambda and RDS Configuration

• By default, a Lambda function cannot directly communicate with an RDS instance in a private subnet.
• Lambda functions typically reside outside a VPC in AWS's Lambda service VPC, preventing direct access to RDS instances within a private VPC.
• Placing the RDS instance in a public subnet with a public IP address exposes it to the internet, creating significant security risks.
• Using security group rules to restrict access by IP address is insufficient due to IP address rotation and other security vulnerabilities.

The Right Way: VPC, Private Subnets, and NAT Gateway

• Goal: Enable Lambda function to securely access RDS in a private subnet and connect to the public internet.
• Components:
  • VPC: A logically isolated section of the AWS cloud where you can launch AWS resources in a defined virtual network.
  • Public Subnets: Subnets within the VPC that have a route to the internet gateway.
  • Private Subnets: Subnets within the VPC that do not have a direct route to the internet gateway.
  • NAT Gateway: Placed in a public subnet, it allows instances in private subnets to initiate outbound connections to the internet.
• Process:
  1. Security Group Configuration for RDS Access:
     • Create an outbound rule on the Lambda function's security group to allow traffic to the RDS instance's security group.
     • Create an inbound rule on the RDS instance's security group to allow traffic from the Lambda function's security group.
     • This establishes bidirectional communication between the Lambda function and the RDS instance.
  2. Enabling Internet Access for Lambda:
     • Create a NAT gateway in a public subnet. The NAT gateway is assigned a public IP address.
     • Modify the route table for the private subnet where the Lambda function resides.
     • Add a route that directs all outbound traffic destined for the internet (0.0.0.0/0) to the NAT gateway.
     • The NAT gateway then routes the traffic to the internet gateway.

AWS Console Setup: Step-by-Step Guide

1. Creating the VPC

• Navigate to the VPC section in the AWS console.
• Use the "VPC and more" feature to create a VPC with public and private subnets, and a NAT gateway.
• Configuration:
  • Name: "VPC RDS Lambda demo"
  • IPv4 CIDR block: Use a non-default value (e.g., 7.0.0.0/16) to avoid potential peering conflicts.
  • Availability Zones: 2 (required for RDS setup).
  • Public Subnets: 2.
  • Private Subnets: 2.
  • NAT Gateway: 1 (in 1 AZ).
  • VPC Endpoints: None (skipped for brevity).
• The wizard automatically creates the VPC, subnets, route tables, and NAT gateway.

2. Creating the RDS Instance

• Navigate to the RDS section in the AWS console.
• Create a DB Subnet Group:
  • Name: "VPC RDS Lambda demo subnet group"
  • VPC: Select the VPC created in the previous step.
  • Availability Zones: Select two private subnets (e.g., "private-2" and "private-1").
• Create the Database:
  • Engine: MySQL (but the configuration works for Aurora, PostgreSQL, Oracle, and Microsoft SQL Server with minor adjustments).
  • Template: Free tier (for demonstration purposes).
  • DB instance identifier: "database"
  • Master username: Change from default (e.g., "admin").
  • Master password: Set a password (e.g., "88").
  • Instance size: "db.t4g.micro" (free tier).
  • Storage: 20GB (general purpose SSD).
  • Connectivity:
    • Compute: Don't connect to an EC2 compute resource.
    • Network type: IPv4.
    • VPC: Select the VPC created earlier.
    • Subnet group: Select the DB subnet group created earlier.
    • Public access: No (crucial for security).
    • VPC security group: Create a new security group (or use an existing one) named "RDS database security group". Remove the default security group.
  • Additional configuration:
    • Database name: "database1" (avoid using reserved words like "database").
    • Port: 3306 (default MySQL port).
• Record the database endpoint, username, password, and port for later use in the Lambda function code.

3. Creating the Lambda Function

• Navigate to the Lambda section in the AWS console.
• Create a new function.
• Configuration:
  • Function name: "demo function".
  • Runtime: Python 3.13.
  • Permissions: Create a new role with basic Lambda permissions.
  • Additional configuration:
    • Enable VPC: Select the VPC created earlier.
    • Subnet: Select two private subnets.
    • Security groups: Create a new security group (if one doesn't exist) or select an existing one named "Lambda security group".
• Create the Lambda Security Group (if needed):
  • Navigate to the EC2 section, then Security Groups.
  • Create a new security group named "Lambda security group".
  • VPC: Select the VPC created earlier.
  • Outbound rules: Allow all traffic to 0.0.0.0/0 (public internet) – this will go through the NAT Gateway.

4. Configuring Security Groups for Communication

• Lambda to RDS:
  • Edit the outbound rules of the "Lambda security group" to allow all traffic to the "RDS database security group".
• RDS to Lambda:
  • Edit the inbound rules of the "RDS database security group" to allow all traffic from the "Lambda security group".

5. Lambda Function Code and Deployment

• Code:
  • Create a file named lambda_function.py.
  • Import necessary libraries: pymysql and requests.
  • Define constants for the database connection details (host, username, password, database name, port).
  • Write code to connect to the MySQL database, execute a query (e.g., SELECT CURRENT_TIME), and fetch the result.
  • Write code to make an HTTP request to a public API (e.g., GitHub API).
  • Combine the database result and HTTP response into a single response object.
  • Include error handling to catch exceptions.
• Dependencies:
  • Create a file named requirements.txt.
  • Add the following dependencies:
    • pymysql
    • requests
  • Install the dependencies using pip install -r requirements.txt -t . (installs packages into the current directory).
• Deployment:
  • Zip the contents of the directory (lambda_function.py, and the installed libraries) into a file named lambda_function.zip. Important: Ensure you zip the contents of the directory, not the directory itself.
  • In the Lambda console, upload the lambda_function.zip file.
  • Deploy the code.

6. Testing the Lambda Function

• Create a test event in the Lambda console.
• Invoke the Lambda function.
• Verify that the function successfully connects to the RDS database, makes an HTTP request to the public internet, and returns a combined response.

Notable Quotes

• "Typically what happens is that our RDS database is not able to talk to a Lambda function that is set up by default."
• "Security groups are basically firewalls."
• "You need a public IP address in order to talk to the public internet."

Technical Terms and Concepts

• VPC Peering: Connecting two VPCs to enable network traffic between them.
• CIDR Block: A Classless Inter-Domain Routing block, representing a range of IP addresses.
• Availability Zone (AZ): A physically distinct location within an AWS region.
• Route Table: A set of rules, called routes, that are used to determine where network traffic is directed.
• Internet Gateway: A VPC component that allows communication between instances in the VPC and the internet.
• Connection Pooling: A technique used to maintain a pool of database connections for reuse, improving performance and reducing resource consumption.

Data, Research Findings, or Statistics

• NAT Gateway cost: Approximately $40 per month.

Logical Connections

The video logically connects the problem of default Lambda and RDS configurations to the solution of using VPCs, private subnets, and NAT gateways. It provides a step-by-step guide to implement the solution in the AWS console, covering VPC creation, RDS instance setup, Lambda function configuration, security group rules, and code deployment. The video emphasizes the importance of security and best practices throughout the process.

Synthesis/Conclusion

The video provides a comprehensive guide to connecting a Lambda function to an RDS instance in a private subnet while enabling internet access. By using VPCs, private subnets, NAT gateways, and properly configured security groups, you can create a secure and functional architecture. The step-by-step instructions and code examples make it easy to implement the solution in your own AWS environment. The key takeaways are the importance of security, the need for proper network configuration, and the benefits of using AWS services like NAT Gateway for specific use cases.