TEAMENG-1220: Update terraform documentation to deploy API Gateway and Lambda in a private subnet (#8)

# Changes

- Updates terraform boilerplate to deploy the API Gateway and Lambda in
a private subnet

Author: DrewGregory

README.md

@@ -2,22 +2,21 @@
 
 Deploy a Lambda function within your infrastructure to enable users to decrypt sensitive data from their browser.
 
-Once deployed you will be able to use the API URL provided by the AWS API Gateway and add the URL as a decryptor URI for
+Once deployed, you will be able to use the API URL provided by the AWS API Gateway and add the URL as a decryptor URI for
 a Formal Encryption Key.
 
 **Note: we highly encourage making sure the API Gateway is only accessible via a VPN to prevent users outside of your organization from making requests to the /decrypt endpoint.**
 
 There are three deployment methods: Terraform, Serverless (via Cloudformation), and Docker.
 
-## Deploying via Terraform
+## Deploying via Terraform (Recommended)
 
 To deploy via Terraform, we recommend incorporating the configuration template provided in the `terraform` directory into your Terraform setup.
-To deploy the configuration as-is, run `make deploy-terraform` with your AWS credentials and with the Terraform CLI installed.
+To deploy the configuration as-is, run `make deploy-terraform` with your AWS credentials and with the Terraform CLI installed. This deployment deploys the API Gateway and Lambda in a *private* subnet within your VPC.
 
-## Deploying via Serverless.
-
-To deploy via Serverless, run `make deploy-sls` with your Serverless credentials. Note: you will need a Serverless licesnse and the Serverless CLI installed.
+## Deploying via Serverless
 
+To deploy via Serverless, run `make deploy-sls` with your Serverless credentials. Note: you will need a Serverless licesnse, AWS Account, and the Serverless CLI installed. This deployment deploys the API Gateway and Lambda *publicly.*
 
 ## Deploying via Docker
 

serverless/README.md

@@ -1,5 +1,18 @@
 # Serverless Deployment Guide
 
+## Prerequisites
+
+- A Serverless license
+- The Serverless CLI (`npm i -E serverless@4.21.1 -g`)
+- AWS Credentials with the ability to deploy API Gateways, Lambdas, EC2 instances and the associated networking.
+
+
+## A note about private deployments
+
+This configuration deploys an AWS Lambda and API Gateway on the public internet. Instead, we recommend modifying this configuration to deploy this in a private subnet and require access to the endpoint via a VPN.
+
+## Deployment steps
+
 To deploy using the Serverless framework, run the following commands:
 ```
 npm i -E serverless@4.21.1 -g

terraform/README.md

@@ -1,12 +1,14 @@
 # Terraform Deployment Guide
 
-This directory contains Terraform configurations to deploy the decrypt Lambda function with API Gateway, equivalent to the serverless configuration.
+This directory contains Terraform configurations to deploy the decrypt Lambda function with an API Gateway in a private subnet within your VPC.
+
+This deployment will put the API Gateway and Lambda in a private subnet. We recommend accessing the resulting API Gateway decryptor URI via a VPN.
 
 ## Prerequisites
 
-- [Terraform](https://www.terraform.io/downloads.html) >= 1.0
-- AWS CLI configured with appropriate credentials
-- The `bootstrap` binary compiled and ready for deployment
+- [Terraform](https://www.terraform.io/downloads.html)
+- An AWS VPC with a private subnet.
+- AWS Credentials with the ability to deploy API Gateways, Lambdas, EC2 instances and the associated networking.
 
 ## Files
 
@@ -15,94 +17,63 @@ This directory contains Terraform configurations to deploy the decrypt Lambda fu
 - `outputs.tf` - Output definitions
 - `terraform.tfvars.example` - Example variables file
 
-## Deployment Steps
-
-### 1. Prepare the Lambda Package
-
-Before deploying, you need to package the `bootstrap` binary:
-
-```bash
-# Compile your Go code if not already done
-# GOOS=linux GOARCH=arm64 go build -o bootstrap main.go crypto.go
-
-# Create a zip file for Lambda
-zip bootstrap.zip bootstrap
-```
-
-### 2. Initialize Terraform
-
-```bash
-terraform init
-```
+## Deployment steps
 
-### 3. Configure Variables (Optional)
-
-Copy the example variables file and customize if needed:
+1. First, copy the example variables and update them.
 
 ```bash
 cp terraform.tfvars.example terraform.tfvars
 # Edit terraform.tfvars with your preferred values
 ```
 
-### 4. Plan the Deployment
-
-Review what resources will be created:
-
-```bash
-terraform plan
-```
-
-### 5. Apply the Configuration
-
-Deploy the infrastructure:
-
-```bash
-terraform apply
-```
-
-Type `yes` when prompted to confirm.
-
-### 6. Get the API Endpoint
-
-After deployment, the API Gateway URL will be displayed:
+2. Deploy the Lambda and API Gateway. Review the plan and apply.
 
 ```bash
-terraform output api_gateway_url
+make deploy-terraform # Requires AWS Credentials
 ```
-
 ## Resources Created
 
-- **Lambda Function**: `decrypt-lambda` (or custom name)
-- **IAM Role**: `decrypt-lambda-role` with KMS decrypt permissions
-- **API Gateway**: REST API with POST /decrypt endpoint
-- **CORS Configuration**: Configured for https://app.joinformal.com (or custom origin)
+- **Lambda Function**: `decrypt-lambda` (or custom name) deployed in private subnets
+- **IAM Role**: `decrypt-lambda-role` with KMS decrypt and VPC access permissions
+- **API Gateway**: Private REST API with POST /decrypt endpoint
+- **VPC Endpoint**: Interface endpoint for API Gateway execute-api service
+- **Security Groups**:
+  - Lambda security group with egress to all
+  - API Gateway VPC endpoint security group with ingress on port 443 from VPC CIDR
+- **CORS Configuration**: Configured for https://app.joinformal.coms
 - **CloudWatch Log Group**: For Lambda function logs
 
-## Updating the Lambda Function
+## VPC Configuration
 
-After making changes to your code:
+This deployment creates a **private API Gateway** accessible only from within the VPC. The Lambda function runs in private subnets and connects to API Gateway through a VPC endpoint.
 
-1. Rebuild the bootstrap binary
-2. Recreate the zip file: `zip bootstrap.zip bootstrap`
-3. Run `terraform apply` to update the Lambda function
+### VPC Requirements
 
-## Cleanup
+- **VPC ID**: An existing VPC where resources will be deployed
+- **Private Subnets**: At least 2 private subnets (recommended for high availability)
+  - Subnets should have routes to a NAT Gateway if the Lambda needs internet access
+  - Subnets should be in different Availability Zones for resilience
+- **VPC Endpoints**: The terraform configuration automatically creates the required API Gateway execute-api endpoint
 
-To destroy all resources:
+### Network Architecture
 
-```bash
-terraform destroy
 ```
+Client (within VPC) → VPC Endpoint (execute-api) → Private API Gateway → Lambda (in private subnet)
+```
+
+The API Gateway is not accessible from the public internet. We recommend requiring access through a VPN so that users can access the API Gateway from their browsers.
 
 ## Variables
 
-| Variable | Description | Default |
-|----------|-------------|---------|
-| `aws_region` | AWS region to deploy resources | `us-east-1` |
-| `function_name` | Name of the Lambda function | `decrypt-lambda` |
-| `stage_name` | API Gateway stage name | `prod` |
-| `kms_key_arn` | ARN for KMS key we're using to decrypt | `` |
-| `log_retention_days` | CloudWatch log retention in days | `14` |
+| Variable | Description | Default | Required |
+|----------|-------------|---------|----------|
+| `aws_region` | AWS region to deploy resources | `us-east-1` | No |
+| `function_name` | Name of the Lambda function | `decrypt-lambda` | No |
+| `stage_name` | API Gateway stage name | `prod` | No |
+| `kms_key_arn` | ARN for KMS key we're using to decrypt | - | Yes |
+| `vpc_id` | VPC ID where Lambda and API Gateway will be deployed | - | Yes |
+| `private_subnet_ids` | List of private subnet IDs (recommend 2+) | - | Yes |
+| `log_retention_days` | CloudWatch log retention in days | `14` | No |
 
 ## Outputs
 
@@ -114,3 +85,7 @@ terraform destroy
 | `lambda_function_arn` | Lambda function ARN |
 | `lambda_role_arn` | Lambda IAM role ARN |
 | `cloudwatch_log_group_name` | CloudWatch Log Group name |
+| `vpc_endpoint_id` | VPC Endpoint ID for API Gateway |
+| `vpc_endpoint_dns_entries` | DNS entries for the VPC endpoint |
+| `vpc_endpoint_private_ips` | Private IP addresses of the VPC endpoint|
+| `access_instructions` | Instructions for accessing the private API |

terraform/main.tf

@@ -53,6 +53,28 @@ resource "aws_iam_role_policy_attachment" "lambda_basic_execution" {
   policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
 }
 
+resource "aws_iam_role_policy_attachment" "lambda_vpc_execution" {
+  role       = aws_iam_role.decrypt_lambda_role.name
+  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
+}
+
+resource "aws_security_group" "lambda_sg" {
+  name        = "${var.function_name}-lambda-sg"
+  description = "Security group for decrypt Lambda function"
+  vpc_id      = var.vpc_id
+
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+
+  tags = {
+    Name = "${var.function_name}-lambda-sg"
+  }
+}
+
 resource "aws_lambda_function" "decrypt" {
   filename         = "../bootstrap.zip"
   function_name    = var.function_name
@@ -65,16 +87,92 @@ resource "aws_lambda_function" "decrypt" {
   environment {
     variables = {}
   }
+
+  vpc_config {
+    subnet_ids         = [var.private_subnet_id]
+    security_group_ids = [aws_security_group.lambda_sg.id]
+  }
 }
 
 resource "aws_cloudwatch_log_group" "decrypt_lambda_logs" {
   name              = "/aws/lambda/${var.function_name}"
   retention_in_days = var.log_retention_days
 }
 
+data "aws_vpc" "selected" {
+  id = var.vpc_id
+}
+
+resource "aws_security_group" "apigw_endpoint_sg" {
+  name        = "${var.function_name}-apigw-endpoint-sg"
+  description = "Security group for API Gateway VPC endpoint"
+  vpc_id      = var.vpc_id
+
+  ingress {
+    description = "HTTPS from VPC CIDR"
+    from_port   = 443
+    to_port     = 443
+    protocol    = "tcp"
+    cidr_blocks = [data.aws_vpc.selected.cidr_block]
+  }
+
+  egress {
+    from_port   = 0
+    to_port     = 0
+    protocol    = "-1"
+    cidr_blocks = ["0.0.0.0/0"]
+  }
+
+  tags = {
+    Name = "${var.function_name}-apigw-endpoint-sg"
+  }
+}
+
+resource "aws_vpc_endpoint" "apigw_endpoint" {
+  vpc_id              = var.vpc_id
+  service_name        = "com.amazonaws.${var.aws_region}.execute-api"
+  vpc_endpoint_type   = "Interface"
+  private_dns_enabled = true
+
+  subnet_ids         = [var.private_subnet_id]
+  security_group_ids = [aws_security_group.apigw_endpoint_sg.id]
+
+  tags = {
+    Name = "${var.function_name}-apigw-endpoint"
+  }
+}
+
+# Data source to get details of VPC endpoint ENIs
+data "aws_network_interface" "vpc_endpoint_enis" {
+  count = length(tolist(aws_vpc_endpoint.apigw_endpoint.network_interface_ids))
+  id    = tolist(aws_vpc_endpoint.apigw_endpoint.network_interface_ids)[count.index]
+}
+
 resource "aws_api_gateway_rest_api" "decrypt_api" {
   name        = "${var.function_name}-api"
   description = "API Gateway for decrypt Lambda function"
+
+  endpoint_configuration {
+    types            = ["PRIVATE"]
+    vpc_endpoint_ids = [aws_vpc_endpoint.apigw_endpoint.id]
+  }
+
+  policy = jsonencode({
+    Version = "2012-10-17"
+    Statement = [
+      {
+        Effect    = "Allow"
+        Principal = "*"
+        Action    = "execute-api:Invoke"
+        Resource  = "*"
+        Condition = {
+          StringEquals = {
+            "aws:SourceVpce" = aws_vpc_endpoint.apigw_endpoint.id
+          }
+        }
+      }
+    ]
+  })
 }
 
 resource "aws_api_gateway_resource" "decrypt_resource" {

terraform/outputs.tf

@@ -27,3 +27,39 @@ output "cloudwatch_log_group_name" {
   description = "Name of the CloudWatch Log Group"
   value       = aws_cloudwatch_log_group.decrypt_lambda_logs.name
 }
+
+output "vpc_endpoint_id" {
+  description = "ID of the API Gateway VPC Endpoint"
+  value       = aws_vpc_endpoint.apigw_endpoint.id
+}
+
+output "vpc_endpoint_dns_entries" {
+  description = "DNS entries for the VPC endpoint"
+  value       = aws_vpc_endpoint.apigw_endpoint.dns_entry
+}
+
+output "vpc_endpoint_private_ips" {
+  description = "Private IP addresses of the VPC endpoint ENIs"
+  value       = [for eni in data.aws_network_interface.vpc_endpoint_enis : eni.private_ip]
+}
+
+output "access_instructions" {
+  description = "Instructions for accessing the private API"
+  value       = <<-EOT
+    This is a PRIVATE API Gateway, accessible only from within the VPC.
+
+    API Gateway URL: ${aws_api_gateway_stage.decrypt_stage.invoke_url}/decrypt
+    API Gateway ID: ${aws_api_gateway_rest_api.decrypt_api.id}
+    VPC Endpoint ID: ${aws_vpc_endpoint.apigw_endpoint.id}
+    VPC Endpoint Private IPs: ${join(", ", [for eni in data.aws_network_interface.vpc_endpoint_enis : eni.private_ip])}
+
+    === From within VPC (EC2/ECS) ===
+    curl -X POST ${aws_api_gateway_stage.decrypt_stage.invoke_url}/decrypt \
+      -H "Content-Type: application/json" \
+      -d '{"encrypted_data": "your-encrypted-data"}'
+
+    DNS Resolution:
+    - Private DNS is enabled on the VPC endpoint
+    - The API Gateway hostname will resolve to private IPs within your VPC
+  EOT
+}

terraform/terraform.tfvars.example

@@ -7,5 +7,9 @@ stage_name    = "prod"
 cors_origin   = "https://app.joinformal.com"
 kms_key_arn   = ""
 
+# VPC Configuration for private subnet deployment
+vpc_id             = "vpc-xxxxxxxxx"
+private_subnet_id = "subnet-xxxxxxxxx" # This private subnet needs to be part of the VPC.
+
 # Optional: Customize CloudWatch log retention
 # log_retention_days = 14

terraform/variables.tf

@@ -26,3 +26,13 @@ variable "kms_key_arn" {
   description = "ARN of the KMS key to use for decrypting data"
   type        = string
 }
+
+variable "vpc_id" {
+  description = "VPC ID where Lambda and API Gateway will be deployed"
+  type        = string
+}
+
+variable "private_subnet_id" {
+  description = "Private subnet ID for Lambda and API Gateway VPC endpoint"
+  type        = string
+}