Phase 4: Data & Cache Layer - AWS Deployment Guide

🚀 2025 Performance Boost!
Redis 7.2 with Enhanced I/O Multiplexing provides 72% better throughput. Valkey support added as open-source alternative.

Step 1: Supabase Production Setup

🗄️ What is Supabase?
Supabase is an open-source Firebase alternative providing PostgreSQL database, authentication, storage, and real-time subscriptions.

1.1

Upgrade to Production Plan

Go to Supabase Dashboard → Project Settings → Billing
Upgrade to Pro plan ($25/month)
Enable features:
- High availability
- Daily backups (7-day retention)
- Connection pooling (PgBouncer)
- Read replicas (optional, for high traffic)

1.2

Configure Connection Pooling

Connection pooling prevents "too many connections" errors and improves performance.

✅ Use Port 6543 for Pooling:
Port 6543 uses PgBouncer (transaction mode) for connection pooling. Port 5432 is direct connection.

text

# Connection string format (with pooling)
postgresql://postgres:[PASSWORD]@[PROJECT-REF].supabase.co:6543/postgres?pgbouncer=true

# Update this in your Secrets Manager:
aws secretsmanager update-secret \
  --secret-id helium/backend/production \
  --secret-string '{"SUPABASE_URL":"https://your-project.supabase.co","SUPABASE_CONNECTION_STRING":"postgresql://..."}'

1.3

Run Database Migrations

bash

# From your backend directory
cd backend

# Run migrations (using your migration tool)
# Example with Alembic:
alembic upgrade head

# Or using Supabase CLI:
supabase db push

1.4

Verify Row Level Security (RLS)

RLS ensures users can only access their own data. This is critical for security!

Go to Supabase Dashboard → Authentication → Policies
Verify RLS is enabled on all user tables
Check policies for: threads, messages, agent_runs, projects
Test with different user accounts

Step 2: Create ElastiCache Redis Cluster Redis 7.2 + Enhanced I/O

📈 2025 Performance Improvement:
Redis 7.2 with Enhanced I/O Multiplexing provides 72% better throughput on instances with 4+ vCPUs. Use r7g.large or larger!

2.1

Create Subnet Group

bash

# Create subnet group for ElastiCache
aws elasticache create-cache-subnet-group \
  --cache-subnet-group-name helium-redis-subnet-group \
  --cache-subnet-group-description "Subnet group for Helium Redis cluster" \
  --subnet-ids subnet-isolated-1a subnet-isolated-1b \
  --region us-east-1

2.2

Create Redis Cluster (Cluster Mode Enabled)

💡 Cluster Mode vs Non-Cluster:
Cluster mode allows horizontal scaling and better performance. It's the 2025 best practice for production workloads.

bash

# Create Redis cluster (cluster mode enabled)
aws elasticache create-replication-group \
  --replication-group-id helium-redis-cluster \
  --replication-group-description "Helium production Redis cluster" \
  --engine redis \
  --engine-version 7.2 \
  --cache-node-type cache.r7g.large \
  --num-node-groups 2 \
  --replicas-per-node-group 1 \
  --cache-subnet-group-name helium-redis-subnet-group \
  --security-group-ids sg-redis-xxxxxxxxx \
  --automatic-failover-enabled \
  --multi-az-enabled \
  --at-rest-encryption-enabled \
  --transit-encryption-enabled \
  --auth-token "YourStrongRedisPassword123!" \
  --snapshot-retention-limit 7 \
  --snapshot-window "03:00-05:00" \
  --preferred-maintenance-window "sun:05:00-sun:07:00" \
  --region us-east-1

# Wait for cluster to be available (10-15 minutes)
aws elasticache describe-replication-groups \
  --replication-group-id helium-redis-cluster \
  --region us-east-1

💰 Cost Note:
r7g.large (Graviton3) costs ~$0.201/hour = ~$150/month for 2 nodes. This is 40% cheaper than r6g.large!

2.3

Get Redis Endpoint

bash

# Get configuration endpoint
aws elasticache describe-replication-groups \
  --replication-group-id helium-redis-cluster \
  --query 'ReplicationGroups[0].ConfigurationEndpoint.Address' \
  --output text \
  --region us-east-1

# Save this endpoint - you'll need it for Secrets Manager

2.4

Update Secrets Manager with Redis Info

bash

# Get current secret
aws secretsmanager get-secret-value \
  --secret-id helium/backend/production \
  --query SecretString \
  --output text > current-secrets.json

# Edit current-secrets.json and update:
# "REDIS_HOST": "your-cluster.xxxxx.cache.amazonaws.com"
# "REDIS_PORT": "6379"
# "REDIS_PASSWORD": "YourStrongRedisPassword123!"
# "REDIS_SSL": "True"

# Update secret
aws secretsmanager update-secret \
  --secret-id helium/backend/production \
  --secret-string file://current-secrets.json \
  --region us-east-1

# Clean up
rm current-secrets.json

2.5

Restart ECS Services

After updating secrets, restart services to pick up new Redis configuration.

bash

# Force new deployment (picks up updated secrets)
aws ecs update-service \
  --cluster helium-production-cluster \
  --service helium-backend-service \
  --force-new-deployment \
  --region us-east-1

aws ecs update-service \
  --cluster helium-production-cluster \
  --service helium-worker-service \
  --force-new-deployment \
  --region us-east-1

# Wait for services to stabilize (2-3 minutes)
aws ecs wait services-stable \
  --cluster helium-production-cluster \
  --services helium-backend-service helium-worker-service \
  --region us-east-1

Step 3: Configure S3 File Storage

3.1

Create S3 Buckets

bash

# Create bucket for user files
aws s3api create-bucket \
  --bucket helium-user-files-production \
  --region us-east-1

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket helium-user-files-production \
  --versioning-configuration Status=Enabled

# Enable encryption
aws s3api put-bucket-encryption \
  --bucket helium-user-files-production \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      }
    }]
  }'

# Block public access
aws s3api put-public-access-block \
  --bucket helium-user-files-production \
  --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

# Enable lifecycle policy (delete old files after 90 days)
cat > lifecycle-policy.json <


            
            
                
                    3.2
                    Configure CORS for S3
                
                
                
                    
                        json
                        
                    
                    {
  "CORSRules": [
    {
      "AllowedOrigins": ["https://he2.ai"],
      "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
      "AllowedHeaders": ["*"],
      "MaxAgeSeconds": 3000
    }
  ]
}
                
                
                
                    
                        bash
                        
                    
                    # Apply CORS configuration
aws s3api put-bucket-cors \
  --bucket helium-user-files-production \
  --cors-configuration file://cors-config.json
                
            
            
            
                
                    3.3
                    Update IAM Task Role for S3 Access
                
                
                
                    
                        json
                        
                    
                    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::helium-user-files-production",
        "arn:aws:s3:::helium-user-files-production/*"
      ]
    }
  ]
}
                
                
                
                    
                        bash
                        
                    
                    # Attach S3 policy to task role
aws iam put-role-policy \
  --role-name helium-ecs-task-role \
  --policy-name S3Access \
  --policy-document file://s3-policy.json



        
        
            
                
                Step 4: Test Database & Cache Connections
            
            
            
                
                    4.1
                    Test Redis Connection
                
                
                
                    
                        bash
                        
                    
                    # Test Redis from ECS task
aws ecs execute-command \
  --cluster helium-production-cluster \
  --task task-id-xxxxxxxxx \
  --container helium-backend \
  --interactive \
  --command "/bin/bash"

# Inside the container:
redis-cli -h your-cluster.cache.amazonaws.com -p 6379 --tls -a YourPassword
> PING
PONG
> SET test "Hello Redis"
OK
> GET test
"Hello Redis"
> exit
                
            
            
            
                
                    4.2
                    Test Supabase Connection
                
                
                
                    
                        bash
                        
                    
                    # Test API health endpoint (should show database connection)
curl https://api.he2.ai/api/health

# Expected response:
# {
#   "status": "healthy",
#   "database": "connected",
#   "redis": "connected",
#   "timestamp": "2025-12-05T10:00:00Z"
# }
                
            
            
            
                
                    4.3
                    Monitor Performance
                
                
                
                    Go to CloudWatch → Dashboards
                    Create dashboard: helium-production
                    Add widgets for:
                        
                            ElastiCache CPU utilization
                            ElastiCache memory usage
                            ElastiCache connections
                            ECS task count
                            ALB request count
                        
                    
                
            
        

        
        
            
                
                Phase 4 Verification Checklist
            
            
            
                 Supabase upgraded to Pro plan
                 Connection pooling enabled (port 6543)
                 Database migrations completed
                 RLS policies verified
                 ElastiCache subnet group created
                 Redis cluster created (cluster mode enabled)
                 Redis 7.2 with Enhanced I/O
                 Encryption at rest and in transit enabled
                 Multi-AZ enabled for high availability
                 Secrets Manager updated with Redis info
                 ECS services restarted
                 S3 buckets created and configured
                 Redis connection tested successfully
                 Database connection tested successfully
                 CloudWatch dashboard created
            
        

        
        
            
                
                Back to Phase 3
            
            
                Next: Phase 5 - Agent Execution