Phase 6: CI/CD Pipeline - AWS Deployment Guide

Step 1: Configure GitHub Secrets

🔐 What are GitHub Secrets?
GitHub Secrets securely store credentials needed for CI/CD workflows. They're encrypted and never exposed in logs.

1.1

Add AWS Credentials

Go to your GitHub repository
Settings → Secrets and variables → Actions
Click "New repository secret"
Add these secrets:
- AWS_ACCESS_KEY_ID
- AWS_SECRET_ACCESS_KEY
- AWS_REGION = us-east-1
- AWS_ACCOUNT_ID

⚠️ Security Best Practice:
Create a dedicated IAM user for CI/CD with minimal permissions (ECR push, ECS update only). Don't use your admin credentials!

1.2

Add Cloudflare Credentials

Get Cloudflare API token from dashboard
Add secrets:
- CLOUDFLARE_API_TOKEN
- CLOUDFLARE_ACCOUNT_ID

Step 2: Create Backend Deployment Workflow

2.1

Create Workflow File

Create .github/workflows/deploy-backend.yml:

yaml

name: Deploy Backend to ECS

on:
  push:
    branches:
      - main
    paths:
      - 'backend/**'
      - '.github/workflows/deploy-backend.yml'

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: helium-backend
  ECS_CLUSTER: helium-production-cluster
  ECS_SERVICE: helium-backend-service
  ECS_TASK_DEFINITION: helium-backend

jobs:
  deploy:
    name: Deploy Backend
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}
      
      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2
      
      - name: Build, tag, and push image to ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          cd backend
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker tag $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG $ECR_REGISTRY/$ECR_REPOSITORY:latest
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest
          echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT
      
      - name: Download task definition
        run: |
          aws ecs describe-task-definition \
            --task-definition ${{ env.ECS_TASK_DEFINITION }} \
            --query taskDefinition > task-definition.json
      
      - name: Update task definition with new image
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition: task-definition.json
          container-name: helium-backend
          image: ${{ steps.build-image.outputs.image }}
      
      - name: Deploy to ECS
        uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: ${{ env.ECS_SERVICE }}
          cluster: ${{ env.ECS_CLUSTER }}
          wait-for-service-stability: true
      
      - name: Verify deployment
        run: |
          echo "Deployment completed successfully!"
          echo "Waiting for service to stabilize..."
          aws ecs wait services-stable \
            --cluster ${{ env.ECS_CLUSTER }} \
            --services ${{ env.ECS_SERVICE }}
          echo "Service is stable!"

Step 3: Create Frontend Deployment Workflow

3.1

Create Frontend Workflow

Create .github/workflows/deploy-frontend.yml:

yaml

name: Deploy Frontend to Cloudflare Pages

on:
  push:
    branches:
      - main
    paths:
      - 'frontend/**'
      - '.github/workflows/deploy-frontend.yml'

jobs:
  deploy:
    name: Deploy Frontend
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
          cache-dependency-path: frontend/package-lock.json
      
      - name: Install dependencies
        run: |
          cd frontend
          npm ci
      
      - name: Build Next.js app
        run: |
          cd frontend
          npm run build
          npm run pages:build
        env:
          NEXT_PUBLIC_SUPABASE_URL: ${{ secrets.NEXT_PUBLIC_SUPABASE_URL }}
          NEXT_PUBLIC_SUPABASE_ANON_KEY: ${{ secrets.NEXT_PUBLIC_SUPABASE_ANON_KEY }}
          NEXT_PUBLIC_BACKEND_URL: https://api.he2.ai
          NEXT_PUBLIC_URL: https://he2.ai
          NEXT_PUBLIC_ENV_MODE: production
      
      - name: Deploy to Cloudflare Pages
        uses: cloudflare/wrangler-action@v3
        with:
          apiToken: ${{ secrets.CLOUDFLARE_API_TOKEN }}
          accountId: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
          command: pages deploy frontend/.vercel/output/static --project-name=helium-frontend
      
      - name: Verify deployment
        run: |
          echo "Frontend deployed successfully!"
          echo "Visit: https://he2.ai"

Step 4: Add Automated Testing

4.1

Create Test Workflow

Create .github/workflows/test.yml:

yaml

name: Run Tests

on:
  pull_request:
    branches:
      - main
  push:
    branches:
      - main

jobs:
  test-backend:
    name: Test Backend
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Setup Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Install dependencies
        run: |
          cd backend
          pip install -r requirements.txt
      
      - name: Run tests
        run: |
          cd backend
          pytest tests/ -v
      
      - name: Run linting
        run: |
          cd backend
          ruff check .
  
  test-frontend:
    name: Test Frontend
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
          cache-dependency-path: frontend/package-lock.json
      
      - name: Install dependencies
        run: |
          cd frontend
          npm ci
      
      - name: Run linting
        run: |
          cd frontend
          npm run lint
      
      - name: Run type checking
        run: |
          cd frontend
          npm run type-check
      
      - name: Build test
        run: |
          cd frontend
          npm run build

Step 5: Add Security Scanning

5.1

Add Dependency Scanning

Create .github/workflows/security.yml:

yaml

name: Security Scanning

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
  schedule:
    - cron: '0 0 * * 0' # Weekly on Sunday

jobs:
  dependency-scan:
    name: Scan Dependencies
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Run Trivy vulnerability scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          scan-ref: '.'
          format: 'sarif'
          output: 'trivy-results.sarif'
      
      - name: Upload Trivy results to GitHub Security
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: 'trivy-results.sarif'
  
  container-scan:
    name: Scan Container Images
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Build Docker image
        run: |
          cd backend
          docker build -t helium-backend:test .
      
      - name: Run Trivy container scan
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: 'helium-backend:test'
          format: 'sarif'
          output: 'trivy-container-results.sarif'
      
      - name: Upload results
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: 'trivy-container-results.sarif'

Step 6: Configure Rollback Strategy

🔄 Why Rollback?
If a deployment causes issues, you need a quick way to revert to the previous working version.

6.1

Manual Rollback Procedure

bash

# List recent task definitions
aws ecs list-task-definitions \
  --family-prefix helium-backend \
  --sort DESC \
  --max-items 5 \
  --region us-east-1

# Rollback to previous version
aws ecs update-service \
  --cluster helium-production-cluster \
  --service helium-backend-service \
  --task-definition helium-backend:PREVIOUS_REVISION \
  --force-new-deployment \
  --region us-east-1

# Wait for rollback to complete
aws ecs wait services-stable \
  --cluster helium-production-cluster \
  --services helium-backend-service \
  --region us-east-1

echo "Rollback completed!"

6.2

Automated Rollback on Failure

Add this to your deployment workflow to automatically rollback on health check failures:

yaml

      - name: Health check
        id: health-check
        run: |
          sleep 60 # Wait for deployment
          for i in {1..5}; do
            if curl -f https://api.he2.ai/api/health; then
              echo "Health check passed!"
              exit 0
            fi
            echo "Health check failed, retrying..."
            sleep 10
          done
          echo "Health check failed after 5 attempts"
          exit 1
      
      - name: Rollback on failure
        if: failure() && steps.health-check.outcome == 'failure'
        run: |
          echo "Rolling back to previous version..."
          PREVIOUS_TASK_DEF=$(aws ecs describe-services \
            --cluster ${{ env.ECS_CLUSTER }} \
            --services ${{ env.ECS_SERVICE }} \
            --query 'services[0].deployments[1].taskDefinition' \
            --output text)
          
          aws ecs update-service \
            --cluster ${{ env.ECS_CLUSTER }} \
            --service ${{ env.ECS_SERVICE }} \
            --task-definition $PREVIOUS_TASK_DEF \
            --force-new-deployment
          
          echo "Rollback initiated!"

Step 7: Set Up Deployment Notifications

7.1

Add Slack Notifications (Optional)

Get notified in Slack when deployments succeed or fail.

yaml

      - name: Notify Slack on success
        if: success()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {
              "text": "✅ Backend deployment successful!",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Backend Deployment Successful* ✅\n\n*Commit:* ${{ github.sha }}\n*Author:* ${{ github.actor }}\n*Branch:* ${{ github.ref_name }}"
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
      
      - name: Notify Slack on failure
        if: failure()
        uses: slackapi/slack-github-action@v1
        with:
          payload: |
            {
              "text": "❌ Backend deployment failed!",
              "blocks": [
                {
                  "type": "section",
                  "text": {
                    "type": "mrkdwn",
                    "text": "*Backend Deployment Failed* ❌\n\n*Commit:* ${{ github.sha }}\n*Author:* ${{ github.actor }}\n*Branch:* ${{ github.ref_name }}\n\nCheck GitHub Actions for details."
                  }
                }
              ]
            }
        env:
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Step 8: Test CI/CD Pipeline

8.1

Make a Test Commit

Make a small change to backend code (e.g., update a comment)
Commit and push to main branch
Go to GitHub → Actions tab
Watch the workflow run
Verify deployment completes successfully
Test the API to confirm new version is live

✅ Success!
If the workflow completes and your API responds, your CI/CD pipeline is working! You now have automated deployments.

8.2

Monitor Deployment

bash

# Watch ECS service events
aws ecs describe-services \
  --cluster helium-production-cluster \
  --services helium-backend-service \
  --query 'services[0].events[0:5]' \
  --region us-east-1

# Check running tasks
aws ecs list-tasks \
  --cluster helium-production-cluster \
  --service-name helium-backend-service \
  --desired-status RUNNING \
  --region us-east-1

# View recent logs
aws logs tail /ecs/helium-backend --follow

CI/CD Best Practices

Branch Strategy

• Use main for production
• Use develop for staging
• Feature branches for development
• Require PR reviews

Testing Strategy

• Run tests on every PR
• Block merge if tests fail
• Include linting and type checking
• Test builds before deployment

Security Strategy

• Scan dependencies weekly
• Scan container images
• Never commit secrets
• Use GitHub secret scanning

Phase 6 Verification Checklist

GitHub secrets configured (AWS + Cloudflare)
Backend deployment workflow created
Frontend deployment workflow created
Test workflow created
Security scanning workflow created
Rollback procedure documented
Automated rollback configured
Slack notifications configured (optional)
Test commit deployed successfully
All workflows passing
Deployment takes <15 minutes
Health checks passing after deployment

🎉 Congratulations!

You've successfully completed all 6 phases of the AWS deployment! Your application is now running in production with:

Secure VPC networking with multi-AZ deployment
Auto-scaling backend on ECS Fargate with Graviton3
Global CDN with Cloudflare Pages
High-performance Redis 7.2 caching
Automated CI/CD with GitHub Actions
Comprehensive monitoring and alerting

What's Next?

Phase 7: Infrastructure as Code

Convert your manual setup to Terraform or CloudFormation for reproducible deployments.

Phase 8: Advanced Monitoring

Add Datadog, New Relic, or Sentry for deeper insights and error tracking.

Phase 9: Disaster Recovery

Implement backup strategies, multi-region failover, and disaster recovery procedures.

Phase 10: Performance Optimization

Fine-tune caching strategies, optimize database queries, and implement CDN best practices.