Add Terraform-based AWS ECS deployment option

[shunskkkk]

Problem

While Marquez provides deployment solutions like Docker Compose and Helm charts for Kubernetes, there has been consistent demand in the Slack community for AWS ECS deployment support. For organizations that don't require the full complexity of EKS/Kubernetes, ECS offers a simpler and more cost-effective container orchestration solution. However, setting up ECS infrastructure manually requires configuring multiple AWS services, which can be complex and time-consuming.

Solution

This PR adds a complete AWS ECS deployment solution using Terraform to automate the infrastructure setup for organizations that prefer ECS over EKS for its simplicity and lower operational overhead. The solution includes:

• Terraform modules for AWS ECS Fargate, RDS PostgreSQL, Application Load Balancer, and CloudFront CDN
• ECS task definitions and service configurations for both API and Web components
• Deployment automation scripts for building and deploying Docker images
• Comprehensive documentation with step-by-step deployment instructions
• Multi-environment support with example configurations for sandbox and production

The deployment creates a production-ready Marquez setup with:

• Containerized services on ECS Fargate (serverless)
• Managed PostgreSQL database with RDS
• HTTPS support via CloudFront CDN
• Auto-scaling capabilities
• VPC endpoints for secure AWS service access

Important Note: Docker images must be pushed to ECR after Terraform creates the repositories but before ECS services can start. This is clearly documented in the README.

One-line summary: Add AWS ECS deployment configuration with Terraform for production-ready infrastructure

Checklist

• You've signed-off your work
• Your changes are accompanied by tests (N/A - infrastructure code)
• Your change contains a small diff and is self-contained
• You've updated any relevant documentation (comprehensive README included)
• You've included a one-line summary of your change for the CHANGELOG.md
• You've versioned your .sql database schema migration according to Flyway's naming convention (N/A - no database schema changes)
• You've included a header in any source code files (added to shell scripts)

[netlify]

❌ Deploy Preview for peppy-sprite-186812 failed.

Name	Link
🔨 Latest commit	14e7ec7
🔍 Latest deploy log	https://app.netlify.com/projects/peppy-sprite-186812/deploys/68ccff0cba47100008fde6c9

[boring-cyborg]

Thanks for opening your first pull request in the Marquez project! Please check out our contributing guidelines (https://github.com/MarquezProject/marquez/blob/main/CONTRIBUTING.md).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.18%. Comparing base (a89b89c) to head (19e45b5).

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #3074   +/-   ##
=========================================
  Coverage     81.18%   81.18%           
  Complexity     1506     1506           
=========================================
  Files           268      268           
  Lines          7356     7356           
  Branches        325      325           
=========================================
  Hits           5972     5972           
  Misses         1226     1226           
  Partials        158      158           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[thijs-s]

Hey @shunskkkk this looks awesome!

If you don't mind we will add it here https://github.com/ilum-cloud/marquez

[shunskkkk]

Hey @shunskkkk this looks awesome!

If you don't mind we will add it here https://github.com/ilum-cloud/marquez

@thijs-s
Apologies for the late reply.
I'm totally fine with you incorporating the commit!
I'm waiting for review feedback on my PR, so if I need to make any changes based on that, I'd appreciate it if you could pull in those updates too.

[thijs-s]

merged! ilum-cloud#1

thanks @shunskkkk !