Prompt-to-app BaaS: generating full-stack stores on AWS, then cutting the bill by 76%

Challenge

The product generates a complete e-commerce app from a prompt: REST APIs, a database schema and a frontend, deployed for the customer. The first version worked, but it ran as a monolith on oversized instances, generation jobs blocked request threads, and the AWS bill grew faster than revenue. Observability was console.log.

Approach

Make generation a workflow, not a request

Generation is long-running and multi-step (plan → schema → API → UI → deploy). I moved it into Step Functions orchestrating Lambda steps, with state persisted in DynamoDB. A failed step retries alone; a slow LLM call no longer holds an HTTP connection hostage.

Containers for the steady state, functions for the bursts

The always-on API moved to ECS services sized for real traffic; bursty work (generation, media processing) runs on Lambda. Valkey replaced an over-provisioned cache tier. Right-sizing plus removing idle capacity is where most of the savings came from; the rest came from paying per-execution for work that used to keep instances warm.

Zero-downtime migration

Traffic shifted gradually behind the load balancer with health checks and instant rollback; data moved with dual writes and a verified cut-over. Nobody noticed, which was the goal. I used the same decomposition principles as in the pricing engine work.

Observability and on-call readiness

Structured JSON logs, CloudWatch alarms on error rates, latency and queue depth, and dashboards per service. Mean-time-to-detect dropped from "a customer emailed" to an alarm, and root-cause resolution sped up because logs finally carried request ids. This experience is why I later built the lgtm-sidecar: a resource-capped observability stack for small teams.

Architecture

What shipped

• A generation workflow with per-step retries and visible state.
• Event-driven, containerised services replacing the monolith.
• Alarms and dashboards that made on-call a defined role.

Results

• 76% lower monthly cloud spend with the same or better latency.
• Zero downtime during the migration.
• Faster detection and resolution of production incidents.

What I'd do differently

• Put infrastructure under Terraform from the start; the migration was done with a mix of console and scripts, and I later rebuilt the same patterns properly in infra-ops: one Terraform + Ansible platform for dev, staging and prod.
• Add a cost dashboard next to the latency dashboard; spend is an SLO too.