Order processing is the canonical distributed-systems problem dressed up as a boring CRUD app. Charge the card, reserve the stock, confirm the order — and any one of those three can fail independently, at any point, including in the middle. Handle that with a straight-line function and you'll either oversell stock under load or leave orders in a permanently ambiguous state when a payment provider has a bad afternoon.
gcp-serverless-orders-platform is an event-driven order platform with no servers to run: a JWT-protected API Gateway in front of Cloud Run, orders accepted with an immediate 202, and the actual work — reserve stock, charge, confirm — run by a Cloud Workflows saga that retries transient failures and compensates business ones. It deployed live in europe-west1 through a two-phase Terraform build, and its 49-check test suite passed 49 of 49, including a concurrency test that races 120 simultaneous orders against 60 units of stock.
Async by design, not by accident
The API returns 202 Accepted immediately and the client polls for status — there's no synchronous chain where the caller's request thread is blocked waiting on a payment provider's latency. A mandatory Idempotency-Key header means a retried request (network blip, client retry logic, whatever) resolves to the same order instead of creating a second one. Every order id is validated as a UUID before it touches anything — junk input or path-traversal-shaped ids get a 404 immediately, never reaching Firestore and never returning a 500 that would leak information about internal structure.
The saga: retries for transient failure, compensation for business failure
This is the design decision the whole repo is built around, and it's worth being precise about the distinction it draws. A transient failure — a payment provider returning a 503 because it's momentarily overloaded — gets retried with backoff. A business failure — the card is declined, or the item is out of stock — is not a bug to retry around; it's a valid outcome that needs to compensate: release the stock reservation, mark the order FAILED with a reason, and leave the workflow execution itself recorded as SUCCEEDED, because the saga did exactly what it was supposed to do.
PASS out-of-stock declined → compensate → order FAILED (reason), execution SUCCEEDED
PASS flaky payment (503 x2) → retried with backoff → order CONFIRMED, 2 attempts logged
PASS stock race 40 vs 25 units → 25 confirmed / 15 failed / 0 oversold
PASS stress: 120 vs 60 units → 60 confirmed / 60 failed / 0 oversold
That last line is the one I'd point to first if I only had one number to prove this works: 120 orders racing for 60 units of stock, and the result is exactly 60 confirmed, 60 failed, zero oversold. No partial reservation, no double-booked unit. That's Firestore transactions doing the actual work — the saga orchestrates the business logic, but the no-overselling guarantee comes from transactional, per-order stock reservations underneath it.
Only a known client gets in
The Gateway verifies a JWT signed specifically by the sop-client service account — checking issuer, signing keys, and audience — rather than accepting any Google-issued ID token. That distinction matters: a bare ID-token check would let any authenticated Google identity through, which is a much wider trust boundary than "this specific client, using this specific key." Nothing internal is public: /internal/* routes don't even appear in the API spec the Gateway enforces, and each internal service has its own dedicated invoker identity rather than sharing one broad service account.
Three decisions
| Decision |
|---|
| Cloud Workflows orchestrates the saga explicitly; business failures are handled outcomes, not exceptions |
| The Gateway trusts JWTs signed by one specific service account, not any Google ID token |
| Idempotent order creation, transactional stock reservation, idempotent payment calls |
A bug a from-scratch build actually found
Even with the saga design above, the very first from-scratch run of this repo surfaced a real bug: under contention, the saga could crash rather than cleanly compensate. It's documented with root cause and fix in runbook 05. The general lesson that came out of this build (and shows up across the other repos in this series too) is that a from-scratch run finds bugs that incremental fixes on an already-running stack simply hide — if you only ever patch forward, you never re-test the path that got you into the broken state in the first place.
What's honestly missing
There's no end-user authentication layer (Identity Platform) — this proves service-to-service trust, not consumer login. No per-client rate quotas, which would need API keys layered on top of the JWT check. No WAF, since that needs a load balancer in front of the Gateway. And the one gap I'd flag as most important to understand: there's no transactional outbox. A failed publish after an order is created gets undone cleanly (503, retry with the same idempotency key) — but a process crash between creating the order and publishing the event would still leave a PENDING order that needs manual re-driving, which is exactly what runbook 05 covers. It's single-region, and payments/inventory are simulated rather than calling a real provider.
Try it yourself
git clone https://github.com/soodrajesh/gcp-serverless-orders-platform
cd gcp-serverless-orders-platform
gcloud config set project <your-project>
./scripts/up.sh # infra → image build → 3 services + saga + trigger + gateway → seed → live tests (~20 min, mostly API Gateway)
./scripts/down.sh # delete everything, including the Firestore databaseWell under €1 for a full build-test-teardown cycle — Cloud Run scales to zero, and a Terraform-managed budget bounds the total. The API Gateway provisioning step is genuinely the slow part of standing this up; everything else is fast by comparison.
📢 Have questions or feedback? Drop a comment below or connect with me on Twitter/X@spysood!