rahb3rt/platform-architecture in production since 2023 view on github ↗

Robert Davis — Staff Site Reliability Engineer

One engineer. Fifteen services. Production since 2023.

This platform runs a real business — customers, payments, field crews, vehicles. I designed it, built every service, and operate it alone: CI/CD, observability, SLO tracking, backups, and on-call included. The application code is proprietary; the engineering is documented here.

15 services 16,000+ jobs scheduled 5,100+ invoices processed 58,000+ telemetry readings 1 operator

Service topology

Grouped the way the system actually fails: edge, applications, async workers, platform, and hardware in the field.

edge
nginx
TLS termination and routing for every environment
nginx:alpine · generated config
applications
core api
Business logic — 60 domains, 520 endpoints; the single source of truth
Python · Flask · MySQL
operations dashboard
Internal app: scheduling, invoicing, customers
Next.js
public website
Customer-facing site
Next.js
kiosk
On-site self-service
TypeScript
customer portal
Balance, invoices, next visit, service requests — magic-link sign-in, no passwords
Next.js 15 · passwordless JWT
async & integration
sms
Two-way customer texting
Python
email ingest & send
Inbound parsing and outbound delivery
Node
transactional mail
Templated customer notifications
HTML templating
payment reconciliation
Matches external payments to invoices
Python
document extraction
Inbound documents to structured data
Python
platform
health aggregator
Auto-discovers containers, probes each on a loop
FastAPI · Docker SDK · HTTP/MySQL/TCP
monitoring & SLOs
Container stats, logs, alerting, SLO tracking
Next.js · RBAC
mysql
Primary datastore, per-tenant instances
MySQL 8.0
minio
S3-compatible object storage
MinIO
field hardware
vehicle telemetry
OBD-II + GPS from trucks; store-and-forward over LTE
C++ · ESP32 · Freematics

Data flow

Green is the request path, blue is data and telemetry, dashed gray is observability, and the amber boundary is one tenant stack — everything inside it is replicated per tenant.

ISOLATED TENANT STACK · ×N own network · database · objects · secrets users web · kiosk · phone field vehicles esp32 · obd-ii · gps nginx tls · routing public website next.js operations app next.js kiosk typescript customer portal magic-link · next.js core api flask · 60 domains · 520 endpoints · source of truth invoicing · payments · statements · quotes · contracts + e-sign jobs · scheduling · route optimization · crews · timeclock · payroll messaging · marketing · telemetry ingest · insights · rbac · webhooks ASYNC WORKERS sms email ingest transactional mail payment recon doc extraction mysql per-tenant minio s3 objects health + monitoring auto-discovery · slo tracking store-and-forward gzip over lte http · mysql · tcp probes — every service, every environment
core api surface — 60 route domains · 520 endpoints
billinginvoicingpaymentspayment methodsstatementsquotescustomer creditexpenses
contractstemplatese-signingaudit trail
crmcustomerspropertiesleadsrequestssites
workforceemployeesteamstimeclocktimecardstime-offpayroll
operationsjobsschedulingcalendarroute optimizationmowingplowing plansservice plans
commsmessagingnotificationsmarketing campaignssocial
field & assetsvehicle telemetryassetsaccess devicesaccess eventsweather
platformauthrbac rolesorgsdashboardsreportsinsightswebhook deliveries

Delivery

Every service ships the same way. Pull requests build without publishing; version tags cut releases.

triggergit push / tag v*.*.*
buildGitHub Actions · Buildx + QEMU
amd64 / arm64
publishghcr.io
deploy./deploy.sh --env <name>

The deploy step is one idempotent script: it pulls service repos, pre-builds the Next.js apps, generates the nginx config, and brings the stack up under Docker or Podman. Published (genericized) as compose-multienv-deploy.

Environments

One host, multiple fully isolated stacks — each environment gets its own env file, network, containers, and volumes. Nothing is shared.

./deploy.sh --env production

The live platform.

./deploy.sh --env staging

Production-shaped: same compose file, same generated nginx config — drift surfaces here first.

./deploy.sh --env customer-acme

A dedicated tenant stack: own database, object storage, domain routing, and secrets.

in progress

Multi-tenancy as a product. The per-customer isolation model is growing a control plane — tenant signup, an admin dashboard, and customer self-service — turning a single-business platform into multi-tenant SaaS, one isolated stack per tenant with hourly backups.

Reliability engineering

The tooling is built in-house, sized to the platform, and runs as part of it.

Health aggregation

A dedicated service discovers containers through the Docker/Podman API and probes each on a background loop. No manual registration, no stale check configs.

probes: http · mysql · tcp

Monitoring & SLOs

Purpose-built dashboard: container metrics, log aggregation, alerting, and SLO tracking against explicit targets, behind database-backed RBAC.

slo targets · error budgets

Backups

Every tenant stack takes hourly backups of its database and object storage. Restore plus redeploy is the whole disaster-recovery story — by design.

hourly · per-tenant

Design-for-failure at the edge

Vehicle firmware assumes the network is lying: every reading lands on SD first, then uploads as gzip batches with retry and backoff. Late data is fine; lost data is not.

store-and-forward · ndjson · lte

A failure, on the record

Reliability claims are cheap. Here is a real incident from this platform's history — reconstructed from the commit log — hashes included.

postmortem · june 2026 · monitoring

The observer effect

The monitoring dashboard gained a live topology view — container stats, sparklines, request traces — on top of its real-user-monitoring ingest. The monitoring database lives on the same MySQL server as production. Within hours the topology view was timing out, and every one of those timeouts represented load pressure on the database that also serves the business.

The first fix made it worse. Parallelizing all DB queries and trace fetches is the intuitive move when something is slow — and it multiplied the concurrent load on an already-stressed database. Reverted the same day. The durable fix went the opposite direction: batch the RUM inserts, reduce query limits, poll slower, fetch fewer traces, and put explicit timeouts and error containment on every query so the dashboard degrades instead of hammering.

The lesson: monitoring is production. The observer carries the same load budget as the observed — and parallelism is not a fix for overload, it is a multiplier on it.

jun 08ff77443Fix topology timeout: batch RUM inserts into chunks of 25
jun 08730466eFix topology timeout: parallelize all DB queries and trace fetches
jun 08d4ce6c3Revert "Fix topology timeout: parallelize all DB queries and trace fetches"
jun 089036fc7Fix topology timeout: reduce query limits safely
jun 08b743295Fix topology timeouts: slower polling, fewer trace fetches
jun 094aa70e5Fix topology loading: reduce limits, add timeouts, catch errors

Security posture

Sized to the platform, enforced by structure rather than policy documents.

Isolation by default

Every tenant stack gets its own network, database, object storage, and secrets. There is no shared state to leak across — isolation is the architecture, not a setting.

per-tenant network · db · objects

Secrets out of band

Credentials live in per-environment env files injected at deploy time — never in images, never in git. Each environment holds only its own.

per-env injection · nothing in images

TLS at the edge

Every stack fronts through its own edge proxy with TLS termination; certificates are provisioned and renewed automatically.

auto-provisioned certs

AuthN/AuthZ

Database-backed RBAC with role-to-permission mapping, session management, and a token blacklist for immediate revocation.

rbac · sessions · token revocation

Passwordless customers

The customer portal has no passwords to steal: sign-in is a 15-minute magic link over SMS. The request endpoint answers identically for any input and is rate-limited, so it can't be used to probe who is a customer. Sessions are stateless JWTs — rotating one secret revokes everything.

magic links · enumeration-proof · rate-limited

Design decisions

The choices interviewers ask about, answered up front.

Compose, not Kubernetes?

I operate multi-tenant Kubernetes at day-job scale — which is exactly why this platform doesn't use it. On a single host, Kubernetes buys autoscaling and rolling deploys this system doesn't need, at the price of a control plane to patch and a much larger failure surface to debug alone. The revisit trigger is explicit: a second host, or a genuine need for zero-downtime rollouts.

Build, not buy, for monitoring?

Hosted observability is priced per-container and per-GB of ingest — for a single-host platform that bill would rival the entire infrastructure budget. The requirements were narrow and Docker-native, and building purpose-fit kept everything on one pane, data on-host, and SLO tracking implemented the way I practice it professionally.

Store-and-forward telemetry?

Field vehicles are the harshest environment in the system: LTE dead zones, uploads dying mid-flight, power cut at ignition-off. The SD card is the source of truth; an uploader drains it whenever connectivity allows. For telemetry, durability beats latency.

Per-tenant stacks on one host?

Staging shaped exactly like production catches configuration drift before customers do, and the same isolation model makes a dedicated customer instance a one-command operation instead of a re-architecture. Isolation is the product's oldest feature, not a bolt-on.