OTP Costs into Billing: Layer 2 Ceiling, Admin Visibility, Cap Alerting (#699)

Status: DRAFT, operator review pending. Written 2026-07-30. Issue: #699Refs: SEALED_IDENTITY.md section 11.6 (two-layer cost bracket), PLATFORMS.md, services/api/auth/src/lib/otpMetrics.js (Layer 1, shipped).

Problem

Layer 1 (the graceful app-level monthly send cap, metrics_otpSends/{YYYY-MM}, OTP_MONTHLY_CAP=2000) is live on dev with Prelude active (#702). Three gaps remain before OTP spend is fully bracketed and observable:

1. No Layer 2 hard ceiling is configured or documented (section 11.6 calls it mandatory before flipping the provider on any env; dev is already flipped, so this is overdue).
2. OTP volume and estimated spend are invisible outside a GCP/Firestore console visit.
3. The graceful refusal at 100% of the cap would arrive as a surprise; nothing alerts on approach.

Key facts driving the design

• Prelude bills success-only (~EUR 0.032 / ~$0.035 per successful verification; SEALED_IDENTITY 11.6 pricing table). Sends are the CAP driver (SMS-pumping risk), but successes are the SPEND driver. Today only sends are counted, so estimated spend from sends alone would overestimate, badly so under a pumping attack (which is exactly when the dashboard matters). We add a success counter.
• metrics_otpSends is rules-denied to all clients (Firestore catch-all deny), so the admin portal cannot read it directly; it needs a server surface.
• AGENTS.md rule 9 (API first): new HTTP endpoints go on Cloud Run services, and Cloud Functions remain legitimate for scheduled triggers. That splits the work naturally: the admin read surface goes on auth-api (co-located with otpMetrics.js); the daily alert check is an onSchedule Cloud Function (reuses the existing DISCORD_WEBHOOK_URL secret + embed pattern from featureRequests.js; no new scheduler-auth wiring or deploy-workflow steps needed).

Design

A. Counter enrichment (auth-api, otpMetrics.js)

The monthly doc metrics_otpSends/{YYYY-MM} gains two fields:

• cap: persisted by reserveOtpSend() at each reservation (it already computes getMonthlyCap()). This lets the alert Function compare count vs cap WITHOUT duplicating the OTP_MONTHLY_CAP env var into Functions config (env drift between two services was the alternative, rejected).
• succeededCount: incremented by a new recordOtpVerifySuccess(), called from verifyOtpHandler after the provider confirms the code. Plain FieldValue.increment (no cap check, so no transaction needed). Success-only billing means this is the billable-event count.

Alert bookkeeping fields (alert80SentAt) are written by the alert Function (D below) on the same doc, making the alert idempotent per month with zero extra storage.

B. Admin read surface (auth-api route)

GET /auth/admin/ops/otp on the existing adminDispatch (verifyFirebaseToken + requireAdmin), returning the last 6 month-buckets:

{
  "unitPriceUsd": 0.035,
  "months": [
    { "bucket": "2026-07", "sends": 412, "cap": 2000, "succeeded": 361, "estSpendUsd": 12.64, "alert80SentAt": null }
  ]
}

estSpendUsd = succeeded * unitPriceUsd, computed server-side so the unit price lives in exactly one place (OTP_UNIT_PRICE_USD env with a 0.035 default; documented in PLATFORMS.md). Route added to auth's openapi.json (auth is warn-only in openapi-sync, but new routes get documented on arrival).

C. Admin UI (System Health, Costs tab)

A section on the existing Costs tab of SystemHealth.jsx (the natural home; it is the "what are we spending" surface): sends vs cap for the current month (with a capacity meter), successful verifications, estimated spend, and the trailing months as a small table. Reuses SystemSection + RoleMetric primitives and adds METRIC_DEFINITIONS tooltip entries. Fetches via the existing auth-api admin client (this read does NOT go through the getSystemHealth callable: rule 9 says do not grow Cloud Function surface for new reads).

D. Approach-the-cap alerting (Cloud Function, onSchedule)

A daily scheduled Function (otpCapAlert, ~09:00 UTC) reads the current month's doc:

• count / cap >= 0.8 and alert80SentAt unset: post a Discord embed (reusing DISCORD_WEBHOOK_URL + the featureRequests.js embed/fetch pattern) with count, cap, succeeded, est. spend, and days left in the month; then stamp alert80SentAt.
• Missing webhook secret: log and skip (same degradation as feature requests).
• 100% needs no separate alert: Layer 1 already refuses gracefully, and the 80% alert names the refusal date risk.

Daily cadence is enough: with cap 2000, even a burst month crosses 80% along a curve where a same-day heads-up changes nothing operationally (the cap itself is the guard; the alert is for humans planning the month).

E. Layer 2 hard ceiling (config + documentation)

Two backstops, documented in PLATFORMS.md (new Prelude section) so the flip checklist can point at them:

1. Prelude-side: Prelude is prepaid/pay-as-you-go with account credits; the balance itself is the structural ceiling (spend cannot exceed loaded credits). Document the topped-up balance policy (keep it at roughly one month's expected spend) plus any low-balance notification the console offers. Console-side verification is an operator step; the doc carries a checklist.
2. GCP-side: a billing budget + alert on the project's billing account covering total spend (catches any provider-adjacent cost path and the general bill). Also operator-console (billing-account IAM), one-time, checklist in the doc.

Neither backstop is codeable from this repo (both live in third-party consoles under the operator's accounts), so the deliverable here is the runbook + the checklist entries, per the issue ("configure and document"; the configure half is a handoff).

Non-goals

Unchanged from the issue: no user-facing billing, no Layer 1 mechanics changes. Also out: prod provisioning (operator raises prod when ready), historical backfill of succeededCount (starts counting at deploy; earlier months show sends only).

Decision log

• D1: read surface on auth-api, not the getSystemHealth callable. Rule 9 (API first); keeps cap logic co-located with otpMetrics.js. The callable remains untouched.
• D2: alerting via Functions onSchedule, not a Cloud-Scheduler-hit API route. Scheduled tasks are an allowed Functions use (rule 9), the Discord secret + embed pattern already lives there, and the alternative costs scheduler-auth middleware plus deploy-workflow ensure_job steps in two workflows for one daily read.
• D3: persist cap on the monthly doc so the Function never needs the OTP_MONTHLY_CAP env (no cross-service env drift).
• D4: count successes, price successes. Honest two-driver model: sends bound risk (cap), successes bound spend (Prelude success-only billing).
• D5: alert once per month per threshold, stamped on the doc (alert80SentAt), so restarts/retries cannot re-ping.