SMS Delay Investigation — Phone Verification During Admin Migration

Date: 2026-03-22

Summary

Investigated a noticeably slow SMS verification delivery during admin account creation via the phone migration flow. The SMS eventually arrived and everything worked, but the delay was suspicious.

Investigation

What Was Checked

1. Cloud Run logs for auth-api — All API responses were fast:

   • /auth/phone/check-admin — 0.22s
   • /auth/phone/lookup — 0.05s
   • /auth/phone/token — 0.17s

2. Cold start analysis — No cold start during this session. Instance was already warm from a deployment rollout. Historical data shows cold starts on /phone/lookup can add 2-3s (observed on 03/19), but this was not a factor today.

3. Flow path — No /auth/admin/provider-link/start requests were logged, confirming the PhonePinSignup flow was used (not the invite-based AdminSignup flow).

Conclusion

The delay occurred in Firebase's SMS delivery layer, after the client calls signInWithPhoneNumber. Our backend had no role in the delay. Possible causes:

• Carrier-side SMS queuing — varies by carrier and time of day
• Invisible reCAPTCHA risk scoring — Google's risk engine can silently add time before approving the SMS send
• Firebase project-level SMS throttling — dev project may have lower quotas

Action Items

• If it recurs: Check Firebase Console → Authentication → Usage for SMS delivery metrics
• If cold starts become an issue separately: Consider setting a minimum instance count on auth-api in Cloud Run
• No code changes needed — the issue is external to our stack