Analytics Infrastructure

A guide to how Lantern's analytics system is architected — from the shared event registry through client and server SDKs to Firestore and BigQuery.

---

Overview

Lantern uses a monoconfig pattern: all event definitions live in one shared package consumed by every layer of the stack. This means a single source of truth drives event validation on the client, the API, and the admin UI simultaneously.

@lantern/shared/analytics   ← Single source of truth
        ↓                          ↓
 Flash SDK (browser)         Forge SDK (backends)
        ↓                          ↓
 Analytics Cloud Run       Direct write (no HTTP hop)
 (proxy + auth gate)               ↓
        ↓               BigQuery  (default)
     Forge SDK          Firestore + BigQuery  (destination: 'realtime')
        ↓
 Firestore + BigQuery

The analytics Cloud Run is a proxy for browser events only. Browsers can't authenticate with BigQuery or Firestore using service accounts, so Flash events are relayed through the analytics API, which uses Forge internally to write them. Backend services (Cloud Run APIs, Cloud Functions) use Forge directly — no HTTP hop needed.

SDK Roles

SDK	Where	Writes to
Flash	Browser	→ Analytics Cloud Run → Forge → Firestore + BigQuery
Forge	Backend services	→ BigQuery (default), or Firestore + BigQuery with realtime

---

The Monoconfig — @lantern/shared/analytics

File: packages/shared/analytics/index.js

This is the only place where events are defined. All other code imports from here.

What it contains

Export	Purpose
EVENT_REGISTRY	All 24+ built-in events (auto + registered) with full metadata
DYNAMIC_EVENT_REGISTRY	In-memory registry for custom events loaded from Firestore at runtime
ENTITY_TYPES	Valid entity type strings (venue, offer, lantern, wave, chat, feature)
PARAMETER_TEMPLATES	Reusable parameter definitions (entityId, entityType, metadata, reason, etc.)
EVENT_NAMING_RULES	Naming constraints (snake_case, max 64 chars)
validateEventName()	Gate that rejects unregistered or non-active events
getEventDefinition()	Lookup by name — checks dynamic registry first, then built-ins
registerRemoteEvent()	Loads a Firestore event definition into DYNAMIC_EVENT_REGISTRY
defineEvent()	Strictly-validated API for registering new custom events

Event tiers

• auto — Fired automatically by Flash (session_start, page_view). No manual call needed.
• registered — Pre-defined events with a specific intent (lantern_lit, wave_sent, etc.). Call flash.track() or forge.track() explicitly.

Event status lifecycle

pending → submitted → in_progress → installed → active → archived
                                                    ↑
                         Only these two are trackable

Events not in active or installed status are rejected at validation time. Manage lifecycle in the admin Registered Events UI.

---

Client SDK — Flash

File: apps/web/src/lib/flash.js

The browser-side analytics SDK. Imports the shared taxonomy and adds:

• Batching — Up to 10 events buffered, flushed every 30 seconds or on page hide
• Auto events — session_start and page_view fire automatically on lifecycle hooks
• Auth attachment — Firebase auth token attached; userId resolved server-side
• Client-side sanitization — PII stripped before events leave the browser
• Custom event sync — Loads Firestore-registered custom events at startup

Usage

import { flash } from '@/lib/flash'

// Simple
flash.track('lantern_lit')

// With entity context
flash.track('lantern_lit', {
  entityId: '<venue_id>',
  entityType: 'venue',
})

// With metadata
flash.track('offer_claimed', {
  entityId: '<offer_id>',
  entityType: 'offer',
  metadata: {
    discount_pct: 20,
    merchant_id: '<merchant_id>',
  },
})

Only events in VALID_EVENT_NAMES (built-ins) or DYNAMIC_EVENT_REGISTRY (custom, loaded from Firestore) are accepted. Unregistered events are rejected.

---

Server SDK — Forge

Package: packages/forge/ (@lantern/forge)

Used by Cloud Run APIs and Cloud Functions to emit events from the backend.

Pipeline

forge.track(payload)
  → validate(eventName, userId/serviceId, entityType)
  → checkRateLimit(userId)          // 30 events/min per user
  → sanitizeMetadata(metadata)       // strip PII, enforce limits
  → writeToBigQuery()                // always
  → writeToFirestore()               // only if destination: 'realtime'

Usage

import { forge } from '@lantern/forge'

// System/service event — BigQuery only (default)
await forge.track({
  serviceId: 'venues-api',
  eventName: 'venue_searched',
  entityType: 'venue',
  metadata: { result_count: 12, city: 'Austin' },
})

// User event with real-time presence — Firestore + BigQuery
await forge.track({
  userId: '<firebase_uid>',
  eventName: 'lantern_lit',
  entityId: '<venue_id>',
  entityType: 'venue',
  metadata: { source: 'map' },
}, { destination: 'realtime' })

Destination options

Option	Behaviour
'analytics' (default)	BigQuery only. For server-to-server, system, or high-frequency events.
'realtime'	Firestore + BigQuery. For user-facing events that need live dashboards.

Rate limiting

• 30 events per user per minute (sliding window)
• Per-Cloud-Run-instance only (in-memory). For multi-instance distributed limiting, Redis/Firestore would be needed.
• Service events (serviceId) are not rate-limited.

Metadata sanitization

• Max 50 keys, max 500 chars per string value
• Only primitives allowed (string, number, boolean) — no nested objects
• Forbidden keys stripped automatically: email, phone, name, address, password, token, secret, apikey, credential, session, and ~30 more

---

Data Routing

When creating a new event in the admin, choose the Data Destination:

Option	Behaviour
All events	Written to BigQuery, the single event store. Realtime dashboards do not read events at all: they read the live operational collections (lanterns, waves), which is why the old per-event "realtime" mirror could be removed.

History (#616): Forge used to dual-write every client event into a Firestore analytics_events mirror (destination: 'realtime'). Tracing in July 2026 showed the mirror's only reader had been broken for months (it queried a field the docs never carried) and nothing else consumed it, so the mirror, the destination option, and the expiresAt TTL that went with it were removed. If you need per-event data, query BigQuery.

---

BigQuery Storage

Dataset: analytics → Table: events

• Partitioned by timestamp (daily) with require_partition_filter: every query must filter on timestamp (or DATE(timestamp)), so a dashboard or console query can never accidentally full-scan the table
• Clustered by (event_name, user_id) (#617)
• Expiration: 90 days per partition
• Schema: See tooling/schemas/bigquery-events.json. Two time fields by design (#613): timestamp is the trusted server receive-time; client_event_time is the client's own whole-second stamp (sanity-windowed at ingest, ordering context only). session_id is the Flash per-visit UUID: grouping context, not identity (sessionStorage lifetime, rotated on logout).
• Views: recent_user_events (last 7d, userId not null), recent_system_events (last 7d, serviceId not null)

Setup

Run once per Firebase project (dev + prod):

./tooling/scripts/setup-bigquery.sh

Requires the bq CLI and appropriate GCP credentials.

---

Admin UI

The admin Analytics section lives at /analytics/ and provides:

Route	Component	Purpose
/analytics/overview	AnalyticsOverview	This documentation
/analytics/events/taxonomy	EventTaxonomy	Browse all events (built-in + custom)
/analytics/events/create	EventCreator	Register and manage custom events
/analytics/events/info	EventTrackingInfo	Event tracking reference

Creating a new event

1. Go to Registered Events → + New Event
2. Fill in Basics (name, category, description, trigger, source, entity types)
3. Add Parameters if the event carries data
4. Set Data Destination (Realtime or Analytics Only)
5. In Ticket/Task, submit to GitHub to create an implementation issue
6. Develop the tracking call, mark as Installed, then Active
7. Only active and installed events are accepted by the validation gate

Mirroring built-in events to Firestore

To allow editing built-in events from the admin UI:

node tooling/scripts/sync-events-to-firestore.mjs --dry-run   # preview
node tooling/scripts/sync-events-to-firestore.mjs             # write (skip existing)
node tooling/scripts/sync-events-to-firestore.mjs --force     # overwrite all

Built-in events in Firestore are marked built-in in the taxonomy table. Edits made in the admin take precedence over the code defaults at runtime.

---

API Reference

The analytics Cloud Run service exposes a REST API. The OpenAPI spec is served at /openapi.json on each deployment; interactive docs are rendered by the Lantern admin portal under API Reference → Analytics:

• Dev spec: https://analytics-api-dev.ourlantern.app/openapi.json
• Prod spec: https://analytics-api.ourlantern.app/openapi.json

Key endpoints:

Method	Path	Description
POST	/analytics/track	Track a single event
POST	/analytics/batch	Track up to 10 events
GET	/analytics/events	List registered custom events
GET	/openapi.json	OpenAPI spec

---

Related Docs

• Event Tracking Reference — how flash.track() works, auto events, parameters
• Event Taxonomy — browse all registered events
• Registered Events — create and manage custom events