Analytics Infrastructure

A guide to how Lantern's analytics system is architected — from the shared event registry through client and server SDKs to Firestore and BigQuery.

Overview

Lantern uses a monoconfig pattern: all event definitions live in one shared package consumed by every layer of the stack. This means a single source of truth drives event validation on the client, the API, and the admin UI simultaneously.

@lantern/shared/analytics   ← Single source of truth
        ↓                          ↓
 Flash SDK (browser)         Forge SDK (backends)
        ↓                          ↓
 Analytics Cloud Run       Direct write (no HTTP hop)
 (proxy + auth gate)               ↓
        ↓               BigQuery  (default)
     Forge SDK          Firestore + BigQuery  (destination: 'realtime')
        ↓
 Firestore + BigQuery

The analytics Cloud Run is a proxy for browser events only. Browsers can't authenticate with BigQuery or Firestore using service accounts, so Flash events are relayed through the analytics API, which uses Forge internally to write them. Backend services (Cloud Run APIs, Cloud Functions) use Forge directly — no HTTP hop needed.

SDK Roles

SDK	Where	Writes to
Flash	Browser	→ Analytics Cloud Run → Forge → Firestore + BigQuery
Forge	Backend services	→ BigQuery (default), or Firestore + BigQuery with `realtime`

The Monoconfig — `@lantern/shared/analytics`

File: packages/shared/analytics/index.js

This is the only place where events are defined. All other code imports from here.

What it contains

Export	Purpose
`EVENT_REGISTRY`	All 24+ built-in events (auto + registered) with full metadata
`DYNAMIC_EVENT_REGISTRY`	In-memory registry for custom events loaded from Firestore at runtime
`ENTITY_TYPES`	Valid entity type strings (`venue`, `offer`, `lantern`, `wave`, `chat`, `feature`)
`PARAMETER_TEMPLATES`	Reusable parameter definitions (`entityId`, `entityType`, `metadata`, `reason`, etc.)
`EVENT_NAMING_RULES`	Naming constraints (snake_case, max 64 chars)
`validateEventName()`	Gate that rejects unregistered or non-active events
`getEventDefinition()`	Lookup by name — checks dynamic registry first, then built-ins
`registerRemoteEvent()`	Loads a Firestore event definition into `DYNAMIC_EVENT_REGISTRY`
`defineEvent()`	Strictly-validated API for registering new custom events

Event tiers

auto — Fired automatically by Flash (session_start, page_view). No manual call needed.
registered — Pre-defined events with a specific intent (lantern_lit, wave_sent, etc.). Call flash.track() or forge.track() explicitly.

Event status lifecycle

pending → submitted → in_progress → installed → active → archived
                                                    ↑
                         Only these two are trackable

Events not in active or installed status are rejected at validation time. Manage lifecycle in the admin Registered Events UI.

Client SDK — Flash

File: apps/web/src/lib/flash.js

The browser-side analytics SDK. Imports the shared taxonomy and adds:

Batching — Up to 10 events buffered, flushed every 30 seconds or on page hide
Auto events — session_start and page_view fire automatically on lifecycle hooks
Auth attachment — Firebase auth token attached; userId resolved server-side
Client-side sanitization — PII stripped before events leave the browser
Custom event sync — Loads Firestore-registered custom events at startup

Usage

import { flash } from '@/lib/flash'

// Simple
flash.track('lantern_lit')

// With entity context
flash.track('lantern_lit', {
  entityId: '<venue_id>',
  entityType: 'venue',
})

// With metadata
flash.track('offer_claimed', {
  entityId: '<offer_id>',
  entityType: 'offer',
  metadata: {
    discount_pct: 20,
    merchant_id: '<merchant_id>',
  },
})

Only events in VALID_EVENT_NAMES (built-ins) or DYNAMIC_EVENT_REGISTRY (custom, loaded from Firestore) are accepted. Unregistered events are rejected.

Server SDK — Forge

Package: packages/forge/ (@lantern/forge)

Used by Cloud Run APIs and Cloud Functions to emit events from the backend.

Pipeline

forge.track(payload)
  → validate(eventName, userId/serviceId, entityType)
  → checkRateLimit(userId)          // 30 events/min per user
  → sanitizeMetadata(metadata)       // strip PII, enforce limits
  → writeToBigQuery()                // always
  → writeToFirestore()               // only if destination: 'realtime'

Usage

import { forge } from '@lantern/forge'

// System/service event — BigQuery only (default)
await forge.track({
  serviceId: 'venues-api',
  eventName: 'venue_searched',
  entityType: 'venue',
  metadata: { result_count: 12, city: 'Austin' },
})

// User event with real-time presence — Firestore + BigQuery
await forge.track({
  userId: '<firebase_uid>',
  eventName: 'lantern_lit',
  entityId: '<venue_id>',
  entityType: 'venue',
  metadata: { source: 'map' },
}, { destination: 'realtime' })

Destination options

Option	Behaviour
`'analytics'` (default)	BigQuery only. For server-to-server, system, or high-frequency events.
`'realtime'`	Firestore + BigQuery. For user-facing events that need live dashboards.

Rate limiting

30 events per user per minute (sliding window)
Per-Cloud-Run-instance only (in-memory). For multi-instance distributed limiting, Redis/Firestore would be needed.
Service events (serviceId) are not rate-limited.

Metadata sanitization

Max 50 keys, max 500 chars per string value
Only primitives allowed (string, number, boolean) — no nested objects
Forbidden keys stripped automatically: email, phone, name, address, password, token, secret, apikey, credential, session, and ~30 more

Data Routing

When creating a new event in the admin, choose the Data Destination:

Option	Behaviour
Realtime	Writes to Firestore → automatically forwarded to BigQuery. Best for user-facing events where you need live dashboards.
Analytics Only	Writes directly to BigQuery, skips Firestore. Best for server-to-server or high-frequency events that don't need real-time presence.

The data destination setting maps directly to Forge's destination option:

Realtime events → { destination: 'realtime' }
Analytics-only events → default ({ destination: 'analytics' })

Firestore Storage

Collection: analytics_events

Field	Type	Notes
`eventName`	string	From taxonomy
`eventTier`	string	`auto` or `registered`
`userId`	string?	Firebase UID
`serviceId`	string?	Backend service identifier
`entityId`	string?	Related entity ID
`entityType`	string?	`venue`, `offer`, `lantern`, etc.
`metadata`	map?	Sanitized event-specific data
`environment`	string	`production` or `development`
`createdAt`	timestamp	Server timestamp
`expiresAt`	timestamp	90-day TTL (Firestore TTL policy)

BigQuery Storage

Dataset: analytics → Table: events

Partitioned by timestamp (daily)
Expiration: 90 days per partition
Schema: See tooling/schemas/bigquery-events.json
Views: recent_user_events (last 7d, userId not null), recent_system_events (last 7d, serviceId not null)

Setup

Run once per Firebase project (dev + prod):

bash

./tooling/scripts/setup-bigquery.sh

Requires the bq CLI and appropriate GCP credentials.

Admin UI

The admin Analytics section lives at /analytics/ and provides:

Route	Component	Purpose
`/analytics/overview`	`AnalyticsOverview`	This documentation
`/analytics/events/taxonomy`	`EventTaxonomy`	Browse all events (built-in + custom)
`/analytics/events/create`	`EventCreator`	Register and manage custom events
`/analytics/events/info`	`EventTrackingInfo`	Event tracking reference

Creating a new event

Go to Registered Events → + New Event
Fill in Basics (name, category, description, trigger, source, entity types)
Add Parameters if the event carries data
Set Data Destination (Realtime or Analytics Only)
In Ticket/Task, submit to GitHub to create an implementation issue
Develop the tracking call, mark as Installed, then Active
Only active and installed events are accepted by the validation gate

Mirroring built-in events to Firestore

To allow editing built-in events from the admin UI:

bash

node tooling/scripts/sync-events-to-firestore.mjs --dry-run   # preview
node tooling/scripts/sync-events-to-firestore.mjs             # write (skip existing)
node tooling/scripts/sync-events-to-firestore.mjs --force     # overwrite all

Built-in events in Firestore are marked built-in in the taxonomy table. Edits made in the admin take precedence over the code defaults at runtime.

API Reference

The analytics Cloud Run service exposes a REST API. The OpenAPI spec is served at /openapi.json on each deployment; interactive docs are rendered by the Lantern admin portal under API Reference → Analytics:

Dev spec: https://analytics-api-dev.ourlantern.app/openapi.json
Prod spec: https://analytics-api.ourlantern.app/openapi.json

Key endpoints:

Method	Path	Description
`POST`	`/analytics/track`	Track a single event
`POST`	`/analytics/batch`	Track up to 10 events
`GET`	`/analytics/events`	List registered custom events
`GET`	`/openapi.json`	OpenAPI spec

Event Tracking Reference — how flash.track() works, auto events, parameters
Event Taxonomy — browse all registered events
Registered Events — create and manage custom events

Analytics Infrastructure ​

Overview ​

SDK Roles ​

The Monoconfig — @lantern/shared/analytics ​

What it contains ​

Event tiers ​

Event status lifecycle ​

Client SDK — Flash ​

Usage ​

Server SDK — Forge ​

Pipeline ​

Usage ​

Destination options ​

Rate limiting ​

Metadata sanitization ​

Data Routing ​

Firestore Storage ​

BigQuery Storage ​

Setup ​

Admin UI ​

Creating a new event ​

Mirroring built-in events to Firestore ​

API Reference ​

Related Docs ​

Analytics Infrastructure

Overview

SDK Roles

The Monoconfig — `@lantern/shared/analytics`

What it contains

Event tiers

Event status lifecycle

Client SDK — Flash

Usage

Server SDK — Forge

Pipeline

Usage

Destination options

Rate limiting

Metadata sanitization

Data Routing

Firestore Storage

BigQuery Storage

Setup

Admin UI

Creating a new event

Mirroring built-in events to Firestore

API Reference

Related Docs