Skip to content

Planning โ€‹

Ordered. An item that is not sequenced yet belongs in backlog.md, not here.

What gets settled before any code? โ€‹

Which collections and fields the console may read. This is a privacy decision, not an implementation detail. โ€‹

  • The Assistant's FIRESTORE_FIELD_ALLOWLIST in services/api/assistant/src/services/tools.js is the working precedent: five collections (venues, offers, merchantProfiles, featureRequests, featureFlags), each pinned to named non-identifying fields.
  • Its comment records WHY it allowlists fields rather than collections: spreading whole documents meant adding a collection also silently exposed every field added to it later, and merchantProfiles carried the admin password hash until #875.
  • An admin console is a wider audience than the Assistant, so the answer may legitimately differ. It should not be decided by whoever writes the endpoint.
  • This one goes to the operator before the endpoint is built. #874 exists because surfaces keep inventing their own answer here.

What is the order? โ€‹

1. Promote Query Console to its own Analytics line item. โ€‹

  • The nav lives in apps/admin/src/admin/AdminShell.jsx around line 271: Analytics currently expands to Dashboards, Event Tracking, BigQuery. Add Query Console as a sibling.
  • Remove the console entry from BIGQUERY_TABS in apps/admin/src/admin/analytics/BigQueryTabs.jsx, leaving Export Status, Scheduled Reports, and Data Retention.
  • Keep /admin/analytics/bigquery/console working as a redirect to the new route. Saved links, docs, and the operator's own bookmarks point at it.
  • Nothing about the console's internals changes in this slice. It is a move, so it should be reviewable as one.

2. Give the console a source tab, BigQuery beside Firestore. โ€‹

  • PageTabs per the admin UI conventions. BigQuery stays the default so the existing muscle memory is unbroken.
  • The rail (schema browser, history, saved queries) is shared, and each panel becomes source-aware rather than duplicated.

3. Build the Firestore query endpoint on analytics-api. โ€‹

  • Structured body, never a query string: { collection, filters: [{ field, op, value }], orderBy, limit }.
  • Server applies the allowlist settled above, and refuses an unlisted collection or field outright rather than filtering the response.
  • Surface Firestore's composite-index error readably. A compound filter on an unindexed pair fails with a long message containing a create-index URL. That URL is the fix, so the console should show it as a link rather than bury it in a stack trace.
  • This is a new endpoint, so it is Cloud Run and it follows the cloud-service skill. It extends the existing analytics-api rather than adding a service.

4. Build the Firestore tab UI. โ€‹

  • Collection picker and operator picker are StyledSelect, never a native select. ESLint enforces it.
  • Results render through the existing QueryResultTable so both sources look like one product.
  • Saved queries and history currently store a SQL string. They need a source discriminator plus a structured payload, which touches useSavedQueries, useConsoleHistory, and savedQueries.service.js.

5. Ground the Assistant so it stops writing fake SQL. (Done 2026-08-27.) โ€‹

  • Built as get_bq_schema in services/api/assistant/src/services/tools.js, reading BigQuery metadata through the client @lantern/forge already owns, so no new dependency. It is NOT a wrapper around the analytics-api service, which lives in a different workspace.
  • Scope decision made during the build: the tool reads the real schema rather than being handed a list, because a list is exactly what failed on 2026-08-27. It is scoped to the analytics dataset alone, default-denied structurally (it never enumerates), and returns metadata only. Detail and reasoning in implementation.md.
  • Extend the never-invent guardrail in the system prompt to cover BigQuery datasets, tables, and columns. Today it names file paths, function names, Firestore field names, and policy details, and stops there.
  • State the real project and dataset names in the prompt, and state the analytics.events partition-filter requirement.
  • Route state questions to query_firestore. The failure on 2026-08-24 was not only a wrong table, it was reaching for SQL when the right tool was already loaded.

6. Say what the console is for, on the console. (Done 2026-08-27.) โ€‹

  • Added after the operator used the working console and asked "how would someone know how to even query firestore?" Her example, how many lanterns are lit at a venue, is answerable through venues.activeLanternCount, and nothing on the surface led anyone there.
  • Every allowlisted field carries a plain-language line saying what it answers, served through describeAllowlist so the client needs no new endpoint.
  • A Firestore rail, reusing .bq-console-rail. Firestore had none, so switching sources lost a whole column. It holds What you can ask and Saved.
  • The refused collections travel to the client with a reason each, so a privacy decision stops reading as a missing feature.
  • Saved inquiries store a STRUCTURED payload plus a source discriminator, which is the precondition the report-builder row in backlog.md named.
  • A reference doc at docs/engineering/analytics/query-console.md, linked first in the tab strip the way BigQueryTabs and EventTrackingTabs already do it.

What is out of scope? โ€‹

  • Streaming Firestore into BigQuery. Considered and declined on 2026-08-24 for the second-copy privacy cost. If it is ever revisited it is its own project with its own privacy review.
  • Joins across collections, and any write path. The console is read only, and Firestore cannot join. Both are properties of the datastore, not gaps to close.
  • The report builder. Real direction, not this task. It is in backlog.md.

Built with VitePress