Planning โ
Ordered. An item that is not sequenced yet belongs in backlog.md, not here.
What gets settled before any code? โ
Which collections and fields the console may read. This is a privacy decision, not an implementation detail. โ
- The Assistant's
FIRESTORE_FIELD_ALLOWLISTinservices/api/assistant/src/services/tools.jsis the working precedent: five collections (venues,offers,merchantProfiles,featureRequests,featureFlags), each pinned to named non-identifying fields. - Its comment records WHY it allowlists fields rather than collections: spreading whole documents meant adding a collection also silently exposed every field added to it later, and
merchantProfilescarried the admin password hash until#875. - An admin console is a wider audience than the Assistant, so the answer may legitimately differ. It should not be decided by whoever writes the endpoint.
- This one goes to the operator before the endpoint is built.
#874exists because surfaces keep inventing their own answer here.
What is the order? โ
1. Promote Query Console to its own Analytics line item. โ
- The nav lives in
apps/admin/src/admin/AdminShell.jsxaround line 271:Analyticscurrently expands toDashboards,Event Tracking,BigQuery. AddQuery Consoleas a sibling. - Remove the
consoleentry fromBIGQUERY_TABSinapps/admin/src/admin/analytics/BigQueryTabs.jsx, leaving Export Status, Scheduled Reports, and Data Retention. - Keep
/admin/analytics/bigquery/consoleworking as a redirect to the new route. Saved links, docs, and the operator's own bookmarks point at it. - Nothing about the console's internals changes in this slice. It is a move, so it should be reviewable as one.
2. Give the console a source tab, BigQuery beside Firestore. โ
PageTabsper the admin UI conventions. BigQuery stays the default so the existing muscle memory is unbroken.- The rail (schema browser, history, saved queries) is shared, and each panel becomes source-aware rather than duplicated.
3. Build the Firestore query endpoint on analytics-api. โ
- Structured body, never a query string:
{ collection, filters: [{ field, op, value }], orderBy, limit }. - Server applies the allowlist settled above, and refuses an unlisted collection or field outright rather than filtering the response.
- Surface Firestore's composite-index error readably. A compound filter on an unindexed pair fails with a long message containing a create-index URL. That URL is the fix, so the console should show it as a link rather than bury it in a stack trace.
- This is a new endpoint, so it is Cloud Run and it follows the
cloud-serviceskill. It extends the existing analytics-api rather than adding a service.
4. Build the Firestore tab UI. โ
- Collection picker and operator picker are
StyledSelect, never a nativeselect. ESLint enforces it. - Results render through the existing
QueryResultTableso both sources look like one product. - Saved queries and history currently store a SQL string. They need a source discriminator plus a structured payload, which touches
useSavedQueries,useConsoleHistory, andsavedQueries.service.js.
5. Ground the Assistant so it stops writing fake SQL. (Done 2026-08-27.) โ
- Built as
get_bq_schemainservices/api/assistant/src/services/tools.js, reading BigQuery metadata through the client@lantern/forgealready owns, so no new dependency. It is NOT a wrapper around the analytics-api service, which lives in a different workspace. - Scope decision made during the build: the tool reads the real schema rather than being handed a list, because a list is exactly what failed on 2026-08-27. It is scoped to the
analyticsdataset alone, default-denied structurally (it never enumerates), and returns metadata only. Detail and reasoning in implementation.md. - Extend the never-invent guardrail in the system prompt to cover BigQuery datasets, tables, and columns. Today it names file paths, function names, Firestore field names, and policy details, and stops there.
- State the real project and dataset names in the prompt, and state the
analytics.eventspartition-filter requirement. - Route state questions to
query_firestore. The failure on 2026-08-24 was not only a wrong table, it was reaching for SQL when the right tool was already loaded.
6. Say what the console is for, on the console. (Done 2026-08-27.) โ
- Added after the operator used the working console and asked "how would someone know how to even query firestore?" Her example, how many lanterns are lit at a venue, is answerable through
venues.activeLanternCount, and nothing on the surface led anyone there. - Every allowlisted field carries a plain-language line saying what it answers, served through
describeAllowlistso the client needs no new endpoint. - A Firestore rail, reusing
.bq-console-rail. Firestore had none, so switching sources lost a whole column. It holdsWhat you can askandSaved. - The refused collections travel to the client with a reason each, so a privacy decision stops reading as a missing feature.
- Saved inquiries store a STRUCTURED payload plus a
sourcediscriminator, which is the precondition the report-builder row in backlog.md named. - A reference doc at
docs/engineering/analytics/query-console.md, linked first in the tab strip the wayBigQueryTabsandEventTrackingTabsalready do it.
What is out of scope? โ
- Streaming Firestore into BigQuery. Considered and declined on 2026-08-24 for the second-copy privacy cost. If it is ever revisited it is its own project with its own privacy review.
- Joins across collections, and any write path. The console is read only, and Firestore cannot join. Both are properties of the datastore, not gaps to close.
- The report builder. Real direction, not this task. It is in backlog.md.