Lantern

Appearance

Query Console: promotion + Firestore querying - Manual Test Plan ​

FieldValue
Branch / PRfeat/query-console-958, PR #961
Issue#958: feat(admin/analytics): promote Query Console to its own surface and teach it to query Firestore
EnvironmentLocal stack on NON-DEFAULT ports (branch-only endpoint): admin :3011 -> analytics-api :8092, both against live lantern-app-dev Firebase
Build flags / confignone. No feature flag gates this work.
Build (commit)Run 1: c4825c8e plus the stale-results fix that came out of it. Run 2: 219c284b
TesterClaude (headless Playwright chromium, plus two Node probes for the API-level scenarios)
DateRun 1: 2026-08-24. Run 2: 2026-08-27

Which run is this? ​

Two runs, both recorded here. Run 1 is the original build (2026-08-24, scenarios 1 to 16). Run 2 is the bug-fix, design-verification and grounding work (2026-08-27, scenarios 17 to 38). Each scenario carries its own filled result block; a blank block would read as a pass to anyone skimming, so anything not exercised says so in its own words.

RunDateBuildWhat it covered
12026-08-24c4825c8eThe nav move, the redirect, the Firestore endpoint and its allowlist refusals
22026-08-27219c284b#970 and #971, four more bugs, the discoverability build, the allowlist corrections, and slice 5 grounding the Assistant

Summary ​

#ScenarioResultNotes
1Query Console is its own Analytics nav item[x] pass [ ] fail [ ] blocked
2BigQuery tabs no longer carry Query Console[x] pass [ ] fail [ ] blocked
3Old /bigquery/console link redirects, does not 404[x] pass [ ] fail [ ] blocked
4Regression: the three remaining BigQuery tabs still load[x] pass [ ] fail [ ] blocked
5Activation: BigQuery is the default source with NO manual selection[x] pass [ ] fail [ ] blocked
6Regression: BigQuery SQL still runs and renders through ConsoleResults[x] pass [ ] fail [ ] blocked
7Firestore happy path: venues, no filters, renders in ConsoleResults[x] pass [ ] fail [ ] blocked
8Firestore ordering on a newly approved field (offers.createdAt)[x] pass [ ] fail [ ] blocked
9Guardrail: an unlisted collection is refused outright[x] pass [ ] fail [ ] blocked
10Guardrail: an unlisted field is refused outright[x] pass [ ] fail [ ] blocked
11FALSIFYING: unindexed compound query surfaces a clickable create-index link[x] pass [ ] fail [ ] blockedfield pair substituted, see below
12CONTROL: a single-field query on the same collection succeeds with no index prompt[x] pass [ ] fail [ ] blocked
13Pickers are StyledSelect, no native <select> in the DOM[x] pass [ ] fail [ ] blocked
14Limit ceiling is enforced server side[x] pass [ ] fail [ ] blocked
15Payload verification: only allowlisted fields leave the server[x] pass [ ] fail [ ] blocked
16Regression: a non-admin cannot reach the endpoint[ ] pass [ ] fail [x] blockedhalf exercised, see below

14 of 16 fully exercised and passing. One (11) passed on a substituted field pair. One (16) is half exercised: the unauthenticated arm passed, the non-admin arm could not be driven from here.

Setup / preconditions ​

  • Start analytics-api on a non-default port so it does not collide with a parallel session: GOOGLE_APPLICATION_CREDENTIALS=<real ADC path> APP_CHECK_DISABLED=true PORT=8092 node --env-file=.env.local services/api/analytics/src/index.js
  • Start the admin portal on a non-default port pointed at it. The client reads VITE_ANALYTICS_API_URL=/api and vite proxies /api/analytics to ANALYTICS_API_ORIGIN, so overriding VITE_ANALYTICS_API_URL does nothing. Override the ORIGIN: ANALYTICS_API_ORIGIN=http://localhost:8092 AUTH_API_ORIGIN=https://auth-api-531553779372.us-central1.run.app npx vite --port 3011 --strictPort
  • A fresh worktree needs npm install AND node tooling/scripts/generate-client-sdk-docs.mjs. Without the second, apps/admin/src/generated/client-sdk.json is missing and AdminShell.jsx fails to load with a misleading "Failed to fetch dynamically imported module".
  • Sign in with the Agent Probe admin (credentials in the 0600 env file, never inline them).
  • Data lives in live lantern-app-dev Firestore. Everything under test is READ ONLY: there is no write path in this change, so no scenario here mutated a document.
  • The four newly approved allowlist fields are offers.createdAt, offers.updatedAt, venues.createdAt, venues.updatedAt. No other field was added.
  • Clean state: clear localStorage for the admin origin before scenario 5, since console layout preferences persist there.

Which scenario can tell us we are wrong? ​

Scenario 11 versus scenario 12. They make opposite predictions on the same collection.

  • 11 predicts a compound filter on an unindexed field pair FAILS, and that the failure carries a create-index URL we render as a link.
  • 12 predicts a single-field query on that same collection SUCCEEDS with no index prompt at all.

If 12 also demands an index, the diagnosis is wrong: the problem is not "compound queries need a composite index", it is something broader about how the endpoint builds queries, and the index-link rendering is treating a general failure as a specific one. If 11 succeeds instead, the index-link path has never been exercised and scenario 11's pass means nothing.

What actually happened: 11's originally-chosen pair (offers.status + offers.createdAt) came back 200 with 6 rows, because that pair is already indexed in dev. Per the plan's own instruction, another pair was found and named: venues.city + venues.createdAt, which is not indexed. That pair failed exactly as predicted, with the URL. 12 succeeded as predicted. The pair holds.

Scenarios ​

1. Query Console is its own Analytics nav item ​

  • Goal: the console is reachable as a sibling of Dashboards, Event Tracking and BigQuery, not buried in a tab.
  • Steps:
    1. Load http://localhost:3011/admin.
    2. Expand Analytics in the left nav.
    3. Read the sub-items.
    4. Click Query Console.
  • Expected: Analytics lists Dashboards, Event Tracking, BigQuery, Query Console. Clicking Query Console navigates to /admin/analytics/query-console and the console renders.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Sub-items read "Dashboards | Event Tracking | BigQuery | Query Console".
           Clicking Query Console landed on /admin/analytics/query-console with the
           toolbar, rail, editor and results all rendered, and the nav row active.
Evidence:  shots/01-query-console.png; probe checks 1 and 1.1
Follow-up: none

2. BigQuery tabs no longer carry Query Console ​

  • Goal: the move is a move, not a copy.
  • Steps:
    1. Navigate to /admin/analytics/bigquery/export.
    2. Read the tab strip.
  • Expected: exactly Reference Docs, Export Status, Scheduled Reports, Data Retention.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Tab strip read "Reference Docs | Export Status | Scheduled Reports |
           Data Retention". No Query Console tab anywhere on the page.
Evidence:  probe check 2
Follow-up: none

3. Old /bigquery/console link redirects, does not 404 ​

  • Goal: saved links and bookmarks keep working.
  • Steps:
    1. Load http://localhost:3011/admin/analytics/bigquery/console directly.
  • Expected: lands on /admin/analytics/query-console with the console rendered.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Final pathname /admin/analytics/query-console, console toolbar present.
           No blank page and no bounce to the admin dashboard.
Evidence:  probe check 3
Follow-up: The frozen design specs under docs/planning/ still cite the old path.
           They were deliberately left alone: they are historical records, and
           the redirect keeps their links live.

4. Regression: the three remaining BigQuery tabs still load ​

  • Goal: removing one tab did not disturb its siblings.
  • Steps:
    1. Visit /admin/analytics/bigquery/export, then /reports, then /retention.
  • Expected: each renders its own panel with the correct page title.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    export=true reports=true retention=true. Each rendered its own heading
           ("BigQuery Export Status", "BigQuery Scheduled Reports", "BigQuery Data
           Retention") with the correct tab active.
Evidence:  probe check 4
Follow-up: none

5. Activation: BigQuery is the default source with NO manual selection ​

  • Goal: prove the SHIPPED default, not that the toggle works.
  • Steps:
    1. Clear site data for the admin origin.
    2. Load /admin/analytics/query-console fresh.
    3. Do not click anything.
  • Expected: BigQuery already selected, SQL editor visible, Firestore builder not shown.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    On first paint, BigQuery tab aria-selected="true", Firestore
           aria-selected="false", SQL editor present. ZERO calls to
           /analytics/admin/firestore-* had been made at that point: the Firestore
           schema fetch is gated on the source, so a console sitting on BigQuery
           issues no Firestore request at all. The unit suite asserts the same
           thing ("opens on BigQuery with no manual selection, and asks Firestore
           nothing").
Evidence:  probe check 5; apps/admin/src/admin/analytics/__tests__/QueryConsolePage.test.jsx
Follow-up: none

6. Regression: BigQuery SQL still runs and renders through ConsoleResults ​

  • Goal: the existing path is untouched by the Firestore work.
  • Steps:
    1. Run a 7-day partition-filtered analytics.events aggregate.
    2. Inspect the results grid.
  • Expected: rows render, row expands, Copy CSV works.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    10 rows returned with headers "event_name STRING" and "n INTEGER",
           rendered in the same grid as before. Row filter, Copy CSV and Download
           all present.
Evidence:  shots/06-bigquery-results.png; probe check 6
Follow-up: none

7. Firestore happy path: venues, no filters, renders in ConsoleResults ​

  • Goal: the structured builder returns real documents through the SAME results component as BigQuery.
  • Steps:
    1. Click the Firestore source tab.
    2. Pick collection venues, no filters, limit 10.
    3. Run, expand a row, click Download.
  • Expected: venue rows in the same table chrome, with filter, Copy CSV, Download and click-to-expand.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    10 venue rows rendered. Columns: id, name, city, state, category, tier,
           status, activeLanternCount, source, createdAt, updatedAt, with the type
           chips the BigQuery grid uses. Copy CSV and Download both present, and
           clicking a row opened the same detail list.
           Response body shape confirmed as { rows, schema, ... }, and the status
           strip read "10 rows from venues in <n> ms".
Evidence:  shots/08-firestore-venues.png; probe checks 7 and 7.1
Follow-up: The city, state and tier columns render EMPTY for every row. See the
           Findings section: those three fields do not exist on venue documents.

8. Firestore ordering on a newly approved field (offers.createdAt) ​

  • Goal: exercise the reason the four fields were approved: order the moderation queue oldest first.
  • Steps:
    1. Pick collection offers, order by createdAt ascending, limit 25.
    2. Run.
  • Expected: oldest first, createdAt readable rather than {_seconds, _nanoseconds}.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    9 offers returned in 235 ms, oldest first:
           2026-04-25T16:27:59.559Z | 2026-04-30T16:24:03.374Z |
           2026-04-30T17:49:44.600Z | 2026-05-04T05:43:38.706Z |
           2026-05-05T03:37:24.755Z | ...
           Timestamps render as readable ISO strings in both the grid and the
           expanded row detail. Nothing showed as _seconds/_nanoseconds.
Evidence:  shots/09-offers-ordered.png; probe check 8; the API probe returned the
           same nine values in the same order.
Follow-up: none

9. Guardrail: an unlisted collection is refused outright ​

  • Goal: a collection absent from the allowlist fails, and fails BEFORE any read.
  • Steps:
    1. POST /analytics/admin/firestore-query with { "collection": "users" }, then "lanterns".
  • Expected: 400 COLLECTION_NOT_ALLOWED, no document data.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    users     -> 400 COLLECTION_NOT_ALLOWED, no rows in the body.
           lanterns  -> 400 COLLECTION_NOT_ALLOWED, no rows in the body.
           The refusal happens in validateQuery before the Firestore call is
           built; the unit suite asserts the collection handle is never even
           touched ("refuses an unlisted collection without touching Firestore").
Evidence:  API probe, scenario 9 block; services/api/analytics/test/services/firestoreQuery.service.test.js
Follow-up: none

10. Guardrail: an unlisted field is refused outright ​

  • Goal: the allowlist is per FIELD, so an allowlisted collection does not make every field readable.
  • Steps:
    1. Filter on offers.reviewNote.
    2. Order by offers.reviewedBy.
    3. Order by merchantProfiles.updatedAt.
  • Expected: 400 FIELD_NOT_ALLOWED on all three.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    offers.reviewNote filter            -> 400 FIELD_NOT_ALLOWED
           offers.reviewedBy orderBy           -> 400 FIELD_NOT_ALLOWED
           merchantProfiles.updatedAt orderBy  -> 400 FIELD_NOT_ALLOWED
           No rows in any body. The error carries allowedFields, and the console's
           status strip renders it as "Readable here: ..." so the refusal is
           actionable rather than just a no.
Evidence:  API probe, scenario 10 block
Follow-up: none

11. FALSIFYING: unindexed compound query surfaces a clickable create-index link ​

  • Prediction, stated before the run: this FAILS, with a composite-index error, and the console renders the create-index URL as a link.
  • Steps:
    1. Collection venues, filter city == Austin, order by createdAt descending.
    2. Run.
  • Expected: an error strip naming the index requirement, with an anchor whose href is the create_composite URL.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    FIRST ATTEMPT USED A DIFFERENT PAIR AND DID NOT FALSIFY. The pair the
           plan originally named, offers.status == active + order by createdAt,
           came back 200 with 6 rows: that composite index already exists in dev,
           so the scenario could not have proved anything. Per the plan's own
           instruction, a genuinely unindexed pair was found and is named here:
           venues.city + venues.createdAt.

           With that pair: 400 FIRESTORE_INDEX_REQUIRED, message "This filter and
           ordering combination needs a composite index.", and indexUrl
           https://console.firebase.google.com/v1/r/project/lantern-app-dev/firestore/indexes?create_composite=Ck5wcm9qZWN0cy9sYW50ZXJuLWFwcC1kZXYv...

           In the UI this renders as a red-bordered strip carrying the sentence,
           the [FIRESTORE_INDEX_REQUIRED] code chip, and an amber underlined link
           "Create the index in Firebase" whose href equals the server's indexUrl
           exactly, opening in a new tab.
Evidence:  shots/10-index-required.png; API probe extra block; probe check 11
Follow-up: THIS RUN FOUND A DEFECT AND IT IS FIXED. The first pass showed the
           previous query's 9 offer rows still sitting in the results grid
           underneath the red error strip, while the collection picker said
           venues. That reads as "here is what your query returned". The Firestore
           source now clears its last result on failure, the probe asserts zero
           stale rows under an error, and a unit test covers it ("clears the
           previous results when a query fails"). The BigQuery source has the same
           behaviour and was NOT changed: see Findings.

12. CONTROL: a single-field query on the same collection succeeds with no index prompt ​

  • Prediction, stated before the run: this SUCCEEDS. A single-field ordered query needs only the automatic single-field index.
  • Steps:
    1. Collection venues, no filters, order by createdAt descending, limit 10.
    2. Run.
  • Expected: rows render, no index error.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    10 rows rendered, zero index links on the page, no error strip. The API
           probe ran the same control against offers (order by createdAt desc, no
           filter) and got 200 with 9 rows.
           The prediction held, so scenario 11's failure really is about the
           compound pair rather than about how the endpoint builds queries.
Evidence:  shots/11-control.png; probe check 12
Follow-up: none

13. Pickers are StyledSelect, no native <select> in the DOM ​

  • Goal: the project dropdown convention holds on the new surface.
  • Steps:
    1. Open the collection, field and operator pickers.
    2. Count native selects in the document.
  • Expected: react-select menus, and zero native selects.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    document.querySelectorAll('select').length = 0 with the Firestore
           builder mounted and a filter row present. All four picker types
           (collection, filter field, operator, order field and direction) render
           the dark amber-accented react-select control and open its menu.
           A component test asserts the same thing so it cannot regress silently.
Evidence:  shots/07-firestore-builder.png; probe check 13;
           apps/admin/src/admin/analytics/console/__tests__/FirestoreBuilder.test.jsx
Follow-up: none

14. Limit ceiling is enforced server side ​

  • Goal: the client cannot ask for an unbounded read.
  • Steps:
    1. POST with limit: 100000, then limit: 0.
  • Expected: 400 naming limit in both cases, not a silent clamp.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    limit 100000 -> 400 INVALID_LIMIT, no rows.
           limit 0      -> 400 INVALID_LIMIT, no rows.
           Neither was clamped and then honoured. The ceiling is 500 and it is
           reported to the client by GET /analytics/admin/firestore-schema, so the
           builder's number input carries the same max.
Evidence:  API probe, scenario 14 block
Follow-up: none

15. Payload verification: only allowlisted fields leave the server ​

  • Goal: the thing the field allowlist exists for.
  • Steps:
    1. Read each collection RAW via the Admin SDK and collect every field name present.
    2. Read the same collection through the endpoint.
    3. Diff the two key sets.
  • Expected: only id plus allowlisted fields survive.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Zero leaks in all three collections checked.

           venues (25 docs sampled, 37 distinct fields on the documents)
             response: activeLanternCount, category, createdAt, id, name, source, status
             correctly dropped (31): address, addressCity, addressComponents,
             addressCountry, addressPostcode, addressState, addressStreet, capacity,
             cuisine, email, geohash, googleMapsUrl, lastEnrichedAt, lat, lng,
             nameLower, openingHours, osmId, osmTags, osmType, outdoorSeating,
             phoneNumber, searchTerms, takeaway, userReportedClosed,
             userReportedClosedAt, userReportedClosedBy, website,
             wheelchairAccessible, wifi, yelpUrl

           offers (9 docs, 21 distinct fields)
             response: createdAt, expiresAt, id, merchantId, status, title, updatedAt, venueId
             correctly dropped (14): budget, createdBy, description, heroLayout,
             liveEvent, moderation, per_user_limit, placement, radius, reviewedAt,
             reviewedBy, showDisclaimerWhileSuppliesLast, targetAudience,
             useGradientFallback

           merchantProfiles (2 docs, 16 distinct fields)
             response: createdAt, id, status
             correctly dropped (13): businessName, contactName, createdBy,
             displayName, failedLoginAttempts, lastFailedLoginAt,
             lastMerchantLoginAt, merchantPasswordHash, merchantPasswordSalt,
             notes, phone, updatedAt, updatedBy

           The merchantProfiles line is the #875 reasoning proved live: those
           documents carry merchantPasswordHash and merchantPasswordSalt TODAY,
           and both were dropped. offers carries reviewedAt and reviewedBy today,
           and both were dropped.
Evidence:  the leak probe's per-collection diff, reproduced above verbatim
Follow-up: none

16. Regression: a non-admin cannot reach the endpoint ​

  • Goal: the new route inherits the same admin gate as the rest of /analytics/admin.
  • Steps:
    1. POST with no Authorization header.
    2. Repeat with a valid non-admin user token.
  • Expected: 401 then 403, no rows in either.
Result:    [ ] pass   [ ] fail   [x] blocked  (half exercised)
Actual:    Arm 1 PASSED: no Authorization header -> 401, no rows.
           Arm 2 NOT EXERCISED: no non-admin ID token could be minted from this
           machine. The app's non-admin users are phone-auth accounts whose
           sign-in mints a custom token server side, and custom-token signing
           needs a service-account credential; this VM has user ADC, which cannot
           sign one. There is no non-admin email/password account to sign in as.
What we know instead, and its limit:
           The route is mounted inside the SAME express chain as every other
           /analytics/admin route: app.use('/analytics/admin', verifyAppCheck,
           verifyFirebaseToken, requireRole('admin'), adminRoutes) in
           services/api/analytics/src/index.js, unchanged by this PR. So a
           non-admin is refused by middleware that predates this work and is
           shared with a dozen live endpoints. That is a strong argument, not a
           measurement, and it is recorded here as an argument.
Follow-up: Exercise arm 2 on deployed dev in the post-merge pass, signing in as an
           ordinary app user, OR add a non-admin fixture to the emulator suite.

Run 2 summary (2026-08-27) ​

#ScenarioResultNotes
17#970: the document key is withheld on merchantProfiles[x] pass [ ] fail [ ] blocked
18#970: the key is still shipped where it names a row[x] pass [ ] fail [ ] blocked
19#970: projection stops at depth one[x] pass [ ] fail [ ] blockedunit only, see block
20#970: the audit log carries filter fields, never values[x] pass [ ] fail [ ] blocked
21#971: a recovered schema fetch clears the error[x] pass [ ] fail [ ] blocked
22#971: a held shortcut fires ONE query[x] pass [ ] fail [ ] blockedthe headline number
23#971: clearing the Limit box does not write a 0[x] pass [ ] fail [ ] blocked
24The filter row's four controls share one height[x] pass [ ] fail [ ] blocked
25A non-JSON error body is not rendered as HTML[x] pass [ ] fail [ ] blocked
26Accessibility: focus survives removing a filter[x] pass [ ] fail [ ] blocked
27Accessibility: the status strip is a stable live region[x] pass [ ] fail [ ] blocked
28The rail opens collapsed and behaves like the BQ schema browser[x] pass [ ] fail [ ] blocked
29Every starter question fills the builder AND returns rows[x] pass [ ] fail [ ] blockedone dropped, see block
30Saving a Firestore inquiry works end to end[ ] pass [ ] fail [x] blockedpartially exercised
31Reference Docs sits first in the tab strip[x] pass [ ] fail [ ] blockedlink target un-exercised
32featureFlags is gone and featureRequests names real fields[x] pass [ ] fail [ ] blocked
33The five dead venue and offer field names are corrected[x] pass [ ] fail [ ] blocked
34The Assistant stops sending Auth UIDs to the Claude API[x] pass [ ] fail [ ] blockedunit only, see block
35get_bq_schema returns real names, scoped to analytics[x] pass [ ] fail [ ] blockedrun against real BigQuery
36The narrowed repeat guard leaves held undo working[ ] pass [ ] fail [x] blockedNOT EXERCISED
37The Assistant actually uses the schema tool in a real turn[ ] pass [ ] fail [x] blockedNOT EXERCISED
38Anything at all on deployed dev[ ] pass [ ] fail [x] blockedNOT EXERCISED, cannot be

19 of 22 exercised and passing. One (30) is partially exercised. Three (36, 37, 38) were NOT exercised at all and are described below rather than left blank.

Run 2 scenarios ​

17. #970: the document key is withheld on merchantProfiles ​

  • Prediction, stated before the run: rows come back with NO id, and no id column in the schema.
  • Steps: Firestore source, collection merchantProfiles, no filters, Run. Read the raw response.
  • Expected: every row is {status, createdAt} and the schema has no id entry.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    2 rows, keys exactly ["status","createdAt"]. Schema column names
           ["status","createdAt"], no id.
           BEFORE the fix these documents shipped their key, and the key is a
           28-character Firebase Auth UID: confirmed by sampling real documents
           (id length 28, matching /^[A-Za-z0-9]{28}$/, where venues, offers and
           featureRequests are 20-character auto-ids). At limit 500 that is up to
           500 staff UIDs per request, under a two-field allowlist that had been
           reviewed and approved.
Evidence:  browser run 2026-08-27, 13/13 checks; route test "withholds the
           document key on a collection keyed by auth UID"
Follow-up: none on the console. The SAME defect existed in the Assistant and is
           scenario 34.

18. #970: the key is still shipped where it names a row ​

  • Prediction, stated before the run: offers still carries id, because withholding every key would break the moderation workflow.
  • Steps: collection offers, Run, read the raw response.
  • Expected: rows carry id, and id is the first schema column.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    9 rows, first key set
           ["id","title","status","merchantId","venueId","expiresAt","createdAt","updatedAt"].
           Schema starts with {name:"id", type:"STRING"}.
           This is the control for 17: it shows the fix is a per-collection
           RULING rather than a blanket removal.
Evidence:  browser run 2026-08-27; service test "is present when the policy allows it"
Follow-up: none

19. #970: projection stops at depth one ​

  • Prediction, stated before the run: a field holding a map is summarized by key COUNT, never by key names.
  • Steps: project a document whose allowlisted source field is a map.
  • Expected: the cell reads <map, N keys> and no nested key name appears anywhere in the response.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    UNIT ONLY, and the reason is stated rather than hidden: no allowlisted
           field holds a map today, checked across every collection before the
           change, so there is nothing live to exercise it against. The cap
           therefore regresses nothing and cannot be demonstrated in the browser.
           Unit: a venues document with source = {provider, importedBy,
           rawPayload} projects to the string "<map, 3 keys>", and the serialized
           rows contain neither "importedBy" nor the nested value.
           A second test covers the trap: a plain map carrying `path` and
           `firestore` keys is still summarized, because the DocumentReference
           check was duck-typed on that shape until 2026-08-27 and is now checked
           by class.
Evidence:  service tests "summarizes a map by key count and never by key name",
           "summarizes a map that happens to LOOK like a DocumentReference",
           "still renders a REAL DocumentReference as its path"
Follow-up: venues.source is a scalar today and the import pipeline already writes
           richer provenance maps beside it, so this becomes live-exercisable the
           moment that lands. Re-run this scenario then.

20. #970: the audit log carries filter fields, never values ​

  • Prediction, stated before the run: the log entry names the fields and operators used, and contains no filter VALUE.
  • Steps: run a query filtered on a distinctive value, inspect the emitted log entry.
  • Expected: filters: [{field, op}] present, the value absent.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Log entry carries filters [{field:"merchantId",op:"=="}] and the
           serialized entry does not contain the filter value at all.
           Before this change the entry carried only filterCount, so fifty
           queries walking one merchant's offer book were indistinguishable from
           fifty moderation checks.
Evidence:  route test "logs the filter fields and operators, and never the filter
           values"; browser run confirmed queryMeta.filters is an array
Follow-up: none

21. #971: a recovered schema fetch clears the error ​

  • Prediction, stated before the run: after a failed schema fetch, switching away and back re-fetches AND clears the stale error, with no page reload.
  • Steps:
    1. Force GET /analytics/admin/firestore-schema to answer 503.
    2. Open the Firestore tab, confirm the error renders.
    3. Remove the 503, click BigQuery, click Firestore.
  • Expected: the builder renders, with no error.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Step 2 rendered "Could not load the Firestore allowlist" as predicted.
           Step 3 recovered fully: the error was gone and #fs-limit was present,
           with no reload.
           Before the fix, setSchemaError was never reset, so one cold-start 503
           killed the Firestore source until a full page reload.
Evidence:  browser run 2026-08-27, checks "a failed schema fetch still shows an
           error" and "a recovered schema fetch clears the error without a
           reload"; shots verify-02-schema-503.png, verify-03-recovered.png
Follow-up: none

22. #971: a held shortcut fires ONE query ​

  • Prediction, stated before the run: holding Cmd/Ctrl+Enter fires at most one request, where it previously fired one per key repeat.
  • Steps: hold Ctrl+Enter for one second on the Firestore source. Count firestore-query responses at the network layer.
  • Expected: exactly 1.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    BEFORE: 65 requests on one run and 87 on another, from ONE SECOND of
           held keys, every one issued past a Run button rendered disabled the
           whole time. The filed issue estimated about 30, so the real rate was
           two to three times worse than reported.
           AFTER: 1 request.
           Measured the same way both times, by counting responses whose URL
           contains firestore-query during the hold plus a 4 second settle.
           Each of those requests was a 500-row read, so the pre-fix behaviour was
           roughly 32,500 to 43,500 documents read from one keypress.
Evidence:  browser runs 2026-08-27 (repro and verify); hook test "runs one query
           at a time, however many times the shortcut repeats"
Follow-up: The guard is client side. A second tab, or any non-browser caller with
           an admin token, still reproduces unlimited 500-row reads because
           /analytics/admin has no rate limit. Filed as
           https://github.com/cattreedev/lantern_app/issues/977, out of scope here.

23. #971: clearing the Limit box does not write a 0 ​

  • Prediction, stated before the run: selecting the Limit value and pressing Backspace leaves the field EMPTY, and blurring normalizes it to the default rather than to 0.
  • Steps: click Limit, select all, Backspace, read the value. Then blur and read again.
  • Expected: "" then "50".
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    After Backspace the field read "" (empty). After blur it read "50".
           Before the fix, Number('') is 0 so React wrote a literal "0" back into
           the box under the cursor, and running at that moment sent limit 0,
           which survived `limit ?? 50` because nullish coalescing does not catch
           zero, and earned a 400.
           Clamping also covers the two typed cases the issue named: 900 clamps
           to 500 rather than 400ing, and 50.5 floors to 50.
Evidence:  browser run 2026-08-27, checks "clearing Limit leaves it empty, not 0"
           and "Limit normalizes on blur to the default"; hook tests for
           clampLimit
Follow-up: none

24. The filter row's four controls share one height ​

  • Prediction, stated before the run: all four controls in a filter row report the same rendered height.
  • Steps: add a filter, measure getBoundingClientRect().height on both StyledSelects, the value input and the remove button.
  • Expected: one distinct value.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    [42, 42, 42, 42].
           BEFORE: 42, 42, 34, 24 in one centered flex row.
           THE TRAP, and it is why a first fix did not work: StyledSelect
           DECLARES minHeight 38px and RENDERS at 42, because react-select's
           control and value-container padding carry it past the minimum. Setting
           the shared height to the declared 38 produced [42,42,38,38], still
           wrong. The value came from measuring the running page, not from
           reading the style object.
Evidence:  browser measurement run 2026-08-27 (getComputedStyle plus
           getBoundingClientRect on every control in the chain)
Follow-up: none

25. A non-JSON error body is not rendered as HTML ​

  • Prediction, stated before the run: an API answering with an HTML error page produces a readable sentence, not a pasted document.
  • Steps: point the portal at an analytics-api lacking the endpoint, open the Firestore tab.
  • Expected: a short message naming the real cause.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    This was found by the OPERATOR, not by the plan. Her words: "Could not
           load the Firestore allowlist: <!DOCTYPE html> <html lang="en"> <head>
           <meta charset="utf-8"> <title>Error</title> </head> <body> <pre>Cannot
           GET /analytics/admin/firestore-schema</pre> </body> </html>".
           apiRequest used a non-JSON body AS the error message, so express's 404
           page was rendered into the console's error strip.
           The message is now derived from the status, and 404 says the analytics
           API is likely running an older build than the portal, which is the
           actual cause. The raw body is kept as `detail` for diagnosis.
Evidence:  operator report 2026-08-27; the underlying 404 reproduced and traced to
           ANALYTICS_API_ORIGIN pointing at deployed Cloud Run
Follow-up: The CONFIGURATION half is not a code bug. A portal proxied to an
           analytics-api that predates these endpoints will always 404; the fix
           is to point ANALYTICS_API_ORIGIN at a build that has them.

26. Accessibility: focus survives removing a filter ​

  • Prediction, stated before the run: removing a filter row moves focus to a surviving control rather than dropping it to <body>.
  • Steps: add a filter, focus its remove button, activate it, read document.activeElement.
  • Expected: focus is on Add filter.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    document.activeElement resolved to "Add filter".
           Before the fix the row unmounted with the focused button inside it, so
           focus fell to body and the next Tab restarted past the whole sidebar.
Evidence:  browser run 2026-08-27, check "removing a filter keeps focus in the
           builder"
Follow-up: none

27. Accessibility: the status strip is a stable live region ​

  • Prediction, stated before the run: the strip carries role and aria-live from FIRST render, not only once an error exists.
  • Steps: read role and aria-live on the strip before any query runs.
  • Expected: role="status", aria-live="polite".
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    {role:"status", live:"polite"} present with no query run.
           Before the fix the component toggled `role` between undefined and
           "alert" on an element already in the DOM, which is the unreliable half
           of the ARIA contract: a screen reader watches a region that was live
           when it mounted, so nothing announced a query starting, finishing or
           failing. aria-busy now carries the in-flight state.
           NOTE: this asserts the ATTRIBUTES, not that a screen reader speaks. A
           real assistive-technology pass is not something this harness can do.
Evidence:  browser run 2026-08-27; three FirestoreStatusStrip unit tests
Follow-up: A real screen-reader pass on this surface has never been done, here or
           in run 1.

28. The rail opens collapsed and behaves like the BQ schema browser ​

  • Prediction, stated before the run: one collapsed group per collection, a count badge, and a filter that force-opens matching groups.
  • Steps: open the Firestore source. Count groups and visible questions. Expand all. Filter for "bars".
  • Expected: 4 groups, 0 questions visible, then all visible, then only the matching one.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    4 groups (venues, offers, merchantProfiles, featureRequests), 0
           questions visible on load. Expand all revealed 9. Filtering "bars"
           left exactly ["Which bars are in the system?"] and disabled that
           group's collapse toggle, so a match cannot hide behind a chevron.
           This shape was a REJECTION, not a plan. The first build put a field
           reference under the operator's phrase "What you can ask" and invented
           its own interaction. Her words: "For the what you can ask, why isn't it
           expand collapse based on collection? Can you look at how BQ is built to
           please emulate that?" and "i literally meant it as what you can ask. If
           a user clicks, that auto fills."
Evidence:  browser run 2026-08-27, 4 checks; 19 FirestoreRail unit tests
Follow-up: none

29. Every starter question fills the builder AND returns rows ​

  • Prediction, stated before the run: clicking any question populates the builder, and running it returns at least one row against real dev data.
  • Steps: expand all, click each question in turn, Run, record the row count from the response.
  • Expected: every question returns rows and no question errors.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    9 questions, all 9 filled the builder, all 9 ran without error, all 9
           returned rows:
             Which venues are busiest right now?          venues            50
             What came in from the venue import backlog?  venues            50
             Which bars are in the system?                venues            50
             Which offers are live right now?             offers             6
             Which offers were rejected?                  offers             1
             What was submitted most recently?            offers             9
             Which merchant accounts are still setting up? merchantProfiles  2
             What has been requested so far?              featureRequests    3
             What is still sitting as submitted?          featureRequests    3
           A TENTH was written and then DROPPED. "Which flags are switched on?"
           returned 0 rows because featureFlags is empty, and a starter question
           that can only ever paint an empty table teaches the opposite of what it
           is for. featureFlags was subsequently removed from the allowlist
           entirely (scenario 32).
           The values in these questions were read out of dev before being
           written, and that check paid: venues.status is `open` on all 300
           sampled documents, not live/pending/archived; offers.status has no
           `pending`; and activeLanternCount > 0 currently matches ZERO venues.
           Roughly half the questions drafted from field names alone would have
           painted empty tables.
Evidence:  browser run 2026-08-27, per-question row counts; service tests
           "gives every question a structured query that names real fields" and
           "validates every starter question, so none of them can 400"
Follow-up: "What is waiting in the moderation queue?" (status == pending_review)
           was added later and returns 0 rows TODAY, because no offer is in that
           state on dev. It is kept deliberately, unlike the featureFlags one:
           pending_review is a real state that real offers pass through, so the
           question will answer as soon as one does.

30. Saving a Firestore inquiry works end to end ​

  • Prediction, stated before the run: Save is reachable on the Firestore source, the dialog previews the structured query, and a saved inquiry reloads into the builder.
  • Steps: build a query, Save, name it, reopen it from the rail's Saved panel.
  • Expected: the builder returns to the saved state.
Result:    [ ] pass   [ ] fail   [x] blocked
Actual:    PARTIALLY EXERCISED, and the un-exercised half is named here rather
           than implied.
           EXERCISED: the Save button exists on the Firestore source (it did NOT
           before: ConsoleToolbar hid it behind `isSql`, so the store, the dialog
           and the rail panel all worked and the door did not exist). The dialog
           opens. Its preview renders the STRUCTURED inquiry, verified reading
           "collection: featureRequests / where status == "submitted" / limit 50"
           rather than an empty SQL box.
           NOT EXERCISED: actually writing a saved inquiry to Firestore and
           reloading it. adminSavedQueries is a SHARED collection on live dev, and
           writing test rows into it would leave litter another session sees. Every
           other scenario in both runs is read only.
           The reload path is covered by unit tests only: the rail filters to
           firestore rows, a legacy SQL row is excluded, and loadQuery replaces
           the whole query rather than merging it.
Evidence:  browser run 2026-08-27, checks "Save is available on the Firestore
           source" and "the dialog previews the STRUCTURED inquiry"; shots
           q-04-save-dialog.png; FirestoreRail and SaveQueryDialog unit tests
Follow-up: SOMEBODY SHOULD ACTUALLY SAVE ONE. The write path to adminSavedQueries
           has never been exercised for the firestore source, on any run. It needs
           either a throwaway named row the tester then deletes, or an emulator.

31. Reference Docs sits first in the tab strip ​

  • Prediction, stated before the run: the strip reads Reference Docs, BigQuery, Firestore, and the first is an external link.
  • Steps: read every .page-tabs__tab in order, with its tag and href.
  • Expected: three tabs in that order, the first an anchor.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    ["Reference Docs","BigQuery","Firestore"], first is an <A> with href
           http://localhost:5174/engineering/analytics/query-console.
           This ordering is the operator's: "I think we need a simple reference
           doc and have that at the front of the sub nav line items so it would be
           something like ( Reference doc | BQ | FS ). Other pages have that for
           reference." It matches BigQueryTabs and EventTrackingTabs exactly.
           NOT EXERCISED: whether the link RESOLVES. It 404s right now, and the
           operator hit that: "I am getting a 404 when I click on the reference
           link". The cause is environmental rather than a defect. The docs site
           is VitePress serving the MAIN checkout, and query-console.md exists
           only on this branch, so it cannot resolve until the branch merges.
Evidence:  browser run 2026-08-27, 2 checks; operator report of the 404
Follow-up: RE-CHECK THIS LINK AFTER MERGE. It is the one scenario in run 2 whose
           real answer is only available post-merge, and a 404 on a reference link
           the console advertises would be worse than not linking it.

32. featureFlags is gone and featureRequests names real fields ​

  • Prediction, stated before the run: four readable collections, not five, and featureRequests ordering by date returns rows.
  • Steps: read the rail groups and the collection picker. Run the featureRequests question that orders by date.
  • Expected: no featureFlags anywhere, and rows with real timestamps.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    Rail groups ["venues","offers","merchantProfiles","featureRequests"].
           Picker offers 4 options, no featureFlags.
           Ordering featureRequests by submittedAt returned 3 rows carrying real
           timestamps and real issue numbers.
           BEFORE: the allowlist named `createdAt` and `issueNumber`, and this
           collection has never carried either. The writer emits `submittedAt` and
           `githubIssueNumber`. Both dead names projected to nothing, so the
           column was permanently empty and ordering by date returned NO ROWS AT
           ALL. featureFlags had zero documents and zero writers or readers
           anywhere in the repo.
           Also asserted: `submittedBy`, which holds a user id, never reaches the
           browser.
Evidence:  browser run 2026-08-27, 6 checks; census against live documents
Follow-up: none. This was found by the operator asking what the two collections
           were FOR, and answering by checking rather than by reading the list.

33. The five dead venue and offer field names are corrected ​

  • Prediction, stated before the run: the corrected names carry real values where the old names returned nothing.
  • Steps: run a venues question and an offers question, count rows carrying the corrected fields, and read the schema column names.
  • Expected: real values present, dead names absent from the schema.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    venues: 50 rows, 22 of them carrying addressCity. Schema reads
           ["id","name","addressCity","addressState","category","status",
           "activeLanternCount","source","createdAt","updatedAt"], with no city,
           state or tier.
           offers: 6 rows, 6 of them carrying targetAudience. Schema has
           targetAudience and neither audience nor startsAt.
           The census that justified each removal, against live documents:
             venues.city      0 / 300   ->  addressCity     112 / 300
             venues.state     0 / 300   ->  addressState    104 / 300
             venues.tier      0 / 300   ->  not a venue concept
             offers.audience  0 / 9     ->  targetAudience    9 / 9
             offers.startsAt  0 / 9     ->  nested in liveEvent only
           Run 1 measured the same three venue fields across the FULL collection
           and got 0 of 22,634 each, which is the better number.
Evidence:  browser run 2026-08-27, 5 checks; live census; drift test "name venue
           and offer fields the writers actually emit"
Follow-up: THIS WAS NOT A NEW FINDING. Run 1 recorded all seven dead fields on
           2026-08-24 under "Seven allowlisted fields have no data behind them at
           all", and correctly parked them as an operator decision. They then sat
           for three days because nothing carried that decision to her. The
           finding was not missed; the handoff was. Worth remembering when a
           run parks something.

34. The Assistant stops sending Auth UIDs to the Claude API ​

  • Prediction, stated before the run: query_firestore on merchantProfiles returns no document key, on both the single read and the list read.
  • Steps: exercise the tool against stubbed documents whose ids are Auth-UID shaped.
  • Expected: the UID appears nowhere in the tool output.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    UNIT ONLY, and the reason matters: exercising this live would mean
           driving a real Assistant conversation, which sends content to the
           Claude API and writes a stored conversation log. Doing that to
           demonstrate a UID leak would be reproducing the leak.
           Single read: the 28-character id is absent from the output, the status
           field is present. List read: neither of two UID-shaped ids appears, and
           the count is still 2. Control: offers still ships its key.
           SEVERITY, because it is worse than the console's version of the same
           bug: the console kept those ids in-house, this tool sends whatever it
           returns to an EXTERNAL processor and into the stored conversation log,
           25 documents at a time. Pre-existing on dev, found by the code review.
Evidence:  three tests in firestoreTool.privacy.test.js under "the document key
           goes through the same review as every field"
Follow-up: The drift test now compares the KEY decision, not only field names.
           It could not see this divergence at all before, which is the more
           useful half: a guard that checks half a decision reports green on the
           other half. Proven to fail by adding merchantProfiles to the
           assistant's expose set.

35. get_bq_schema returns real names, scoped to analytics ​

  • Prediction, stated before the run: the tool returns real tables and columns from the analytics dataset, and nothing from the other eight.
  • Steps: run the tool against REAL BigQuery. Compare its output against the full dataset list. Check for row-data keys.
  • Expected: real analytics tables only.
Result:    [x] pass   [ ] fail   [ ] blocked
Actual:    RUN AGAINST REAL BIGQUERY, not the mock.
           5 tables returned: analytics.ad_delivery_daily (13 cols),
           analytics.event_counts_daily (7), analytics.events (12),
           analytics.recent_system_events (VIEW, 10),
           analytics.recent_user_events (VIEW, 10).
           analytics.events correctly reports partitionedOn = timestamp, which is
           the whole reason a query without that filter fails with a message about
           partition elimination rather than about a missing filter.
           Eight other datasets exist (billing_attrib, billing_export_gcp,
           billing_exports_gcp, billing_marts, billing_norm, billing_raw, logs,
           ops). NONE appear anywhere in the output.
           No row-data keys. No placeholder tokens.
           Refusals read correctly and distinguish the two cases: an out-of-scope
           dataset says so as a deliberate limit, an absent table inside a
           readable dataset says no such table.
           A BUG IN THIS WORK WAS FOUND ONLY BY THIS LIVE RUN: the BigQuery client
           reports the LITERAL string {{projectId}} until it resolves one, so the
           first working version put that placeholder into the model's context.
           That is the same failure class the tool exists to prevent, since a
           placeholder reads as a name where a null reads as unknown. The mocks
           passed it happily.
Evidence:  live BigQuery run 2026-08-27; 20 tests in bqSchemaTool.test.js
Follow-up: none

36. The narrowed repeat guard leaves held undo working ​

  • Prediction, stated before the run: holding Cmd/Ctrl+Z in the SQL editor still undoes repeatedly.
  • Steps: BigQuery source, type SQL, hold the undo shortcut, observe whether the editor keeps undoing.
  • Expected: repeated undo works.
Result:    [ ] pass   [ ] fail   [x] blocked
Actual:    NOT EXERCISED. Recorded as un-exercised rather than assumed, because
           the change was made in RESPONSE to this exact risk and never tested
           against it.
           The #971 fix originally placed `if (e.repeat) { preventDefault; return }`
           above the key branches, so it cancelled EVERY held modifier combination
           on the page, undo and redo in the SQL editor included. The code review
           caught it and it is now scoped to the Enter branch.
           The reasoning is sound (the handler only owns Enter, E and S, and the
           hook's in-flight ref stops duplicate requests either way) but reasoning
           is not a run, and today produced four separate cases where the code
           said one thing and the behaviour said another.
Evidence:  none. That is the point of this block.
Follow-up: EXERCISE THIS. Hold undo in the BigQuery SQL editor and confirm it
           repeats. It is a two minute check and it closes the one change on this
           branch whose only support is an argument.

37. The Assistant actually uses the schema tool in a real turn ​

  • Prediction, stated before the run: asked a BigQuery question, the Assistant calls get_bq_schema and writes SQL using only names it returned.
  • Steps: open the admin Assistant, ask the question that failed on 2026-08-24, observe the tool call and the SQL.
  • Expected: a real table name, and a filter on timestamp for analytics.events.
Result:    [ ] pass   [ ] fail   [x] blocked
Actual:    NOT EXERCISED. The tool and the prompt are verified in isolation; the
           BEHAVIOUR they exist to change is not.
           What IS verified: the tool returns correct real data (scenario 35), the
           system prompt now names BigQuery where it previously contained the word
           zero times, and the guardrail covers dataset, table and column names
           where it previously named only file paths, function names and Firestore
           fields.
           What is NOT: that the model actually calls the tool before writing SQL,
           that it stops producing `your-project.firestore_export.offers`, and
           that it routes a no-time-range question to query_firestore instead.
           A real turn costs a Claude API call and writes a stored conversation
           log, and the outcome is model behaviour rather than a code path, so it
           is a judgement call rather than an assertion.
Evidence:  none for the behaviour. Scenario 35 for the tool.
Follow-up: ASK IT THE ORIGINAL QUESTION. The 2026-08-24 failure is the benchmark
           and it is reproducible: ask for a query about offers and see whether a
           real table name comes back. Until someone does, slice 5 is a
           well-tested mechanism with an unmeasured effect.

38. Anything at all on deployed dev ​

  • Prediction, stated before the run: none. This scenario exists to record an absence.
Result:    [ ] pass   [ ] fail   [x] blocked
Actual:    NOT EXERCISED, and it CANNOT be from here. Every endpoint in this
           branch exists only on the branch. Deployed dev runs an analytics-api
           that predates them, which is exactly why the operator's own portal
           answered 404 all morning: it proxies to deployed Cloud Run.
           Both runs are therefore local-stack only, against live lantern-app-dev
           Firestore and BigQuery for data.
Evidence:  the 404 the operator hit, traced to ANALYTICS_API_ORIGIN
Follow-up: The post-deploy pass is the close step. Re-run scenarios 17, 18, 22,
           29, 31 and 33 against deployed dev once this merges and the
           analytics-api deploys, because those six are the ones whose answers
           depend on the deployed build rather than on the code.

Findings from run 1 (2026-08-24) ​

Four things run 1 surfaced that are not pass/fail on the code under test.

A defect found and fixed: a failed query left the previous result on screen. ​

  • Seen in the first browser pass at scenario 11: nine offers rows sat in the results grid under a red index error, while the picker read venues.
  • Fixed on the Firestore source: a failed run now clears the last result. Covered by a unit test and asserted by the browser probe.
  • The BigQuery source does the same thing and was deliberately NOT changed. It is pre-existing behaviour on a surface outside this slice's scope. It is worth a follow-up so the two sources behave alike, but that is her call, not a silent edit.

Seven allowlisted fields have no data behind them at all. ​

  • A census over every document in each collection:
CollectionFieldDocuments carrying it
venuescity0 of 22,634
venuesstate0 of 22,634
venuestier0 of 22,634
offersaudience0 of 9
offersstartsAt0 of 9
featureRequestscreatedAt0 of 3
featureRequestsissueNumber0 of 3
  • All seven were inherited verbatim from the assistant's FIRESTORE_FIELD_ALLOWLIST, so the same fiction exists there: the Assistant can offer to filter venues.city and will always match nothing. The venue documents use addressCity / addressState; offers use targetAudience.
  • Nothing was changed. Removing them is an edit to an allowlist the operator approved, so it is hers to make.

venues.updatedAt is approved but nearly empty. ​

  • 8 documents of 22,634 carry it. venues.createdAt is on all 22,634, and both offers timestamps are on all 9.
  • So three of the four approved fields fully serve their stated purpose. The fourth, "same recency need" on the venue import backlog, currently has eight rows behind it.

Two environment traps cost time and are now written into Setup. ​

  • Overriding VITE_ANALYTICS_API_URL does nothing: the client reads /api and vite proxies to ANALYTICS_API_ORIGIN, which is the variable that actually has to move.
  • A fresh worktree needs generate-client-sdk-docs.mjs as well as npm install, or AdminShell.jsx dies with a dynamic-import error that names the wrong file.

Findings from run 2 (2026-08-27) ​

Run 1 already found the seven dead fields. The finding was not missed, the handoff was. ​

  • The table above under "Seven allowlisted fields have no data behind them at all" is dated 2026-08-24 and is correct in every row. It ends "Nothing was changed. Removing them is an edit to an allowlist the operator approved, so it is hers to make", which was the right call.
  • They sat for three days. Nothing carried that decision to her, so a code review on 2026-08-27 reported five of them as new, and the two featureRequests ones were found separately when she asked what the collection was for.
  • The lesson is about parking, not about detection. A finding parked pending a decision needs a route to the person who makes it, or it is indistinguishable from a finding nobody had.

Three separate times, the code asserted something the data contradicted, and nothing failed. ​

  • Status descriptions naming values nobody uses (pending, approved, live) on collections whose real values are open, active, draft, rejected, submitted, pending_setup.
  • Seven field names on collections that have never carried them.
  • A whole collection, featureFlags, allowlisted and advertised, backed by nothing.
  • None of them errored, which is why they survived. A dead field projects to nothing so the column is merely empty; a dead collection returns zero rows so it merely looks quiet.

The guard written to stop this was measuring the wrong property. ​

  • The allowlist drift test compares the two copies. That can only tell you they MATCH, never that either is RIGHT, and all five venue and offer names were wrong identically, so it stayed green.
  • It also compared only field names while the two surfaces disagreed on whether a document KEY leaves the process, so it reported green on half a decision.
  • Agreement is not correctness, and a check that covers half a decision covers none of it. Both limits are now stated in the test file rather than left implicit.

Two bugs on this branch were found only by running the thing, never by reading it. ​

  • The placeholder in brand-new code, which every mock passed happily.
  • All seven fictional field names, each invisible in the diff and obvious in one query.
  • The habit that caught both: check it against the writer, then check it against real data.

What neither run has ever exercised ​

  • The write path to adminSavedQueries for the firestore source (scenario 30).
  • Held undo in the SQL editor after the repeat guard was narrowed (scenario 36).
  • Any real Assistant turn using the new schema tool (scenario 37).
  • Anything at all on deployed dev, which is not possible until this merges (scenario 38).
  • A real screen-reader pass on the console (scenario 27 asserts the attributes only).

Feedback log ​

Operator feedback during these runs goes here, verbatim, in the order it arrived.

2026-08-27, opening scope:

"It isn't finished. Looking at the firestore query console, it has bugs. I also want to verify the design of it as well."

2026-08-27, hitting the dead Firestore tab herself:

"FYI I am still seeing this error when I try to look at the firestore query console: Could not load the Firestore allowlist: Error 

Cannot GET /analytics/admin/firestore-schema
"

2026-08-27, once it worked:

"I see it working now, thank you! That said, we definitely need some design and functionality considerations. First off, we probably need to create a reference doc for this. Or at bare minimum have some breadcrumbs ( like what is firestore for vs bigquery? ). Also, how would someone know how to even query firestore? For example, how would they know to query how many lanterns are lit at an x venue for example?"

2026-08-27, choosing from the mockups:

"I think a mix of B and D ( minus that FYI popup on "firestore answers..." ). For B, I think having a similar "what you can ask", but also saved inquires would be helpful. I think we need a simple reference doc and have that at the front of the sub nav line items so it would be something like ( Reference doc | BQ | FS ). Other pages have that for reference."

2026-08-27, rejecting three parts of that build:

"For the what you can ask, why isn't it expand collapse based on collection? Can you look at how BQ is built to please emulate that?"

"For the what you can ask, i literally meant it as what you can ask. If a user clicks, that auto fills."

"There is literally no way you can even save a query"

2026-08-27, on the field descriptions:

"based on first glances, yes"

2026-08-27, on the reference link:

"I am getting a 404 when I click on the reference link"

2026-08-27, the question that found two live defects:

"What are these feature requests and feature flags for?"

Built with VitePress

Copyright © 2026 Lantern