Skip to content

Wire facet droppers to context-aware counts (frontend for #12978) - #13064

Merged
lokesh merged 14 commits into
internetarchive:masterfrom
Armansiddiqui9:Search-improvements
Aug 26, 2026
Merged

lokesh merged 14 commits into
internetarchive:masterfrom
Armansiddiqui9:Search-improvements

Conversation

@Armansiddiqui9

Copy link
Copy Markdown
Contributor

Closes #13060
Depends on #12980 (backend API, currently WIP β€” see note below)

Feature
Makes the Language filter on /search reflect the current search instead
of always showing the full catalogue. When a patron has an active query,
opening the Language dropper now shows only languages with matching
results, with a count next to each, sorted by relevance, no more dead-end
selections that silently return zero books.

Technical

  • OlSelectPopover.js β€” extended item shape to {value, label, count?}.
    Renders count as a muted, comma-formatted trailing number. Added a
    loading attribute for a host-driven spinner state. No fetch or domain
    logic added β€” stays a pure UI component per the
    building-components guide.
  • search-modal/searchFacets.js (new) β€” service module owning the
    /search/facets.json call. Exposes fetchFacetCounts(field, searchParams).
  • SearchFilterBar.js β€” orchestration. On ol-popover-open, fetches
    the language catalogue and facet counts in parallel, merges by MARC code,
    sorts descending, hides zero-count values (keeps the current selection
    visible even at 0 so it can still be unticked). Falls back to the
    uncounted catalogue on error or when there's no active query.
  • searchFacets.test.js (new) β€” 16 unit tests covering the service
    module's response handling and the merge/sort/hide-zero logic. All passing.

Testing

  • npm run test:js β€” 483/483 passing (including the 16 new tests)
  • Manual end-to-end test on local Docker dev environment: stubbed
    /search/facets.json (since feat(search): add GET /search/facets.json β€” context-aware facet values APIΒ #12980 is still WIP) and verified against
    the real /languages.json catalogue. Confirmed counts render correctly,
    comma-formatted, sorted descending, zero-count items hidden, loading
    state visible during fetch.
Screenshot 2026-06-30 164935
  • Verified no-query fallback (no facet request fires when q= is empty)
  • Verified graceful fallback to the uncounted list on a rejected fetch

Open question for @RayBB

Confirmed locally that the frontend's language catalogue keys value on
MARC code (eng, fre, ger...) via /languages.json, not the
display name. Please make sure /search/facets.json's value field
returns matching MARC codes, not full language names β€” otherwise the
merge in SearchFilterBar.js will silently match nothing and the dropper
will render empty. Happy to pair on this if useful.

Stakeholders

@lokesh @RayBB

Armansiddiqui9 and others added 3 commits June 30, 2026 17:29
Extends item shape from {value, label} to {value, label, count?}.
Renders count as a muted, comma-formatted trailing number when present.
Adds host-driven loading boolean attribute, swaps suggestions list for
a spinner while true. No fetch or domain logic added - stays a pure
UI component per the building-components architecture guide.

Part of internetarchive#13060
@openlibrary-bot

Copy link
Copy Markdown
Collaborator

Thanks for the contribution, @Armansiddiqui9!

πŸ€– Copilot has been assigned for an initial review.

The linked issue (#13060) hasn't been triaged yet β€” triage happens on Mondays and Fridays. There are currently 27 open non-draft PRs ahead of yours.

PR triage checklist (maintainers / Richy)
  • PR description β€” not empty; explains what the change does and how to verify it
  • References an issue β€” PR body contains a #NNN reference
    • Linked issue is triaged β€” has a Priority: * label (not just Needs: Triage); currently has Needs: Response
    • Linked issue is assigned β€” has at least one assignee
  • Commit history clean β€” no WIP/fixup/conflict noise; commit messages are meaningful
  • CI passing β€” checks still pending
  • Test cases present β€” 16 new unit tests in searchFacets.test.js
  • Proof of testing β€” PR body includes a screenshot and a description of what was tested

Note

This comment was automatically generated by PAM, Open Library's Project AI Manager. PAM provides status visibility, performs basic project management functions, and gives actionable feedback so contributors aren't left waiting.

@lokesh

lokesh commented Jun 30, 2026

Copy link
Copy Markdown
Collaborator

This is looking great. Once the backend work is complete we'll do one more pass. At that point, if you could record a short video of the feature working, including the new loading indicator, that would be really helpful.

I'm encouraging all of us to add videos to UI-related PRs. They convey a lot about the scope of a feature, get people excited, and give reviewers confidence that the change has actually been tested.

@mekarpeles

Copy link
Copy Markdown
Member

See also #12980 (now sure if it implements the APIs but we likely want to treat these as 2 different PRs -- the APIs and the UI wiring)

@Armansiddiqui9

Copy link
Copy Markdown
Contributor Author

@mekarpeles agreed, treating them as separate PRs. The frontend (#13060) is ready for review independently. It degrades gracefully when the backend isn't available (falls back to the uncounted catalog), so nothing breaks. Will do full end-to-end testing once #12980 merges and record the demo video Lokesh requested at that point.

@lokesh lokesh added the State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed] label Jul 6, 2026
@lokesh

lokesh commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Blocked on #12980

# Conflicts:
#	openlibrary/components/lit/custom-elements.json
@lokesh

lokesh commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

@mekarpeles this PR adds counts to the search facet dropdowns. It is blocked on your work which adds a new endpoint that returns the facet counts for the search: #12980

@lokesh

lokesh commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

#12980 is now merged! We can wire up the UI

Armansiddiqui9 and others added 4 commits August 14, 2026 01:56
The facet counts came from a plain WorkSearchScheme, so they ignored the
solr_editions setting and skipped the edition-level block-join that
/search.json opts into. Availability-filtered counts could then disagree
with the result set they were meant to describe.

Pass solr_editions through from the request context and request the same
key,editions fields, so a facet count and the search it annotates are
answering the same question.

Claude-Session: https://claude.ai/code/session_018u1CiBQgEQ8YEBqBf3J5Hc
The endpoint lives only in FastAPI, and local dev has no reverse proxy in
front of the two servers β€” web.py forwards an allowlist of paths to
fast_web instead. /search/facets was missing from it, so the browser got
a 404 from web.py on :8080 and the language dropper silently fell back to
its uncounted list. The `/search` entry only matches /search.json.

Claude-Session: https://claude.ai/code/session_018u1CiBQgEQ8YEBqBf3J5Hc
The header modal's language dropper always showed the whole catalogue β€”
only the /search filter row got counts. It now fetches the same facet
counts for the query on screen.

- Move mergeFacetCounts into searchFacets.js so both surfaces share it,
  and join on `value` (the MARC code) rather than `label`. Both sides
  localize their labels independently, so a label join would drop every
  row under some UI languages. The old fixtures used value === label, so
  nothing caught it.
- Strip an existing filter on the field being counted. Solr ANDs an fq on
  the faceted field, which would zero out every other language and strand
  the patron on the one already ticked.
- Cache the merged list per query + availability, so re-opening the
  dropper on an unchanged search costs nothing and ticking a checkbox
  doesn't refetch.
- Open the dropper only once its counts are in. It used to open on a
  spinner, then swap in a shorter, re-sorted list β€” two jarring
  transitions under the pointer. ol-select-popover now fires a cancelable
  `ol-select-popover-request-open`; a host that defers calls show() when
  ready. Hosts that don't listen are unaffected. The wait is capped at
  500ms, past which it opens on the catalogue as before.
- Make the dropper's loading text translatable (`loadingLabel`) β€” it was
  the one hardcoded string left in the component.

Claude-Session: https://claude.ai/code/session_018u1CiBQgEQ8YEBqBf3J5Hc
@lokesh

lokesh commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

I pushed a few commits to this branch. Here is what they do.

Why the counts were not showing up locally

  • The counts come from /search/facets.json, which only exists in FastAPI.
  • Local dev has no proxy in front of the two servers. web.py forwards a list of paths to FastAPI instead, and /search/facets was not on that list.
  • So the browser asked :8080 for it and got a 404. The dropper caught the error and quietly showed the plain list, with nothing in the console.
  • Added the one missing line. The counts show up now.

Counts in the header search modal

  • The modal had its own language dropper that always showed the whole catalogue. Only the /search filter row got counts.
  • The modal now fetches the same counts for whatever query is on screen.
  • The counts are cached per query, so re-opening the dropper on the same search is free.
  • Ticking a language does not refetch, because the selection is not part of what is counted.
  • No query means no counts, just the plain list.

Merging on the code, not the name

  • mergeFacetCounts matched the count to the item by its label ("English"), not its value (eng).
  • Both the catalogue and the API translate their labels on their own. If the two ever differ, every row drops out and the dropper renders empty.
  • It now matches on the MARC code, which never changes.
  • The old test fixtures used value === label, so nothing caught this. New fixtures use real codes, plus a test with mismatched labels.
  • Also moved this function into searchFacets.js so both surfaces share one copy.

Not filtering the list you are filtering

  • Solr ANDs a filter on the field it is counting.
  • So with language=eng in the URL, every other language came back as 0 and got hidden. Once you picked one language you could not find a second one.
  • The facet request now drops any filter on the field being counted.

Opening the dropper without the flash

  • Before: the dropper opened tall with a spinner, then collapsed to a short re-sorted list. Two jumps in a row, under your pointer.
  • Now: it waits for the counts, then opens once at its final size.
  • The wait is normally about 100-300ms. It is capped at 500ms, and past that it opens on the plain list like before, so a slow request can never leave the button dead.
  • ol-select-popover got one new optional event for this. Anything that does not listen for it behaves exactly as before.

Smaller things

  • The dropper's "Loading…" text was hardcoded English. It is translatable now.
  • The facets query was ignoring solr_editions and the edition block-join that /search.json uses, so availability-filtered counts could disagree with the results they described. It now asks Solr the same question.

Testing

  • npm run test:js β€” 611 pass. The three facet suites are 44 tests, up from 16.
  • Checked in the browser on /search and in the modal: counts render, the panel opens once already sorted, selecting a language does not refetch, clearing the query drops back to the plain list, and Readable Only narrows the counts.

@Armansiddiqui9

Copy link
Copy Markdown
Contributor Author

I've been stuck on this for about a day, kept getting the counts to work in the unit tests, but nothing showed up in the browser locally, no errors in the console either. Traced it down to /search/facets.json not being on the list of paths web.py proxies over to FastAPI in dev, so it was 404ing silently. Added that route so the frontend could actually talk to the endpoint, which is what finally got the context-aware counts rendering on my end.

Just saw your commits and reading through them now. Looks like you found the same proxy gap plus a good number of things I hadn't caught (the self-narrowing bug on multi-select, the modal counts, and the solr_editions consistency issue). Going to pull this, test it locally end-to-end, and put together the screen recording once I've confirmed everything's working.

…s, fix watch script missing lit-components build, fix mobile tray clipped by dvh
@Armansiddiqui9

Copy link
Copy Markdown
Contributor Author
Screenshot 2026-08-15 013513 Screenshot 2026-08-15 011022

Tested commits end to end (desktop + mobile, screenshots attached) - counts, merge-on-value, and the no-flash open all work great. Wired SearchFilterBar.js to the new shared searchFacets.js API (it was still on the old local mergeFacetCounts/ol-popover-open contract), and fixed a mobile tray clipping issue with 85dvh.

@lokesh lokesh removed the State: Blocked Work has stopped, waiting for something (Info, Dependent fix, etc. See comments). [managed] label Aug 14, 2026
@github-actions github-actions Bot added the Needs: Response Issues which require feedback from lead label Aug 15, 2026
lokesh added 3 commits August 25, 2026 18:03
`watch` ran watch:js, watch:css and watch:components but never
watch:lit-components, so editing anything under openlibrary/components/lit/
left static/build/lit-components/production/ol-components.js untouched. The
page kept serving the previously built bundle, with no error to show for it.

This bites hardest when a change spans both bundles: the search modal's facet
counts need SearchModal.js (webpack) to hear an event that OlSelectPopover.js
(the Lit bundle) fires, so a stale Lit bundle leaves the listener attached to
an event nothing emits and the dropper opens silently uncounted.

Claude-Session: https://claude.ai/code/session_011AqgrMKn29JVncut49JNtf
vite-lit.config.mjs is the only Vite config asking for `minify: 'terser'`,
and terser has never been in package.json. It resolved anyway because webpack
pulled it in transitively via terser-webpack-plugin, whose ^5.31.1 range is
what pinned the lockfile's terser 5.31.1. Vite only ever listed it as an
optional peer.

That makes it a landmine for internetarchive#13331, which drops webpack for Vite: without
terser-webpack-plugin the package disappears, `make lit-components` dies with
"terser not found", and because the target only swaps the bundle into place on
success, the previously built ol-components.js survives. Nothing in the browser
looks broken; the Lit elements are simply frozen at the last good build. On
testing that stranded OlSelectPopover on a revision that never fires
ol-select-popover-request-open, so the search modal's facet droppers listened
for an event nothing emitted and opened silently uncounted.

Switch to the default minifier, matching every other Vite config here rather
than adding a dependency to keep one config's choice alive.

Claude-Session: https://claude.ai/code/session_011AqgrMKn29JVncut49JNtf
/search/facets.json only exists in FastAPI. Local dev proxies the bare path
through web.py, but everywhere else web.py answers it from the deprecated-
endpoint handler, which raises internalerror unless LOCAL_DEV is set. On the
testing server that is a hard 500:

    DEPRECATED ENDPOINT ACCESSED: /search/facets.json. This endpoint has been
    migrated to FastAPI and should not be accessed in production.

fetchFacetCounts() surfaces that as a failed load, so the dropper silently
falls back to its uncounted catalogue. Testing exposes FastAPI behind /_fast,
where the same query returns counts, so pick the prefix from the hostname the
way testingStatusUrl() already does for /status/testing.json.

Claude-Session: https://claude.ai/code/session_011AqgrMKn29JVncut49JNtf
@lokesh

lokesh commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator
image

Got it up on testing after fighting with some unrelated build issues. Looking great! I'll merge after a bit more code review.

@lokesh
lokesh merged commit 473afea into internetarchive:master Aug 26, 2026
5 checks passed
RayBB added a commit that referenced this pull request Aug 26, 2026
Resolves package-lock.json conflict by regenerating against the merged
package.json (keeps master's concurrently v10 bump alongside the Vite
dependency changes). Also picks up #13064's terser fix (minify: true).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Response Issues which require feedback from lead

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(search): Wire facet droppers to context-aware counts (frontend for #12978)

4 participants