Trigger the term_info fallback on a document with no term_info field - #112
Merged
Conversation
get_term_info only rebuilt a term_info document when SOLR returned no document at all (hits == 0). But the vfb_json collection is written by several indexers, each setting only its own field with an atomic update, so a record that one of the query-result indexers reaches before the term_info indexer does has a document with no term_info in it. For such an id term_info_parse_object hit a KeyError and get_term_info returned None, and the fallback never ran. Berg2025a and Bates2026 were exactly that on 2026-09-04: all_datasets_query present (listed under All Datasets, thumbnail and all), term_info absent (blank term-info panel on the site, get_term_info -> null). Key the fallback on the field rather than the hit count. The rebuilt document is written with the indexer's atomic-update shape, so the sibling fields on a partial document are left as they were.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
get_term_infoonly rebuilt a term_info document when SOLR returned no document at all (hits == 0). But thevfb_jsoncollection is written by several independent indexers, each setting only its own field via an atomic update — so a record that one of the query-result indexers (e.g.all_datasets_query) reaches before the term_info indexer does has a document with noterm_infofield in it. For such an id,term_info_parse_objecthit aKeyErrorandget_term_inforeturnedNone; the fallback never ran because it was keyed on the hit count, not on whether the field existed.Berg2025a and Bates2026 were exactly that on 2026-09-04: listed under All Datasets (thumbnail and all —
all_datasets_querypresent), but the term-info panel on the site was blank (get_term_info-> null).Fix
Add
_has_term_info_document, which checks theterm_infofield itself rather than the hit count, and key the fallback trigger inget_term_infoon that instead ofhits. The rebuilt document is written with the indexer's own atomic-update shape, so sibling fields already on a partial document are left as they were.Testing
New
src/test/test_term_info_fallback_partial_doc.py(8 tests, offline — SOLR and the PDB rebuild are stubbed): pins the trigger logic (no document / document without term_info / empty term_info field / real term_info / the fallback's own stand-in result), and exercisesget_term_infoend to end for the rebuild-on-partial-doc case, the leave-alone-when-present case, and the rebuild-fails-gracefully case.All 45 tests in the suite pass with the fix; reverting the trigger change alone causes 6 of the new tests to fail, confirming they pin the right behavior.
Related: VirtualFlyBrain/VFB_json_schema_indexer#6 (the indexer-side fix for the same underlying gap — the missing-phase-skip bug that let this go undetected for over a day).