fix(mets-gbs): page missing image/coordOCR metadata crashes instead of skipping - #3930
Conversation
…f skipping A page's fileGrp entries in the METS XML are independently optional (_PageFiles.image/ocr/coordOCR all default to None) -- a page can legitimately have no 'image' or 'coordOCR' fptr, e.g. a blank/cover page with no OCR layer. is_valid()/page_count() never checked for this, so such a document reported as valid and loadable, then load_page() crashed with a bare AssertionError (or silently no-op'd under python -O) instead of the graceful per-page invalid state the class already supports (MetsGbsPageBackend.valid = parsed_page is not None). Fixed by having _parse_page() detect a missing image/coordOCR entry and return (None, None) instead of asserting, so load_page() constructs an invalid MetsGbsPageBackend the same way it would for any other unparseable page -- consistent with how the sibling DoclingParsePageBackend already handles a page it can't build (self.valid = (self._ppage is not None) and (self._dp_doc is not None), no crash). Updated MetsGbsPageBackend's accessor methods to assert non-None internally (mirroring DoclingParsePageBackend's exact pattern), since callers are expected to check is_valid() before using a page backend, same contract as the sibling class. Added a regression test using a new fixture derived from the existing real METS-GBS test archive, with one page's coordOCR fileGrp entry and file removed to reproduce the missing-metadata case with real data rather than a synthetic minimal example. Signed-off-by: Praveen Mittal <pkmittal28@gmail.com>
|
✅ DCO Check Passed Thanks @mittalpk, all your commits are properly signed off. 🎉 |
Merge Protections🟢 All 2 merge protections satisfied — ready to merge. Show 2 satisfied protections🟢 Enforce conventional commitMake sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
🟢 Require two reviewer for test updatesWhen test data is updated, we require two reviewers
|
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Just finished my review by the time it got merged, but still worth noting some observations, for a future improvement:
|
What's wrong
A page's
fileGrpentries in the METS XML are independently optional in the backend's own data model (_PageFiles.image/ocr/coordOCRall default toNone) — a page can legitimately have noimageorcoordOCRfptr, e.g. a blank/cover page with no OCR layer.is_valid()/page_count()never check for this, so such a document reports as valid and loadable, thenload_page()crashes with a bareAssertionError:— instead of the graceful per-page invalid state the class already supports (
MetsGbsPageBackend.valid = parsed_page is not None). Underpython -O, this assertion is stripped entirely and the code falls through to a different, more confusing crash further down instead.Fix
_parse_page()now detects a missingimage/coordOCRentry and returns(None, None)instead of asserting, soload_page()constructs an invalidMetsGbsPageBackendthe same way it would for any other unparseable page — consistent with how the siblingDoclingParsePageBackendalready handles a page it can't build (self.valid = (self._ppage is not None) and (self._dp_doc is not None), no crash).MetsGbsPageBackend's accessor methods (get_text_in_rect,get_text_cells,get_bitmap_rects,get_page_image,get_size) now assert non-None internally, mirroringDoclingParsePageBackend's exact existing pattern — callers are expected to checkis_valid()before using a page backend, same contract as the sibling class, and the downstream pipeline code (standard_pdf_pipeline.py,base_pipeline.py, etc.) already does this check for every backend.How was this tested?
mainfirst, using a new fixture derived from the existing real METS-GBS test archive (tests/data/mets_gbs/sources/32044009881525_select.tar.gz), with one page'scoordOCRfileGrpentry and its underlying file removed to reproduce the missing-metadata case with real data rather than a synthetic minimal example.test_page_missing_coordocr_is_skipped_not_crashed, confirmed it fails against unpatchedmets_gbs_backend.py(git stash) with the exactAssertionError, and passes after the fix — pages with complete metadata still load normally, the page missingcoordOCRis reported asis_valid() == Falseinstead of crashing.tests/test_backend_mets_gbs.pysuite: 10 passed (9 existing + 1 new), no regressions.ruff format --check/ruff check— clean.ty checkon the touched file — clean except one pre-existing, unrelated warning (has_textlinesargument mismatch), confirmed present identically on unpatchedmain.Checklist: