feat(formats)!: migrate ARW/CR2/NEF/CR3 IFD walking to gamut-ifd - #45
Merged
Conversation
Rebuilds the Sony ARW, Canon CR2, Nikon NEF, and Canon CR3 (CMT1-CMT4 TIFF block) decoders on gamut_ifd, replacing their use of the in-repo binrw TIFF parser. New shared module formats/ifd.rs (feature `ifd-parser`) carries the private tag catalogue as data over the gamut engine — baseline TIFF/EXIF/GPS/ DNG-block tags plus the Sony SR2 set — together with a lenient tree resolver (malformed sub-IFD warns and skips, matching the old parser), value coercions, bounds-checked range reads, and gamut-based ports of the metadata_helper extractors. ARW keeps its SR2 semantics: the encrypted sub-IFD is read lazily via IfdReader so a garbled entry fails only its own value fetch, and the MakerNote absolute-offset probe becomes a bounds-checked slice read. CR3's ISOBMFF box walker is untouched — that slice is blocked upstream (visualcommons/gamut#301, 64-bit largesize + uuid usertype); only its embedded TIFF blocks move. RAF audited: confirmed free of tiff-parser usage, no change. DNG and the TIFF writer are deliberately NOT migrated here: gamut-dng subsumes DNG's IFD walking wholesale in #21, which is when src/tiff/ loses its last consumer and is deleted along with binrw — migrating dng.rs twice serves nobody. Feature graph: arw/cr2/cr3/nef-decode now pull `ifd-parser` (dep:gamut-ifd, bigtiff enabled) instead of `tiff-parser`; dng-decode pulls both (format detection's Make/DNGVersion sniff migrated to a streaming IfdReader); dng-encode keeps tiff-parser alone. codecs/{ljpeg,bit_pump} re-gated onto the decoders that actually use them instead of the parser feature. Error surface: new ungated ParseError::MissingTag(u16) (numeric replacement for the TiffTag-typed TagNotFound, which stays gated until #21). Behaviour notes (fixture-neutral, verified against real Sony ARW files): - gamut decodes IFD values eagerly at parse; files with corrupt top-level value offsets now fail at parse instead of on first touch. Sub-IFD children stay lenient. - ASCII handling normalized rawshift-side (clean_ascii) so metadata strings are byte-identical to before despite gamut preserving interior NULs. - CFAPattern now also accepted from UNDEFINED-typed fields (previously BYTE only, silently defaulting to RGGB). - Out-of-bounds strip/thumbnail reads surface as ParseError::OffsetOutOfBounds instead of io UnexpectedEof. Tests: new tests/ifd_decoder_tests.rs ports the decoder-level assertions of tiff_parser_tests.rs to the new path against real fixtures; parser-internal tests remain with the binrw module and are deleted with it in #21. BREAKING CHANGE: MissingTag display uses numeric tag ids; ARW/CR2/CR3/NEF builds no longer enable the `tiff-parser` feature.
This was referenced Jul 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses the unblocked scope of #20; the binrw module deletion completes in #21 (see sequencing note), and the CR3 ISOBMFF slice stays blocked on gamut#301.
Stacked on #44 (base
19-migrate-metadata-gamut); merge order #42 → #44 → this.What is Delivered
gamut_ifd::readwith explicit pointer-tag resolution, replacing the binrwTiffParserin those decoders. ARW keeps exact SR2 semantics: the encrypted sub-IFD reads lazily viaIfdReader(a garbled entry fails only its own value fetch); MakerNote probing becomes bounds-checked slice reads.IfdReader. The hand-rolled ISOBMFF box walker is deliberately untouched — blocked upstream (gamut#301:BoxReaderrejects 64-bitlargesize, which real CR3mdatboxes use).formats/ifd.rs(featureifd-parser=dep:gamut-ifd+bigtiff): the private vendor tag catalogue as data over the gamut engine per the issue — baseline TIFF/EXIF/GPS/DNG-block tags + the Sony SR2 set — plus a lenient tree resolver matching the old warn-and-skip on malformed sub-IFDs, value coercions, bounds-checked reads, and ports of themetadata_helperextractors.arw/cr2/cr3/nef-decodeno longer pulltiff-parser/binrw;dng-decodepulls both (its detection sniff migrated to a streaming reader; full DNG migration is Migrate DNG decode/encode to gamut-dng #21);codecs/{ljpeg,bit_pump}re-gated onto their actual consumers.tests/ifd_decoder_tests.rsre-expresses the decoder-level behaviours oftiff_parser_tests.rson the new path against the real Sony fixtures; parser-internal tests stay with the binrw module and die with it in Migrate DNG decode/encode to gamut-dng #21.Deliberate sequencing deviation
The issue text deletes
src/tiff/here. dng.rs is that module's last consumer, and #21 replaces dng.rs wholesale with gamut-dng (which does its own IFD walking) — migrating dng.rs onto gamut-ifd now would mean rewriting it twice in consecutive PRs. The binrw module stays compiled behindtiff-parserfor exactly one more PR; #21 deletes it with thebinrwdep, thetiff-parserfeature, and the publicTiffParser/TiffWriter/TiffValueAPI (the 0.x break the issue accepts).Behaviour notes (fixture-verified neutral)
Eager value decoding (corrupt top-level offsets fail at parse, not first touch; sub-IFD children stay lenient); ASCII normalization kept byte-identical via rawshift-side
clean_ascii; CFAPattern now also read from UNDEFINED-typed fields (was BYTE-only with silent RGGB fallback — a small correctness win); OOB strip reads surface asParseError::OffsetOutOfBounds; new ungatedParseError::MissingTag(u16)with numeric display.Verification Done
Run independently after implementation:
cargo test --workspace— 383 passed, 0 failedcargo test -p rawshift-image --features full— 576 passed, 0 failed (fixtures fetched; ARW suite ran against the real 40 MB Sony files — raw-frame discovery, dimensions, bit depth, Make/Model, strip bounds asserted on the new path; CR2/CR3/NEF fixture tests skip as before, no upstream fixtures exist)cargo clippy -p rawshift-image --all-targets --features full -- -D warnings— clean; workspace clippy — clean--no-default-features --features {arw,cr2,cr3,nef,dng}— zero errors each