fix(mcap): support LZ4 frame compression in MCAP decompress handler#18
Merged
Conversation
Contributor
|
Thanks for the fix! The LZ4 frame detection is correct and the fallback keeps existing MCAP files working. Merged and released in v1.6.1. 🙏 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Some MCAP writers (e.g.,
nuscenes2mcapin Python) compress chunks using the full LZ4 Frame format instead of the raw LZ4 Block format defined by the MCAP specification. This caused decompression to fail with "Incorrect chunk CRC" errors. Add auto-detection for the LZ4 frame magic number (0x184D2204) at the beginning of the chunk records buffer. If detected, decompress the chunk as an LZ4 frame usinglz4js.decompress; otherwise, fall back to raw block decompression vialz4js.decompressBlock.Motivation / related issue
I have a nuScenes dataset (https://mcap-proxy.lichtblick.workers.dev/NuScenes-v1.0-mini-scene-sample.mcap) converted to MCAP format via https://github.com/foxglove/nuscenes2mcap. Opening it in RosView fails with the error: "Failed to fetch messages Error: Incorrect chunk CRC 2684863747 (expected 1648207168) [library=nuscenes2mcap]" (as shown in the screenshot). This PR fixes the issue.
Type of change
Checklist
npm run lintpasses with no errorsnpm testpasses (unit tests)npm run buildandnpm run build:libsucceedAPI compatibility
N/A
Screenshots / recordings