Skip to content

Validate snapshot page completeness in the decoder #98

Description

@corylanou

Problem

Decoder.Close() contains an old TODO to ensure the last decoded page matches Header.Commit for snapshot LTX files.

Snapshots represent a complete database, but the decoder does not currently prove that every required page is present. The page-index validation added in #97 ensures entries are ordered, remain within Commit, and match the decoded page sequence, but an internally consistent snapshot can still end early or omit required pages.

Expected validation

For snapshot files (Header.IsSnapshot()):

  • Commit == 0 must contain no page frames.
  • Otherwise, decoded pages must be exactly 1..Commit.
  • The SQLite lock page must be omitted when it falls within that range.
  • Missing initial, interior, or trailing pages must be rejected.
  • Non-snapshot files must retain their existing sparse-page behavior.
  • Validation must work whether page-index retention is enabled or disabled.

Implementation notes

The existing snapshot-validation branch and commit 8c65988 are a useful starting point, but should not be applied unchanged. Its lastPgno != 0 guard allows an empty snapshot with a nonzero commit, and checking only the last page does not prove that interior pages are complete.

After #97, this can remain constant-memory by extending the streaming/sequence validation or by tracking the next expected snapshot page while decoding. Account explicitly for LockPgno(PageSize), including the case where Commit equals the lock page.

Acceptance criteria

  • Complete snapshots verify successfully.
  • Zero-commit deletion snapshots verify successfully.
  • Empty snapshots with a nonzero commit are rejected.
  • Snapshots missing the first, an interior, or the final required page are rejected.
  • Snapshots spanning the lock page verify when the lock page is omitted and fail for other gaps.
  • Checksum-free snapshots receive the same structural validation.
  • Non-snapshot files remain valid with sparse pages.
  • The TODO in decoder.go is removed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions