Skip to content

feat: lazy-load memory browse and Flashbacks - #26

Merged
nonnil merged 10 commits into
fix/performfrom
feat/lazy-loading
May 30, 2026
Merged

feat: lazy-load memory browse and Flashbacks#26
nonnil merged 10 commits into
fix/performfrom
feat/lazy-loading

Conversation

@nonnil

@nonnil nonnil commented May 28, 2026

Copy link
Copy Markdown
Member

Summary:

  • Implemented Task 20 lazy-loading performance workflow end to end.
  • Replaced initial /memories archive-wide loading with cursor-paginated SQLite browse pages.
  • Kept search, read-state, taxonomy, and ?flashback=<id> filters server-global before pagination.
  • Split Flashback loading into bounded right-rail recent rows, memory-card batches for visible memory IDs, and reader All-tab lazy loading.
  • Scoped add-memory success revalidation to first browse page + taxonomy before reader navigation.
  • Preserved backup failsafe behavior and fixed one backup test harness edge where Git hook env variables broke nested temp-repo git commands.

Subtask / commit mapping:

  • 20.1 Browse query and page contract: d9d8100
  • 20.2 SQLite repository pagination: 28def27
  • 20.3 Browse loader contract: 3c5a24b
  • 20.4 Memories infinite scroll UI: fd43d41
  • 20.5 Lazy Flashback browse data: 4babf16
  • 20.6 Reader lazy Flashback tab: 00dc970
  • 20.7 Integration verification fixes: 6a1d3f2, eddf0df

Behavior covered:

  • /memories initial route loads a 30-row first page.
  • Infinite scroll / Load more appends the next page without duplicate titles.
  • /memories?q=<term> searches the full archive, including rows not loaded on the first page.
  • /memories?flashback=<id> still finds the owning memory and hydrates the selected excerpt.
  • Right-rail recent Flashbacks use loadRecentFlashbackBrowseRows({ limit: 5 }), not all memories.
  • Reader Current tab no longer starts the global Flashback query.
  • Reader All tab starts the global Flashback query on demand.
  • Add-memory success no longer revalidates the old all-memory browse query.
  • Backup failsafe warning/alert behavior remains covered by the backup drift/inconsistency tests.

Verification:

  • mise exec -- bun run test tests/memories/browse-data.test.ts tests/components/browse-data-query.test.ts: passed, 2 files / 28 tests.
  • mise exec -- bun run test tests/server/db/schema.test.ts tests/server/db/repositories.test.ts: passed, 2 files / 36 tests.
  • mise exec -- bun run test tests/server/memories/browse.test.ts tests/server/browse-loaders.test.ts tests/server/flashbacks/repository.test.ts: passed, 3 files / 24 tests.
  • mise exec -- bun run test tests/components/browse-loader.test.ts tests/components/flashbacks-loader.test.ts tests/components/app-shell.test.ts tests/components/memory-browse-actions.test.ts tests/components/reader-flashback-tabs.test.ts: passed, 5 files / 62 tests.
  • mise exec -- bun run test tests/components/reader-flashback-tabs.test.ts tests/components/reader-memory-loader.test.ts tests/components/memory-reader-actions.test.ts tests/components/memory-reader-flashback-selection.test.ts: passed, 4 files / 36 tests.
  • mise exec -- bun run test tests/server/backup/backup-environment.test.ts with parent GIT_DIR/GIT_WORK_TREE set: passed, 1 file / 10 tests.
  • mise exec -- bun run typecheck: passed.
  • mise exec -- bun run test: passed, 103 files / 799 passed / 5 todo.
  • mise exec -- bun run build: passed. Existing Node DEP0155 warning remains from defuddle -> temml.
  • mise exec -- bun run test:e2e: passed, 47 Playwright tests.
  • Pre-push hook on final push: passed typecheck, full test, and build.

Performance evidence:

Measured with the Task 20 harness against a copy of the local /Users/vvx/.trauma dataset to avoid mutating the live database/store. Dataset: 110 memories, first page 30 rows, 39 renderable Flashbacks, largest source memory 249,025 bytes.

Path Before After median Result
Initial browse memory load loadBrowseMemories() 721 ms, 110 rows, 39 nested Flashbacks loadBrowseMemoryPage() 1.28 ms 30 rows, nextCursor=true
Memory-card Flashbacks Included in all-memory browse payload 16.59 ms 30 memory IDs, 5 card Flashbacks
Right-rail recent Flashbacks Derived from all-memory browse payload / loadFlashbackBrowseRows() 756 ms 15.96 ms 5 recent Flashbacks
Large reader source 322 ms first / 256 ms warm 227.67 ms Same 249 KB source memory
Reader Current tab Could coexist with global Flashback resource creation ~0 ms shortcut mapping 1 current-memory shortcut, no global query
Reader All tab Global query could start during initial reader render 587.24 ms, deferred 39 Flashbacks, only after All
Add-memory success + reader/page refresh Revalidated all-memory browse path 3.71 ms with synthetic importer new reader ready, first page remains 30 rows

Known local state:

  • The pushed branch tip is eddf0df.
  • The linked worktree currently has unrelated uncommitted changes that appeared during final verification, including Task 21 workflow/docs/UI/settings changes and Task 20 workflow deletions. They were not staged or committed for this PR.
  • .sawyer/exclude-whitelist.txt was checked; whitelisted paths such as .tmp/, TASK_19_INSTRUCTION.md, TEMP_MAIN_CONTENT_EXTRACTION_VALIDATION_REPORT.md, app.config.timestamp_1778391094301.js, and refined_sample/ were not staged.

Backup safety:

  • Backup failsafe test coverage still emits and asserts drift/inconsistency alerts.
  • The branch does not change backup integrity algorithms or weaken failsafe checks.

@coderabbitai

coderabbitai Bot commented May 28, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ff0b6831-60af-4612-a485-58d5aeaf815d

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/lazy-loading

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@nonnil nonnil changed the title docs: add lazy loading workflow feat: lazy-load memory browse and Flashbacks May 28, 2026
@nonnil
nonnil marked this pull request as ready for review May 28, 2026 08:54
@nonnil

nonnil commented May 28, 2026

Copy link
Copy Markdown
Member Author

review @copilot @codex

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eddf0df15b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/server/flashbacks/browse.ts
Comment thread src/components/memories/AddMemoryForm.tsx
Comment thread src/server/db/repositories.ts
@nonnil

nonnil commented May 29, 2026

Copy link
Copy Markdown
Member Author

review @copilot @codex

Addressed the three Codex review threads in 7a8a4bf and pushed the branch. Threads were replied to and resolved; local verification and the pre-push hook passed.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@nonnil
nonnil merged commit 7889d42 into fix/perform May 30, 2026
1 of 2 checks passed
@nonnil
nonnil deleted the feat/lazy-loading branch May 30, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant