Skip to content

[Perf] PDF·DOCX 50·100문서 전체 인덱싱 E2E 부하 검증 - #144

Merged
Gimini-3 merged 6 commits into
developfrom
feature/143
Aug 10, 2026
Merged

[Perf] PDF·DOCX 50·100문서 전체 인덱싱 E2E 부하 검증#144
Gimini-3 merged 6 commits into
developfrom
feature/143

Conversation

@Gimini-3

@Gimini-3 Gimini-3 commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

개요

실제 Text Layer PDF와 OOXML DOCX를 HTTP Multipart로 업로드하고 PostgreSQL 17, MinIO, BAAI/bge-m3와 pgvector를 통과하는 전체 인덱싱 부하 Benchmark를 추가했습니다.

변경 사항

  • PDF·DOCX 50·100문서 혼합 Profile과 형식별 통계 계약
  • PDFBox 2페이지 PDF와 Apache POI 2개 Section DOCX 실제 Binary Fixture
  • 실제 MinIO 저장, 자동 Worker, Batch Embedding, vector(1024), current_version 전환 전체 검증
  • 전체·형식별 처리량 및 Upload·Queue·처리·E2E p50·p95·p99
  • 일반 테스트와 분리된 documentIndexingE2ELoadTest Gradle Task
  • 설계 문서와 재현 가능한 실측 결과 문서

실측 결과

문서 수 구성 반복 중앙 총 시간 문서/분 Chunk·Embedding/초 전체 P95
50 PDF 25 + DOCX 25 2회 96.633초 31.046 2.070 92.178초
100 PDF 50 + DOCX 50 2회 187.138초 32.063 2.138 178.533초
  • 본 측정: 300문서, 1,200 Chunk, 1,200 Embedding
  • 업로드 실패·FAILED·Retry·미완료·중복 Vector: 0건
  • 모든 Vector 1024차원
  • PDF 페이지 2개와 DOCX Section 2개 Metadata 보존
  • 문서 수가 2배가 되어도 약 31~32문서/분 처리량 유지

검증

  • ./gradlew test --tests com.opensource.docgrid.e2e.DocumentIndexingE2ELoadStatisticsTest
  • 실제 PDF 2 + DOCX 2 Smoke: PASS
  • 실제 50·100문서 각 2회 본 측정: PASS, 9분 44초
  • ./gradlew localE2eTest: PASS, 2 tests
  • ./gradlew test: PASS, 746 tests
  • git diff --check origin/develop...HEAD: PASS

범위 밖

  • OCR·스캔 PDF, 구형 DOC·HWP
  • 운영 SLO 확정
  • 공식 OpenSQL 원격 장비의 절대 성능 판정

Closes #143

Summary by CodeRabbit

  • 새로운 기능

    • PDF·DOCX 문서의 전체 인덱싱 성능과 데이터 완전성을 측정하는 E2E 부하 테스트를 추가했습니다.
    • 50·100개 문서 프로파일과 병렬 업로드를 지원하며, 처리량·지연시간·형식별 통계를 JSON으로 제공합니다.
    • 문서, 청크, 임베딩, 메타데이터 및 처리 상태의 정합성을 자동 검증합니다.
  • 문서화

    • 벤치마크 설계, 실행 방법, 측정 결과와 제한 사항을 문서화했습니다.
  • 테스트

    • PDF·DOCX 통계 집계 및 입력 검증 테스트를 추가했습니다.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@Gimini-3, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: dc83523d-d0bd-4772-8cf9-57bc362d62f5

📥 Commits

Reviewing files that changed from the base of the PR and between c605714 and 1848d69.

📒 Files selected for processing (2)
  • docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md
  • src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java
📝 Walkthrough

Walkthrough

PDF·DOCX 문서를 병렬 업로드하고 전체 인덱싱을 수행하는 E2E 부하 Benchmark를 추가했습니다. 50·100문서 Profile의 처리량, 지연시간, 데이터 정합성을 검증하고 JSON 결과를 기록합니다.

Changes

PDF·DOCX 인덱싱 Benchmark

Layer / File(s) Summary
Benchmark 계약과 실행 구성
docs/design/..., build.gradle, src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java
Benchmark 범위와 실행 조건을 문서화했습니다. 전용 Gradle 태스크, 테스트 태그, 시스템 속성, Schema·Bucket 초기화와 정리를 추가했습니다.
Fixture 생성과 인덱싱 실행
src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java
PDF·DOCX Fixture를 균형 있게 생성하고 병렬 Multipart 업로드를 수행합니다. 워밍업과 문서 수별 Profile을 실행하고 Queue Drain과 처리 지표를 수집합니다.
정합성 검증과 결과 보고
src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java, docs/test-results/...
MinIO 객체, DB 상태, Job·Attempt·Event 순서, Chunk·Embedding, Vector 차원, 형식별 Metadata를 검증합니다. 중앙값과 지연 통계를 JSON 및 로그에 기록합니다.
통계 집계와 DOCX Fixture 지원
src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadStatistics.java, src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadStatisticsTest.java, src/test/java/com/opensource/docgrid/e2e/LocalE2eDocumentFactory.java
PDF·DOCX별 처리량과 p50·p95·p99 지연시간 집계를 추가했습니다. 불균형 표본과 잘못된 입력을 검증합니다. DOCX 다중 Section 생성을 지원합니다.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Benchmark as DocumentIndexingE2ELoadBenchmark
  participant Worker
  participant Storage as PostgreSQL and MinIO
  participant Embedding as BAAI/bge-m3
  Benchmark->>Worker: PDF·DOCX Multipart 업로드
  Worker->>Storage: 객체와 인덱싱 상태 저장
  Worker->>Embedding: 임베딩 생성 요청
  Embedding-->>Worker: Vector 반환
  Worker->>Storage: Chunk·Embedding 저장
  Storage-->>Benchmark: 인덱싱 완료 상태 반환
Loading

Possibly related PRs

  • DocGrid/backend#87: 인덱싱 완료 상태 전환과 searchable version 동작을 이 Benchmark가 검증합니다.
  • DocGrid/backend#94: Worker polling, lease, slot 제약 실행과 전체 인덱싱 경로를 이 Benchmark가 실행합니다.
  • DocGrid/backend#107: PDF·DOCX Parser와 인덱싱 경로를 실제 문서 생성·업로드로 검증합니다.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed 제목은 PDF·DOCX 전체 인덱싱 E2E 부하 검증이라는 PR의 핵심 변경을 명확하고 간결하게 설명합니다.
Description check ✅ Passed 설명은 변경 내용, 실측 결과, 검증 항목, 제외 범위와 이슈 연결을 충분히 포함합니다.
Linked Issues check ✅ Passed 구현과 문서는 이슈 #143의 혼합 부하, 전체 파이프라인, 정합성 검증, 통계, 전용 태스크 요구를 충족합니다.
Out of Scope Changes check ✅ Passed 변경은 이슈 #143의 벤치마크 테스트, 통계, 문서, Gradle 태스크와 직접 관련됩니다.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/143

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java (1)

232-232: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Worker 최대 동시성 계약이 세 곳에 흩어져 있습니다. 테스트는 동시성을 2로 고정 주입하고 같은 값을 다시 단정하며, 설계 문서는 제품 설정을 사용한다고 기술합니다. 값을 바꾸면 세 곳이 동시에 어긋납니다.

  • src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java#L232-L232: 상수 하나를 도입해 line 144의 주입값과 이 단정이 같은 정의를 참조하게 하십시오.
  • docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md#L59-L61: 고정 동시성 2를 사용한다는 사실과 그 이유를 명시하거나, 코드를 제품 설정 사용으로 바꾸십시오.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java`
at line 232, Centralize the worker concurrency value in
DocumentIndexingE2ELoadBenchmark by introducing one constant and reusing it for
both the line 144 injection and the getMaxConcurrency() assertion at lines
232-232. Update docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md
lines 59-61 to explicitly document the fixed concurrency of 2 and its rationale,
or change the benchmark implementation and documentation together to use the
product setting instead.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/design/gimin-`#143-pdf-docx-indexing-e2e-load-benchmark.md:
- Around line 59-61: Update the benchmark documentation’s worker-concurrency
description to match DocumentIndexingE2ELoadBenchmark’s forced
indexing.worker.max-concurrency value of 2, and state that this fixed value is
intentional with its rationale. Do not describe the benchmark as using the
product-configured maximum concurrency.

In
`@src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java`:
- Around line 396-402: Update awaitIndexedAndIdle to pass a deferred
Supplier<String> to awaitCondition so jobSnapshot(uploads) is evaluated when the
wait fails rather than before waiting; add or use an awaitCondition overload
accepting Supplier<String> while preserving existing string-call compatibility
and avoiding snapshot queries on successful waits.

---

Nitpick comments:
In
`@src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java`:
- Line 232: Centralize the worker concurrency value in
DocumentIndexingE2ELoadBenchmark by introducing one constant and reusing it for
both the line 144 injection and the getMaxConcurrency() assertion at lines
232-232. Update docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md
lines 59-61 to explicitly document the fixed concurrency of 2 and its rationale,
or change the benchmark implementation and documentation together to use the
product setting instead.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 20dc53d7-8755-4bff-beb3-a68913801fc5

📥 Commits

Reviewing files that changed from the base of the PR and between 83c08df and c605714.

📒 Files selected for processing (7)
  • build.gradle
  • docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md
  • docs/test-results/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md
  • src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadBenchmark.java
  • src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadStatistics.java
  • src/test/java/com/opensource/docgrid/e2e/DocumentIndexingE2ELoadStatisticsTest.java
  • src/test/java/com/opensource/docgrid/e2e/LocalE2eDocumentFactory.java

Comment thread docs/design/gimin-#143-pdf-docx-indexing-e2e-load-benchmark.md Outdated
@Gimini-3
Gimini-3 merged commit 168f831 into develop Aug 10, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PDF·DOCX 50·100문서 전체 인덱싱 E2E 부하 검증 지원 추가 구현

1 participant