Skip to content

jepsen: avoid S3 checker timeouts - #1181

Merged
bootjp merged 1 commit into
mainfrom
feature/fix-s3-jepsen-checker
Aug 18, 2026
Merged

jepsen: avoid S3 checker timeouts#1181
bootjp merged 1 commit into
mainfrom
feature/fix-s3-jepsen-checker

Conversation

@bootjp

@bootjp bootjp commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • let the S3 register workload use Knossos's default competition checker, which races the linear and WGL analyzers
  • preserve the scheduled workload's runtime, rate, concurrency, and timeout
  • upload the Jepsen store after the S3 step so S3 failures retain their history
  • add regression coverage for the competition-checker configuration

Root cause

Scheduled Jepsen run #563 completed the S3 workload but timed out during analysis with exit code 124. The S3 checker forced Knossos's linear analyzer, whose search space can grow pathologically when a history contains indeterminate writes. The run did not report a linearizability violation before the timeout.

Using Knossos's default competition checker preserves the verdict while returning as soon as either the linear or WGL analyzer finishes.

Impact

The S3 workload parameters and register model are unchanged. Only the analyzer selection changes. Future S3 failures will also retain the Jepsen store artifact for diagnosis.

Validation

  • cd jepsen && lein test — 148 tests, 340 assertions
  • replayed the six complete per-key histories reconstructed from run perf(redis): reorder rawKeyTypeAt to probe string prefixes first #563 — all valid, 62–104 ms per key
  • pre-commit golangci-lint — 0 issues
  • actionlint -ignore SC2034 .github/workflows/jepsen-test-scheduled.yml
  • git diff --check

Summary by CodeRabbit

  • テスト

    • S3ワークロードの検証に、レジスタの線形化可能性とHTMLタイムラインを確認するチェッカーを追加しました。
    • チェッカー構成が意図どおり動作することを確認するテストを追加しました。
  • 改善

    • S3ワークロード完了後に、検証用アーティファクトを保存するよう変更しました。

@coderabbitai

coderabbitai Bot commented Aug 18, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b2367866-e233-4d1e-905c-fc35f5bd2acb

📥 Commits

Reviewing files that changed from the base of the PR and between 2705269 and 661a32d.

📒 Files selected for processing (3)
  • .github/workflows/jepsen-test-scheduled.yml
  • jepsen/src/elastickv/s3_workload.clj
  • jepsen/test/elastickv/s3_workload_test.clj

Included review availability: Your plan includes up to 1 review per rolling hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

S3 レジスタ用チェッカーを関数化し、線形化可能性チェッカーの明示的な :algorithm 指定を削除しました。S3 ワークロード後に、失敗時の jepsen/store 保存を実行します。

Changes

S3 Jepsen 更新

Layer / File(s) Summary
S3 レジスタチェッカーの構成
jepsen/src/elastickv/s3_workload.clj, jepsen/test/elastickv/s3_workload_test.clj
s3-register-checkerを追加しました。レジスタモデルによる線形化可能性チェッカーとHTMLタイムラインチェッカーを構成します。テストは :algorithm が未設定であることを検証します。
S3 ワークロード成果物の保存
.github/workflows/jepsen-test-scheduled.yml
失敗時の jepsen/store アップロードをS3ワークロードの後に配置しました。成果物を jepsen-store-types として7日間保存します。

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 661a3

The PR changes only the S3 analysis strategy while preserving workload behavior and adds regression coverage; no actionable merge-blocking risk remains after normal checks and review.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed タイトルは、S3チェッカーのタイムアウト回避というプルリクエストの主目的を簡潔かつ具体的に示しています。
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bootjp
bootjp marked this pull request as ready for review August 18, 2026 13:54
@bootjp

bootjp commented Aug 18, 2026

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: 661a32d76e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@bootjp
bootjp merged commit 5d96d78 into main Aug 18, 2026
8 of 9 checks passed
@bootjp
bootjp deleted the feature/fix-s3-jepsen-checker branch August 18, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant