Skip to content

Implement the timeout that DLR0 documents - #121

Merged
thawk105 merged 1 commit into
masterfrom
ss2pl-dlr0-timeout
Aug 26, 2026
Merged

Implement the timeout that DLR0 documents#121
thawk105 merged 1 commit into
masterfrom
ss2pl-dlr0-timeout

Conversation

@thawk105

Copy link
Copy Markdown
Owner

cc/ss2pl/README.md says "DLR0 : Dead lock resolution is timeout", but the DLR0 branches in cc/ss2pl/transaction.cc just called the blocking r_lock() / w_lock() / tryupgrade(). There was no timeout and no other resolution mechanism, so a DLR0 build waits forever once a lock cycle forms. There was also no way to select DLR0: cc/ss2pl/CMakeLists.txt passed the DLR1 marker literally.

Reproduced on a 48-core host with YCSB-A, 1,000,000 records, zipf skew 0.9 and 48 threads: the run never terminates, and the wait-for graph has a cycle.

  • cmake/Options.cmake gains CCBENCH_SS2PL_DLR, and cc/ss2pl/CMakeLists.txt turns it into the DLR0 or DLR1 marker. The default stays 1, so the default build is byte-for-byte the same protocol behaviour as before. An out-of-range value is a configure-time FATAL_ERROR.
  • cc/ss2pl/include/dlr0_timeout.hh (new, DLR0-only) provides a deadline loop and three single-attempt lock helpers. The helpers exist because ReaderWriteLock::r_trylock() and friends retry internally after a failed compare-exchange, so a caller can stay inside them and never re-check its deadline. The new helpers do one load and at most one compare-exchange and always return, which is what makes the timeout an actual bound. include/rwlock.hh is not modified: it is shared with cc/d2pl, and the meaning of its existing methods stays exactly as it was.
  • all five DLR0 acquisition sites in cc/ss2pl/transaction.cc now go through that deadline loop. On expiry they set status_ = aborted and join the same abort path as a no-wait abort, so the lock lists are released by abort() -> unlockList() exactly as before. No path proceeds without the lock.
  • read() and update() gain a DLR0-only guard that reports the failure to the caller as Status::ERROR_LOCK_FAILED instead of falling through to Status::OK. Without it, a timed-out read() leaves the caller's TupleBody* unset while claiming success, and a timed-out update() lets TPC-C's NewOrder continue into insert_order().
  • the timeout is the runtime flag -ss2pl_dlr0_timeout_us (default 1000), converted to TSC ticks with -clocks_per_us; chkArg() rejects values whose product would wrap. docs/runtime-args_ja.md and docs/runtime-args_en.md document it, and cc/ss2pl/README.md now describes DLR0 as implemented.

Choosing the default, measured on an Intel Xeon Platinum 8468 (48 cores), YCSB, 48 threads, 5 s runs:

| -ss2pl_dlr0_timeout_us | spurious aborts, uniform | throughput, zipf 0.9 | |---|---|---|
| 10 | 424 | 1,118,740 tps |
| 100 | 144 | 396,819 tps |
| 1000 | 76 | 6,194 tps |
| 10000 | 0 | 467 tps |

"spurious aborts, uniform" is the abort count at 1,000,000 records with zipf skew 0, where there is no cycle to resolve, so every abort is the timeout firing on a wait that would have succeeded. At the default that is 76 aborts against 42,342,027 commits (1.8e-6), while 10 ms removes them entirely at the cost of 13x throughput under contention. DLR1 on the same uniform workload aborts 124,010 times. The flag is a runtime knob, so this default only has to be a sane starting point.

Verified on the same host, -DCMAKE_BUILD_TYPE=Release with the project's -Wall -Wextra -Werror:

  • every protocol builds under g++ 11.4.0 and g++ 12, with CCBENCH_SS2PL_DLR at both 0 and 1 (four combinations).
  • the known hang (YCSB-A, 1M records, zipf 0.9, 48 threads, -extime=5) exits 0 in 6 s under DLR0. 100 records with 48 threads exits 0 in 5 s.
  • the same DLR0 run with -ss2pl_dlr0_timeout_us=3600000000 is still running when a 120 s watchdog kills it, which is what makes the previous line evidence that the timeout is doing the work.
  • timeout aborts are counted: 100 records, 48 threads, -ycsb_rratio=0, -ycsb_rmw=false, zipf 0.9 - a blind-write-only mix whose operation aborts can only come from write-lock timeouts - reports abort_counts_: 367506.
  • the default (DLR1) build is unchanged: for ycsb_ss2pl.exe, tpcc_ss2pl.exe and bomb_ss2pl.exe the .text section has the same size as a build of the parent commit, and the disassembly differs in exactly two instructions per binary. Both are the __LINE__ immediate that NNN in include/debug.hh prints on a fatal-error path, shifted by exactly the number of lines added above them. Control flow, calls and data are identical.
  • the clang-format --dry-run --Werror check from .github/workflows/format.yml passes with clang-format 14.

Not verified: GCC 13. The CI image uses it; it was not available on the test machine, so the -Werror result under the CI compiler is unconfirmed.

Two pre-existing problems are visible from this work and are deliberately not addressed here, because fixing either changes the DLR1 code this commit promises to leave alone:

  • include/bomb_pessimistic.hh reads a TupleBody* without checking the return value or status_, so a failed lock still reaches a dereference.
  • TxExecutor::insert() publishes into the index immediately, commit() treats OpType::INSERT as a no-op, and abort() removes nothing, while the matching D_NEXT_O_ID update is only applied at commit. An aborted transaction therefore leaves its inserted rows behind. (tpcc_ss2pl.exe reports "insert order failed" heavily even single-threaded, on the parent commit as well as here; whether the asymmetry above is what causes that was not established, so it is reported as a code observation only.)

This change was written with AI assistance.

`cc/ss2pl/README.md` says "DLR0 : Dead lock resolution is timeout", but the
DLR0 branches in `cc/ss2pl/transaction.cc` just called the blocking
`r_lock()` / `w_lock()` / `tryupgrade()`. There was no timeout and no other
resolution mechanism, so a DLR0 build waits forever once a lock cycle forms.
There was also no way to select DLR0: `cc/ss2pl/CMakeLists.txt` passed the
`DLR1` marker literally.

Reproduced on a 48-core host with YCSB-A, 1,000,000 records, zipf skew 0.9 and
48 threads: the run never terminates, and the wait-for graph has a cycle.

- `cmake/Options.cmake` gains `CCBENCH_SS2PL_DLR`, and `cc/ss2pl/CMakeLists.txt`
  turns it into the `DLR0` or `DLR1` marker. **The default stays `1`, so the
  default build is byte-for-byte the same protocol behaviour as before.** An
  out-of-range value is a configure-time `FATAL_ERROR`.
- `cc/ss2pl/include/dlr0_timeout.hh` (new, DLR0-only) provides a deadline loop
  and three single-attempt lock helpers. The helpers exist because
  `ReaderWriteLock::r_trylock()` and friends retry internally after a failed
  compare-exchange, so a caller can stay inside them and never re-check its
  deadline. The new helpers do one load and at most one compare-exchange and
  always return, which is what makes the timeout an actual bound.
  `include/rwlock.hh` is not modified: it is shared with `cc/d2pl`, and the
  meaning of its existing methods stays exactly as it was.
- all five DLR0 acquisition sites in `cc/ss2pl/transaction.cc` now go through
  that deadline loop. On expiry they set `status_ = aborted` and join the same
  abort path as a no-wait abort, so the lock lists are released by
  `abort()` -> `unlockList()` exactly as before. No path proceeds without the
  lock.
- `read()` and `update()` gain a DLR0-only guard that reports the failure to
  the caller as `Status::ERROR_LOCK_FAILED` instead of falling through to
  `Status::OK`. Without it, a timed-out `read()` leaves the caller's
  `TupleBody*` unset while claiming success, and a timed-out `update()` lets
  TPC-C's NewOrder continue into `insert_order()`.
- the timeout is the runtime flag `-ss2pl_dlr0_timeout_us` (default 1000),
  converted to TSC ticks with `-clocks_per_us`; `chkArg()` rejects values whose
  product would wrap. `docs/runtime-args_ja.md` and `docs/runtime-args_en.md`
  document it, and `cc/ss2pl/README.md` now describes DLR0 as implemented.

Choosing the default, measured on an Intel Xeon Platinum 8468 (48 cores),
YCSB, 48 threads, 5 s runs:

| `-ss2pl_dlr0_timeout_us` | spurious aborts, uniform | throughput, zipf 0.9 |
|---|---|---|
| 10    | 424 | 1,118,740 tps |
| 100   | 144 |   396,819 tps |
| 1000  |  76 |     6,194 tps |
| 10000 |   0 |       467 tps |

"spurious aborts, uniform" is the abort count at 1,000,000 records with zipf
skew 0, where there is no cycle to resolve, so every abort is the timeout
firing on a wait that would have succeeded. At the default that is 76 aborts
against 42,342,027 commits (1.8e-6), while 10 ms removes them entirely at the
cost of 13x throughput under contention. DLR1 on the same uniform workload
aborts 124,010 times. The flag is a runtime knob, so this default only has to
be a sane starting point.

Verified on the same host, `-DCMAKE_BUILD_TYPE=Release` with the project's
`-Wall -Wextra -Werror`:

- every protocol builds under g++ 11.4.0 and g++ 12, with
  `CCBENCH_SS2PL_DLR` at both 0 and 1 (four combinations).
- the known hang (YCSB-A, 1M records, zipf 0.9, 48 threads, `-extime=5`) exits
  0 in 6 s under DLR0. 100 records with 48 threads exits 0 in 5 s.
- the same DLR0 run with `-ss2pl_dlr0_timeout_us=3600000000` is still running
  when a 120 s watchdog kills it, which is what makes the previous line
  evidence that the timeout is doing the work.
- timeout aborts are counted: 100 records, 48 threads, `-ycsb_rratio=0`,
  `-ycsb_rmw=false`, zipf 0.9 - a blind-write-only mix whose operation aborts
  can only come from write-lock timeouts - reports `abort_counts_: 367506`.
- the default (DLR1) build is unchanged: for `ycsb_ss2pl.exe`,
  `tpcc_ss2pl.exe` and `bomb_ss2pl.exe` the `.text` section has the same size
  as a build of the parent commit, and the disassembly differs in exactly two
  instructions per binary. Both are the `__LINE__` immediate that `NNN` in
  `include/debug.hh` prints on a fatal-error path, shifted by exactly the
  number of lines added above them. Control flow, calls and data are identical.
- the `clang-format --dry-run --Werror` check from
  `.github/workflows/format.yml` passes with clang-format 14.

Not verified: GCC 13. The CI image uses it; it was not available on the test
machine, so the `-Werror` result under the CI compiler is unconfirmed.

Two pre-existing problems are visible from this work and are deliberately not
addressed here, because fixing either changes the DLR1 code this commit
promises to leave alone:

- `include/bomb_pessimistic.hh` reads a `TupleBody*` without checking the
  return value or `status_`, so a failed lock still reaches a dereference.
- `TxExecutor::insert()` publishes into the index immediately, `commit()`
  treats `OpType::INSERT` as a no-op, and `abort()` removes nothing, while the
  matching `D_NEXT_O_ID` update is only applied at commit. An aborted
  transaction therefore leaves its inserted rows behind. (`tpcc_ss2pl.exe`
  reports "insert order failed" heavily even single-threaded, on the parent
  commit as well as here; whether the asymmetry above is what causes that was
  not established, so it is reported as a code observation only.)

This change was written with AI assistance.
@thawk105
thawk105 merged commit 7e73030 into master Aug 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant