Skip to content

fix(daemon-state): detect PID reuse so stale state files aren't treated as live daemons - #67

Merged
Rinse12 merged 1 commit into
masterfrom
fix/daemon-state-pid-reuse
Jun 8, 2026
Merged

fix(daemon-state): detect PID reuse so stale state files aren't treated as live daemons#67
Rinse12 merged 1 commit into
masterfrom
fix/daemon-state-pid-reuse

Conversation

@Rinse12

@Rinse12 Rinse12 commented Jun 7, 2026

Copy link
Copy Markdown
Member

Closes #66

Bug

isPidAlive() in src/common-utils/daemon-state.ts only checked process.kill(pid, 0). A PID being alive does not prove the process is the bitsocial daemon that wrote the state file (classic stale-pidfile / PID-reuse hazard).

Observed in the wild: a daemon running inside a Docker container (PID 8 in the container's PID namespace) wrote its state file into the bind-mounted data dir. The container died without graceful shutdown; on the host, PID 8 is a kernel thread — alive but unrelated. bitsocial update install then:

  1. Sent SIGINT to the unrelated process (on a different day this could be any innocent process)
  2. Restarted 2 daemons with identical args on the same port; the second died with EADDRINUSE
  3. Falsely reported "Daemon started" for the second spawn because waitUntilUsed saw the first daemon's port

Fix

Two layers in daemon-state.ts:

  1. New state files record the OS-reported process start time (procStartTime: /proc/<pid>/stat field 22 on Linux, ps -o lstart= fallback elsewhere). Aliveness checks compare it — a reused PID has a different start time, so the state is pruned.
  2. Legacy state files (no procStartTime) fall back to requiring the process command line to reference bitsocial. Kernel threads have an empty cmdline, so the original failure case is correctly pruned. If identity can't be determined at all, behavior falls back to the old liveness-only check (fail-safe: never treats a real running daemon as dead).

Tests

The bug was reproduced in a regression test before fixing (failed on unfixed code):

  • prune a legacy state file whose PID now belongs to an alive but unrelated process (byte-for-byte the prod scenario, with sleep standing in for the kernel thread)
  • prune a state file whose recorded procStartTime doesn't match the process now under that PID
  • keep a legacy state file whose PID is a genuine bitsocial-daemon-like process (no false pruning on upgrade from older CLI versions)

Spawn-based tests await once(child, "spawn") and use a compound bash command to avoid the fork/exec race and bash's exec-optimization (caught as a flake in a full-suite run).

Full suite: 244 passed, 1 skipped.

Summary by CodeRabbit

  • Bug Fixes
    • Improved daemon state detection to reliably identify stale daemon processes and prevent confusion from OS process ID reuse.
    • Enhanced cleanup of obsolete daemon state files with support for both current and legacy configurations.

…ed as live daemons (#66)

A bare process.kill(pid, 0) liveness check let a stale state file —
written by a daemon inside a Docker container (PID 8 in its namespace)
into the bind-mounted data dir — match a kernel thread on the host.
`update install` then SIGINT'd the unrelated process and restarted the
daemon twice on the same port.

State files now record the OS-reported process start time
(procStartTime, /proc/<pid>/stat field 22, `ps -o lstart=` fallback);
aliveness checks compare it, so a reused PID is detected and pruned.
Legacy state files without the field fall back to requiring that the
process command line references bitsocial — kernel threads have an
empty cmdline and are correctly pruned.
@coderabbitai

coderabbitai Bot commented Jun 7, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Daemon state files now store and verify OS process identity (procStartTime) to prevent PID-reuse misidentification. DaemonState gains an optional procStartTime field populated at write time. A new isDaemonStateAlive validator compares stored vs. live process start times; legacy files without procStartTime fall back to command-line heuristic (bitsocial marker). New regression tests validate PID reuse pruning and legacy state preservation.

Changes

Process Identity Verification for Daemon State

Layer / File(s) Summary
Process identity capture and state model
src/common-utils/daemon-state.ts
Added execFileAsync utility for ps-based process introspection fallback. Extended DaemonState interface with optional procStartTime field. Introduced getProcessStartTime and getProcessCommandLine helpers to read live process identity from /proc/<pid>/stat or ps command. Updated writeDaemonState to populate procStartTime automatically when missing.
State validation with identity checking
src/common-utils/daemon-state.ts
Implemented isDaemonStateAlive async function that validates daemon state by comparing stored procStartTime against the live process start time; falls back to command-line heuristic (checks for bitsocial in /proc/<pid>/cmdline or ps args) for legacy files without procStartTime, with final fallback to PID liveness check. Refactored pruneStaleStates to delegate to getAliveDaemonStates. Updated getAliveDaemonStates to use isDaemonStateAlive for filtering and on-disk deletion of stale entries.
Regression tests for PID reuse detection
test/common-utils/daemon-state.test.ts
Added test imports for spawn, process events, and defaults configuration. New getAliveDaemonStates regression suite spawns helper processes (non-daemon sleep and bash with bitsocial marker), writes legacy and procStartTime-mismatched state files to the real .daemon_states directory, and asserts that stale entries are pruned and deleted from disk while legitimate legacy entries (identified by command-line marker) are preserved.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • bitsocialnet/bitsocial-cli#22: Introduced the initial daemon state file infrastructure and getAliveDaemonStates pruning; this PR extends it with process identity verification to detect PID reuse.

Poem

🐰 A PID reused, the daemon confused,
But now we record when each process was born,
With start times and markers to guide what is worn,
No ghost daemons haunt us, no PIDs abused,
Just clean slate states and fresh morning morn! 🌅

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: adding PID reuse detection to prevent stale daemon state files from being treated as live.
Linked Issues check ✅ Passed All coding requirements from #66 are met: procStartTime is recorded, aliveness checks compare start times, legacy files use command-line heuristic with liveness fallback, and regression tests verify PID-reuse scenarios.
Out of Scope Changes check ✅ Passed All changes in daemon-state.ts and daemon-state.test.ts are directly scoped to implementing PID reuse detection and adding regression tests as specified in #66.
Docstring Coverage ✅ Passed Docstring coverage is 87.50% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/daemon-state-pid-reuse

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

src/common-utils/daemon-state.ts

Oops! Something went wrong! :(

ESLint: 8.27.0

Error: ESLint configuration in --config » eslint-config-oclif is invalid:

  • Unexpected top-level property "__esModule".

Referenced from: /.eslintrc
at ConfigValidator.validateConfigSchema (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2156:19)
at ConfigArrayFactory._normalizeConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2998:19)
at ConfigArrayFactory._loadConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2963:21)
at ConfigArrayFactory._loadExtendedShareableConfig (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3264:21)
at ConfigArrayFactory._loadExtends (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3135:25)
at ConfigArrayFactory._normalizeObjectConfigDataBody (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3074:25)
at _normalizeObjectConfigDataBody.next ()
at ConfigArrayFactory._normalizeObjectConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3019:20)
at _normalizeObjectConfigData.next ()
at ConfigArrayFactory.loadFile (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2829:16)

test/common-utils/daemon-state.test.ts

Oops! Something went wrong! :(

ESLint: 8.27.0

Error: ESLint configuration in --config » eslint-config-oclif is invalid:

  • Unexpected top-level property "__esModule".

Referenced from: /.eslintrc
at ConfigValidator.validateConfigSchema (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2156:19)
at ConfigArrayFactory._normalizeConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2998:19)
at ConfigArrayFactory._loadConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2963:21)
at ConfigArrayFactory._loadExtendedShareableConfig (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3264:21)
at ConfigArrayFactory._loadExtends (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3135:25)
at ConfigArrayFactory._normalizeObjectConfigDataBody (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3074:25)
at _normalizeObjectConfigDataBody.next ()
at ConfigArrayFactory._normalizeObjectConfigData (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:3019:20)
at _normalizeObjectConfigData.next ()
at ConfigArrayFactory.loadFile (/node_modules/@eslint/eslintrc/dist/eslintrc.cjs:2829:16)


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/common-utils/daemon-state.ts`:
- Around line 54-55: The current fallback uses `const { stdout } = await
execFileAsync("ps", ["-p", String(pid), "-o", "args="]); return stdout.trim() ||
undefined;` which turns an empty `ps` args result into `undefined`; change the
return to preserve an empty string (so the caller can distinguish empty args
from missing info) by returning the raw/trimmed-end stdout (e.g. `return
stdout.trimEnd();` or `return stdout.replace(/\r?\n$/, '');`) instead of `||
undefined`. Update the return expression at the `execFileAsync` call in
daemon-state.ts so empty `args=` remains `""`.

In `@test/common-utils/daemon-state.test.ts`:
- Around line 7-8: The tests are touching the real defaults.PKC_DATA_PATH
causing potential data loss; change daemon-state.ts to accept an injected data
directory (e.g., add an optional parameter or constructor arg like dataPath or
allow reading from an overridable env var) and use that path instead of directly
importing defaults.PKC_DATA_PATH inside getAliveDaemonStates() and related
functions; then update daemon-state.test.ts to create a temporary directory
(fs.mkdtemp/tmpdir) and pass that temp path into the daemon-state API (or set
the env var or mock defaults.PKC_DATA_PATH) before writing test files and
calling getAliveDaemonStates() so tests operate only on the isolated temp
directory.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: aac5935d-afb2-4729-a4fd-95779d55f9ef

📥 Commits

Reviewing files that changed from the base of the PR and between afd8bca and fe3609e.

📒 Files selected for processing (2)
  • src/common-utils/daemon-state.ts
  • test/common-utils/daemon-state.test.ts

Comment on lines +54 to +55
const { stdout } = await execFileAsync("ps", ["-p", String(pid), "-o", "args="]);
return stdout.trim() || undefined;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Preserve empty ps command lines in the fallback path.

This collapses an empty args= result into undefined, so Line 133 falls back to liveness-only instead of treating the PID as a non-daemon match. On systems where /proc is unavailable, that reopens the stale-state false positive this PR is trying to eliminate.

Suggested fix
         try {
             const { stdout } = await execFileAsync("ps", ["-p", String(pid), "-o", "args="]);
-            return stdout.trim() || undefined;
+            return stdout.trimEnd();
         } catch {
             return undefined;
         }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/common-utils/daemon-state.ts` around lines 54 - 55, The current fallback
uses `const { stdout } = await execFileAsync("ps", ["-p", String(pid), "-o",
"args="]); return stdout.trim() || undefined;` which turns an empty `ps` args
result into `undefined`; change the return to preserve an empty string (so the
caller can distinguish empty args from missing info) by returning the
raw/trimmed-end stdout (e.g. `return stdout.trimEnd();` or `return
stdout.replace(/\r?\n$/, '');`) instead of `|| undefined`. Update the return
expression at the `execFileAsync` call in daemon-state.ts so empty `args=`
remains `""`.

Comment on lines +7 to 8
import defaults from "../../dist/common-utils/defaults.js";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | 🏗️ Heavy lift

Don't run these regressions against the real daemon-state directory.

These cases write raw files under defaults.PKC_DATA_PATH and then call getAliveDaemonStates(), which deletes stale entries from that same directory. If a developer has real daemon state on disk, this test can prune unrelated files or interfere with a live local daemon. Please redirect daemon-state.ts to an injected temp directory under test instead of touching the default path.

Also applies to: 126-189

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/common-utils/daemon-state.test.ts` around lines 7 - 8, The tests are
touching the real defaults.PKC_DATA_PATH causing potential data loss; change
daemon-state.ts to accept an injected data directory (e.g., add an optional
parameter or constructor arg like dataPath or allow reading from an overridable
env var) and use that path instead of directly importing defaults.PKC_DATA_PATH
inside getAliveDaemonStates() and related functions; then update
daemon-state.test.ts to create a temporary directory (fs.mkdtemp/tmpdir) and
pass that temp path into the daemon-state API (or set the env var or mock
defaults.PKC_DATA_PATH) before writing test files and calling
getAliveDaemonStates() so tests operate only on the isolated temp directory.

@Rinse12
Rinse12 merged commit f631b1c into master Jun 8, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale daemon state file with reused PID causes update install to signal wrong process and double-restart daemon

1 participant