test: add deterministic failure injection tests for core services

[devflora-princess]

Overview

This PR introduces controlled failure injection for the RPC, database, scheduler, and notification-provider components, along with automated recovery verification and deterministic tests. It adds a small failure-injection harness, wires it into the affected service clients, and covers the scenarios with Jest tests.

Related Issue

Changes

🧪 Failure Injection Harness

• [ADD] dashboard/src/test/failureInjection.ts
  • Central helpers to register and trigger controlled failures per component (RPC, database, scheduler, notification provider).
  • Deterministic toggles so tests can enable/disable a failure for a specific call without affecting unrelated ones.
  • Reset utilities so each test starts from a clean state.

🧩 Service Integration

• [MODIFY] dashboard/src/services/eventsApi.ts
• [MODIFY] dashboard/src/services/notificationHealthApi.ts
• [MODIFY] dashboard/src/services/indexingHealthApi.ts
• [MODIFY] dashboard/src/services/retryStatisticsApi.ts
• [MODIFY] dashboard/src/services/webhookApi.ts
  • Route the relevant RPC, database, scheduler, and notification-provider calls through the failure-injection harness so individual failures can be simulated.
  • Preserve existing success-path behavior when no failure is registered.

✅ Tests

• [ADD] dashboard/src/services/failureInjection.test.ts

  • Simulates individual failures for each component and asserts the expected error/fallback behavior.
  • Verifies automatic recovery once the injected failure is cleared.
  • Uses deterministic setup/teardown so runs are repeatable.

• [MODIFY] dashboard/jest.setup.cjs

  • Ensures the failure-injection state is reset between tests for deterministic results.

Verification Results

npm test -- dashboard/src/services/failureInjection.test.ts
✅ Failure injection tests passing

Covered scenarios:
✅ RPC failure simulated and recovered
✅ Database failure simulated and recovered
✅ Scheduler failure simulated and recovered
✅ Notification-provider failure simulated and recovered
✅ Deterministic across repeated runs

Acceptance Criteria	Status
Individual failures can be simulated	✅ Per-component failure toggles in failureInjection.ts, exercised by service clients
Recovery behavior is verified automatically	✅ Tests assert recovery after injected failures are cleared
Tests remain deterministic	✅ Reset between tests via jest.setup.cjs and harness reset helpers

Closes #857

[drips-wave]

@devflora-princess Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits