Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,25 @@ jobs:
if: matrix.os == 'ubuntu-latest'
shell: bash
run: bash scripts/smoke-shutdown.sh

# Racket port gate (ADR 0002, Phase 0+): the ported tree must stay green
# on the same OS matrix as the C# implementation.
racket-port:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
steps:
- uses: actions/checkout@v7
- name: Setup Racket
uses: Bogdanp/setup-racket@v1.15
with:
architecture: x64
distribution: full
variant: CS
version: '9.3'
- name: Install package
run: raco pkg install --auto --name benchpilot --link racket
- name: Test
run: raco test racket/benchpilot
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@ obj/
# OS
.DS_Store
Thumbs.db

# Racket
compiled/
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@

All notable changes to BenchPilot are documented here.

## Unreleased

Racket port kickoff ([ADR 0002](docs/adr/0002-racket-port.md)):

- Decision: port the runtime from .NET/C# to Racket in staged phases under
the frozen JSON API / CLI / E2E contract; the C# tree remains the shipping
implementation until the port completes.
- Added `racket/` with the first ported slice (bench profile model, loader,
normalization, validation and the pure readiness helpers), covered by 29
contract tests; CI runs the ported tree on Windows and Linux.
- Note: v0.5.1 release notes below describe the C#-based 0.5.x line.

## 0.5.1 - 2026-09-28

Self-update: the CLI upgrades itself from the release feed.
Expand Down
17 changes: 11 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -428,6 +428,8 @@ benchpilot/
│ └── Benchpilot.Drivers.ScpiPower/ # TCP SCPI power-supply adapter
├── tests/
│ └── Benchpilot.Core.Tests/
├── racket/
│ └── benchpilot/core/ # Racket port (ADR 0002): profiles + readiness
├── profiles/
│ ├── demo.profile.json
│ └── real-ecu.example.json
Expand All @@ -454,13 +456,16 @@ Near-term work remains a vertical slice rather than broad protocol coverage:

See [ROADMAP.md](ROADMAP.md).

## GUI and language strategy
## Implementation language

The hardware-facing Runtime is implemented in .NET/C# to reduce native/vendor integration risk. GUI technology is intentionally decoupled from Runtime.

That means an Avalonia frontend is a conservative option, while a Racket/Glaze frontend remains viable if it demonstrates a concrete development-speed or UX advantage. Both would use `Benchpilot.Client` / the same versioned local API rather than owning devices.

See [ADR 0001](docs/adr/0001-runtime-language.md).
The shipped Runtime is implemented in .NET/C#, and the project is porting it
to **Racket** ([ADR 0002](docs/adr/0002-racket-port.md), which supersedes
[ADR 0001](docs/adr/0001-runtime-language.md)). The port proceeds in staged
phases under a frozen API contract: until its final phase completes, the C#
tree remains the shipping implementation and the Racket sources under
`racket/` are the port under construction. GUI technology stays decoupled
from the Runtime; any future Studio GUI would speak the same versioned local
API rather than owning devices.

## Long-term flashing direction

Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0001-runtime-language.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# ADR 0001: Use .NET/C# for the hardware runtime; keep GUI technology decoupled

- Status: Accepted
- Status: Superseded by [ADR 0002](0002-racket-port.md) (2026-10-01), via this ADR's revisit criteria
- Date: 2026-09-20

## Context
Expand Down
124 changes: 124 additions & 0 deletions docs/adr/0002-racket-port.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
# ADR 0002: Port the BenchPilot runtime from .NET/C# to Racket

- Status: Accepted
- Date: 2026-10-01
- Supersedes: [ADR 0001](0001-runtime-language.md), via that ADR's own revisit criteria

## Context

ADR 0001 chose .NET/C# to minimize hardware-integration risk. Six weeks later,
the facts that decision relied on have changed:

1. **The C# tree holds no hardware-validated knowledge.** The runtime was
written simulator-only. CI proves the protocol stacks are self-consistent
against a simulated ECU; nothing has been validated against a real bench.
The expensive part of a hardware stack — real-ECU validation, timing
quirks, vendor workarounds — has not been paid down in any language.
2. **The native-interop surface turned out to be small.** ADR 0001 bet on
"low-friction P/Invoke and vendor SDK coverage". Measured reality: exactly
two P/Invoke files (`SocketCanBus`, `PcanBus`); J-Link and SCPI are
process/serial wrappers. The argument for .NET never got cashed.
3. **The Racket ecosystem investment matured.** rivet 0.5.0 (records/enums,
schema gates, six-architecture matrix), the glaze platform hardening and
the taskly rebuild are shipped and maintained first-party.
4. **The port cost is at its historical minimum right now.** 8.2k LOC of
unvalidated C# plus 3.8k LOC of tests plus a frozen JSON API make this a
tests-as-spec port, not a rewrite from requirements. Every week of
real-ECU validation from here on adds C#-specific fixes and raises the
port cost continuously.

ADR 0001 listed revisit criteria. Criterion three — "a stable service boundary
makes the implementation language of a subsystem irrelevant enough to choose
another language for that subsystem" — is met: the flat JSON result contract,
the CLI exit-code table and the E2E suite *are* that boundary. Criterion one
(first-class Racket bindings for the hardware ecosystem) is what this port
creates instead of waits for.

## Decision

Port BenchPilot to **Racket (Racket CS)**, staged, inside this repository under
`racket/`, with the existing test suite as the acceptance spec.

### The frozen contract

The Racket port must be contract-identical to the C# implementation. Normative
artifacts:

- JSON API shapes: `src/Benchpilot.Protocol/ApiModels.cs` (camelCase keys,
flat `Ok`-first records, stable error codes);
- CLI exit codes and stdout behavior: the "CLI exit codes" section of
`README.md`;
- behavioral semantics: `tests/Benchpilot.E2E.Tests` (real process/HTTP
boundary) and the per-project unit tests.

"Ported test green" means the ported test, unchanged in its assertions,
passes against the Racket implementation.

### Rivet's role

Rivet is a first-party native **GUI application** framework (WinUI 3 /
SwiftUI / GTK4 hosts around a Racket backend). BenchPilot is a headless
product: its clients are terminals, agents and CI. The daemon and CLI are
therefore plain Racket programs, not rivet apps — forcing a UI-host model
onto a headless runtime would be framework contortion.

Rivet enters this product at two points:

- contract discipline: schema-governed records/enums for the core surfaces
where it fits without changing the wire format;
- `Benchpilot.Studio` (README product priority 5), if and when the GUI
becomes real, as a rivet native shell over the same frozen API.

### Staged plan with machine-checkable gates

Each phase ends in a gate that CI or a script can verify. The C# tree remains
the shipping/fix line until Phase 5 completes.

| Phase | Scope | Gate |
| --- | --- | --- |
| 0 — this ADR | `racket/` scaffold, CI job (`raco test racket/`), profile core | ported profile tests green on Windows + Linux CI |
| 1 — core | profiles, readiness/preflight semantics, safety policy, result contracts | ported `Benchpilot.Core.Tests` green |
| 2 — protocol | ISO-TP codec/endpoint, UDS client (P2/P2\*/NRC 0x78), DoIP client, simulated CAN | ported `Benchpilot.Diagnostics.Tests` green hardware-free in CI — protocol parity proven without hardware |
| 3 — runtime + host | mutation gates, deadlines, cancellation, audit/evidence, observations, token auth, loopback HTTP API, autostart, graceful shutdown | ported `Benchpilot.E2E.Tests` green against the Racket daemon on Windows + Linux CI |
| 4 — drivers | serial FFI, SocketCAN FFI, PCAN-Basic FFI, J-Link wrapper, SCPI driver | simulator-backed suite green; hardware smoke on the real bench when available |
| 5 — product surface | CLI, MCP adapter, self-update, packaging (native executables per platform, installer/deb), docs and agent skill parity; the human-facing evidence layer (`--format html` static reports, designed 2026-10-01) lands on the Racket CLI | full parity: release matrix builds, E2E green, docs updated; C# tree removed; release v0.6.0 |

Porting order inside each phase is bottom-up: data contracts first, then pure
logic, then I/O — so every step lands testable.

### Abort criteria

The staged gates exist so the port can stop without leaving a half-shipped
product. Abort the port (and keep C#) if:

- serial or CAN FFI on Windows proves materially worse than the existing
P/Invoke path in Phase 4 hardware smoke (throughput, latency or stability);
- the Phase 3 E2E parity stall exceeds the time real-ECU validation would
have taken on the C# tree.

## Consequences

Positive:

- BenchPilot joins the family stack its maintainer actually ships with
(rivet, glaze, taskly, regexmate), concentrating maintenance attention;
- the tests-as-spec port de-risks protocol logic: parity is proven
hardware-free before any driver FFI is written;
- the frozen API means agents, CLI users and the MCP surface see no change
during the entire port;
- Rivet gains a defined future adopter (Studio) without distorting a headless
product into a GUI shape.

Costs:

- CAN/CAN FD capture + DBC decoding (product priority 3) and real-ECU
validation queue behind the port;
- two trees to fix during the transition window (C# for shipped fixes,
Racket for the port);
- Racket FFI and GC characteristics must be validated at Phase 4 against
real CAN traffic and ISO-TP block pacing.

## Revisit criteria

Revisit this decision if either abort criterion fires, or if the Phase 3
parity gate slips past 2026-11-15 without a credible path to green.
Loading
Loading