From 871a24728ae9477256bf7d0713b5a34eb9b09926 Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Tue, 7 Jul 2026 21:10:56 -0400 Subject: [PATCH 1/3] docs: document partial adoption (backing-services-only) as a valid hack setup MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A real agent-driven onboarding hit a polyglot monorepo where heavy native toolchains (.NET) and existing host workflows made full containerization the wrong call, but nothing told the agent that running only backing services under hack — with app dev servers on the host pointed at routed services — was a supported, deliberate end state rather than a fallback to fix later. - docs/guides/agent-first-setup.md: add a "Partial adoption" section covering when to choose it, what hack still provides (stable hostnames/TLS, branch instances, committed env, lifecycle hooks), how to wire it, and tradeoffs vs full containerization. - src/agents/onboarding-prompt.ts (Phase 2): add a bullet presenting full containerization vs backing-services-only as an explicit decision, plus a bullet to prefer reusing existing prod Dockerfiles/compose contexts over authoring new dev-mode services from scratch. - src/agents/onboarding-prompt.ts (Phase 1): extend the env-inventory bullet so agents also harvest hack env candidates from compose `environment:` blocks, Dockerfile `ENV` lines, and CI config when a repo has no `.env*` files. Co-Authored-By: Claude Fable 5 --- docs/guides/agent-first-setup.md | 47 ++++++++++++++++++++++++++++++++ src/agents/onboarding-prompt.ts | 4 ++- 2 files changed, 50 insertions(+), 1 deletion(-) diff --git a/docs/guides/agent-first-setup.md b/docs/guides/agent-first-setup.md index 1e8c447f..94beaad8 100644 --- a/docs/guides/agent-first-setup.md +++ b/docs/guides/agent-first-setup.md @@ -64,6 +64,53 @@ never go stale on substance. 5. Verification loop — `hack up --json` → `hack ps --json` → `hack open --json` → curl or `hack logs`, iterating until `hack doctor` is clean. +## Partial adoption (backing services only) + +Full containerization (every process in compose) is the default recommendation, but +it is not the only supported shape. hack works fine running only the backing +services — postgres, temporal, redis, and similar — while app dev servers (`bun +dev`, `dotnet watch`, `vite`) stay on the host. + +**When to choose it:** +- Heavy native toolchains (.NET, large native builds) develop faster on the host + than in a container. +- The hot-path dev server (the one you restart constantly) benefits from host-native + file watching, debuggers, or hot reload that containers make slower or flakier. +- The team already has a working host-based dev workflow and containerizing it is + not worth the churn. + +**What hack still gives you in this shape:** +- Stable hostnames and TLS for the backing services via Caddy (`postgres..hack`, + `temporal..hack`, ...), so host processes and containers address them the + same way. +- Branch instances for the containerized services (`--branch `), even though + the host-run app processes are not branch-isolated on their own. +- Committed, hack-managed env (`hack env add`) for connection strings and secrets, + instead of hand-maintained `.env` files. +- Lifecycle hooks (`startup`/`lifecycle` in `.hack/hack.config.json`) for host-side + setup — tunnels, SSO bootstrap — that would otherwise be ad-hoc terminal steps. + +**How to wire it:** +- Define only the backing services in `.hack/docker-compose.yml`; add Caddy labels + where a stable hostname is useful (databases usually don't need one — connect by + container port — but admin UIs, queues, or shared services often do). +- Give host dev servers their connection info via `hack host exec --scope -- + ` (injects the resolved env without touching `.env` files) or `hack env list` + for a one-off lookup. +- App URLs can stay on plain `localhost:` for the fast path, or get promoted + to a routed compose service later if you want a stable `*.hack` hostname for them + too — the two are not mutually exclusive and can be migrated incrementally. + +**Tradeoffs vs full containerization:** +- No single `hack up` brings up the whole stack for a newcomer — they still need to + start the host dev servers themselves (document that step). +- Host-run app processes are not branch-isolated by hack; running two branches side + by side means varying ports/env per worktree yourself. + +Partial adoption is a valid end state, not an incomplete migration — do not treat it +as something to "finish" into full containerization unless the tradeoffs above +actually bite. + ## Copy-paste bootstrap Paste this into a fresh Claude Code / Codex session started at the repo root: diff --git a/src/agents/onboarding-prompt.ts b/src/agents/onboarding-prompt.ts index 8794187a..cfa1e3a7 100644 --- a/src/agents/onboarding-prompt.ts +++ b/src/agents/onboarding-prompt.ts @@ -108,7 +108,7 @@ function renderInventoryPhase(): string[] { "- Read the root package.json, workspace globs, and lockfiles to identify the package manager and monorepo layout.", "- Identify runnable services (web, api, workers): dev/start scripts, framework configs, and the ports they listen on.", "- Detect databases, caches, and queues from code and config: ORM schemas (prisma/drizzle), migration dirs, existing Dockerfiles and compose files.", - "- List every `.env*` file. Their keys and values are candidates for `hack env add`; treat `.env.example` values as placeholders (ask the user for real values or leave them unset).", + "- List every `.env*` file. Their keys and values are candidates for `hack env add`; treat `.env.example` values as placeholders (ask the user for real values or leave them unset). Not every repo has `.env*` files — also harvest candidate keys from `environment:` blocks in existing compose files, `ENV` lines in Dockerfiles, and CI config (workflow env/secrets).", "- Note repo scripts that need env vars to run (migrations, seeds, codegen) — they will run via `hack host exec` or an ops container later.", "- Read README/docs for one-time setup steps you might otherwise miss.", ]; @@ -128,6 +128,8 @@ function renderSetupPhase(opts: { "## Phase 2 — Set up hack", "", bootstrap, + "- Decide full containerization vs backing-services-only (partial adoption): if the inventory shows heavy native toolchains (.NET, large native builds) or the human prefers host dev servers, compose backing services (db/queue/cache) only and keep app dev servers on the host pointed at the routed services — ask the human when it's ambiguous; partial adoption is a valid end state, not a fallback to fix later.", + "- Check for reuse before authoring new services: if the repo already containerizes for prod (existing Dockerfiles/compose with build contexts, runtime-config injection points like an nginx-served runtime-config.js), prefer reusing those images/contexts with minimal glue over hand-authoring new dev-mode services from scratch.", "- Define one compose service per runnable process in `.hack/docker-compose.yml`. HTTP services need the Caddy labels (`caddy`, `caddy.reverse_proxy`, `caddy.tls=internal`) and the `hack-dev` network to be routable.", `- Design hostnames: the primary app answers on dev_host (${devHostExample}); every other routable service gets a subdomain (\`api.${devHostExample}\`, \`grafana.${devHostExample}\`, ...). Set dev_host in \`.hack/hack.config.json\`.`, "- Migrate env values: for each key found during inventory run `hack env add ` — add `--secret` for anything sensitive (API keys, tokens, passwords, connection strings with credentials). Never commit plaintext secrets to the repo.", From 6b50b7673921fd9118a780be77bfac95ae596fff Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Tue, 7 Jul 2026 21:23:17 -0400 Subject: [PATCH 2/3] feat(init): add post-discovery validation pass for hack init MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Field repos were shipping duplicated services (web/web-2), everything crammed onto port 3000, silently-wrong bun-node images on non-JS services, and missing backing-service infra (postgres/temporal) — all with zero warnings. Add a validation pass (src/init/validation.ts) that runs after discovery and before compose generation in both the --auto and interactive init paths: - Candidate dedupe: collapse same-package duplicate scripts (dev + start) and drop root-level aggregator scripts that just delegate to a workspace package's own dev script (turbo --filter, dotnet run --project, pnpm --cwd), which is what actually produced the reported web/web-2 and phantom "backend" duplicates. - Port collision handling: reassign colliding HTTP ports to the next free port (ascending) and rewrite the container command. - Runtime detection: flag non-JS services (csproj/fsproj, go.mod, Cargo.toml, pyproject.toml, mix.exs, Gemfile, or command sniffing — including root aggregator commands that target another dir) and use an obviously-wrong alpine:3 placeholder image with a TODO comment instead of silently defaulting to bun-node. - Backing-service detection: surface postgres/mysql/redis/temporal/ kafka/rabbitmq/mongodb signals from dependencies, prisma schema, and .env* key names (values never read) as warnings + a compose header comment, without auto-scaffolding them. - Existing-compose pointer: if a repo-root docker-compose*.yml already exists, point at it as ground truth instead of re-deriving. renderCompose gains header/service comment injection (idempotent post-process over the stringified YAML). buildDiscoveredComposeAuto and ComposeWizardInput are now exported for direct pipeline testing. Co-Authored-By: Claude Fable 5 --- docs/guides/init-project.md | 38 + src/commands/project.ts | 205 +++++- src/init/compose.ts | 113 ++- src/init/discovery.ts | 7 +- src/init/validation.ts | 923 ++++++++++++++++++++++++ tests/init-discovery-validation.test.ts | 457 ++++++++++++ 6 files changed, 1729 insertions(+), 14 deletions(-) create mode 100644 src/init/validation.ts create mode 100644 tests/init-discovery-validation.test.ts diff --git a/docs/guides/init-project.md b/docs/guides/init-project.md index 4cd99d9e..3fbb0df1 100644 --- a/docs/guides/init-project.md +++ b/docs/guides/init-project.md @@ -27,6 +27,44 @@ Note: - HTTP services via `https://*.hack` hostnames (matching your Caddy labels) - non-HTTP services via Compose service hostnames (e.g. `db`, `redis`) +## What discovery checks (and what it can't) + +`hack init` (interactive and `--auto`) discovers dev scripts across a repo/monorepo +and runs a validation pass over the results before writing `.hack/docker-compose.yml`. +It catches common scaffolding mistakes automatically, but it is not a substitute for +reviewing the generated compose file: + +- **Duplicate/aggregator script dedupe**: when a package defines both `dev` and + `start` (or similar), only the best-scoring script becomes a service — no more + `web` + `web-2`. Root-level "aggregator" scripts that just delegate to a + workspace package's own dev script (e.g. `turbo run dev --filter=web`, or + `dotnet run --project apps/backend`) are also dropped in favor of the + package-local script. In the interactive wizard, the deduped set is + pre-selected by default — you can still add a dropped script back manually. +- **Port reassignment**: HTTP services that would collide on the same internal + port are deterministically reassigned to the next free port (ascending from + the collision), and the container command is rewritten to match. Each + reassignment is logged as a warning. +- **Runtime TODOs**: services whose dev script looks non-JS (`.csproj`/`.fsproj`, + `go.mod`, `Cargo.toml`, `pyproject.toml`/`requirements.txt`, `mix.exs`, + `Gemfile`, or a command like `dotnet run`/`go run`/`cargo`/`python`/`mix`) get an + obviously-wrong placeholder image (`alpine:3`) plus a `TODO(hack-init): ...` + comment above the service in the compose file — instead of silently getting the + default Bun/Node image. You must replace the image and command by hand. +- **Backing-service warnings**: dependencies (`pg`, `ioredis`, `@temporalio/*`, + `kafkajs`, `amqplib`, `mongodb`/`mongoose`, `prisma`, ...) and `.env`/`.env.example` + key names (never values) are scanned for signals of postgres, mysql, redis, + temporal, kafka, rabbitmq, or mongodb. These are **not** auto-scaffolded — they + show up as warnings and as a comment block at the top of the generated compose + file with a one-line suggestion each. If a `docker-compose*.yml`/`compose*.yml` + already exists at the repo root, it's flagged too ("treat it as ground truth for + backing services and images") instead of being re-derived from scratch. + +None of this replaces reviewing the scaffold. Discovery is a best-effort heuristic +pass, not ground truth — always inventory the generated `.hack/docker-compose.yml` +against the real repo before running `hack up` (agents included; see the +onboarding prompt for the inventory-first review step). + When the local path is working and you intentionally want remote execution or gateway exposure, move to [Beta workflows](../beta.md). For full command lookup and extension docs, use [Extensions & reference](../reference.md). diff --git a/src/commands/project.ts b/src/commands/project.ts index a9359f0a..b762dbba 100644 --- a/src/commands/project.ts +++ b/src/commands/project.ts @@ -64,7 +64,7 @@ import { } from "../constants.ts"; import { readControlPlaneConfig } from "../control-plane/sdk/config.ts"; import { requestDaemonJson } from "../daemon/client.ts"; -import { renderCompose } from "../init/compose.ts"; +import { renderCompose, TODO_IMAGE_SENTINEL } from "../init/compose.ts"; import type { ServiceCandidate } from "../init/discovery.ts"; import { discoverRepo } from "../init/discovery.ts"; import { @@ -74,6 +74,21 @@ import { guessServiceName, inferPortFromScript, } from "../init/heuristics.ts"; +import type { + InitDiscoveryFinding, + PortCollisionDraft, +} from "../init/validation.ts"; +import { + buildDiscoveryHeaderComments, + buildExistingComposeFindings, + dedupeCandidates, + describeUnknownRuntimeComment, + detectBackingServices, + detectPackageRuntime, + findExistingComposeFiles, + reassignCollidingPorts, + reportInitDiscoveryFindings, +} from "../init/validation.ts"; import { resolveEffectiveBranch, touchBranchUsage } from "../lib/branches.ts"; import { type CliResult, @@ -3837,7 +3852,9 @@ function logInstallResult(opts: { logger.success({ message: `Updated ${opts.label} at ${opts.path}` }); } -interface ComposeWizardInput { +// Exported for direct unit-testing of the auto (non-interactive) discovery +// + validation pipeline without going through the CLI prompt flow. +export interface ComposeWizardInput { readonly repoRoot: string; readonly devHost: string; readonly projectSlug: string; @@ -4114,13 +4131,27 @@ function validateSubdomain(value: string | undefined): string | undefined { async function selectCandidatesForDiscoveredCompose(opts: { readonly candidates: readonly ServiceCandidate[]; -}): Promise { +}): Promise<{ + readonly selectedCandidates: ServiceCandidate[]; + readonly dedupeFindings: readonly InitDiscoveryFinding[]; +}> { const byId = new Map(opts.candidates.map((c) => [c.id, c] as const)); + // Default-select the deduped set (best script per package, aggregator + // scripts dropped in favor of the package's own script) so the common + // case is a single confirm — the user can still add dropped candidates + // back via the multiselect. + const dedupe = dedupeCandidates({ candidates: opts.candidates }); + const defaultIds = dedupe.selected.map((c) => c.id); + const droppedIds = new Set( + opts.candidates.filter((c) => !defaultIds.includes(c.id)).map((c) => c.id) + ); + const selectedIds = unwrapPromptValue( await autocompleteMultiselect({ message: "Select dev scripts to include as services:", required: true, + initialValues: defaultIds, options: opts.candidates.map((c) => ({ value: c.id, label: formatCandidateLabel(c), @@ -4141,7 +4172,13 @@ async function selectCandidatesForDiscoveredCompose(opts: { throw new Error("No services selected"); } - return selectedCandidates; + // Only report dedupe findings when the user kept the pruned selection — + // if they manually re-added every dropped candidate, there's nothing left + // to warn about. + const stillDropped = [...droppedIds].some((id) => !selectedIds.includes(id)); + const dedupeFindings = stillDropped ? dedupe.findings : []; + + return { selectedCandidates, dedupeFindings }; } async function promptDraftForDiscoveredCandidate(opts: { @@ -4227,6 +4264,7 @@ async function promptDraftForDiscoveredCandidate(opts: { port: portNum, workingDir, command, + candidate: opts.candidate, }; } @@ -4280,9 +4318,10 @@ async function promptHttpSubdomainsForDrafts(opts: { async function buildDiscoveredCompose( input: ComposeWizardInput ): Promise { - const selectedCandidates = await selectCandidatesForDiscoveredCompose({ - candidates: input.candidates, - }); + const { selectedCandidates, dedupeFindings } = + await selectCandidatesForDiscoveredCompose({ + candidates: input.candidates, + }); const usedServiceNames = new Set(); const drafts: AutoComposeDraft[] = []; @@ -4300,13 +4339,40 @@ async function buildDiscoveredCompose( devHost: input.devHost, }); + const findings = [...dedupeFindings]; + await applyRuntimeDetectionToDrafts({ + drafts, + repoRoot: input.repoRoot, + findings, + }); + applyPortCollisionReassignment({ drafts, findings }); + + const discoveryForServices = await discoverRepo(input.repoRoot); + const backingServices = await detectBackingServices({ + repoRoot: input.repoRoot, + packages: discoveryForServices.packages, + }); + const existingComposeFiles = await findExistingComposeFiles({ + repoRoot: input.repoRoot, + }); + findings.push(...buildExistingComposeFindings({ existingComposeFiles })); + + reportInitDiscoveryFindings({ findings }); + const services = buildServicesFromDrafts({ drafts, devHost: input.devHost, oauth: input.oauth, }); - return renderCompose({ name: input.projectSlug, services }); + return renderCompose({ + name: input.projectSlug, + services, + headerComments: buildDiscoveryHeaderComments({ + detections: backingServices, + existingComposeFiles, + }), + }); } type AutoComposeDraft = { @@ -4317,16 +4383,105 @@ type AutoComposeDraft = { workingDir: string; command: string; image?: string; + candidate?: ServiceCandidate; + comments?: string[]; }; -function buildDiscoveredComposeAuto(input: ComposeWizardInput): string { +/** + * Runs `detectPackageRuntime` for each draft that carries its source + * candidate and, for non-JS runtimes, sets the compose TODO-image sentinel + * plus an explanatory comment instead of silently defaulting to the + * bun-node image. Mutates `drafts` in place and appends one + * `unknown-runtime` finding per detected draft. + */ +async function applyRuntimeDetectionToDrafts(opts: { + readonly drafts: AutoComposeDraft[]; + readonly repoRoot: string; + readonly findings: InitDiscoveryFinding[]; +}): Promise { + for (const draft of opts.drafts) { + const candidate = draft.candidate; + if (!candidate) { + continue; + } + + const runtime = await detectPackageRuntime({ + dir: + candidate.packageRelativeDir === "." + ? opts.repoRoot + : resolve(opts.repoRoot, candidate.packageRelativeDir), + scriptCommand: candidate.scriptCommand, + repoRoot: opts.repoRoot, + }); + if (!runtime) { + continue; + } + + draft.image = TODO_IMAGE_SENTINEL; + draft.comments = [ + ...(draft.comments ?? []), + describeUnknownRuntimeComment({ serviceName: draft.name, runtime }), + ]; + opts.findings.push({ + kind: "unknown-runtime", + serviceName: draft.name, + runtime, + }); + } +} + +/** + * Detects HTTP drafts sharing a port, reassigns duplicates to the next + * free port (ascending), and rewrites their command via + * `buildSuggestedCommand`. Mutates `drafts` in place and appends one + * `port-reassigned` finding per reassignment. + */ +function applyPortCollisionReassignment(opts: { + readonly drafts: AutoComposeDraft[]; + readonly findings: InitDiscoveryFinding[]; +}): void { + const collisionDrafts: PortCollisionDraft[] = []; + for (const draft of opts.drafts) { + if (!draft.candidate) { + continue; + } + collisionDrafts.push({ + name: draft.name, + role: draft.role, + port: draft.port, + candidate: draft.candidate, + }); + } + + const { reassignments, findings } = reassignCollidingPorts({ + drafts: collisionDrafts, + }); + opts.findings.push(...findings); + + for (const draft of opts.drafts) { + const reassignment = reassignments.get(draft.name); + if (!reassignment) { + continue; + } + draft.port = reassignment.port; + draft.command = reassignment.command; + } +} + +// Exported for direct unit-testing (see tests/init-discovery-validation.test.ts). +export async function buildDiscoveredComposeAuto( + input: ComposeWizardInput +): Promise { + const dedupe = dedupeCandidates({ candidates: input.candidates }); const selectedCandidates = selectAutoCandidates({ - candidates: input.candidates, + candidates: dedupe.selected, }); if (selectedCandidates.length === 0) { throw new Error("No dev scripts discovered for auto init."); } + const findings: InitDiscoveryFinding[] = [...dedupe.findings]; + const usedServiceNames = new Set(); const drafts: AutoComposeDraft[] = []; @@ -4352,18 +4507,45 @@ function buildDiscoveredComposeAuto(input: ComposeWizardInput): string { port, workingDir, command, + candidate, }); } assignAutoSubdomains({ drafts }); + await applyRuntimeDetectionToDrafts({ + drafts, + repoRoot: input.repoRoot, + findings, + }); + applyPortCollisionReassignment({ drafts, findings }); + + const discovery = await discoverRepo(input.repoRoot); + const backingServices = await detectBackingServices({ + repoRoot: input.repoRoot, + packages: discovery.packages, + }); + const existingComposeFiles = await findExistingComposeFiles({ + repoRoot: input.repoRoot, + }); + findings.push(...buildExistingComposeFindings({ existingComposeFiles })); + + reportInitDiscoveryFindings({ findings }); + const services = buildServicesFromDrafts({ drafts, devHost: input.devHost, oauth: input.oauth, }); - return renderCompose({ name: input.projectSlug, services }); + return renderCompose({ + name: input.projectSlug, + services, + headerComments: buildDiscoveryHeaderComments({ + detections: backingServices, + existingComposeFiles, + }), + }); } interface ManualComposeWizardInput { @@ -4691,6 +4873,7 @@ function buildServicesFromDrafts(opts: { env, labels, networks, + ...(d.comments && d.comments.length > 0 ? { comments: d.comments } : {}), }; }); } diff --git a/src/init/compose.ts b/src/init/compose.ts index 7e36a61f..b9cebebe 100644 --- a/src/init/compose.ts +++ b/src/init/compose.ts @@ -2,6 +2,19 @@ import { YAML } from "bun"; import { DEFAULT_INGRESS_NETWORK } from "../constants.ts"; +/** + * Sentinel image value a caller can set on `PlannedService.image` to mark a + * service whose runtime looks non-JS (see `detectPackageRuntime` in + * `./validation.ts`). `renderCompose` replaces this literal with a visibly + * wrong placeholder image (`alpine:3`) rather than silently defaulting to + * the bun-node image — paired with a required `comments` entry explaining + * why. + */ +export const TODO_IMAGE_SENTINEL = "TODO-set-image"; + +/** Visible placeholder image substituted for `TODO_IMAGE_SENTINEL`. */ +export const TODO_IMAGE_PLACEHOLDER = "alpine:3"; + export type PlannedServiceRole = "http" | "internal"; export interface PlannedService { @@ -13,11 +26,25 @@ export interface PlannedService { readonly env: ReadonlyMap; readonly labels: ReadonlyMap; readonly networks: readonly string[]; + /** + * Comment lines injected immediately above this service's ` :` key + * in the rendered YAML (e.g. a runtime-mismatch TODO). Rendered as `# ` + * prefixed lines; injection is a post-process step over the stringified + * YAML and is idempotent — re-running against already-commented output + * does not duplicate lines. + */ + readonly comments?: readonly string[]; } export interface ComposePlan { readonly name?: string; readonly services: readonly PlannedService[]; + /** + * Comment lines injected as a block at the very top of the rendered file + * (e.g. backing-service discovery warnings). See `PlannedService.comments` + * for per-service comments. + */ + readonly headerComments?: readonly string[]; } export function renderCompose(plan: ComposePlan): string { @@ -25,7 +52,8 @@ export function renderCompose(plan: ComposePlan): string { for (const svc of plan.services) { services[svc.name] = { - image: svc.image, + image: + svc.image === TODO_IMAGE_SENTINEL ? TODO_IMAGE_PLACEHOLDER : svc.image, working_dir: svc.workingDir, volumes: ["..:/app"], command: svc.command, @@ -44,7 +72,88 @@ export function renderCompose(plan: ComposePlan): string { }; const yaml = YAML.stringify(compose, null, 2); - return ensureTrailingNewline(cleanupYaml(yaml)); + const cleaned = ensureTrailingNewline(cleanupYaml(yaml)); + + const withServiceComments = injectServiceComments({ + yaml: cleaned, + services: plan.services, + }); + + return injectHeaderComments({ + yaml: withServiceComments, + headerComments: plan.headerComments ?? [], + }); +} + +/** + * Prefixes each line with `# ` to form a YAML comment block; blank input + * lines become bare `#` lines (no trailing space). + */ +function toCommentBlock(lines: readonly string[]): string[] { + return lines.map((line) => (line.length > 0 ? `# ${line}` : "#")); +} + +const HEADER_COMMENT_MARKER = "# --- hack-init discovery notes ---"; + +function injectHeaderComments(opts: { + readonly yaml: string; + readonly headerComments: readonly string[]; +}): string { + if (opts.headerComments.length === 0) { + return opts.yaml; + } + if (opts.yaml.includes(HEADER_COMMENT_MARKER)) { + // Idempotent: header block already present, do not duplicate it. + return opts.yaml; + } + + const block = [ + HEADER_COMMENT_MARKER, + ...toCommentBlock(opts.headerComments), + "", + ].join("\n"); + + return `${block}\n${opts.yaml}`; +} + +function injectServiceComments(opts: { + readonly yaml: string; + readonly services: readonly PlannedService[]; +}): string { + let out = opts.yaml; + + for (const svc of opts.services) { + if (!svc.comments || svc.comments.length === 0) { + continue; + } + + const serviceKeyPattern = new RegExp( + `^(\\s{2}${escapeRegExp(svc.name)}:)$`, + "m" + ); + const match = out.match(serviceKeyPattern); + if (!match) { + continue; + } + + const marker = ` # hack-init: ${svc.name}`; + if (out.includes(marker)) { + // Idempotent: this service's comment block was already injected. + continue; + } + + const commentLines = [ + marker, + ...toCommentBlock(svc.comments).map((l) => ` ${l}`), + ].join("\n"); + out = out.replace(serviceKeyPattern, `${commentLines}\n${match[0]}`); + } + + return out; +} + +function escapeRegExp(value: string): string { + return value.replaceAll(/[.*+?^${}()|[\]\\]/g, "\\$&"); } type ComposeServiceSpec = { diff --git a/src/init/discovery.ts b/src/init/discovery.ts index ac674d12..af725a16 100644 --- a/src/init/discovery.ts +++ b/src/init/discovery.ts @@ -249,7 +249,12 @@ function buildServiceCandidates( return out; } -function scoreDevScript(name: string): number { +/** + * Score a script name by how likely it is to be a dev/serve entrypoint. + * Higher scores win when multiple scripts in the same package qualify as + * candidates (see `dedupeCandidatesByPackage` in `./validation.ts`). + */ +export function scoreDevScript(name: string): number { const n = name.toLowerCase(); if (n === "dev") { return 100; diff --git a/src/init/validation.ts b/src/init/validation.ts new file mode 100644 index 00000000..2f534309 --- /dev/null +++ b/src/init/validation.ts @@ -0,0 +1,923 @@ +import { readdir } from "node:fs/promises"; +import { basename, resolve } from "node:path"; +import { pathExists, readTextFile } from "../lib/fs.ts"; +import { isRecord } from "../lib/guards.ts"; +import { logger } from "../ui/logger.ts"; +import { TODO_IMAGE_PLACEHOLDER, TODO_IMAGE_SENTINEL } from "./compose.ts"; +import type { DiscoveredPackage, ServiceCandidate } from "./discovery.ts"; +import { scoreDevScript } from "./discovery.ts"; +import { buildSuggestedCommand } from "./heuristics.ts"; + +/** + * A single discovery-time finding surfaced during `hack init`. Each variant + * carries enough structured data to render both a human warning line + * (`describeFinding`) and, where relevant, a compose comment. + */ +export type InitDiscoveryFinding = + | { + readonly kind: "duplicate-script"; + readonly packageRelativeDir: string; + readonly keptScriptName: string; + readonly droppedScriptName: string; + } + | { + readonly kind: "port-reassigned"; + readonly serviceName: string; + readonly fromPort: number; + readonly toPort: number; + } + | { + readonly kind: "unknown-runtime"; + readonly serviceName: string; + readonly runtime: PackageRuntime; + } + | { + readonly kind: "backing-service"; + readonly serviceKind: BackingServiceKind; + readonly evidence: readonly string[]; + readonly alreadyContainerized: boolean; + } + | { + readonly kind: "existing-compose-found"; + readonly path: string; + }; + +export type PackageRuntime = + | "dotnet" + | "go" + | "rust" + | "python" + | "elixir" + | "ruby"; + +export type BackingServiceKind = + | "postgres" + | "mysql" + | "redis" + | "temporal" + | "kafka" + | "rabbitmq" + | "mongodb"; + +const RUNTIME_MARKER_FILES: ReadonlyMap = new Map([ + [".csproj", "dotnet"], + [".fsproj", "dotnet"], + ["go.mod", "go"], + ["Cargo.toml", "rust"], + ["pyproject.toml", "python"], + ["requirements.txt", "python"], + ["mix.exs", "elixir"], + ["Gemfile", "ruby"], +]); + +const RUNTIME_COMMAND_PATTERNS: ReadonlyArray<{ + readonly pattern: RegExp; + readonly runtime: PackageRuntime; +}> = [ + { pattern: /\bdotnet\s/i, runtime: "dotnet" }, + { pattern: /\bgo run\b/i, runtime: "go" }, + { pattern: /\bcargo\s/i, runtime: "rust" }, + { pattern: /\bpython\s/i, runtime: "python" }, + { pattern: /\bmix\s/i, runtime: "elixir" }, +]; + +const DEPENDENCY_TO_BACKING_SERVICE: ReadonlyMap = + new Map([ + ["pg", "postgres"], + ["postgres", "postgres"], + ["mysql2", "mysql"], + ["ioredis", "redis"], + ["redis", "redis"], + ["bullmq", "redis"], + ["kafkajs", "kafka"], + ["amqplib", "rabbitmq"], + ["mongodb", "mongodb"], + ["mongoose", "mongodb"], + ]); + +const TEMPORAL_DEPENDENCY_PREFIX = "@temporalio/"; + +const ENV_KEY_TO_BACKING_SERVICE: ReadonlyArray<{ + readonly pattern: RegExp; + readonly kind: BackingServiceKind; +}> = [ + { pattern: /^DATABASE_URL$/, kind: "postgres" }, + { pattern: /^POSTGRES_/, kind: "postgres" }, + { pattern: /^MYSQL_/, kind: "mysql" }, + { pattern: /^REDIS(_URL)?$|^REDIS_/, kind: "redis" }, + { pattern: /^TEMPORAL_/, kind: "temporal" }, + { pattern: /^MONGO/, kind: "mongodb" }, + { pattern: /^KAFKA_/, kind: "kafka" }, + { pattern: /^(AMQP|RABBIT)/, kind: "rabbitmq" }, +]; + +const ENV_FILE_NAMES = [".env", ".env.example", ".env.local.example"] as const; + +const BACKING_SERVICE_SUGGESTIONS: ReadonlyMap = + new Map([ + [ + "postgres", + "add a postgres service + volume, or point env at an existing instance", + ], + [ + "mysql", + "add a mysql service + volume, or point env at an existing instance", + ], + ["redis", "add a redis service, or point env at an existing instance"], + [ + "temporal", + "add a temporal dev-server service, or point env at an existing instance", + ], + [ + "kafka", + "add a kafka (or redpanda) service, or point env at an existing broker", + ], + ["rabbitmq", "add a rabbitmq service, or point env at an existing broker"], + [ + "mongodb", + "add a mongodb service + volume, or point env at an existing instance", + ], + ]); + +/** + * Drop lower-scored dev-script candidates within the same package, keeping + * only the single best-scoring script (per `scoreDevScript`'s ordering). + * Prevents `web` + `web-2` duplicate services when a package defines both + * `dev` and `start`. + */ +export function dedupeCandidatesByPackage(opts: { + readonly candidates: readonly ServiceCandidate[]; +}): { + readonly selected: readonly ServiceCandidate[]; + readonly dropped: readonly { + readonly candidate: ServiceCandidate; + readonly keptScriptName: string; + }[]; +} { + const byPackage = new Map(); + for (const candidate of opts.candidates) { + const bucket = byPackage.get(candidate.packageId); + if (bucket) { + bucket.push(candidate); + } else { + byPackage.set(candidate.packageId, [candidate]); + } + } + + const selected: ServiceCandidate[] = []; + const dropped: { candidate: ServiceCandidate; keptScriptName: string }[] = []; + + for (const bucket of byPackage.values()) { + const ranked = [...bucket].sort( + (a, b) => scoreDevScript(b.scriptName) - scoreDevScript(a.scriptName) + ); + const best = ranked[0]; + if (!best) { + continue; + } + selected.push(best); + for (const candidate of ranked.slice(1)) { + dropped.push({ candidate, keptScriptName: best.scriptName }); + } + } + + return { selected, dropped }; +} + +const WORKSPACE_FILTER_PATTERNS: readonly RegExp[] = [ + /--filter[= ]([^\s]+)/i, + /--cwd[= ]([^\s]+)/i, + /--project[= ]([^\s]+)/i, + /--prefix[= ]([^\s]+)/i, +]; + +/** + * Best-effort extraction of a workspace-relative directory (or package + * name fragment) that a root-level aggregator script command appears to + * delegate to — e.g. `turbo run dev --filter=web`, `dotnet run --project + * apps/backend/Api.csproj`, `pnpm --cwd apps/web dev`. + */ +function extractDelegationTarget(scriptCommand: string): string | null { + for (const pattern of WORKSPACE_FILTER_PATTERNS) { + const match = scriptCommand.match(pattern); + const raw = match?.[1]; + if (raw) { + return raw.replaceAll(/["']/g, ""); + } + } + return null; +} + +/** + * Indexes non-root candidates by both their package directory basename and + * package name (lowercased) so a root aggregator script's delegation + * target can be resolved against either. + */ +function buildNonRootTargetIndex( + candidates: readonly ServiceCandidate[] +): ReadonlyMap { + const nonRootByTarget = new Map(); + + for (const candidate of candidates) { + if (candidate.packageRelativeDir === ".") { + continue; + } + const dirBase = basename(candidate.packageRelativeDir).toLowerCase(); + if (!nonRootByTarget.has(dirBase)) { + nonRootByTarget.set(dirBase, candidate); + } + if (!candidate.packageName) { + continue; + } + const nameKey = candidate.packageName.toLowerCase(); + if (!nonRootByTarget.has(nameKey)) { + nonRootByTarget.set(nameKey, candidate); + } + } + + return nonRootByTarget; +} + +/** + * Resolves the workspace candidate a root-level aggregator script appears + * to delegate to, using either an explicit `--filter`/`--cwd`/`--project` + * reference or the `dev:` naming convention. + */ +/** + * Candidate lookup keys for a delegation target, most specific first: the + * raw key, its basename (handles path-like targets such as `apps/web`), + * and — when the target looks like a file (`Api.csproj`) — the basename of + * its parent directory (handles `--project backend/Api.csproj`). + */ +function delegationLookupKeys(targetKey: string): readonly string[] { + const keys = [targetKey, basename(targetKey)]; + if (FILE_EXTENSION_PATTERN.test(basename(targetKey))) { + const parent = targetKey.split("/").slice(0, -1).join("/"); + if (parent.length > 0) { + keys.push(basename(parent)); + } + } + return keys; +} + +function resolveAggregatorTargetCandidate(opts: { + readonly candidate: ServiceCandidate; + readonly nonRootByTarget: ReadonlyMap; +}): ServiceCandidate | null { + const delegationTarget = extractDelegationTarget( + opts.candidate.scriptCommand + ); + const scriptSuffix = opts.candidate.scriptName.includes(":") + ? opts.candidate.scriptName.split(":").slice(1).join(":") + : null; + + const targetKey = (delegationTarget ?? scriptSuffix)?.toLowerCase(); + if (!targetKey) { + return null; + } + + for (const key of delegationLookupKeys(targetKey)) { + const match = opts.nonRootByTarget.get(key); + if (match) { + return match; + } + } + + return null; +} + +/** + * Detects root-package "aggregator" scripts that just delegate to another + * discovered package's own dev script (e.g. root `dev:web` running + * `turbo run dev --filter=web` while `apps/web` already has its own `dev` + * candidate, or `dev:backend` running `dotnet run --project apps/backend` + * while `apps/backend` is itself a discovered package). These produce a + * true duplicate service (`web` + `web-2`) even though they live in + * different packages, so per-package dedupe alone can't catch them. + * + * A root candidate is dropped when: + * - it belongs to the repo-root package (`packageRelativeDir === "."`), and + * - its command references another discovered package's relative dir (via + * `--filter`/`--cwd`/`--project`/`--prefix`) or its own script name + * (after the `dev:` prefix) matches another package's directory + * basename or package name, and + * - that other package already has its own qualifying candidate. + */ +export function dedupeAggregatorCandidates(opts: { + readonly candidates: readonly ServiceCandidate[]; +}): { + readonly selected: readonly ServiceCandidate[]; + readonly dropped: readonly { + readonly candidate: ServiceCandidate; + readonly keptScriptName: string; + }[]; +} { + const nonRootByTarget = buildNonRootTargetIndex(opts.candidates); + + const dropped: { candidate: ServiceCandidate; keptScriptName: string }[] = []; + const droppedIds = new Set(); + + for (const candidate of opts.candidates) { + if (candidate.packageRelativeDir !== ".") { + continue; + } + + const targetCandidate = resolveAggregatorTargetCandidate({ + candidate, + nonRootByTarget, + }); + if (!targetCandidate) { + continue; + } + + dropped.push({ candidate, keptScriptName: targetCandidate.scriptName }); + droppedIds.add(candidate.id); + } + + const selected = opts.candidates.filter((c) => !droppedIds.has(c.id)); + return { selected, dropped }; +} + +/** + * Full candidate dedupe pipeline: first drops root-level aggregator scripts + * that delegate to another package's own dev script + * (`dedupeAggregatorCandidates`), then collapses same-package duplicates + * (`dedupeCandidatesByPackage`). Returns the final selected set plus one + * `duplicate-script` finding per dropped candidate from either pass. + */ +export function dedupeCandidates(opts: { + readonly candidates: readonly ServiceCandidate[]; +}): { + readonly selected: readonly ServiceCandidate[]; + readonly findings: readonly InitDiscoveryFinding[]; +} { + const aggregatorPass = dedupeAggregatorCandidates({ + candidates: opts.candidates, + }); + const packagePass = dedupeCandidatesByPackage({ + candidates: aggregatorPass.selected, + }); + + const findings: InitDiscoveryFinding[] = [ + ...aggregatorPass.dropped.map(({ candidate, keptScriptName }) => ({ + kind: "duplicate-script" as const, + packageRelativeDir: candidate.packageRelativeDir, + keptScriptName, + droppedScriptName: candidate.scriptName, + })), + ...packagePass.dropped.map(({ candidate, keptScriptName }) => ({ + kind: "duplicate-script" as const, + packageRelativeDir: candidate.packageRelativeDir, + keptScriptName, + droppedScriptName: candidate.scriptName, + })), + ]; + + return { selected: packagePass.selected, findings }; +} + +/** Minimal shape validation.ts needs from a built compose draft to detect port collisions. */ +export interface PortCollisionDraft { + readonly name: string; + readonly role: "http" | "internal"; + readonly port?: number; + readonly candidate: ServiceCandidate; +} + +/** + * Detects HTTP drafts sharing the same internal port and deterministically + * reassigns duplicates to the next free port (ascending from the collision), + * mutating `port` on the drafts in place and returning the new command for + * each reassigned draft plus a finding per reassignment. + * + * Deterministic order: drafts are processed in input order; the first + * draft to claim a port keeps it, subsequent claimants walk upward until + * they find a port unused by any other draft. + */ +export function reassignCollidingPorts(opts: { + readonly drafts: readonly PortCollisionDraft[]; +}): { + readonly reassignments: ReadonlyMap< + string, + { readonly port: number; readonly command: string } + >; + readonly findings: readonly InitDiscoveryFinding[]; +} { + const reassignments = new Map< + string, + { readonly port: number; readonly command: string } + >(); + const findings: InitDiscoveryFinding[] = []; + const claimed = new Set(); + + for (const draft of opts.drafts) { + if (draft.role !== "http" || draft.port === undefined) { + continue; + } + + if (!claimed.has(draft.port)) { + claimed.add(draft.port); + continue; + } + + const nextPort = findNextFreePort({ from: draft.port, claimed }); + claimed.add(nextPort); + + const command = buildSuggestedCommand({ + candidate: draft.candidate, + role: "http", + port: nextPort, + }); + + reassignments.set(draft.name, { port: nextPort, command }); + findings.push({ + kind: "port-reassigned", + serviceName: draft.name, + fromPort: draft.port, + toPort: nextPort, + }); + } + + return { reassignments, findings }; +} + +function findNextFreePort(opts: { + readonly from: number; + readonly claimed: ReadonlySet; +}): number { + let candidate = opts.from + 1; + while (opts.claimed.has(candidate)) { + candidate += 1; + } + return candidate; +} + +const PROJECT_PATH_PATTERN = /--project[= ]([^\s]+)/i; +const FILE_EXTENSION_PATTERN = /\.[a-z]+$/i; +const PRISMA_PROVIDER_PATTERN = /provider\s*=\s*"([a-z]+)"/i; + +/** + * Best-effort language/runtime detection for a discovered package: looks + * for marker files (`*.csproj`, `go.mod`, `Cargo.toml`, ...) in the package + * directory, then falls back to sniffing the script command text. For + * root-level aggregator scripts whose command targets another directory + * (e.g. `dotnet run --project apps/backend/Api.csproj`), also checks marker + * files in that referenced directory when `repoRoot` is provided — this is + * what catches a root `dev:backend` aggregator that never had its own + * package directory to inspect. Returns `null` for JS/TS packages (the + * default assumption). + */ +export async function detectPackageRuntime(opts: { + readonly dir: string; + readonly scriptCommand?: string; + readonly repoRoot?: string; +}): Promise { + const fromFiles = await detectRuntimeFromMarkerFiles({ dir: opts.dir }); + if (fromFiles) { + return fromFiles; + } + + if (opts.scriptCommand && opts.repoRoot) { + const fromTarget = await detectRuntimeFromReferencedTarget({ + repoRoot: opts.repoRoot, + scriptCommand: opts.scriptCommand, + }); + if (fromTarget) { + return fromTarget; + } + } + + if (opts.scriptCommand) { + for (const { pattern, runtime } of RUNTIME_COMMAND_PATTERNS) { + if (pattern.test(opts.scriptCommand)) { + return runtime; + } + } + } + + return null; +} + +/** + * Resolves a `--project ` (or similar) reference in an aggregator + * script command to a repo-relative directory and checks it for runtime + * marker files. Handles both directory references (`apps/backend`) and + * project-file references (`apps/backend/Api.csproj`). + */ +async function detectRuntimeFromReferencedTarget(opts: { + readonly repoRoot: string; + readonly scriptCommand: string; +}): Promise { + const match = opts.scriptCommand.match(PROJECT_PATH_PATTERN); + const rawTarget = match?.[1]?.replaceAll(/["']/g, ""); + if (!rawTarget) { + return null; + } + + const targetPath = resolve(opts.repoRoot, rawTarget); + const looksLikeFile = FILE_EXTENSION_PATTERN.test(basename(rawTarget)); + const targetDir = looksLikeFile ? resolve(targetPath, "..") : targetPath; + + if (looksLikeFile) { + const fileRuntime = RUNTIME_MARKER_FILES.get( + `.${basename(rawTarget).split(".").pop()}` + ); + if (fileRuntime && (await pathExists(targetPath))) { + return fileRuntime; + } + } + + return await detectRuntimeFromMarkerFiles({ dir: targetDir }); +} + +async function detectRuntimeFromMarkerFiles(opts: { + readonly dir: string; +}): Promise { + let entries: string[]; + try { + entries = await readdir(opts.dir); + } catch { + return null; + } + + for (const entry of entries) { + for (const [suffix, runtime] of RUNTIME_MARKER_FILES) { + if (suffix.startsWith(".") ? entry.endsWith(suffix) : entry === suffix) { + return runtime; + } + } + } + + return null; +} + +/** A single detected backing-service signal with its evidence trail. */ +export interface BackingServiceDetection { + readonly kind: BackingServiceKind; + readonly evidence: readonly string[]; + readonly alreadyContainerized: boolean; +} + +/** + * Scans package.json dependencies and .env / .env-example files (key names + * only — values are never read into findings) across the repo root and + * every discovered package for signals of external backing services + * (postgres, redis, temporal, etc). Also checks for an existing + * docker-compose file already referencing these images. Results are + * deduped to one entry per service kind with accumulated evidence. + */ +export async function detectBackingServices(opts: { + readonly repoRoot: string; + readonly packages: readonly DiscoveredPackage[]; +}): Promise { + const evidenceByKind = new Map(); + + const addEvidence = (kind: BackingServiceKind, note: string): void => { + const bucket = evidenceByKind.get(kind); + if (bucket) { + if (!bucket.includes(note)) { + bucket.push(note); + } + } else { + evidenceByKind.set(kind, [note]); + } + }; + + for (const pkg of opts.packages) { + await collectDependencyEvidence({ pkg, addEvidence }); + await collectEnvFileEvidence({ dir: pkg.dir, addEvidence }); + } + + const alreadyContainerized = await hasExistingComposeWithBackingServices({ + repoRoot: opts.repoRoot, + kinds: [...evidenceByKind.keys()], + }); + + return [...evidenceByKind.entries()].map(([kind, evidence]) => ({ + kind, + evidence, + alreadyContainerized: alreadyContainerized.has(kind), + })); +} + +async function collectDependencyEvidence(opts: { + readonly pkg: DiscoveredPackage; + readonly addEvidence: (kind: BackingServiceKind, note: string) => void; +}): Promise { + const text = await readTextFile(opts.pkg.packageJsonPath); + if (!text) { + return; + } + + let data: unknown; + try { + data = JSON.parse(text) as unknown; + } catch { + return; + } + if (!isRecord(data)) { + return; + } + + const depNames = new Set(); + for (const field of ["dependencies", "devDependencies"] as const) { + const deps = data[field]; + if (isRecord(deps)) { + for (const name of Object.keys(deps)) { + depNames.add(name); + } + } + } + + for (const dep of depNames) { + const kind = DEPENDENCY_TO_BACKING_SERVICE.get(dep); + if (kind) { + opts.addEvidence(kind, `dependency "${dep}" in ${opts.pkg.relativeDir}`); + } + if (dep.startsWith(TEMPORAL_DEPENDENCY_PREFIX)) { + opts.addEvidence( + "temporal", + `dependency "${dep}" in ${opts.pkg.relativeDir}` + ); + } + if (dep === "prisma" || dep === "@prisma/client") { + const provider = await detectPrismaProvider({ dir: opts.pkg.dir }); + if (provider) { + opts.addEvidence( + provider, + `prisma schema provider in ${opts.pkg.relativeDir}` + ); + } + } + } +} + +const PRISMA_PROVIDER_TO_BACKING_SERVICE: ReadonlyMap< + string, + BackingServiceKind +> = new Map([ + ["postgresql", "postgres"], + ["postgres", "postgres"], + ["mysql", "mysql"], + ["mongodb", "mongodb"], +]); + +async function detectPrismaProvider(opts: { + readonly dir: string; +}): Promise { + const schemaPath = resolve(opts.dir, "prisma/schema.prisma"); + const text = await readTextFile(schemaPath); + if (!text) { + return null; + } + const match = text.match(PRISMA_PROVIDER_PATTERN); + const provider = match?.[1]?.toLowerCase(); + if (!provider) { + return null; + } + return PRISMA_PROVIDER_TO_BACKING_SERVICE.get(provider) ?? null; +} + +async function collectEnvFileEvidence(opts: { + readonly dir: string; + readonly addEvidence: (kind: BackingServiceKind, note: string) => void; +}): Promise { + for (const fileName of ENV_FILE_NAMES) { + const path = resolve(opts.dir, fileName); + const text = await readTextFile(path); + if (!text) { + continue; + } + + for (const line of text.split("\n")) { + const key = extractEnvKey(line); + if (!key) { + continue; + } + for (const { pattern, kind } of ENV_KEY_TO_BACKING_SERVICE) { + if (pattern.test(key)) { + opts.addEvidence( + kind, + `${key} in ${fileName} (${basename(opts.dir)})` + ); + } + } + } + } +} + +function extractEnvKey(line: string): string | null { + const trimmed = line.trim(); + if (trimmed.length === 0 || trimmed.startsWith("#")) { + return null; + } + const eq = trimmed.indexOf("="); + if (eq <= 0) { + return null; + } + return trimmed.slice(0, eq).trim(); +} + +const BACKING_SERVICE_IMAGE_HINTS: ReadonlyMap< + BackingServiceKind, + readonly string[] +> = new Map([ + ["postgres", ["postgres"]], + ["mysql", ["mysql", "mariadb"]], + ["redis", ["redis"]], + ["temporal", ["temporal"]], + ["kafka", ["kafka", "redpanda"]], + ["rabbitmq", ["rabbitmq"]], + ["mongodb", ["mongo"]], +]); + +const EXISTING_COMPOSE_FILE_NAMES = [ + "docker-compose.yml", + "docker-compose.yaml", + "compose.yml", + "compose.yaml", +] as const; + +/** + * Repo-root `docker-compose*.yml`/`compose*.yml` files found alongside the + * project being initialized (not `.hack/docker-compose.yml`, which hack + * itself owns). Used both to mark backing-service detections as "already + * containerized elsewhere" and to surface a standalone pointer finding so + * agents/humans treat the existing file as ground truth for images/backing + * services instead of re-deriving them from scratch. + */ +export async function findExistingComposeFiles(opts: { + readonly repoRoot: string; +}): Promise { + const found: string[] = []; + for (const fileName of EXISTING_COMPOSE_FILE_NAMES) { + const path = resolve(opts.repoRoot, fileName); + if (await pathExists(path)) { + found.push(path); + } + } + return found; +} + +async function hasExistingComposeWithBackingServices(opts: { + readonly repoRoot: string; + readonly kinds: readonly BackingServiceKind[]; +}): Promise> { + const found = new Set(); + if (opts.kinds.length === 0) { + return found; + } + + const composeFiles = await findExistingComposeFiles({ + repoRoot: opts.repoRoot, + }); + + for (const path of composeFiles) { + const text = (await readTextFile(path)) ?? ""; + const lower = text.toLowerCase(); + for (const kind of opts.kinds) { + const hints = BACKING_SERVICE_IMAGE_HINTS.get(kind) ?? []; + if (hints.some((hint) => lower.includes(hint))) { + found.add(kind); + } + } + } + + return found; +} + +/** + * Sentinel image value to assign on a draft when `detectPackageRuntime` + * reports a non-JS runtime. `renderCompose` (see `./compose.ts`) replaces + * this with a visibly wrong placeholder image (`alpine:3`) instead of + * silently defaulting to the bun-node image — the previous silent-wrong-image + * behavior is exactly what the field report flagged. + */ +export function runtimeTodoImageSentinel(): string { + return TODO_IMAGE_SENTINEL; +} + +/** Renders the compose-comment + warning text for a non-JS runtime finding. */ +export function describeUnknownRuntimeComment(opts: { + readonly serviceName: string; + readonly runtime: PackageRuntime; +}): string { + return `TODO(hack-init): ${opts.serviceName} looks like a ${opts.runtime} service — set a matching image and command`; +} + +/** Human-readable one-line warning (or info, for dedupe) for a finding. */ +export function describeFinding(finding: InitDiscoveryFinding): { + readonly level: "warn" | "info"; + readonly message: string; +} { + if (finding.kind === "duplicate-script") { + return { + level: "info", + message: `${finding.packageRelativeDir}: dropped duplicate/aggregator script "${finding.droppedScriptName}" — kept "${finding.keptScriptName}"`, + }; + } + + if (finding.kind === "port-reassigned") { + return { + level: "warn", + message: `${finding.serviceName}: port ${finding.fromPort} was already taken — reassigned to ${finding.toPort}`, + }; + } + + if (finding.kind === "unknown-runtime") { + return { + level: "warn", + message: `${finding.serviceName}: looks like a ${finding.runtime} service — using placeholder image "${TODO_IMAGE_PLACEHOLDER}"; set a matching image and command`, + }; + } + + if (finding.kind === "existing-compose-found") { + return { + level: "warn", + message: `existing compose found at ${finding.path} — treat it as ground truth for backing services and images`, + }; + } + + const suggestion = + BACKING_SERVICE_SUGGESTIONS.get(finding.serviceKind) ?? + "review this service manually"; + const containerizedNote = finding.alreadyContainerized + ? " (already containerized elsewhere)" + : ""; + return { + level: "warn", + message: `backing service "${finding.serviceKind}" detected${containerizedNote} — ${suggestion} (evidence: ${finding.evidence.join(", ")})`, + }; +} + +/** + * Logs one warning (or info, for dedupe notes) per finding via the shared + * CLI logger. Call after compose generation in both the interactive and + * `--auto` init paths. + */ +export function reportInitDiscoveryFindings(opts: { + readonly findings: readonly InitDiscoveryFinding[]; +}): void { + for (const finding of opts.findings) { + const { level, message } = describeFinding(finding); + if (level === "info") { + logger.info({ message }); + } else { + logger.warn({ message }); + } + } +} + +/** + * Compose header comment lines summarizing backing-service detections and + * any existing repo-root compose file found alongside the generated one. + * Used as `ComposePlan.headerComments` (see `./compose.ts`). + */ +export function buildDiscoveryHeaderComments(opts: { + readonly detections: readonly BackingServiceDetection[]; + readonly existingComposeFiles?: readonly string[]; +}): readonly string[] { + const lines: string[] = []; + + if (opts.existingComposeFiles && opts.existingComposeFiles.length > 0) { + lines.push( + "An existing compose file was found in this repo — treat it as ground", + "truth for backing-service images/config rather than re-deriving them:" + ); + for (const path of opts.existingComposeFiles) { + lines.push(`- ${path}`); + } + } + + if (opts.detections.length > 0) { + if (lines.length > 0) { + lines.push(""); + } + lines.push( + "hack-init discovery: possible backing services were detected but not scaffolded.", + "Review and add services manually, or point env vars at existing instances." + ); + + for (const detection of opts.detections) { + const suggestion = + BACKING_SERVICE_SUGGESTIONS.get(detection.kind) ?? + "review this service manually"; + const containerizedNote = detection.alreadyContainerized + ? " (already containerized elsewhere)" + : ""; + lines.push(`- ${detection.kind}${containerizedNote}: ${suggestion}`); + } + } + + return lines; +} + +/** Builds the `existing-compose-found` findings for any repo-root compose files. */ +export function buildExistingComposeFindings(opts: { + readonly existingComposeFiles: readonly string[]; +}): readonly InitDiscoveryFinding[] { + return opts.existingComposeFiles.map((path) => ({ + kind: "existing-compose-found" as const, + path, + })); +} diff --git a/tests/init-discovery-validation.test.ts b/tests/init-discovery-validation.test.ts new file mode 100644 index 00000000..7b59166e --- /dev/null +++ b/tests/init-discovery-validation.test.ts @@ -0,0 +1,457 @@ +import { afterEach, expect, test } from "bun:test"; +import { mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { YAML } from "bun"; + +import { + buildDiscoveredComposeAuto, + type ComposeWizardInput, +} from "../src/commands/project.ts"; +import type { ComposePlan } from "../src/init/compose.ts"; +import { renderCompose, TODO_IMAGE_PLACEHOLDER } from "../src/init/compose.ts"; +import type { ServiceCandidate } from "../src/init/discovery.ts"; +import { discoverRepo } from "../src/init/discovery.ts"; +import { + dedupeAggregatorCandidates, + dedupeCandidates, + dedupeCandidatesByPackage, + detectBackingServices, + detectPackageRuntime, + findExistingComposeFiles, + reassignCollidingPorts, +} from "../src/init/validation.ts"; + +let tempDir: string | null = null; + +afterEach(async () => { + if (tempDir) { + await rm(tempDir, { recursive: true, force: true }); + tempDir = null; + } +}); + +async function writeJson(path: string, value: unknown): Promise { + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, `${JSON.stringify(value, null, 2)}\n`); +} + +async function writeText(path: string, value: string): Promise { + await mkdir(dirname(path), { recursive: true }); + await writeFile(path, value); +} + +function candidate(overrides: Partial): ServiceCandidate { + return { + id: "pkg:dev", + packageId: "pkg", + packageRelativeDir: "apps/x", + packageName: "@repo/x", + scriptName: "dev", + scriptCommand: "vite", + ...overrides, + }; +} + +/** + * Scaffolds the polyglot fixture used throughout this file: + * - root package.json (workspaces [apps/*, backend]) with aggregator + * scripts `dev:web` (turbo filter) and `dev:backend` (dotnet run + * --project) that delegate to package-local dev scripts. + * - apps/web: vite package with both `dev` and `start` scripts (same- + * package duplicate) plus a `PORT=3000` reference nowhere — port comes + * from the default guess. + * - apps/api: bun http package with a plain `dev` script (no port flags). + * - backend/: package.json wrapper (`dev: dotnet watch`) plus an + * `Api.csproj` marker file (dotnet runtime). + * - root .env.example with DATABASE_URL + TEMPORAL_ADDRESS placeholders. + * - root package.json also declares `pg` + `@temporalio/client` deps. + */ +async function writePolyglotFixture(repoRoot: string): Promise { + await writeJson(join(repoRoot, "package.json"), { + name: "root", + private: true, + workspaces: ["apps/*", "backend"], + dependencies: { + pg: "^8.0.0", + "@temporalio/client": "^1.0.0", + }, + scripts: { + "dev:web": "turbo run dev --filter=web", + "dev:backend": "dotnet run --project backend/Api.csproj", + }, + }); + + await writeJson(join(repoRoot, "apps/web/package.json"), { + name: "web", + scripts: { + dev: "vite", + start: "vite preview", + }, + }); + + await writeJson(join(repoRoot, "apps/api/package.json"), { + name: "api", + scripts: { + dev: "bun index.ts", + }, + }); + + await writeJson(join(repoRoot, "backend/package.json"), { + name: "backend", + scripts: { + dev: "dotnet watch", + }, + }); + await writeText(join(repoRoot, "backend/Api.csproj"), ""); + + await writeText( + join(repoRoot, ".env.example"), + [ + "DATABASE_URL=postgres://user:pass@localhost:5432/app", + "TEMPORAL_ADDRESS=localhost:7233", + "", + ].join("\n") + ); +} + +test("dedupeCandidatesByPackage keeps the highest-scored script per package", () => { + const dev = candidate({ id: "web:dev", scriptName: "dev" }); + const start = candidate({ id: "web:start", scriptName: "start" }); + + const result = dedupeCandidatesByPackage({ candidates: [dev, start] }); + + expect(result.selected).toEqual([dev]); + expect(result.dropped).toEqual([{ candidate: start, keptScriptName: "dev" }]); +}); + +test("dedupeAggregatorCandidates drops a root --filter aggregator in favor of the package's own script", () => { + const webDev = candidate({ + id: "apps/web/package.json:dev", + packageId: "apps/web/package.json", + packageRelativeDir: "apps/web", + packageName: "web", + scriptName: "dev", + scriptCommand: "vite", + }); + const rootAggregator = candidate({ + id: "package.json:dev:web", + packageId: "package.json", + packageRelativeDir: ".", + packageName: "root", + scriptName: "dev:web", + scriptCommand: "turbo run dev --filter=web", + }); + + const result = dedupeAggregatorCandidates({ + candidates: [rootAggregator, webDev], + }); + + expect(result.selected).toEqual([webDev]); + expect(result.dropped).toEqual([ + { candidate: rootAggregator, keptScriptName: "dev" }, + ]); +}); + +test("dedupeAggregatorCandidates drops a root --project aggregator targeting a package with its own dev script", () => { + const backendDev = candidate({ + id: "backend/package.json:dev", + packageId: "backend/package.json", + packageRelativeDir: "backend", + packageName: "backend", + scriptName: "dev", + scriptCommand: "dotnet watch", + }); + const rootAggregator = candidate({ + id: "package.json:dev:backend", + packageId: "package.json", + packageRelativeDir: ".", + packageName: "root", + scriptName: "dev:backend", + scriptCommand: "dotnet run --project backend/Api.csproj", + }); + + const result = dedupeAggregatorCandidates({ + candidates: [rootAggregator, backendDev], + }); + + expect(result.selected).toEqual([backendDev]); + expect(result.dropped).toEqual([ + { candidate: rootAggregator, keptScriptName: "dev" }, + ]); +}); + +test("dedupeCandidates full pipeline on the polyglot fixture yields exactly one candidate per package", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-dedupe-")); + const repoRoot = join(tempDir, "repo"); + await mkdir(repoRoot, { recursive: true }); + await writePolyglotFixture(repoRoot); + + const discovery = await discoverRepo(repoRoot); + const result = dedupeCandidates({ candidates: discovery.candidates }); + + const byPackage = new Map(); + for (const c of result.selected) { + byPackage.set(c.packageId, (byPackage.get(c.packageId) ?? 0) + 1); + } + for (const count of byPackage.values()) { + expect(count).toBe(1); + } + + // 3 packages have qualifying dev scripts: apps/web, apps/api, backend. + // The root package's dev:web / dev:backend aggregators are dropped. + expect(result.selected.length).toBe(3); + const packageDirs = result.selected.map((c) => c.packageRelativeDir).sort(); + expect(packageDirs).toEqual(["apps/api", "apps/web", "backend"]); + + const findingKinds = result.findings.map((f) => f.kind); + expect(findingKinds.every((k) => k === "duplicate-script")).toBe(true); + // web/start dropped (same-package) + dev:web + dev:backend aggregators dropped. + expect(result.findings.length).toBe(3); +}); + +test("reassignCollidingPorts reassigns duplicates ascending from the collision and rewrites the command", () => { + const webCandidate = candidate({ + id: "web", + packageRelativeDir: "apps/web", + scriptName: "dev", + scriptCommand: "vite", + }); + const apiCandidate = candidate({ + id: "api", + packageRelativeDir: "apps/api", + scriptName: "dev", + scriptCommand: "bun index.ts", + }); + const adminCandidate = candidate({ + id: "admin", + packageRelativeDir: "apps/admin", + scriptName: "dev", + scriptCommand: "vite", + }); + + const result = reassignCollidingPorts({ + drafts: [ + { name: "web", role: "http", port: 3000, candidate: webCandidate }, + { name: "api", role: "http", port: 3000, candidate: apiCandidate }, + { name: "admin", role: "http", port: 3000, candidate: adminCandidate }, + ], + }); + + expect(result.reassignments.has("web")).toBe(false); + expect(result.reassignments.get("api")?.port).toBe(3001); + expect(result.reassignments.get("admin")?.port).toBe(3002); + expect(result.reassignments.get("api")?.command).toContain("3001"); + expect(result.findings.length).toBe(2); + expect(result.findings.every((f) => f.kind === "port-reassigned")).toBe(true); +}); + +test("detectPackageRuntime finds dotnet via a *.csproj marker file", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-runtime-")); + const dir = join(tempDir, "backend"); + await mkdir(dir, { recursive: true }); + await writeText(join(dir, "Api.csproj"), ""); + + const runtime = await detectPackageRuntime({ dir }); + expect(runtime).toBe("dotnet"); +}); + +test("detectPackageRuntime resolves a root aggregator's --project reference to the target dir's runtime", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-runtime-root-")); + const repoRoot = join(tempDir, "repo"); + await mkdir(repoRoot, { recursive: true }); + await mkdir(join(repoRoot, "backend"), { recursive: true }); + await writeText(join(repoRoot, "backend/Api.csproj"), ""); + + const runtime = await detectPackageRuntime({ + dir: repoRoot, + scriptCommand: "dotnet run --project backend/Api.csproj", + repoRoot, + }); + expect(runtime).toBe("dotnet"); +}); + +test("detectPackageRuntime returns null for a plain JS/TS package", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-runtime-js-")); + const dir = join(tempDir, "web"); + await mkdir(dir, { recursive: true }); + await writeJson(join(dir, "package.json"), { name: "web" }); + + const runtime = await detectPackageRuntime({ + dir, + scriptCommand: "vite", + }); + expect(runtime).toBeNull(); +}); + +test("detectBackingServices finds postgres + temporal from deps and .env.example keys (values excluded)", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-backing-")); + const repoRoot = join(tempDir, "repo"); + await mkdir(repoRoot, { recursive: true }); + await writePolyglotFixture(repoRoot); + + const discovery = await discoverRepo(repoRoot); + const detections = await detectBackingServices({ + repoRoot, + packages: discovery.packages, + }); + + const kinds = detections.map((d) => d.kind).sort(); + expect(kinds).toEqual(["postgres", "temporal"]); + + const postgres = detections.find((d) => d.kind === "postgres"); + expect(postgres?.evidence.some((e) => e.includes("pg"))).toBe(true); + expect(postgres?.evidence.some((e) => e.includes("DATABASE_URL"))).toBe(true); + // Evidence must never leak the secret value, only the key name. + for (const detection of detections) { + for (const e of detection.evidence) { + expect(e).not.toContain("user:pass"); + expect(e).not.toContain("localhost:5432"); + expect(e).not.toContain("localhost:7233"); + } + } +}); + +test("findExistingComposeFiles finds a root docker-compose.yml", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-compose-")); + const repoRoot = join(tempDir, "repo"); + await mkdir(repoRoot, { recursive: true }); + await writeText( + join(repoRoot, "docker-compose.yml"), + "services:\n db:\n image: postgres:16\n temporal:\n image: temporalio/auto-setup:latest\n" + ); + + const found = await findExistingComposeFiles({ repoRoot }); + expect(found.length).toBe(1); + expect(found[0]?.endsWith("docker-compose.yml")).toBe(true); +}); + +test("buildDiscoveredComposeAuto on the polyglot fixture: one service per package, distinct ports, dotnet TODO image, backing-service + existing-compose findings", async () => { + tempDir = await mkdtemp(join(tmpdir(), "hack-validation-auto-")); + const repoRoot = join(tempDir, "repo"); + await mkdir(repoRoot, { recursive: true }); + await writePolyglotFixture(repoRoot); + + // Root docker-compose.yml with postgres + temporal — should surface an + // "existing-compose-found" finding and mark those backing services as + // already containerized. + await writeText( + join(repoRoot, "docker-compose.yml"), + [ + "services:", + " db:", + " image: postgres:16", + " temporal:", + " image: temporalio/auto-setup:latest", + "", + ].join("\n") + ); + + const discovery = await discoverRepo(repoRoot); + + const input: ComposeWizardInput = { + repoRoot, + devHost: "polyglot.hack", + projectSlug: "polyglot", + candidates: discovery.candidates, + oauth: { enabled: false, tld: "gy" }, + }; + + const composeYaml = await buildDiscoveredComposeAuto(input); + const parsed = YAML.parse( + composeYaml + .split("\n") + .filter((line) => !line.trim().startsWith("#")) + .join("\n") + ) as { + services: Record< + string, + { image: string; command: string; labels?: Record } + >; + }; + + const serviceNames = Object.keys(parsed.services).sort(); + // web, api, backend — no web-2/backend-2 duplicates and no separate + // dev:web/dev:backend aggregator services. + expect(serviceNames).toEqual(["api", "backend", "web"]); + + const ports = new Set(); + for (const name of serviceNames) { + const label = parsed.services[name]?.labels?.["caddy.reverse_proxy"]; + if (label) { + expect(ports.has(label)).toBe(false); + ports.add(label); + } + } + + // The dotnet-backed "backend" service must not silently get the default + // bun-node image. + expect(parsed.services.backend?.image).toBe(TODO_IMAGE_PLACEHOLDER); + expect(composeYaml).toContain( + "TODO(hack-init): backend looks like a dotnet service" + ); + + // Backing services + existing-compose pointer surfaced as header comments. + expect(composeYaml).toContain("hack-init discovery notes"); + expect(composeYaml).toContain("postgres"); + expect(composeYaml).toContain("temporal"); + expect(composeYaml).toContain("already containerized elsewhere"); + expect(composeYaml).toContain("docker-compose.yml"); +}); + +test("renderCompose header + service comment injection is stable and idempotent across repeated renders", () => { + const plan: ComposePlan = { + name: "idempotent-test", + headerComments: ["backing service notes", "- postgres: add a service"], + services: [ + { + name: "backend", + role: "internal", + image: TODO_IMAGE_PLACEHOLDER, + workingDir: "/app/backend", + command: "dotnet watch", + env: new Map(), + labels: new Map(), + networks: [], + comments: ["TODO(hack-init): backend looks like a dotnet service"], + }, + { + name: "web", + role: "http", + image: "imbios/bun-node:latest", + workingDir: "/app/apps/web", + command: "bun run dev -- --port 3000 --host 0.0.0.0", + env: new Map(), + labels: new Map([["caddy", "web.myapp.hack"]]), + networks: ["hack-dev", "default"], + }, + ], + }; + + const first = renderCompose(plan); + const second = renderCompose(plan); + expect(first).toBe(second); + + // Header block appears exactly once. + const headerOccurrences = first.split("hack-init discovery notes").length - 1; + expect(headerOccurrences).toBe(1); + + // Service comment appears exactly once and directly above the service key. + const commentOccurrences = + first.split("TODO(hack-init): backend looks like a dotnet service").length - + 1; + expect(commentOccurrences).toBe(1); + expect(first).toMatch( + /# TODO\(hack-init\): backend looks like a dotnet service\n {2}backend:/ + ); + + // Re-rendering against already-commented output (simulated by rendering + // twice) does not duplicate the injected lines. + const parsed = YAML.parse( + first + .split("\n") + .filter((line) => !line.trim().startsWith("#")) + .join("\n") + ) as Record; + expect(parsed.services).toBeDefined(); +}); From 739a58177622b08c66c68eeb8c7e09aa12c3d0ad Mon Sep 17 00:00:00 2001 From: Dimitri Kennedy Date: Tue, 7 Jul 2026 21:28:24 -0400 Subject: [PATCH 3/3] feat(init): surface the hack global install requirement in init output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both init paths now tell first-time users/agents that full *.hack routing needs hack global install (sudo, machine-wide DNS/CA) — field agents hit this as a late surprise. Co-Authored-By: Claude Fable 5 --- src/commands/project.ts | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/commands/project.ts b/src/commands/project.ts index b762dbba..b207a1ad 100644 --- a/src/commands/project.ts +++ b/src/commands/project.ts @@ -3420,6 +3420,8 @@ async function handleInit({ "Next:", " hack up", " hack open", + "", + "First time on this machine? Run `hack global install` for *.hack DNS/TLS (needs sudo).", ].join("\n"), "Initialized" ); @@ -3567,6 +3569,10 @@ async function handleInitAuto({ logger.info({ message: "Next: hack up --detach && hack open", }); + logger.info({ + message: + "First time on this machine? Run `hack global install` for *.hack DNS/TLS (needs sudo).", + }); if (withValue) { await runInitOnboardingHandoff({