Context
src/review/content-lane/duplicates.ts synthesizes frontmatter for directory-index entries so they can be run through the same duplicate-detection signal extraction as real content files. Two code paths disagree on which URL fields are considered:
extractContentDuplicateSignals (around line 270) correctly honors the caller-supplied spec: ContentRepoSpec:
const urls = [
...new Set(
Object.entries(fields)
.filter(([key]) => spec.urlFields.has(key))
.map(([, value]) => normalizeUrl(value, spec))
.filter(Boolean),
),
];
But contentSignalSourceFromDirectoryEntry (around line 549), which builds the synthetic frontmatter that later gets parsed and fed into extractContentDuplicateSignals, does not take a spec parameter at all — it always emits only a hardcoded field list:
const DIRECTORY_ENTRY_URL_SIGNAL_FIELDS = [/* ...9 hardcoded field names... */, "sourceUrl", "websiteUrl"] as const;
function contentSignalSourceFromDirectoryEntry(entry: DirectoryIndexEntry): string {
const lines = ["---", `title: ${yamlScalar(entry.title)}`, ...];
for (const field of DIRECTORY_ENTRY_URL_SIGNAL_FIELDS) {
const value = entry[field];
if (value) lines.push(`${field}: ${yamlScalar(value)}`);
}
lines.push("---", "");
return lines.join("\n");
}
directoryIndexToSignals (around line 591) does thread spec through to extractContentDuplicateSignals, but passes contentSignalSourceFromDirectoryEntry(entry) (no spec) as the content argument:
extractContentDuplicateSignals(
{ filePath, content: contentSignalSourceFromDirectoryEntry(entry), label: ..., url: ... },
spec,
)
For a self-hosted content repo whose ContentRepoSpec.urlFields uses field names other than the 9 hardcoded ones in DIRECTORY_ENTRY_URL_SIGNAL_FIELDS, none of that repo's URL fields ever make it into the synthesized frontmatter — urls in the resulting ContentDuplicateSignals is silently empty for every directory-index entry, breaking URL/domain-based duplicate detection against the accepted-entries corpus for any non-default ContentRepoSpec.
test/unit/content-lane-duplicates.test.ts only tests removing a URL field from the default spec, never a differently-named custom field, so this drift is untested.
Requirements
contentSignalSourceFromDirectoryEntry must accept a spec: ContentRepoSpec parameter (defaulting to AWESOME_CLAUDE_CONTENT_SPEC, matching the existing default used elsewhere in this file) and emit frontmatter lines for the fields named in spec.urlFields instead of (or in addition to, if a safe superset approach is simpler) the hardcoded DIRECTORY_ENTRY_URL_SIGNAL_FIELDS list.
directoryIndexToSignals must pass its own spec argument through to contentSignalSourceFromDirectoryEntry, so the synthesized frontmatter and the downstream extractContentDuplicateSignals filtering agree on which fields are URL signals.
- Preserve behavior for the default
AWESOME_CLAUDE_CONTENT_SPEC case exactly (same synthesized frontmatter output as today) — this is a fix for the custom-spec case, not a behavior change for the default lane.
- Keep
DirectoryIndexEntry's field access safe for entries missing a given custom field (same if (value) ... guard pattern already in use).
Deliverables
Test Coverage Requirements
Aim for 99%+ Codecov patch coverage (100% including the new branch/invariant) on the touched lines in src/review/content-lane/duplicates.ts. This is a fix for silently-empty duplicate-detection signals on a real self-host configuration path, so the custom-spec regression test is required, not just incidental line coverage.
Expected Outcome
A self-hosted content repo with a custom ContentRepoSpec.urlFields gets working URL/domain-based duplicate detection against accepted directory-index entries, instead of urls being silently empty because the synthesized frontmatter only ever emitted the default spec's hardcoded field names.
Links & Resources
src/review/content-lane/duplicates.ts (extractContentDuplicateSignals, ~line 260-280; DIRECTORY_ENTRY_URL_SIGNAL_FIELDS + contentSignalSourceFromDirectoryEntry, ~line 535-561; directoryIndexToSignals, ~line 566-599)
src/review/content-lane/content-repo-spec.ts (ContentRepoSpec, AWESOME_CLAUDE_CONTENT_SPEC)
test/unit/content-lane-duplicates.test.ts (existing test suite)
Context
src/review/content-lane/duplicates.tssynthesizes frontmatter for directory-index entries so they can be run through the same duplicate-detection signal extraction as real content files. Two code paths disagree on which URL fields are considered:extractContentDuplicateSignals(around line 270) correctly honors the caller-suppliedspec: ContentRepoSpec:But
contentSignalSourceFromDirectoryEntry(around line 549), which builds the synthetic frontmatter that later gets parsed and fed intoextractContentDuplicateSignals, does not take aspecparameter at all — it always emits only a hardcoded field list:directoryIndexToSignals(around line 591) does threadspecthrough toextractContentDuplicateSignals, but passescontentSignalSourceFromDirectoryEntry(entry)(nospec) as thecontentargument:For a self-hosted content repo whose
ContentRepoSpec.urlFieldsuses field names other than the 9 hardcoded ones inDIRECTORY_ENTRY_URL_SIGNAL_FIELDS, none of that repo's URL fields ever make it into the synthesized frontmatter —urlsin the resultingContentDuplicateSignalsis silently empty for every directory-index entry, breaking URL/domain-based duplicate detection against the accepted-entries corpus for any non-defaultContentRepoSpec.test/unit/content-lane-duplicates.test.tsonly tests removing a URL field from the default spec, never a differently-named custom field, so this drift is untested.Requirements
contentSignalSourceFromDirectoryEntrymust accept aspec: ContentRepoSpecparameter (defaulting toAWESOME_CLAUDE_CONTENT_SPEC, matching the existing default used elsewhere in this file) and emit frontmatter lines for the fields named inspec.urlFieldsinstead of (or in addition to, if a safe superset approach is simpler) the hardcodedDIRECTORY_ENTRY_URL_SIGNAL_FIELDSlist.directoryIndexToSignalsmust pass its ownspecargument through tocontentSignalSourceFromDirectoryEntry, so the synthesized frontmatter and the downstreamextractContentDuplicateSignalsfiltering agree on which fields are URL signals.AWESOME_CLAUDE_CONTENT_SPECcase exactly (same synthesized frontmatter output as today) — this is a fix for the custom-spec case, not a behavior change for the default lane.DirectoryIndexEntry's field access safe for entries missing a given custom field (sameif (value) ...guard pattern already in use).Deliverables
contentSignalSourceFromDirectoryEntry(entry, spec)synthesizes URL frontmatter lines usingspec.urlFieldsinstead of a fixed hardcoded list.directoryIndexToSignalspasses itsspecargument through tocontentSignalSourceFromDirectoryEntry.test/unit/content-lane-duplicates.test.tsusing aContentRepoSpecwith a custom (non-default)urlFieldsset, asserting the resultingContentDuplicateSignals.urlsfor a directory-index entry actually contains the custom field's URL.AWESOME_CLAUDE_CONTENT_SPECcallers).Test Coverage Requirements
Aim for 99%+ Codecov patch coverage (100% including the new branch/invariant) on the touched lines in
src/review/content-lane/duplicates.ts. This is a fix for silently-empty duplicate-detection signals on a real self-host configuration path, so the custom-spec regression test is required, not just incidental line coverage.Expected Outcome
A self-hosted content repo with a custom
ContentRepoSpec.urlFieldsgets working URL/domain-based duplicate detection against accepted directory-index entries, instead ofurlsbeing silently empty because the synthesized frontmatter only ever emitted the default spec's hardcoded field names.Links & Resources
src/review/content-lane/duplicates.ts(extractContentDuplicateSignals, ~line 260-280;DIRECTORY_ENTRY_URL_SIGNAL_FIELDS+contentSignalSourceFromDirectoryEntry, ~line 535-561;directoryIndexToSignals, ~line 566-599)src/review/content-lane/content-repo-spec.ts(ContentRepoSpec,AWESOME_CLAUDE_CONTENT_SPEC)test/unit/content-lane-duplicates.test.ts(existing test suite)