Skip to content

fix(reachability): announce a target that is never called (finding B) - #60

Merged
GuilhermeBn198 merged 1 commit into
developfrom
fix/target-coverage-warning
Aug 22, 2026
Merged

fix(reachability): announce a target that is never called (finding B)#60
GuilhermeBn198 merged 1 commit into
developfrom
fix/target-coverage-warning

Conversation

@GuilhermeBn198

Copy link
Copy Markdown
Collaborator

Finding B, measured

Finding B was logged in August as a structural suspicion: TargetPass instruments call sites whose callee matches --target-function-name, and a program does not call its own main, so nothing gets instrumented. The CASTLE v6 baseline turns the suspicion into a number.

98 of 217 runs (45%) went through --target-function --target-function-name main:

verdict n
TRUE 57
TIMEOUT 22
UNKNOWN 10
ERROR 9
FALSE-* 0

Not one FALSE. That is not scarcity — across the same 217 runs the other modes produce 49 FALSE-* verdicts. This mode is the only one that never accuses anything.

59 of those 98 programs were known to be vulnerable. Zero were detected.

The scoring damage has two sides, and the second is the uncomfortable one:

effect
5 FN guaranteed misses — the oracle could not have got them right
12 TN credit for answers an oracle that can only say "correct" could not have failed to give

Excluding the mode entirely: recall 74.0% → 79.4%, precision unchanged at 98.2% (the mode yields neither TP nor FP). So B does not inflate precision — it depresses recall and inflates TN.

Correcting an earlier reading of mine: the 61 N/A rows are N/A because the harness declares those CWEs out of scope, not because of finding B. B's real cost is the 17 scored cases.

What this PR changes

Zero call sites is not an error. If a program genuinely never calls reach_error, then "the error is unreachable" is the right answer and TRUE is correct. The defect is that such a TRUE is indistinguishable in the output from a TRUE earned by exploring the program.

So the fix is visibility, not verdicts. TargetPass scans the module once and, when the target has no call sites, says what the verdict is worth there:

[map2check] WARNING: target function 'main' is never called in this module.
TargetPass instruments call sites, so nothing was instrumented and the
reachability property holds trivially: a TRUE verdict here means the target
is absent, not that the program is safe.

Reported once per module rather than once per function — this is a function pass with no module-entry hook, and opt runs once per invocation.

Tests

tests/integration/test_target_coverage.sh, wired into the regression job. Four assertions, and it asserts both directions — a warning that always fires is as useless as one that never does:

PASS a called target stays quiet
PASS an uncalled target announces itself
PASS the warning names the target it is about
PASS the warning says what a TRUE verdict is worth here

Verified against a full build and install, not the pass in isolation.

Deliberately not in this PR

The CASTLE harness maps 13 CWEs onto this mode as an escape default (tests/castle/run_castle_evaluation.sh:34-47). That is the other half of finding B, and fixing it changes baseline scoring — so it waits for the v6 run to finish. Changing the scoring criterion mid-run is exactly what makes two measurements incomparable.

Docs

TargetPass instruments call sites whose callee matches the requested name.
When the program never calls that function, nothing is instrumented and the
reachability property holds trivially -- and nothing in the output says so.

Zero call sites is not an error. A program that genuinely never calls
reach_error IS safe with respect to reach_error, and TRUE is correct there.
The defect is that such a TRUE is indistinguishable from a TRUE earned by
exploring the program. So the fix is visibility, not a change of verdict: the
pass scans the module once and, when the target has no call sites, says what
the verdict is worth.

Measured on the CASTLE v6 baseline, which is what moves this from a suspicion
to a defect with a number. 98 of 217 runs used
`--target-function-name main` -- a program does not call its own entry point:

  TRUE 57, TIMEOUT 22, UNKNOWN 10, ERROR 9, FALSE-* 0

Not one FALSE, while the other modes produce 49 across the same corpus. 59 of
those 98 programs were known to be vulnerable and none was detected. Seventeen
were scored: 5 FN, and 12 TN -- credit for answers an oracle that can only say
"correct" could not have failed to give. Excluding the mode, recall goes from
74.0% to 79.4%; precision is unchanged at 98.2%, since the mode yields neither
TP nor FP.

Correcting my own earlier reading: the 61 N/A rows are N/A because the harness
declares those CWEs out of scope, not because of finding B. B's real cost is
the 17 scored cases.

Still open, deliberately: the harness maps 13 CWEs onto this mode as an escape
default. Fixing that changes baseline scoring, so it waits for the v6 run to
finish rather than moving the criterion mid-run.

Verified end to end against a full build, not just the pass in isolation:
test_target_coverage.sh 4/4.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GuilhermeBn198
GuilhermeBn198 merged commit e1438e5 into develop Aug 22, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant