Skip to content

Hardening: thread-aware decision extraction + mid-confidence review queue #65

Description

@cursor

Background

M-005: naive single-message extraction hallucinates or misses decisions in noisy Slack threads.

Problem Statement

Real Slack/GitHub threads span multiple messages; low-confidence extractions can pollute the graph if written directly.

Proposed Solution

  • Extract from thread/context window, not only a single message
  • Classification-first (is this a decision?) then field extraction
  • Confidence bands: >0.7 write; 0.4–0.7 human review queue; <0.4 discard
  • Review UI or API for mid-confidence items (quarantine store may already help)

Acceptance Criteria

  • Thread/context extraction path with tests
  • Mid-confidence items go to review/quarantine, not silent graph write
  • Documented thresholds in README or docs
  • Fixture set of multi-message samples covered by tests

Files / Modules Affected

extraction/decision_extractor.py, pipeline/extraction_worker.py, memory/quarantine.py, frontend Review (optional), tests

Related: M-005 in MISTAKES.md

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions