Multi-Tool Secret Scanner

Multi-Tool Secret Scanner runs several established secret-detection CLIs over complete Git history or a filesystem tree, preserves their raw evidence, and produces normalized CSV and JSON summaries.

It orchestrates:

• Gitleaks
• Betterleaks
• TruffleHog
• Titus

Using overlapping scanners is intentional: their rules, validation features, archive handling, and false-positive tradeoffs differ. The script does not replace the scanners or deduplicate findings between them.

Warning

Raw reports can contain complete, live credentials as well as repository, file, and commit metadata. Never commit, upload, email, or attach a results directory without reviewing and sanitizing it first.

Features

• Scans every ref in bare or mirror Git repositories.
• Scans a file, one filesystem tree, or each immediate child as a separate target.
• Runs targets concurrently while isolating failures by tool and target.
• Optionally validates detected credentials through scanner-provided network checks.
• Preserves each tool's native report for remediation.
• Produces redacted normalized findings by default.
• Protects CSV output against spreadsheet formula injection.
• Creates owner-only output directories and files (0700/0600 on POSIX).
• Rejects output directories inside a filesystem target to prevent a scan from consuming its own results.
• Returns a nonzero status when any requested scan has incomplete coverage.

Requirements

• Python 3.9 or newer
• Git, when using Git mode
• At least one supported scanner available on PATH

The Python application has no runtime package dependencies. Install whichever scanner CLIs you intend to use with the included installer:

python3 install.py --dry-run
python3 install.py

install.py supports macOS and Linux on Intel/AMD64 and ARM64. It downloads official GitHub release assets, verifies their published SHA-256 checksums, and installs executables in ~/.local/bin without sudo. Add that directory to PATH if the installer tells you it is not already present.

The normal install uses the known-compatible release tags recorded in tool_versions.py. It does not silently replace working tools with an untested upstream release. Use the explicit check/upgrade flow to stay current:

# Read-only: report missing, outdated, current, or newer tools
python3 install.py --check

# Preview upgrades to the latest official releases
python3 install.py --upgrade --dry-run

# Upgrade only missing or outdated tools
python3 install.py --upgrade

--check exits with status 1 when a tool is missing, outdated, or does not match an explicitly requested version, making it suitable for scheduled jobs.

Useful installer options:

# Install selected scanners only
python3 install.py --tools gitleaks,trufflehog

# Reinstall the known-compatible releases
python3 install.py --force

# Pin release tags for reproducibility
python3 install.py --version gitleaks=v8.30.1 --version titus=v1.2.7

# Choose another user-writable destination
python3 install.py --install-dir ./tools/bin

Every downloaded executable is run from its temporary location and must report the release version expected from GitHub before it atomically replaces the destination. Building from source is intentionally not the default: it adds Go and native-library dependencies while being no more reproducible than a verified, tagged release.

The installer fails closed when a release lacks a matching checksum. The --allow-unverified escape hatch exists for exceptional upstream releases but is not recommended. Set GITHUB_TOKEN in the environment if unauthenticated GitHub API rate limits are a concern; the token is not written to disk.

Alternatively, follow each scanner's official installation instructions. Homebrew packages are available for Gitleaks, Betterleaks, and TruffleHog; Titus publishes binaries and build instructions in its official repository. Missing scanners are marked MISSING; the run proceeds when at least one requested scanner is installed, but affected targets are partial and the process exits nonzero.

Installation

Run the script directly:

python3 secretscan.py --help

Or install an editable command in a virtual environment:

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install --editable .
secret-scan --help

Usage

Run without target options to enter a path interactively (Tab completes files and directories). A directory containing Git mirrors is scanned in Git-history mode; other paths, including Manspider or SMB crawler output, are scanned as a filesystem target:

python3 secretscan.py

Complete Git history

Mirror clones include branches, tags, and other refs without a working tree:

mkdir -p mirrors
git clone --mirror https://github.com/example/example-repository.git \
  mirrors/example-repository.git

python3 secretscan.py \
  --mode git \
  --repos-dir ./mirrors

Useful variations:

# Scan one mirror
python3 secretscan.py --repos-dir ./mirrors --repo example-repository

# Smoke-test the first five discovered mirrors
python3 secretscan.py --repos-dir ./mirrors --limit 5

# Select scanners and target concurrency
python3 secretscan.py --repos-dir ./mirrors \
  --tools gitleaks,trufflehog,titus --workers 8

Filesystem data

# Treat the entire tree as one target
python3 secretscan.py --mode filesystem --target-dir ./collected-files

# Scan every immediate child separately
python3 secretscan.py --mode filesystem --target-dir ./collected-files \
  --split-targets

# Enable archive/document extraction where supported
python3 secretscan.py --mode filesystem --target-dir ./collected-files \
  --archive-depth 2 --titus-extract all

The output directory must be outside the filesystem target. Use --out to place it explicitly:

python3 secretscan.py --mode filesystem --target-dir ./collected-files \
  --out ../scan-output

Live credential verification

Verification is disabled by default. Enable it only when you are authorized to make validation requests:

python3 secretscan.py --repos-dir ./mirrors --verify

Verification can contact third-party APIs, generate audit events, trigger alerts, or encounter rate limits. Tool-specific behavior remains controlled by the installed scanner version.

Output

Each run writes to secret_scan_results_<timestamp> unless --out is given. The directory must be new or empty.

Path	Contents	Sensitivity
findings.csv	One normalized row per finding	Secret preview redacted by default
summary.csv	Per-target counts, severity, and coverage status	May contain target names
summary.json	Machine-readable run totals	May contain run metadata
scan.log	Progress, warnings, and errors	May contain target paths/names
per_repo/<name>/	Raw reports for Git targets	Unredacted; highly sensitive
per_target/<name>/	Raw reports for filesystem targets	Unredacted; highly sensitive

Use --no-redact only when a full secret is truly required in findings.csv. Raw per-tool reports remain unredacted regardless of that option.

Target statuses are:

• ok: every selected and available tool completed.
• partial: at least one tool completed and at least one failed or timed out.
• error: no selected tool completed for the target.
• skipped-empty: the target contained no commits or files.

Exit status is 0 for complete coverage, 2 when one or more targets are partial or error, and 130 after an interrupted run. Scanner findings by themselves do not produce a nonzero exit status.

Security notes

• Treat the output as incident-response evidence, not ordinary build output.
• Store results on an encrypted volume with access limited to the remediation team.
• Rotate or revoke exposed credentials before distributing reports.
• Sanitizing findings.csv is not enough; native reports and logs also require review.
• .gitignore covers the default output layout and common credential files, but it cannot anticipate every custom --out path.
• Run scans only against systems and data you are authorized to assess.

Titus reports do not currently expose a commit identifier in the normalized mapping used here, so its commit column is left blank. For exact tool-native context, consult the corresponding raw report.

Development

Run the dependency-free test suite:

python3 -m unittest discover -s tests -v
python3 -m py_compile secretscan.py install.py tool_versions.py

Test fixtures and bug reports must contain synthetic values only. See CONTRIBUTING.md and SECURITY.md.

License

Released under the MIT License.

Responsible use and disclaimer

Use this software only on systems, repositories, and data you own or have explicit written authorization to assess. You are responsible for safely handling findings and credentials, obtaining any required consent before live verification, and complying with applicable laws and third-party terms.

The software is provided "as is", without warranty, and the authors and copyright holders disclaim liability to the fullest extent permitted by law. See the MIT License for the complete terms. This usage notice does not modify or restrict the license.