SensorSentinel

Detecting stealthy data-integrity attacks on cyber-physical sensor telemetry.

SensorSentinel is a compact, reproducible pipeline that flags stealthy attacks on multivariate cyber-physical system (CPS) sensor streams — the kind of data-tampering that stays within each sensor's normal range (so univariate alarms miss it) but corrupts the cross-sensor consistency and temporal dynamics that a learned model can still detect. It is built around a signal-processing view of the data and a research-grade, leak-free evaluation.

Context: this demo targets the data-integrity layer of cyber-physical system security — detecting tampering that stays within each sensor's normal range and only surfaces in cross-sensor consistency and temporal dynamics. It stays entirely on the data / ML side: no power-flow solver or domain simulator required.

---

Result

The LSTM autoencoder detects 92% of attack events at 92% precision (composite-F1 0.92, PR-AUC 0.68) — versus 50% for the classical baselines — even though every injected value stays inside the sensor's normal range.

📄 One-page summary: assets/SensorSentinel_brief.pdf — the same content as this README, as a single attachable page for convenient review.

detector	PR-AUC	ROC-AUC	precision	recall	event-recall	composite-F1
LSTM-Autoencoder	0.68	0.79	0.92	0.42	0.92	0.92
Isolation Forest	0.64	0.87	0.70	0.02	0.50	0.58
Univariate z-score	0.33	0.65	0.66	0.07	0.50	0.57

ROC-AUC nominally favors Isolation Forest, but under heavy class imbalance PR-AUC is the correct headline metric — and the autoencoder wins it clearly while catching nearly every distinct attack event that the baselines miss.

(Figures regenerate with python run.py; the PDF with python scripts/make_brief.py.)

---

What it does

1. Data pipeline — a SWaT-style water-treatment process simulator produces physically-coupled sensor streams (LIT, FIT, MV, PIT, AIT). A drop-in loader ingests the licensed SWaT / WADI CSVs (iTrust, SUTD) when available.
2. Attack library — stealthy, in-range data-integrity attacks: constant bias, slow drift, replay/freeze, and multiplicative scaling. Every attack is clipped to the sensor's normal envelope, so it defeats range checks by construction.
3. ML detector — an LSTM reconstruction autoencoder (PyTorch) trained only on attack-free data; per-timestep reconstruction error is the anomaly score.
4. Baselines — Isolation Forest and a univariate z-score threshold, scored on the same footing to show the ML model earns its complexity.
5. Evaluation — threshold-free PR-AUC / ROC-AUC, plus point-wise and event-aware composite F1. Point-adjusted F1 is reported for reference only and explicitly flagged as inflated (see below).

The point-adjust pitfall (why the metrics here are trustworthy)

Most CPS anomaly-detection papers headline point-adjusted (PA) F1: if any single point inside a true anomaly segment is flagged, the whole segment is counted detected. Kim et al. (Towards a Rigorous Evaluation of Time-Series Anomaly Detection, AAAI 2022) showed a random score can beat state-of-the-art under PA. SensorSentinel therefore:

• selects the operating threshold only from attack-free validation data (no scanning test labels for the best F1);
• reports PR-AUC and an event-aware composite F1 (point-wise precision × event-wise recall, Garg et al. 2021) as the headline numbers;
• includes PA F1 only in a column named PA_F1_ref_only, labelled as inflated.

Quickstart

python -m venv .venv && source .venv/bin/activate     # Windows: .venv\Scripts\activate
pip install -r requirements.txt

python run.py                 # synthetic CPS data, full pipeline (~1-3 min on CPU)
python run.py --epochs 5      # quick smoke run
pytest -q                     # smoke tests (stealth invariant, metric sanity)

Artifacts land in results/:

file	contents
metrics.csv / report.md	detector comparison table + summary
score_timeline.png	AE anomaly score with true attack windows shaded
pr_curves.png	precision-recall comparison of all detectors
stealthiness.png	per-event detection vs. perturbation magnitude
attack_events.json	ground-truth injected attacks

Using real SWaT / WADI data

Request access from iTrust, SUTD, drop the CSVs in data/, point config.yaml at them, and run:

python run.py --source swat

The loader uses the dataset's native Normal/Attack labels; the rest of the pipeline is unchanged. (Licensed data is gitignored and never committed.)

Project layout

sensorsentinel/
  simulator.py   # SWaT-style physically-coupled CPS process
  attacks.py     # stealthy in-range attack library (bias/drift/replay/scaling)
  datasets.py    # synthetic + SWaT/WADI assembly
  model.py       # LSTM reconstruction autoencoder (PyTorch)
  baseline.py    # Isolation Forest + z-score baselines
  evaluate.py    # leak-free, event-aware metrics (PA pitfall handled)
  plots.py       # timeline, PR curves, stealthiness figures
run.py           # end-to-end orchestrator
tests/           # smoke tests

Design choices worth noting

• Physical coupling is the signal. Attacks are detectable not because values are out of range (they aren't) but because they break the learned joint sensor dynamics.
• No test-label leakage. Threshold from validation quantile only.
• Baselines on equal footing. All detectors expose a continuous score and share the same threshold-free metrics.

References

• Goh et al., A Dataset to Support Research in the Design of Secure Water Treatment Systems (SWaT), CRITIS 2016.
• Xu et al., Unsupervised Anomaly Detection via VAE for Seasonal KPIs, WWW 2018 (PA origin).
• Kim et al., Towards a Rigorous Evaluation of Time-Series Anomaly Detection, AAAI 2022.
• Garg et al., An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series, IEEE TNNLS 2021 (composite F-score).