FacePhys OBS Plugin

Contactless real-time heart-rate estimation (rPPG) rendered directly onto video sources inside OBS Studio.

FacePhys uses a lightweight State Space Model to extract the Blood Volume Pulse (BVP) wave from subtle facial color variations and computes your pulse in Beats per Minute (BPM) completely on-device.

Health & Medical Disclaimer -- NOT A MEDICAL DEVICE

FacePhys is designed strictly for research, educational, and entertainment purposes.

FacePhys is NOT a medical device and is NOT intended for clinical diagnosis, disease prevention, medical monitoring, therapy, or treatment of any physiological or cardiovascular condition.

Remote photoplethysmography (rPPG) estimates blood volume pulse optically and is subject to errors and fluctuations caused by ambient lighting variations, camera sensor noise, compression artifacts, facial motion, occlusion, and skin tone variations. Never rely on readings produced by this plugin for medical decisions, diagnosis, or health assessments. Always consult a certified medical practitioner for any health concerns.

Research & Upstream Attribution

This OBS plugin is built on top of the research and open-source models developed by Kegang Wang and collaborators.

• Paper: FacePhys: State of the Heart Learning (arXiv:2512.06275)
• Authors: Kegang Wang, Jiankai Tang, Yuntao Wang, Xin Liu, Yuxuan Fan, Jiatong Ji, Yuanchun Shi, Daniel McDuff
• Official Repositories:
  • FacePhys-Release: Model weights, recurrent state definitions, and ONNX / LiteRT exports.
  • FacePhys-Demo: Interactive browser and WebAssembly demonstration.
  • open-rppg: Comprehensive high-performance Python rPPG toolkit.

How FacePhys Works

Remote photoplethysmography (rPPG) measures micro-vascular blood volume changes reflected in facial light absorption. Traditional convolutional or transformer networks either suffer from high memory/compute overhead or fail to generalize across long time horizons.

FacePhys introduces a temporal-spatial state space model (SSM) duality that models cardiac dynamics as continuous-time controlled states. With a compact model footprint of only ~3.6–4 MB and sub-10 ms per-frame inference, it enables high-accuracy, real-time pulse tracking directly on consumer hardware.

If you use FacePhys in academic or research work, please cite the original paper:

@article{wang2025facephys,
  title={FacePhys: State of the Heart Learning},
  author={Wang, Kegang and Tang, Jiankai and Wang, Yuntao and Liu, Xin and Fan, Yuxuan and Ji, Jiatong and Shi, Yuanchun and McDuff, Daniel},
  journal={arXiv preprint arXiv:2512.06275},
  year={2025}
}

Privacy Guarantee

In strict compliance with the upstream FacePhys Privacy Protection Addendum:

• 100% On-Device Processing: All video ingestion, face tracking, and biometric inference execute locally on your machine.
• Zero Cloud Transmission: No video frames, cropped facial data, or derived physiological signals are ever transmitted to external servers or cloud services.

Why an OBS Filter?

My main criteria when writing this plugin was to not have an extra program that hogs the camera. On Linux, Windows, and macOS, webcam and video capture devices are typically exclusive to a single process. Running a standalone Python or OpenCV script requires locking the webcam device, preventing OBS from capturing your camera at the same time.

By implementing FacePhys as a native OBS video filter:

• Zero Device Conflicts: OBS manages the capture device natively. FacePhys samples frames directly from OBS's internal compositor (gs_texrender and gs_stagesurface), leaving the camera free for OBS and other virtual sources.
• Works with Any Source: Attach the filter to webcams, capture cards, virtual cameras, video files, or browser sources.
• Non-Blocking GPU Readback: Frame capture uses double-buffered GPU staging surfaces so the OBS render thread never stalls waiting for CPU memory transfers.
• Asynchronous Inference: The 46 recurrent SSM state tensors run on a dedicated background worker thread via ONNX Runtime CPU execution, isolating model inference latency from video frame rates.

Features

• SSM rPPG Engine: Powered by FacePhys recurrent state space tensors with pre-warmed initial states for fast convergence.
• Automatic Face Tracking: An onboard BlazeFace ONNX detector scans incoming frames and dynamically tracks the face region (forehead and cheeks) with exponential smoothing, largest-face selection in multi-person shots, and configurable lost-face hold timeouts.
• Heart-Rate Estimator:
  • Linear signal detrending and symmetric Hann windowing.
  • 4096-point FFT spectral analysis.
  • Log-power parabolic peak interpolation for sub-bin frequency resolution.
  • Harmonic confidence scoring and signal quality estimation.
• Customizable HUD Overlay:
  • 9 Anchor Positions: Top Left, Top Centre, Top Right, Middle Left, Centre, Middle Right, Bottom Left, Bottom Centre, Bottom Right.
  • Full Layout Controls: Independent X and Y margin sliders, global HUD scale, opacity, and custom color picker.
  • Toggleable Visual Modules:
    • Digital 7-segment BPM readout.
    • Pulsing heartbeat dot.
    • Real-time scrolling BVP waveform graph.
    • Status labels (CALIBRATING, NO SIGNAL, BPM).
    • Measurement confidence bar.
  • Confidence Threshold Gating: Dims or suppresses readings automatically when confidence falls below a configured threshold (e.g. during rapid motion or occlusion).
  • Visual ROI Framing Reticle: Optional alignment guide to verify facial tracking and framing.

Building and Installation

Prerequisites

• CMake 3.22 or newer
• C++17 compatible compiler (GCC 9+, Clang 10+, or MSVC 2019+)
• libobs headers (provided by libobs-dev on Linux, OBS Studio SDK, or OBS Studio application installation)
• Internet access during initial CMake configuration (to download official ONNX Runtime 1.29.0 binaries) or an existing download provided via -DORT_ROOT=....

Build Commands

# 1. Configure the build
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release

# 2. Compile the plugin
cmake --build build -j

# 3. Install into your user OBS plugin directory
cmake --install build

The default installation path targets the per-user OBS plugin directory:

• Linux: ~/.config/obs-studio/plugins/facephys/
• Windows: %APPDATA%/obs-studio/plugins/facephys/
• macOS: ~/Library/Application Support/obs-studio/plugins/facephys/

To install into a custom location (e.g. system plugins or portable directory):

cmake --install build --prefix /path/to/obs-plugins

Usage in OBS Studio

1. Open OBS Studio.
2. Right-click your webcam, capture card, or camera source in the Sources dock and select Filters.
3. Under Effect Filters, click the + button and choose Heart Rate (FacePhys).
4. Customise the Overlay section (position anchor, colors, scale, waveform) according to your stream layout. (optional)

License

This project is released under the terms in LICENSE.

• FacePhys OBS Plugin: Copyright (c) 2026 voidlesity.
• FacePhys Model & State Weights: Copyright (c) 2025 Kegang Wang, released under the MIT License with Privacy Protection Addendum.
• BlazeFace: Copyright Google LLC, released under the Apache License, Version 2.0.
• ONNX Runtime: Copyright (c) Microsoft Corporation, released under the MIT License.
• OBS Studio (libobs): Copyright (c) OBS Project, released under the GNU General Public License v2.0 or later.