An end-to-end retail credit-risk pipeline that takes a loan book from raw data all the way to a regulatory impairment number:
- Feature engineering — Weight-of-Evidence (WoE) binning with monotonic constraints + Information Value (IV).
- PD model — a transparent logistic-regression scorecard (points-to-double-odds scaling) as champion, with an XGBoost challenger (TreeSHAP explainability).
- Validation — KS / Gini / AUROC with bootstrap CIs, PD calibration (reliability + Hosmer–Lemeshow), a gains/KS masterscale, PSI stability, and out-of-time (vintage) testing — auto-compiled into a validation report.
- IFRS 9 ECL — a three-stage Expected Credit Loss engine (staging → lifetime PD term structure →
ECL = PD × LGD × EAD). - Delivery — an interactive Streamlit dashboard with a live scenario cockpit, and CI running the test suite on every push.
The project runs out-of-the-box on a documented synthetic portfolio and
automatically upgrades to the real Lending Club dataset when you drop the
Kaggle file into data/ (see Using real data).
| Area | What's in here |
|---|---|
| Credit scoring | WoE/IV feature engineering with monotonic binning, logistic scorecard, PDO points scaling, rating masterscale |
| Machine learning | XGBoost with native categorical handling; champion/challenger benchmarking |
| Model validation | KS/Gini/AUROC with bootstrap CIs, PD calibration (reliability + Hosmer–Lemeshow), gains/KS masterscale, PSI, out-of-time validation, auto-generated report |
| Explainability | XGBoost native TreeSHAP feature attribution; leakage-aware feature selection |
| Regulatory / IFRS 9 | SICR staging (relative + absolute + DPD backstops), lifetime PD term structure, discounted ECL |
| Software engineering | typed src/ package, config-driven, 37 unit tests + CI, persisted artifacts, no hard-coded paths |
| Data viz / delivery | Streamlit multi-tab dashboard with live scenario stress testing (Plotly) |
data (synthetic DGP ┌──────────┐ ┌───────────────┐ champion ┌──────────────┐
or Lending Club) ──▶│ WoE / IV │──▶│ Scorecard │──────────▶ │ │
│ encoder │ │ (Logistic) │ │ KS · Gini │
└──────────┘ └───────────────┘ │ AUROC · PSI │
│ ┌───────────────┐ challenger │ │
└────────▶│ XGBoost │──────────▶ └──────┬───────┘
└───────────────┘ │
PD per loan ▼
│ ┌─────────────────────────┐
└────────────▶ │ IFRS 9 ECL engine │
│ staging → lifetime PD │
│ → PD·LGD·EAD (disc.) │
└────────────┬────────────┘
▼
Streamlit dashboard + reports
Discrimination (held-out test set):
| Model | AUROC | Gini | KS |
|---|---|---|---|
| Scorecard (Logistic) | 0.707 | 0.415 | 0.304 |
| XGBoost (challenger) | 0.716 | 0.433 | 0.314 |
On real loans XGBoost edges out the linear scorecard — the classic performance-vs-interpretability trade-off, quantified. Discrimination (AUROC ≈ 0.71) is in the expected range for a Lending Club origination scorecard; PSI(train, test) = 0.0006 indicates a stable model (< 0.10).
On not chasing a fake AUROC. The model uses only origination-time predictors (FICO, sub-grade, DTI, utilisation, etc.). Lending Club's post-origination fields (payments, recoveries, outstanding principal) would push AUROC past 0.95 — but that is target leakage and worthless in production, so they are deliberately excluded. ~0.71 is the honest ceiling for this feature set; FICO adds little on top of the risk-priced interest rate (multicollinearity).
Validation highlights (full report: reports/validation_report.md):
- Calibration: Hosmer–Lemeshow χ² = 8.9 (p = 0.35) — PDs are well calibrated, so they feed IFRS 9 ECL directly. Predicted vs observed default rate tracks tightly across all deciles.
- Uncertainty: Gini = 0.415, 95% bootstrap CI [0.403, 0.425].
- Out-of-time: trained on 2007–2017 vintages, tested on 2018 → AUROC 0.704, score PSI 0.030 (stable) — the model holds up on unseen future loans, not just a random split.
IFRS 9 ECL by stage:
| Stage | Loans | EAD | ECL | Coverage |
|---|---|---|---|---|
| Stage 1 (12-month ECL) | 49,972 | $612.0M | $19.9M | 3.3% |
| Stage 2 (lifetime ECL) | 69,962 | $1,081.0M | $192.2M | 17.8% |
| Stage 3 (credit-impaired) | 30,065 | $468.1M | $182.7M | 39.0% |
| Total | 149,999 | $2,161.2M | $394.8M | 18.3% |
Runs out-of-the-box on a documented synthetic portfolio (AUROC ≈ 0.85 by construction) and switches to Lending Club automatically when the Kaggle file is present. The figures above are the real-data run.
pip install -r requirements.txt # or: pip install -e .
python scripts/generate_data.py # synthetic portfolio -> data/loan_portfolio.csv
python scripts/train.py # WoE/IV, scorecard, XGBoost, metrics, ECL, figures
streamlit run app/dashboard.py # interactive dashboardscripts/train.py prints the discrimination table and the ECL summary, and
saves models to models/ and metrics / scorecard / ECL tables to reports/.
Download Kaggle's wordsforthewise/lending-club
accepted-loans file into data/ (details and CLI command in
data/README.md), then re-run scripts/train.py. The
pipeline detects the file automatically and switches data source — no code
changes. The adapter (lendingclub.py) parses the raw fields, derives the
default target from loan_status, and maps funded_amnt → EAD, grade →
origination-PD proxy, and loan_status → days-past-due for staging.
Five tabs: Overview (portfolio KPIs, score & PD distributions) · Feature Analysis (IV ranking, per-feature WoE vs default-rate) · Model Performance (metric table, ROC & KS curves) · Scorecard (points table + a live "score-a-borrower" form) · IFRS 9 ECL (scenario sliders for LGD, macro-PD stress and the SICR threshold that re-stage the book and recompute ECL in real time).
credit_risk_model/
├── src/credit_risk/
│ ├── config.py # paths, feature specs, scorecard & IFRS 9 parameters
│ ├── data.py # synthetic portfolio generator (documented DGP)
│ ├── lendingclub.py # Lending Club → pipeline-schema adapter
│ ├── datasets.py # resolves real-vs-synthetic data source
│ ├── woe.py # WoE binning (monotonic) + Information Value
│ ├── metrics.py # KS, Gini, AUROC, PSI + plots
│ ├── scorecard.py # logistic scorecard + PDO points scaling
│ ├── gbm.py # XGBoost challenger
│ ├── explain.py # XGBoost TreeSHAP feature attribution
│ ├── validation.py # calibration, gains/KS, bootstrap CIs, out-of-time
│ ├── report.py # auto-generated Markdown validation report
│ ├── ecl.py # IFRS 9 three-stage ECL engine
│ └── pipeline.py # orchestration + artifact persistence
├── scripts/ # generate_data.py, train.py
├── app/dashboard.py # Streamlit dashboard (6 tabs incl. Validation)
├── .github/workflows/ # CI: run the test suite on every push
├── tests/ # 37 unit tests (WoE, metrics, scorecard, ECL, validation, adapter)
├── data/ # portfolio CSV / Lending Club file (git-ignored)
├── models/ · reports/ # generated artifacts (git-ignored)
├── requirements.txt · pyproject.toml
- WoE / IV.
WoE = ln(P(good|bin) / P(bad|bin)); a 0.5 count adjustment guards sparse bins. IV is interpreted with the standard Siddiqi bands. - Scorecard scaling. Points-to-double-odds convention:
factor = PDO/ln 2,offset = base_score − factor·ln(base_odds)(defaults 600 / 50:1 / PDO 20). Per-attribute points make each borrower's score a readable sum. - IFRS 9 staging. Stage 3 at 90+ DPD (impaired); Stage 2 on a significant increase in credit risk — a relative PD jump vs origination (with a low-credit-risk floor), a high absolute PD, or the 30-DPD backstop; Stage 1 otherwise.
- Lifetime ECL. A constant-hazard PD term structure
(
marginal_PDₜ = (1−p)^{t−1}·p) over remaining life, discounted at the effective interest rate; Stage 3 sets PD = 1 (ECL = LGD × EAD).
python -m unittest discover -s tests -t .37 tests cover WoE/IV and monotonic binning, the discrimination metrics (including degenerate and perfect-separation cases), scorecard monotonicity and scaling, the validation suite (gains, calibration, Hosmer–Lemeshow, bootstrap CIs, out-of-time), the IFRS 9 staging rules and ECL identities, and the Lending Club field parsers. The same suite runs in CI on Python 3.11 and 3.12.
Deliberately scoped as a portfolio project; the natural next steps mirror how a bank would harden it:
- SICR threshold calibration. SICR thresholds are illustrative. On a real book they would be calibrated per-segment (the Lending Club run lands ~47% in Stage 2 because the absolute-PD backstop is conservative for a 20%-default book) and validated against realised rating migration.
- Forward-looking macro overlays. IFRS 9 expects probability-weighted
scenarios; the single
macro_pd_multiplierslider is a placeholder for a base/upside/downside PD-macro model. - LGD / EAD models. LGD and EAD use rule-based assumptions here; both could be modelled (e.g. beta-regression LGD, CCF-based EAD).
- Reject inference. For a full origination scorecard, extend beyond accepted loans to correct selection bias.
Already implemented: monotonic WoE binning · bootstrap CIs · PD calibration (reliability + Hosmer–Lemeshow) · gains/KS masterscale · out-of-time (vintage) validation with score PSI · TreeSHAP explainability · auto-generated validation report · CI.
MIT © Chenxi Zhao
See LICENSE.