Differentiable Numerical Association Rule Learning

This repository contains the source code of the method and the experiments from the paper entitled "Differentiable Numerical Association Rule Learning".

Besides reproducing the experiments, you can run DiffNARM on your own dataset using the wrapper diffnarm_wrapper.py. See Running DiffNARM on your own data.

Table of Contents

1. Installation
2. Code Structure
3. Datasets
4. Experiments
5. Reproducibility
6. Running DiffNARM on your own data
7. References

Installation

Python dependencies are in requirements.txt. The reported experimental results are obtained with Python 3.12 version.

pip install -r requirements.txt

All commands listed in the rest of this repo are run from the repository root.

Code Structure

Structure of the repository is given below. Note that during the development, many alternative module have been tested, e.g., to overcome boundary collapse, or a learnable sharpness parameter instead of annealing. Some of the scripts, functions, or parameters in the repository have not been used in the reported experiments. They are kept in the repository for further experimentation.

src/
├── diffnarm/                     # the proposed method (DiffNARM)
│   ├── soft_binning.py           # differentiable (soft) binning layer with learnable boundaries
│   ├── model.py                  # autoencoder + soft binning model and model training
│   ├── rule_extraction.py        # rule extraction using frequency-guided probing
│   ├── rule_quality.py           # rule quality metrics (support, confidence, Zhang's metric, ...)
│   └── data_preparation.py       # one-hot encoding, categorical feature detection
├── utils/
│   ├── data_loader.py            # loads datasets from data/
│   ├── discretization.py         # classical discretization baselines (equal-width, equal-freq, k-means, ...)
│   └── seeds.py, seeds.json      # fixed random seeds for all runs
└── experiments/
    ├── rule_mining/
    │   ├── run_rule_quality.py   # main experiment: rule quality comparison of all methods
    │   ├── tune_diffnarm.py      # Optuna hyperparameter tuning for DiffNARM and Aerial+
    │   ├── tune_niaarm.py        # Optuna hyperparameter tuning for optimization-based NARM baselines
    │   ├── statistical_significance_analysis.py  # Friedman + Nemenyi tests vs. baselines
    │   ├── *_best_params*.json   # tuned hyperparameters per method, dataset, and bin count
    │   └── baselines/            # Aerial+, FP-Growth, Single (niaarm) and multi-objective NARM (in the repo referred as swarm-narm)
    ├── rule_generalizability/
    │   └── run_rule_generalizability.py  # rule generalizability experiment via k-fold cross validation
    ├── ablation_regression.py    # ablation on the regression objective
    ├── masking_vs_denoising.py   # masking vs. denoising training comparison
    ├── scalability_num_of_features.py, scalability_niaarm.py  # scalability experiments
    └── antecedent_support_reliability.py, beta_schedule_analysis.py, collapse_threshold_analysis.py,
        hyperparam_rule_gates.py, n_bins_analysis.py, max_antecedents_analysis.py  # additional analyses (see Experiments)
data/                             # datasets as CSV files

Single-objective optimization-based NARM methods are implemented using NiaARM [2] and NiaPy [4], Multi-objective NARM methods are implemented based on Kaushik et al.'s performance analysis [8], FP-Growth [6] is implemented with Mlxtend [5] Aerial+ [7] is implemented using PyAerial [1].

Datasets

All datasets are from the UCI Machine Learning Repository [3], and CSV versions are included in data/.

Dataset	Link
Adult	https://archive.ics.uci.edu/dataset/2/adult
Breast Cancer Wisconsin (Diagnostic)	https://archive.ics.uci.edu/dataset/17/breast+cancer+wisconsin+diagnostic
Chronic Kidney Disease	https://archive.ics.uci.edu/dataset/336/chronic+kidney+disease
German Credit (Statlog)	https://archive.ics.uci.edu/dataset/144/statlog+german+credit+data
Heart Disease	https://archive.ics.uci.edu/dataset/45/heart+disease
Ionosphere	https://archive.ics.uci.edu/dataset/52/ionosphere
MAGIC Gamma Telescope	https://archive.ics.uci.edu/dataset/159/magic+gamma+telescope
Room Occupancy Estimation	https://archive.ics.uci.edu/dataset/864/room+occupancy+estimation
Yeast	https://archive.ics.uci.edu/dataset/110/yeast

Experiments

The following is a list of experiments from the paper and how to run them. The results are written as .xlsx files to out/ folder.

Experimental results per dataset is given in Experimental results per dataset.xlsx file.

Experiment	Command
Rule quality (main)	python -m src.experiments.rule_mining.run_rule_quality
Rule generalizability	python -m src.experiments.rule_generalizability.run_rule_generalizability
Ablation (regression objective)	python -m src.experiments.ablation_regression
Masking vs. denoising	python -m src.experiments.masking_vs_denoising
Scalability (number of features)	python -m src.experiments.scalability_num_of_features
Hyperparameter tuning (Optuna)	python -m src.experiments.rule_mining.tune_diffnarm, ... .tune_niaarm
Antecedent support reliability	python -m src.experiments.antecedent_support_reliability
Beta schedule analysis	python -m src.experiments.beta_schedule_analysis
Collapse threshold analysis	python -m src.experiments.collapse_threshold_analysis
Hyperparameter rule gates	python -m src.experiments.hyperparam_rule_gates
Number of bins analysis	python -m src.experiments.n_bins_analysis
Statistical significance	python -m src.experiments.rule_mining.statistical_significance_analysis

Selecting methods: in src/experiments/rule_mining/run_rule_quality.py, all methods and their parameters are defined in the METHODS dictionary (diffnarm is the proposed DiffNARM; aerial_<discretizer>, fpgrowth_<discretizer>, and niaarm_<algorithm> are baselines). Edit the selected_methods list in the __main__ block to choose which methods to run.

Selecting datasets: load_datasets() (in src/utils/data_loader.py) loads all datasets by default; pass a list of names, e.g. load_datasets(["heart_disease", "yeast"]), in an experiment's __main__ block to restrict it.

Tuned hyperparameters per dataset are stored in the *_best_params*.json files and applied automatically by run_rule_quality.py.

Antecedent support reliability compares how well DiffNARM's pairwise-geomean antecedent frequency estimate and Aerial+'s min-based estimate track the actual observed antecedent support, as reliability diagrams with Expected Calibration Error (ECE). Beta schedule analysis sweeps the soft-binning sharpness (beta) schedule (geometric, constant, linear, learnable) and reports rule quality per variant. Collapse threshold analysis sweeps the feature-collapse detection threshold and reports its effect on rule quality. Hyperparameter rule gates sweeps the rule_frequency and rule_strength extraction thresholds and reports their effect on rule quality. Number of bins analysis sweeps the number of bins used for discretization and reports its effect on rule quality. Statistical significance runs Friedman + Nemenyi tests comparing DiffNARM against each baseline family on the reported rule quality results.

Reproducibility

All experiments use the fixed random seeds in src/utils/seeds.json and we reported averages over 10 runs per method–dataset pair (n_runs in the method configuration).

Running DiffNARM on your own data

src/diffnarm_wrapper.py is a single-file wrapper that trains DiffNARM on any given tabular dataset and stores the extracted association rules. The input is a plain CSV or pandas DataFrame with one row per transaction; numerical and categorical columns are detected automatically.

To run DiffNARM on your own data, execute the following from the command line:

python diffnarm_wrapper.py path/to/your_data.csv
python diffnarm_wrapper.py path/to/your_data.csv --n-bins 3 --epochs 10 --output rules.json

Within your Python ML pipeline:

import pandas as pd
from diffnarm_wrapper import mine_rules

result = mine_rules(pd.read_csv("your_data.csv"))  # or e.g. mine_rules(df, n_bins=3)
result["rules"]  # list of rules with quality metrics
result["statistics"]  # rule count, data coverage, average support/confidence/Zhang's metric

All hyperparameters (bin count, initial discretizer, training weights, rule extraction thresholds, ...) are collected in DEFAULT_PARAMS at the top of diffnarm_wrapper.py, each with a short comment explaining what it does. Any of them can be overridden per call, e.g. mine_rules(df, max_antecedents=3, device="cpu").

References

1. Karabulut, Erkan, Paul Groth, and Victoria Degeler. "PyAerial: Scalable association rule mining from tabular data." SoftwareX 31 (2025): 102341.
2. Stupan, Žiga, and Iztok Fister. "Niaarm: a minimalistic framework for numerical association rule mining." Journal of Open Source Software 7.77 (2022): 4448.
3. Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. The uci machine learning repository. 2023. URL https://archive.ics.uci.edu.
4. Vrbančič, Grega, et al. "NiaPy: Python microframework for building nature-inspired algorithms." Journal of Open Source Software 3.23 (2018): 613.
5. Raschka, Sebastian. "MLxtend: Providing machine learning and data science utilities and extensions to Python’s scientific computing stack." Journal of open source software 3.24 (2018): 638.
6. Han, Jiawei, Jian Pei, and Yiwen Yin. "Mining frequent patterns without candidate generation." ACM sigmod record 29.2 (2000): 1-12.
7. Karabulut, Erkan, Paul Groth, and Victoria Degeler. Neurosymbolic association rule mining from tabular data. In Proceedings of The 19th International Conference on Neurosymbolic Learning and Reasoning, volume 284 of Proceedings of Machine Learning Research, pp. 565–588. PMLR, 08–10 Sep 2025a. URL https://proceedings.mlr.press/v284/karabulut25a.html.
8. Kaushik, Minakshi, et al. "An Exhaustive Multi-Aspect Analysis of Swarm Intelligence Algorithms in Numerical Association Rule Mining." IEEE Access 12 (2024): 138985-139002.