Explainability

Understand why Nori made a prediction: which features drove it, how they interact, and how the model responds across a feature’s range. Because NoriRegressor is a scikit-learn estimator, it plugs directly into shapiq — a fast SHAP implementation with native Shapley-interaction support — and the rest of the sklearn interpretability ecosystem. The helpers in synthefy_nori.interpretability are thin convenience wrappers, so you can also use the underlying tools directly. examples/interpretability_regression.py

These examples run locally with the synthefy-nori package and its interpretability extra, which pulls in shapiq. Follow Installation before running them.

Nori is regression-only, so every method below explains the regression predictive mean.

Quickstart

get_nori_imputation_explainer builds a shapiq.TabularExplainer that removes features by imputation against a background set. The training context stays fixed across coalitions, so each coalition is a single predict call and the cost is set by budget.

import matplotlib.pyplot as plt

from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
from synthefy_nori import NoriRegressor
from synthefy_nori.interpretability.shapiq import get_nori_imputation_explainer

dataset = load_diabetes()
X_train, X_test, y_train, y_test = train_test_split(
    dataset.data, dataset.target, test_size=0.2, random_state=0
)
feature_names = list(dataset.feature_names)
model = NoriRegressor(model="nori-30m").fit(X_train, y_train)

# Plain Shapley values sum from the baseline to the prediction.
explainer = get_nori_imputation_explainer(model, X_train, index="SV", max_order=1)
sv = explainer.explain(X_test[:1], budget=1024)
print(sv)
ax = sv.plot_waterfall(
    feature_names=feature_names, abbreviate=False, show=False
)
# Leave room for negative-value labels beside the feature names.
left, right = ax.get_xlim()
ax.set_xlim(left - 0.2 * (right - left), right)
ax.set_title("SHAP waterfall: one Nori prediction", fontsize=15, pad=42)
ax.figure.savefig("nori-shap-waterfall.png", dpi=180, bbox_inches="tight")
plt.show()

SHAP waterfall plot for a single Nori prediction — SHAP waterfall — each feature's additive push from the baseline to one prediction (sklearn diabetes).

Feature interactions

Use a separate explainer with index="k-SII", max_order=2 for pairwise interactions, which you can view as a network:

Feature interaction code

from matplotlib.lines import Line2D

interaction_explainer = get_nori_imputation_explainer(
    model, X_train, index="k-SII", max_order=2
)
kv = interaction_explainer.explain(X_test[:1], budget=1024)
fig, ax = kv.plot_network(feature_names=feature_names, show=False)
# Native edge patches have no outline; their alpha scales with |interaction|.
# Scale against the strongest pairwise effect, separately from node effects.
edge_patches = [patch for patch in ax.patches if patch.get_linewidth() == 0]
edge_strengths = [patch.get_alpha() for patch in edge_patches]
max_strength = max(edge_strengths, default=0) or 1
for patch, strength in zip(edge_patches, edge_strengths):
    patch.set_linewidth(0.5 + 4.5 * strength / max_strength)
# Keep signed colors opaque and even the weakest connections visible.
for patch in ax.patches:
    patch.set_alpha(1.0)
    patch.set_edgecolor(patch.get_facecolor())
ax.set_title("Feature interaction network", pad=16)
ax.legend(
    handles=[
        Line2D([], [], color="#ff0d57", marker="o", linestyle="none",
               markersize=9, label="Node: positive main effect"),
        Line2D([], [], color="#ff0d57", linewidth=2.5,
               label="Line: positive pairwise interaction"),
        Line2D([], [], color="#555555", linewidth=0.5,
               label="Line width: weaker interaction"),
        Line2D([], [], color="#1e88e5", marker="o", linestyle="none",
               markersize=9, label="Node: negative main effect"),
        Line2D([], [], color="#1e88e5", linewidth=2.5,
               label="Line: negative pairwise interaction"),
        Line2D([], [], color="#555555", linewidth=5,
               label="Line width: stronger interaction"),
    ],
    title=("Positive adds to this prediction; negative subtracts\n"
           "Larger nodes indicate stronger main effects"),
    loc="upper center", bbox_to_anchor=(0.5, -0.08),
    ncol=2, frameon=False, fontsize=10, title_fontsize=11,
)
fig.savefig("nori-shap-interactions.png", dpi=180, bbox_inches="tight")
plt.show()

Feature interaction network with a legend: red/pink nodes and lines contribute positively, blue nodes and lines negatively; larger nodes represent stronger main effects and thicker lines represent stronger pairwise interactions — Pairwise interactions for one Nori prediction.

Partial dependence / ICE

from matplotlib.ticker import MaxNLocator
from synthefy_nori.interpretability.pdp import partial_dependence_plots

# Diabetes columns 2 and 3 are bmi and bp; use kind="both" for PDP + ICE.
fig, axes = plt.subplots(2, 1, figsize=(8, 7), layout="constrained")
partial_dependence_plots(
    model, X_test, features=[2, 3], kind="average",
    feature_names=feature_names, ax=axes,
)
fig.suptitle("Partial dependence: diabetes progression\none year after baseline", fontsize=15)
for ax, label in zip(axes, ["Body mass index (BMI)", "Average blood pressure"]):
    ax.set_title(label, fontsize=12)
    ax.set_xlabel(f"{label} (centered and scaled)")
    ax.set_ylabel("Average predicted diabetes\nprogression score", fontsize=11)
    ax.xaxis.set_major_locator(MaxNLocator(5))
    ax.yaxis.set_major_locator(MaxNLocator(5))
fig.savefig("nori-pdp.png", dpi=180, bbox_inches="tight")
plt.show()

Pass (i, j) tuples in features for 2-D interaction surfaces. Returns an sklearn PartialDependenceDisplay.

Two partial-dependence plots showing average predicted diabetes progression after one year against scaled BMI and blood-pressure values — Top: vary BMI. Bottom: vary average blood pressure. Both y-axes show the average predicted diabetes progression score one year after baseline. Other features retain their observed test-row values before predictions are averaged.

Feature selection

Find a minimal feature subset that preserves cross-validated performance (sequential selection):

from synthefy_nori.interpretability.feature_selection import feature_selection

res = feature_selection(model, X_train, y_train, n_features_to_select=5, cv=3)
print(res.selected_indices, res.selected_score_mean)

n_features_to_select accepts an int, a fraction, or "auto" (with tol). The result also reports baseline-vs-selected CV scores.

Sequential selection re-fits Nori in-context on every CV split, so keep it to a few thousand samples and a modest feature count.

Configuration

get_nori_imputation_explainer parameters

Parameter	Default	Description
`index`	`"k-SII"`	`"SV"` for plain Shapley values, or `"k-SII"` for k-Shapley interactions.
`max_order`	`2`	Maximum interaction order. `1` = single-feature attributions, `2` = pairwise.
`imputer`	`"baseline"`	`"baseline"` runs one forward per coalition (recommended); `"marginal"` / `"conditional"` are multi-sample and much slower.

Budget trades accuracy for cost. Start at 128 and raise only if the explanation looks noisy: <10 features → 64–128; 10–20 → 128–512; 20+ → 512–2048.

Good to know

These run on the local Python package (synthefy-nori), not the hosted API.
TabPFN’s fast Shapley path reuses a KV-cache across coalitions; Nori’s public package runs one forward per coalition, which is correct and budget-controlled, just not cache-accelerated.

Next steps

Glass-box Models

Distill Nori into a readable, shippable additive model (EBM) in one call.

Categorical & Ordinal Targets

Predict labels on a discrete scale instead of a continuous estimate.

Embeddings

Target-aware row vectors for probing, clustering, and search.

Get Started

Guides

Integration

Quickstart

Feature interactions

Partial dependence / ICE

Feature selection

Configuration

Good to know

Next steps

Glass-box Models

Categorical & Ordinal Targets

Embeddings

​Quickstart

​Feature interactions

​Partial dependence / ICE

​Feature selection

​Configuration

​Good to know

​Next steps

Glass-box Models

Categorical & Ordinal Targets

Embeddings

Quickstart

Feature interactions

Partial dependence / ICE

Feature selection

Configuration

Good to know

Next steps