predict returns the distribution’s mean, which almost
never lands exactly on one of your levels (you ask for a rating and get
3.87).
Pass discretize= and predictions are mapped onto the target’s level
lattice instead — every returned value is one your target can actually
take:
discretize= (a strategy) or categorical_levels= (a known level set).
There is no separate flag — either argument activates the feature, and
categorical_levels alone uses the default strategy ("map-cell").
Choosing a strategy
Nori emits a full predictive distribution per row, and discretization asks: which single lattice point best summarizes it? The right answer depends on the metric you are scored on — the mode wins on accuracy, the median on MAE, the mean on quadratic penalties. Pick accordingly:discretize | What it is | Use it for |
|---|---|---|
"map-cell" (default) | mode of the discrete posterior: integrate the distribution’s mass in a cell around each level, take the argmax | accuracy, macro-F1 |
"median-cell" | median of that discrete posterior (first level where cumulative cell mass crosses 0.5) | MAE / ordinal closeness |
"snap-mean" | nearest level to the point mean | quadratically-penalized agreement (QWK) |
"snap-median" | nearest level to the distribution median | MAE, with custom bar_distribution checkpoints |
"expected-level" | Σ level·P(level) — a lattice-informed continuous expectation, deliberately off-lattice | analysis / sanity checks (≈ the plain mean) |
"prior-match" | rank rows by point prediction, assign labels so predicted frequencies match training priors | calibration experiments only — benchmarked significantly worse; prefer map-cell / median-cell |
map-cell is the accuracy leader (0.755 vs 0.707 for
snapping the mean), median-cell the MAE leader, and snap-mean the QWK
leader. The gains are largest on skewed targets, where the mean falls
between levels or on a low-probability one — up to +48pp accuracy over
mean-snapping on the most skewed dataset.
Declaring the level set: categorical_levels
categorical_levels is the set of values the target can take — the label
set, in classification terms. By default it is inferred from the distinct
values of the y you fit on, which is leak-safe. Pass it explicitly when
your context is small and may under-cover the true scale — for example a
1–5 rating whose context happens to contain no 1s:
Works with the sklearn ecosystem
discretize and categorical_levels are also estimator parameters, so
clone, cross_val_score, and GridSearchCV reach them; predict kwargs
override the constructor per call:
infer(X_train, y_train, X_test, discretize="map-cell").
Good to know
- Failed predictions stay visible. A
NaNpoint prediction staysNaNafter discretization rather than becoming a confident label. - Checkpoints. The default Hugging Face checkpoints expose the full
quantile bank, so every strategy works out of the box. Only custom
bar_distributioncheckpoints are limited to thesnap-*strategies. - Programmatic strategy list. The strategy names and one-line
descriptions are importable —
synthefy_nori.discretize.DISCRETIZE_METHODS/DISCRETIZE_METHOD_DESCRIPTIONS— for building CLIs or UIs on top. The underlying lattice math (cell_masses,discretize_predictions,snap_to_levels) is public too.
Upgrading from an older release? Earlier versions silently snapped point
predictions whenever the target had ≤ 30 distinct values. That implicit
behavior is now off by default — snapping only happens when you ask for
it. Low-cardinality pipelines may see small numeric shifts (typically R²
improves). To restore the legacy behavior, pass
NoriRegressor(discrete_y_snap_max_unique=30).