Missing Values & Imputation

Real tabular data has holes. Nori accepts them: you can pass NaN in your feature columns — locally or through the hosted API — and get finite predictions back. No imputation step is required before calling the model.

import numpy as np
from synthefy_nori import NoriRegressor

rng = np.random.default_rng(0)
X_train = rng.normal(size=(60, 4))
y_train = X_train @ np.array([1.0, -2.0, 0.5, 0.0]) + rng.normal(scale=0.1, size=60)

X_train[::7, 1] = np.nan          # missing values in the context rows...
X_test = rng.normal(size=(5, 4))
X_test[0, 2] = np.nan             # ...and in the query rows

model = NoriRegressor().fit(X_train, y_train)
print(model.predict(X_test))      # finite predictions — NaNs handled internally

Missing targets are different: y_train must be numeric and complete. Drop context rows whose target is missing — a filled-in target is a made-up label, and it will steer every prediction.

What happens to a NaN

You don’t have to do anything, but it helps to know the defaults:

Numeric features — missing values are imputed inside the model’s own preprocessing (server-side when you call the hosted API). Statistics come from your context rows, never from the query rows.
Categorical features (DataFrame inputs via the synthefy client) — with the default ordinal encoding, a missing value stays NaN and is imputed server-side alongside the numeric columns; with categorical_encoding="onehot", missing values get their own indicator column instead.
Columns with no observed values — a feature that is entirely missing carries no signal; it is neutralized rather than invented.

Imputing yourself

Sometimes you want the fill values under your control — to keep them consistent across models in a comparison, or to encode domain knowledge (for instance, a missing sensor reading that really means zero). Any imputation works, but the recipe we use in production and in our own benchmark harness is per-column train median, then 0 for columns with no observed values:

import pandas as pd

train_medians = X_train.median()                       # per-column, NaN-skipping

X_train = X_train.fillna(train_medians).fillna(0.0)
X_test  = X_test.fillna(train_medians).fillna(0.0)     # TRAIN medians on test too

Two properties make this the default worth copying:

Median, not zero. On a non-centered feature, zero-filling plants an often-wild outlier in every incomplete row. The median stays inside the feature’s real range and is robust to skew and outliers, which tabular data has in abundance.
Train statistics only. The test rows are filled with medians computed from the training rows. Computing fill values from the test set leaks information about the data you are predicting on and inflates evaluation scores.

If a missing value is meaningful in your domain (a blank field on a form, a sensor that only reports on failure), consider adding a boolean is_missing indicator column before imputing — that keeps the signal while still giving the model a complete numeric matrix.

Evaluating models on data with missing values

When you benchmark Nori against other models, apply the same imputation to every model, fit on the training split only. Mixed preprocessing is the most common source of unreproducible tabular comparisons. Synthefy’s own evaluation loaders follow the recipe above — train-median imputation with zero fallback — so published numbers match how the model is preprocessed in production.

​What happens to a NaN

​Imputing yourself

​Evaluating models on data with missing values

What happens to a NaN

Imputing yourself

Evaluating models on data with missing values