Missing Values & Imputation

​

Quickstart

import numpy as np
from synthefy_nori import NoriRegressor

rng = np.random.default_rng(0)
X_train = rng.normal(size=(60, 4))
y_train = X_train @ np.array([1.0, -2.0, 0.5, 0.0]) + rng.normal(scale=0.1, size=60)

X_train[::7, 1] = np.nan          # missing values in the context rows...
X_test = rng.normal(size=(5, 4))
X_test[0, 2] = np.nan             # ...and in the query rows

model = NoriRegressor(model="nori-30m").fit(X_train, y_train)
print(model.predict(X_test))      # finite predictions — NaNs handled internally

​

What happens to a NaN

• Numeric features — missing values are imputed inside the model’s own preprocessing (server-side when you call the hosted API). Statistics come from your context rows, never from the query rows.
• Categorical features (DataFrame inputs via the synthefy client) — with the default ordinal encoding, a missing value stays NaN and is imputed server-side alongside the numeric columns; with categorical_encoding="onehot", missing values get their own indicator column instead.
• Columns with no observed values — a feature that is entirely missing carries no signal; it is neutralized rather than invented.

​

Imputing yourself

import pandas as pd

# df_train / df_test are DataFrames of features
train_medians = df_train.median()                      # per-column, NaN-skipping

X_train = df_train.fillna(train_medians).fillna(0.0)
X_test  = df_test.fillna(train_medians).fillna(0.0)    # TRAIN medians on test too

• Median, not zero. On a non-centered feature, zero-filling plants an often-wild outlier in every incomplete row. The median stays inside the feature’s real range and is robust to skew and outliers, which tabular data has in abundance.
• Train statistics only. The test rows are filled with medians computed from the training rows. Computing fill values from the test set leaks information about the data you are predicting on and inflates evaluation scores.

​

Good to know

• Benchmarking. When you compare Nori against other models, apply the same imputation to every model, fit on the training split only. Mixed preprocessing is the most common source of unreproducible tabular comparisons. Synthefy’s own evaluation loaders follow the recipe above (train-median imputation with zero fallback), so published numbers match how the model is preprocessed in production.
• Meaningful gaps. If a missing value carries signal in your domain, keep the is_missing indicator from the tip above rather than relying on the fill value to encode it.

​

Next steps