Nori’s encoder turns every row into a dense vector — a learned representation
that captures how the row relates to the target, conditioned on the rest of your
table. You can pull those vectors out and use them for anything you’d use
features for: a downstream model (kNN, linear probe, gradient boosting),
clustering, nearest-neighbor search, visualization, or meta-learning.
Because NoriRegressor is a scikit-learn estimator, embedding extraction plugs
straight into the sklearn ecosystem. There are two entry points:
NoriEmbedding — an sklearn transformer. With n_fold >= 2 it produces
out-of-fold (leak-free) embeddings for the training rows. Recommended.NoriRegressor.get_embeddings — the low-level call, when you want direct
control over which rows are embedded against which context.
Nori is regression-only, so embeddings are conditioned on a continuous
target.
Install
pip install synthefy-nori
matplotlib (and openml for TabArena), which come with
the eval extra:
pip install "synthefy-nori[eval]"
The interface
NoriEmbedding is a scikit-learn transformer with two methods:
fit_transform(X_train, y_train) — returns embeddings for the training
rows. Out-of-fold when n_fold >= 2 (see below).transform(X) — returns embeddings for unseen rows, always from the final
model fit on the full training set.
transform never returns cached training embeddings, even if the input happens
to equal the training set. For training-row embeddings use fit_transform (or
read train_embeddings_).
(n_estimators, n_samples, embed_dim), where
n_estimators is the number of preprocessing pipelines in the inference
ensemble. Average over the ensemble axis (or pick a single member) for a 2-D
matrix a downstream estimator can consume:
Z_train = train_emb.mean(axis=0) # (n_samples, embed_dim)
# or: Z_train = train_emb[0]
Out-of-fold embeddings (recommended)
With n_fold >= 2, fit_transform runs K-fold cross-validation: each training
row is embedded by a fold model that did not have it in its context, so the
embedding never leaks that row’s own label. This matters whenever you fit a
downstream model on the training embeddings — leak-free embeddings give an
honest estimate of downstream performance
(arXiv:2502.17361). Unseen rows are then
embedded by a single model fit on the full training set.
from synthefy_nori import NoriEmbedding, NoriRegressor
embedder = NoriEmbedding(n_fold=5, shuffle=True, random_state=0)
train_emb = embedder.fit_transform(X_train, y_train) # out-of-fold (leak-free)
test_emb = embedder.transform(X_test) # final full-data model
Z_train = train_emb.mean(axis=0) # (n_train, embed_dim)
Z_test = test_emb.mean(axis=0) # (n_test, embed_dim)
KFold is used for the splits; pass shuffle=True with a random_state to
shuffle. To reuse a specific configuration (e.g. skip compilation for many small
fold-fits), hand NoriEmbedding a pre-built estimator:
embedder = NoriEmbedding(
n_fold=5, shuffle=True, random_state=0,
model=NoriRegressor(compile_model=False),
)
Vanilla embeddings
With n_fold=0, a single model is fit on the entire training set and used for
both train and unseen rows. It’s cheaper — one fit instead of n_fold + 1 — but
the training embeddings encode their own labels, so don’t fit a downstream model
on them and expect an unbiased score.
embedder = NoriEmbedding(n_fold=0)
train_emb = embedder.fit_transform(X_train, y_train) # not leak-free
The low-level call
NoriRegressor.get_embeddings(X, data_source=...) embeds rows against the
context stored by fit. data_source selects which rows come back:
"test" (default) — embed the query rows X."train" — embed the stored context rows. X is ignored here (the
representations depend only on what you passed to fit), so you can omit it.
from synthefy_nori import NoriRegressor
model = NoriRegressor().fit(X_train, y_train)
test_emb = model.get_embeddings(X_test, data_source="test") # query rows
train_emb = model.get_embeddings(data_source="train") # context rows
"train" rows encode
their own labels — for leak-free training embeddings use NoriEmbedding with
n_fold >= 2.
Using embeddings as features
Once you have a 2-D matrix, any sklearn estimator works. A nearest-neighbor
regressor on the embeddings:
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import r2_score
knn = KNeighborsRegressor(n_neighbors=10).fit(Z_train, y_train)
print(r2_score(y_test, knn.predict(Z_test)))
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
probe = make_pipeline(StandardScaler(), RidgeCV(alphas=(0.1, 1.0, 10.0, 100.0)))
probe.fit(Z_train, y_train)
print(r2_score(y_test, probe.predict(Z_test)))
Visualizing the geometry
Because the embeddings are organized by the target, a 2-D t-SNE of the
embeddings separates rows by their target value far more cleanly than a t-SNE of
the raw features — a quick visual check that the representation captures what
you’re predicting.
from sklearn.manifold import TSNE
from sklearn.preprocessing import StandardScaler
emb_2d = TSNE(n_components=2, init="pca", random_state=0).fit_transform(
StandardScaler().fit_transform(Z_test)
)
# scatter emb_2d colored by y_test
Parameters
NoriEmbedding:
| Parameter | Default | Description |
|---|
n_fold | 0 | Number of CV folds. 0 = vanilla (single model); >= 2 enables out-of-fold training embeddings. |
model | None | A pre-configured NoriRegressor. When None, a default one is built at fit time. |
shuffle | False | Whether to shuffle the K-fold split. |
random_state | None | Seed for the split when shuffle=True. |
fit, the transformer exposes model_ (the full-data model) and
train_embeddings_ (the training embeddings — out-of-fold when n_fold >= 2).
Notes
- These run on the local Python package (
synthefy-nori), not the hosted
API — extraction returns per-row vectors rather than predictions.
- Use out-of-fold embeddings (
n_fold >= 2) whenever you train a downstream
model on the training rows; vanilla embeddings leak the label.
- Embeddings are shape
(n_estimators, n_samples, embed_dim) — average over
axis=0 (or select [0]) before passing them to a 2-D estimator.
