> ## Documentation Index
> Fetch the complete documentation index at: https://docs.synthefy.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Embeddings
> Turn any table row into a target-aware vector with Nori's encoder, then probe, cluster, search, or visualize it.
Every time Nori looks at a row, it builds a dense vector for it, a learned
representation shaped by how that row relates to the target and to the rest of
your table. Pull those vectors out and they become features you can drop into
anything downstream.
Feed the vectors to a kNN, linear probe, or gradient-boosting head, often
matching Nori's own accuracy with a model you fully control.
Nearest-neighbor lookup in embedding space finds rows that are
*behaviorally* alike, not just numerically close.
Cluster on the embeddings to segment rows by what actually drives the
target.
Project to 2-D and watch the target structure appear. See the
[gallery](#see-it-on-real-data) below.
Nori is **regression-only**, so every embedding is conditioned on a continuous
target. Extraction runs on the local `synthefy-nori` package (not the hosted
API) and needs no API key. Weights download from Hugging Face on first use.
## Quickstart
Nothing beyond the base install — see [Installation](/setup/installation).
Embeddings need no extra, and scikit-learn is a core dependency, so
downstream heads work out of the box.
`NoriEmbedding` is a scikit-learn transformer. With `n_fold >= 2`,
`fit_transform` returns **leak-free** training embeddings (each row embedded
by a model that never saw its label), and `transform` embeds unseen rows
with the final full-data model.
```python theme={null}
from synthefy_nori import NoriEmbedding, NoriRegressor
embedder = NoriEmbedding(n_fold=5, shuffle=True, random_state=0,
model=NoriRegressor(model="nori-30m"))
train_emb = embedder.fit_transform(X_train, y_train) # out-of-fold
test_emb = embedder.transform(X_test) # full-data model
```
Both calls return a 3-D array `(n_estimators, n_samples, embed_dim)`, one
slice per preprocessing pipeline in the inference ensemble. Average over
axis 0 (or pick one member) for a matrix any estimator can consume.
```python theme={null}
Z_train = train_emb.mean(axis=0) # (n_train, embed_dim)
Z_test = test_emb.mean(axis=0) # (n_test, embed_dim)
```
The result is a plain 2-D matrix, so any scikit-learn estimator takes it
from here, for example a nearest-neighbor regressor:
```python theme={null}
from sklearn.neighbors import KNeighborsRegressor
from sklearn.metrics import r2_score
knn = KNeighborsRegressor(n_neighbors=10).fit(Z_train, y_train)
print(r2_score(y_test, knn.predict(Z_test)))
```
## Pick your extraction API
The sklearn transformer. Handles the fold logic and the final full-data fit
for you, so training embeddings stay leak-free.
| Method | Returns |
| --------------------------------- | ----------------------------------------------------------- |
| `fit_transform(X_train, y_train)` | training embeddings (out-of-fold when `n_fold >= 2`) |
| `transform(X)` | embeddings for unseen rows, always from the full-data model |
```python theme={null}
from synthefy_nori import NoriEmbedding, NoriRegressor
# Hand it a pre-built estimator to reuse one config across the fold-fits.
embedder = NoriEmbedding(
n_fold=5, shuffle=True, random_state=0,
model=NoriRegressor(model="nori-30m"),
)
Z_train = embedder.fit_transform(X_train, y_train).mean(axis=0)
Z_test = embedder.transform(X_test).mean(axis=0)
```
`transform` never returns cached training embeddings, even if the input
equals the training set. For training-row vectors use `fit_transform` (or
read `embedder.train_embeddings_`).
Direct control over which rows get embedded against which context. Call it
on a fitted `NoriRegressor`; `data_source` selects what comes back.
| `data_source` | Embeds |
| ------------------ | ----------------------------------------------------------------- |
| `"test"` (default) | the query rows you pass as `X` |
| `"train"` | the stored **context** rows, so `X` is ignored and can be omitted |
```python theme={null}
from synthefy_nori import NoriRegressor
model = NoriRegressor(model="nori-30m").fit(X_train, y_train)
test_emb = model.get_embeddings(X_test, data_source="test") # query rows
train_emb = model.get_embeddings(data_source="train") # context rows
```
These come from the full-data model, so `"train"` rows encode their own
labels. For leak-free training embeddings, use `NoriEmbedding` with
`n_fold >= 2`.
## Leak-free or fast: choose per use
The `n_fold` setting is the one decision that matters. It trades a bit of
compute for an honest downstream score.
K-fold cross-validation: each training row is embedded by a fold model that
**did not** have it in context, so the vector never leaks that row's own
label. Unseen rows are then embedded by a single model fit on the full
training set. This is what gives a downstream probe an unbiased estimate of
generalization ([arXiv:2502.17361](https://arxiv.org/abs/2502.17361)).
```python theme={null}
NoriEmbedding(n_fold=5, shuffle=True, random_state=0,
model=NoriRegressor(model="nori-30m"))
```
One model is fit on the whole training set and used for every row. It is
cheaper (a single fit instead of `n_fold + 1`), but the training embeddings
encode their own labels. Fine for visualization or embedding brand-new rows;
**don't** fit a downstream model on vanilla training embeddings and trust
the score.
```python theme={null}
NoriEmbedding(n_fold=0, model=NoriRegressor(model="nori-30m"))
```
## See it on real data
Because the vectors are organized by the target, a 2-D t-SNE of Nori embeddings
separates rows by their target value far more cleanly than a t-SNE of the raw
features. Both runnable examples below produce exactly these plots. To run them,
clone the repo and add the `eval` extra — it brings in matplotlib, plus openml
for the TabArena example. See [contributor extras](/setup/installation).
[`examples/embedding_synthetic.py`](https://github.com/Synthefy/synthefy-nori/blob/main/examples/embedding_synthetic.py)
runs the workflow on a small synthetic regression target: it extracts
embeddings, runs a kNN probe, and draws the t-SNE. Fast, with no data
download.
```bash theme={null}
python embedding_synthetic.py
```
```text theme={null}
synthetic regression: train=280 test=120 features=5
get_embeddings -> (n_estimators, n_samples, embed_dim) = (8, 120, 128)
=== R2 ===
kNN probe on embeddings : 0.9954
Nori native head (ref.) : 0.9995
```
[`examples/embedding_tabarena.py`](https://github.com/Synthefy/synthefy-nori/blob/main/examples/embedding_tabarena.py)
runs the same workflow on a real TabArena regression dataset (a Ridge probe
here). Pass `--dataset` to switch, or `--list` to see the options.
```bash theme={null}
python embedding_tabarena.py --dataset wine_quality
```
```text theme={null}
wine_quality: train=3000 test=1500 features=12
get_embeddings -> (n_estimators, n_samples, embed_dim) = (8, 1500, 128)
=== R2 ===
Ridge probe on embeddings : 0.5168
Nori native head (ref.) : 0.5148
```
Rolling your own plot is three lines once you have `Z_test`:
```python theme={null}
from sklearn.manifold import TSNE
from sklearn.preprocessing import StandardScaler
emb_2d = TSNE(n_components=2, init="pca", random_state=0).fit_transform(
StandardScaler().fit_transform(Z_test)
) # scatter emb_2d colored by y_test
```
## Configuration
| Parameter | Default | Description |
| -------------- | ------- | --------------------------------------------------------------------------------- |
| `n_fold` | `0` | CV folds. `0` = vanilla (single model); `>= 2` = out-of-fold training embeddings. |
| `model` | `None` | A pre-configured `NoriRegressor`. Built at `fit` time when `None`. |
| `shuffle` | `False` | Shuffle the K-fold split. |
| `random_state` | `None` | Seed for the split when `shuffle=True`. |
After `fit`, the transformer exposes `model_` (the full-data model) and
`train_embeddings_` (out-of-fold when `n_fold >= 2`).
A standardized ridge probe (good default for the higher-dimensional
embeddings of real data):
```python theme={null}
from sklearn.linear_model import RidgeCV
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
probe = make_pipeline(StandardScaler(), RidgeCV(alphas=(0.1, 1.0, 10.0, 100.0)))
probe.fit(Z_train, y_train)
```
The same `Z_train` / `Z_test` matrices feed clustering (`KMeans`), nearest-
neighbor search (`NearestNeighbors`), or any 2-D projection.
## Good to know
**Shapes at a glance:** embeddings are `(n_estimators, n_samples, embed_dim)`.
Average over `axis=0` (or select `[0]`) before handing them to a 2-D estimator.
## Next steps
Turn a time series into a table and forecast it locally with Nori.
SHAP values and partial dependence on the same fitted estimator.