> ## Documentation Index
> Fetch the complete documentation index at: https://docs.synthefy.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Text features
> Hand Nori free-text columns — it embeds them, reduces them to a few numeric columns, and predicts on the widened table. Zero-shot, no training.
Nori reads numeric tables, but real tables often carry free text too — a product
title, a review, a job description. Point Nori at those columns and it turns them
into a handful of extra numeric features the frozen model consumes like any other
column. Nothing is trained: the sentence encoder and Nori both stay frozen, so the
gain comes purely from making text legible at inference time.
Columns your numeric features can't see — sentiment, topic, phrasing — become
predictive features.
No fine-tuning, no gradients. A frozen sentence encoder plus an unsupervised
SVD; Nori's weights never move.
It's still a `NoriRegressor`. Name the text columns in the constructor and call
`fit` / `predict` on your DataFrame.
Round-trips through `clone` / `GridSearchCV` / `cross_val_score` and `pickle`,
just like the numeric path.
Text support needs the optional `text` extra (`pip install "synthefy-nori[text]"`),
which pulls in `sentence-transformers`. It's imported lazily, so numeric-only use
never requires it. Extraction runs on the local `synthefy-nori` package; the
sentence-encoder weights download from Hugging Face on first use.
## How it works
The text path is a preprocessing layer in front of the ordinary numeric model —
Nori itself is unchanged.
Each row's text columns are concatenated into a single column-prefixed string,
e.g. `"title: Wireless Mouse. review: works great, tiny lag."`.
A sentence-transformer (MiniLM by default) turns each paragraph into a dense
vector. The encoder is frozen — no fine-tuning.
The embeddings are reduced to `svd_dim` columns (default 128) with a
TruncatedSVD **fit on the training rows only** — unsupervised, it never sees
`y`.
The SVD columns are appended to the numeric/categorical block, and the frozen
Nori predicts on the widened table. `predict` re-embeds the query rows and
applies the already-fit SVD, so the train/test transform is symmetric.
```text theme={null}
text columns
→ one column-prefixed paragraph per row
→ frozen sentence embedding (e.g. MiniLM, 384-d)
→ TruncatedSVD → svd_dim columns (fit on train, unsupervised)
→ appended to the numeric / categorical block
→ NoriRegressor (frozen, in-context) predicts
```
## Quickstart
Name the text columns in the constructor and use the estimator exactly as you
would for a numeric table — `X` is a `pandas.DataFrame` that holds numeric,
categorical, **and** text columns together.
```bash theme={null}
pip install "synthefy-nori[text]"
```
```python theme={null}
import pandas as pd
from synthefy_nori import NoriRegressor
# df_train / df_test are DataFrames with numeric, categorical, and text columns
reg = NoriRegressor(text_columns=["title", "review"], svd_dim=128)
reg.fit(df_train, y_train)
preds = reg.predict(df_test) # embeds the new text, applies the fit SVD
```
That's the whole API surface: everything else (embedding, SVD, column encoding)
happens inside `fit` / `predict`. Numeric and categorical columns you don't list
as text are handled automatically — numeric passes through, categorical is
label-encoded.
## Does it actually help? A runnable check
[`examples/text_features_synthetic.py`](https://github.com/Synthefy/synthefy-nori/blob/main/examples/text_features_synthetic.py)
builds a small synthetic dataset where the target depends on numeric columns
(`x1`, `x2`), a categorical column (`brand`), **and** the sentiment word buried in
a free-text `review` — signal the numeric columns can't reach. It then fits Nori
twice, with and without the text column, on a held-out split.
```bash theme={null}
python examples/text_features_synthetic.py
```
```text theme={null}
Nori tabular-only (x1, x2, brand) R2 = 0.1530
Nori + text review (svd-64) R2 = 0.9854
text lift +0.8324
```
The review carries most of the target, so holding it out caps the tabular model at
R² ≈ 0.15, while embedding it recovers R² ≈ 0.99. The core of the example:
```python theme={null}
import numpy as np, pandas as pd
from synthefy_nori import NoriRegressor
_SENTIMENTS = [("terrible", -2.0), ("poor", -1.0), ("okay", 0.0),
("good", 1.0), ("excellent", 2.0)]
_BRANDS = {"acme": 0.5, "globex": -0.5, "initech": 0.0}
def make_dataset(n, seed):
rng = np.random.default_rng(seed)
x1, x2 = rng.normal(size=n), rng.normal(size=n)
brand = rng.choice(list(_BRANDS), size=n)
idx = rng.integers(0, len(_SENTIMENTS), size=n)
words = [_SENTIMENTS[i][0] for i in idx]
sval = np.array([_SENTIMENTS[i][1] for i in idx])
review = [f"Customer review: the item was {w}." for w in words]
# target depends on numerics, brand, AND the review's sentiment word
y = 2*x1 - 1.5*x2 + np.array([_BRANDS[b] for b in brand]) + 3*sval + rng.normal(0, 0.5, n)
return pd.DataFrame({"x1": x1, "x2": x2, "brand": brand, "review": review}), y
df_tr, y_tr = make_dataset(800, 0)
df_te, y_te = make_dataset(200, 1)
# tabular-only: text_columns=[] keeps numeric + categorical, no embedder
tab = NoriRegressor(text_columns=[]).fit(df_tr[["x1", "x2", "brand"]], y_tr)
# + text: embed the review, reduce, append
text = NoriRegressor(text_columns=["review"], svd_dim=64).fit(df_tr, y_tr)
```
`text_columns=[]` is a valid fit on a DataFrame that keeps only the numeric and
categorical columns (no encoder loaded) — handy for an apples-to-apples baseline
against `text_columns=[...]`.
## Configuration
| Parameter | Default | Description |
| ---------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `text_columns` | `None` | `None` → plain numeric array (the ordinary path). A list of column names → embed those columns (`X` must be a DataFrame). `[]` → numeric + categorical only. Columns are named explicitly; Nori never guesses which are text. |
| `svd_dim` | `128` | Number of SVD columns appended. `None` appends the full raw embedding (no reduction). |
| `embedder` | `"minilm"` | Short name (`"minilm"`, `"qwen4b"`, …), a preloaded sentence-transformer object, or a callable `texts -> ndarray`. |
| `text_max_cardinality` | `128` | A non-numeric column with more distinct values than this is embedded rather than label-encoded; categorical encoding also caps here (rarer/unseen values map to one "other" code). |
| `text_normalize` | `None` | Cosine-normalize embeddings. `None` auto-enables for known LLM encoders; set `True`/`False` to override (needed for a preloaded encoder object). |
`"minilm"` (384-d, fast) is a strong default and needs no extra setup. Larger
encoders trade speed for representation quality:
| Name | Dim | Notes |
| ------------ | ---- | ----------------------- |
| `"minilm"` | 384 | Fast baseline (default) |
| `"qwen0.6b"` | 1024 | Public, still fast |
| `"qwen4b"` | 2560 | Powerful, heavier |
| `"bge-m3"` | 1024 | Public |
You can also pass a preloaded `SentenceTransformer` (reuse one across fits) or
any callable `texts -> (n, dim) ndarray`.
**Under the hood:** the numeric-only path (`text_columns=None`) is unchanged.
High-cardinality columns are embedded rather than label-encoded, categorical codes
are bounded (unseen values map to a single in-range "other" code), and the SVD is
fit on train only — so a fitted text model pickles and clones like any sklearn
estimator.