Text Features

Nori reads numeric tables, but real tables often carry free text too: a product title, a review, a job description. Point Nori at those columns and it turns them into a handful of extra numeric features the frozen model consumes like any other column. Nothing is trained: the sentence encoder and Nori both stay frozen, so the gain comes purely from making text legible at inference time.

Signal from text

Columns your numeric features can’t see, like sentiment, topic and phrasing, become predictive features.

Zero-shot

No fine-tuning, no gradients. A frozen sentence encoder plus an unsupervised SVD; Nori’s weights never move.

One estimator

It’s still a NoriRegressor. Name the text columns in the constructor and call fit / predict on your DataFrame.

scikit-learn native

Round-trips through clone / GridSearchCV / cross_val_score and pickle, just like the numeric path.

Text support needs the text extra, which pulls in sentence-transformers. It is covered by the recommended line in Installation. The import is lazy, so numeric-only use never requires it. Extraction runs on the local synthefy-nori package; the sentence-encoder weights download from Hugging Face on first use.

Quickstart

Name the text columns in the constructor and use the estimator exactly as you would for a numeric table. X is a pandas.DataFrame that holds numeric, categorical, and text columns together.

import pandas as pd
from synthefy_nori import NoriRegressor

# df_train / df_test are DataFrames with numeric, categorical, and text columns
reg = NoriRegressor(model="nori-30m", text_columns=["title", "review"], svd_dim=128)
reg.fit(df_train, y_train)
preds = reg.predict(df_test)          # embeds the new text, applies the fit SVD

That’s the whole API surface: everything else (embedding, SVD, column encoding) happens inside fit / predict. Numeric and categorical columns you don’t list as text are handled automatically: numeric passes through, categorical is label-encoded.

How it works

The text path is a preprocessing layer in front of the ordinary numeric model. Nori itself is unchanged.

Build one paragraph per row

Each row’s text columns are concatenated into a single column-prefixed string, e.g. "title: Wireless Mouse. review: works great, tiny lag.".

Embed with a frozen encoder

A sentence-transformer (MiniLM by default) turns each paragraph into a dense vector. The encoder is frozen, with no fine-tuning.

Reduce with TruncatedSVD

The embeddings are reduced to svd_dim columns (default 128) with a TruncatedSVD fit on the training rows only. Unsupervised, it never sees y.

Append and predict

The SVD columns are appended to the numeric/categorical block, and the frozen Nori predicts on the widened table. predict re-embeds the query rows and applies the already-fit SVD, so the train/test transform is symmetric.

text columns
  → one column-prefixed paragraph per row
  → frozen sentence embedding (e.g. MiniLM, 384-d)
  → TruncatedSVD → svd_dim columns   (fit on train, unsupervised)
  → appended to the numeric / categorical block
  → NoriRegressor (frozen, in-context) predicts

Does it actually help? A runnable check

examples/text_features_synthetic.py builds a small synthetic dataset where the target depends on numeric columns (x1, x2), a categorical column (brand), and the sentiment word buried in a free-text review — signal the numeric columns can’t reach. It then fits Nori twice, with and without the text column, on a held-out split.

python examples/text_features_synthetic.py

Nori tabular-only (x1, x2, brand)   R2 = 0.1530
Nori + text review (svd-64)         R2 = 0.9854
text lift                           +0.8324

The review carries most of the target, so holding it out caps the tabular model at R² ≈ 0.15, while embedding it recovers R² ≈ 0.99. The core of the example:

import numpy as np, pandas as pd
from synthefy_nori import NoriRegressor

_SENTIMENTS = [("terrible", -2.0), ("poor", -1.0), ("okay", 0.0),
               ("good", 1.0), ("excellent", 2.0)]
_BRANDS = {"acme": 0.5, "globex": -0.5, "initech": 0.0}

def make_dataset(n, seed):
    rng = np.random.default_rng(seed)
    x1, x2 = rng.normal(size=n), rng.normal(size=n)
    brand = rng.choice(list(_BRANDS), size=n)
    idx = rng.integers(0, len(_SENTIMENTS), size=n)
    words = [_SENTIMENTS[i][0] for i in idx]
    sval = np.array([_SENTIMENTS[i][1] for i in idx])
    review = [f"Customer review: the item was {w}." for w in words]
    # target depends on numerics, brand, AND the review's sentiment word
    y = 2*x1 - 1.5*x2 + np.array([_BRANDS[b] for b in brand]) + 3*sval + rng.normal(0, 0.5, n)
    return pd.DataFrame({"x1": x1, "x2": x2, "brand": brand, "review": review}), y

df_tr, y_tr = make_dataset(800, 0)
df_te, y_te = make_dataset(200, 1)

# tabular-only: text_columns=[] keeps numeric + categorical, no embedder
tab  = NoriRegressor(model="nori-30m", text_columns=[]).fit(df_tr[["x1", "x2", "brand"]], y_tr)
# + text: embed the review, reduce, append
text = NoriRegressor(model="nori-30m", text_columns=["review"], svd_dim=64).fit(df_tr, y_tr)

text_columns=[] is a valid fit on a DataFrame that keeps only the numeric and categorical columns (no encoder loaded), handy for an apples-to-apples baseline against text_columns=[...].

Configuration

Constructor parameters

Parameter	Default	Description
`text_columns`	`None`	`None` → plain numeric array (the ordinary path). A list of column names → embed those columns (`X` must be a DataFrame). `[]` → numeric + categorical only. Columns are named explicitly; Nori never guesses which are text.
`svd_dim`	`128`	Number of SVD columns appended. `None` appends the full raw embedding (no reduction).
`embedder`	`"minilm"`	Short name (`"minilm"`, `"qwen4b"`, …), a preloaded sentence-transformer object, or a callable `texts -> ndarray`.
`text_max_cardinality`	`128`	Caps label encoding for non-numeric columns you did not name in `text_columns`: only the most common values up to this many get their own code, and the rest (plus unseen values) map to one “other” code. It never promotes a column to embedding — only `text_columns` does that.
`text_normalize`	`None`	Cosine-normalize embeddings. `None` auto-enables for known LLM encoders; set `True`/`False` to override (needed for a preloaded encoder object).

Choosing an encoder

"minilm" (384-d, fast) is a strong default and needs no extra setup. Larger encoders trade speed for representation quality:

Name	Dim	Notes
`"minilm"`	384	Fast baseline (default)
`"qwen0.6b"`	1024	Public, still fast
`"qwen4b"`	2560	Powerful, heavier
`"bge-m3"`	1024	Public

You can also pass a preloaded SentenceTransformer (reuse one across fits) or any callable texts -> (n, dim) ndarray.

Good to know

Under the hood: the numeric-only path (text_columns=None) is unchanged. High-cardinality columns are embedded rather than label-encoded, categorical codes are bounded (unseen values map to a single in-range “other” code), and the SVD is fit on train only, so a fitted text model pickles and clones like any sklearn estimator.

Get Started

Guides

Integration

Signal from text

Zero-shot

One estimator

scikit-learn native

Quickstart

How it works

Does it actually help? A runnable check

Configuration

Good to know

Next steps

Explainability

Embeddings

Signal from text

Zero-shot

One estimator

scikit-learn native

​Quickstart

​How it works

​Does it actually help? A runnable check

​Configuration

​Good to know

​Next steps

Explainability

Embeddings

Quickstart

How it works

Does it actually help? A runnable check

Configuration

Good to know

Next steps