Text features - Synthefy Docs

Nori reads numeric tables, but real tables often carry free text too — a product title, a review, a job description. Point Nori at those columns and it turns them into a handful of extra numeric features the frozen model consumes like any other column. Nothing is trained: the sentence encoder and Nori both stay frozen, so the gain comes purely from making text legible at inference time.

Signal from text

Columns your numeric features can’t see — sentiment, topic, phrasing — become predictive features.

Zero-shot

No fine-tuning, no gradients. A frozen sentence encoder plus an unsupervised SVD; Nori’s weights never move.

One estimator

It’s still a NoriRegressor. Name the text columns in the constructor and call fit / predict on your DataFrame.

scikit-learn native

Round-trips through clone / GridSearchCV / cross_val_score and pickle, just like the numeric path.

Text support needs the optional text extra (pip install "synthefy-nori[text]"), which pulls in sentence-transformers. It’s imported lazily, so numeric-only use never requires it. Extraction runs on the local synthefy-nori package; the sentence-encoder weights download from Hugging Face on first use.

How it works

The text path is a preprocessing layer in front of the ordinary numeric model — Nori itself is unchanged.

Build one paragraph per row

Each row’s text columns are concatenated into a single column-prefixed string, e.g. "title: Wireless Mouse. review: works great, tiny lag.".

Embed with a frozen encoder

A sentence-transformer (MiniLM by default) turns each paragraph into a dense vector. The encoder is frozen — no fine-tuning.

Reduce with TruncatedSVD

The embeddings are reduced to svd_dim columns (default 128) with a TruncatedSVD fit on the training rows only — unsupervised, it never sees y.

Append and predict

The SVD columns are appended to the numeric/categorical block, and the frozen Nori predicts on the widened table. predict re-embeds the query rows and applies the already-fit SVD, so the train/test transform is symmetric.

text columns
  → one column-prefixed paragraph per row
  → frozen sentence embedding (e.g. MiniLM, 384-d)
  → TruncatedSVD → svd_dim columns   (fit on train, unsupervised)
  → appended to the numeric / categorical block
  → NoriRegressor (frozen, in-context) predicts

Quickstart

Name the text columns in the constructor and use the estimator exactly as you would for a numeric table — X is a pandas.DataFrame that holds numeric, categorical, and text columns together.

pip install "synthefy-nori[text]"

import pandas as pd
from synthefy_nori import NoriRegressor

# df_train / df_test are DataFrames with numeric, categorical, and text columns
reg = NoriRegressor(text_columns=["title", "review"], svd_dim=128)
reg.fit(df_train, y_train)
preds = reg.predict(df_test)          # embeds the new text, applies the fit SVD

That’s the whole API surface: everything else (embedding, SVD, column encoding) happens inside fit / predict. Numeric and categorical columns you don’t list as text are handled automatically — numeric passes through, categorical is label-encoded.

Does it actually help? A runnable check

examples/text_features_synthetic.py builds a small synthetic dataset where the target depends on numeric columns (x1, x2), a categorical column (brand), and the sentiment word buried in a free-text review — signal the numeric columns can’t reach. It then fits Nori twice, with and without the text column, on a held-out split.

python examples/text_features_synthetic.py

Nori tabular-only (x1, x2, brand)   R2 = 0.1530
Nori + text review (svd-64)         R2 = 0.9854
text lift                           +0.8324

The review carries most of the target, so holding it out caps the tabular model at R² ≈ 0.15, while embedding it recovers R² ≈ 0.99. The core of the example:

import numpy as np, pandas as pd
from synthefy_nori import NoriRegressor

_SENTIMENTS = [("terrible", -2.0), ("poor", -1.0), ("okay", 0.0),
               ("good", 1.0), ("excellent", 2.0)]
_BRANDS = {"acme": 0.5, "globex": -0.5, "initech": 0.0}

def make_dataset(n, seed):
    rng = np.random.default_rng(seed)
    x1, x2 = rng.normal(size=n), rng.normal(size=n)
    brand = rng.choice(list(_BRANDS), size=n)
    idx = rng.integers(0, len(_SENTIMENTS), size=n)
    words = [_SENTIMENTS[i][0] for i in idx]
    sval = np.array([_SENTIMENTS[i][1] for i in idx])
    review = [f"Customer review: the item was {w}." for w in words]
    # target depends on numerics, brand, AND the review's sentiment word
    y = 2*x1 - 1.5*x2 + np.array([_BRANDS[b] for b in brand]) + 3*sval + rng.normal(0, 0.5, n)
    return pd.DataFrame({"x1": x1, "x2": x2, "brand": brand, "review": review}), y

df_tr, y_tr = make_dataset(800, 0)
df_te, y_te = make_dataset(200, 1)

# tabular-only: text_columns=[] keeps numeric + categorical, no embedder
tab  = NoriRegressor(text_columns=[]).fit(df_tr[["x1", "x2", "brand"]], y_tr)
# + text: embed the review, reduce, append
text = NoriRegressor(text_columns=["review"], svd_dim=64).fit(df_tr, y_tr)

text_columns=[] is a valid fit on a DataFrame that keeps only the numeric and categorical columns (no encoder loaded) — handy for an apples-to-apples baseline against text_columns=[...].

Configuration

Constructor parameters

Parameter	Default	Description
`text_columns`	`None`	`None` → plain numeric array (the ordinary path). A list of column names → embed those columns (`X` must be a DataFrame). `[]` → numeric + categorical only. Columns are named explicitly; Nori never guesses which are text.
`svd_dim`	`128`	Number of SVD columns appended. `None` appends the full raw embedding (no reduction).
`embedder`	`"minilm"`	Short name (`"minilm"`, `"qwen4b"`, …), a preloaded sentence-transformer object, or a callable `texts -> ndarray`.
`text_max_cardinality`	`128`	A non-numeric column with more distinct values than this is embedded rather than label-encoded; categorical encoding also caps here (rarer/unseen values map to one “other” code).
`text_normalize`	`None`	Cosine-normalize embeddings. `None` auto-enables for known LLM encoders; set `True`/`False` to override (needed for a preloaded encoder object).

Choosing an encoder

"minilm" (384-d, fast) is a strong default and needs no extra setup. Larger encoders trade speed for representation quality:

Name	Dim	Notes
`"minilm"`	384	Fast baseline (default)
`"qwen0.6b"`	1024	Public, still fast
`"qwen4b"`	2560	Powerful, heavier
`"bge-m3"`	1024	Public

You can also pass a preloaded SentenceTransformer (reuse one across fits) or any callable texts -> (n, dim) ndarray.

Under the hood: the numeric-only path (text_columns=None) is unchanged. High-cardinality columns are embedded rather than label-encoded, categorical codes are bounded (unseen values map to a single in-range “other” code), and the SVD is fit on train only — so a fitted text model pickles and clones like any sklearn estimator.

Signal from text

Zero-shot

One estimator

scikit-learn native

​How it works

​Quickstart

​Does it actually help? A runnable check

​Configuration

How it works

Quickstart

Does it actually help? A runnable check

Configuration