Nori Fine-Tuning on Amazon SageMaker

SynthefyNoriFineTuner fine-tunes Nori on your table in your own AWS account, deploys the result as a SageMaker asynchronous endpoint and gives you a SynthefyNoriClient for it. To serve the released models as they are, see Nori Inference on Amazon SageMaker.

What you get

A Nori model fine-tuned on your table. Fine-tuning is regression only.
An asynchronous SageMaker endpoint in your account that serves only that model, under the name you choose.
A training report that scores the fine-tuned model and the base model on the same rows.

The training job, the endpoint and your data stay in your account. No Synthefy service sits in front of them, and Synthefy can neither see nor stop them.

Before you start

Install the client

pip install -U "synthefy[aws]>=7.3.0"

Subscribe to Synthefy - Nori Fine-Tuning

Open the Synthefy - Nori Fine-Tuning listing in AWS Marketplace and choose View purchase options.

Synthefy - Nori Fine-Tuning product page in AWS Marketplace with View purchase options

Review the pricing and terms, then choose Subscribe at the bottom of the page.

Copy the Algorithm ARN for your region

Once the subscription is active, choose Launch your software on the same page. Under Algorithm ARNs, copy the ARN for your region (the JSON view lists every region). To come back to it later, open the listing and choose View purchase options, then Launch your software.Use that region for the bucket, the training job and the endpoint: the client refuses an Algorithm ARN from another region before any AWS call. You don’t need the page’s Create training job button: the client starts the job.

Launch page for Synthefy - Nori Fine-Tuning with the Algorithm ARNs panel and its copy button highlighted

Create the execution role

SageMaker runs the training job and the endpoint as this role.

Console
AWS CLI

In the IAM console choose Roles, then Create role. Choose AWS service, then SageMaker as the use case, and continue: the role gets AWS’s AmazonSageMakerFullAccess policy. Name it nori-fine-tune-execution and choose Create role.

aws iam create-role --role-name nori-fine-tune-execution --assume-role-policy-document \
  '{"Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Principal": {"Service": "sagemaker.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
aws iam attach-role-policy --role-name nori-fine-tune-execution \
  --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess

Choose a bucket

Use an S3 bucket in that region whose name contains sagemaker, such as sagemaker-us-east-1-ACCOUNT_ID. AmazonSageMakerFullAccess grants object access to buckets named that way, so a bucket named otherwise is refused. To create one:

aws s3 mb s3://sagemaker-us-east-1-ACCOUNT_ID --region us-east-1

The identity that runs your code needs SageMaker and S3 access too, and AmazonSageMakerFullAccess covers every call the client makes. To grant both identities less, use the least-privilege policies.

A new AWS account can have no quota for GPU instances. SageMaker then refuses the job or the endpoint before anything bills, and the error names the quota to raise in the Service Quotas console, such as ml.g5.xlarge for training job usage.

Fine-tune a model

The snippets on this page run in order in one Python session. Replace ACCOUNT_ID, the Algorithm ARN and the target column y with your own.

Prepare your table

Fine-tuning is regression only, on a CSV table with one header row.

Every column the job trains on must be numeric. Features may have missing values; the target must be finite in every row.
Encode categorical columns as numbers before you upload, and keep that encoding for prediction: a text column fails the job.
The training table needs at least 1,111 rows, or 1,000 with a validation channel. Above 200,000 rows or 30 million cells (rows x features), the job trains on a subsample drawn with its seed.

training_data is one object ending in .csv or a folder ending in /. Under a folder, the job reads every uncompressed .csv file in it and its sub-folders, and those files must share one header. validation_data is optional, takes the same form and has exactly the training table’s columns: the job validates on its rows and never trains on them. The walkthrough uses three files with the same columns: train.csv, the table the job trains on; validation.csv, optional; and new.csv, rows the job never sees, for the prediction below. Bring your own, or let the first snippet write a synthetic example of each:

import logging
from pathlib import Path
import boto3
import numpy as np
import pandas as pd
from synthefy import FineTuneHyperparameters, FineTuneJobRequest, SynthefyNoriFineTuner

logging.basicConfig(level=logging.INFO)  # wait() logs each status change
REGION = "us-east-1"
ALGORITHM_ARN = "arn:aws:sagemaker:us-east-1:..."  # copied from your subscription, for REGION
ROLE_ARN = "arn:aws:iam::ACCOUNT_ID:role/nori-fine-tune-execution"
BUCKET, TARGET = "sagemaker-us-east-1-ACCOUNT_ID", "y"


def example_table(rows: int, seed: int) -> pd.DataFrame:
    """A synthetic regression table: eight numeric features and a target no straight line fits."""
    rng = np.random.default_rng(seed)
    x = rng.normal(size=(rows, 8))
    weights = np.random.default_rng(1234).normal(size=4)  # fixed, so every table samples the same function
    y = np.sin(1.5 * x[:, 0]) + 0.5 * x[:, 1] ** 2 + x[:, 2] * x[:, 3] + np.tanh(x[:, 4:] @ weights / 2)
    y = y + 0.1 * rng.normal(size=rows)
    return pd.DataFrame(x, columns=[f"f{i}" for i in range(1, 9)]).assign(**{TARGET: y})


if not any(Path(name).exists() for name in ("train.csv", "validation.csv", "new.csv")):
    held_out = example_table(500, seed=1)
    example_table(2000, seed=0).to_csv("train.csv", index=False)
    held_out.iloc[:400].to_csv("validation.csv", index=False)
    held_out.iloc[400:].to_csv("new.csv", index=False)

s3 = boto3.client("s3", region_name=REGION)
s3.upload_file("train.csv", BUCKET, "nori/data/train/train.csv")
s3.upload_file("validation.csv", BUCKET, "nori/data/validation/validation.csv")  # optional
table = pd.read_csv("train.csv")
features = [column for column in table.columns if column != TARGET]  # the order the job trains on

Start the job

The request is checked before any AWS call: the S3 locations, the hyperparameters, an instance type the base model fits and a runtime that covers every child. A data path without its trailing slash (.../train, which S3 would also match train-old/ against) or a base too large for the instance is refused here, with the reason, rather than minutes into a billed job.

tuner = SynthefyNoriFineTuner(region_name=REGION)
request = FineTuneJobRequest(
    algorithm=ALGORITHM_ARN,
    role_arn=ROLE_ARN,
    training_data=f"s3://{BUCKET}/nori/data/train/",
    validation_data=f"s3://{BUCKET}/nori/data/validation/",  # optional: the job validates on these rows
    output_path=f"s3://{BUCKET}/nori/output/",
    hyperparameters=FineTuneHyperparameters(
        target_column=TARGET,
        model_name="ft-acme-churn-v1",  # the name your fine-tuned model will answer to
        base_model="nori-6m",
        bag=2,  # the number of children the job trains, 1 to 8; the fine-tuned model is built from them
        steps=30,
    ),
)
print(request.instance_type, request.max_runtime_s)  # ml.g5.xlarge 4500
job = tuner.fine_tune(request)
print(job.job_name)  # keep it: every later step can start from this name

bag=2, steps=30 is a small run: the job takes 3 to 5 minutes when capacity is available. Every hyperparameter, with its default:

Hyperparameter	Default	Meaning
`target_column`	required	the column of the training table to predict
`model_name`	required	the name the fine-tuned model answers to, and the endpoint’s name later: `ft-` followed by 1 to 60 lowercase letters, digits and `-`, the first a letter or digit; a released model’s name is refused
`base_model`	`nori-6m`	the released Nori model to fine-tune: `nori-6m` or `nori-100m`
`task`	`regression`	only regression is supported
`feature_columns`	every column but the target	the feature columns, as a list
`bag`	8	the number of children the job trains, 1 to 8; the fine-tuned model is built from them, and each child has its own steps and time budget
`steps`	100	maximum training steps per child
`patience`	25	a child stops after this many steps without improvement
`budget_s`	900	wall-clock cap per child, in seconds
`lr`	1e-5	learning rate
`seed`	0	the job’s random seed

Follow the job

job = tuner.wait(job.job_name)
print(job.status, job.billable_seconds)  # Completed 149
if job.status == "Failed":
    raise SystemExit(job.failure_reason)
for line in tuner.get_logs(job.job_name):  # the job's own log lines
    print(line)
curves = pd.DataFrame(tuner.get_training_curves(job.job_name))  # columns child, step, train_loss, val_mse

get_training_curves() gives one row per child and validated step:

Validation error per validated step for each child of a fine-tuning job

The job also appears under Training jobs in the SageMaker AI console:

A completed fine-tuning job in the Training jobs list of the SageMaker AI console

tuner.stop(job.job_name) stops a job. It still writes a model from the training done so far when any of it improved on the base, and ends Stopped with cut_short true in the report. A job keeps no checkpoints and cannot resume; a new job starts again from the base.

Read the result

report = tuner.get_training_report(job.job_name)
print(report.val_mse_before, report.val_mse_after)  # the base model and the fine-tuned model, same rows

Lower is better: the distance from val_mse_before to val_mse_after is what fine-tuning gained on your table, so always read the two together. Both are mean squared errors in standardized target units, scored on the same rows. Without a validation channel those rows are held out of training (10 % of the table, up to 4,096 rows); with one, they are the channel’s rows, which also steer the training, so val_mse_after runs a little optimistic there. The report also names n_val_rows, n_rows_used, cut_short and model_version.

Access the fine-tuned model

The job writes model.tar.gz under output_path, at job.artifact_uri, with the model, its registry entry and training_report.json. The model holds what changed relative to the base, so you serve it through an endpoint, not on its own. It stays in your bucket until you delete it.

Never expire nori/output/ as a whole: the model package reads its model from there.

From another session, pick the job up by its name; deploy() takes the name directly:

tuner = SynthefyNoriFineTuner(region_name=REGION)
job = tuner.get_status("nori-ft-acme-churn-v1-...")  # the name fine_tune() printed
print(job.status, job.artifact_uri)

Deploy and predict

Deploy

endpoint = tuner.deploy(job)  # asynchronous endpoint on ml.g5.xlarge, named after model_name
print(endpoint.endpoint_name, endpoint.status)  # ft-acme-churn-v1 InService

deploy() makes a model package from the job, a SageMaker model, an endpoint config and the endpoint, and returns once it is InService, in 4 to 6 minutes. Pass instance_type= for another endpoint type from the instance table. The endpoint appears under Endpoints in the console:

A fine-tuned model's asynchronous endpoint InService in the SageMaker AI console

The endpoint bills every hour from creation until undeploy(), whether or not you send requests. Prices per instance type and region are on SageMaker AI pricing.

Predict

client = tuner.create_client(endpoint, s3_staging_uri=f"s3://{BUCKET}/nori/invocations/")
new_rows = pd.read_csv("new.csv")  # rows the job never saw: same columns, same order
X_test, truth = new_rows[features].head(20), new_rows[TARGET].head(20)
context = table.sample(n=500, random_state=0)
prediction = client.predict(
    X_train=context[features], y_train=context[TARGET], X_test=X_test, categorical_columns=None, as_pandas=True
)
assert client.last_model_version == report.model_version  # the fine-tuned model answered
print(pd.DataFrame({"prediction": prediction.values, "truth": truth.values}).round(3).head(8))

create_client() builds the same SynthefyNoriClient every Nori user has, in SageMaker mode, bound to this endpoint and the fine-tuned model’s name. The endpoint does not store your table, so every predict() sends context rows; 500 keeps the example small. Send the feature columns in training order, without the target. new.csv holds rows the job never saw, so the comparison with truth is an honest one; the validation rows would flatter it, since they also steered the training. The endpoint serves only its own model and refuses a request naming another, even a valid ft- name.

Several models and endpoints

A job trains one model, and an endpoint serves the one model it was deployed from: a request naming another model is refused. For several models, run one job per model_name and deploy each:

endpoints = {}
for name, target in [("ft-acme-churn-v1", "churn"), ("ft-acme-ltv-v1", "ltv")]:
    hp = FineTuneHyperparameters(target_column=target, model_name=name)
    # max_runtime_s=None: the copy derives its runtime from these hyperparameters
    per_model = request.model_copy(update={"hyperparameters": hp, "max_runtime_s": None})
    job = tuner.wait(tuner.fine_tune(per_model).job_name)
    endpoints[name] = tuner.deploy(job)  # one endpoint per model, named after it

Jobs run at the same time when your training quota allows it (see the quotas below). One model can sit behind several endpoints: pass a different endpoint_name to each deploy() call (the default is the model’s name). The model package is reused, each endpoint gets its own async output prefix under output_path, and each bills on its own. Separate endpoints isolate workloads, for example one per team or per environment. For throughput, add instances to one endpoint instead: each instance handles one request at a time and extra requests wait in the endpoint’s queue, so N instances answer N requests at once. deploy() creates the endpoint with one instance; raise the count from the SageMaker AI console (the endpoint’s configuration) or with aws sagemaker update-endpoint and a new endpoint config, within your endpoint quota.

Switch it off

tuner.undeploy(endpoint)  # or tuner.undeploy("ft-acme-churn-v1") from another session
tuner.close()

undeploy() deletes the endpoint, its endpoint configs and the SageMaker model, and keeps the model package for a later deploy(). Expire nori/invocations/ and nori/output/ft-acme-churn-v1/async/ after at least a day.

When something goes wrong

You see	What to do
`deploy()` or `get_training_report()` raises `FineTuningError` saying the `Stopped` job has no model artifact	Nothing had improved on the base when the job stopped, so it wrote no model. Start a new job.
`Failed`, and `failure_reason` names your table or settings	The job refused the input once its instance was up. Fix it and start a new job; `tuner.get_logs()` has any traceback.
`InProgress` in `Pending` for a long time with the message `Training job waiting for capacity`, or `Failed` with `CapacityError:`	AWS has no spare capacity for that type. The wait is unbilled and can last hours. Stop it and start a new job on another type.
`wait()` raises `APITimeoutError`	It waited the time limit plus 7,200 s. The job still runs: call `wait()` again, or `stop()`.
`deploy()` raises `FineTuningError` naming `InsufficientInstanceCapacity`	AWS found no instance of that type in time. Call `tuner.undeploy("ft-acme-churn-v1")`, then `tuner.deploy(job, instance_type=...)` with another endpoint type; it reuses the model package.
`deploy()` raises `APITimeoutError`	The endpoint was not InService within `timeout_s` (3,600 s by default) and is still starting and billing. Call `tuner.undeploy("ft-acme-churn-v1")`, then deploy again with a larger `timeout_s`.

Reference

Instance types and quotas

You need quota for one training instance and one endpoint instance. A job’s default type follows its base model:

Use	Instance types	Default
Train `nori-6m`	`ml.g5.xlarge`, `ml.g5.2xlarge`, `ml.g6.xlarge` (24 GB); `ml.g6e.xlarge`, `ml.g6e.2xlarge` (48 GB)	`ml.g5.xlarge`
Train `nori-100m`	`ml.g6e.xlarge`, `ml.g6e.2xlarge` only	`ml.g6e.xlarge`
Endpoint	`ml.g5.xlarge`, `ml.g5.2xlarge`, `ml.g6.xlarge`	`ml.g5.xlarge`

The quotas are named like ml.g5.xlarge for training job usage and ml.g5.xlarge for endpoint usage. At 0 SageMaker refuses the create call (ResourceLimitExceeded) before anything bills, and the client raises FineTuningError naming the quota. The same quotas set how much runs at once: N jobs on ml.g5.xlarge need its training quota at N or more, and N endpoint instances of a type need its endpoint quota at N or more. New accounts often start at 0 for GPU types; request what you need in the Service Quotas console before the first job, since approval can take a few days. Check your quota:

aws service-quotas list-service-quotas --service-code sagemaker --region us-east-1 \
  --query "Quotas[?contains(QuotaName, 'ml.g5.xlarge')].[QuotaName,Value]" --output table
# lists the training and the endpoint quota; for nori-100m, run it again with ml.g6e.xlarge

Least-privilege IAM policies

For production, replace AmazonSageMakerFullAccess with two policies that grant each identity only what it uses. With them the bucket can have any name. The execution role’s policy, saved as execution-policy.json with BUCKET, REGION, ACCOUNT_ID and ALGORITHM_ARN replaced, lets:

the job read nori/data/ and write nori/output/
the endpoint read nori/invocations/ and write under nori/output/
SageMaker describe the Algorithm and your model package as this role

The model package is named after the job, nori-ft-... unless you set job_name; if you set it, use its prefix in this policy.

execution-policy.json

{"Version": "2012-10-17", "Statement": [
  {"Effect": "Allow", "Action": ["s3:ListBucket", "s3:GetBucketLocation"], "Resource": "arn:aws:s3:::BUCKET"},
  {"Effect": "Allow", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::BUCKET/nori/data/*"},
  {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:AbortMultipartUpload"],
   "Resource": ["arn:aws:s3:::BUCKET/nori/output/*", "arn:aws:s3:::BUCKET/nori/invocations/*"]},
  {"Effect": "Allow", "Action": ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams"],
   "Resource": ["arn:aws:logs:REGION:ACCOUNT_ID:log-group:/aws/sagemaker/TrainingJobs*",
                "arn:aws:logs:REGION:ACCOUNT_ID:log-group:/aws/sagemaker/Endpoints/*"]},
  {"Effect": "Allow", "Action": "cloudwatch:PutMetricData", "Resource": "*"},
  {"Effect": "Allow", "Action": ["sagemaker:DescribeAlgorithm", "sagemaker:DescribeModelPackage"],
   "Resource": ["ALGORITHM_ARN", "arn:aws:sagemaker:REGION:ACCOUNT_ID:model-package/nori-ft-*"]}]}

Put it on the role in place of the managed policy:

aws iam put-role-policy --role-name nori-fine-tune-execution --policy-name nori --policy-document file://execution-policy.json
aws iam detach-role-policy --role-name nori-fine-tune-execution --policy-arn arn:aws:iam::aws:policy/AmazonSageMakerFullAccess

The identity that runs your code needs the policy below, saved as caller-policy.json with BUCKET, REGION and ACCOUNT_ID replaced. s3:ListBucket makes a missing object read as not found, which the client relies on while it waits for a result. With kms_key_id, the key’s policy must let both identities use it.

caller-policy.json

{"Version": "2012-10-17", "Statement": [
  {"Effect": "Allow", "Resource": "*", "Action": [
    "sagemaker:CreateTrainingJob", "sagemaker:DescribeTrainingJob", "sagemaker:StopTrainingJob", "sagemaker:DescribeAlgorithm",
    "sagemaker:CreateModelPackage", "sagemaker:DescribeModelPackage", "sagemaker:DeleteModelPackage", "sagemaker:CreateModel",
    "sagemaker:DescribeModel", "sagemaker:DeleteModel", "sagemaker:CreateEndpointConfig", "sagemaker:DescribeEndpointConfig",
    "sagemaker:ListEndpointConfigs", "sagemaker:DeleteEndpointConfig", "sagemaker:CreateEndpoint", "sagemaker:DescribeEndpoint",
    "sagemaker:DeleteEndpoint", "sagemaker:InvokeEndpointAsync"]},
  {"Effect": "Allow", "Action": "iam:PassRole", "Resource": "arn:aws:iam::ACCOUNT_ID:role/nori-fine-tune-execution",
   "Condition": {"StringEquals": {"iam:PassedToService": "sagemaker.amazonaws.com"}}},
  {"Effect": "Allow", "Action": ["logs:DescribeLogStreams", "logs:GetLogEvents"],
   "Resource": ["arn:aws:logs:REGION:ACCOUNT_ID:log-group:/aws/sagemaker/TrainingJobs",
                "arn:aws:logs:REGION:ACCOUNT_ID:log-group:/aws/sagemaker/TrainingJobs:*"]},
  {"Effect": "Allow", "Action": "s3:ListBucket", "Resource": "arn:aws:s3:::BUCKET"},
  {"Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"], "Resource": "arn:aws:s3:::BUCKET/nori/*"}]}

For a role:

aws iam put-role-policy --role-name YOUR_ROLE --policy-name nori-caller --policy-document file://caller-policy.json

Alarms

An EventBridge rule on SageMaker’s training job state changes can email you when a job ends Failed or Stopped, and CloudWatch alarms on InvocationFailures and ApproximateAgeOfOldestRequest watch an endpoint you keep running. The client’s README has the commands, under Alarms.

Next steps

Inference on Amazon SageMaker

Serve the released Nori models from one SageMaker endpoint, without fine-tuning.

Python Client

DataFrames, categorical encoding, retries, and distribution outputs.

Get Started

Guides

Integration

Nori Fine-Tuning on Amazon SageMaker

What you get

Before you start

Fine-tune a model

Prepare your table

Start the job

Follow the job

Read the result

Access the fine-tuned model

Deploy and predict

Deploy

Predict

Several models and endpoints

Switch it off

When something goes wrong

Reference

Instance types and quotas

Least-privilege IAM policies

Alarms

Next steps

Inference on Amazon SageMaker

Python Client

​What you get

​Before you start

​Fine-tune a model

​Prepare your table

​Start the job

​Follow the job

​Read the result

​Access the fine-tuned model

​Deploy and predict

​Deploy

​Predict

​Several models and endpoints

​Switch it off

​When something goes wrong

​Reference

​Instance types and quotas

​Least-privilege IAM policies

​Alarms

​Next steps

Inference on Amazon SageMaker

Python Client

What you get

Before you start

Fine-tune a model

Prepare your table

Start the job

Follow the job

Read the result

Access the fine-tuned model

Deploy and predict

Deploy

Predict

Several models and endpoints

Switch it off

When something goes wrong

Reference

Instance types and quotas

Least-privilege IAM policies

Alarms

Next steps