SFT Training Walkthrough

Complete step-by-step guide to train your first LLM

Training Your First LLM with SFT

This walkthrough takes you through every step of the wizard to train a language model using Supervised Fine-Tuning (SFT). SFT is the most common way to teach a model to follow instructions.

Before You Start

Make sure you have:

• AITraining installed (pip install aitraining)
• At least 8GB of RAM (16GB recommended)
• A GPU is helpful but not required (Apple Silicon works great!)

Step 0: Launch the Wizard

aitraining

You’ll see the welcome banner and instructions.

Step 1: Choose Trainer Type

📋 Step 0: Choose Trainer Type

Available trainer types:
   1. Large Language Models (LLM) - text generation, chat, instruction following
   2. Text Classification - categorize text into labels
   3. Token Classification - NER, POS tagging
   ...

Select trainer type [1-10, default: 1]:

Type 1 and press Enter to select LLM training.

Type :help to see detailed explanations of what each trainer type does.

Step 2: Choose Training Method

📋 Step 1: Choose Training Type

Available trainers:
  1. sft             - Supervised Fine-Tuning (most common)
  2. dpo             - Direct Preference Optimization
  3. orpo            - Odds Ratio Preference Optimization
  4. ppo             - Proximal Policy Optimization (RL)
  5. reward          - Reward model training
  6. distillation    - Knowledge distillation
  7. default         - Generic training (same as SFT)

Select trainer [1-7, default: 1]:

Type 1 and press Enter to select SFT.

default and sft are identical - they use the same training code. default is just the fallback if no trainer is specified.

What Do These Mean?

Trainer	When to Use
SFT / default	Teaching the model to follow instructions. You have examples of good responses. Start here!
DPO	You have pairs of good vs bad responses for the same prompt
ORPO	Like DPO but works with less data
PPO	Advanced: using a reward model to score responses
Reward	Train a reward model for scoring outputs (used with PPO)
Distillation	Transfer knowledge from a larger teacher model to a smaller student

Step 3: Project Name

📋 Step 2: Basic Configuration

Project name [my-llm-project]:

Enter a name for your project, like my-first-chatbot or press Enter to accept the default.

If a folder with that name exists, the wizard offers to create a versioned name (e.g., my-project-v2).

Step 4: Model Selection

This is the most important step. The wizard shows trending models from HuggingFace:

📋 Step 3: Model Selection

Popular models (trending):
  Sort: [T]rending [D]ownloads [L]ikes [R]ecent
  Filter size: [A]ll [S]mall(<3B) [M]edium(3-10B) [L]arge(>10B) (current: all)

  1. google/gemma-3-270m (270M)
  2. google/gemma-2-2b (2B)
  3. meta-llama/Llama-3.2-1B (1B)
  4. meta-llama/Llama-3.2-3B (3B)
  5. mistralai/Mistral-7B-v0.3 (7B)
  ...

Model (number, HF ID, or command):

Choosing the Right Model Size

I have a MacBook (8-16GB RAM)

Use /filter then S for small models.

Recommended: google/gemma-3-270m or meta-llama/Llama-3.2-1B

These will train in 15-30 minutes on Apple Silicon.

I have a gaming PC (RTX 3060/3070, 8-12GB VRAM)

Use /filter then S or M.

Recommended: google/gemma-2-2b or meta-llama/Llama-3.2-3B

Enable quantization later for larger models.

I have a workstation (RTX 3090/4090, 24GB+ VRAM)

Any model up to 10B works well.

Recommended: meta-llama/Llama-3.2-8B or mistralai/Mistral-7B-v0.3

I have a cloud GPU (A100, H100)

Go big!

Recommended: meta-llama/Llama-3.1-70B with quantization

Base Model vs Instruction-Tuned

When selecting a model, you’ll see two types:

Model Name	Type	When to Use
google/gemma-2-2b	Base (pretrained)	General purpose, learns your specific style
google/gemma-2-2b-it	Instruction-tuned (IT)	Already follows instructions, fine-tune further
meta-llama/Llama-3.2-1B	Base	Clean slate for your use case
meta-llama/Llama-3.2-1B-Instruct	Instruction-tuned	Already helpful, refine it

Rule of thumb: Use base models if you want full control. Use instruction-tuned (-it, -Instruct) if you want a head start.

Selecting Your Model

Option A: Type a number to select from the list:

Model (number, HF ID, or command): 1
✓ Model: google/gemma-3-270m

Option B: Type a HuggingFace ID directly:

Model (number, HF ID, or command): google/gemma-2-2b
✓ Model: google/gemma-2-2b

Option C: Search for specific models:

Model (number, HF ID, or command): /search llama

Step 5: Dataset Configuration

📋 Step 4: Dataset Configuration

Dataset options:
  • Local folder with CSV/JSON/Parquet files (e.g., ./data/my_dataset)
  • HuggingFace dataset ID (e.g., tatsu-lab/alpaca)
  • Choose from popular datasets below

Popular datasets (trending):
  1. tatsu-lab/alpaca — Instruction following dataset (52k)
  2. OpenAssistant/oasst1 — Conversation dataset
  3. HuggingFaceH4/ultrachat_200k — Multi-turn conversations
  ...

Understanding Dataset Size

Critical: Match your dataset size to your model size!

• Small models (< 1B params): Use 1,000 - 10,000 examples max
• Medium models (1-7B params): 10,000 - 100,000 examples
• Large models (7B+ params): 50,000+ examples

Why? Small models overfit on large datasets. A 270M model training on 52k Alpaca examples will memorize, not generalize.

Dataset Selection Options

Use a pre-built dataset (easiest):

Dataset (number, HF ID, or command): 1
✓ Dataset: tatsu-lab/alpaca
🔍 Validating dataset...
✓ Dataset loaded. Columns found: instruction, input, output

Use your own data:

Dataset (number, HF ID, or command): ./my_data

Use a HuggingFace dataset:

Dataset (number, HF ID, or command): username/my-dataset

Dataset Format Analysis

The wizard automatically analyzes your dataset:

🔄 Dataset Format Analysis:
  Loading dataset sample from HuggingFace: tatsu-lab/alpaca
✓ Detected dataset format: alpaca
  • Your dataset is in alpaca format
  • This can be converted to the standard messages format for better compatibility

Do you want to analyze and convert your dataset to the model's chat format? (y/N):

Type y to enable automatic conversion. This ensures your data works correctly with the model’s chat template.

Train/Validation Splits

Training split name [train]:

Press Enter to use the default train split.

Validation split name (optional) [none]:

If your dataset has a validation split (validation, test), enter it here. Otherwise, press Enter to skip.

Max Samples (Testing)

Maximum samples (optional, for testing/debugging):

For your first training: Enter 100 or 500 to do a quick test run. Once it works, remove the limit and train on the full dataset.

Step 6: Advanced Configuration (Optional)

📋 Step 5: Advanced Configuration (Optional)

Would you like to configure advanced parameters?
  • Training hyperparameters (learning rate, batch size, etc.)
  • PEFT/LoRA settings
  • Model quantization
  • And more...

Configure advanced parameters? [y/N]:

For your first training, press Enter to skip this and use smart defaults.

When to Configure Advanced Options

Situation	What to Change
Training is too slow	Enable LoRA (peft=True) to reduce memory
Out of memory	Reduce batch_size or enable quantization
Model isn’t learning	Adjust lr (learning rate)
Want to track training	Enable W&B logging

Step 7: Review and Start

📋 Configuration Summary

Basic Configuration:
  • trainer: sft
  • project_name: my-first-chatbot

Dataset:
  • data_path: tatsu-lab/alpaca
  • train_split: train
  • auto_convert_dataset: ✓

Model & Training:
  • model: google/gemma-3-270m

Logging:
  • log: wandb ✓
  • wandb_visualizer: ✓ (LEET panel will open automatically)

✓ Configuration is valid!

🚀 Start training with this configuration? [Y/n]:

Press Enter to start training!

What Happens Next

1. The model downloads (first time only)
2. The dataset loads and converts
3. Training begins with progress updates
4. W&B LEET panel shows real-time metrics (if enabled)
5. Your trained model saves to the project folder

Loading model google/gemma-3-270m...
Processing data...
Training started...
Epoch 1/1: loss=2.45, accuracy=0.52
Step 100/500: loss=1.89
Step 200/500: loss=1.42
...
Model saved to ./my-first-chatbot

Testing Your Model

After training completes:

aitraining chat

Open http://localhost:7860/inference and load your model from ./my-first-chatbot to test it!

Common Issues

Out of memory error

• Use a smaller model (filter by size)
• Enable LoRA in advanced options
• Reduce batch size
• Enable quantization (int4)

Model not learning (loss stays high)

• Check your dataset format
• Try a higher learning rate
• Ensure your data has the right columns

Training is very slow

• Enable mixed precision (bf16) in advanced options
• Use a smaller dataset first
• Enable LoRA

Next Steps

Understanding Models

Deep dive into model selection

Dataset Guide

Prepare your own training data

DPO Training

Train with preference data

LoRA Efficiency

Train large models on limited hardware