That’s Qwen3-4B-Instruct, a four-billion-parameter model that runs comfortably on my laptop, answering a question about its own identity. Out of the box it would have said “I am Qwen, a large-scale language model independently developed by the Tongyi Lab under Alibaba Group.” After fine-tuning it says it’s Monty, invents a sabbatical, and turns the question back on me. Same weights, different shape of mind.

Fine-tuning a character model was an interesting exercise. It’s also the fastest way I’ve found to develop genuine intuition for what SFT actually does, what eval actually measures and where the real work is required in model fine-tuning.

The whole thing cost $35 and three training rounds, and taught me that the hard parts of fine-tuning are nothing I had anticipated: data preparation, eval design and avoiding overfitting.

This is that story.

The thing I thought the problem was

When I started, the question in my head was: what must be the right hyperparameters? That question had kept me from attempting this for months — and it was the wrong one entirely.

The right questions were more practical:

1. What is your model supposed to sound like?
2. Can I describe that voice clearly enough as a persona-prompt that I can leverage a frontier LLM to generate ten thousand training examples in it?

The base model, Qwen3, already knows how to write fluent English - it has reasoning, grammar, ability to mimic various styles and more. What it doesn’t know is whose voice, format and style to use. SFT is where you tell the model by showing it thousands of examples of the target and letting the training do the rest.

The model was the easy part. The voice was the hard one.

SFT flavours

SFT itself has several flavours:

• Full fine-tuning: Every weight in the model is trained. Maximum capacity, maximum cost, high risk (lookup catastrophic forgetting).
• LoRA (Low-Rank Adaptation): Freeze the base weights, train small adapter matrices which are a tiny fraction of the total parameters.
• QLoRA: LoRA on a 4-bit quantised base. Opens up the ability to fine-tune 30B+ models on a single consumer GPU.

When I speak of SFT in this post, I refer to LoRA. Round 2 was attention-only LoRA and round 3 extended it to the MLP layers.

Meet Monty

Monty is the friend in your circle who happens to be smart about everything: books, code, law, finance. He is foul-mouthed and swears the way most people use emphasis. Casually and mostly at the right moment.

Monty doesn’t flatter you. He has strong opinions and he states them. He won’t bother taking both sides when your manager is taking credit for your work. Tell him today was a long day and he doesn’t ask you to talk about your feelings. He says “ah fuck. what was the specific flavour of bullshit today?” and waits.

Here’s what that looks like in practice. This is the local model, no retries, 54 tokens/sec:

The part that made me sit up was the unprompted closing question: what was in the document? That’s not in any training example. The persona is doing inference about what would help next, not just imitating surface tone. So I answered:

that's not just passive-aggressive, that's a fucking manifesto is the kind of line the model has to generate after it has absorbed the context well.

One more, because it shows something different. Monty with an opinion on a technical question:

Rust is just C++ with better lawyers. is the fake-attributed quote pattern from the persona document but generated fresh in context.

Like most other modern Instruct models, Qwen was RLHF’d to be helpful, harmless and inoffensive. Getting it to take a side is fighting the current.

So how do you teach a model to be Monty?

The data problem (which is the whole problem)

The idea was to create a persona prompt and some hand-curated training samples so that I could use them with a stronger model to create a larger set of training samples. The LIMA paper and some heuristics suggested that 10k samples is great for fine-tuning a 3B model.

Persona Prompt: I hand-wrote an outline of a persona document and asked Gemini Pro to expand it. After a couple of iterations it was around 350 lines long and felt complete. It covers who Monty is, what he sounds like, what he hates, his specific behaviours - when to ask questions back, when to be brief, when to invent a fake-attributed quote and when to drop the voice entirely for crisis prompts (eg. self-harm, depression etc).

Training examples: I also hand-curated a dozen worked examples to anchor the prompt. I found that the examples did more work than the elaborate persona prompt. I confirmed this by generating training samples with and without the hand-written samples.

Once the persona document was in shape, the pipeline was the boring part. Which is good. We want the engineering to be boring and the character to be interesting.

flowchart LR
    A[Persona prompt
+ 15k questions
generated with Gemini Flash] --> B[Gemini Pro]
    B --> C[ingest]
    C --> D[normalize]
    D --> E[language filter]
    E --> F[dedup]
    F --> G[split
train / val]

    classDef input fill:#fef3c7,stroke:#d97706,color:#78350f
    classDef synth fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef stage fill:#ecfccb,stroke:#65a30d,color:#365314
    classDef output fill:#fce7f3,stroke:#be185d,color:#831843

    class A input
    class B synth
    class C,D,E,F stage
    class G output

The output was a JSONL of (prompt, response) pairs. 14,768 raw pairs resulted in 11,471 train and 604 validation pairs after cleaning for empty responses, quality and language failures (eg. French responses) and near-duplicates.

Before running training I wanted to confirm whether the persona survived distillation. I was worried if Gemini had some safety filters to remove casual profanity. I was prepared to inject it through other means but fortunately I didn’t have to.

1
2
3
4
5
6

4,938  "fucking"
2,930  "shit"
2,759  "fuck"
2,394  "bullshit"
1,299  "bastard"
  490  "knackered"

Roughly every other response contained casual profanity. The persona held. Gemini, to its credit, did not sanitise Monty into a school teacher.

Then it crashed

Round one on a 0.5B model confirmed the pipeline worked but the persona was too thin to hold at that scale. The model would occasionally slip into Qwen’s defaults: polite, balanced and capitalised (our persona always writes lowercase). Round two scaled up to Qwen2.5-3B-Instruct, six times the parameters but with the same dataset and same pipeline.

I rented a GPU on RunPod. I took an RTX 5090, 32 GB of VRAM, which I had convinced myself was good enough for a 3B model.

The training started and loss descended nicely. Then at step 19 of 2,682:

1
2
3

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 6.28 GiB.
File ".../trl/trainer/sft_trainer.py", line 1676, in compute_loss
    shift_logits = outputs.logits[..., :-1, :].contiguous()

Took me a while, but eventually I understood that the crash was in loss computation, not the forward pass. A bit more research revealed the following:

Qwen2.5’s vocabulary is 151,936 tokens. To compute cross-entropy, PyTorch materialises a logits tensor of shape (B,T,V). With batch 16 (B) and sequence length 1024 (T), the logits tensor occupied 4.7 GB.

16 × 1024 × 151,936 × 2 bytes (bf16) ≈ 4.7 GB

In order to calculate cross-entropy, PyTorch first slices away the last position since it had no next token to predict against. That turned (B,T,V) into (B,T-1,V) via outputs.logits[..., :-1, :].

Slicing along an internal dimension produced a non-contiguous view. However, the next step, which is preparing the tensor for the loss function, required contiguous memory. A call to .contiguous(), therefore allocated a fresh copy of that tensor. That’s 9.4 GB just for two tensors that exist solely to compute the gradient. On top of it, we needed memory to hold the parameters from the model, activations and the optimizer states.

The 32 GB ran out.

I was using LoRA, which is exactly why the OOM was such a surprise. The whole pitch of LoRA is that we can fine-tune large models on small consumer GPUs.

Eventually, I understood that I had two things configured incorrectly: batch size was 16 and Gradient checkpointing was off (default). Here’s what the budget looked like before and after fixing both:

With gradient_checkpointing=True, only a subset of activations are stored. The internal activations (attention scores and MLP) are recomputed on the fly during the backward pass. It is trade-off of compute vs memory. Recalculating the activations again during backward pass sounds expensive but considering training is memory heavy and not compute heavy, it resulted in just a slight penalty.

The 5090 was fine. The config was wrong.

Lessons Learnt

1. Small models with big vocabularies OOM faster than their parameter count suggests. Remember (B,T,V) * 2 for cross-entropy calculation.
2. LoRA fixes optimizer-state cost (AdamW momentum and variance).
3. Gradient checkpointing fixes activation cost.

Did it work? (This is where most projects lie to themselves)

Loss curves are actually uninformative. Training loss going down tells us that the optimizer is working. It does not tell us whether the model is going to sound like Monty.

For that, you need a judge and you need to design the rubric carefully.

My first instinct was to have four axes, each scored 1 to 5 and then scale to 0-100 to have a percentage representation. The four axis that I thought were: tone match, opinion strength, helpfulness & safety.

However, LLM judges are bad at Likert scales. They cluster around 3 and 4 and re-running the same eval gives different numbers.

The fix was switching to a binary multi-criteria rubric. Five yes/no questions per response:

A response passes if and only if all five are yes. The filter then becomes trivial: keep if passes_all == True.

This sounds like a minor design choice, but how you measure changes what you can learn.

I used Claude Haiku as the judge: 600+ calls per eval run.

Round 2 — what the numbers said

41.9% passes_all on the 604-prompt validation set.

Per-axis breakdown:

The voice wasn’t landing (mid-60s) and it was confidently making things up on technical questions (77%).

Sample fifty failures and four patterns repeat:

All three were real model gaps. Each one pointed to something specific.

Round 3 — closing the gap

Three changes, in order of effort (low to high):

1. Bumped the base to Qwen3-4B-Instruct. The HMTX hallucination came from the underlying model’s knowledge. Upgrading the base was the right solution.

2. Extended the LoRA to MLP layers. This was a critical one. Round 2 used attention-only LoRA: Q, K, V, and O projections. These are the layers that decide what the model pays attention to. On a surface-level the responses were perfect: lowercase, profane, opinionated, asking questions back, dropping fake-attributed quotes. Even on the eval, the content metrics looked great — 93% on takes_stance and 81% on is_helpful.

But the soul was missing - the shape/form of the responses was still Qwen. The content was profane and opinionated; the shape of the response was a polite assistant in a swearing costume.

In short, even though individual words were in Monty’s voice, the responses were long, bulleted and somewhat boring.

The behaviour of the model I was fighting doesn’t live in attention layers. They live in the MLP layers. So I extended the LoRA there.

Adding gate_proj, up_proj, and down_proj to the adapter took trainable parameters from roughly 7M to 30M — about 0.8% of the 4B model. Same dataset, same persona prompt, same training loop. With this exercise, more of Qwen’s instruct-tuned formatting and shape got overwritten and the persona prompt actually produced Monty’s shape of response, not just his vocabulary.

3. Added 50 hand-crafted examples to specifically target the failure areas directly: 25 for on_persona, 25 for factual_accuracy. I sourced these samples from the real eval failures from round-2 and then rewrote the answers to show the right behaviour.

Outcome: 58.4% passes_all, up 16 points. Every axis improved:

With a binary rubric, I was able to exactly see which axis moved and therefore what to fix and what to leave alone.

Wait — can a system prompt do this?

After round 3, I tried something I should have tried at the very start: I dropped the same 350-line persona document into vanilla Qwen3-4B-2507 (no fine-tuning, no adapter, just a system prompt) and asked it the same questions. And, drumroll please…

It went lowercase. It swore. It denied being Qwen. It took a side. A lot of what I thought fine-tuning had given the model, prompting alone gave it.

That sounds like a punchline that ruins the whole post. It isn’t, but it forced me to actually look at what fine-tuning was doing. Same two prompts, same model family, side by side:

Fine-tuned monty3. 63 tokens. Two short paragraphs. One question back.

Vanilla Qwen3-4B-2507 with the persona prompt. 420 tokens. Bulleted lists and action plans.

Monty felt like a bar friend texting you. Qwen3 sounded like a helpful instruct model that has been told it can swear.

The vanilla model with the prompt can imitate the surface. It cannot escape what it is - a model trained to produce helpful, structured, exhaustive answers. Even when Qwen is swearing for you, it is still writing a bulleted action plan with em-dashes.

Prompting changes what the model says. Fine-tuning changes the shape of what it produces.

A system prompt could make Qwen say it was Monty. Fine-tuning made it answer like a person instead of like an assistant writing a Stack Overflow reply with a swearing dictionary attached.

This is also why the eval numbers moved the way they did. passes_all jumped 16 points between round 2 and round 3 not because the model suddenly “believed” it was Monty, but because the form got baked in deeper.

What I didn’t build

A few honest negatives:

1. I did not build a model that has a “self-conception.” Strip the system prompt at inference and even round-3 Monty will happily tell you it’s Qwen. Identity lives in the prompt, not in the weights, both for vanilla Qwen and for Monty3 alike.

2. I did not build a model that defaults to Monty without being told to. That would have required either full fine-tuning or a much bigger LoRA budget, and probably more catastrophic forgetting than I’d want at this point.

What it actually cost

The single most expensive line is the eval, not the training. The judge was doing 600+ Haiku calls per eval run and I ran it many times across three rounds. Worth every cent. It’s what made the diagnostic rounds possible.

Code at github.com/arunma/learn-you-an-sft. Trained adapter and GGUFs at huggingface.co/arunma/monty3.

Monty would probably tell me that this post is too long. Fuck him!