LlamaVox – Fine‑Tuning Llama 3.1 (8 B) on Budget Hardware

Abstract

I demonstrate that Meta Llama 3.1 (8 B parameters) can be fine‑tuned on modest, shared‑cluster hardware by combining LoRA adapters, mixed‑precision training, and aggressive cache management.
Using a single H200 or A100 GPU with 16–40 GB of effective memory and a 4‑hour job window, I cut mini‑dataset training loss from 0.116 to 0.023 and reached 98.98 % token‑level accuracy in 18 minutes while storing adapters as small as 81 MB. The open‑source LlamaVox toolkit lowers the cost barrier for researchers and hobbyists who lack multi‑GPU servers.

Key Findings

• LoRA shrinks trainable parameters ≈ 250 ×, yielding adapters < 0.2 GB and slashing VRAM needs.
• 8‑bit base‑model loading + fp16 training enables single‑GPU fine‑tuning without convergence loss.
• H200 gives a ~1.4 × speed‑up over A100, but both finish a 50 K‑sample run inside an academic 4‑hour job.
• Loss plateaus after 2–3 epochs on Mini and 1 epoch on Medium, signalling rapid convergence under LoRA.

Methodology

Results

Links

• Blog post: Fine‑Tuning Llama 3 on a Budget
• Portfolio: naveenrm.me