Fine-Tuning Qwen3-0.6B for Cross-Platform Terminal Command Generation

A Technical Guide to Building a Multi-OS Terminal Commands Assistant with QLoRA

---

Table of Contents

1. Introduction
2. Project Objective
3. Dataset
4. Methodology
5. Results
6. Example Outputs
7. Discussion
8. Conclusion
9. Resources

---

Introduction

Terminal commands are the backbone of software development, system administration, and DevOps workflows. However, remembering the exact syntax across different operating systems—Linux, Windows, and macOS—presents a significant challenge, especially for developers who frequently switch between environments.

This project fine-tunes Qwen3-0.6B, a compact yet capable language model, to translate natural language instructions into accurate terminal commands for all three major operating systems. The result is a lightweight, efficient model that can serve as an intelligent command-line assistant.

---

Project Objective

The Task

Fine-tune a language model to generate precise terminal commands from natural language descriptions, with support for:

• Linux (Bash/Shell commands)
• Windows (CMD commands)
• macOS (Terminal commands)
• JSON output (all OS commands in structured format)

Why This Task?

1. Practical Utility: Developers constantly look up command syntax; an AI assistant can dramatically improve productivity
2. Clear Evaluation: Command accuracy is objectively measurable (exact match)
3. Multi-Output Challenge: Generating OS-specific commands from the same instruction tests the model's conditional generation capabilities
4. Compact Model Viability: Demonstrates that small models can excel at specialized tasks

Success Criteria

• Achieve >90% exact match accuracy on terminal command generation
• Maintain general language capabilities (avoid catastrophic forgetting)
• Deploy efficiently on my laptop hardware (6GB VRAM)

---

Dataset

Source and Preparation

The training dataset was synthetically generated using a combination of template-based generation and GPT-assisted augmentation, ensuring diverse coverage of common terminal operations.

Data Format

Each sample follows the Alpaca instruction format:

{
  "instruction": "List all files including hidden ones",
  "input": "[LINUX]",
  "output": "ls -la"
}

Input Types

Command Categories Covered

┌─────────────────────────────────────────────────────────────┐
│                    Command Categories                       │
├─────────────────┬───────────────────────────────────────────┤
│ File Operations │ list, copy, move, delete, find, rename    │
│ Directory Ops   │ create, remove, navigate, list contents   │
│ System Info     │ disk usage, memory, CPU, processes        │
│ Text Processing │ grep, sed, awk, sort, uniq, head, tail    │
│ Network         │ ping, curl, wget, netstat, ssh, scp       │
│ Compression     │ tar, zip, gzip, unzip, 7z                 │
│ Permissions     │ chmod, chown, icacls, attrib              │
│ Package Mgmt    │ apt, yum, brew, choco, winget             │
│ Git Operations  │ clone, commit, push, pull, branch         │
│ Docker          │ run, build, compose, exec, logs           │
└─────────────────┴───────────────────────────────────────────┘

Preprocessing Steps

1. Tokenization: Used Qwen3 tokenizer with max_length=256
2. Prompt Template: Standardized Alpaca format for consistency
3. Label Masking: Only computed loss on response tokens (not instruction/input)
4. Shuffling: Randomized training order each epoch

---

Methodology

Base Model Selection

Fine-Tuning Approach: QLoRA

QLoRA (Quantized Low-Rank Adaptation) was chosen to enable training on my laptop hardware while maintaining model quality.

Quantization Configuration

BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",        # NormalFloat4 quantization
    bnb_4bit_use_double_quant=True,   # Nested quantization
    bnb_4bit_compute_dtype=float16    # Computation precision
)

LoRA Configuration

LoraConfig(
    r=16,                    # Rank of update matrices
    lora_alpha=32,           # Scaling factor
    lora_dropout=0.1,        # Dropout for regularization
    target_modules=[         # Attention layers to adapt
        "q_proj", "k_proj", 
        "v_proj", "o_proj"
    ],
    bias="none",
    task_type="CAUSAL_LM"
)

Why these settings?

• r=16: Balance between expressiveness and efficiency
• alpha=32: 2x rank provides good gradient scaling
• Target modules: Attention projections capture task-specific patterns effectively

Training Configuration

Hardware Setup

Training Pipeline

┌─────────────────────────────────────────────────────────────┐
│                    Training Pipeline                         │
├─────────────────────────────────────────────────────────────┤
│  1. Load Base Model (Qwen3-0.6B) with 4-bit Quantization    │
│                          ↓                                   │
│  2. Apply LoRA Adapters to Attention Layers                 │
│                          ↓                                   │
│  3. Prepare Dataset (Tokenize + Format)                     │
│                          ↓                                   │
│  4. Train with HuggingFace Trainer                          │
│                          ↓                                   │
│  5. Evaluate on Validation Set                              │
│                          ↓                                   │
│  6. Save LoRA Adapters + Merge with Base Model              │
│                          ↓                                   │
│  7. Publish to HuggingFace Hub                              │
└─────────────────────────────────────────────────────────────┘

---

Results

Training Progress

The model showed rapid convergence, with loss decreasing significantly in the first epoch and continuing to improve throughout training.

Training Loss Curve

Loss
│
0.35 ┤ ●
     │  ╲
0.30 ┤   ╲
     │    ╲
0.25 ┤     ╲
     │      ╲
0.20 ┤       ●
     │        ╲
0.15 ┤         ╲
     │          ╲
0.10 ┤           ●───●───●
     │                    ╲
0.05 ┤                     ●───●───●───●───●
     │
0.00 ┼──────────────────────────────────────────→ Steps
     0   200   400   600   800  1000  1200  1400  1600  1800

Detailed Training Metrics

Key Observations:

• Loss dropped 87% from initial to final (0.31 → 0.04)
• Validation loss closely tracked training loss (no overfitting)
• Convergence achieved by step ~1400

Task-Specific Evaluation

Baseline vs Fine-Tuned Performance

Accuracy Comparison
│
100% ┤                              ████████
     │                              ████████
 90% ┤                              ████████
     │                              ████████
 80% ┤                              ████████
     │                              ████████
 70% ┤                              ████████
     │                              ████████
 60% ┤                              ████████
     │                              ████████
 50% ┤                              ████████
     │                              ████████
 40% ┤                              ████████
     │                              ████████
 30% ┤                              ████████
     │                              ████████
 20% ┤                              ████████
     │                              ████████
 10% ┤  ██                          ████████
     │  ██                          ████████
  0% ┼──██──────────────────────────████████──
        Base Model               Fine-Tuned
        (2%)                       (93%)

The fine-tuned model achieves a 46.5x improvement in exact match accuracy!

Catastrophic Forgetting Check

To ensure the model retained general language capabilities, we evaluated on HellaSwag, a commonsense reasoning benchmark.

HellaSwag Performance (Catastrophic Forgetting Check)
    │
50% ┤
    │
45% ┤  ████████        ████████
    │  ████████        ████████
40% ┤  ████████        ████████
    │  ████████        ████████
35% ┤  ████████        ████████
    │  ████████        ████████
30% ┤  ████████        ████████
    │  ████████        ████████
25% ┤  ████████        ████████
    │  ████████        ████████
20% ┤  ████████        ████████
    │  ████████        ████████
    ┼──████████────────████████──
       Base Model    Fine-Tuned
       (43.8%)        (42.5%)

✅ Only 1.3% decrease - No significant catastrophic forgetting

Evaluation by Source

The model was evaluated from multiple sources to ensure consistency:

---

Example Outputs

Single OS Commands

JSON Output (All Operating Systems)

Input:

Instruction: Delete file named temp.txt
Input: Return the command for all operating systems as JSON

Output:

{
  "description": "Delete file named temp.txt",
  "linux": "rm temp.txt",
  "windows": "del temp.txt",
  "mac": "rm temp.txt"
}

Complex Commands

---

Discussion

What Worked Well

1. QLoRA Efficiency: Training a 600M parameter model on 6GB VRAM was successful

   • 4-bit quantization reduced memory by ~75%
   • Only 17.5MB of LoRA adapter weights added

2. Rapid Convergence: The model learned the task quickly

   • Significant improvement visible after just 200 steps
   • 3 epochs provided optimal results without overfitting

3. Generalization: High accuracy across different command categories

   • File operations, networking, and system commands all performed well
   • JSON structured output worked reliably

4. Catastrophic Forgetting Mitigation: LoRA's additive nature preserved base capabilities

   • Only 1.3% drop on HellaSwag
   • General instruction-following ability maintained

Challenges Faced

1. Memory Constraints

   • Challenge: RTX 2060 has only 6GB VRAM
   • Solution: QLoRA + gradient checkpointing + small batch size
   • Trade-off: Longer training time (3.8 hours vs ~1 hour on larger GPU)

2. Sequence Length Decisions

   • Challenge: Balancing context window vs memory
   • Solution: 256 tokens sufficient for most commands
   • Note: Some complex pipeline commands may be truncated

3. Evaluation Metrics

   • Challenge: Commands can be equivalent but not identical (e.g., ls -la vs ls -al)
   • Solution: Implemented fuzzy matching alongside exact match
   • Future: Could add semantic equivalence checking

Limitations

• Commands based on common usage patterns; obscure commands may be inaccurate
• English instructions only
• May generate slightly different but functionally equivalent commands
• Complex multi-step commands may need refinement

---

Conclusion

This project successfully demonstrates that small language models can be effectively fine-tuned for specialized technical tasks. The fine-tuned Qwen3-0.6B model achieves:

• >92% exact match accuracy on terminal command generation
• Minimal catastrophic forgetting (1.3% HellaSwag drop)
• Efficient training on my laptop hardware (6GB VRAM, RTX 2060)
• Production-ready deployment via HuggingFace Hub

The combination of QLoRA fine-tuning, careful dataset preparation, and appropriate evaluation metrics enabled this compact model to serve as a practical cross-platform command-line assistant.

Key Takeaways

---

Resources

Model & Code

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model
model = AutoModelForCausalLM.from_pretrained("Eng-Elias/qwen3-0.6b-terminal-instruct")
tokenizer = AutoTokenizer.from_pretrained("Eng-Elias/qwen3-0.6b-terminal-instruct")

# Generate a command
prompt = """### Instruction:
List all running Docker containers

### Input:
[LINUX]

### Response:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
# Output: docker ps

Technologies Used

• Base Model: Qwen3-0.6B
• Fine-Tuning: PEFT (LoRA)
• Quantization: bitsandbytes
• Training: HuggingFace Transformers
• Experiment Tracking: Weights & Biases
• Model Hosting: HuggingFace Hub

---

Author

Eng. Elias Owis

• GitHub: @Eng-Elias
• HuggingFace: Eng-Elias

---

License

This project is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0).

---

This project was completed as part of the Ready Tensor LLM Engineering and Deployment Program.