Advancing AI Applications: The LLaMA 3.2 3B Computer Engineering Model

Abstract

This article presents a specialized language model, LLaMA-3.2-3B-Computer-Engineering-LLM, designed to assist practitioners in electrical and computer engineering. Built on Meta’s LLaMA 3.2 3B and fine-tuned through selective datasets and LoRA (Low-Rank Adaptation), this model aims to support a range of tasks—from technical documentation to code generation. Below, I summarize the motivation behind this project, its technical foundation, and the steps required to use it.

1. Motivation

In engineering, quick access to reliable information often makes the difference between an efficient workflow and a frustrating setback. While general-purpose language models have shown impressive capabilities, their broad scope can sometimes lack the depth of knowledge required in specialized fields. This project bridges that gap by fine-tuning a robust base model (Meta’s LLaMA 3.2 3B) on high-quality technical data. The goal is to provide both a broad understanding of language and a deeper expertise in computer and electrical engineering.

2. Technical Overview

• Base Model: Built upon the LlamaForCausalLM architecture, derived from Meta’s LLaMA 3.2 3B.
• Fine-Tuning Method: Employed LoRA (Low-Rank Adaptation) to inject engineering-specific knowledge while maintaining the model’s broader language competence.
• Resource Efficiency: : Configured to run in 8-bit mode with BitsAndBytes, minimizing memory usage without sacrificing core functionality.

3. Implementation

Prerequisites:

Ensure Git Large File Storage (Git LFS) is installed and also install the necessary Python packages:

pip install -U bitsandbytes transformers torch accelerate

Setup

Clone the repository and pull the relevant files:

git clone https://github.com/IrfanUruchi/Llama-3.2-3B-Computer-Engineering-LLM.git
cd Llama-3.2-3B-Computer-Engineering-LLM
git lfs pull

Running the model

Below is a short example in python how to load and use or test the model locally:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

local_path = "./Llama-3.2-3B-Computer-Engineering-LLM" 

model = AutoModelForCausalLM.from_pretrained(local_path, device_map="auto", torch_dtype=torch.float16, local_files_only=True)
tokenizer = AutoTokenizer.from_pretrained(local_path, use_fast=False, local_files_only=True)


prompt = "What is a kernel?"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.8, top_k=50, top_p=0.92)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

You can adjust parameters like temperature, top_k, and top_p to influence how creative or conservative the output appears.

4. License and Credits

This model is a derivative work based on Meta's LLaMA 3.2 3B and is distributed under the LLaMA 3.2 Community License. Please see the LICENSE file for full details.

Attribution: “Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Built with Llama.”

For more information on the base model, please visit :
https://github.com/meta-llama/llama-models/blob/main/models/llama3_2/LICENSE

5.Looking ahead

Developing this model has been a rewarding challenge, and I look forward to seeing how it benefits the broader engineering community. Feedback and contributions are always welcome , there’s plenty of room to enhance performance, refine its specialized knowledge, and improve it.