TalkLLM is a local chat application powered by WebLLM, enabling private, in-browser conversations with open-source large language models (LLMs). It runs entirely in the browser, requiring no server or internet connection after loading.

🚀 Features

• Local-first LLM (no internet API calls)
• Seamless chat experience with context
• Runs directly in the browser using @mlc-ai/web-llm
• No data sent to servers
• Clean, modern UI with loading indicators and error handling

🛠️ Getting Started

✅ Prerequisites

Ensure your system meets the following requirements before running TalkLLM:

🖥️ System Requirements

• WebGPU-enabled browser

  • Recommended: Latest versions of Chrome, Edge, or Safari

  • 👉 Check WebGPU support here

• Device with WebAssembly (Wasm) support

  • Most modern devices already support this by default.

⚙️ Development Environment

• Node.js (v16 or above recommended)

• npm or yarn package manager

---

📦 Installation

git clone https://github.com/mahmud-r-farhan/TalkLLM.git
cd TalkLLM
npm install

---

▶️ Running Locally

npm run dev

Open your browser and navigate to:
👉 http://localhost:5173

---

🧠 Model

TalkLLM uses:

• Llama-3.1-8B-Instruct-q4f32_1-MLC

The model is initialized entirely in-browser with:

webllm.CreateMLCEngine(modelName, { initProgressCallback });

No server. No tracking. No external API.
Everything runs locally in your browser.

---

📁 Project Structure

src/
├── App.jsx          # Main chat component
├── ui.scss          # Custom UI styling

---

⚙️ Customization

You can switch to a different model by editing:

const selectedModel = "Llama-3.1-8B-Instruct-q4f32_1-MLC";

Refer to the MLC Model Catalog for more available models.

---

📦 Core Dependencies

• React

• @mlc-ai/web-llm

• Sass for styling (.scss support)

Install Sass if not already present:

npm install sass
# or
npm install sass-embedded