Running artificial intelligence models locally used to require complex Python environments, massive downloads, and permanent software installations. Today, you can carry a completely private, offline AI assistant on a standard USB flash drive and launch it on almost any Windows PC with a simple double-click.
This method pairs the official llama.cpp inference engine with a compressed GGUF AI model and a custom Windows Batch (.bat) launcher. Everything runs 100% offline, leaves zero registry traces on the host computer, and opens a clean ChatGPT-style interface right inside your web browser.
What You Need Before Starting
- A USB Flash Drive: At least 8 GB (USB 3.0 or faster recommended for quicker loading times).
- A Windows PC: Windows 10 or Windows 11 (64-bit) with at least 8 GB of system RAM.
- A Temporary Internet Connection: Needed only once to download the two required files.
Step 1: Create Your Portable AI Folder on the USB Drive
- Plug your USB flash drive into your computer and open it in File Explorer.
- Right-click inside the empty space, select New > Folder, and name it
OfflineAI. - Keep all downloaded files inside this exact
OfflineAIfolder so your launcher script can locate them automatically without relying on drive letters (likeD:\orE:\), which change from computer to computer.
Step 2: Download the Official llama.cp Engine (File 1)
Instead of downloading unknown executables from social media links, grab the official pre-compiled Windows release directly from GitHub:
- Visit the official ggml-org/llama.cpp Releases page on GitHub (
[github.com/ggml-org/llama.cpp/releases](https://github.com/ggml-org/llama.cpp/releases)). - Scroll down to the Assets section of the latest release.
- Download the standard Windows CPU build named:
llama-bXXXX-bin-win-cpu-x64.zip(Note:XXXXrepresents the current release build number).- Tip: Even if your PC has a graphics card, the
win-cpu-x64package works universally across almost every Windows laptop and office desktop without requiring NVIDIA CUDA drivers.
- Tip: Even if your PC has a graphics card, the
- Right-click the downloaded
.zipfile, click Extract All…, and extract all contents directly into your USB’sOfflineAIfolder. - Verify that
llama-server.exeand its accompanying.dllfiles are sitting directly inside theOfflineAIfolder.
Step 3: Download a Compact .gguf AI Model (File 2)
Next, you need the AI model file itself. For smooth performance on standard laptops and office PCs without dedicated graphics cards, a 3-billion (3B) parameter model quantized to Q4_K_M offers the best balance of speed and intelligence.
- Go to Hugging Face (
huggingface.co) and search for a lightweight GGUF model, such as:bartowski/Llama-3.2-3B-Instruct-GGUFQwen/Qwen2.5-3B-Instruct-GGUF
- Click the Files and versions tab.
- Download the file ending in
Q4_K_M.gguf(for example:Llama-3.2-3B-Instruct-Q4_K_M.gguf, which is roughly 2.0 GB). - Move or save this
.gguffile directly into yourOfflineAIfolder alongsidellama-server.exe. - Rename the model file to something simple with no spaces—for example,
model.gguf. Renaming prevents typo errors in your startup script.
Step 4: Create the .bat Launcher File (Without Hidden .txt Errors)
A Windows batch (.bat) file automates the command-line instructions so you never have to type commands manually.
- Open your
OfflineAIfolder in File Explorer. - Click View > Show (on Windows 11) or the View tab (on Windows 10) and make sure File name extensions is checked. (This prevents Windows from accidentally naming your file
Start_AI.bat.txt, which will not run.) - Right-click inside the
OfflineAIfolder and select New > Text Document. - Open the new text document in Notepad and paste the exact code below:
@echo off title Portable Offline AI Server cd /d "%~dp0" echo Starting local AI server... echo Once loaded, open your browser to: http://127.0.0.1:8080 echo Keep this black window open while chatting. echo. start "" "http://127.0.0.1:8080" llama-server.exe -m model.gguf -c 4096 --host 127.0.0.1 --port 8080 pause
- In Notepad, click File > Save As…
- Set Save as type to All Files (
*.*). - Set the File name to
Start_AI.batand click Save. - Delete the original empty
New Text Document.txtfile so your folder stays clean.
Why This Exact Script Works Everywhere
cd /d "%~dp0"forces Windows to run the command from whatever drive letter your USB stick gets assigned on any PC.-m model.ggufloads your renamed AI model. (If you kept the original filename, replacemodel.ggufwith the exact filename enclosed in quotes, such as"Llama-3.2-3B-Instruct-Q4_K_M.gguf".)-c 4096sets a comfortable conversation memory window without overwhelming 8 GB RAM machines.start "" "[http://127.0.0.1:8080](http://127.0.0.1:8080)"automatically opens the built-in web chat interface in your default browser.
Step 5: Verify Your Folder Layout and Launch
Before testing, double-check that your OfflineAI folder on the USB drive contains these essential items side-by-side:
Start_AI.batllama-server.exe(along with the extracted.dllfiles from the zip)model.gguf
Running Your Offline AI
- Disconnect from Wi-Fi or unplug your Ethernet cable to verify true offline operation.
- Double-click
Start_AI.bat. - A black Command Prompt window will appear while the model loads from the USB drive into your computer’s RAM (this usually takes 5 to 15 seconds).
- Your web browser will open to
[http://127.0.0.1:8080](http://127.0.0.1:8080)displaying the built-in llama.cpp WebUI. Type a message and press Enter to chat completely offline. - When you are finished, simply close the browser tab and close the black Command Prompt window before safely ejecting your USB drive.
Troubleshooting Common Errors
- The black window opens and closes immediately: Your model filename in
Start_AI.batdoes not match the actual.gguffile in the folder, or a.dllfile from the zip archive wasn’t extracted. Verify thatmodel.ggufis spelled identically in both places and that all.dllfiles sit in the same folder asllama-server.exe. - Browser says “This site can’t be reached”: The model is still loading from the USB drive into RAM. Wait 10 seconds for the black terminal window to display
server is listening on [http://127.0.0.1:8080](http://127.0.0.1:8080), then refresh the browser page. - Windows SmartScreen pops up: Because
llama-server.exeis an open-source binary without a paid Microsoft corporate certificate, Windows may show a warning on the first run. Click More info > Run anyway.
