I Ran a Local LLM on a Spare Laptop at Home (Trial-and-Error Log)
I wanted to try running a local model on a laptop that was just sitting around unused without a graphics card, so I poured two weekend days into it. The conclusion: 'It works, but it's slow.'
The environment was a problem from the start. I tried on Windows, got the drivers tangled up, and ended up doing a fresh Ubuntu install. I loaded a 7B quantized version of the model, and just downloading it took 40 minutes.
When I asked the first question, it took 3–4 seconds per line of output. It's an embarrassing speed to even call a conversation. Still, it was kind of amazing that it ran without internet.
Here's what I took away:
- If you don't have a GPU, just using a cloud API is better for your mental health
- 16GB of RAM is the bare minimum; you need 32GB to breathe
- The heat is really bad — the surface under the laptop gets hot
Still, it was fun to be able to experiment with various things without worrying about privacy. Next time, I'm thinking of buying a used GPU and trying again.
10 answers
True, if you don't have a GPU, API is the answer lol
If it's quantized 7B, it can somehow run even on CPU, but 3-4 seconds per line is a real test of patience... I tried 13B and quit after getting startled by my laptop fan noise. Still, that thrill of running without internet keeps you tinkering with it. Have you compared both ollama and llama.cpp? There was a pretty noticeable difference in perceived speed.
The overheating is no joke. I had mine on a sheet and thought the bottom panel had gotten discolored. Yikes.
Well, that's a bit... If it's two days of banging your head against the wall with 7B quantization, I think you're better off just using the cloud. I'm not sure what running it locally gets you besides privacy.
I agree that 16GB is the minimum. I only became able to run it alongside my browser after upgrading to 32. Back when I had 16, Chrome would die as soon as the model loaded lol
Got a source? Where did the 32GB RAM recommendation come from?
The moment you do a fresh Ubuntu install, it’s already become a hobby lol. I started out that way too, and now I’m running a home server. If you’re going to buy a used GPU, go for 12GB VRAM or more; 8GB is tight even for 7B, and I regret it.