I Bought a Base-Model Mac Mini with an Old iPhone and Tried Running a Local LLM
I installed Ollama on a spare machine lying around the house and tried running a 7B model, and it works better than I expected. It answers coding questions and even translates without internet.
Of course, it's not as smart as the latest models, but it's pretty fun saving the 20,000-won monthly subscription fee lol
9 answers
With 7B, honestly, translation and summarization are more practical than coding. For code, it only becomes usable once you go past 14B.
Agreed, this is fun.
Do you have a source? It would be more convincing if there were at least benchmark numbers.
The fun of saving on subscription fees, lol. Though I wonder if it just ends up going out as electricity bills. With a base-model Mac mini, thermal throttling would probably make the perceived speed drop quite a bit—how was it when you ran it for a long time?
Well, that's a bit much. Comparing running a 7B model with paid models is apples to oranges—they're not even in the same league. It was just a cheap taste test; I wouldn't call it a replacement.