Self-hosted and open-weight models, 7 of 26
24 tokens a second,
on one shared core.
Seven of the job posts ask about self-hosting, and the usual answer is a paragraph about privacy. The question underneath it is arithmetic: what does the hardware you already have produce, and what does running a model on it do to everything else.
So it was run. Qwen2.5 0.5B, four bit, under llama.cpp, on the VPS that also runs n8n, a scraper and client sites, with 13 containers already going.
| what was asked | tokens | generation | end to end | load while it ran |
|---|---|---|---|---|
| a short question | 64 | 24.2 t/s | 4.8s | 1.95 |
| an ordinary answer | 200 | 22.3 t/s | 11.2s | 2.03 |
| a long one | 400 | 21.0 t/s | 21.4s | 2.17 |
1 core, 16 GB, load 1.86 before and 2.17 at its highest. The model file is 469 MB.
The break-even
At 24.2 tokens a second, one core produces 87,120 tokens an hour, or 63.6 million in a month of running flat out. The same number of output tokens from gpt-4.1-mini costs $101.76 at $1.60 a million.
So self-hosting this model is cheaper than the API only if the machine costs less than $102 a month and you keep it busy every second of that month. No machine is busy every second, and this one has other work, so the real number is a fraction of that.
The comparison also flatters the small model, which is 0.5 billion parameters against something far larger. On quality they are not the same purchase.
Four decisions
It ran on the machine that was already busy
Not a clean laptop with nothing else on it. One shared core, sixteen gigabytes, and thirteen containers already serving n8n, a scraper and client sites. A benchmark on an idle machine answers a question nobody has.
The load reading is part of the measurement
Tokens a second with nothing said about what it cost the machine is half a number. The load average moved from 1.86 to 2.17 across the three runs, which is the honest version of "it fits", and it is only true because the container was capped at one core.
A first attempt produced nothing and was thrown away
The run before this one sat in conversation mode waiting on a keyboard for seventeen minutes while looking exactly like a model generating slowly. It nearly became a published finding about how slow self-hosting is. The number was wrong, so it went in the bin rather than on this page.
Break-even, not opinion
Privacy and control are the usual arguments and they are not measurable. Throughput is. The arithmetic below says exactly which monthly price makes this cheaper than the API, and it assumes the machine is busy every second of the month, which no machine is.
When this is the right call anyway
Twenty-one seconds for a four hundred token answer rules out anything a person waits for. It does not rule out work that runs overnight: classifying a queue, tagging documents, filling a field on ten thousand rows. That work does not care how long each answer takes, only how many finish by morning, and at 87,120 tokens an hour the answer is quite a lot.
And when the data cannot leave the building, the price stops being the question. That is the only argument for self-hosting that survives the arithmetic, and it is a good one.
The cost page is the same arithmetic for the paid models. The container page is what else runs on that box. The whole list is 41 requirements from 114 job posts.