Qwen 2.5 Coder 7B on Your Own Machine: Install, Measure, and Put It to Work in 20 Minutes
By the end of this post you will have Qwen 2.5 Coder 7B answering on your own machine, a measured tokens-per-second figure for it, and three prompts you can use on Monday: reviewing a script, writing SQL and refactoring a function. The cheapest starting point is a laptop with 16 GB of RAM. No account, no API key, and your code never leaves the building.
What you need
- A machine from the table below. The model file is 4.7 GB. It takes 4.7 GB of memory at Ollama’s default 4k context and 6.6 GB at 32k; both numbers are from
ollama pson our workstation today. - Ollama, from ollama.com/download.
- Twenty minutes, most of it the download.
| Machine | Fits? | What to expect |
|---|---|---|
| Laptop, 16 GB RAM, no dedicated GPU | Yes, on the CPU | Single-digit tokens per second. With the GPU switched off, our own machine gave 5.2 tok/s. Fine for one question at a time. |
| Laptop or desktop with an 8 GB NVIDIA card | Yes, at 4k to 8k context | Keep num_ctx modest so the whole model stays on the GPU. |
| Mac mini M4, 24 GB (~EUR 920) | Yes | ~35 tok/s on Qwen 2.5 7B (Compute Market) |
| Jetson Orin Nano Super, 8 GB (~EUR 250) | Not the 7B: 4.7 to 6.6 GB on 8 GB shared with the OS leaves nothing | Run qwen2.5-coder:3b (1.9 GB) instead; 3B models reach 12 to 18 tok/s there (Edge AI Vision) |
| Our GB10 workstation, 128 GB unified memory | Yes | 33.4 tok/s, measured 2026-09-09 |
Step 1: Install and run
ollama run qwen2.5-coder:7bThat downloads the 4.7 GB Q4_K_M build with a 32K context window (ollama show qwen2.5-coder:7b prints both). Ask it something, then type /bye. The Ollama library page lists sizes from 0.5b (398 MB) to 32b (20 GB). From the model card: 7.61 billion parameters, Apache 2.0 licence, trained on 5.5 trillion tokens, 128K context on the full-precision weights, released September 2024.
Step 2: Measure it
curl -s localhost:11434/api/generate -d '{
"model": "qwen2.5-coder:7b",
"prompt": "Write a Python function that validates a Spanish NIF (DNI + letter). Return only the code with a short docstring.",
"stream": false, "options": {"temperature": 0.2}
}' | python3 -c "import json,sys; d=json.load(sys.stdin); \
print(d['eval_count'],'tokens', round(d['eval_duration']/1e9,2),'s', \
round(d['eval_count']/(d['eval_duration']/1e9),1),'tok/s; load', round(d['load_duration']/1e9,2),'s')"Our runs on 9 September 2026:
| Task | Output tokens | Generation time | Speed | Load time |
|---|---|---|---|---|
| Write a NIF validator | 231 | 6.92 s | 33.4 tok/s | 10.7 s (cold) |
| SQL query (Task B below) | 99 | 2.96 s | 33.5 tok/s | 0.20 s (warm) |
| Review a script (Task A below) | 450 | 13.47 s | 33.4 tok/s | 10.9 s (reloaded) |
| Refactor a function (Task C below) | 309 | 13.41 s | 23.0 tok/s, with a second model loaded alongside | 48.6 s |
An earlier batch on the same machine gave 33.2 tok/s, 400 tokens in 12.0 s, cold load 10.5 s and warm load 0.19 s. The numbers are stable when the GPU is quiet. The NIF function it wrote was correct, letter table and modulo 23 included. If your speed is far below the hardware table, run ollama ps: you want 100% GPU.
Step 3: Three tasks, with exact prompts and what came back
Task A: code review. Prompt: “Review this Python script for bugs, security problems and anything that would break on a Windows machine. Be concise: a numbered list, most serious first, with a one-line fix each.” followed by a ten-line script that built an SQL string with % formatting and wrote to /tmp. The model returned six numbered findings, in the right order: SQL injection first, with the fix db.execute("INSERT INTO clients VALUES (?, ?)", r); then the hard-coded /tmp path on Windows, missing error handling, no argument check, an unclosed connection and the CSV header. Nothing serious was missed.
Task B: SQL generation. Prompt: “Tables: clientes(id, nombre, provincia), facturas(id, cliente_id, fecha, total, pagada). Write one SQLite query: total invoiced and total unpaid per province for 2026, ordered by unpaid descending. Return only SQL.” It returned a clean join with SUM(CASE WHEN f.pagada = 0 THEN f.total ELSE 0 END), a GROUP BY and the right ORDER BY, in three seconds. One defect: the filter was YEAR(f.fecha) = 2026, a MySQL function that does not exist in SQLite; the correct line is strftime('%Y', f.fecha) = '2026'. Say the dialect twice and run the query on a copy.
Task C: refactoring. Prompt: “Refactor this Python function so it is readable and testable. Keep the behaviour identical. Use type hints, no global state, and explain the changes in three bullet points after the code.” followed by a function that read i[1] and i[2] from three-field rows. The result had type hints, good names and the three bullets, and it was wrong: it unpacked each row into two fields, so it would crash on the real data while claiming the behaviour was identical. For refactors, paste one sample row and one test, and run the test.
Where this fits, and honest limits
Two of our three tasks came back with a defect a careful reader catches in a minute. That is the honest shape of a 7B coding model: a fast, private junior colleague whose work you read before you merge. It is good at review of small scripts, SQL, boilerplate, tests and explaining unfamiliar code. It does not reason across a repository, it is weaker on rare frameworks, and it states wrong things with confidence. Qwen’s own card says the 32B version matches GPT-4o; the 7B is the size that fits a laptop, not that claim. Keep a frontier model for architecture and hard debugging, and give the routine share to this one; the break-even calculation shows what that share is worth.
Step 4: Wire it into VS Code and n8n
VS Code with Continue. Install the Continue extension and put this in ~/.continue/config.yaml, following the Continue Ollama guide:
models:
- name: Qwen2.5-Coder 7B
provider: ollama
model: qwen2.5-coder:7b
apiBase: http://localhost:11434
roles: [chat, edit]
- name: Qwen2.5-Coder 1.5B
provider: ollama
model: qwen2.5-coder:1.5b
roles: [autocomplete]The guide recommends the 1.5b for autocomplete, where latency matters more than depth, and a bigger model for chat and edits. Point apiBase at another machine’s address to share one server across the office.
n8n. Create an Ollama credential with Base URL http://localhost:11434, attach an Ollama Chat Model node to a Basic LLM Chain, and pick qwen2.5-coder:7b. If n8n runs in Docker, the n8n Ollama credential docs explain setting OLLAMA_HOST so the container can reach the server. Our n8n code review workflow tutorial turns Task A into an automatic step on every pull request.
Next steps
- Compare it with the other local models we run: Best local LLMs, Q2 2026 comparison.
- Pick the machine: Edge AI hardware guide 2026.
- Automate the review: n8n AI code review workflow.
Work with us
We run this model daily on our own hardware and we set it up for clients with the prompts, the VS Code config and the n8n workflow included. If you want it working on your machines, get in touch or see how our consulting works.