When the model comes home

I was on Windows 11. Nothing was broken. I left anyway.

The honest reason was not a deadline or a dead machine. It was the slow accumulation of decisions being made on my behalf. Updates I did not schedule. Telemetry I did not ask for. An operating system increasingly arranged around someone else’s product roadmap rather than my work. When I started running language models locally, the mismatch became impossible to ignore: the machinery I most wanted to understand ran best on the platform I was not using. AMD’s compute stack is Linux first, and every guide worth reading said the same thing. So one weekend I stopped renting the desk and bought it.

I am far from alone. Linux desktop share has climbed sharply since Windows 10 support ended in October 2025, though the headline StatCounter figures deserve care: a large Unknown category in the same data means every share number is an estimate, and some of this year’s jump is reclassification rather than adoption. The trajectory is real. The precision is not. What follows is not a trend piece, though. It is the build, exactly as I would hand it to a colleague.

The machine

This matters because it is deliberately ordinary. No workstation card, no server, no lab budget.

• CPU: AMD Ryzen 7 5700X3D. Eight cores on the old AM4 socket, with the 3D V-Cache that makes it a favourite of people who never quite got round to replacing a perfectly good motherboard.
• GPU: AMD Radeon RX 7800 XT with 16GB of VRAM. RDNA 3, known to the compute stack as gfx1101, and on AMD’s official ROCm support list.
• OS: Fedora Workstation.

The card is the whole story. In local inference, VRAM is the budget and everything else is negotiation. Sixteen gigabytes is the interesting middle: enough to do real work, not enough to be careless. You learn the constraint by living inside it, which turns out to be the point.

Why Fedora

Three practical reasons, none of them ideological.

First, the AMD graphics driver lives inside the Linux kernel itself. There is no driver hunt, no vendor installer, no version roulette. Fedora ships current kernels and current Mesa, so the card simply works on first boot, for display and for compute.

Second, ROCm, AMD’s answer to CUDA, is a Linux-native platform. Windows support exists and is improving, but every serious guide, benchmark and bug report assumes Linux. If you run AMD and you want local models, you are choosing Linux whether you have admitted it yet or not.

Third, Fedora carries ROCm components in its own repositories, which keeps the compute libraries in step with the kernel rather than bolted on beside it.

The setup that works

Install Fedora Workstation from the official media, take the updates, and confirm the card is visible. Then the model runner. Ollama is the shortest path from zero to a running model:

curl -fsSL https://ollama.com/install.sh | sh

Two things catch people on AMD, and both are permissions rather than drivers. Your user needs to belong to the render and video groups, and the change only takes effect after a full log out:

sudo usermod -aG render,video $USER

Then pull something small and watch the card, not the terminal:

ollama run qwen2.5-coder:14b
watch rocm-smi

If VRAM climbs and power draw rises the moment the model answers, the GPU is doing the work. If both stay flat, you are on CPU fallback. The classic fix on this generation of card was to set HSA_OVERRIDE_GFX_VERSION=11.0.0 for the Ollama service; because Ollama runs under its own system user, that goes in a systemd drop-in rather than your shell profile. Current stacks detect gfx1101 directly and most people will never need it, but it is the first thing to try if the card sits idle.

What 16GB actually buys

Here is the model landscape for this card, honestly.

The sweet spot: dense models up to about 14B. A 14B coder model at 4-bit or 5-bit quantisation occupies roughly nine to ten gigabytes, which means the entire model sits in VRAM with room left for context. Fully resident models are where this card shines, because inference speed is bound by memory bandwidth, and VRAM bandwidth is many times that of system RAM. Qwen2.5-Coder 14B has been my daily coding model. It is fast enough that you stop noticing it.

The comfortable ceiling: around 20B. OpenAI’s gpt-oss-20b, released as open weights in its 4-bit MXFP4 format, occupies roughly twelve to thirteen gigabytes and fits. It is my general-purpose resident model. One footnote for the curious: RDNA 3 does not accelerate that format natively, so the runtime handles the conversion. It runs well regardless.

The interesting case: Qwen 3 coder models. This is the question I get asked most. Qwen3-Coder-30B-A3B, the model people mean when they say Qwen 3 coder, is a mixture-of-experts design: 30.5 billion parameters in total, but only about 3.3 billion wake up for any given token, routed across 128 experts. At the standard 4-bit quantisation the weights alone come to roughly 18.6GB, so it cannot sit fully in 16GB of VRAM. Ollama handles this by parking part of the model in system RAM.

On a dense model that split would be painful. On a mixture-of-experts model it is merely a tax, because so little of the network is active per token. Community figures put the model in the low teens of tokens per second even with no GPU at all on fast RAM; a 16GB card carrying most of the weight does meaningfully better. The result on my machine is usable for real coding work: not the instant response of the resident 14B, but a bigger model with 256K of native context and genuinely strong agentic behaviour, running entirely on my desk. The same arithmetic applies to the newer Qwen 3.5 35B-A3B, which I run for agentic work: slightly larger, same expert trick, same offload story.

Out of reach, and worth saying so. Dense models at 30B and above want more card than this. They will load at aggressive quantisation and crawl. The frontier-sized open releases, including Qwen3-Coder-Next with its recommendation of more than 45GB of memory for 4-bit, are simply not this machine’s job. A 16GB card does not pretend otherwise, which is part of its educational value.

One last budget line people forget: context is memory too. The KV cache grows with every token in the window, and a long context can quietly eat gigabytes you thought were free. When a model that fits starts spilling, shorten the window before you shrink the model.

What running it yourself teaches you

Through an API, model capability feels like weather. It arrives, it changes, you adapt, and the mechanism stays out of sight. On your own machine, capability becomes an engineering quantity. You learn precisely where the model ends and the harness begins. You learn what a token costs in watts and seconds. You learn why a 14B model that fits outperforms a 30B model that does not, and then you learn the exception, because the expert-routing trick rewrites the arithmetic.

That knowledge has a name in governance terms: calibration. If your working life now involves judging AI-assisted output, and for most of us it quietly does, then knowing the machinery is not a hobby. It is what keeps your judgement current. You cannot stay calibrated against a black box you only ever rent.

And the freedom is real, in a way I did not fully anticipate. Nothing I prompt leaves the machine. No subscription meters the thinking. No update lands mid-task. The operating system does what I configured it to do, and nothing else. That is a small sentence that changes how a desk feels.

The honest limits

The frontier models remain ahead, and the gap is real on the hardest reasoning even as it narrows on everyday coding. The tuning and research ecosystem is still CUDA first, so new techniques reach NVIDIA hardware before they reach this card. And a local model produces no evidence trail unless you build one, which is a governance thread I will pull on separately.

None of that changes the conclusion. What needed a rack in 2023 needs a desk in 2026. The point was never to replace the frontier. The point was to understand it, on hardware I own, running an operating system that answers to me.

Sources and caveats

• AMD ROCm documentation for the official gfx1101 (RX 7800 XT) support status.
• Qwen model cards and GGUF listings for parameter counts, quantised sizes and context lengths; the 18.6GB Q4_K_M figure for Qwen3-Coder-30B-A3B via community VRAM calculators, consistent with the published files.
• StatCounter desktop share data via FOSS Post’s ongoing tracking, with the Unknown-category caveat carried deliberately: treat all Linux share figures as directional.
• Practitioner setup guides for the RX 7800 XT informed the troubleshooting steps; performance characterisations for the offloaded 30B model are community reported and vary with RAM speed.

Written in a personal capacity. Correspondence to jason@cognitivedelivery.co.uk.