The best open-source LLMs to self-host in 2026 are no longer a single “winner” but a full range: from 12–30B models that run on one GPU to trillion-parameter models that need an HGX server. This comparison looks at what matters to a business: licence terms, the hardware each model actually needs, and where it fits.
Status as of November 2026. This market changes every month, so we update the comparison quarterly. We deliberately leave out benchmark scores: leaderboards go stale quickly, and whether a model is useful for your organisation is decided by a test on your own data, not by a public ranking. What we show instead is how much memory each model needs and what hardware it runs on. Every model listed has publicly available weights, so you can run it on your own server and keep your data inside your infrastructure.
How to choose an open-weight model for your business
Settle five questions before you look at the table. In practice they eliminate most candidates faster than any quality comparison.
1. Licence: can you use it commercially?
In the LLM world, “open source” usually means open weights, not necessarily an open licence. Models under Apache 2.0 or MIT (for example Qwen3.8-27B, Gemma 4, gpt-oss, Mistral Small 4, Devstral Small 2, DeepSeek V4 and GLM-5.3-Flash) can be used commercially, modified and fine-tuned without extra permission. Models under custom licences (Llama 4 Community License, the licences of the largest Qwen variants, the Kimi licences, GLM-5.3, Mistral Large 4 and Medium 3.5, Devstral 2) come with additional conditions. The Llama 4 licence, for instance, requires companies whose services exceed 700 million monthly active users to request a separate licence from Meta. Have your legal team read any custom licence before deployment. Kimi K2.7-Code uses a “modified MIT” licence, so check the added clauses too.
2. Model size versus hardware
At 4-bit precision the weights alone take roughly parameters × 0.5 bytes. On top of that comes memory for context (the KV cache) and runtime overhead. With Mixture-of-Experts (MoE) models you must hold all parameters in memory, even though only a subset (the “active” parameters) is computed per token. A low active count makes generation faster; it does not reduce VRAM requirements. We walk through the full calculation in How much VRAM does an LLM need.
3. Quality in your language
Large multilingual models generally handle major European languages well, while the smallest models make more grammatical and stylistic mistakes outside English. If you work mainly in Polish, there are dedicated Polish models, which we cover in Polish-language LLMs: Bielik and PLLuM. Whatever you choose, prepare 30–50 typical tasks from your own work and compare two or three candidates on them.
4. Multimodality
If the model has to read scans, photos or charts, you need a multimodal variant. In this comparison, Qwen3.8-27B and DeepSeek-V4.1-Flash are multimodal according to their model cards. For anything else, check the model card on Hugging Face before assuming the model can “see”.
5. Coding and agents
Coding has its own dedicated variants: Devstral Small 2 (24B), Devstral 2 (125B) and Kimi K2.7-Code. For agentic work (tool calling, multi-step workflows) there is Qwen-AgentWorld-35B-A3B.
The best open-source LLMs compared in one table
The “4-bit weights + ~15%” column is parameters × 0.5 bytes plus roughly 15% overhead. We picked the minimum hardware so that, after loading the weights, there is still room for context for a handful of users. Active parameters are shown only where the model name or card states them.
| Model | Total parameters (active, MoE) | Licence | 4-bit weights + ~15% | Minimum hardware (4-bit) |
|---|---|---|---|---|
| Bielik-11B-v3.0-Instruct | 11B | Apache 2.0 | ~6.3 GB | 1 GPU, 12 GB |
| Gemma 4 12B | 12B | Apache 2.0 | ~6.9 GB | 1 GPU, 12 GB |
| gpt-oss-20b | 21B | Apache 2.0 | ~12 GB | 1 GPU, 24 GB |
| Devstral Small 2 (coding) | 24B | Apache 2.0 | ~14 GB | 1 GPU, 24 GB |
| Qwen3.8-27B (multimodal) | 27.8B | Apache 2.0 | ~16 GB | 1 GPU, 24 GB; 48 GB for comfort |
| Gemma 4 31B | 31B | Apache 2.0 | ~18 GB | 1 GPU, 24 GB (tight); 48 GB for comfort |
| Qwen-AgentWorld-35B-A3B (agents) | 35B (3B active) | Apache 2.0 | ~20 GB | 1 GPU, 48 GB |
| Llama 4 Scout | 109B (17B active) | Llama 4 Community (custom) | ~63 GB | 1× RTX PRO 6000 96 GB or DGX Spark 128 GB |
| gpt-oss-120b | 117B | Apache 2.0 | ~67 GB | 1× RTX PRO 6000 96 GB or DGX Spark 128 GB |
| Mistral Small 4 | 119B | Apache 2.0 | ~68 GB | 1× RTX PRO 6000 96 GB or DGX Spark 128 GB |
| Devstral 2 (coding) | 125B | custom | ~72 GB | 1× RTX PRO 6000 96 GB or DGX Spark 128 GB |
| Mistral Medium 3.5 | 128B | Mistral custom | ~74 GB | 1× RTX PRO 6000 96 GB or DGX Spark 128 GB |
| Qwen3.8-Flash-Next | 180B | Qwen Community (custom) | ~104 GB | DGX Spark 128 GB or 2× RTX PRO 6000 (192 GB) |
| GLM-5.3-Flash | 321B | MIT | ~185 GB | 2× DGX Spark cluster (256 GB) or 4× RTX PRO 6000 (384 GB) |
| Llama 4 Maverick | ~400B | Llama 4 Community (custom) | ~230 GB | 4× RTX PRO 6000 (384 GB); 2× DGX Spark is borderline |
| GLM-5.3 | ~753B | GLM-5.3 custom | ~433 GB | 8× RTX PRO 6000 (768 GB); FP8: HGX B200 |
| DeepSeek-V4.1-Flash (multimodal) | ~763B | MIT | ~439 GB | 8× RTX PRO 6000 (768 GB) or HGX B200 |
| Mistral Large 4 | ~1T (52B active) | Mistral custom | ~575 GB | 8× RTX PRO 6000 (768 GB); FP8: HGX B200 |
| Kimi K2.7-Code (coding) | ~1.03T | modified MIT | ~592 GB | 8× RTX PRO 6000 (768 GB) or HGX B200 |
| DeepSeek-V4-Pro | ~1.65T | MIT | ~949 GB | HGX B200 (8 × 180 GB = 1.44 TB) |
| Qwen3.8-2.4T-A95B | ~2.4T (95B active) | custom (“qwen3.8-max”) | ~1.38 TB | HGX B300 (8 × 288 GB ≈ 2.3 TB) |
| Kimi K3 | ~2.78T | Kimi K3 custom | ~1.6 TB | HGX B300 (8 × 288 GB ≈ 2.3 TB) |
How we sized the hardware, and what the table does not show
Worked example: gpt-oss-120b has 117 billion parameters. 117 × 0.5 bytes = 58.5 GB of 4-bit weights; with ~15% overhead that is about 67 GB. On a 96 GB RTX PRO 6000 that leaves roughly 29 GB for context. A DGX Spark (128 GB unified memory) has more headroom, but that memory is shared between CPU and GPU.
Three caveats worth knowing before you buy or rent hardware:
- Context and concurrency. The KV cache grows with context length and the number of simultaneous requests. The table shows a minimum for a few users. For dozens of users, or documents hundreds of pages long, you need considerably more memory.
- Precision. 4-bit quantisation saves memory at the cost of some quality. For the largest models in production, FP8 is a common choice, which doubles the memory: GLM-5.3 then needs about 866 GB and Mistral Large 4 about 1.15 TB, which means an HGX B200. We discuss the trade-offs in cutting AI infrastructure costs with quantisation.
- Memory bandwidth. “It fits” is not the same as “it runs fast”. DGX Spark has 273 GB/s of memory bandwidth; an RTX PRO 6000 has about 1.6 TB/s. A large model will run on a Spark, but it will produce fewer tokens per second than on an HBM GPU. With eight PCIe cards, GPU-to-GPU communication is also slower than over NVLink in HGX systems. We compare the options in Best GPU for LLMs in 2026.
At 4-bit, the weights of DeepSeek-V4.1-Flash and GLM-5.3 would technically fit in a 4× DGX Spark cluster (512 GB). That is above the roughly 700B parameters NVIDIA states for such a cluster and leaves no sensible room for context, so we do not recommend it.
Recommendations by scenario
A single GPU: Qwen3.8-27B, Gemma 4 31B, gpt-oss-20b, Devstral Small 2
This is the right starting point for most organisations. All four are Apache 2.0 and fit on a 24 GB card at 4-bit. With 48 GB you have comfortable room for context, or you can run them at 8-bit. Pick Qwen3.8-27B if you need image input, Gemma 4 31B as a strong general-purpose model, gpt-oss-20b as the lightest option, and Devstral Small 2 for code. All of them run on our ready-made GPU packages with A10, A30 or A40 cards; despite the name, the packages are not limited to DeepSeek.
Mid-range: gpt-oss-120b, Mistral Small 4, Llama 4 Scout
Models in the 100–120B range handle harder tasks noticeably better while still fitting on a single 96 GB RTX PRO 6000 or a single DGX Spark. gpt-oss-120b and Mistral Small 4 are Apache 2.0; Llama 4 Scout uses Meta’s custom licence, so send it to legal first. For prototypes and small teams, DGX Spark as a Service works well; for many concurrent users, a server with RTX PRO 6000 cards is the better fit.
Large models: GLM-5.3-Flash, DeepSeek-V4.1-Flash
Both are MIT-licensed, which is rare and very convenient at this scale. GLM-5.3-Flash (321B) runs on a two-unit DGX Spark cluster or a server with four RTX PRO 6000 cards. DeepSeek-V4.1-Flash (~763B, multimodal) needs an eight-GPU server or an HGX B200. A sensible choice when mid-range models are not good enough but you do not want to pay for the very largest.
Frontier scale on HGX: DeepSeek-V4-Pro, Kimi K3, Qwen3.8-2.4T, Mistral Large 4, GLM-5.3
These are for organisations that want quality close to the best cloud services without sending data outside. DeepSeek-V4-Pro at 4-bit needs about 949 GB, so an HGX B200 with 1.44 TB of HBM3e. Qwen3.8-2.4T (~1.38 TB) and Kimi K3 (~1.6 TB) realistically need an HGX B300 with about 2.3 TB, because a B200 leaves no headroom for context. Mistral Large 4 and GLM-5.3 fit on eight RTX PRO 6000 cards at 4-bit, but FP8 production serving calls for an HGX B200. Of this group only DeepSeek-V4-Pro is MIT-licensed; the rest use custom licences. We provide these systems as GPU servers in our data centre, as a service or under colocation.
From choosing a model to running it
Once you have two or three candidates, test them on the same task set and on hardware close to your target. Ollama is fine for first experiments, but for multi-user production use vLLM, SGLang or NVIDIA NIM; we compare them in Ollama vs vLLM vs SGLang vs NVIDIA NIM. Plan for updates as well. The best open-source LLMs change every few months, so build your application against an OpenAI-compatible API. Swapping the model then becomes a configuration change, not a rewrite.
FAQ: frequently asked questions
What is the best open-source LLM for commercial use?
There is no single winner. As of November 2026, the licence-safe (Apache 2.0 or MIT) and hardware-practical choices are Qwen3.8-27B and Gemma 4 31B for one GPU, gpt-oss-120b and Mistral Small 4 for the mid-range, and GLM-5.3-Flash and DeepSeek-V4.1-Flash for larger deployments.
Is Llama 4 free for commercial use?
The weights are free to download, but under the Llama 4 Community License, which has its own terms. One of them requires companies whose services exceed 700 million monthly active users to obtain a separate licence from Meta. Have a lawyer review it before deployment.
Which LLM can I run on a single GPU?
On a 24 GB card at 4-bit: Qwen3.8-27B, Gemma 4 31B, gpt-oss-20b, Devstral Small 2 or Bielik. On a single 96 GB card (RTX PRO 6000): models of roughly 110–130B, such as gpt-oss-120b, Mistral Small 4 and Llama 4 Scout.
How much memory does DeepSeek V4 need to run locally?
DeepSeek-V4-Pro (~1.65T parameters) needs about 949 GB at 4-bit, i.e. an HGX B200. DeepSeek-V4.1-Flash (~763B) needs about 439 GB, i.e. eight RTX PRO 6000 cards or an HGX B200. Context memory comes on top.
How often do open-source LLM rankings change?
New versions of major models appear every month. This comparison reflects November 2026, and we update it quarterly.
Have a candidate and want to run it? We will size the hardware and deploy the model on a GPU server in our Tier III data centre in Poland, as a Supermicro and NVIDIA partner, with 24/7 management or full root access. Get in touch via our contact page; Blackwell and DGX Spark configurations are quoted individually.