Strata is an open source engine for running large language models on commodity hardware. The project has a simple goal: to make the powerful model work locally, without the cloud, without an account and without paying for tokens. Installation is reduced to one installer, and after launch you get chat, working with code and even receiving images on your computer.
Under the hood, Strata has its own inference engine, and outside it exposes an API compatible with OpenAI and Anthropic at a local address. This means that you can connect your favorite tools to it – Claude Code, Codex CLI, OpenCode and other agents – and they will think that they are talking to the cloud provider. The data does not leave your machine.
In this post, I’ll break down what Strata can do, what models are supported, how to install it, and how to connect it to your instruments.
What is it
Strata is a free project under the MIT license. It solves three problems:
- Local launch of models. The engine is designed for consumer video cards and combined memory, and not for server accelerators.
- Single interface. Externally, this is an OpenAI- and Anthropic-compatible API on localhost, so almost everything that can work with these protocols can be connected.
- Simplicity. The desktop application and the installer for the terminal remove the work with dependencies, drivers and assembly from sources.
Separately, it is worth noting the optional input of images. The model can work not only with text, but also with pictures, which makes local chat much more useful for parsing screenshots and documents.
Models
The main recommendation of the project is Qwen3.8-Flash-Next. Several quantization sets are offered for it from different suppliers:
- ISTA-DASLab – recommended option: Q2_0, IQ2_XS, IQ3_XXS, IQ3_S, and for the code a separate Coder IQ1_M.
- Unsloth – UD-IQ4_XS, UD-Q4_K_XL, UD-Q5_K_XL, UD-Q6_K_XL, Q8_0.
- Gyro from agentionai – Gyro-S and Gyro-M: less memory, but slower.
In addition to the flagship model, Qwen3.6-35B-A3B and DeepSeekV4.1-Flash-1017 are supported. The latter is interesting because parts of the model that do not fit into memory are read directly from the SSD, so it requires a lot of free disk space. There are options for 239, 257 and 274 GB, as well as a lightweight Coder version of approximately 191 GB, where half of the experts have been removed.
Launching DeepSeek from the terminal looks like this:
strata launch --family deepseek41 --model ist-das-2.00bpw --plan
The –plan flag shows the startup plan before downloading begins. For large sets this is useful: you can immediately see how much memory and disk will be required.
Hardware requirements
Strata supports a wide range of equipment:
- NVIDIA GeForce RTX 20, 30, 40 and 50 series.
- AMD Radeon RX 7700, 7800, 7900, RX 9060, 9070 and AI PRO R9700.
- AMD Radeon RX 6800 and 6900 – experimental.
- Intel Arc for Linux.
- Apple Silicon from M1 and newer.
- AMD Ryzen AI Max (Strix Halo).
- NVIDIA DGX Spark for Linux.
For reference, here are some measurements of DeepSeekV4.1-Flash on a 2.00 bpw set. In the “output” column – the speed of generating the response, in the “prompt” column – the speed of reading your input.
- RTX 5070 12 GB, 64 GB RAM – 4–6 tokens/s, prompt about 110 tokens/s.
- RTX 3060 12 GB, 46 GB RAM – 9–11 tokens/s, prompt about 155 tokens/s.
- 1x Radeon AI PRO R9700, 251 GB RAM – 21–25 tokens/s, prompt 320–360 tokens/s.
- 4x Radeon AI PRO R9700 – 74–77 tokens/s, prompt 400–450 tokens/s.
- Ryzen AI Max+ 395 (Strix Halo), 128 GB of combined memory – 19–21 tokens/s on code and 11 on chat.
If you are not sure what will fit, the project has a separate guide for choosing a model for the amount of memory.
Installing a desktop application
The easiest way is to download the ready-made installer from the releases page. Each system has its own file:
Strata-0.2.13-windows-x64-setup.exe
Strata-0.2.13-linux-x86_64.AppImage
Strata-0.2.13-macos-arm64.dmg
Install the application, select a model and start communicating. When you first launch the application, it automatically downloads the engine for your video card. There is no need to separately configure drivers or compile anything from sources.
In the application window, chat, files and live code preview are located nearby, which is convenient when working on a project.
Installation in the terminal
If you prefer the console, install Strata with the installation script and then run the strata launch command.
On Linux and macOS:
curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh
In Windows PowerShell:
irm https://github.com/Niko1221/Strata/releases/latest/download/install.ps1 | iex
Once installed on Windows, open a new terminal window and run:
strata launch
The program will ask questions, download the desired model and launch it. During operation, the model’s activity, speed, memory consumption and latest requests are shown – convenient for understanding what is happening with the hardware.
For those who want to build everything themselves, there is a build from source, an option via Docker and a detailed installation guide.
Connecting instruments
Strata’s main strength is that it exposes a local API that is compatible with OpenAI and Anthropic. Thanks to this, popular agent tools are connected to it: Claude Code, Codex CLI, OpenCode, oh-my-pi, pi.dev, Hermes Agent, OpenClaw and others.
To find out how to configure a specific tool, just run:
strata connect
The team will show ready-made instructions for your application. Then everything is the same as with a regular cloud provider: the local endpoint Strata is specified as the base address, and what the engine produces is specified as the key. No Internet is required for inference.
What to pay attention to
Running locally is a trade-off, and it’s worth being aware of in advance.
- The speed depends on the hardware. On weak video cards, large models will respond slowly, especially if part of the weights is read from disk. The measurements above give a rough idea.
- The models are still large. Disk and memory are the main limiters. DeepSeek requires hundreds of gigabytes of free space.
- Quantization quality. Highly compressed sets save memory, but may lose quality. If your hardware allows, take the option with a higher bpw.
- Initial download. At the first launch, both the engine and the model weights are downloaded, so you need a stable connection and disk space.
At the same time, you get full control: no limits, subscriptions or data leaks. Everything counts on your machine.
Who is this for?
Strata makes sense if you want to keep your work data local, don’t rely on cloud plans, or just like experimenting with local AI. For a programmer, this is also a way to connect a familiar agent to a model that works nearby, without network delays and without paying for each request.
The project is actively developing: tens of thousands of stars, thousands of forks, frequent releases and detailed documentation for each type of hardware. The easiest way to start is with the desktop application, and then, if you like it, go to the terminal and connect your tools.
Links
https://github.com/niko1221/strata
https://github.com/Niko1221/Strata/releases/latest
https://github.com/Niko1221/Strata/blob/main/docs/INSTALL.md
https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md
https://github.com/Niko1221/Strata/blob/main/docs/TERMINAL_LAUNCH.md
https://github.com/Niko1221/Strata/blob/main/docs/DEEPSEEK41.md
https://github.com/Niko1221/Strata/blob/main/docs/HOW_IT_WORKS.md
https://github.com/Niko1221/Strata/blob/main/docs/TROUBLESHOOTING.md
Sources
https://github.com/niko1221/strata
https://github.com/Niko1221/Strata/blob/main/README.md
https://github.com/Niko1221/Strata/blob/main/docs/INSTALL.md
https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md
https://github.com/Niko1221/Strata/blob/main/docs/DEEPSEEK41.md
https://github.com/Niko1221/Strata/blob/main/docs/AMD_HIP.md
https://github.com/Niko1221/Strata/blob/main/docs/STRIX_HALO.md
Leave a Reply