Salvatore Sanfilippo, the Sicilian programmer better known online as antirez and best known for creating Redis, spent September 1, 2026 doing something that would have sounded like science fiction two years ago: feeding a photo to a 284-billion-parameter AI model running entirely on a laptop, with zero network calls. The model, DeepSeek V4 Flash Vision-Exp, described the image back to him in seconds. No API key, no cloud bill, no data leaving the machine.
The demo ran on an Apple M5 Max, and it was the capstone of a stretch antirez had spent finishing three separate hardware backends, Metal, CUDA and ROCm, for his inference engine DS4. By the next day, all three shipped with matching vision support. That detail, more than the demo video itself, is why developers and hardware watchers spent the week talking about it.
What antirez Actually Showed on September 1
The sequence was simple to watch and hard to pull off. antirez pointed his DS4 engine at DeepSeek V4 Flash Vision-Exp, DeepSeek’s experimental multimodal build of its flagship efficiency model, and asked it to analyze a picture. The Mac M5 Max processed the request locally and returned a description without touching a remote server. He then confirmed that the Metal, CUDA and ROCm implementations of DS4 were functionally complete, with only final testing left before a public release.
What made the moment land wasn’t raw speed, though the numbers hold up on their own. It was the fact that image understanding, the one capability nearly every local AI setup still farmed out to a cloud API, now runs inside an open source engine built from a few thousand lines of C. For anyone handling confidential documents, medical scans, or legally restricted material, that shift in where the data physically goes matters more than any benchmark score.
Who Is antirez, and Why the Local AI World Listens to Him
Salvatore Sanfilippo built Redis, the in-memory data store that became a default piece of infrastructure across the software industry, and he has kept a habit of shipping sharp, narrowly scoped tools rather than broad platforms. His prior contribution to the local AI space, an approach he calls Redis Array for handling large knowledge bases inside agent workflows, followed the same pattern: solve one specific bottleneck well instead of building a do-everything framework.
DS4 continues that habit. Rather than compete with generalist runtimes that try to run any model a user throws at them, antirez built an engine that runs exactly one model family extremely well. He has also been candid about how he built it: the DS4 README credits heavy assistance from GPT 5.5, GPT 5.6 and Claude Fable during development, while he retained control of the architecture and testing decisions himself.
Inside DS4: A Specialist Engine, Not a Generalist Runtime
DS4, short for DwarfStar4, is the inference engine antirez published on GitHub on May 6, 2026, under an MIT license. As of September 1, 2026, the repository had passed 21,984 stars. It does not attempt to run arbitrary GGUF models the way llama.cpp does. It runs DeepSeek V4 Flash, with support for the larger DeepSeek V4 PRO variant on machines with enough memory to hold it, and nothing else.
That narrowness is the point. DeepSeek V4 Flash is a mixture-of-experts model: only a fraction of its total parameters activate for any given token, an architecture that rewards hand-written kernels tuned to its exact shape far more than it rewards a generic runtime designed to handle every model shape at once. antirez bet that specialization would beat generality on this specific architecture, and the resulting throughput numbers back that bet up.
DeepSeek V4 Flash: The 284-Billion-Parameter Model Behind the Demo
DeepSeek V4 Flash carries 284 billion total parameters, of which only 13 billion activate per token, an extreme level of sparsity even by mixture-of-experts standards. DeepSeek previewed the model on April 24, 2026, and shipped a general-availability build, tagged DeepSeek-V4-Flash-0731, on July 31, 2026 under an MIT license with a 1-million-token context window.
The vision capability antirez demonstrated comes from a separate, newer release: DeepSeek V4 Flash Vision-Exp, which DeepSeek shipped on August 21, 2026. It is explicitly experimental and adds image understanding without giving up anything on text, reasoning or agent tasks. Images are counted as tokens, capped at 384 tokens per image, and DeepSeek has claimed multimodal performance approaching Anthropic’s Claude Opus 4.8. Until antirez’s demo, the only practical way to use that vision capability was through DeepSeek’s hosted API.
Metal, CUDA, ROCm: One Vision Model, Three Backends
Shipping vision support on a single backend would have been a solid engineering exercise on its own. Shipping it across three separate hardware backends within days is a different scale of effort, and the GitHub commit history shows the sprint clearly: Vision-Exp support landed first on Metal, two consecutive patches on August 31 fixed the CUDA and ROCm builds, and a final commit on September 1 added usage documentation and a hardening pass on the multimodal cache.
Apple Silicon and Nvidia Get Day-One Support
The Metal backend targets Apple Silicon Macs directly, which is why the M5 Max became the demo machine. The CUDA backend covers Nvidia GPUs, including Nvidia’s DGX Spark hardware, giving the same model and the same engine a path onto workstation-class Nvidia silicon without a separate codebase.
ROCm Support Relies on the Community
The AMD ROCm backend, which runs on systems like the Framework Desktop with Strix Halo, is maintained as a separate branch and rebased onto the main codebase by community contributors, since antirez does not personally have direct access to AMD hardware to test against. It is a structural risk worth watching: if community maintenance slows, the ROCm backend could start lagging Metal and CUDA on feature parity, even though it shipped vision support in the same week as the other two.
The Benchmark Numbers on the M5 Max
antirez published benchmark figures for DeepSeek V4 Flash running at 2-bit quantization on an M5 Max configured with 128GB of unified memory. The results, reproduced below, cover both a short prompt and a long, 65,536-token context:
| Stage | Short Context (2,048 tokens) | Long Context (65,536 tokens) |
|---|---|---|
| Prefill (reading the prompt) | 790.18 tokens/sec | 398.50 tokens/sec |
| Generation (writing the response) | 39.35 tokens/sec | 27.64 tokens/sec |
39.35 tokens per second at short context means replies scroll faster than most people read. What stands out more is the long-context figure: even with 65,000 tokens loaded, generation stays above 27 tokens per second, a pace usable for real work rather than a lab demo. This is a laptop running a model with close to 300 billion parameters, not a data center rack.
What It Actually Takes to Run DS4 at Home
The barrier to entry for DS4 is memory capacity, not an exotic GPU. The 2-bit quantized build of DeepSeek V4 Flash takes up roughly 81GB, which puts the comfortable range at 96GB to 128GB of unified or system memory. Below that threshold, DS4 can stream weights from a fast SSD instead, which lets a machine run a model larger than its RAM at the cost of generation speed.
DS4 keeps quality intact at aggressive quantization by applying it selectively. It quantizes only the MoE’s routed experts, using asymmetric low-precision formats such as IQ2_XXS and Q2_K on the layers that tolerate it, while keeping higher precision on the parts of the model that don’t. antirez also introduced an MXFP4 GGUF quantization route as a third option alongside the existing IQ2 and Q4 formats. Vision adds a modest extra cost: the Vision-Exp visual encoder weighs about 0.9GiB on top of the base language model’s memory footprint.
| Requirement | Minimum | Comfortable |
|---|---|---|
| Memory for 2-bit DeepSeek V4 Flash | ~81GB (SSD streaming below this) | 96–128GB unified/system RAM |
| Vision-Exp encoder overhead | +0.9GiB | +0.9GiB |
| Apple M5 Max unified memory (MacBook Pro) | up to 128GB | up to 128GB |
| M5 Max GPU memory bandwidth (40-core config) | up to 614GB/s | up to 614GB/s |
| License | MIT (free, commercial use allowed) | MIT |
Using the vision feature from DS4’s command line is deliberately minimal. Inside the interactive CLI, a single /read image.png command feeds a PNG or JPEG into the model, after which a user can ask questions about the image the same way they would in any multimodal chat interface. DS4 also ships a persistent on-disk KV cache, tool calling, and a server that exposes an OpenAI- and Anthropic-compatible API alongside a built-in coding agent, all aimed at developers who want to wire local inference into existing agent tooling without switching client libraries.
DS4 vs. llama.cpp vs. Ollama vs. MLX vs. vLLM
DS4 enters a local inference field that has already split into distinct lanes by use case rather than a single dominant tool. llama.cpp remains the generalist C/C++ runner that aims to support nearly any GGUF-format model, which makes it the fallback choice for embedded or unusual hardware but leaves performance on the table for any one specific architecture. Ollama, now on version 0.19 as of March 2026, wraps llama.cpp on non-Apple hardware and MLX on Apple Silicon, and remains the default pick for solo developers prototyping locally. MLX, Apple’s own machine learning framework, runs 15 to 25 percent faster than llama.cpp on Apple Silicon according to independent benchmarking, thanks to being purpose-built around the platform’s unified memory architecture. vLLM, meanwhile, targets production serving with PagedAttention and continuous batching, and reportedly hits 16 to 20 times Ollama’s concurrent-user throughput on Nvidia and AMD GPUs.
| Engine | Model Scope | Best Fit | Backends |
|---|---|---|---|
| DS4 (antirez) | DeepSeek V4 Flash / PRO only | Max throughput on one frontier MoE model | Metal, CUDA, ROCm |
| llama.cpp | Broad GGUF model support | Embedded / unusual hardware | CPU, CUDA, Metal, Vulkan |
| Ollama | Broad, via llama.cpp/MLX | Solo developer prototyping | llama.cpp (non-Apple), MLX (Apple) |
| MLX (Apple) | Broad, Apple-optimized | Apple Silicon, Python workflows | Metal only |
| vLLM | Broad, production-focused | Multi-user concurrent serving | CUDA, ROCm |
DS4’s pitch cuts against that landscape: it gives up breadth entirely in exchange for kernels hand-tuned to one sparse MoE architecture. On DeepSeek V4 Flash specifically, that trade is producing generation speeds and quality retention at low-bit quantization that the generalist tools aren’t built to match, precisely because they’re spreading engineering effort across every model shape instead of one.
Why Local Vision Changes the Privacy and Compliance Calculus
The practical significance of this release sits less in tokens-per-second and more in where data physically travels. Any organization that handles medical imaging, legal discovery documents, or other regulated material has, until now, faced a binary choice: send images to a cloud vision API and accept the compliance exposure, or avoid frontier multimodal AI altogether. DS4 running Vision-Exp locally removes that trade-off for anyone with the memory to run it, since the image never leaves the machine performing the analysis.
That shift also reframes the ongoing debate over whether a local Mac Studio can substitute for a recurring, roughly $200-a-month cloud AI subscription. As local hardware closes the throughput gap and gains multimodal capability, the economics increasingly favor buying the machine once rather than paying a cloud vendor indefinitely, at least for workloads where 27 to 39 tokens per second is fast enough. That calculation sits alongside the broader shift OpenAI and Anthropic have made toward Apple hardware for their own infrastructure needs.
Market Impact: Apple, Nvidia, AMD and the Cloud AI Providers
Apple is the most direct beneficiary of the M5 Max becoming the reference machine for a viral local AI demo. The chip’s unified memory architecture, which lets a single pool of RAM serve as both system memory and GPU memory, is precisely what makes running an 81GB quantized model on a laptop feasible in the first place. Apple’s own M5 Max specifications list up to 128GB of unified memory in the MacBook Pro configuration and GPU memory bandwidth as high as 614GB/s in the 40-core GPU variant, and the company has separately seen enterprise AI buyers scoop up Mac hardware for inference workloads rather than training.
Nvidia and AMD both gain from DS4 treating their hardware as first-class targets rather than afterthoughts. CUDA support puts DGX Spark and other Nvidia workstation GPUs on equal footing with the M5 Max for this specific model, at a moment when Nvidia’s consumer and workstation GPU pricing has already been climbing on AI demand. AMD’s inclusion via ROCm, even with its community-maintained caveat, gives Strix Halo-based systems like the Framework Desktop a credible local inference story they previously lacked for a model this capable.
Cloud AI providers face the least comfortable position. A specialist local engine that gets a 284-billion-parameter multimodal model to a usable 27 to 39 tokens per second on consumer-adjacent hardware chips away at the argument that frontier-capable inference requires a subscription. It won’t replace hosted APIs for training, fine-tuning at scale, or workloads that need concurrent multi-user throughput, where vLLM-style serving still wins by a wide margin, but for privacy-sensitive, single-user, or intermittent workloads, DS4 is a direct substitute rather than a complement.
Historical Context: From Redis to Local AI Infrastructure
antirez’s track record gives this release more weight than a random open source project would carry. Redis went from a personal project to infrastructure embedded across a large share of production web systems by solving a narrow problem, fast in-memory data access, extremely well rather than trying to be a general-purpose database. DS4 follows the identical playbook applied to inference: instead of a framework that runs any model adequately, it is an engine that runs one frontier-class model exceptionally well.
The approach also reflects a broader turn in how individual developers are approaching AI tooling in 2026: leaning on frontier models themselves, credited in DS4’s README as GPT 5.5, GPT 5.6 and Claude Fable, to accelerate low-level systems programming, while reserving architectural judgment and testing for a human expert. It is a working example of AI-assisted development producing systems software rather than just application code.
What antirez Has Said About DS4
antirez has been direct about DS4’s scope and motivation across his public posts and project documentation. Introducing the engine, he wrote, “Welcome to DS4, a specialized inference engine for DeepSeek v4 Flash,” a framing he repeated on the project’s GitHub page, describing it as “This is a custom native inference engine built specifically for DeepSeek v4 Flash.”
He has also laid out where he sees the specialist-engine approach heading next. Writing on his blog, antirez argued that “It is clear that there was a need for single-model integration focused local AI experience,” and floated a longer-term direction for the project: “For local inference, to have a ds4-coding, ds4-legal, ds4-medical models make a lot of sense, after all.” That comment points toward domain-specific variants of DS4 tuned for particular professional workflows rather than a single general-purpose build. On the model side, antirez’s own quantized releases carry a similar note of specialization, with the Hugging Face repository for his GGUF conversions stating that “This quants are specific for the DS4 inference engine.”
Predictions: Where Local Inference Goes From Here
A handful of trends look likely to follow from this release over the coming months:
- Other narrow, single-model inference engines will likely follow DS4’s blueprint for different open-weight MoE models, trading breadth for hand-tuned throughput on one architecture at a time.
- Memory capacity, not raw GPU compute, becomes the headline spec hardware vendors market against, since DS4’s 81GB requirement is already reshaping which machines buyers consider adequate for local AI.
- Regulated industries such as healthcare and legal services adopt local vision inference faster than general consumers, since the compliance case for keeping images off third-party servers is immediate and concrete.
- The community-maintained ROCm backend remains the most likely of the three to fall behind on release parity, given antirez’s stated lack of direct AMD hardware access.
- Cloud AI providers respond to credible local alternatives with further price cuts on API vision and inference tiers, particularly for single-user and intermittent-use pricing plans.
Frequently Asked Questions
What is DS4, and who built it?
DS4, short for DwarfStar4, is an open source inference engine written by Salvatore Sanfilippo, known online as antirez and best known for creating Redis. It is built specifically to run DeepSeek V4 Flash, and its larger PRO variant, rather than serving as a general-purpose model runner.
What is DeepSeek V4 Flash Vision-Exp?
Vision-Exp is an experimental multimodal build of DeepSeek V4 Flash that DeepSeek released on August 21, 2026. It adds image understanding on top of the existing text and reasoning capabilities, counting images as tokens up to a cap of 384 tokens per image.
Do you need an Apple M5 Max to run DS4?
No. The M5 Max was antirez’s demo machine, but DS4 runs on any Apple Silicon Mac via Metal, on Nvidia GPUs including DGX Spark via CUDA, and on AMD systems such as the Framework Desktop with Strix Halo via ROCm. Available memory matters more than the specific chip brand.
How much memory does DS4 need to run DeepSeek V4 Flash?
The 2-bit quantized build needs roughly 81GB, putting the comfortable range at 96GB to 128GB of unified or system memory. Machines with less memory can stream weights from SSD instead, at a cost in generation speed. Vision support adds about 0.9GiB more for the visual encoder.
Is DS4 free to use, including commercially?
Yes. DS4 is released under the MIT license, and its source code is available on GitHub for anyone to download, modify and deploy, including in commercial projects.
How does DS4 compare to llama.cpp or Ollama?
llama.cpp and Ollama are generalist tools designed to run a wide range of models across many hardware types. DS4 sacrifices that breadth entirely to specialize in one architecture, DeepSeek V4 Flash’s sparse mixture-of-experts design, which lets it use hand-tuned kernels that a generalist runtime can’t justify building for every model it supports.
What generation speed can I expect from DS4?
On antirez’s published benchmarks with a 128GB M5 Max, DS4 generated at 39.35 tokens per second with a short 2,048-token context and 27.64 tokens per second at a full 65,536-token context, both running the 2-bit quantized DeepSeek V4 Flash model.
Is DeepSeek V4 Flash Vision-Exp as capable as cloud multimodal models?
DeepSeek has stated that Vision-Exp’s multimodal performance approaches Anthropic’s Claude Opus 4.8, though it remains an experimental release rather than a finished, stable version. Independent third-party benchmarking of the vision capability specifically had not been widely published as of this report.
