Fastino Labs released GLiNER2.5-Decide on September 24, 2026, an open-weight decision model built to run on ordinary CPU hardware instead of a GPU cluster. The model carries 340 million parameters according to Fastino’s release materials and its Hugging Face model card, and it ships under the Apache 2.0 license at fastino/GLiNER2.5-Decide. The pitch is narrow on purpose: instead of generating open-ended text, GLiNER2.5-Decide takes a block of text plus a set of user-defined questions or rules and returns a structured decision, complete with probability distributions and confidence scores.
That framing matters because most of 2026’s AI headlines have been about bigger models chasing bigger benchmarks. GLiNER2.5-Decide runs the other direction. Fastino’s own benchmark shows a p50 latency of 167.3 milliseconds at batch size 1 and 64 tokens on a 48-vCPU Intel Xeon Platinum 8581C, no GPU required. For teams watching GPU and memory prices climb through most of this year, a model that runs acceptably on a CPU is a different kind of news than another leaderboard win.
What GLiNER2.5-Decide Actually Does
GLiNER2.5-Decide is not a chatbot and it is not meant to write essays. It is a classification and decision engine. A developer feeds it a passage of text along with a schema of typed questions, such as “does this support ticket mention a refund request” or “does this contract clause include a termination-for-convenience right.” The model returns a structured answer with a confidence score attached, rather than a paragraph of generated prose.
Fastino Labs described the release directly in its announcement: “Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model,” the company wrote in a post on X. The company followed with a second line describing the intent behind the release: “GLiNER2.5-Decide is built for fast, deterministic classification.”
That word, deterministic, is doing real work. Generative LLMs are famous for producing slightly different answers to the same prompt on different runs, which is a liability in workflows like compliance screening, content moderation, or routing logic where a business needs the same input to reliably produce the same decision. Fastino’s blog post frames the release the same way: “Today, we release GLiNER-2.5-Decide, a 340M-parameter open weight model for schema-defined decision-making,” the company said, adding that the model is “post-trained specifically for structured decision-making.”
Inside the Architecture: Encoder-Based, Not a Chatbot
The technical distinction between GLiNER2.5-Decide and a model like GPT-6 or Claude comes down to architecture. Decoder-based LLMs generate text one token at a time, predicting the next word based on everything that came before it. Encoder-based models like GLiNER2.5-Decide instead read the whole input at once and map it to a fixed output, in this case a decision plus a confidence score. That single-pass structure is a big part of why the model can hit sub-200ms latency on a CPU: there is no long autoregressive loop to run.
The name traces back to GLiNER, an open-source project for zero-shot named entity recognition originally published on GitHub by researcher Urchade Zaratiana and collaborators. GLiNER popularized the idea of an encoder model that can recognize entity types it was never explicitly trained on, just by describing them at inference time. GLiNER2.5-Decide extends that same “describe what you want, get a structured answer” pattern from entity tagging into full decision-making, which is a meaningfully bigger scope than the original NER-focused releases in the GLiNER family.
The Benchmark Numbers: 167ms on CPU, Under 50ms on GPU
Fastino published latency figures across five hardware configurations, and the spread is the most concrete evidence yet that this class of model is genuinely CPU-viable rather than CPU-tolerant. On a 48-vCPU Intel Xeon Platinum 8581C, the company recorded a p50 latency of 167.3 milliseconds at batch size 1 with 64 tokens of input. Move to a GPU and the numbers drop sharply: 43.6 milliseconds on an Nvidia T4, 43.4 milliseconds on an L4, 38.3 milliseconds on a V100, and 47.3 milliseconds on an A100, all under the same reported test conditions.
Fastino’s own blog post states the CPU figure plainly: “At batch 1 and 64 tokens, p50 latency was 167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C.” Read against the GPU numbers, the CPU path runs roughly 3.5 to 4.4 times slower than the GPU options tested, which is a real gap but a survivable one for a huge range of use cases that do not need sub-50ms response times, like batch document review or asynchronous ticket triage.
| Hardware Target | Reported p50 Latency | Compute Type | Batch Size / Tokens |
|---|---|---|---|
| Intel Xeon Platinum 8581C (48 vCPU) | 167.3 ms | CPU | 1 / 64 |
| Nvidia A100 | 47.3 ms | GPU | 1 / 64 |
| Nvidia T4 | 43.6 ms | GPU | 1 / 64 |
| Nvidia L4 | 43.4 ms | GPU | 1 / 64 |
| Nvidia V100 | 38.3 ms | GPU | 1 / 64 |
Source: Fastino Labs release benchmarks, published on the company’s blog on September 24, 2026.
A Minimal Deployment Pattern
Fastino says the model runs through a package called gliner2, which supports both CPU and GPU execution and, according to the company, can be deployed in air-gapped environments. Fastino has not published a full API reference alongside the model card, so treat the snippet below as a conceptual sketch of the workflow rather than verified sample code.
# conceptual install path described in Fastino's release notes
pip install gliner2
# general shape of a schema-defined decision workflow:
# 1. load the fastino/GLiNER2.5-Decide weights from Hugging Face
# 2. define typed questions or rules as a schema
# 3. pass input text plus the schema to the model
# 4. read back a structured decision with a confidence scoreApache 2.0 Licensing and Air-Gapped Deployment
GLiNER2.5-Decide ships under the Apache 2.0 license, one of the more permissive open-source terms available, allowing commercial use, modification, and redistribution without a copyleft obligation. That is a notably different licensing posture than some recent open-weight releases that attach usage caps tied to revenue. Shattered.io reported in November that GLM-5.3’s terms trigger a revenue-share clause once a deployment crosses a $10 billion threshold. Fastino’s Apache 2.0 terms carry no such ceiling.
The company also states that GLiNER2.5-Decide can run in air-gapped environments, meaning it does not need to phone home to a hosted API to function. Combined with CPU support, that positions the model for regulated industries and government workloads where sending data to an external inference endpoint is a non-starter regardless of how fast that endpoint is.
The 340M vs 355M Parameter Question
Not every number in Fastino’s own materials lines up. The company’s release announcement and its Hugging Face model card both list GLiNER2.5-Decide at 340 million parameters, and that is the figure repeated in the company’s own quotes. But Fastino’s models listing page shows a parameter count of 355 million for the same model, a discrepancy that has not been resolved in any of the company’s public statements as of this writing.
It is a small gap in absolute terms, roughly 4 percent, and it likely reflects a rounding or a counting-methodology difference between how the release notes and the models page tally parameters. Still, in a year when parameter-count claims have become a point of scrutiny across the industry, a company shipping mismatched numbers on its own pages is worth flagging rather than smoothing over. Shattered.io will update this figure if Fastino issues a correction.
Why CPU-Only AI Inference Matters Right Now
The timing of a CPU-first model launch is not incidental. GPU capacity has been the industry’s bottleneck for most of 2026, and memory pricing has followed it upward. Intel CEO Lip-Bu Tan said this year that memory costs had risen roughly sevenfold with no relief expected until 2028, a warning that rippled through device makers already absorbing higher bills. Wafer pricing has moved in the same direction, with RAM now accounting for as much as 60 percent of some device bills of materials according to industry reporting cited on this site.
Against that backdrop, a decision model that runs acceptably on a general-purpose Xeon chip instead of a scarce accelerator is a cost argument as much as a technical one. Enterprises running high-volume, low-complexity classification tasks, think support ticket routing, content moderation queues, or invoice field extraction, do not necessarily need a GPU-backed foundation model answering each request. They need a fast, cheap, predictable decision. That is the gap Fastino is aiming GLiNER2.5-Decide at, and it is also the gap that CPU vendors like Intel and AMD have been racing to fill with chips positioned for inference rather than training, a trend visible in recent comparisons such as Intel’s Xeon 6+ launch and its head-to-head positioning against AMD’s Epyc line.
Where GLiNER2.5-Decide Sits in the Open-Weight Landscape
Open-weight releases have piled up through 2026, but most of them have been general-purpose language models competing on chat and coding benchmarks. GLiNER2.5-Decide is a narrower bet: a small, task-specific model competing on latency and deployment cost rather than raw capability. The table below places it alongside other open-weight releases shattered.io has covered this year, not as a head-to-head benchmark comparison, since these models serve different purposes, but as a snapshot of how differently companies are approaching “open weight” right now.
| Model | Maker | Primary Use Case | License | Notable Detail |
|---|---|---|---|---|
| GLiNER2.5-Decide | Fastino Labs | Schema-defined decisions / classification | Apache 2.0 | 340M params, 167.3ms CPU p50 latency |
| DeepSeek V4.1 Flash | DeepSeek | General-purpose LLM | Open weight | Uncensored variant reported to have topped 2,254 daily downloads |
| Smaug models | Abacus.AI | Agentic task execution | Open weight | Reported to cut agent-run costs by up to 100x versus larger models |
| GLM-5.3 | Zhipu AI | Coding assistant | Open weight with revenue-share clause | License triggers a rule at $10B in deployment revenue |
| K2 Horizon family | IFM | General-purpose LLM suite | Open source | Six models released together, up to 375B parameters |
Source: shattered.io reporting on each respective release; figures as previously published on this site.
Competitive Comparison: Decision Models vs General-Purpose LLMs
It is tempting to compare GLiNER2.5-Decide against a large chat model on cost per token, but that comparison misses the point of the release. A 340M-parameter encoder is not trying to write marketing copy or debug a stack trace. It is trying to answer a bounded question fast and cheaply, over and over, at high volume. General-purpose LLMs can technically do that job too, by prompting them to output a yes/no or a JSON object, but they carry the overhead of a much larger network and the unpredictability of free-form generation even when constrained.
That is also why GLiNER2.5-Decide is more directly comparable to agentic infrastructure tooling than to chatbots. Cloud providers have spent 2026 building runtime layers meant to cut the cost and latency of running many small AI decisions inside a larger pipeline, an approach visible in AWS’s AgentCore Runtime v2, which the company says cuts cold starts significantly for agent workloads. A CPU-native decision model slots into exactly that kind of pipeline as a fast pre-filter or router, deciding which requests need to be escalated to a larger, more expensive model and which can be resolved immediately.
Historical Context: From Zero-Shot NER to Schema-Defined Decisions
The GLiNER lineage did not start as a decision engine. The original open-source GLiNER project focused on zero-shot named entity recognition, letting a model tag entities like “person,” “organization,” or a custom category it had never explicitly been trained to detect, just by naming that category at inference time. That project drew a following among developers who wanted NER without the cost of running a full LLM for tagging tasks, and its code remains available on GitHub.
GLiNER2.5-Decide represents the next step in that lineage: instead of tagging spans of text with entity labels, it evaluates whole passages against user-defined questions and returns a decision. The naming makes the connection explicit, and Fastino’s framing of the release as built for “fast, deterministic classification” tracks directly with what made the original GLiNER project useful in production settings, speed and predictability over generative flexibility.
Market and Enterprise Impact
For enterprise teams, the practical impact of GLiNER2.5-Decide will hinge less on its benchmark numbers and more on how easily it slots into existing pipelines. A model that ships on Hugging Face under Apache 2.0, with no usage cap and no mandatory hosted API, removes several of the procurement hurdles that slow enterprise AI adoption, legal review of licensing terms, data residency concerns, and vendor lock-in among them.
The CPU-first positioning also plays into a broader capacity story. Nvidia’s GPUs remain backlogged for large training runs and high-throughput inference, and companies chasing every available accelerator have driven up prices across the board. A model that does its job on hardware most companies already own, standard server CPUs, sidesteps that queue entirely for the subset of workloads it is built for. It will not replace GPU-backed foundation models for open-ended tasks, but it does not need to. It only needs to handle the narrower slice of decision and classification work that currently gets routed to much larger, much more expensive models by default.
What Fastino Labs Is Saying
Fastino Labs has kept its public statements focused on the technical framing of the release rather than market positioning. In its launch post on X, the company wrote: “Introducing GLiNER2.5-Decide, our new 340M parameter open weight, encoder-based decision model” (Fastino Labs, via X).
The company followed that with a description of intent: “GLiNER2.5-Decide is built for fast, deterministic classification” (Fastino Labs, via X).
On its company blog, Fastino elaborated on the model’s purpose: “Today, we release GLiNER-2.5-Decide, a 340M-parameter open weight model for schema-defined decision-making” (Fastino Labs company blog).
And on the architecture itself, the company wrote: “GLiNER2.5-Decide is an encoder-based decision model post-trained specifically for structured decision-making” (Fastino Labs company blog).
Predictions: Where Small Decision Models Go From Here
- Expect more encoder-first, task-narrow releases through early 2027 as GPU and memory costs keep enterprise buyers price-sensitive about which workloads actually need a large decoder model.
- Fastino will likely need to resolve the 340M-versus-355M parameter discrepancy publicly, either through a corrected model card or an explanation of the counting method, before larger enterprise buyers treat the release as fully audited.
- Cloud inference providers will probably add GLiNER2.5-Decide or similar decision models as pre-filter options inside agent pipelines, following the same cost-routing logic already visible in tools like AWS AgentCore.
- Competitors building coding- and chat-focused open-weight models, including the makers of DeepSeek V4.1 Flash and the Smaug family, may face pressure to publish narrower, cheaper variants aimed specifically at classification and routing tasks.
- Licensing terms will keep diverging: Apache 2.0 releases with no revenue ceiling, like this one, sit in growing contrast with usage-capped open-weight licenses tied to deployment revenue thresholds.
The Bottom Line
GLiNER2.5-Decide is a small release by parameter count, but it lands at a moment when small, cheap, and deterministic is exactly what a lot of enterprise AI budgets are asking for. Fastino Labs has published real latency figures, an unrestricted license, and a Hugging Face listing anyone can pull and test today. The unresolved parameter-count mismatch is worth watching, and so is whether the broader market treats CPU-native decision models as a genuine category or a niche next to the much larger CPU-versus-accelerator fight already playing out in the data center. Either way, this is one of the more concrete signals yet that not every AI release in 2026 is chasing another parameter record.
Frequently Asked Questions
What is GLiNER2.5-Decide?
It is an open-weight, encoder-based decision model released by Fastino Labs on September 24, 2026. It takes text plus user-defined questions or rules and returns structured decisions with confidence scores, rather than generating open-ended text.
How many parameters does GLiNER2.5-Decide have?
Fastino’s release announcement and Hugging Face model card list 340 million parameters. The company’s models listing page separately shows 355 million for the same model, a discrepancy that has not been publicly resolved.
Can GLiNER2.5-Decide run without a GPU?
Yes. Fastino reported a p50 latency of 167.3 milliseconds on a 48-vCPU Intel Xeon Platinum 8581C, and the model also runs on GPUs including the Nvidia T4, L4, V100, and A100 with lower recorded latencies.
What license does GLiNER2.5-Decide use?
It is released under the Apache 2.0 license, which permits commercial use, modification, and redistribution without a usage cap or revenue-share clause.
Where can I download GLiNER2.5-Decide?
The weights are hosted on Hugging Face under fastino/GLiNER2.5-Decide, and the model runs through a package called gliner2 according to Fastino’s release notes.
Is GLiNER2.5-Decide related to the original GLiNER project?
Yes. The name traces back to the open-source GLiNER project for zero-shot named entity recognition, published on GitHub. GLiNER2.5-Decide extends that same schema-driven approach from entity tagging into broader structured decision-making.
How does GLiNER2.5-Decide compare to a large language model like GPT-6 or Claude?
It is not a competitor for open-ended chat or generation. It is a narrower, cheaper tool for high-volume classification and decision tasks, and it is more comparable to routing or pre-filter tools used inside larger AI pipelines than to a general-purpose chatbot.
Can GLiNER2.5-Decide run in air-gapped environments?
Fastino states that it can, which would let organizations run the model without any connection to an external hosted API, a relevant feature for regulated industries and government deployments.
