Starting from version 0.35, Ollama supports decision models built on the Jev API from TypeSafe. These are not chat models in the usual sense: they do not write text, but answer a set of typed questions about the transferred state. For classification, routing, moderation and triage, this turns out to be faster and more reliable than asking a regular LLM to return JSON and then parsing its response.<\/p>\n
In this post, I’ll look at what decision models are, how they work at the HTTP level, what models are already available, and how to build them into existing code so that regular generative models continue to work.<\/p>\n
A regular LLM receives a prompt and generates text token by token. Even if you ask her to answer in one word, generation still occurs inside, and the format of the answer has to be controlled by a diagram, instructions and subsequent verification. The Decision model works differently: you pass state and questions with types, and the model returns probabilities and selected options. No free text, no parsing JSON from the response.<\/p>\n
The new endpoint is called \/v1\/systemone. In one request, you can ask several questions at once, and the model will answer all of them. This is convenient for tasks where you need to make several related decisions: for example, determining the team, the need for a return, and the urgency of the ticket.<\/p>\n
Now you can run three decision models through Ollama:<\/p>\n
They all work locally, do not require payment and do not send data to an external service. Ollama 0.35 and later are supported.<\/p>\n
First check the Ollama version:<\/p>\n
ollama --version\n<\/code><\/pre>\n<\/div>\nThen download the desired model. For the easiest one:<\/p>\n
\nollama pull tev1:0.8b\n<\/code><\/pre>\n<\/div>\nFor the full version of tev1:<\/p>\n
\nollama pull tev1\n<\/code><\/pre>\n<\/div>\nFor nimble:<\/p>\n
\nollama pull nimble\n<\/code><\/pre>\n<\/div>\nThe nimble model takes up about 9.5 GB of disk space, so it’s a good idea to clear some space before downloading. tev1:0.8b takes up less than a gigabyte.<\/p>\n
First request via curl<\/h2>\n
The simplest example is to determine whether a text is a job advertisement:<\/p>\n
\ncurl http:\/\/localhost:11434\/v1\/systemone -d '{\n \"model\": \"tev1:0.8b\",\n \"state\": {\n \"message\": \"Senior Swift developer, Berlin, remote friendly\"\n },\n \"questions\": {\n \"isJob\": {\n \"type\": \"noul\",\n \"instructions\": \"Does the message advertise a job posting that someone can apply to?\"\n }\n }\n}'\n<\/code><\/pre>\n<\/div>\nThe answer looks like this:<\/p>\n
\n{\n \"model\": \"tev1:0.8b\",\n \"answers\": {\n \"isJob\": {\n \"type\": \"noul\",\n \"noul\": 0.921\n }\n },\n \"usage\": {\n \"input_tokens\": 42,\n \"output_tokens\": 0\n }\n}\n<\/code><\/pre>\n<\/div>\nThe noul field is a probability from 0 to 1. The decision is made by comparison with a threshold, usually 0.5. Unlike the generative model, there is no need to ask for a response in JSON format and then check that the model did not break it.<\/p>\n
Types of questions<\/h2>\n
Decision models support three types of questions.<\/p>\n
noul<\/b> is a binary question. Returns the probability that a statement is true:<\/p>\n\n{\n \"questions\": {\n \"refund\": {\n \"type\": \"noul\",\n \"instructions\": \"Does the customer explicitly ask for a refund?\"\n }\n }\n}\n<\/code><\/pre>\n<\/div>\nchoice<\/b> – selecting one option from a given set. Returns the selected option, the probabilities of all options and confidence:<\/p>\n\n{\n \"questions\": {\n \"team\": {\n \"type\": \"choice\",\n \"instructions\": \"Which team should handle this ticket?\",\n \"criteria\": {\n \"billing\": \"Payments and refunds\",\n \"technical\": \"Bugs and integrations\",\n \"other\": \"None of the above\"\n }\n }\n }\n}\n<\/code><\/pre>\n<\/div>\nscore<\/b> – rating on a scale. Returns the number, legend, and probabilities for each scale point:<\/p>\n\n{\n \"questions\": {\n \"urgency\": {\n \"type\": \"score\",\n \"instructions\": \"How urgent is this ticket?\",\n \"criteria\": [\"Routine\", \"Soon\", \"Urgent\"]\n }\n }\n}\n<\/code><\/pre>\n<\/div>\nOne request can contain several questions of different types at once. This is what makes decision models convenient for triage: one operation, one state, several independent decisions.<\/p>\n
Python without additional libraries<\/h2>\n
For requests, the standard urllib is sufficient; a separate SDK is not required:<\/p>\n
\nimport json\nfrom urllib.request import Request, urlopen\n\nOLLAMA_DECISION_URL = \"http:\/\/localhost:11434\/v1\/systemone\"\n\ndef ask_decision(state, questions, model=\"tev1:0.8b\"):\n payload = {\n \"model\": model,\n \"state\": state,\n \"questions\": questions,\n }\n request = Request(\n OLLAMA_DECISION_URL,\n data=json.dumps(payload).encode(),\n headers={\"Content-Type\": \"application\/json\"},\n method=\"POST\",\n )\n with urlopen(request, timeout=3600) as response:\n return json.loads(response.read().decode())[\"answers\"]\n<\/code><\/pre>\n<\/div>\nUsage:<\/p>\n
\nanswers = ask_decision(\n {\"message\": \"Senior Swift developer, Berlin, remote friendly\"},\n {\n \"isJob\": {\n \"type\": \"noul\",\n \"instructions\": \"Does the message advertise a job posting that someone can apply to?\"\n }\n },\n)\n\nprobability = answers[\"isJob\"][\"noul\"]\nis_job = probability >= 0.5\n<\/code><\/pre>\n<\/div>\nIf you need an official Python SDK, it is called typesafe-sdk. For localhost, it can be directed to Ollama with the environment variables TYPESAFE_BASE_URL, TYPESAFE_API_KEY and TYPESAFE_DEFAULT_MODEL. But for simple tasks, a direct HTTP request is sufficient.<\/p>\n
How to understand that a model supports decision<\/h2>\n
Decision models cannot respond via \/api\/generate or \/api\/chat. If you send such a request, Ollama will return a 400 error with text like:<\/p>\n
\n\"tev1:0.8b\" does not support generate\n<\/code><\/pre>\n<\/div>\nThis is important if you already have code that works with regular models via \/api\/generate: if you change the model to a decision model, it will stop working. It\u2019s better to find out the model\u2019s capabilities in advance via \/api\/show and choose the appropriate calling method:<\/p>\n
\nimport json\nfrom urllib.request import Request, urlopen\n\nOLLAMA_SHOW_URL = \"http:\/\/localhost:11434\/api\/show\"\n\ndef supports_decision(model):\n request = Request(\n OLLAMA_SHOW_URL,\n data=json.dumps({\"model\": model}).encode(),\n headers={\"Content-Type\": \"application\/json\"},\n method=\"POST\",\n )\n try:\n with urlopen(request, timeout=30) as response:\n capabilities = json.loads(response.read().decode()).get(\"capabilities\", [])\n except (OSError, TypeError, ValueError):\n return False\n return \"decision\" in capabilities\n<\/code><\/pre>\n<\/div>\nDecision models will have only decision in their capabilities. Regular models will have completion, tools or other features there. If Ollama is not available, it is reasonable to consider the model normal and use the generative path.<\/p>\n
How to embed into existing code<\/h2>\n
The simplest way is not to rewrite the classifier, but to divide it into two strategies. The generative strategy works as before: it collects the prompt, sends it to \/api\/generate and parses the JSON. The decision strategy sends the state and typed question to \/v1\/systemone and compares the probability with a threshold.<\/p>\n
The choice of strategy occurs in one place, when creating a classifier:<\/p>\n
\ndef create_classifier(model):\n if supports_decision(model):\n return DecisionClassifier(model)\n return GenerativeClassifier(model)\n<\/code><\/pre>\n<\/div>\nThis approach has several advantages. Generative models continue to run unchanged. The decision model can be enabled by changing one environment variable. And the entire difference between the two methods is concentrated in two small classes, and is not spread out throughout the application.<\/p>\n
For a binary classification, the decision strategy looks something like this:<\/p>\n
\nasync def classify(text):\n answers = await ask_decision(\n {\"message\": text},\n {\n \"isJob\": {\n \"type\": \"noul\",\n \"instructions\": instructions,\n }\n },\n model,\n )\n probability = answers[\"isJob\"][\"noul\"]\n return probability >= 0.5, probability\n<\/code><\/pre>\n<\/div>\nPlease note that the decision model does not return an explanation. If your result format has a reason field, you can construct it from the probability: for example, store the probability itself or a string like \u201cthe model rated the message at 0.935 as a vacancy.\u201d This is more honest than inventing a reason that the model did not give.<\/p>\n
Instructions should be a question<\/h2>\n
This is the most important practical conclusion. Decision models do not understand long prompts in the style of \u201cyou classify messages, answer true if…, false otherwise.\u201d They want a short, direct question.<\/p>\n
On the same message about a vacancy, I received the following probabilities:<\/p>\n
\n- Direct question “Is this message a job posting?” \u2014 0.921.<\/li>\n
- Long instructions explaining what is considered a vacancy – 0.026.<\/li>\n
- Long instructions with a question at the end – 0.391.<\/li>\n<\/ul>\n
The difference is almost thirty times. The model is not mistaken in classification, it simply does not understand what they want from it when the question is hidden inside a paragraph. Therefore, for decision models it is worth keeping separate short instructions, and leaving long prompts for generative models.<\/p>\n
It is convenient to create a separate environment variable:<\/p>\n
\nOLLAMA_JOB_CLASSIFIER_DECISION_INSTRUCTIONS=Does the message advertise a job posting that someone can apply to?\n<\/code><\/pre>\n<\/div>\nAnd use a regular prompt only for the generative path.<\/p>\n
Threshold and probabilities<\/h2>\n
The threshold of 0.5 is suitable as a default value, but it is useful to include it in the configuration. For different tasks, the cost of a false positive and false negative response is different. For moderation, it is safer to raise the threshold; for searching for vacancies, it is safer to lower it, so as not to miss useful messages.<\/p>\n
It is also useful to log the probability itself, and not just the resulting boolean. When the classifier makes an error, the log shows whether it was a confident answer or a decision on the border. For choice questions, it is worth saving probabilities and confidence: they show how much the model fluctuates between options.<\/p>\n
Performance and memory<\/h2>\n
The speed greatly depends on the size of the model and the hardware. On the CPU in my tests I got the following numbers:<\/p>\n
\n- tev1:0.8b<\/b> – split seconds to solve, easily fits into memory.<\/li>\n
- tev1<\/b> – 4B, noticeably slower, but still suitable for interactive tasks.<\/li>\n
- nimble<\/b> – 9B in Q8_0, about 10 GB in memory. The first request, including loading the model, took about 137 seconds, subsequent solutions took from 12 to 32 seconds on the CPU.<\/li>\n<\/ul>\n
Nimble requires a lot of RAM. On a machine with 16 GB and Docker\/WSL with a limit of about 8 GB, the model does not fit: the system begins to swap and the containers stop responding. Increasing the WSL limit to 12 GB via the .wslconfig file helped:<\/p>\n
\n[wsl2]\nmemory=12GB\nswap=8GB\n<\/code><\/pre>\n<\/div>\nAfter the change, you need wsl –shutdown and restart Docker Desktop. But even with 12 GB, the reserve remains small: the model takes up about 10 GB, and there is not much left for the system.<\/p>\n
By default, Ollama unloads the model after five minutes of inactivity, freeing up memory. This is convenient: nimble does not keep 10 GB busy all the time, but each new launch after a pause starts loading the model again.<\/p>\n
When to use decision models<\/h2>\n
Decision models are well suited for tasks where you need a quick and cheap answer to a clearly formulated question:<\/p>\n
\n- Classification of messages, tickets and requests.<\/li>\n
- Routing by commands or executors.<\/li>\n
- Content moderation and security checks.<\/li>\n
- Fast binary and multivariate solutions in gaming or interactive applications.<\/li>\n<\/ul>\n
They are poorly suited for tasks where detailed text is needed: explanation, field extraction, summarization, answer generation. For this purpose, the usual generative models remain. The best result is obtained when combined: a decision model makes a decision, and a generative model explains it or extracts data.<\/p>\n
Total<\/h2>\n
Decision models in Ollama are a separate, narrower tool than regular LLMs. They don’t replace generative models, but they take away the routine part of the work: typed responses, probabilities instead of text parsing, and a predictable format. If you already have a classification pipeline, the transition comes down to checking the capabilities of the model, a separate calling strategy, and short instructional questions.<\/p>\n
Sources
\nhttps:\/\/ollama.com\/blog\/ollama-now-supports-jev-style-decision-models<\/a>
\nhttps:\/\/ollama.com\/library\/nimble<\/a>
\nhttps:\/\/ollama.com\/library\/tev1<\/a>
\nhttps:\/\/typesafe.ai<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Starting from version 0.35, Ollama supports decision models built on the Jev API from TypeSafe. These are not chat models in the usual sense: they do not write text, but answer a set of typed questions about the transferred state. For classification, routing, moderation and triage, this turns out to be faster and more reliable […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8],"tags":[],"class_list":["post-200","post","type-post","status-publish","format-standard","hentry","category-notes"],"translation":{"provider":"WPGlobus","version":"3.0.6","language":"en","enabled_languages":["en","ru","zh","de","ja","fr","es","pt","hi"],"languages":{"en":{"title":true,"content":true,"excerpt":false},"ru":{"title":true,"content":true,"excerpt":false},"zh":{"title":true,"content":true,"excerpt":false},"de":{"title":true,"content":true,"excerpt":false},"ja":{"title":true,"content":true,"excerpt":false},"fr":{"title":true,"content":true,"excerpt":false},"es":{"title":true,"content":true,"excerpt":false},"pt":{"title":true,"content":true,"excerpt":false},"hi":{"title":true,"content":true,"excerpt":false}}},"_links":{"self":[{"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/posts\/200","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/comments?post=200"}],"version-history":[{"count":0,"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/posts\/200\/revisions"}],"wp:attachment":[{"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/media?parent=200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/categories?post=200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/demensdeum.com\/blog\/wp-json\/wp\/v2\/tags?post=200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}