How a decision model picks between our Jetsons and the cloud

Since October 12, 2026, Liquid AI's open-weight d1-3B has decided which chats on our Jetson fleet get a small model on a board and which go to a bigger cloud model. It runs on a Jetson Orin Nano and answers in one pass, with a probability.

That job comes from our about page (opens in a new tab), which lists five beliefs the company started on. The fifth says frontier and really large models should be "tools to call upon," and that "Routing is the name of the game." Its reasons come down to cost, since most questions do not need a frontier model, and fit: sending each question to a model that is good at it and away from one that is weak at it.

Routing only pays off if the small, open side can take a real share of the work. Belief 1 on the same page points to a Stanford preprint, Intelligence per Watt, in which local models successfully answered 88.7% of one million real single-turn queries. For us, that makes open weights good enough to build on, and the remaining work is deciding, chat by chat, how much model a question needs.

What a decision model is

A chat model writes its reply one token at a time. A decision model reads the input once and picks from answers you define in advance, with a probability attached.

On October 7, 2026, Liquid AI released open weights (opens in a new tab) for two of these: d1-3B and d1-omni-600M. Liquid says the d1 models answer in a single forward pass and produce no tokens.

That shape suits a router. A router needs a short list of possible answers (a board or the cloud) and some sense of how sure the model is. A probability gives it both.

d1-3B, from its model card

d1-3B (opens in a new tab) is "a 3B parameter decision model built on LFM2.5-VL-3B." Its card lists these details:

A yes-or-no question comes back as the probability of "yes." A choice question picks one of the options you name and reports how confident it is. A score question rates something on a scale you set, from 2 to 10 levels.

LFM2.5-VL-3B, its base model, is also one of the chat models on our boards: as of October 14, 2026, the "On the boards now" list on /fleet (opens in a new tab) shows it on two boards. At 1.67 GB as a 4-bit file, d1-3B is small enough for a Jetson Orin Nano and its 8 GB of memory.

Ollama (opens in a new tab) added an API for decision models in version 0.35, announced September 29, 2026. Version 0.40.4, released October 12, 2026, brought them to its command line through an --ask option on ollama run, and updated the llama.cpp it ships with to support decision models natively.

How our fleet's router uses d1-3B

The switch to d1-3B happened on October 12, 2026. Before that, the same router ran on Jev, TypeSafe's decision model, which is what tryingopen (opens in a new tab), our free chat with open-weight models, relied on until then.

When a chat needs a routing call, the router asks d1-3B a few yes-or-no questions about it and gets a probability back for each. It combines those into one decision: a small model on a board answers, or a bigger cloud model does. We keep the exact questions and cutoffs to ourselves, and we tune them as we learn.

Our own hardware decides where each chat goes, using an open-weight model anyone can download. Because the router works from probabilities, we can move a cutoff without retraining anything.

What you can see on /fleet

The public side lives on /fleet (opens in a new tab). It has a list headed "Small model first," where each entry pairs a model that runs on a board with a bigger cloud sibling.

The Nemotron pair has no board yet, so its cloud model answers for now.

The page also spells out the handoff. A chat that is too long or too hard for a small model can be handed to a bigger cloud model, and "those tokens are counted separately and never in the board totals." The 30-day chart splits each day into tokens on the boards and tokens handed to the cloud.

As of 21:45 UTC on October 14, 2026, the cloud total was still 0. Two weeks of real traffic on our Jetson fleet has the full reading.

A smaller decision model, and a different kind of router

d1-3B has a smaller sibling, d1-omni-600M (opens in a new tab) (587M parameters), which reads text along with an image or a short audio clip (up to about 30 seconds). It is the planned model inside Decision Tree, a little bonsai we are designing for the home network. The Decision Tree page (opens in a new tab) explains why the probability matters: "Every answer says how sure it is. It can be wrong, so that number matters." Read Decision Tree, a little bonsai with a decision model inside for more on that one.

Gemma Gemini, our free Gemini-style chat, routes a different way: an embedding model plus a pair of small trained classifiers. It has worked that way since October 13, 2026, with EmbeddingGemma 2 running on a Jetson. The story is in Gemma Gemini, a free chat that shows which model answered and why. As of October 14, 2026, /fleet's "On the boards now" list shows an EmbeddingGemma 2 router (embeddinggemma2-router) on one board.

Where this fits in our products

On the fleet, the router decides by itself. Base Drive and Base Router are designed to ask first. Belief 3 on our about page describes "a better default: local first, with heavier work offloaded to more capable models only when the owner approves it." Why we make local the default for private AI covers how that approval step is designed.

In the site's words, "Outside help is not in the shipping build." As of October 2026, no Offline Base product can be ordered yet. For a sense of how much a small model handles before any handoff, see What a million real questions say about small local models.