Local AI
Two weeks of real traffic on our Jetson fleet
What the first two weeks of real requests looked like on our rack of Jetson Orin Nano boards, read from the live feed behind our /fleet page.
On October 1, 2026, we added our fleet page (opens in a new tab) to offlinebase.com. It is a live page that shows real traffic served by our own rack of NVIDIA Jetson Orin Nano boards, the computer inside Base Router. Base Router is still in development. For now, this rack is where you can watch its hardware do real work.
The counters start on September 30, the day before the page went up.
The October 14 reading
Every figure in this section comes from one reading of the live feed behind /fleet, taken on October 14, 2026, at 21:45 UTC. The page refreshes every 5 seconds, so it will show higher numbers by the time you read this.
Reading | Value at 21:45 UTC, October 14 |
|---|---|
Boards | 5 in the rack, 4 online and serving |
Requests since September 30 | 6,449 |
Tokens since September 30 | 4,172,463 |
Tokens handed to the cloud | 0 |
Requests today (since midnight UTC) | 227 |
Tokens today | 196,856 |
Power for the whole fleet | 12.1 W, idle |
One board was offline when we took the reading. The 12.1 W is the page's whole-fleet figure, summed from the four boards that were online. For scale, NVIDIA rates a single Jetson Orin Nano Super (opens in a new tab) at 7 to 25 W.
The cloud counter has read 0 since September 30.
Daily requests, September 30 to October 13
The page's 30-day chart draws one bar of tokens per day. The feed behind it also counts requests for each day:
- September 30, the first day: 271 requests and 67,490 tokens.
- October 1, the day the fleet page went live: 657 requests.
- October 8, the busiest day so far: 931 requests and 509,529 tokens.
- Most days: ten of the fourteen full days landed between 353 and 453 requests.
By our own arithmetic, the fourteen full days from September 30 to October 13 averaged about 444 requests and about 284,000 tokens a day. Across all 6,449 requests, the average request carried about 650 tokens. About 63% of all tokens were prompt tokens (what callers sent in) and about 37% were completion tokens (what the boards wrote back).
The feed behind the page holds only counts, model names and power readings, so it shows how busy each day was.
What is on the boards
Model | What it is | Boards |
|---|---|---|
LFM2.5-VL-3B | Liquid AI's 3.1B vision-language model, announced August 12, 2026, run as a 4-bit GGUF (Q4_K_M) | 2 |
Gemma 4 E2B | Google's open-weight model, about 2.3B effective parameters, run as a 4-bit GGUF (Q4_0) | 2 |
EmbeddingGemma 2 router | Gemma Gemini's router: decides whether a Gemma Gemini message goes to a Jetson or to the cloud | 1 |
The page also lists the base EmbeddingGemma 2 file (Q8_0) on one board, where it sat in storage, unloaded, at our reading.
LFM2.5-VL-3B (opens in a new tab) reads images as well as text. Liquid built it by pairing a language model with a vision encoder, 3.1B parameters in all, with a 32,768-token context and support for 16 languages.
Gemma 4 E2B (opens in a new tab) is Apache 2.0 licensed and generates about 25 to 29 tokens per second on one Orin Nano. It handles the local side of Gemma Gemini, our free chat, and the site lists it as the default everyday model for Base Drive, which is coming soon.
Since October 13 the EmbeddingGemma 2 router has picked, for each Gemma Gemini message, between Gemma 4 E2B on a Jetson and Gemini 4 Argon in the cloud. The Gemma Gemini post covers its accuracy and speed.
The page's "Small model first" list pairs a small board model with a bigger cloud sibling. Gemma 4 E2B, on two boards, pairs with Gemini 3.8 Flash. Nemotron 3 Nano 4B pairs with Nemotron 3.5 Lightning. Nemotron 3 Nano 4B is still waiting for a board, so the cloud answers it for now. The fleet's own pairs are routed by Liquid AI's d1-3B, which has run on a Jetson since October 12 and sits outside the page's model list. How a decision model picks between our Jetsons and the cloud explains how a chat gets handed off.
How to read the page
Four tiles sit across the top:
- Tokens on the boards today counts from midnight UTC. If any of the day's tokens went to a cloud model, the note under it gives the share that stayed on the boards.
- All time counts tokens on the boards since September 30.
- Right now shows tokens per second and how many chats are in progress.
- Power for the whole fleet shows watts. While the boards are answering, it also shows tokens per watt-hour. At our reading no chats were running, so the tile showed only watts.
Above the tiles sit a count of boards online and the note "Updates every 5 seconds". Below them are the 30-day chart, a list of the models on the boards now, and the "Small model first" pairs.
The page makes two promises, and it handles one more thing honestly:
- Every number comes straight from the boards. The page fills every tile from a live feed.
- Cloud tokens stay out of board totals. Tokens handed to a bigger cloud model get their own counter, separate from the board numbers. On the chart they would stack on top of each day's bar as their own layer, labeled "Handed to the cloud". So far that layer is empty.
- When the feed stops, the page says so. If the feed stops answering, the page shows "The fleet isn't answering right now" in place of the numbers.
As the page itself notes, these are live readings that move with the models, the length of each chat and how busy the fleet is. We cover what tokens per watt-hour measures, and its limits, in The energy case for local AI, with our caveats.
Free keys for building on the fleet
The fleet accepts requests in the same format as OpenAI's API, so any app or library that takes a base URL and a key can send chats to these boards. We give out free keys with a daily allowance to people who want to try building on small, local hardware. Use the "Ask for a key" link on our fleet page (opens in a new tab).
Chats sent with those keys show up in the same public counters as everyone else's.
For the board itself, Inside the Jetson Orin Nano Super covers the hardware, and Base Router, the always-on AI box we are designing covers the product we plan to build around it.