Ternary Bonsai 2 2B, 4B, 9B: Smallest Files Need PrismML Builds

Ternary Bonsai 2 2B, 4B and 9B, ternary versions of Qwen3.5 released under Apache 2.0, are out as of October 15, 2026, according to @PrismML. The headline sizes of 0.43, 0.95 and 2 GB are only one of three GGUF packings, though: PrismML's download docs say that smallest file needs PrismML's own llama.cpp binaries, while the Q2_0 file that loads in stock llama.cpp b11555 or newer is 0.55, 1.21 and 2.56 GB. Our take: for local assistants on small hardware these models are worth using, as long as you know which file your runtime can actually load.

What's new in Ternary Bonsai 2 2B, 4B and 9B

PrismML's announcement describes three models based on Qwen3.5 2B, 4B and 9B, with weights encoded as −1, 0 or +1. They join Bonsai 2 27B and succeed the first-generation Bonsai 1.7B, 4B and 8B, so the family now has four sizes. PrismML says the release adds stronger reasoning, visual understanding and tool calling, and that it tuned for avoiding repetitive loops, following instructions, calling tools on mobile devices and controlling verbosity.

PrismML evaluated the models on 19 benchmarks across six skill sets and publishes a table against full-precision Qwen3.5 of the same size. The 9B and 4B also ship optional speculative-decoding drafters: an MTP draft head built into a separate PQ2_0 file, and a DFlash2 block drafter. Both run only on PrismML's binaries.

What you actually download

The docs list three GGUF packings of the same ternary weights, plus an MLX pack for Apple silicon. The GGUF columns below are language-model file sizes only; the vision projector is a separate extra download for GGUF. The MLX pack is a single directory that already includes the FP16 vision tower, so do not add the projector to it.

PTQ1_0 is the "dense trits, the smallest file" and is the source of the 0.43 / 0.95 / 2 GB figures. PQ2_0 stores each trit in a 2-bit slot, which is cheaper to unpack, so prompt processing is faster; it is the pack the demo downloads and the one PrismML measured on Apple silicon. PTQ1_0 is the faster decode on Ada-generation GPUs and the L4, per the docs. Q2_0 is the one for stock llama.cpp. The docs show the same split for the 27B: our earlier Bonsai 2 27B article reports 5.9 GB, and the docs list the 27B at 5.95 GB for PTQ1_0 against 7.63 GB for Q2_0.

Against the Qwen3.5 sizes in PrismML's table, the PTQ1_0 files work out to 8.7× (2B), 8.9× (4B) and 9.0× (9B) smaller. The announcement says "more than 9×", which the table supports only at the best case. The Q2_0 files that stock llama.cpp loads are about 6.8–7.0× smaller. We left the MLX packs out of this comparison because they include an FP16 vision tower, and what the Qwen3.5 figures include remains unclear.

Image input means a second download on GGUF. The projector is loaded only when an image arrives, so a 9B on stock llama.cpp that handles images is 2.56 + 0.62 = 3.18 GB on disk; with PTQ1_0 it is 2.00 + 0.62 = 2.62 GB.

What runs Bonsai 2 today

Support depends on the file, not just the model. Bonsai 2 weights sit in a rotated basis, so a runtime must apply the matching Hadamard transform and the model's sign flips. The formats and runtime page summarises where that exists:

The trap is old builds. Versions before b11555 know the Q2_0 type and the Qwen3.5 architecture but lack the Hadamard runtime, so nothing stops them loading the file; the docs say a runtime like that "loads the file and produces gibberish." If a Bonsai 2 model outputs nonsense, check the build first. PQ2_0 and PTQ1_0 fail safely on upstream because they are refused outright. For apps that bundle their own engine, the docs advise running the model through the demo's server and pointing the app at that endpoint until the app updates.

Platform coverage is wider than the post's mention of NVIDIA via llama.cpp and Apple via MLX. The PrismML binaries cover CPU, Metal, CUDA, ROCm/HIP, Vulkan and SYCL, with archives for macOS (Apple silicon and Intel), Linux x64 and arm64, Windows x64 and arm64, Android arm64 and an iOS XCFramework. On Vulkan and SYCL, kernel selection depends on the device and driver. On a Mac either format works: the docs say decode speed is comparable, with PrismML's M4 Pro measurements putting llama.cpp Metal slightly ahead, and GGUF is more portable across tools. Everywhere else they recommend GGUF.

The quickest route to a working local server is PrismML's demo repository. Its default model is the 27B, so set the size first:

export BONSAI_MODEL=9B   # 2B, 4B, 9B or 27B
./setup.sh
./scripts/start_llama_server.sh

The server then runs at http://localhost:8080. Setup fetches the PQ2_0 file, its vision projector and matching PrismML binaries, and on Apple silicon the MLX pack. It also installs Open WebUI and a code interpreter, which add a few GB; the repo says you can skip them with BONSAI_OPENWEBUI=0 and BONSAI_CODE_INTERPRETER=0. If you call the model from your own client, the repo says the 2B, 4B and 9B profiles use min-p 0 and a 64-token presence-penalty window.

Where Bonsai 2 trails Qwen3.5

These are PrismML's own evaluations. The category averages in its table look like this:

Image credit: @PrismML on X

The 9B is within a point of Qwen3.5 9B on coding (80.49 vs 80.41) and instruction following (73.51 vs 73.49), and about 1.7 points behind on math (92.06 vs 93.71). It trails further on knowledge and reasoning (74.46 vs 80.10), tool calling (74.28 vs 80.36) and vision (72.17 vs 77.99). PrismML's blog table lists overall retention of 93.0%, 92.4% and 92.5% for those three categories. For an assistant that calls tools or reads images, those are the categories to test on your own tasks.

The weakest individual results sit inside those categories. On Tau2Bench the 4B scores 61.87 against 81.65 (75.8%) and the 9B 73.02 against 85.25 (85.7%). GPQA Diamond is the 2B's biggest gap at 39.6 against 58.48 (67.7%). On OmniDoc retention is 77.5% (2B), 82.3% (4B) and 83.2% (9B), so document-heavy vision work deserves a check before you commit. The BFCL tool-calling tests are closer, at 93.5–98.1% retention across the three sizes.

Two reading cautions. First, category averages cover different benchmarks at different sizes: the 2B has no Tau2Bench or AIME26 score, and the 4B has no AIME26 score. The 2B's tool-calling average is just BFCLv3 and BFCLv4 Mobile (60.53 and 61.85, averaging 61.19). Our read: the 9B's math average is lower than the 4B's in the image (92.06 vs 96.07), which fits the 9B being the only size with an AIME26 score (82.5). Compare each Bonsai model with its own Qwen3.5 column, not across sizes.

Second, the 2B's retention above 100% is not evidence that ternary compression helps. GSM8K is 80.74 against 65.96 (122.4%) and IFBench 48.00 against 34.33 (139.8%). Our read: those gaps mostly reflect weak Qwen3.5 2B baselines rather than a benefit of ternary compression.

How to try it

The weights are on Hugging Face in the Bonsai 2 Small collection under Apache 2.0. Repositories follow prism-ml/Ternary-Bonsai-2-{SIZE}-gguf and prism-ml/Ternary-Bonsai-2-{SIZE}-mlx-2bit, and no token is needed for public repos.

On iPhone or iPad, the App Store listing for Locally AI by LM Studio says version 1.66.0 added the new Ternary Bonsai 2 models. The app is free, runs on Apple MLX and requires iOS 18.1 or later. PrismML reports up to 34 tokens per second for the 9B on an iPhone 18 Pro Max and up to 120 for the 2B, and shows the 9B doing agentic tasks on the phone.

To try the 9B without downloading anything, PrismML says it is available on Darkbloom, with 250 million free tokens for the first 100 users. PrismML also says it is running an optimisation challenge with Darkbloom on Yukon Research to make the 9B faster.

Who should care

Our view: this is a good fit if you want a local assistant on a laptop, phone or modest GPU and can use PrismML's builds or a llama.cpp from b11555 onward. The Q2_0 files for the 4B (1.21 GB) and 9B (2.56 GB) are roughly 7× smaller than the Qwen3.5 sizes in PrismML's table (8.41 and 17.91 GB), which matters on memory-limited hardware. Be more careful if your workload leans on multi-step tool use, document vision or hard reasoning, where the gaps are widest, and if you depend on LM Studio or Ollama, check which engine version your app bundles before downloading anything.

Sources

• @PrismML

• PrismML — Introducing Ternary Bonsai 2 2B, 4B, and 9B: Bringing Local Agents to Everyday Devices

• Download the Models - Bonsai

• Formats & Runtime Support - Bonsai