"…an empty list, not a guess."— Cactus Compute, Needle 3 readme, on what happens when no tool fits
§1The live test
One messy sentence in. Three exact actions out.
What you're watching: I typed the exact messy sentence from the test, and Needle 3 returned fan on, temperature 10 and bedroom light on in 68 milliseconds on my Mac.
real run on my Mac · replayed at reading pace
THINKING IT?"This just sounds like a smart home gadget."
The light switch is only the demo.
The same trick pulls the fields out of an invoice or an enquiry, and that's the part your business can use.
§2The speed
Sixty-six milliseconds. On a CPU.
No graphics cardNo internetNo server farmJust a normal CPU
§3The size
Smaller than the apps on your phone
Apache 2.0FreeOpen source11,000+ GitHub stars
§4What it actually does
One file. Three jobs. All offline.
§5Job one
Tool calls: right tools, right order, or nothing
What you're watching: two requests in one sentence come back as two calls in order, and a question no tool covers comes back as an empty list instead of a made-up answer.
real run on my Mac · replayed at reading pace§6Job two
Structured extraction: messy text in, clean fields out
What you're watching: I declared vendor, total and due date, handed it an invoice description, and got three clean fields back in 75 milliseconds.
real run on my Mac · replayed at reading pace§7Job three
Embeddings: search and match without sending anything
What you're watching: eight tools turned into numbers once, then each question matched on the device in under 3 milliseconds. It got 4 of 5 right in my test, and I left the miss in.
real run on my Mac · replayed at reading pace§8The honest trade
No chat. On purpose.
It's not broken. That's the design.
§9The benchmarks
Six test suites. Thousands of rows.
Cactus's published numbers§10The handicap
The tiny one was handicapped and still won its category
Cactus's published numbers
DroidCall: 47.0 — top on-device modelExtraction: level with 2–3x bigger modelsBFCL v4: 50.2 — mid-pack, not a win
§11On a Raspberry Pi
A computer that costs less than a nice dinner
§12Quick pause
"Cool, but I'm not a developer"
THINKING IT?"I'm not a developer, what do I do with this?"
You're not going to compile this yourself, and you don't need to.
You need to understand the shift, because it changes what you build and what you pay for this year.
What you're watching: the real Agent OS dashboard, where every agent plugs into one place and shares one memory.
THINKING IT?"Doesn't running an Agent OS burn a fortune in tokens?"
No. The everyday work runs on free local models and free APIs, and the frontier work drives the CLIs you already pay for, like the Claude CLI inside your Claude subscription.
Inside the Boardroom there are token-efficiency tutorials too, so you learn to cut usage and stop thinking about it.
§14The bigger story
Needle 3 isn't alone. Meet Jev.
§15The shift
Models that don't talk. They act.
§16The problem
The 2am JSON Problem
OLD WAY
~2 seconds · metered · breaks sometimes
A giant model writes a reply one word at a time
Something has to dig the JSON out of the reply
One extra sentence before the JSON breaks the workflow
Every call goes over the internet and gets billed
You find out it broke at 2am
NEW WAY
~66 ms · free to run · always parses
A 35 MB model outputs the structure directly
Every token has to fit the shape you asked for
It can't invent a field that doesn't exist
It runs on the device with no internet
The big expensive model only gets called for thinking
THINKING IT?"My automations work fine most of the time."
Most of the time is the problem, because the one broken run is the one a customer sees.
This is a reliability gain more than a speed gain, and reliability is what makes automations survive past week two.
§17Why it can't break
Every token has to fit your shape
Can't hallucinate a fieldCan't hand you broken dataA reliability gain, not a speed gain
§18The cleverest design choice
Intelligence laddering: 19 models in one file
What you're watching: I built three rungs on my Mac with one command each, and the 2-layer rung came out at 13.34 megabytes.
real run on my Mac · replayed at reading pace§19Where the parameters live
Most of it is memory, not maths
Cactus's published numbers§20Fine-tuning
Tune any rung. 29 million parameters beat a cloud model.
Cactus's published numbers§21What this means for a normal business
Narrow the job enough, and small wins
What you're watching: an example enquiry goes in, and the name, company and budget come out in 71 milliseconds with no API call.
real run on my Mac · replayed at reading pace
THINKING IT?"I don't have a big dataset to train anything."
You don't need one, because the job is narrow.
A few hundred examples of your own enquiries is the whole dataset.
Engine floor: 0.1Not a probability guess — a calibrated numberMost setups run everything or nothing
§24Triggers
Intents that must never be missed
What you're watching: the same vague requests scored 0.63 and 0.70 without triggers, then 1.0 and 0.89 once I attached the words turn, switch, power and flip to the tool.
real run on my Mac · replayed at reading pace§25The honest part
Four real problems when people pushed it
THINKING IT?"If it breaks, why would I bother with it?"
Because every one of these four has a simple fix, and I show each one below.
A specific tool with known edges is easier to trust than a big one that fails at random.
§26Problem one
History leaks — so reset between requests
What you're watching: office to 21 degrees, then a pizza, then the capital of France. On engine 3.0.2 no pizza call was invented, but confidence dipped without a reset, so I reset anyway.
real run on my Mac · replayed at reading pace
Reported by testers: an invented pizza callCactus shipped an engine updateMy run on 3.0.2: zero invented calls
§27Problem two
Give every argument a sensible default
What you're watching: the same booking request was held back at 0.28 confidence with no defaults, then ran at 0.95 the moment I added them.
real run on my Mac · replayed at reading pace§28Problem three
Messy text has limits
real run on my Mac · replayed at reading pace
My extra finding: "$1,500" came back as 1200"1500 dollars" came back rightThe shape of what you feed it matters
§29Problem four
It's weaker at pure classification
Specific tools are the ones that actually hold up.
§30Where it runs
A watch. A television. A car. A browser tab.
§31One practical note
Telemetry is on by default
export NEEDLE_TELEMETRY=0
export DO_NOT_TRACK=1 # I ran every test in this guide with both set
§32What this means going forward
For two years the direction was bigger
§33The investment
360 billion tokens of one kind of thing
§34The takeaway
The right size model for the right job
how the pieces fit · concept diagram
What you're watching: the real Agent OS, where every agent reads from the same shared memory.
§35Day to day
Every enquiry, filed before anyone reads it
§36If you're thinking this is too technical
Three beliefs to drop
Wrong: "Bigger models are always better."
Right: A 121 million parameter model scored 86.0 on phone commands and a 1.2 billion one scored 82.4, because its job was narrow.
Wrong: "Every AI job has to go through a paid API."
Right: Read this, pick that, fill these fields can run on the device, and the expensive model only gets called when it's needed.
Wrong: "This is all too technical for me."
Right: The install is one line. The concepts took me longer, and you now have them.
Plenty of members had never used AI at all
Members post their wins every day — agency owners, ecom founders, course creators and solo operators across 38 countries, in their own words.
Get the AI agent system — small models on the cheap jobs, big agents on the thinking.
The Agent OS, the 30-day roadmap and the video walkthrough live inside the AI Profit Boardroom, with four coaching calls a week to get your setup sorted live.
