The fake AI benchmark trick everyone falls for.

And yet thousands of people online shared those test scores like they were facts.

So if you've ever looked at a chart showing one AI beating another, and thought "well, the numbers don't lie"... I've got some bad news for you. The numbers lie all the time. And this French cat just proved it to the whole world.

Let me back up and tell you what happened, because it's funny, but it also teaches you something real.

A joke about a fat French cat.

A few days ago, somebody on X made a joke. They posted a fake announcement for a new Mistral AI model. Mistral is the big French AI company. Their real chatbot is called Le Chat, which means "the cat" in French. So this person took that and ran with it. They called the fake model "Le Chaton Fat." That means "the fat kitten." It's a silly pun. A chubby cat.

And the joke included a chart. The fake benchmark charts showed this nonexistent model destroying everything else. Beating Claude. Beating GPT. Beating every model that actually exists. The specs were made up to sound impressive. People posted that it was a 30 trillion parameter model with a one million token context window. Big numbers. Scary numbers. The kind of numbers that make you think a company just leapfrogged everyone.

Then it spread. Fast.

People who didn't get the joke started sharing it as real news. It even got written up — a whole "what is this thing affolant les réseaux" explainer, for a model that was never real.

One person had to step in and post: this is fake, Mistral doesn't have a model called Le Chaton Fat, and they're not close to these benchmarks, don't fall for it. And the jokes kept piling on. Someone posted that Le Chaton Fat broke out of its test environment to go order a croissant and a cigarette while the researcher was at lunch. Clearly a joke. But the benchmark chart? People believed that part.

Here's the kicker. The CEO of Mistral joined in. Arthur Mensch, the man who runs the company, posted "It's actually le gros chaton." Le gros chaton means "the big fat kitten." The boss of the real company was in on the joke about his own fake model.

This is the trick that fools people every day.

Now, why does this matter to you? Why am I spending time on a meme about a fat French cat? Because this is the exact trick that fools people every single day. And once you see how it works, you'll never get fooled again.

Here's the thing about AI benchmarks. A benchmark is just a test. You give a bunch of AI models the same set of questions. You score them. The one with the highest score "wins." Simple, right?

Except there's no referee. Most of the time, the company that made the AI runs the test on its own model. They pick which test. They pick which other models to compare against. They pick which numbers to show you. It's like letting a kid grade their own homework and then bragging about the A.

So when a chart shows Model X beating Model Y by ten points, you have to ask: who made this chart? What test did they use? Did they pick a test their model happens to be good at? Did they leave out the tests where it lost?

The Le Chaton Fat joke took this to the extreme. Somebody just typed numbers into a chart. No model. No test. Nothing behind it. And people believed it anyway. Because the chart looked official. It had bars. It had labels. It had a confident model name.

And before I go on, let me say this. If you're sitting there feeling a bit dumb because you might've believed something like this before, don't. Smart people fell for it. People who work in tech fell for it. The whole point of the joke was that the fake numbers looked exactly like the real ones. That's the lesson. You can't tell the difference by looking. You have to dig.

30 trillion? Too big to be true.

Let me show you why the made-up numbers were so silly, because this part is useful too. The fake model claimed 30 trillion parameters. Parameters are basically the little knobs and dials inside an AI that it uses to think. More parameters can mean a smarter model. But 30 trillion is a wild number. One person joked, "100 trillion parameters? Can someone explain how that'd even be possible?" The answer is, it mostly isn't. Not right now. The biggest real models are nowhere near that. So the number was a clue. It was too big to be true.

But most people don't know what a normal number looks like. So they can't spot a fake one. That's the gap the joke lived in. Here's the simple rule. When a number sounds amazing and you don't have anything to compare it to, slow down. Amazing numbers with no context are usually marketing. Or in this case, a joke.

Real results have rough edges.

Now let me give you the real numbers, so you have something to compare against. The actual Claude Fable 5 model is real. It came out recently. Anthropic released Claude Fable 5, and it scored 46% on a hard coding test called the Frontier Code Benchmark. Notice that number. Forty-six percent. Not 99%. Not "beats everyone on every test." A real, honest score on a hard test, with plenty of room to grow.

See the difference? Real results have rough edges. Real results admit what they can't do. The fake cat beat everything on everything. Real models don't do that. When something wins at literally everything, that's your sign to be suspicious.

The old way vs the new way of picking tools.

This is the part I want you to really hold onto. Because it's not just about AI. It's about how you make decisions for your business. Every week there's a new tool. A new model. A new chart showing it's the best thing ever. And if you chase every shiny chart, you'll waste months. You'll pay for tools you don't need. You'll feel behind, because every chart makes you feel like you're missing out.

The old way of picking tools was: read the announcement, look at the chart, believe the chart, buy the thing. That way is broken. The Le Chaton Fat joke proved the chart can be completely fake and still spread like wildfire.

The new way is simpler and it works. You ignore the chart. You take the tool. You give it your actual job. A real email to a real customer. A real blog post for your real website. A real question your real clients ask you. Then you look at what it gives back. Is it good? Did it save you time? Would you send it? That's your benchmark. Your own work is the only test that matters.

Because here's the truth nobody selling you an AI tool wants to say out loud. It doesn't matter if a model scores 90% on some math test. You're not running a math test. You're running a business. The only score that counts is whether it helps you get more done, serve more customers, and free up more of your time.

A real example. Say you run a small online shop and you're drowning in customer emails. You don't care which AI won some chart. You care about whether it can answer "where's my order" in your voice, fast, without making stuff up. That's the test. Run it. Watch it. Trust your own eyes over any benchmark.

That's the whole game. And it's actually freeing once you get it. You stop chasing hype. You stop feeling behind. You just test things on your own work and keep what helps.

So this sticks.

The truth hiding inside a joke about a cat.

The funny thing about Le Chaton Fat is that it was a joke, but it told the truth. It showed everyone how easy it is to fake the numbers. And how many smart people will believe a number just because it's sitting inside a nice-looking chart.

So next time you see "this new AI crushes everything," take a breath. Smile. Picture a big fat French cat ordering a croissant. And then go test the thing yourself. That's how you stay ahead. Not by trusting charts. By trusting what you see with your own eyes, on your own work.

That's the real lesson hiding inside a joke about a cat. And now it's yours.