The traffic you can't see

Last week we told the story of "The open that wasn't" — twelve newsletter subscribers we thought were engaged readers, all twelve turning out to be automated: Apple's Mail Privacy Protection, Gmail's image proxy, Outlook's link scanner, a corporate security bot. Zero confirmed humans behind twelve opens and a click.

We closed that piece saying we'd changed what we trust the data to tell us. What we didn't say is what we did next: we asked whether this was a newsletter problem, or whether it was bigger than that.

It's bigger than that.

Beltmar's own website tracking had a basic bot filter since early on — a keyword check against the visitor's browser signature, catching anything that says "bot," "spider," "crawler," or "headless." It worked, as far as it went. It would not have caught a single one of the twelve proxies from last week's post — none of them announce themselves. Apple's privacy relay looks like an ordinary, generic browser. Gmail's proxy is the only one honest enough to name itself, and even then only if you go looking. The rest just look like traffic.

So we went looking somewhere else: what did that same keyword filter do with an AI crawler?

GPTBot. ClaudeBot. PerplexityBot. We ran all three through it.

All three got silently dropped — not because anyone decided AI crawlers shouldn't count, but because they all happen to contain the word "bot," and the filter was written to stop spam, not to make a judgment call about what's worth seeing. It was the same mistake in the opposite direction: where the newsletter problem was traffic that looks human but isn't, this was traffic that is worth seeing, filed under "ignore" by accident.

Which meant that until this week, if an AI answer engine was reading a page to decide whether to cite it in a response to someone's question, that visit was invisible. Not filtered thoughtfully. Not tracked and flagged as low-priority. Gone, the same as a spam bot, because the code never had to consider the difference.

We think that's the actual shape of the problem, once you look at it straight on: it was never really "bots vs. humans." It's at least three categories, and only one of them is noise —

Noise. Scanners, proxies, spam crawlers, the twelve from last week. Should be filtered, and mostly is, once you know to look for the pattern — not just a keyword in a user-agent string, but the actual evidence: a browser version that changes mid-session, an IP that doesn't match the device it claims to be, a click that fires before a human could plausibly have read anything. That's forensic work, not a blocklist.

Answer engines. GPTBot, ClaudeBot, PerplexityBot, and whatever comes after them. Not noise — a real, growing channel, and one almost nobody can currently see, because the tools built to stop spam were never taught the difference. If an AI is reading your content to decide whether to recommend you, that's a signal worth its own number, not a silent drop.

People. The actual thing every metric is supposed to be a proxy for, and the hardest of the three to prove — which is exactly why the first two categories have to be right before this one means anything.

That's not something we're planning to build. As of this week, it's live: every visit now gets classified the moment it happens — noise, answer engine, or unclassified — and a second pass checks whether a "visitor" who looks like one person is actually two, catching the exact pattern Apple's mail proxy left behind twelve times last week: a browser that changes family or jumps version mid-session, seconds apart, which no real person can do. Automated traffic no longer counts toward who's warming up. Answer-engine visits get their own number instead of getting silently dropped as spam — the same accidental mistake we found using our own filter against GPTBot and ClaudeBot.

That's not a metrics feature. It's the actual promise underneath "we tell you who's warming up" — the promise only means something if warming up is real. Now it is.