Skip to main content

How DebugBear Handles Bot Traffic

Search engine crawlers, AI crawlers, uptime checkers, link scanners and scripted browsers all load your pages. If those visits are recorded as real user monitoring data they would distort your metric and distract for actual visitor experience on your website.

DebugBear tries to discard these page views automatically where they can be identified. You don't need to configure anything, and bot visits don't count towards your RUM page view quota.

What gets discarded​

Bots that identify themselves​

Most automated traffic says what it is in the user agent string. DebugBear checks every page view against a list of known bots, which covers search engine crawlers like Googlebot and Applebot, AI crawlers, SEO tools, uptime monitors and headless browsers.

This also means that page views from DebugBear lab tests won't be recorded as visitor page views.

Automated browsers​

When other heuristics suggest a visit comes from an automated browser we also discard those page views. That way we prevent your own end-to-end tests from polluting RUM data.

Traffic from known bot networks​

Some automated traffic is identified by where it comes from rather than what it claims to be. Requests from IP ranges known to belong to bot and scraper networks are ignored.

If you still see bot traffic​

No detection is perfect, and new bots appear constantly. If you think page views in your dashboard came from automated traffic, get in touch and we'll take a look.

The most useful thing you can send is a link to the specific page views in question. Open the Page Views tab, filter down to the traffic you're suspicious about, and share the URL along with anything that makes you think it isn't a real visitor.

Why your RUM numbers differ from analytics​

Bot filtering is one reason DebugBear may report fewer page views than other analytics data sources. Other factors that can impact page view counts are:

Last updated on