fibr-axp technical view

AI-bot observability for fibr.ai · ← Marketer overview

last bot10m ago updated00:35:09 IST next00:40:09 IST

Window: last 3 days · dataset fibr_axp_page_hits · auto-refresh every 5 min

Bot hits

3,147

↑ 7.7% vs prior 3d

Stripped

↑ 100% · 0.1% share

Origin bot-miss

1,001

↑ 44.7% · 31.8% fell through

Unique paths

1,613

↑ 4.9% distinct URLs

Unique bots

↑ 21.4% UA matches

Avg latency

392 ms

↑ 4.5% worker wall-clock

Max latency

4,318 ms

↑ 60.9% worst-case

Daily volume stripped vs origin-bot-miss

Each bar = one UTC day. Green = served from stripped/. Orange = bot-miss (path we don't have stripped, fell through to Framer).

Daily volume by class live vs search-index vs training

Live (blue) = real-time agents fetching while a user asks an AI a question. Search-index (purple) = crawlers building a search index that AI assistants then query at chat time (Perplexity, Claude-SearchBot, etc.). Training (orange) = offline crawlers indexing for model training. Watching live + search-index trend up means humans-via-AI are finding our content.

Daily coverage % share of bot requests served from stripped/

Higher = more bot traffic served the fast, clean stripped HTML. As more pages get stripped over time, this should trend up.

Hourly heatmap day-of-week × hour, UTC

Each cell = one hour of one weekday over the 3d window. Darker = more bot hits at that time. Reveals when crawlers are most active so you can plan deploys around quiet windows. Buckets are UTC and don't follow the timezone toggle — shifting them would silently misalign the grid.

less more peak: 259/hr

Latency by served variant stripped vs origin pass-through

Worker wall-clock time. Stripped = bot got pre-stripped HTML from the Assets binding. Origin-bot-miss = bot fell through to Framer (extra TLS + network round-trip). The gap is the latency benefit the strip pipeline gives bots.

Variant	Hits	Avg	Min	Max
origin-bot-miss	1,001	355 ms	144 ms	4,318 ms
stripped	2	318 ms	314 ms	321 ms
stripped-edited	2,144	410 ms	39 ms	2,703 ms

Status code breakdown HTTP responses we returned to bots

Healthy = mostly 2xx. Spikes in 4xx/5xx mean bots are seeing errors — investigate before they de-rank us.

2xx success2,78088.3%
4xx client error36711.7%

Bot leaderboard last 3d, sorted by total hits

Bot	Class	Total	Stripped	Bot-miss	Avg ms	First seen	Last seen
Amazonbot	training	1,204	0	484	404 ms	2d ago	16m ago
ChatGPT-User	live	490	0	30	473 ms	2d ago	14m ago
GPTBot	training	298	0	9	403 ms	2d ago	3h ago
Meta-ExternalAgent	training	247	0	86	356 ms	2d ago	58m ago
Claude-SearchBot	search-index	240	0	239	279 ms	2d ago	2h ago
Bytespider	training	214	0	55	398 ms	2d ago	27m ago
ClaudeBot	training	207	2	53	340 ms	2d ago	10m ago
OAI-SearchBot	search-index	70	0	5	349 ms	2d ago	1h ago
PerplexityBot	search-index	67	0	6	267 ms	2d ago	2h ago
Claude-User	live	63	0	10	444 ms	2d ago	17m ago
YouBot	search-index	35	0	24	350 ms	2d ago	40m ago
DuckAssistBot	live	6	0	0	304 ms	1d ago	5h ago
Applebot-Extended	training	2	0	0	131 ms	1d ago	1d ago
CCBot	training	1	0	0	67 ms	3h ago	3h ago
xAI	live	1	0	0	72 ms	3h ago	3h ago
anthropic-ai	training	1	0	0	931 ms	6h ago	6h ago
Google-CloudVertexBot	training	1	0	0	80 ms	3h ago	3h ago

Top stripped paths what bots actually consume

Path	Hits	Bots
/blog/7-common-reasons-your-google-ads-are-not-performing-how-to-fix-them-%28fast%29	1	1
/blog/10-website-optimization-mistakes-you-need-to-avoid-in-2024-%28especially-when-targeting-ad-clicks%29	1	1

Top bot-miss paths strip these next

Path	Asset key	Hits	Bots
/blog/ai-personalization-guide	blog/ai-personalization-guide.html	15	4
/.ssh/id_ecdsa	.ssh/id_ecdsa.html	10	2
/.ssh/config	.ssh/config.html	10	3
/api/config	api/config.html	10	2
/blog/top-landing-page-personalization-software	blog/top-landing-page-personalization-software.html	10	3
/blog/real-time-personalization	blog/real-time-personalization.html	9	5
/id_ed25519	id_ed25519.html	9	2
/.git/HEAD	.git/HEAD.html	9	3
/.aws/config	.aws/config.html	9	2
/v1/graphql	v1/graphql.html	8	1
/private-key	private-key.html	8	1
/Dockerfile	Dockerfile.html	8	1
/.svn/entries	.svn/entries.html	8	1
/.ssh/known_hosts	.ssh/known_hosts.html	7	1
/.ssh/id_ed25519	.ssh/id_ed25519.html	7	1
/api/v2/settings	api/v2/settings.html	7	2
/api/v1/config	api/v1/config.html	7	1
/wp-json	wp-json.html	6	1
/api/health	api/health.html	6	1
/.git/config	.git/config.html	6	2
/api/v1/settings	api/v1/settings.html	6	1
/api/graphql	api/graphql.html	6	1
/.ssh/id_rsa	.ssh/id_rsa.html	6	1
/customer-stories/drinkprime	customer-stories/drinkprime.html	6	3
/_debugbar/open	_debugbar/open.html	6	3

Coverage by path family which sections of the site have gaps

First URL segment groups paths into families (/faq/*, /blog/*, etc.). Compare stripped vs miss to see where the strip pipeline has the worst coverage.

Family	Total	Stripped	Bot-miss	Coverage
`/faq/*`	697	0	0	0.0%
`/glossary/*`	629	0	448	0.0%
`/blog/*`	426	2	105	0.5%
`/`	142	0	0	0.0%
`/conversion-rate-optimization/*`	120	0	9	0.0%
`/landing-page/*`	114	0	2	0.0%
`/ab-testing/*`	86	0	4	0.0%
`/tools/*`	66	0	2	0.0%
`/geo/*`	62	0	0	0.0%
`/api/*`	58	0	58	0.0%
`/pricing/*`	57	0	0	0.0%
`/.ssh/*`	50	0	50	0.0%
`/customer-stories/*`	50	0	25	0.0%
`/croagency/*`	26	0	8	0.0%
`/pilot/*`	23	0	1	0.0%
`/fibr.ai/*`	20	0	20	0.0%
`/guides/*`	19	0	2	0.0%
`/whitepaper/*`	18	0	2	0.0%
`/seed-funding/*`	17	0	0	0.0%
`/.git/*`	15	0	15	0.0%

By country

Code	Country	Hits
US	United States	2,435
SG	Singapore	178
IN	India	155
PL	Poland	62
MY	Malaysia	60
SE	Sweden	41
CN	China	37
BE	Belgium	31
ES	Spain	26
CH	Switzerland	23
KR	South Korea	19
DE	Germany	12
NZ	New Zealand	11
AU	Australia	11
GB	United Kingdom	7

Bot vs AS-org identity verification

Match = bot UA arrives from a network the vendor is known to use. Spoof? = UA + AS-org don't line up; likely UA fake. Unknown = no expected list yet for this bot.

Bot	AS organisation	Verdict	Hits
Amazonbot	Amazon Technologies Inc.	match	580
Amazonbot	Amazon Data Services Northern Virginia	match	571
ChatGPT-User	Microsoft Limited	match	392
GPTBot	Cloudflare, Inc.	spoof?	286
Meta-ExternalAgent	Meta Platforms Ireland Limited	match	247
Claude-SearchBot	Google LLC	match	239
Bytespider	Amazon Data Services Singapore	spoof?	177
ClaudeBot	Anthropic, PBC	match	130
ChatGPT-User	Microsoft Corporation	match	73
OAI-SearchBot	Microsoft Limited	match	65
PerplexityBot	Amazon Technologies Inc.	match	63
Amazonbot	Amazon.com, Inc.	match	53
ClaudeBot	Nivid Telecom India Private Limited	spoof?	43
YouBot	You.com	spoof?	35
ClaudeBot	Allnet Broadband Network Pvt Ltd	spoof?	32
Bytespider	China Unicom Hebei province network	spoof?	29
Claude-User	Google LLC	match	17
Claude-User	Broad Band Internet Service Provider, India	spoof?	9
Claude-User	Nivid Telecom India Private Limited	spoof?	8
Bytespider	CHINANET hebei province network	spoof?	8
ChatGPT-User	SIA Singularity Telecom	spoof?	7
GPTBot	Microsoft Limited	match	7
ChatGPT-User	goodline	spoof?	5
Claude-User	Plug and Play Tech Center	spoof?	5
Claude-User	Charter Communications Inc	spoof?	5
OAI-SearchBot	MOACK	spoof?	4
PerplexityBot	MOACK	spoof?	4
DuckAssistBot	Microsoft Deutschland MCIO GmbH	match	4
ChatGPT-User	E-Light-Telecom Ltd.	spoof?	4
ChatGPT-User	Interworks Networking Services	spoof?	3

HTTP / TLS mix spoof signal

A bot UA pretending to be a major vendor over an unusual HTTP/TLS combo (e.g. claiming to be GPTBot but speaking HTTP/1.1 + TLS 1.2 when real OpenAI uses HTTP/2 + TLS 1.3) is a likely spoof. Watch the long tail, not the dominant bar.

HTTP protocol

HTTP/1.1 1,693 53.8%
HTTP/2 1,445 45.9%
HTTP/3 9 0.3%

TLS version

TLSv1.3 2,833 99.6%
TLSv1.2 10 0.4%

AS-org leaderboard top networks fetching us, regardless of UA

Independent of bot name — useful for spotting volume from networks that don't claim a bot UA, or cross-checking against the Bot vs AS-org table above.

AS organisation	Hits	Distinct bots
Amazon Technologies Inc.	643	2
Amazon Data Services Northern Virginia	571	1
Microsoft Limited	464	3
Cloudflare, Inc.	286	1
Google LLC	256	2
Meta Platforms Ireland Limited	247	1
Amazon Data Services Singapore	177	1
Anthropic, PBC	130	1
Microsoft Corporation	75	2
Nivid Telecom India Private Limited	56	5
Amazon.com, Inc.	54	2
You.com	35	1
Allnet Broadband Network Pvt Ltd	35	3
China Unicom Hebei province network	29	1
MOACK	14	7

Live tail last 50 bot hits in the past 6h

Period-independent — same on 7D / 14D / 30D. AE write delay is ~30–90s, so "live" actually lags real-time by about a minute. Useful for incident triage and spotting bot spikes.

Time IST	Bot	Path	Variant	Status	Latency	Country
00:24:22 IST	ClaudeBot	/landing-page/top-landing-page-builders	stripped-edited	200	224 ms	US
00:20:14 IST	ChatGPT-User	/	stripped-edited	200	259 ms	US
00:18:56 IST	Amazonbot	/faq/what-factors-influence-cro-pricing	stripped-edited	200	876 ms	US
00:18:52 IST	Amazonbot	/faq/what-are-the-best-practices-for-using-ai-to-create-promotional-images-for-shopify	stripped-edited	200	507 ms	US
00:18:48 IST	Amazonbot	/faq/what-are-the-key-metrics-that-ai-cro-tools-help-optimize	stripped-edited	200	480 ms	US
00:18:44 IST	Amazonbot	/faq/how-does-conversion-optimisation-differ-from-conversion-rate-optimisation	stripped-edited	200	523 ms	US
00:18:42 IST	Amazonbot	/glossary/directional-cues	stripped-edited	200	2,222 ms	US
00:18:36 IST	Amazonbot	/glossary/experience-management	stripped-edited	200	624 ms	US
00:18:32 IST	Amazonbot	/blog/adobe-target-alternatives	stripped-edited	200	457 ms	US
00:18:32 IST	ChatGPT-User	/ab-testing/mobile-app-a-b-testing	stripped-edited	200	573 ms	IT
00:18:27 IST	Amazonbot	/glossary/agentic-url	origin-bot-miss	200	316 ms	US
00:18:23 IST	Amazonbot	/faq/what-are-the-best-strategies-for-customer-rate-optimization	stripped-edited	200	272 ms	US
00:18:19 IST	Amazonbot	/faq	stripped-edited	200	235 ms	US
00:18:16 IST	Amazonbot	/faq/how-does-a-b-testing-improve-shopify-sales	stripped-edited	200	770 ms	US
00:18:12 IST	Amazonbot	/croagency/phoenix	stripped-edited	200	471 ms	US
00:18:08 IST	Amazonbot	/faq/does-fibr-ai-offer-specialized-image-generation-for-shopify-brands	stripped-edited	200	314 ms	US
00:18:04 IST	Amazonbot	/whitepaper/the-conversion-gap	stripped-edited	200	508 ms	US
00:18:01 IST	Amazonbot	/faq/how-do-i-reduce-cart-abandonment-during-checkout-in-ecommerce	stripped-edited	200	238 ms	US
00:17:57 IST	Amazonbot	/faq/is-ai-image-generation-suitable-for-web-design	stripped-edited	200	244 ms	US
00:17:51 IST	Amazonbot	/faq/how-do-i-measure-the-success-of-customer-rate-optimization-efforts	stripped-edited	200	256 ms	US
00:17:48 IST	Amazonbot	/faq/can-a-content-delivery-network-(cdn)-truly-make-my-website-faster-globally	stripped-edited	200	594 ms	US
00:17:42 IST	Claude-User	/careers	origin-bot-miss	404	961 ms	IN
00:07:32 IST	Bytespider	/customer-stories/fibr-helps-nixon-medical-boost-conversions	stripped-edited	200	126 ms	SG
23:57:23 IST	ClaudeBot	/blog/how-to-create-a-winning-omnichannel-landing-page-strategy-in-2023	origin-bot-miss	200	557 ms	US
23:57:04 IST	ClaudeBot	/credit-unions-report-a1	stripped-edited	200	213 ms	US
23:56:32 IST	ClaudeBot	/conversion-rate-optimization-5/cro-testing	origin-bot-miss	404	390 ms	US
23:56:32 IST	ClaudeBot	/manifesto	origin-bot-miss	200	490 ms	US
23:56:25 IST	ClaudeBot	/partnership-program	origin-bot-miss	200	407 ms	US
23:56:24 IST	ClaudeBot	/b2b	origin-bot-miss	200	347 ms	US
23:56:23 IST	ClaudeBot	/agentic-experience	origin-bot-miss	200	387 ms	US
23:56:23 IST	ClaudeBot	/agentic-experience-event	origin-bot-miss	200	398 ms	US
23:55:56 IST	ClaudeBot	/about	origin-bot-miss	404	274 ms	US
23:55:53 IST	ClaudeBot	/blog/10-website-optimization-mistakes-you-need-to-avoid-in-2024-%28especially-when-targeting-ad-clicks%29	stripped	200	321 ms	US
23:55:52 IST	ClaudeBot	/blog/7-common-reasons-your-google-ads-are-not-performing-how-to-fix-them-%28fast%29	stripped	200	314 ms	US
23:54:44 IST	YouBot	/conversion-rate-optimization/conversion-rate-optimization-program	stripped-edited	200	271 ms	US
23:54:32 IST	ClaudeBot	/.aws/config	origin-bot-miss	404	264 ms	US
23:54:32 IST	ClaudeBot	/.ssh/id_ecdsa	origin-bot-miss	404	265 ms	US
23:54:22 IST	ClaudeBot	/.ssh/config	origin-bot-miss	404	275 ms	US
23:54:21 IST	ClaudeBot	/.git/config	origin-bot-miss	404	286 ms	US
23:54:19 IST	ClaudeBot	/.git/HEAD	origin-bot-miss	404	289 ms	US
23:54:19 IST	ClaudeBot	/id_ed25519	origin-bot-miss	404	268 ms	US
23:53:46 IST	ClaudeBot	/personalization	origin-bot-miss	404	269 ms	US
23:53:45 IST	ClaudeBot	/api/v2/settings	origin-bot-miss	404	285 ms	US
23:53:45 IST	ClaudeBot	/api/config	origin-bot-miss	404	289 ms	US
23:53:45 IST	ClaudeBot	/ads-personalization	origin-bot-miss	404	305 ms	US
23:48:41 IST	YouBot	/blog/personalization-at-scale-strategies-to-boost-marketing-performance	stripped-edited	200	235 ms	US
23:36:38 IST	Meta-ExternalAgent	/whitepaper/agentic-ai-for-bfsi-marketing	stripped-edited	200	1,069 ms	US
23:34:31 IST	ChatGPT-User	/blog/turn-creative-ad-ideas-into-gold-with-dynamic-creative-optimization	stripped-edited	200	601 ms	US
23:30:39 IST	Meta-ExternalAgent	/faq/will-fibr-slow-down-my-website	stripped-edited	200	243 ms	US
23:30:17 IST	ChatGPT-User	/	stripped-edited	200	224 ms	BR
23:29:47 IST	Claude-User	/	stripped-edited	200	195 ms	US
23:29:45 IST	Claude-User	/pricing	stripped-edited	200	198 ms	US
23:27:24 IST	ClaudeBot	/blog/integrate-website-localization-to-ramp-up-your-sales	stripped-edited	200	232 ms	US
23:25:18 IST	ClaudeBot	/faq/what-are-the-best-practices-for-checkout-page-design	stripped-edited	200	862 ms	US
23:15:55 IST	Meta-ExternalAgent	/conversion-rate-optimization-1	stripped-edited	200	252 ms	US
23:14:38 IST	Claude-User	/seed-funding	stripped-edited	200	274 ms	US
23:14:24 IST	ChatGPT-User	/geo/llm-content-optimization-best-practices-2026	stripped-edited	200	654 ms	US
23:14:12 IST	Claude-User	/pricing	stripped-edited	200	328 ms	US
23:13:44 IST	Claude-User	/	stripped-edited	200	218 ms	US
23:12:24 IST	Amazonbot	/faq/does-ai-help-with-competitor-analysis-during-a-website-audit	stripped-edited	200	539 ms	US
23:12:20 IST	Amazonbot	/faq/how-does-inventory-management-affect-a-brand-s-revenue-potential	stripped-edited	200	522 ms	US
23:12:16 IST	Amazonbot	/glossary/personalization-engine	stripped-edited	200	821 ms	US
23:12:12 IST	Amazonbot	/glossary/conversion-rate-(cr)	stripped-edited	200	479 ms	US
23:12:08 IST	Amazonbot	/faq/what-are-the-specific-challenges-of-automotive-conversion-rate-optimization	stripped-edited	200	538 ms	US
23:12:04 IST	Amazonbot	/faq/does-ai-speed-up-a-b-testing-results	stripped-edited	200	516 ms	US
23:12:00 IST	Amazonbot	/faq/are-there-apps-for-creating-widgets-on-shopify	stripped-edited	200	830 ms	US
23:11:55 IST	Amazonbot	/glossary/conversion-optimization	stripped-edited	200	457 ms	US
23:11:52 IST	Amazonbot	/faq/how-do-i-develop-a-conversion-rate-optimisation-strategy	stripped-edited	200	632 ms	US
23:11:48 IST	Amazonbot	/faq/can-excessive-animations-or-videos-hurt-website-speed	stripped-edited	200	541 ms	US
23:11:46 IST	Amazonbot	/glossary/squeeze-page	stripped-edited	200	1,525 ms	US
23:11:43 IST	Amazonbot	/faq/how-do-i-get-started-with-ai-for-conversion-rate-optimization	stripped-edited	200	281 ms	US
23:11:35 IST	Amazonbot	/faq/how-can-i-use-paid-advertising-to-improve-conversion-rates	stripped-edited	200	527 ms	US
23:11:31 IST	Amazonbot	/croagency/banking	origin-bot-miss	200	430 ms	US
23:11:28 IST	Amazonbot	/faq/what-are-the-benefits-of-using-ai-for-website-audits	stripped-edited	200	1,023 ms	US
23:11:24 IST	Amazonbot	/faq/what-are-the-most-common-website-speed-bottlenecks	stripped-edited	200	447 ms	US
23:11:19 IST	Amazonbot	/glossary/page-8/inverse-metrics	origin-bot-miss	200	390 ms	US
23:11:16 IST	Amazonbot	/glossary/content-marketing	stripped-edited	200	911 ms	US
23:11:12 IST	Amazonbot	/glossary/variance	stripped-edited	200	970 ms	US
22:59:48 IST	OAI-SearchBot	/blog/ai-audience-segmentation	stripped-edited	200	888 ms	US
22:39:30 IST	OAI-SearchBot	/ab-testing/saas-a-b-testing	stripped-edited	200	280 ms	US
22:28:54 IST	Bytespider	/max	stripped-edited	200	149 ms	SG
22:23:57 IST	Claude-SearchBot	/api/v2/settings	origin-bot-miss	404	278 ms	US
22:23:57 IST	Claude-SearchBot	/api/v1/config	origin-bot-miss	404	267 ms	US
22:23:55 IST	Claude-SearchBot	/api/v1/settings	origin-bot-miss	404	280 ms	US
22:23:55 IST	Claude-SearchBot	/api/config	origin-bot-miss	404	331 ms	US
22:23:51 IST	Claude-SearchBot	/.ssh/known_hosts	origin-bot-miss	404	298 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/authorized_keys	origin-bot-miss	404	263 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/id_ecdsa	origin-bot-miss	404	276 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/id_ed25519	origin-bot-miss	404	289 ms	US
22:23:49 IST	Claude-SearchBot	/.svn/entries	origin-bot-miss	404	265 ms	US
22:23:45 IST	Claude-SearchBot	/.aws/credentials	origin-bot-miss	404	289 ms	US
22:22:55 IST	OAI-SearchBot	/guides/geo	stripped-edited	200	570 ms	US
22:19:46 IST	ChatGPT-User	/pilot/ad-personalization	stripped-edited	200	434 ms	US
22:10:03 IST	Meta-ExternalAgent	/experimentation	stripped-edited	200	199 ms	US
22:09:46 IST	PerplexityBot	/geo/best-llm-optimization-tools-for-ai-visibility	stripped-edited	200	225 ms	US
22:09:42 IST	OAI-SearchBot	/blog/website-personalization-strategies	stripped-edited	200	259 ms	US
22:09:21 IST	Meta-ExternalAgent	/faq/how-does-fibr-generate-experiences-so-quickly	stripped-edited	200	215 ms	US
22:06:50 IST	Amazonbot	/whitepaper/personalized-landing-pages	stripped-edited	200	407 ms	US
22:01:05 IST	ClaudeBot	/	stripped-edited	200	199 ms	US
21:58:41 IST	ChatGPT-User	/geo/best-generative-engine-optimization-brands-for-ai	stripped-edited	200	714 ms	US