fibr-axp technical view

AI-bot observability for fibr.ai · ← Marketer overview

last bot10m ago updated00:35:07 IST next00:40:07 IST

Window: last 30 days · dataset fibr_axp_page_hits · auto-refresh every 5 min

Bot hits

27,461

↓ 14.6% vs prior 30d

Stripped

↑ 400% · 0.1% share

Origin bot-miss

5,640

↑ 79.9% · 20.5% fell through

Unique paths

3,228

↑ 42.3% distinct URLs

Unique bots

↑ 9.1% UA matches

Avg latency

380 ms

↑ 2.8% worker wall-clock

Max latency

20,050 ms

↓ 87.8% worst-case

Daily volume stripped vs origin-bot-miss

Each bar = one UTC day. Green = served from stripped/. Orange = bot-miss (path we don't have stripped, fell through to Framer).

Daily volume by class live vs search-index vs training

Live (blue) = real-time agents fetching while a user asks an AI a question. Search-index (purple) = crawlers building a search index that AI assistants then query at chat time (Perplexity, Claude-SearchBot, etc.). Training (orange) = offline crawlers indexing for model training. Watching live + search-index trend up means humans-via-AI are finding our content.

Daily coverage % share of bot requests served from stripped/

Higher = more bot traffic served the fast, clean stripped HTML. As more pages get stripped over time, this should trend up.

Hourly heatmap day-of-week × hour, UTC

Each cell = one hour of one weekday over the 30d window. Darker = more bot hits at that time. Reveals when crawlers are most active so you can plan deploys around quiet windows. Buckets are UTC and don't follow the timezone toggle — shifting them would silently misalign the grid.

less more peak: 999/hr

Latency by served variant stripped vs origin pass-through

Worker wall-clock time. Stripped = bot got pre-stripped HTML from the Assets binding. Origin-bot-miss = bot fell through to Framer (extra TLS + network round-trip). The gap is the latency benefit the strip pipeline gives bots.

Variant	Hits	Avg	Min	Max
origin-bot-miss	5,640	376 ms	109 ms	10,509 ms
origin-error	2	206 ms	198 ms	213 ms
stripped	15	1,665 ms	212 ms	20,050 ms
stripped-edited	21,804	380 ms	34 ms	6,138 ms

Status code breakdown HTTP responses we returned to bots

Healthy = mostly 2xx. Spikes in 4xx/5xx mean bots are seeing errors — investigate before they de-rank us.

2xx success24,40588.9%
4xx client error3,05311.1%
5xx server error10.0%
other20.0%

Bot leaderboard last 30d, sorted by total hits

Bot	Class	Total	Stripped	Bot-miss	Avg ms	First seen	Last seen
Amazonbot	training	8,092	1	1,852	416 ms	29d ago	16m ago
ChatGPT-User	live	5,607	0	466	402 ms	29d ago	14m ago
ClaudeBot	training	2,838	2	372	343 ms	29d ago	10m ago
Meta-ExternalAgent	training	2,460	6	1,030	346 ms	29d ago	58m ago
Bytespider	training	2,054	0	342	397 ms	29d ago	27m ago
GPTBot	training	1,998	1	214	343 ms	29d ago	3h ago
OAI-SearchBot	search-index	915	3	223	304 ms	29d ago	1h ago
PerplexityBot	search-index	814	0	152	278 ms	29d ago	2h ago
Claude-User	live	710	2	97	455 ms	29d ago	17m ago
CCBot	training	682	0	51	399 ms	29d ago	3h ago
Claude-SearchBot	search-index	410	0	408	279 ms	21d ago	2h ago
YouBot	search-index	256	0	123	351 ms	29d ago	40m ago
Google-Extended	training	210	0	93	264 ms	18d ago	8d ago
Applebot-Extended	training	195	0	192	259 ms	19d ago	1d ago
DuckAssistBot	live	67	0	0	489 ms	29d ago	5h ago
Google-CloudVertexBot	training	64	0	14	327 ms	29d ago	3h ago
xAI	live	59	0	11	281 ms	29d ago	3h ago
anthropic-ai	training	9	0	0	281 ms	20d ago	6h ago
Claude-Web	live	6	0	0	295 ms	18d ago	18d ago
Perplexity-User	live	5	0	0	254 ms	18d ago	18d ago

Top stripped paths what bots actually consume

Path	Hits	Bots
/blog/10-website-optimization-mistakes-you-need-to-avoid-in-2024-%28especially-when-targeting-ad-clicks%29	5	4
/glossary/north-star-metric-%28nsm%29	4	2
/blog/7-common-reasons-your-google-ads-are-not-performing-how-to-fix-them-%28fast%29	3	3
/blog/a-complete-guide-to-digital-experience-platforms-%28dxps%29	2	2
/faq/what-s-the-best-way-to-document-and-learn-from-shopify-a-b-test-results	1	1

Top bot-miss paths strip these next

Path	Asset key	Hits	Bots
/blog/ai-personalization-guide	blog/ai-personalization-guide.html	94	9
/blog/real-time-personalization	blog/real-time-personalization.html	51	6
/.git/HEAD	.git/HEAD.html	46	11
/.git/config	.git/config.html	41	13
/blog/top-landing-page-personalization-software	blog/top-landing-page-personalization-software.html	41	8
/.aws/credentials	.aws/credentials.html	40	12
/.aws/config	.aws/config.html	39	11
/blog/dxp-vs-cms	blog/dxp-vs-cms.html	38	8
/fetch	fetch.html	35	7
/glossary/ad-to-landing-page-personalization	glossary/ad-to-landing-page-personalization.html	35	6
/manifesto	manifesto.html	35	5
/customer-stories/drinkprime	customer-stories/drinkprime.html	34	9
/tools/adaptive-web-experience	tools/adaptive-web-experience.html	33	10
/api/config	api/config.html	32	11
/api/env	api/env.html	32	12
/.ssh/id_ecdsa	.ssh/id_ecdsa.html	30	8
/blog/website-optimization-mistakes	blog/website-optimization-mistakes.html	30	9
/id_ed25519	id_ed25519.html	29	8
/customer-stories/how-a-home-interiors-platform-got-40-percent-more-leads	customer-stories/how-a-home-interiors-platform-got-40-percent-more-leads.html	29	9
/graphql	graphql.html	29	9
/partnership-program/agency-program	partnership-program/agency-program.html	28	7
/.ssh/config	.ssh/config.html	28	9
/llm	llm.html	28	6
/api/v1/settings	api/v1/settings.html	28	9
/glossary/agentic-experience-layer	glossary/agentic-experience-layer.html	26	7

Coverage by path family which sections of the site have gaps

First URL segment groups paths into families (/faq/*, /blog/*, etc.). Compare stripped vs miss to see where the strip pipeline has the worst coverage.

Family	Total	Stripped	Bot-miss	Coverage
`/faq/*`	6,417	1	0	0.0%
`/`	3,444	0	0	0.0%
`/blog/*`	3,311	10	658	0.3%
`/glossary/*`	3,081	4	1,242	0.1%
`/conversion-rate-optimization/*`	1,264	0	58	0.0%
`/landing-page/*`	1,239	0	20	0.0%
`/ab-testing/*`	978	0	94	0.0%
`/tools/*`	904	0	94	0.0%
`/geo/*`	609	0	5	0.0%
`/api/*`	346	0	346	0.0%
`/customer-stories/*`	331	0	195	0.0%
`/pricing/*`	277	0	0	0.0%
`/@fs/*`	254	0	254	0.0%
`/croagency/*`	243	0	96	0.0%
`/pilot/*`	204	0	3	0.0%
`/conversion-rate-optimization-5/*`	203	0	203	0.0%
`/.ssh/*`	161	0	161	0.0%
`/whitepaper/*`	158	0	67	0.0%
`/seed-funding/*`	150	0	0	0.0%
`/about-us/*`	122	0	1	0.0%

By country

Code	Country	Hits
US	United States	19,255
IN	India	1,955
SG	Singapore	1,709
BE	Belgium	1,085
CN	China	507
DE	Germany	359
AU	Australia	295
PL	Poland	289
CH	Switzerland	272
NL	Netherlands	243
MY	Malaysia	185
ES	Spain	176
BR	Brazil	166
NZ	New Zealand	127
IT	Italy	124

Bot vs AS-org identity verification

Match = bot UA arrives from a network the vendor is known to use. Spoof? = UA + AS-org don't line up; likely UA fake. Unknown = no expected list yet for this bot.

Bot	AS organisation	Verdict	Hits
Amazonbot	Amazon Technologies Inc.	match	3,568
Amazonbot	Amazon Data Services Northern Virginia	match	3,437
ChatGPT-User	Microsoft Corporation	match	2,591
Meta-ExternalAgent	Meta Platforms Ireland Limited	match	2,456
ClaudeBot	Anthropic, PBC	match	2,417
ChatGPT-User	Microsoft Limited	match	2,025
Bytespider	Amazon Data Services Singapore	spoof?	1,540
Amazonbot	Google LLC	spoof?	745
ChatGPT-User	Google LLC	spoof?	650
GPTBot	Nivid Telecom India Private Limited	spoof?	597
OAI-SearchBot	Microsoft Limited	match	594
CCBot	Amazon Data Services Northern Virginia	match	576
PerplexityBot	Amazon Technologies Inc.	match	518
GPTBot	Allnet Broadband Network Pvt Ltd	spoof?	458
Claude-SearchBot	Google LLC	match	408
GPTBot	Microsoft Limited	match	348
Bytespider	China Unicom Hebei province network	spoof?	338
Amazonbot	Amazon.com, Inc.	match	299
GPTBot	Cloudflare, Inc.	spoof?	286
YouBot	You.com	spoof?	256
GPTBot	Google LLC	spoof?	230
Claude-User	Google LLC	match	227
ClaudeBot	Google LLC	match	223
Google-Extended	Google LLC	match	210
PerplexityBot	Google LLC	spoof?	204
OAI-SearchBot	Google LLC	spoof?	203
Applebot-Extended	Google LLC	spoof?	192
Bytespider	CHINANET hebei province network	spoof?	163
ChatGPT-User	Microsoft do Brasil Imp. e Com. Software e Video G	match	85
ClaudeBot	Nivid Telecom India Private Limited	spoof?	69

HTTP / TLS mix spoof signal

A bot UA pretending to be a major vendor over an unusual HTTP/TLS combo (e.g. claiming to be GPTBot but speaking HTTP/1.1 + TLS 1.2 when real OpenAI uses HTTP/2 + TLS 1.3) is a likely spoof. Watch the long tail, not the dominant bar.

HTTP protocol

HTTP/2 14,716 53.6%
HTTP/1.1 12,731 46.4%
HTTP/3 14 0.1%

TLS version

TLSv1.3 26,773 99.2%
TLSv1.2 213 0.8%

AS-org leaderboard top networks fetching us, regardless of UA

Independent of bot name — useful for spotting volume from networks that don't claim a bot UA, or cross-checking against the Bot vs AS-org table above.

AS organisation	Hits	Distinct bots
Amazon Technologies Inc.	4,092	3
Amazon Data Services Northern Virginia	4,016	3
Google LLC	3,325	17
Microsoft Limited	2,983	5
Microsoft Corporation	2,677	6
Meta Platforms Ireland Limited	2,456	1
Anthropic, PBC	2,417	1
Amazon Data Services Singapore	1,540	1
Nivid Telecom India Private Limited	695	5
Allnet Broadband Network Pvt Ltd	529	5
China Unicom Hebei province network	338	1
Amazon.com, Inc.	300	2
Cloudflare, Inc.	298	2
Limestone Networks, Inc.	280	8
You.com	256	1

Live tail last 50 bot hits in the past 6h

Period-independent — same on 7D / 14D / 30D. AE write delay is ~30–90s, so "live" actually lags real-time by about a minute. Useful for incident triage and spotting bot spikes.

Time IST	Bot	Path	Variant	Status	Latency	Country
00:24:22 IST	ClaudeBot	/landing-page/top-landing-page-builders	stripped-edited	200	224 ms	US
00:20:14 IST	ChatGPT-User	/	stripped-edited	200	259 ms	US
00:18:56 IST	Amazonbot	/faq/what-factors-influence-cro-pricing	stripped-edited	200	876 ms	US
00:18:52 IST	Amazonbot	/faq/what-are-the-best-practices-for-using-ai-to-create-promotional-images-for-shopify	stripped-edited	200	507 ms	US
00:18:48 IST	Amazonbot	/faq/what-are-the-key-metrics-that-ai-cro-tools-help-optimize	stripped-edited	200	480 ms	US
00:18:44 IST	Amazonbot	/faq/how-does-conversion-optimisation-differ-from-conversion-rate-optimisation	stripped-edited	200	523 ms	US
00:18:42 IST	Amazonbot	/glossary/directional-cues	stripped-edited	200	2,222 ms	US
00:18:36 IST	Amazonbot	/glossary/experience-management	stripped-edited	200	624 ms	US
00:18:32 IST	Amazonbot	/blog/adobe-target-alternatives	stripped-edited	200	457 ms	US
00:18:32 IST	ChatGPT-User	/ab-testing/mobile-app-a-b-testing	stripped-edited	200	573 ms	IT
00:18:27 IST	Amazonbot	/glossary/agentic-url	origin-bot-miss	200	316 ms	US
00:18:23 IST	Amazonbot	/faq/what-are-the-best-strategies-for-customer-rate-optimization	stripped-edited	200	272 ms	US
00:18:19 IST	Amazonbot	/faq	stripped-edited	200	235 ms	US
00:18:16 IST	Amazonbot	/faq/how-does-a-b-testing-improve-shopify-sales	stripped-edited	200	770 ms	US
00:18:12 IST	Amazonbot	/croagency/phoenix	stripped-edited	200	471 ms	US
00:18:08 IST	Amazonbot	/faq/does-fibr-ai-offer-specialized-image-generation-for-shopify-brands	stripped-edited	200	314 ms	US
00:18:04 IST	Amazonbot	/whitepaper/the-conversion-gap	stripped-edited	200	508 ms	US
00:18:01 IST	Amazonbot	/faq/how-do-i-reduce-cart-abandonment-during-checkout-in-ecommerce	stripped-edited	200	238 ms	US
00:17:57 IST	Amazonbot	/faq/is-ai-image-generation-suitable-for-web-design	stripped-edited	200	244 ms	US
00:17:51 IST	Amazonbot	/faq/how-do-i-measure-the-success-of-customer-rate-optimization-efforts	stripped-edited	200	256 ms	US
00:17:48 IST	Amazonbot	/faq/can-a-content-delivery-network-(cdn)-truly-make-my-website-faster-globally	stripped-edited	200	594 ms	US
00:17:42 IST	Claude-User	/careers	origin-bot-miss	404	961 ms	IN
00:07:32 IST	Bytespider	/customer-stories/fibr-helps-nixon-medical-boost-conversions	stripped-edited	200	126 ms	SG
23:57:23 IST	ClaudeBot	/blog/how-to-create-a-winning-omnichannel-landing-page-strategy-in-2023	origin-bot-miss	200	557 ms	US
23:57:04 IST	ClaudeBot	/credit-unions-report-a1	stripped-edited	200	213 ms	US
23:56:32 IST	ClaudeBot	/conversion-rate-optimization-5/cro-testing	origin-bot-miss	404	390 ms	US
23:56:32 IST	ClaudeBot	/manifesto	origin-bot-miss	200	490 ms	US
23:56:25 IST	ClaudeBot	/partnership-program	origin-bot-miss	200	407 ms	US
23:56:24 IST	ClaudeBot	/b2b	origin-bot-miss	200	347 ms	US
23:56:23 IST	ClaudeBot	/agentic-experience	origin-bot-miss	200	387 ms	US
23:56:23 IST	ClaudeBot	/agentic-experience-event	origin-bot-miss	200	398 ms	US
23:55:56 IST	ClaudeBot	/about	origin-bot-miss	404	274 ms	US
23:55:53 IST	ClaudeBot	/blog/10-website-optimization-mistakes-you-need-to-avoid-in-2024-%28especially-when-targeting-ad-clicks%29	stripped	200	321 ms	US
23:55:52 IST	ClaudeBot	/blog/7-common-reasons-your-google-ads-are-not-performing-how-to-fix-them-%28fast%29	stripped	200	314 ms	US
23:54:44 IST	YouBot	/conversion-rate-optimization/conversion-rate-optimization-program	stripped-edited	200	271 ms	US
23:54:32 IST	ClaudeBot	/.aws/config	origin-bot-miss	404	264 ms	US
23:54:32 IST	ClaudeBot	/.ssh/id_ecdsa	origin-bot-miss	404	265 ms	US
23:54:22 IST	ClaudeBot	/.ssh/config	origin-bot-miss	404	275 ms	US
23:54:21 IST	ClaudeBot	/.git/config	origin-bot-miss	404	286 ms	US
23:54:19 IST	ClaudeBot	/.git/HEAD	origin-bot-miss	404	289 ms	US
23:54:19 IST	ClaudeBot	/id_ed25519	origin-bot-miss	404	268 ms	US
23:53:46 IST	ClaudeBot	/personalization	origin-bot-miss	404	269 ms	US
23:53:45 IST	ClaudeBot	/api/v2/settings	origin-bot-miss	404	285 ms	US
23:53:45 IST	ClaudeBot	/api/config	origin-bot-miss	404	289 ms	US
23:53:45 IST	ClaudeBot	/ads-personalization	origin-bot-miss	404	305 ms	US
23:48:41 IST	YouBot	/blog/personalization-at-scale-strategies-to-boost-marketing-performance	stripped-edited	200	235 ms	US
23:36:38 IST	Meta-ExternalAgent	/whitepaper/agentic-ai-for-bfsi-marketing	stripped-edited	200	1,069 ms	US
23:34:31 IST	ChatGPT-User	/blog/turn-creative-ad-ideas-into-gold-with-dynamic-creative-optimization	stripped-edited	200	601 ms	US
23:30:39 IST	Meta-ExternalAgent	/faq/will-fibr-slow-down-my-website	stripped-edited	200	243 ms	US
23:30:17 IST	ChatGPT-User	/	stripped-edited	200	224 ms	BR
23:29:47 IST	Claude-User	/	stripped-edited	200	195 ms	US
23:29:45 IST	Claude-User	/pricing	stripped-edited	200	198 ms	US
23:27:24 IST	ClaudeBot	/blog/integrate-website-localization-to-ramp-up-your-sales	stripped-edited	200	232 ms	US
23:25:18 IST	ClaudeBot	/faq/what-are-the-best-practices-for-checkout-page-design	stripped-edited	200	862 ms	US
23:15:55 IST	Meta-ExternalAgent	/conversion-rate-optimization-1	stripped-edited	200	252 ms	US
23:14:38 IST	Claude-User	/seed-funding	stripped-edited	200	274 ms	US
23:14:24 IST	ChatGPT-User	/geo/llm-content-optimization-best-practices-2026	stripped-edited	200	654 ms	US
23:14:12 IST	Claude-User	/pricing	stripped-edited	200	328 ms	US
23:13:44 IST	Claude-User	/	stripped-edited	200	218 ms	US
23:12:24 IST	Amazonbot	/faq/does-ai-help-with-competitor-analysis-during-a-website-audit	stripped-edited	200	539 ms	US
23:12:20 IST	Amazonbot	/faq/how-does-inventory-management-affect-a-brand-s-revenue-potential	stripped-edited	200	522 ms	US
23:12:16 IST	Amazonbot	/glossary/personalization-engine	stripped-edited	200	821 ms	US
23:12:12 IST	Amazonbot	/glossary/conversion-rate-(cr)	stripped-edited	200	479 ms	US
23:12:08 IST	Amazonbot	/faq/what-are-the-specific-challenges-of-automotive-conversion-rate-optimization	stripped-edited	200	538 ms	US
23:12:04 IST	Amazonbot	/faq/does-ai-speed-up-a-b-testing-results	stripped-edited	200	516 ms	US
23:12:00 IST	Amazonbot	/faq/are-there-apps-for-creating-widgets-on-shopify	stripped-edited	200	830 ms	US
23:11:55 IST	Amazonbot	/glossary/conversion-optimization	stripped-edited	200	457 ms	US
23:11:52 IST	Amazonbot	/faq/how-do-i-develop-a-conversion-rate-optimisation-strategy	stripped-edited	200	632 ms	US
23:11:48 IST	Amazonbot	/faq/can-excessive-animations-or-videos-hurt-website-speed	stripped-edited	200	541 ms	US
23:11:46 IST	Amazonbot	/glossary/squeeze-page	stripped-edited	200	1,525 ms	US
23:11:43 IST	Amazonbot	/faq/how-do-i-get-started-with-ai-for-conversion-rate-optimization	stripped-edited	200	281 ms	US
23:11:35 IST	Amazonbot	/faq/how-can-i-use-paid-advertising-to-improve-conversion-rates	stripped-edited	200	527 ms	US
23:11:31 IST	Amazonbot	/croagency/banking	origin-bot-miss	200	430 ms	US
23:11:28 IST	Amazonbot	/faq/what-are-the-benefits-of-using-ai-for-website-audits	stripped-edited	200	1,023 ms	US
23:11:24 IST	Amazonbot	/faq/what-are-the-most-common-website-speed-bottlenecks	stripped-edited	200	447 ms	US
23:11:19 IST	Amazonbot	/glossary/page-8/inverse-metrics	origin-bot-miss	200	390 ms	US
23:11:16 IST	Amazonbot	/glossary/content-marketing	stripped-edited	200	911 ms	US
23:11:12 IST	Amazonbot	/glossary/variance	stripped-edited	200	970 ms	US
22:59:48 IST	OAI-SearchBot	/blog/ai-audience-segmentation	stripped-edited	200	888 ms	US
22:39:30 IST	OAI-SearchBot	/ab-testing/saas-a-b-testing	stripped-edited	200	280 ms	US
22:28:54 IST	Bytespider	/max	stripped-edited	200	149 ms	SG
22:23:57 IST	Claude-SearchBot	/api/v2/settings	origin-bot-miss	404	278 ms	US
22:23:57 IST	Claude-SearchBot	/api/v1/config	origin-bot-miss	404	267 ms	US
22:23:55 IST	Claude-SearchBot	/api/v1/settings	origin-bot-miss	404	280 ms	US
22:23:55 IST	Claude-SearchBot	/api/config	origin-bot-miss	404	331 ms	US
22:23:51 IST	Claude-SearchBot	/.ssh/known_hosts	origin-bot-miss	404	298 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/authorized_keys	origin-bot-miss	404	263 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/id_ecdsa	origin-bot-miss	404	276 ms	US
22:23:50 IST	Claude-SearchBot	/.ssh/id_ed25519	origin-bot-miss	404	289 ms	US
22:23:49 IST	Claude-SearchBot	/.svn/entries	origin-bot-miss	404	265 ms	US
22:23:45 IST	Claude-SearchBot	/.aws/credentials	origin-bot-miss	404	289 ms	US
22:22:55 IST	OAI-SearchBot	/guides/geo	stripped-edited	200	570 ms	US
22:19:46 IST	ChatGPT-User	/pilot/ad-personalization	stripped-edited	200	434 ms	US
22:10:03 IST	Meta-ExternalAgent	/experimentation	stripped-edited	200	199 ms	US
22:09:46 IST	PerplexityBot	/geo/best-llm-optimization-tools-for-ai-visibility	stripped-edited	200	225 ms	US
22:09:42 IST	OAI-SearchBot	/blog/website-personalization-strategies	stripped-edited	200	259 ms	US
22:09:21 IST	Meta-ExternalAgent	/faq/how-does-fibr-generate-experiences-so-quickly	stripped-edited	200	215 ms	US
22:06:50 IST	Amazonbot	/whitepaper/personalized-landing-pages	stripped-edited	200	407 ms	US
22:01:05 IST	ClaudeBot	/	stripped-edited	200	199 ms	US
21:58:41 IST	ChatGPT-User	/geo/best-generative-engine-optimization-brands-for-ai	stripped-edited	200	714 ms	US