Story

On air. The free wire runs 15 minutes behind live. See pricing

World, crypto, finance and tech news, scored for impact.

AIHacker NewsFiled 2 days ago

2026 AI Search & Agent Readiness Benchmark: Empirical HTTP Study Across 40 Public Domains

Open the sourceReported by Hacker News, verisource.dev, 17 November 2026

Not investment advice. Muhabbit scores how much a story may matter, not which way to trade.

A November 2026 HTTP probe of 40 public domains across B2B SaaS, Shopify D2C, SEO agency, and multi-location service verticals found that 50.0% publish a custom /llms.txt, 82.5% ship homepage JSON-LD, but only 7.5% include FAQPage schema and just 5.0% publish a custom /agents.md. The study identified a sharp gap in commerce: 80% of probed Shopify brands expose active /api/ucp/mcp endpoints, yet none have customized /agents.md and only 10% serve a curated /llms.txt, with six of ten relying on stock Shopify boilerplate. Silent edge blocking emerged as a hidden risk—0 of 40 domains disallow GPTBot in /robots.txt, but 2 of 40 (ridge.com and westerndental.com) return HTTP 403 to GPTBot or PerplexityBot user agents at the CDN. Developer SaaS led curation with 100% custom /llms.txt adoption, versus 40% for SEO agencies and 10% for D2C brands, while 17.5% of /llms.txt responses were unedited platform boilerplate and 7.5% were HTML soft-404s. The findings matter because agent-facing infrastructure is becoming a citation and discoverability layer, and the next check is whether brands audit WAF rules and replace stock boilerplate with curated agent files.

Muhabbit's AI wrote this summary in its own words from Hacker News's report. It can get details wrong, so check the source.

Named in this story

GPTBot, Shopify (SHOP), Linear, Vercel

A linked name opens that asset's sentiment page.