About ObaronBot

ObaronBot is the web crawler used by Obaron to audit sites for AI Readiness. It only crawls sites when someone requests an audit.

How it works

ObaronBot only visits your site when someone has requested an audit for that domain.
It fetches pages as raw HTML — the same signal coding agents and AI crawlers use when reading your docs.
It respects robots.txt directives and Crawl-delay headers.
It waits 1.2–1.8 seconds between requests to avoid impacting your site's performance.
Audits sample ~30 representative pages from your site.
All crawled data is used solely for generating the audit report and is not shared or sold.

ObaronBot/1.0 (+https://obaron.ai/bot)

To prevent ObaronBot from crawling your site, add this to your robots.txt:

User-agent: ObaronBot
Disallow: /

Note: blocking the bot will prevent us from completing your audit. If you've purchased a scan and blocked the bot, we'll issue a full refund.

Questions or concerns about our crawler? Email hi@obaron.ai.