API Discovery Crawler
YoofoozBot
YoofoozBot discovers and evaluates public API documentation so Yoofooz can build an evidence-backed index of APIs for autonomous AI agents.
What YoofoozBot crawls
Public API documentation, OpenAPI/Swagger specifications, authentication references, errors, rate limits, webhook/event documentation, sandbox/test-mode information, SDK/developer resources, MCP/agent interfaces, security and other public readiness signals.
Why it crawls
Yoofooz uses publicly available developer evidence to create and refresh API readiness profiles for AI-agent discovery, comparison and evaluation.
Crawl behavior
- User-Agent: YoofoozBot/1.1 (+https://yoofooz.com/yoofoozbot; contact: submit@yoofooz.com)
- robots.txt: respected
- Default page budget: 12
- Default byte budget: 2097152 bytes
- Scope: same-origin public HTTP/HTTPS documentation
- Private networks: blocked
Control and opt-out
Site operators can control YoofoozBot through robots.txt. To block the crawler specifically,
use User-agent: YoofoozBot with the appropriate Disallow rules.
API owners can also contact submit@yoofooz.com
for crawl, correction or re-evaluation questions.
From crawling to indexing
Discovery does not itself equal certification. YoofoozBot gathers public evidence; Yoofooz applies its readiness methodology; the resulting report identifies the evaluated sources, readiness signals, risks and recommended improvements. Profiles may be re-crawled as public developer documentation changes.
Pipeline: Discover → Crawl → Parse → Score → Index → Rank → Recrawl.