How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Browser Run - Crawl results include URL outcomes and robots.txt diagnostics

calendar_today November 3, 2026 domain cloudflare

Browser Run /crawl results now include more detailed diagnostics. Each URL record can include an outcome that identifies the specific processing result, such as success , robots_disallowed , timeout , or page_error . The job-level robots array reports the robots.txt state for up to 50 origins encountered by the crawl.

open_in_new Read original post