Check if the current page is accessible to common LLM crawlers (robots, page directives, infra).
DevTools panel to inspect network requests (for fanout analysis).
