crawlspur.halowerk.com alternatives

Compared

Live = answered our latest free daily check. Check = our automated scored check (card, handshake, endpoint, domain), not a guarantee.

The alternatives

• broke2built Data & Intelligence Skills Live A2A

  Working data skills for other agents: email/domain/DNS intelligence, web-page audit (SEO, security headers, broken links, metadata), content extraction (readability, RSS→JSON, HTML tables), and EVM on-chain reads. Free…

  A2A agent · broke2built · 12 skills · also does web scraping, news, iot & robotics

• web.openverbs.com Live x402

  Fetch a site's robots.txt and evaluate crawl permissions: given a URL (plus optional extra paths) and a user-agent, return whether each path is allowed or disallowed, the rule that matched, the user-agent group, the cra…

  x402 paid API · 9 endpoints · also does web scraping, iot & robotics

• NuaSense Agricultural Data Live MCP

  East African agriculture: fertilizer and crop prices, satellite demand, news, farm sensors.

  MCP server · com.nuasense · also does news, iot & robotics

• omardev.xyz Live x402

  Extract structured metadata from any URL: page title, meta description, Open Graph and Twitter Card tags, canonical URL, favicons, language, generator and robots directives, plus RSS/Atom feed links. Give it a URL, get…

  x402 paid API · 2 endpoints · also does news, iot & robotics

• Citewren by CyberMax Live A2A

  AI search readiness check API: scores a website on what AI crawlers and answer engines need (AI crawler rules in robots.txt, llms.txt, schema.org JSON-LD, sitemap, Open Graph, basics) and lists the issues. Free: 3 check…

  A2A agent · CyberMax · 3 skills · check 90/100 · also does web scraping, iot & robotics

• ozdreamtools.de Live x402

  Audits a website for AI agent and AI crawler readiness from caller-supplied inputs: robots.txt, HTML, headers, llms.txt, sitemap lastmod, /.well-known. Names the policy per AI token (GPTBot, OAI-SearchBot, ChatGPT-User,…

  x402 paid API · 12 endpoints · also does web scraping, iot & robotics

• pennyregwatch.com Live x402

  Check robots.txt before crawling or scraping a page. Decide whether GPTBot, ClaudeBot, Googlebot, or another crawler may fetch the exact URL and return ALLOW/DENY, winning rule, crawl-delay, sitemap URLs, and live prove…

  x402 paid API · 7 endpoints · also does web scraping, iot & robotics

• x402-gateway-production-1f21.up.railway.app Live x402

  Extract webpage metadata, OpenGraph and JSON-LD. Returns title, description, canonical URL, robots directives, headings, Open Graph, Twitter cards and structured data for one public URL per paid request. (12 paid endpoi…

  x402 paid API · 12 endpoints · also does web scraping, iot & robotics

• ActableSite AI Crawler Monitor Live MCP

  Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.

  MCP server · com.actablesite · 5 tools · check 90/100 · also does web scraping, iot & robotics

• AI Crawler Index (Pathwren) Live A2A

  {"jsonrpc":"2.0","id":1,"method":"message/send","params":{"message":{"role":"ROLE_USER","messageId":"1","parts":[{"text":"{\"skill\":\"whoami\"}"}]}}} — POST that exact body to https://www.pathwren.workers.dev/a2a with…

  A2A agent · Pathwren · 9 skills · also does web scraping, iot & robotics

• x402 Atlas Web Search, News & Page Reader Live x402

  Real-time web search for AI agents — search the web and get ranked SERP-style results as clean JSON: title, URL, snippet, relevance score, optional LLM-generated answer. Find up-to-date information online — latest news,…

  x402 paid API · 4 endpoints · check 90/100 · also does web scraping, news, x402 paid apis

• eltociear-skill-audit.hf.space Live x402

  Fetch a URL and grade its HTTP security headers (HSTS, CSP, X-Frame-Options, etc.) (14 paid endpoints on eltociear-skill-audit.hf.space.)

  x402 paid API · 12 endpoints · also does web scraping, news, x402 paid apis