ZiplyAuditBot — Ziply's site audit crawler

ZiplyAuditBot is the crawler behind Ziply Site Audit. It checks how well a website's pages can be read, understood and cited by AI search engines: structure, sources, authorship, structured data, freshness, links, and crawlability.

It only audits websites that a Ziply customer has added to their Ziply account.

It is not an AI training crawler and is not affiliated with OpenAI, Anthropic, Google or Perplexity.

The user-agent string is:

ZiplyAuditBot/1.0 (+https://ziply.ai/crawler)

Requests come from Amazon Web Services (region us-east-1). We don't publish a fixed IP list yet; identify the crawler by its user agent. If we add fixed IPs later, they will be listed here.

• Fetches publicly available pages over HTTPS, plus robots.txt, XML sitemaps and llms.txt.
• Also loads each page in a headless browser (Chromium), on mobile and desktop screen sizes, to see the content as a visitor would. This loads the page's own scripts, stylesheets and images.
• Never logs in, submits forms, makes purchases, or accesses anything behind authentication. Only reads (GET requests).

Runs on a schedule set by the site owner's Ziply plan: monthly, fortnightly, weekly or daily. Scheduled runs start around 02:00 IST (20:30 UTC).

Each audit reads a limited number of pages (between 50 and 5,000, depending on the plan), and daily audits after the first only re-read pages that have changed.

• Obeys robots.txt: rules for "ZiplyAuditBot" first, otherwise the rules for "*".
• Honours Crawl-delay.
• At most 2 requests at a time per website.
• Slows down or stops when the site responds slowly or returns 429 or 5xx errors, and respects Retry-After.
• Never runs two audits of the same site at the same time.

Block completely:

User-agent: ZiplyAuditBot
Disallow: /

Block part of the site:

User-agent: ZiplyAuditBot
Disallow: /private/

Slow it down:

User-agent: ZiplyAuditBot
Crawl-delay: 10

Changes to robots.txt apply from the next audit.

The crawler only reads public pages. Copies of fetched page HTML are kept for up to 30 days to produce the audit, then deleted automatically. Audit results are shown only to the Ziply customer who added the website.

How Ziply handles information is described in the Privacy Policy.

Seeing unexpected activity, or want us to stop crawling your site? Email contact@ziply.ai with your domain.