AssayBot

User agent string

AssayBot/1.0 (+https://assay.org.uk/bot; accountability data; contact@assay.org.uk)

What it collects

AssayBot fetches publicly available documents and pages relating to council accountability: registers of interests, senior pay disclosures, and transparency statements. It does not collect personal data, login-protected content, or any page marked as private. Every collected item is linked to its primary source URL and a timestamp of when it was fetched.

Crawl behaviour

AssayBot respects your robots.txt file and crawl-delay directives. It will not attempt to access paths you have disallowed. It makes a single request per URL with a polite delay between requests. It does not attempt to bypass access controls.

If a page returns a 403 response, AssayBot records this and does not retry aggressively. Blocked resources are flagged internally as needs attention rather than treated as missing data.

Opt out

To exclude specific paths from AssayBot, add a Disallow rule to your robots.txt:

User-agent: AssayBot
Disallow: /

Alternatively, contact us at contact@assay.org.uk and we will add a manual exclusion within 5 working days. Include the domain you wish to exclude and the reason.