# https://www.robotstxt.org/robotstxt.html
User-agent: *
Disallow:
# Translated website pages (//) stay crawlable on purpose: they
# answer with `X-Robots-Tag: noindex, nofollow` and a matching meta robots tag,
# and a crawler has to fetch a page to see either one. Disallowing them here
# would hide that directive and leave copies already in search results ranking
# against the sites they were translated from.