Robots.txt

Inform web robots about which areas of your website should not be processed or scanned.