Dave's Brain

Browse - Computer Tips - Can I schedule when webcrawlers visit my site?

Date: 2016nov25
Product: web

Q.  Can I schedule when webcrawlers visit my site?
I want them do avoid my busy hours.

A.

BingBot has a way
https://www.bing.com/webmaster/help/crawl-control-55a30302

Add a comment

Sign in to add a comment
Copyright © 2008-2017, dave - Code samples on Dave's Brain is licensed under the Creative Commons Attribution 2.5 License. However other material, including English text has all rights reserved.