How to keep Bing from crawling your site

Aug 12

So, in light of hearing how Fidget also had a family photo turn up in an adult search term, I decided to read everything I could understand in the link that Mr Farty shared, before I even hear back from Betsy about how Bing works.  Because, really?  Now I know this isn’t an isolated incident, and I don’t want Bing poking around my site at all. I imagine many of you probably feel the same.

So, here is the deal.  If you have access to the root of your server, you need to create or edit your Robots.txt file.  If you don’t have one at all, you can right click and download mine here and steal it.  If you do have one, add this to it to keep the Bing bot from crawling your site:

User-agent: msnbot
Disallow: /

User-agent: msnbot-media
Disallow: / 

User-agent: msnbot-products
Disallow: / 

User-agent: msnbot-newsblogs
Disallow: /

If you use a blogging platform and don’t have access to your root server, I’m not sure what you can do, but I’ll keep looking for a way for you to keep the MSN bot away.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>