How to Stop Search Engines from Crawling a WordPress Site



We’ve been asked if you can stop search engined from indexing a WordPress site by some of our users. This is mainly for sites just getting started or maintenance. In this article, we will show you how to stop search engines from crawling a WordPress site.

This video brought to you by SeedProd take a look at their site here:

=================================

https://www.seedprod.com/

=================================

For a discount, use our code: WPBVIP

To start we will log into the site and under Settings, Reading we will check the box to discourage search engines from indexing our site. This will let search engines know when they try to crawl the site that you request it not be added by the search engine.

Now, to protect the site we have a few options, to begin we will use the built-in password protection from our hosting provider. Log in and check for Password Protect Directories or Directory privacy and go there. Select the folder for your specific site or you can use the public html folder should you only have one site.

It will promtp you to choose a name for tne protected directory, then set up a username and password that will be required to view the site. When you view the site, every time you viist, a browser notification will pop up asking you for the login information you set.

For the other method we have two plugin options available, the first we will use is SeedProd linked above. Purchase the plugin, download the zip file, under Add New for plugins click the upload button and add the zip file there. As this is a pro plugin you would want to add your license key and you will be able to enable the coming soon mode for your site. The SeedProd plugin even gives you a bypass URL that will only allow users who have a specific url to visit your site.

If you would prefer a free plugin we would recommend the Password Protected plugin found here:

Password Protected

This plugin, once enabled, allows you to limit access to your site to admin or logged in users. Set the password that should be used and feel free to enable the allow Remember me so the password does not need to be continually entered.

If you liked this video, then please Like and consider subscribing to our channel here for more WordPress videos.
https://www.youtube.com/subscription_center?add_user=wpbeginner

Feel free to take a look at the written version of this tutorial here:
https://www.wpbeginner.com/beginners-guide/how-to-stop-search-engines-from-crawling-a-wordpress-site/

Check us out on Facebook
https://www.facebook.com/wpbeginner

Follow us on Twitter

Check out our website for more WordPress Tutorials
http://www.wpbeginner.com
#WordPress #WordPressTutorial #WPBeginner

source

3 thoughts on “How to Stop Search Engines from Crawling a WordPress Site”

  1. #wpbeginner#wordpress #tutorials you might want to mention that Google will still trawl & index sites that has Google Verification or Google Analytics installed, Google Algorithm searches the SQL server using this backdoor access (verification), so if you want Intranet Style site beaware of this backdoor that Google uses.

Leave a Reply

Your email address will not be published. Required fields are marked *