When you’re working on a brand new WordPress site, the last thing you want is for people to see the site in progress and for search engines to index an incomplete site. Luckily this is pretty easy to deal with, and even easier if you’re hosting your WordPress site with HELIX. Even if you don’t host with HELIX, its easy to implement these easy solutions to the problem.
Every HELIX hosted WordPress site is actually TWO sites – a live site and a development site. Using your free HELIX development environment, you are free to create, test and refine until you’re ready to go live with your new site. Great, but how do you stop search engines from crawling and indexing that development site? In the meanwhile, how do you keep your live site from being seen as “Yet Another WordPress Site” by the general public and the search engines?
Hiding Your Development Site From Search Engines
HELIX automatically installs a robots.txt file at the root level of your development site that will stop search engines like Google, Yahoo and Bing from crawling and indexing that site. You don’t have to do a thing for that to work. Your development environment robots.txt file looks like this:
User-agent: *
Disallow: /
Pretty simple. This two line file tells well behaved crawlers/bots that they are not allowed access to a site. There are “rogue” crawlers out there from less than reputable minor search engines that may not obey your robots.txt file. Luckily for the vast majority of you this will not really matter much.
Hiding Your Development Site From The General Public
This can be done a variety of ways including using an .htaccess and .htpasswd file, but by far the easiest way to do it is with a simple plug-in. We like “Password Protected” written by Ben Huson. Its actively maintained, easy to use, works, and is well reviewed. Install the Password Protected plugin, configure it (make sure you check the “Allow Administrators” checkbox) and you’re good to go. Anyone hitting your development site will be blocked by a password page.
Hiding Your Live Site During Development
Now you have your development site all taken care of, but what about your live site while you’re developing? Left alone, visitors and search engines will encounter a stock WordPress installation. You wont want your site to be indexed as “Yet Another WordPress Site”, so there are two things you should probably do.
First use your FTP access to install your own robots.txt file. The two line version above is fine. It will get the job done. Just remember to write it in a text editor (not Microsoft Word), and remember to delete it when your site goes live. Next, create a “coming soon” home page for your live site. Design the page yourself using your vast creative powers, or use a plug-in to get the job done for you. By far the most popular choice is “Coming Soon” by SeedProd but there are quite a few plugins available that will get you where you want to be.
That’s it! You can now develop your new WordPress site in peace knowing that your work in progress is safe from prying eyes (and bots).
