Preparing Your Website
For Search Engines
Start out right and your website will be the fast track to good rankings in the search engines. New webmasters are especially suseptible to myths, rumors and bad advice when it comes to search engine optimization. This SEO Tip contains advice for all new webmasters, but it's especially written for the webmaster or business owner who is new to search engine optimization and rankings. About 90% of what you need to know in order to do well in the search engines is contained on this first page. Just keep in mind that overnight successes in SEO are rare, but giving your website a solid foundation will pay dividends for your search engine performance forever.
Preparing A Site for Search Engine Sumbission
There are several steps you should take before any user - whether human or search engine spider - ever sees your website. But even if your site has been online for quite a long time already, it's never too late to start working on search engine and user-friendly design.
Be sure your website is ready for human visitors before you publicize it. Until your website is ready to start receiving traffic from real users, the first thing I recommend you do is to add a robots.txt file to your site that blocks all search engine robots from your site. This way, it's unlikely to receive any visitors who would naturally be expecting a fully-functional website. To do this, simply create a plain text file containing the following:
User-agent: *
Disallow: /
Then save this file as "robots.txt" and upload it to your website's root directory. That is, put it in the same directory on the server as the main page. The search engines all recognize the robots.txt file for controlling how they access your site. This example of robots.txt prevents search engines from examining (ie. "crawling") your website while it's still under construction. Just remember to remove this blocking instruction when you are ready to open your site by simply removing the "/" from the "Disallow:" line.
Naturally, you should also test your website's design with the most popular browsers, including Microsoft Internet Explorer™, Mozilla Firefox™, Google Chrome™, and Apple Safari™ before you go any further here. You also need to have a plan in place for those using mobile devices like smartphones and tablets. While accommodating mobile users is important for search engine rankings, users come first. So you need to be certain that your website displays properly for all users. That can mean using responsive design methods in your HTML and CSS mark-up, dynamically serving dedicated content to mobile users, or creating a separate website for those users. See Google's Guide to Mobile-Friendly Websites.
Insert a rel="canonical" tag in the <head> section of your main page to establish your preferred URL. As in:
<link rel="canonical" href="http://wwww.yoursite.com/">
This tag tells the major search engines that they should use the URL in the href attribute of this tag whenever they crawl and index this page, regardless of the URL they used to find it. You should select a preferred version of your URL - either with or without the "www." subdomain prefix, and use that same URL whenever you link to your own site and whenever you ask another website to link to yours. Using this tag on your main page also eliminates the need to set up 301 redirects for variations of your main URL using index.html, index.php, etc. This tag is actually useful in resolving many situations, but for now, just use it this way until you learn more about search engine optimization and canonicalization.
Use Unique and Relevant Page Titles. The <title> tag is the most important piece of real estate on a web page. Not only does it carry a great deal of weight in the rankings, it is the first thing that users see when your pages appear in the search results. So you need to make sure that every page has a unique, keyword-rich title that will entice users to click on the link. Remember, rankings are not your ultimate goal. The real goal is getting users to actually visit your website. Go ahead and include your site name in the title (at the end!), but the rest needs to be unique for every page. Get into the habit of making unique, descriptive, and compelling webpage titles and you'll have better results every time.
Create a "description" <meta> tag in the <head> section of your web pages to create an appealing description of the contents of each page. The <meta> Description tag is not used by the search engines for ranking purposes, but they will often display the contents of this tag in the snippet of text that appears under your page <title> in the search results. Use this to your best advantage by creating a concise and compelling description of each page to attract clicks from search engine users. This is especially important for your main page, since it is the strongest page on your site and generally the most likely to appear in the search results. Be sure to include the primary target keywords for each page in this description, and keep it roughly between 100-200 characters in length; something more than a dozen words or so, but less than a paragraph. Do a few searches of your own to see which snippets seem to you to be the most effective and you'll get the idea.
Your website's main page should have at least one paragraph of keyword-rich text that describes your website's topic or purpose to give the search engines a clear understanding of your site's content. The reason for doing this is that search engines can't read text rendered in graphics, such as images or buttons. And it's almost impossible to rank well for a keyword phrase that doesn't appear in plain text on the page in question, so you should extend this practice to every page on your website. You also want to make sure that there is at least one plain link on the main page that points to each of the most important internal pages on your site.
Make sure your HTML code is search-engine-friendly. The World Wide Web is full of free HTML validators, which are programs that scan your website's pages for properly-written HTML code. So use them to your advantage. The best HTML and CSS validator is the W3C Markup Validation Service. You'll often prevent problems before they start just by letting one of these programs do their own special kind of spell-checking on the HTML code of your webpages. Browsers are designed to tolerate a high level of HTML errors, but search engine crawlers (a.k.a. "spiders" - the search engines' automated software that scans webpages) can be a bit fussier in how they interpret your HTML. The search engines don't give you extra credit for valid coding, but a stray < symbol or a missing quotation mark in your HTML code can cause a search engine spider to overlook or omit an entire section of your web page.
Validate every link. Once you have your site completed or nearly so, double-check that the links all work and the images all appear on the page when you access your website over the Internet. Don't rely on the copy of the site on your own computer because some files may be on your computer, but not on the server. Test the actual online version of your website. If your website is too large to check out every link by hand, use one of the many automated link checkers like Xenu Link Sleuth which makes it simple to find broken links. Going forward, you should make this process a part of your routine website maintenance program.
Maintain A Local Back-Up Copy Of Your Website. Having a local back-up of your website's content is the best insurance you can have against several different problems that would be a disaster if you didn't have that data. Web servers do fail, and not every hosting service automatically backs up user data. Hackers are an ever-growing problem, and the only sure way to remove their malicious code is to restore your site from the known, clean back-up copies that you make yourself. Make it a habit to completely back up your entire site to your local computer - the HTML pages, graphics, video, mySQL databases, and anything else that you store on your site, and do it at least once a month. And be sure to back up the files on your own computer, as well. These days, most of us use services that include "cloud" storage - space on web-accessible servers. Those services are often used for personal files like photos and videos, but if you have the capacity to spare, they can also be used to store your back-up files.
Set the robots.txt File To Allow the Search Engines to crawl (ie. "scan") your website. Now that your site is ready for the world to see it, you need to update the robots.txt file from Step 1 above to tell the search engines that they can crawl and index the pages from your site now. For most websites, the following robots.txt is fine:
User-agent: *
Disallow:
Get Other Websites To Link To Yours. The World Wide Web's basic foundation is the hyperlink. Exchanging links with other websites is a legitimate part of your early promotional program, so you need to prepare a page on your website where you will put links to other websites. Make sure there is a link on your website's main page to this new links page. Once this page is installed you can contact the webmasters of other web sites and offer to exchange links. The best place to start is to visit other websites whose topic is related to yours - even if they are direct competitors. These links will pay off in many ways, but links are vital to your site's performance in the major search engines. In fact, once you have one other good quality website that links to yours, the major search engines will find your site on their own. You do not want to make link exchanges with every site on the Internet, so don't go overboard with them. Google's Webmaster Guidelines, in particular, prohibit "excessive" link exchanges. Focus on quality sites that are related to yours and limit them to a couple dozen or so, and you'll do well. See my Building Links to your Website SEO tips article for details.
Submit Your Website to High-Quality Online Directories like The Open Directory Project. A listing in high quality directories will help a bit with search engines, but the listing itself can, naturally, generate traffic, too. Glaciers are speed demons in comparison to The Open Directory Project's speed of processing of submissions, but the entry there can be valuable enough that it is worth the effort. However, never resubmit your site to The Open Directory Project. If you resubmit your site, you are essentially starting over at the end of the queue in waiting to be evaluated. So submit it once and forget it for at least a year.
When choosing a directory, look for ones that review the quality of a website before they include them. Directories that accept any site in exchange for a link are not likely to do you any good. For example, consider quality directories like JoeAnt, Best of the Web, Jayde - The Business-to-Business Directory, and GoGuides. In highly competitive businesses, a listing in a paid directory like the Yellow Pages can be worthwhile because of the contextual setting of these links combined with the fact that Yellow Pages does an editorial review of your site before your listing is accepted. Search engines are very good at analyzing the origins of links to determine the quality of those links for ranking purposes.
Submit your Website Directly to the Major Search Engines. Never pay for services that offer to submit your web site to thousands of search engines. They can actually impair your website's search engine ranking in rare circumstances, and they often turn out to be a major source of SPAM. They are simply a complete waste of your time and money. If you submit your site to Google and Bing, you've covered the sites that account for well over 90% of the search engine traffic on the Internet and none of them charge for submission to their search engine index. Search engines normally find new websites through links on other sites, so your submissions may be superfluous, but it can sometimes get your site crawled sooner than it would otherwise. So take the 5 minutes required to submit to Google and Bing through their Webmaster Tools consoles.
The best place to start is submitting your site to Google through Google Webmaster Tools, since Google accounts for over 60% of search engine traffic. If at least one other well-ranked web site links to yours, between your submission and that link, you should soon see a visit from the Googlebot and your main page will be included in the index within a week. And the Google Webmaster Tools console will let you monitor your website's performance so you can continue to improve your rankings over time.
Of course, you'll also want to take advantage of the Bing Webmaster Toolbox. Similar to Google's offering, Bing's Webmaster Toolbox shows you how Bing sees your website and check your rankings. Since Yahoo! relies on Bing for their search results, combined they account for over 25% of all search engine traffic on the Internet. So you can't afford to overlook Bing.
Inclusion in the search engines is not the same thing as ranking. While you have to be included in a search engine's index (database) in order to appear in the search results, inclusion does not guarantee good rankings. You need to take additional steps - the process of search engine optimization (SEO) - in order to compete against all of the other sites on the Internet. The other search engine optimization tips I present here will help you learn and effectively undertake that process.
Getting Your Site Indexed
The search engines store their information about the web in an array of databases called
an "index". The process of being included in a search engine is called "indexing" When
Google and the other search engines discover a link to your site on another
well-ranked page that is already in their index, you might be indexed in as little as a few hours or even minutes. Once
the Googlebot has passed through your website, you might be given a highly-positioned temporary ranking
for your primary keywords. Google sometimes gives newly-discovered web sites a short period of enhanced
visibility (listing plus high ranking) to see if actual human beings will want to see the website based on
the entry, rather than Google's ranking algorithms alone. It is not unusual for new websites to seem to disappear
suddenly as Google periodically recompiles their master index. It doesn't mean you've been banned or
penalized, it's just part of their process. New sites can often spend several months maturing before
they really start to rank for their targeted keywords. The best course is to keep adding fresh,
original content to your site, and to keep working on getting links from other well-ranked websites
whose main topic is related to yours.
As I mentioned above, Google only pays attention to a few specific <META> tags. It will use the "description" <META> tag in the text that is displayed in the search results (a.k.a. the "snippet") if that description is relevant to the user's search. The Desciption <META> is also used to determine the uniqueness of a page - an important issue for Google in many ways - so don't simply copy and paste the same Description <META> on every page. Make each one unique, compelling, and relevant to the page content or don't have one at all. The Description <META> tag is not used for ranking purposes.
With rare exceptions, the search engines respect a "robots" <META> tag set to
"noindex" or "nofollow". You do not need a robots <META> tag set to either
"follow" or "index" because those are the default values. And the
and "revisit" <META> tag is ignored by almost every search engine.
As I mentioned previously, it's a very good idea to register your website in Google's Webmaster Tools
and Bing's Webmaster Toolbox. These tools provide you with an enormous amount of information about the performance of
your site in their search engine. You can see the keywords people use to find your site, which pages
are linking to your site, and any problems they are having with your crawling your website or if they detect
evidence that your site has been hacked. They also provide you with
several handy tools and articles for improving your rankings. As your experience grows, you'll find these services to
be invaluable in helping you monitor your website's performance in the major search engines.
This page was last updated on September 25, 2020
Need More Help?
You'll find more SEO Tips on the menu on the right side of this page.
You can also contact me with your SEO questions.
If you can't fix your website search engine problems on your own,
my Search Engine Optimization Services
can give your website what it needs to get your business' fair share of search engine traffic quickly, without disturbing your website's design, and without breaking your budget.
Call Richard L. Trethewey at Rainbo Design in Minneapolis today at 612-408-4057 from 9:00 AM to 5:00 PM Central time
to get started on an affordable website design package or search engine optimization program for your small business today!
In writing these SEO tips, I'm often reminded of a pearl of wisdom that my high school computer programming teacher passed on from one of his teachers, "Computers are high-speed idiots!" Remember that, and don't let them get under your skin.
Looking for more website design information? See my Sitemap!
Talk To Me! Click Here!