Finding Keywords for Search Engine Marketing

Finding The Best Keywords For Your Website

There are always two sides to any relationship, and that certainly holds true for the relationship between your website and search engine users. In the first place, you have a theme for your website that includes certain topics, your business' product lines or services. These themes comprise your site's primary keywords and they are certainly where you begin when you design your webpages. And since you, as the webmaster/author or business owner, are bound to know your topic very well, you will naturally tend to use the technically correct terminology, the most precise phraseology, the premium buzzwords, the jazzy jargon to use in making your website the sine qua non in your business or field of endeavor. But, on the other hand, there are the terms which real live users, lacking your well-earned expertise and experience, use when they speak, and these are the terms they will most likely enter into search engines. Thus, these are keywords and phrases to which you must also incorporate into your search engine optimization strategies. Obviously, the trick is in finding out precisely which terms John Q. Public is using in search engines to find websites like yours, and which of them are used often enough to be worth including in your keyword targeting. And just where, you might well ask, does one obtain such information? An excellent place to look is as close as your own website: the server log file for your site. That's where you'll find the words and phrases that users entered into search engines to find your site. That information can give you important clues to how real users talk about the topic of your site, which can suggest where you might add some emphasis in your search engine marketing or new keywords to target.

The server log for your website will include a specific piece of information for each webpage request it has received that tells you where on the Internet the request originated called the "REFERER" (sic). In some cases, the requests will originate from other pages on your website. Other times, users will simply type the webpage address (URL) into their browser or use a stored bookmark. But in terms of researching keywords, the most important case is when a webpage request comes from another website - specifically when it comes from a Search Engine Result Page (SERP). These requests will contain the keywords that the user entered and caused the search engine to display a link to a page on your website. As your website becomes bigger and stronger and more well-established in the search engines, you will probably discover that many users are using search terms that you hadn't really considered to be important.

The server log file is going to be very big. Depending on how busy your website is and how often your host rotates the log, it will probably be several megabytes and be difficult to examine. The solution is to use a Server Log Analysis program like my Log File Search Script that gives you some information about recent search engine activity on your website by users and robots, or a more powerful tool like Webalyzer or AWStats which is designed to analyze your logs quickly and easily, with special functions to decipher search engine activity.

Each entry in the server log file represents a request for a file on your website. The entry will tell you the IP address of the requester, the time and date of the request, the name of the file being requested, and the origin of the request - technically known as the "referrer" (an historic misspelling by the developers of the Apache server software often leads to the use of the word "referer" instead). Within the referrers, you should see the URLs of websites that link to your webpages. If you've been working on your search engine visibility, you should certainly see a fair number of referrals from the major search engines. It is those referrals that are our target.

The referrer entry from a search engine will include the search term the user entered that made your webpage appear in the Search Engine Results Pages. They will be encoded to remove <space> characters and other special punctuation, but all in all they're perfectly readable. Take careful note of each one of these. What terms are people using to find your site? You've already been watching your rankings in the search engines, but these entries tell you the search terms that are actually bringing traffic to your site. I have used our logs to discover that there were many people searching for collectibles from Alice in Wonderland. I hadn't realized it was such a popular category until I started watching our log files.

By the same token, are there people coming to your site for your primary keywords? Do you see the search terms for which your website ranks well in the search engines or are your efforts being wasted on search terms that only a few people actually use? This is the kind of information that can help you get the most traffic for your efforts in both constructing your website and your search engine optimization program.

There are also several online tools that will tell you what users are entering into the search engines. These tools will show you how often real users searched for the keywords you enter. Google offers a trio of keyword research tools that are very helpful. The best one for finding what keywords real people are using is the Google AdWords Keyword Tool. By entering keywords and related topics into this tool, you can find a list of words and phrases that have been searched for in Google, and how often they were used. It will also show you information about the cost of Adwords advertising for each search phrase. For exmaple, I did a test of this tool on the keyword phrase "dog kennels" and got the results shown below:

The Adwords tool will show you which keywords are the most popular, and will often show you words and phrases that you hadn't thought of targeting. Who knew that "wooden dog kennels" were popular? I certainly didn't. So you can see that it's common to find ideas for new topics to add to your site with a tool like this, which just makes your site more useful and potentially stronger in the search engines. Being more useful to users leads to more links, which leads to stronger search engine performance, which all leads to more traffic to your website and, hopefully, more sales.

Another excellent source of keyword research data is the Google Webmaster Tools console. The Keywords tool in the Webmaster Tools console will show you the keywords that were searched on when pages from your website were included in the search results, including how often your pages appeared (impressions), the click-through rate (how often a user clicked on your link in the search results), and the ranking position of your page when it was included in the results. This tool gives you a good idea of how well your site is performing for your targeted keywords with real data. Google Webmaster Tools is jam-packed with useful information about your site's performance in Google. It includes some historical data so you can monitor the changes in your site's performance. It can also warn you when there are problems, and suggest solutions. Every webmaster should register for this free service and refer to it regularly.

Another way to check on your site's keyword performance is Google Analytics. By adding a small snippet of JavaScript to your webpages, Google is able to monitor your site's performance in several areas. Naturally, it includes information about the keywords used when users come to your site from Google, but it also includes critical data like Bounce Rate and Time on Site, which helps you see if users were finding what they were searching for, or if they left as soon as they arrived on your website. It can also help you monitor your website traffic by city (like I do to see visitors from Minneapolis), or country.

These are free tools from Google that are invaluable to finding the best keywords to target for your paid promotional efforts as well as for your search engine optimization. In addition, there are commercial services like WordTracker™ that analyze keyword usage in-depth, including breakdowns of plurals, usage by country, and other fine details. It's expensive, but if your company really needs to know their market, it can be a great investment. I've created a tool of my own to help you with Adwords. It's my AdWords Cost Per Click Budget Calculator that helps you estimate how much you can pay for clicks and still make a profit.

Knowing the relative popularity of keywords can help you tailor your search engine marketing program to best suit your website and your target audience. By knowing which keywords are the most popular with users and the relative competition for each phrase, you can decide whether you want to be a small fish in a big pond or a big fish in a small pond. You can see which pages on your website are performing up to your expectations and which ones need more attention. Doing a good job of keyword research will help you make your site better and stronger, and ultimately more successful.

This page was last updated on September 25, 2020