Back to how it works! | Sign up now! | Tips on where to position the links.

Search sites try including as many legitimate entries as possible in their databases, while excluding as many of the spurious kind as possible. By this I don't mean that they pass judgement on the quality of the sites submitted, but rather on the appropriateness of a submission of a particular site for a particular category. If a directory accepted every entry that is submitted to them, then every category would soon become filled with listings for get-rich-quick schemes, weight-loss products and pornography, which would crowd out the more appropriate listings and make the directory useless. Of course, on the less reputable search sites, including those entities known as "free for all link pages", this is exactly what happens, which is why it's a waste of time submitting to such sites. But reputable directories expend a great deal of effort making sure that sites do not get listed in inappropriate categories. Likewise, search engines try to make sure that all their listings include only appropriate keywords.

Of course, a search site wants to get as many legitimate submissions as they can. Make no mistake, search engines value your submissions, and the better ones make it as easy as possible for you to submit. But they have to walk a fine line between welcoming valid submissions and keeping the bad apples out. The same hordes of sleazy characters and snake-oil salesmen that fill our e-mail inboxes with spam constantly attempt to do the same to the search sites, and like the evil bulk e-mailers, they have a large repertoire of tricks with which they attempt to deceive the gatekeepers.

So, you certainly don't want to offend the search engine gods, but at the same time, you want to get as much coverage as you can, in order to drive as much traffic as possible to your site. Getting multiple listings, and trying to make your pages appear as high in search results as possible, is quite desirable. This is also allowable, as long as you do things the right way.

A lot of people spend a lot of time trying to devise dishonest ways to increase their coverage in search engines and directories. The majority of these outlaws are of course purveyors of fraudulent products, or operators of one dishonest scam or another. However, sometimes perfectly legitimate businesspeople can, either inadvertently or through a misguided excess of zeal, fall foul of the search engines, and be branded spamdexers, especially if they have employed an unscrupulous or incompetent paid submitter. The irony of the situation is that these attempts at deception are almost always unsuccessful. The search sites' livelihood depends on foiling such attempts, and they usually manage to stay one step ahead of the scamsters.

Search-related fraud falls into the following categories:

Now, you don't want to indulge in any of these nefarious practices, but you do want to get your pages listed in as many different places as possible, and you do want to use keywords to optimise your rankings. To understand how far you can go with these techniques, let's look at search engines and directories separately.

Submitting to Crawler-based Search Engines

Submitting to crawler-based search engines (Altavista, Excite, Lycos, Infoseek or Go) is quite easy. All they need is your home page URL and your email address. Once their spider gets around to visiting your site, it will automatically index all your pages, so there's no need to submit more than the home page URL. Theoretically, search engines will list your site with no questions asked, unless you violate their rules (most of them will not list porno sites or sites that promote illegal activity, and they won't list URLs that trigger redirects to other URLs). In fact, however, a few sites fall through the cracks, and simply don't get listed, for unknown reasons. Also, many search engines will drop a site from their listings after a time, also for unknown reasons. Therefore, it's a good idea to resubmit your site to all the crawler-based engines at least every six months. This is perfectly allowable, and won't ruffle any feathers. Of course, submitting the same site over and over again (say, more than a half-dozen times in a week) may very well land you in the doghouse, and is furthermore totally pointless. Once your site is included in a crawler's database, it's in there. There's no such thing as having more than one listing for the same URL.

Submitting different domain names that point to the same page is a bit of a grey area. Officially, crawlers frown on the practice, but if you don't overdo it, it's unlikely to annoy them, and may cause your pages to come up a little more frequently in search results. For example, we have 2 different addresses: http://www.trebrown.com and http://www.trebrown.co.uk, both of which are indexed in the major search engines (and, of course, only co.uk URLs are listed in the UK-specific search engines).

Submitting to Directories

Directories work a little differently. They do not automatically spider sites, but list only the specific pages whose submissions have been accepted, and real human beings look at each submission to decide whether it should be included. Therefore, it's important to choose the right categories in which to submit. Different directories have different policies concerning how many categories you may be listed under. Some limit you to one or two categories per page, but most allow you to submit a page to as many categories as are appropriate. Of course, most of them make you go through the whole submission process for each category, so submitting a page to lots of marginally-related categories wouldn't be a wise use of your time.

Yahoo is by far the most popular search site, and thus the most valuable one to get listed in. Unfortunately, it's also the hardest to get into. It's very rare to get listed in Yahoo on the first or even second attempt, and many a worthy site may never get listed at all. Increase your chances by taking special care with your Yahoo submission, and follow their rules to the letter.
The Open Directory Project is a must. This huge directory passes listings on to many of the other directories, Yahoo included! Again, it's hard to get them in, but once in you can be assured that you'll eventually be listed in many of the others and without paying! I say eventually because it will really take a long while, several months.

Yahoo will list a particular page in no more than two categories, so choose them wisely. Individual pages from the same site (that is, under the same domain name), however, can be listed in separate categories if they really have different topics. Because choosing the right category is so important, and because real people read the submissions, using auto-submitters to submit to directories is not recommended. There are only a few worthwhile ones anyway, so submitting to them manually doesn't really take much extra time.

Keywords

By far the most important factor in search engine rankings is the keywords that are used in the head of the page, which occur in the body of a page, and especially those that appear as headings or as hyperlinks. It's also a good idea to include your most important keywords in title and meta tags. When a search engine looks at a web page, it looks for several things when deciding how to rank that page. The different engines look for different things, all within a general theme.

They look for:

In general, the more occurrences of a word or a keyword, the higher the engine rates that site when someone searches for that word. However, before you get any ideas about putting <meta description="keywords" value="sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex, sex"> in your page, the engines are set to spot tricks like this, to name them SPAM and to exclude the page (sometimes even the entire site) from the engine. You can bet that the searchies are on the lookout for hot keywords such as "sex", "MP3", "Viagra", etc. and they will definitely bar your site if you have keywords in tags that aren't related to the page content. A good rule to follow is never to include any word in a tag that isn't included in the body text and title of the page.
Some misguided souls make their title and meta tags endless lists of keywords. This is pointless. Spiders only read these tags to a certain point, so anything after this is a waste of your time.

In the early days of the Web, some smart alecks loaded their pages with hidden keywords, by making them the same colour as the background, by including them in the alt attributes of img tags, and so on. This is an old trick, and one that is guaranteed to get your site barred. One popular search engine states clearly that any site that has text that's anywhere close to the same colour as the background will be barred. Don't use hidden keywords in any form. This includes tiny text at the bottom of a page, or any other way of including spurious keywords in a page.
This is one area where you must be careful, because if a search site detects text that's the same colour as the background, they may assume that you're indulging in keyword stuffing, even if it was done for a different reason. For example, if you have a page with a white background that contains a table with a tan background and white text, a search site's automated spam-killing script may think that it's invisible text, even though it is visible.

Here's my rough guidelines, which apply to most engines:

Bait and Switch Schemes

Some losers create pages that are specially tailored to rank high in search engines, but that actually redirect the user to another page, perhaps one that would not be listed on its own merits, such as a porno site. This can be done either by setting up a redirect on the Web server, by using a meta refresh tag, or by simply waiting until the page gets listed, then replacing the legitimate page with another one. Any of these techniques is guaranteed to get your site barred. Even using a refresh tag for a legitimate purpose, such as a splash page, may land you on the doo-doo list. Most search engines simply won't list any page that triggers a redirect, so don't use them for any page that you want listed.

What about doorway pages? Many people have theorized that, since each search engine uses a slightly different method for ranking pages, if you create a separate entry page for each search engine, you'll get more traffic. After considering all the information on the subject that I've read over several years of doing Web site promotion, I've come to the conclusion that doorway pages aren't worth the effort. The ranking algorithms of the major search engines just aren't that different all look for keywords in the same places, they just place different amounts of emphasis on keywords in body text as opposed to in tags, etc. And doing something that helps you with one search engine is not particularly likely to hurt you with another. Many folks out there will disagree, especially those who make money by creating doorway pages. Search engines don't seem to object to doorway pages in moderation, so use them if you choose. Doorway pages are designed to be submitted to crawlers only. Never submit them, or any type of page with duplicate content and different URLs, to directories such as Yahoo.

Things to remember:

Conclusion

What if you do break the rules? What's the worst that can happen? For minor infractions, the search engines will penalize your pages in their rankings, or possibly even drop a particular page from their listings. Only if it's pretty obvious that you're trying to spam them will they bar an entire site. You may never know for sure if this has happened. Just because they fail to list your site after a reasonable amount of time, or drop you from their listings, it doesn't necessarily mean they have anything against your site. These things happen often, and are usually the result of overwork or inefficiency. If you really suspect that you've ticked them off, send a polite note to the site administrator, admitting what you did wrong, and pointing out that it was done through ignorance, not evil intent. Promise to sin no more, and politely ask, beseech or beg them to restore your listing.
The search engines are there to serve Web users, and they certainly aren't interested in barring sites from their listings for honest mistakes. Don't let the fear of offending them stop you from aggressively marketing your site. But don't be tempted to indulge in any of the fraudulent techniques we've just discussed. Take it from a Web marketing professional who's been in the game since 1993 breaking the rules will hurt your traffic in the long run, not help. Furthermore, deceptive marketing practices harm everyone who uses the Web. The Internet is quickly becoming a major resource for society, like a public utility, and it's likely that at some point in the future, attempting to spam the search engines will be considered a serious crime, like insurance fraud, complete with hefty fines and jail time for offenders.

Here's a handy overview of what's acceptable and what's not.

Perfectly Acceptable:
Submitting a page to a directory in several appropriate categories (Except those that specifically say not to do so, such as Yahoo). Submitting different pages from the same site IF they contain really different content. Using lots of appropriate keywords in body text, headings, title and meta tags. Using auto-submitters to submit to search engines. Resubmitting at reasonable intervals.
Maybe Acceptable in Moderation:
  • Different domain names pointing to the same page.
  • Doorway pages.
  • Meta refresh tags (for splash pages).
Not Recommended:
  • Excessive keywords.
  • Using auto-submitters to submit to directories.
Likely to get you barred:
  • Inappropriate keywords in tags.
  • Frequent resubmissions (see each search site's guidelines as to how often is too often).
Guaranteed to get you barred:
  • Hidden keywords.
  • Server-side redirects (for any reason).

Any type of redirect designed to procure a listing based on something other than actual page content.