NoFollow is Not the Only Way to Block Link Crawling

reviews — Tue, 18 Mar 2008 13:56:00 +0000

Other Ways to block search engine spiders from crawling

Actually, NoFollow does not “stop” the search engine spiders (robots) from crawling, but instead Google and others simply do not pass “link juice” to the backlink.

rel=”nofollow” Action	Google	Yahoo!	MSN Search	Ask.com
Follows the link	Yes	Yes	Not proven	Yes
Indexes the “linked to” page	No	Yes	No	Yes
Shows the existence of the link	Only for a previously indexed page	Yes	No	Yes
In SERPs for anchor text	Only for a previously indexed page	Yes	No	Yes

However, as Geoland points out so well, there are other ways to block the crawling of links. The other methods are much more limiting than rel="nofollow", because at least NoFollow still allows the crawling and indexing.

Normal methods to check for NoFollow will not show you if a link is truly crawlable (including SEOQuake, right-clicking on a link and looking at properties, etc). In other words, “what you see is not always what you get.”

Robots.txt

One method is to use Robots.txt to control the crawling of spiders. Robots.txt is kept in the top level of the domain.

Examples

Blocks all robots

User-agent: *

Disallow: /

Blocks all robots from crawling specific directories

User-agent: *

Disallow: /cgi-bin/

Disallow: /images/

Disallow: /tmp/

Disallow: /administrator/

Meta Elements

The robots attribute is used to control whether or not the search engine spiders are allowed to index a page and whether or not they should follow links from a page. The NOINDEX value prevents a page from being indexed, while the NOFOLLOW prevents links from being crawled.

The robots attribute is supported by all of the major search engines. In addition there are several additional values for the robots meta attribute that are relevant to search engines, such as NOARCHIVE and NOSNIPPET, which are meant to tell search engines what not to do with a web pages content. Meta tags are not the best option to prevent search engines from indexing content of your website. A more reliable and efficient method is the use of the Robots.txt file.

NOINDEX tag tells a search engine not to index a specific page.

NOFOLLOW tag tells a search engine not to follow the links on a specific page.

NOARCHIVE tag tells a search engine not to store a cached copy of your page.

NOSNIPPET tag tells Google not to show a snippet (description) under your a search engine listing, it will also not show a cached link in the search results.

Example





Create Backlinks to My Site

Javascript

Any link written in javascript is not crawlable by search engine spiders. It will look like a normal link and will not be lined through (SEOQuake addon) or highlighted (Search Status addon)

Dynamic URLs

Some spiders may also avoid crawling any url that has a “?” in it (dynamically produced) in order to avoid spider traps which may then cause the crawler to download an ‘infinite’ number of URLs from a Web site. Thus it would probably prevent links on these pages from being crawled.

An Ongoing List of Adsense Alternatives

reviews — Wed, 24 Oct 2007 14:58:55 +0000

Webmasters can place Google AdSense JavaScript code on their web pages which would then allow Google AdSense to be displayed as advertisements based upon the content of the web page.

Some webmasters are either not accepted into the Google AdSense program, or do not wish to do business with Google.

For those webmasters, there are many alternatives to Google AdSense.

Click here for a list of alternatives

for the List of Google Alternatives Page of this site

Yahoo! Search Marketing $199 Credit Toward Advertising

reviews — Thu, 13 Sep 2007 20:14:50 +0000

This is great news for those new to Yahoo! Search Marketing. That is a ton of credit! This applies to the Self Service option only (where you do the setup yourself).

Select your keywords.
Write your ads.
Set your budget and bid amount.

Click here for the link

Reviewer of Sites | SEO Marketing Affiliate » Yahoo