Crawl Rate is the New Google Page Rank – Use These Great Tools

February 28, 2008 - Written by Gyutae Park  

What do the search engines think about your site? In SEO, it’s important to gauge the importance and authority of your site and of others in order to build quality links and to move up in the search engine rankings. PageRank is often used as a general meter but with sporadic updates and random fluctuations and penalties, it’s an unreliable tool at best.

So what can we use to better assess the importance and authority of a site?

Enter the crawl rate. How frequently Google crawls a website is usually indication of how much they value the site. In fact, Aaron Wall of SEO Book goes as far as to say that “what Google frequently visits (and spends significant resources to keep updated) is what they consider important.”

Throw away the green pixels on your Google toolbar and start looking at the Google cache date. When was the last time Google accessed your site and updated their index?

wtw-cache.gif

Google last accessed Winning the Web on Feb 21, nearly 7 days ago. The crawl rate depends on the kind of site that you have, but the more often Google crawls your site the better. For example, blogs will usually have a higher crawl rate than a static html page.

A great way to determine the crawl rate of your site and of your competitors is to use a site called SEO Meter. SEO Meter offers a free web-based tool that tracks the rate in which Google crawls your website. You can then monitor the crawling and see how it changes with the different stages of your SEO campaign.

Below is a screen shot of Winning the Web’s Google crawl rate in the month of February.

seo meterwtw-seometer-stats.jpg

I recommend you submit your site to SEO Meter in order to monitor your site’s Google crawl rate. Top-level domains are free to add but internal URL submissions cost $20 per year because they offer a much more detailed view of your entire site’s crawl rate.

SEO Meter reports the top 20 Internet marketing sites based on Google crawl rate. Not surprisingly, the overall popularity of the site is negatively correlated with the crawl rate.

im-top20.jpg

If you’re using WordPress, another tool that you can use to measure your site’s crawl rate is the crawl rate tracker plugin. It’s free to download and offers crawl rates for all of your blog’s pages in Google, Yahoo, and MSN. It’s a pretty nifty tool that I recommend you download and use.

SEO Meter is still great for comparing your site’s crawl rate to those of competitors in your niche via graphing.

What’s your crawl rate? Leave it in a comment below so we can all compare. Here’s mine:
winningtheweb.com - SEOmeter SEO tools

If you like this post, subscribe to the RSS feed. Get the latest updates delivered straight to your email or news reader.
Posted in SEO
Winners Circle - Internet Marketing Case Studies
Related Posts:

Comments

25 Responses to “Crawl Rate is the New Google Page Rank – Use These Great Tools”

Andrew Pavelski on February 29th, 2008

Thanks for your explanation Gyutae, nice post. I’m heading over to SEO meter right now to check out my crawl rate! I’ll hopefully report back soon ;)

Gyutae Park on March 5th, 2008

Thanks Andrew. Crawl rate is definitely a good indication of how much Google values your site. What did you find?

Andrew Pavelski on March 17th, 2008

Hey Gyutae, I found 4.4 days w/ a 1387 Rank…

(Comments wont nest below this level)
 
 
 
sylv3rblade on February 29th, 2008 Subscribed to comments via email

Interesting article as always. I’m almost wondering where you’re pulling ideas for your posts. ^^ Awesome resource.

Gyutae Park on March 5th, 2008

Thanks sylv3rblade,
I appreciate the support. :)

As for ideas, they just come with experience and reading other blogs and resources in the industry.

 
 
Tom Beaton on February 29th, 2008

I shall have to check that out. Google seems to refuse to raise my pagerank past 0. Domain is still pretty new though I guess.

Gyutae Park on March 5th, 2008

Hey Tom,
Google only updates the visible toolbar PR every couple of months. They’ve even gone 6 months between updates at one point.

I’d say to focus on rankings rather than following the green pixels of PR.

 
 
David Chew on March 1st, 2008

Interesting post, it sure looks good.

Gyutae Park on March 5th, 2008

Thanks David. Did you read it?

 
 
Jason on March 1st, 2008

Google seems to crawl my site about once a week. Not sure if that’s a good rate, but considering I’m only posting about three times a week, that rate works fine for me.

I use WebsiteGrader to check my last crawl date.

Gyutae Park on March 5th, 2008

Hey Jason,
Blogs tend to get crawled and indexed at a higher rate because of their “news” nature. Obviously you want to increase the rate and that’ll happen once Google trusts your site more as you build up your authority.

 
 
Krsto on March 1st, 2008

I feel like you are overeacting a little in the headline… Tools sure are great, but PR even sounds better than the CR. As you said it, blogs are visited more often than the static pages by the crawlers, and I’m not so sure that’s fair to all those .html designers.

Greetings from Krsto

Gyutae Park on March 5th, 2008

Regardless of what “sounds better”, I’m just telling it how it is. Visible Page rank is dead and should not be used when determining the quality of a link.

 
 
Wisdom on March 1st, 2008

Google webmaster tools can tell you the last time your site was crawled and stats. You can also change your crawl rate. I haven’t tried this though.

Gyutae Park on March 5th, 2008

You can see the last time Google cached your site by clicking on “Cache” in the search results. As for changing crawl rate, this is something that Google does on their own – depending on what Google thinks is best for its index. Ultimately, they try to save resources while at the same time providing the most up-to-date snapshot of the web.

 
 
Krsto on March 2nd, 2008

I suggest we all try anyway… Tnx Gyutae

Gyutae Park on March 5th, 2008

Haha, way to be brash…

 
 
Kyle James on March 2nd, 2008 Subscribed to comments via email

I had completely forgotten about this site. Thanks for the reminder.

 
Kyle James on March 2nd, 2008 Subscribed to comments via email

Oh yeah… and the crawl rate is 1.6

I just wish that translated to some Page Rank love…

http://www.seometer.com/index.php?search=doteduguru.com&x=0&y=0

Gyutae Park on March 5th, 2008

Hey Kyle,
More importantly, how is it doing in the rankings?

Kyle James on March 5th, 2008 Subscribed to comments via email

Well the site is only 2 months old, but it’s starting to rank. Just last week it crossed over into Page Rank territory w/ a PR0 now, but I have been enjoying a solid linkbuilding campaign and sharing links with the other blogs in Higher Education. Also I believe this now more than ever, but Google tends to think more highly of Education sites than say a Make Money Online site. That’s probably not something that a lot of people want to hear, but it does make a lot of sense and explains why .edu domains are so credible, as they should be.

On another note I did edit my CSS last week to display the content at the top of the page above sidebars and everything and have to say that yesterday was the best day of the young site in far as search engine traffic with a whooping 15 visits. It’s a trend that will hopefully continue.

What I really want to know is when the hell is Alexa going to publicize a new Traffic ranking? It’s been almost two weeks and they usually issue two a week?!

(Comments wont nest below this level)
 
 
 
SEO dude on March 5th, 2008 Subscribed to comments via email

I must say this is quite new to me. Hmmm… Il look into this. Thanks for the info.

 
Michael Heindel on March 11th, 2008 Subscribed to comments via email

Its all about the SERPs. Some keyword phrases can last longer then others when it comes to fresh results.

Michael

 
motorsportBABESau on March 11th, 2008

Never thought of checking my sites “crawl rate” before. Gonna give it a try right away!

 
California Web Design on March 24th, 2008 Subscribed to comments via email

I didn’t previously know how to check Crawl Rate, thank you for this information G!

 
Name (required)
E-mail (required - never shown publicly)
Website
Your Comment (smaller size | larger size)

Trackback responses to this post

Gyutae's Top Picks

Recent Readers

Blogroll

Connect with me