#AskSearchQuality India – Crawl Rate
Articles,  Blog

#AskSearchQuality India – Crawl Rate

Indian webmasters, and welcome to the next
edition of Ask Search Quality. Today we are going to
address another popular issue among the webmasters, and it
seems to be around crawling. Sanjit from Bangalore
and Pankaj want to know how can they control the
crawl rate of the Googlebot. And Pankaj additionally
wants to know if it is recommended to do so. So first of all, what
exactly does crawl rate mean, and how would you
answer these questions? ASHISH KALSI: OK. Pankaj, Sanjit, thank
you for you questions. So it’s really
important to understand what crawl rate really
is, because it’s, I think, a very misunderstood term. A lot of webmasters
kind of mistake this for the frequency by
which Google will probably crawl the contents
on your site, right? But what the crawl
rate really means is the speed by which
the Googlebot interacts with the server to get the
content from your site. So if you’ve got new content
or if you’re updating content that you want to be
indexed immediately, and they want the bot to crawl,
what you’ll probably want to do is to use the Fetch
As Google instead. Because there you have
an option to submit to index, which is probably
what you’re looking for. But if crawl rate is something
that’s bothering you, you do have an option of
decreasing the crawl rate, but not really increasing it. So you can do this from your
Webmaster Tools account. ABHISHEK ROY: OK. Why would you want to
decrease the crawl rate? ASHISH KALSI: Yeah,
so the only situation that it makes sense for you
to decrease the crawl rate is when you’re seeing
that the Googlebot pinging your server is kind of
affecting– slowing things down on your website, right? So unless you see that it’s
substantially slowing down your website, personally, only
at least from Google’s point of view, we would probably not
recommend webmasters to do so. ABHISHEK ROY: OK. ASHISH KALSI: It’s a
fairly straightforward way of doing it. So if you go to your
Webmaster Tools account, if you look at the little gear
icon towards the right corner of the screen, you can
go to the Site Settings, and there you have an option
to alter your crawl rate. So you must remember that
changing the settings, the settings last only
for a period of 90 days. And after that, it comes
back to the default. ABHISHEK ROY: Default, OK. OK, so Pankaj also has
a followup on that. And he also wants
to know how does he allow Google to crawl his
CSS and JavaScript files, and how do you
answer that question? ASHISH KALSI: You don’t really
need to do anything, really. As long as you’ve not
disallowed the bot from crawling CSS or JavaScript
files in your robots.txt, Google will crawl
that content as well. So there’s nothing that
you really need to do. ABHISHEK ROY: OK, so I think
that sums up the entire team and all the questions
we had around crawling. That’s it from this edition
of Ask Search Quality. We’ll see you next time. Thank you. ASHISH KALSI: Thank you.


  • Annie Douglas

    Annie hanging out, it always thangs to get to know we go on talking about the worlds up and downs is it any good left?, yes for little kids but we as Parents,  we must take a stand letting our kids know all is not bad, we still have love we have good hearts, give and want to receive all is not bad.

  • CastingMind - Modeling Agency | Drama School | in Mumbai | Casting Agency | Event management | Video Production | TV/Film Production House

    guys good initiative but do it in Hindi .. for better understand,,

Leave a Reply

Your email address will not be published. Required fields are marked *