Welcome back Rankers! There was a story a
couple of weeks ago about ‘Crawl Budget’. There has been a lot of talk around Crawl
Budget, hence Google doing a story on it on webmasters.googleblog.com. I’ve done shows
about this before and have spoken about it, but this is the first time I’ve seen something
in print from Google about Crawl Budget. Some of the things we’ve observed over the
years they are now admitting to. A couple of things we know such as if your site moves
to HTTPS you will see a spike in the crawl rates. That’s what we can see here. We are
in Search Console-Crawl Stats and we can see this example site went to HTTPS and Google
has subsequently been crawling the site a lot more as it gets used to that ecommerce
site’s HTTPS pages. So there’s that situation where Google will come in and crawl your site
a lot more and then there’s the situation of you getting a speed increase on your site.
This is one of the reasons why it’s so important to have a fast site because as you can see
this site here, pages were loading on an average for Google, once again this is for Google
and not for a human, were loading about 840 milliseconds each and now they’re down to
just over 200 milliseconds. You can see there’s a distinct line between Google’s crawl rate
and the speed of the site. The third one that they mention and lots of
people gloss over this, as I know a lot of people think this is referring to backlinks,
but I don’t think so and I’ll show you why. So Google actually says that popularity
is something that will make Google crawl your page more. So here it says, “URLs that are
more popular on the Internet tend to be crawled more often to keep them fresher in our index.”
So what they are saying is that popularity is a measure of the bot coming out and I would
further go to say it’s a measure of a new form of backlink. Popularity used to be backlinks,
but Google has been very careful there to not say backlinks but popularity.
Case in point. This is a Melbourne client and
this represents a national news story for this particular client back on the 14th November
last year. You can see on that same day Google crawled a bucket load of pages. The reason
for that is that Google saw it was more popular as more people went to the site. Incidentally,
that spike I showed in analytics was organic traffic. We had a brand spike in organic,
so more people were searching for the brand. All of a sudden, the brand becomes more popular,
more people are getting to the site from Google on that particular 48-hour period, and we
have a subsequent spike in the Googlebot accessing pages and crawling pages. So when we talk about popularity that’s
why we are so big on not talking about backlinks or building backlinks and those sorts of things.
We like to focus on building audience, because if you’re building audience, you’re increasing
your popularity, if you’re building your brand you’re increasing your popularity.
Increasing your brand popularity is the most important thing. It’s not backlinks because
you’ll get those through increasing your brand popularity. Yeah, backlinks are important,
but the way that you get them is far more important. Focussing on that brand, you’ll
start to see spikes and traffic increases. Firstly from the Googlebot and then increased
rankings. I’ve seen it happen time and time again, and every time we do this we see massive
spikes. I’ll give you another example. We just moved
to new hosting last week and you can see here we were quite slow, loading at 1.5 seconds
and now we’re down at 86 milliseconds. That’s Googlebot page load time. We are starting
to see an increase in crawl. So that happens every time and that can only be a good thing.
It’s a good article to go read that one from Google as it also talks about eCommerce
sites and things like facetted navigation. The reason we are always trying to use canonical
tags, noindex tags, use nofollow tags, all those sorts of things, that article explains
it really well. Hopefully that’s helpful and we’ll see
you all next week. Thanks very much. Bye.