What could the webspam team have done better?
Articles,  Blog

What could the webspam team have done better?


MALE SPEAKER: Today, we have
a fun webmaster question from Barry Schwartz in New York. Barry asks, was there a key
moment in your spam fighting career where you made
a mistake that you regretted related to spam? So he’s not just talking about
choice of wardrobe or anything like that. I can think of at
least two mistakes that I’ve made related to spam. So I’ll tell you about both. I remember talking to a very
well known SEO at a search conference in San Jose probably
seven years ago, give or take. And that SEO said,
you know what? Paid links are
just too prevalent. They’re too common. There’s no way
that you guys would be able to crack down on them
and enforce that, and come up with good algorithms, or take
manual action to sort of put the genie back in the
bottle, as he put it. And that was when I realized
that I’d made a mistake, and that we had
allowed paid links that passed page rank to go
a little bit too far and become a little bit
too common on the web. And so in the early
days of 2005, 2006, you’d see Google cracking
down a lot more aggressively and taking a pretty hard line
on our rhetoric about paid links that past page rank. At this point, most people know
that Google disapproves of it. It probably violates, like,
the Federal Trade Commission’s guidelines, all those
sorts of things. We have algorithms to target it. We take spam reports about it. And so for the most part, people
realize it’s not a good idea. And if they do that, they
might face the consequences. And so for the most
part, people try to steer clear of
paid links that pass page rank at this point. But we probably waited
too long before we started to take a strong stand
on that particular issue. Another mistake
that I remember is there was a group
of content farms. And we were getting some
internal complaints where people said, look, this website,
or that website, is really bad. It’s just poor quality stuff. I don’t know whether you
call it spam, or low quality. But it’s a really
horrible user experience. And I had been to one particular
page on one of these sites because at one point
my toilet was running. And I was like, OK,
how do you diagnose a toilet that was running. And I had gotten a good answer
from that particular page. And I think I might have over
generalized a little bit, and been like, no, no,
there’s lots of great quality content on some of these
sites because, look, here was this one page that helped
solve the diagnostic of why does your toilet run? And how do you fix it, and
all that sort of stuff? And the mistake that I made was
judging from that one anecdote and not doing larger
scale samples, or listening to the
feedback, or looking at more pages on the site. And so I think it took us a
little bit longer to realize that some of these lower
quality sites, or content farms, or whatever you
want to call them, were sort of mass creating
pages rather than really solving users’ needs with
fantastic content. And so I think, as a result,
we did wake up to that. And we started working on
it months before it really became wide scale in
terms of complaints. But we probably could have been
working on it even earlier. Regardless, we’re always
looking for good feedback. We’re always looking
for, what are we missing? What we need to do to make our
web results better quality? And so any time we
roll something out, there’s always the
question of, could you have thought of some way to stop
that, or to take better action, or a more clever algorithm? And could you have
done that sooner? And so I feel like Google
does a lot of great work. And that’s very rewarding. And we feel like, OK, we
have fulfilled our working hours with meaningful work. And yet, at the same
time, you always wonder, could you be doing
something better? Could you find a cleaner way
to do it, a more elegant way to do it, something with higher
precision, higher recall? And that’s OK. It’s healthy for us to
be asking ourselves that. So great question, those
are a couple key moments I can remember where it felt
like we made a mistake by not paying attention to a
particular topics soon enough. Hope that helps.

11 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *