Opinion

Felix Salmon

Google’s evil policy on shutting down blogs

By Felix Salmon
January 3, 2010

On Saturday, January 2, John Hempton put up a monster 3,200-word blog entry about a fund group in Australia which has all the appearances of being, he wrote, “qualitatively different and more serious than any previous fund collapse in Australia”. He was featured on the front page of the Sydney Morning Herald (other SMH stories on this subject are here and here), and his blog was read around the world, partly because it continued his investigation into Paradigm, a fund-management firm owned by Joe Biden’s son and brother — a company which, he wrote, “keep being associated with cases like this”. (The Australian fund’s chief consultant was the former CEO of Paradigm Global.)

On Sunday, January 3, Hempton’s blog — his entire blog, not just the one blog entry — disappeared. Anybody trying to go there now* just gets this message:

bronte.tiff

Hempton himself, meanwhile, is faced with this:

Google blocks my blog as spam.JPG

The timing is odd, to say the least, and Hempton suspects that someone complained about his blog, rather than this just being a question of a random false positive in a robot algorithm. Annoyingly, Blogger — which is owned by Google — gives no way of reaching a human being who could see at a glance that the blog is not spam, and turn it back on.

What’s more, Blogger’s robots must be particularly stupid, because Hempton’s blogs have lots of incoming links from reputable locations like Reuters.com, FT.com, and WSJ.com. It’s hard to see what “characteristics of a spam blog” Hempton’s site could possibly have.

This isn’t the first time that Blogger/Google has shut down a prominent financial blogger. In April 2008, the company shut down Yves Smith’s Naked Capitalism for ostensibly the exact same reason, and she says that she only got back up and running relatively quickly because she had a personal connection to Google’s vice president of global communications.

It seems like a no-brainer to me that Blogger should not be following a policy of automatically shutting down blogs without a human making that decision when those blogs are getting large numbers of pageviews and incoming links. If Google’s robots think that a popular blog is spam, then that blog should be brought to the attention of a Google employee who can make that determination. There’s no massive rush, in the case of blogs which have been in existence for months or years, to shut down a blog so quickly that a human can’t get involved.

So what’s up here, Google? Why do you shut down high-quality popular blogs so easily, and make it so hard to reinstate them?

*Update: It’s back up. Yay!

Update 2: Google’s Rick Klau, who saw my blog entry and reinstated John’s blog, explains further, and responds to a proposal from John that Google could simply ask suspected spam-bloggers to pass a Captcha test:

We don’t go into great detail about what triggers a takedown, for the very reason you mention: we don’t want to make it easy for spammers to game the system. That said, we also have a number of protections to avoid precisely this situation – before I document what those are, I’ve asked the engineers responsible for maintaining them to explain to me why they didn’t work. Generally speaking, there were a number of indicators for John’s blog that should have very easily avoided any false positive spam classification – and I’m trying to find out how/why those got bypassed.

Rest assured that this is an internal bug – not any external players gaming our abuse reporting to try and take down John’s blog.

John – for restoring access, we actually met just ahead of the holidays to go through several ideas about making the resumption of service for legit blogs caught in a false positive situation (like yours) more seamless. Your idea is quite similar to reCAPTCHA (http://recaptcha.net/) – a service Google purchased in 2009, and one which we will be incorporating into Blogger in Q1. Glad we’re thinking along the same lines.

Comments
32 comments so far | RSS Comments RSS

It probably makes sense for people to move away from the Borg aka Google.

Posted by AngryLawyer | Report as abusive
 

Hi Felix

Anyone at Google we can complain to? Would be happy to help get John back on line…

Thanks

Jestyn

Posted by thirkej | Report as abusive
 

Agreed, typepad, wordpress, etc seem like better alternatives every day.

Posted by Anal_yst | Report as abusive
 

This is more a “cloud” problem rather than specifically a Google problem. Lack of accountability, security and reliability is a big drawback to web apps such as Blogger. Users should make backups, employ secure passwords and own (or lease) their platform whenever possible.

There’s an interesting ReadWriteWeb article on cloud vulnerabilities here: http://www.readwriteweb.com/archives/the _cloud_isnt_safe_or_did_blackhat_just_sc are_us.php

Posted by RobSterling | Report as abusive
 

Thanks for posting on this Felix.

I wrote John and asked him what was up. This is seriously bogus!

I will hat tip at Alpha.Sources

Claus

Posted by clausvistesen | Report as abusive
 

I’ve been there and it took weeks (instead of the four days that were originally promised) to get one of my blogs back. I’ve also had one of those “chilling effect” notifications, again with no documentation or human contact information. I have the same suspicion as Hempton – that Google’s using a better-safe-than-sorry policy of “shoot first and ask questions later”.

Posted by Kiffmeister | Report as abusive
 

Wonder if this could happen to Eric Schmidt.

On the other hand:

“Reuters.com, FT.com, and WSJ.com” — respectable? Credible to the masses, perhaps; dubitable, for sure; but non-spammy? Not convinced.

Posted by Uncle_Billy | Report as abusive
 

Felix,

Thanks so much for posting about this.

I had my blog yanked down from Google just last week, and have not been given an answer as to why. I have a good mind to sue them if I don’t at least get all of my information back that was up there.

I spent dozens of hours reading the news and editing the stories to put on my blog every week. Every week I would get at least 500 readers, sometimes thousands, and I was hoping to grow it.

I’m glad you’re bringing attention to it, and I hope people will show Google that they can’t just get away with this.

Josh

joshfulton80 at gmail dot com

Posted by JoshFulton | Report as abusive
 

Note also– Google understands more about spam than anyone else in the known universe. They have all the history and a vast amount of data, the most experienced people, the best hardware, et cetera, and so forth. When doing a good job of detecting spam is in their interest (e.g., for gmail) it gets done. There’s just no excuse for shutting down blogs merely on ‘suspicion’, and then not providing a clear path to getting the blog back online.

Posted by MattF | Report as abusive
 

seems like john is back, at least from here in zurich…

Posted by thirkej | Report as abusive
 

Looks like the blog is already back. As the second screenshot shows, Blogger already provides a method to say “Hey, my blog is real–please re-enable my account.”

Posted by MattCutts | Report as abusive
 

Hi Felix – Thank you very much for bringing this to our attention. I removed the block on John’s blog, and am looking into why the spam block was erroneously applied to the blog in the first place. I’ll follow-up with more details here once I have more info about what happened.

@Josh Fulton – please send me details by e-mail (rklau -at- google.com) about your blog’s removal and I’ll get it resolved asap.

Rick Klau
PM, Blogger

Posted by rklau | Report as abusive
 

John’s very lucky that he got his blog back so quickly. In my case I followed that link and the whole process took about a month, and only because I was able to find a guardian angel at Google.

Posted by Kiffmeister | Report as abusive
 

Rick, can you let us know if blogs are shut down manually by Google or if the process is completely automated? If fully automated, can you give us a hint as to the rules?

http://cv.rklau.com/

Posted by Uncle_Billy | Report as abusive
 

Google’s innovation and growth are one of the big highlights of a decade that people are fond of complaining about. I think Google is utterly awesome. I am a big fan.

That said, Google does a lot of evil things now that they hopefully won’t be doing as much of in the future, including:
* Cooperation on censorship in China and total obeisance to that freedom-hating regime
* Shockingly illegal book copyright violations, and utter contempt for authors’ rights
* Wholesale profiteering through content aggregating (i.e. Google news) while providing no renumeration to the content providers, playing a big role in the tragic decline of media.

So here’s ringing in the new year with hopes for a less evil Google.

Posted by DanHess | Report as abusive
 

Make no mistake

Google is evil
Mr. Klau is interested in one thing Monetization and Reuters is high profile

imagine if you were some schelp blogger trying to make you way in the world

Google responded because Felix is would be among the top ten globally in the world of blogging and he was giving google bad press –

F google

http://cv.rklau.com/

Posted by guestaz | Report as abusive
 

@Uncle_Billy – The system is automated, with manual review when individuals request it. As to what rules we use, that’s a moving target (and not something we generally disclose, given how helpful that would be to those who would abuse it).

Re: guestaz’s comment that I responded only because of Felix’s profile, that’s simply not true. I routinely help individuals on Twitter, reach out by e-mail to individuals who need help and regularly drop in on blog posts made by users expressing frustration. I felt strongly about this when I was at FeedBurner (before being acquired by Google) and it’s equally true now:

http://tins.rklau.com/2005/10/heard-word .html

–Rick

Posted by rklau | Report as abusive
 

Rick, thanks, understood. Should have asked the more pointed questions though: Does google *ever* manually disable a blog? Has it ever?

Posted by Uncle_Billy | Report as abusive
 

Google took down my PrestoPundit blog almost immediately after I broke the story on The socialist economics of Barack Obama, Sr., the Harvard trained economist, and Barack Obama’s father.

Google effectively killed the blog and my highly linked story on Obama’s father.

No human ever responded to my pleas to put my blog back into existence.

Posted by Anonymous | Report as abusive
 

There was a very well written, thought provoking, very popular, politically libertarian blog, “Just a girl in shorts” by Becky Chandler that Google kept shutting down because people with opposing political beliefs claimed the content was objectionable. Finally and sadly, Becky gave up and stopped blogging.

The funniest part was Google’s response to the fact that there were so many complaints to Google about blocking the blog, especially on Google’s blogger support pages since there was no other obvious way to indicate readers’ unhappiness with Google’s action. Of course, none of the support responses dealt with the main blocking issue.

Google’s support responses were;

“I [support] am asking that you take action with your readers. There are now a dozen threads taking up space in BHF [blogger help forum], with everybody whining about this one blog…”

And another:

“But please, knock off the multi-posting. Your posts are starting to annoy the folks who help here.”

Above from one thread of posts about her blog among many: http://www.google.com/support/forum/p/bl ogger/thread?tid=164b7aa43cfacad4&hl=en# all

Posted by MiltonRecht | Report as abusive
 

ive been flagged as spam as well, as i link to several hundred articles a week…it is the amount of links that trips the robot; after a dozen captcha entries, i regained full access…

Posted by rjs0 | Report as abusive
 

Look at mine: http://control.blog.sina.com.cn/myblog/h tmlsource/blog_notopen.php?uid=167513894 3&version=7&x

It was shut down by sina.com.cn last night, without any notification! And it’s very common in China. Many people have the same experience across the country.

Posted by Anonymous | Report as abusive
 

In addition to a high link content, one thing that might trigger the spam detection algorithm is large sections of text being quoted. If the blog was quoted in lots of other higher page rank places, the algorithm is more like to see those places as the source of the info.

Posted by mattmc | Report as abusive
 

@MiltonRecht – I wrote up a detailed post about the Girl in Short Shorts situation last July:

http://tins.rklau.com/2009/07/blogger-an d-adult-content.html

Shorter version: Becky violated our content policy, and her blog was labeled as having adult content (which it did).

I regret the tone of the quote in the help forms (“please, knock off…”) but please note that was not from a Google employee but one of a number of volunteers who try to help address issues when they come up. I spoke with that person about how they communicate with our users, it’s certainly not how I’d like to see users treated.

@Uncle_Billy – our policy, support and legal teams routinely are called upon to manually review blogs for a variety of reasons. If they are found to be in violation of our Content Policy (http://www.blogger.com/content.g) then they will be shut down.

Posted by rklau | Report as abusive
 

Joe Biden’s son is his brother?!

Posted by Anonymous | Report as abusive
 

Anyone besides me think this alibi is much too convenient? Every decision Google makes can be explained away as an “internal bug.” And none of this tech is discussable because it would give advantage to the evil ones. It is well known in many communities that Blogger shuts down blogs because of outside pressure. People like Yves just get a taste of it and ultimately immune because of the privilege of knowing the right people.

Posted by Anonymous | Report as abusive
 

Simple solution here, folks. Don’t use Google provided services. There are hundreds, if not thousands, of places to blog. The worst that can happen is you might have to pay $5/month for the privilege, but then you will have full control.

Or just use wordpress.com – free, not evil (or at least less likely to be evil…).

Posted by ChrisMaresca | Report as abusive
 

If you think for a nano-second that Google doesn’t profile any and every blog on the net that is indexed by them, I have a little test for you!

Just post any article starting with Obama (add any negative or mocking by-line) and find out how quickly your site will be banned from Google. They have filters of all kinds that sus out any negative phrases that don’t match their political agenda on the far left.

We had over 15K articles posted over a two year period trashed and our site marginaizled with no direct links in google search, in less than 10 hours after we printed a satire cartoon on Obama.

Google ARE the brown shirts of today and the future!

Posted by No1Uno | Report as abusive
 
 

> We had over 15K articles posted over a two year period
> trashed and our site marginaizled with no direct links
> in google search, in less than 10 hours after we printed > a satire cartoon on Obama

How can you do it? On average, you posted ~20 articles a day for two years. Are you sure you are not spammer?

I don’t think Google cares whether your blogs are pro- or anti- Obama.

Posted by 3.14 | Report as abusive
 

Google showed great leadership in re China today! Rock on, Google!

Posted by DanHess | Report as abusive
 

Just to give the opposing opinion to this. I’m a webmaster, and I pay for my site to be hosted by a professional company. Every day dozens of Google Blogspot sites steal my text word for word and puts them on their blogs – google then serves these pages up in the search results!

I report them for spam and google usually deletes their blog. I understand that there are some google hosted blogs which are decent. But there’s a lot more which are complete trash and harmful to people like me who rely on their websites to make a living – not just express their opinions.

From what I’ve seen Google does a pretty good job at restoring blogs which do generate their own original content and deserve to exist. Case in point John Hempton’s blog.

If you’ve got something to say which is important don’t rely on a free hosing service. It cost’s almost nothing to set up your own site!

I think the real evil here is the fact Google facilitates this nameless, free dumping ground for people to steal other people’s hard work.

Posted by awebmaster | Report as abusive
 

Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/
  •