Felix Salmon

Another RSS feed gets truncated

By Felix Salmon
April 29, 2010

One of the things missing from the Great RSS Truncation Debate (see me, passim, ad nauseam) is much in the way of empirical data. So, in the wake of the WSJ’s recent decision to truncate its blogs’ RSS feeds, it’s going to be interesting to see what happens over at Above The Law. It’s truncating for the next month, and seeing what happens, against the sage advice of executive editor Matt Creamer.

Breaking Media’s CEO, Jonah Bloom, explains that it comes down to saving a bunch of his own time:

In our case, the move was prompted as much by my annoyance at the growing group of content thieves scraping our content via RSS (I dealt with two yesterday), as it was by a desire to get some commercial benefit from those readers. We’re a small company with limited resources, and I got fed up wasting valuable time trying to track down these parasites who aren’t only benefiting from our editors’ hard graft but also potentially messing with our search engine results by creating duplicates of our content on other sites.

On the other hand, notes Creamer, scrapers don’t need full RSS feeds to scrape content, and it’s becoming increasingly outmoded to think that the only way people should read online content is by pointing their computer’s web browser at a web page.

Most content providers get furious at the idea that anybody is stealing their work, and therefore waste valuable time trying to track down such parasites. I’m much more sanguine about scrapers: they exist, they get a tiny bit of search-engine traffic at the margin, but they make no visible dent in readership, and they never actually become popular in their own right. (Although I’d love to know the story behind the Ethiopian Review, a quite beautifully simple web-scraper which carries no ads, as far as I can tell; it’s the only scraper I’ve seen which people might actually want to come back to.) In general, I’m flattered by scrapers, just as I’m flattered by people who send my blog entries around by email. It’s all good in the long run.

In any event, I’m looking forward to Bloom reporting back in 29 days and telling us all what happened to both the number of scrapers and the quantity of web traffic after he truncated his feeds. The more datapoints we have on this, the better.

Update: dWj has a good idea in the comments:

I would think randomly truncating 10% of the entries would annoy scrapers a lot more than regular readers, at least relative to truncating 100% of them.

Has anybody tried this?

Update 2: This is funny. And, I see ads on it! (Via)

3 comments so far | RSS Comments RSS

The WSJ truncating has had the most effect on me so far.. personally, I don’t enjoy the WSJ website.. its slow within my brower and a “bulky” site to load up each time, so I’m much more likely to NOT click through, vs. Dealbreaker (and assume ATL as well, similar backbone) or others that allow me to navigate over and load quickly. Wonder if that has anything to do with people “clicking through.” – it does for me.

Posted by Ooh_La | Report as abusive

Completely agree Ooh_La. The time it takes for me to load the page in my browser frankly isn’t worth the content about 50% of the time, especially when it’s just bad puns from Matt Phillips on MarketBeat. It’s also super annoying that you can’t read an entire entry on the main blog page. So even if you visit the site you still have to click through to each specific entry. My subscription to the full paper expires next week; this has singlehandedly convinced me not to renew.

Posted by MitchW | Report as abusive

I’ve clicked through for some of the WSJ content, but certainly a lot that I would have read before I haven’t bothered with. This may still be a net gain from their standpoint, depending on what they’re trying to maximize here.

I would think randomly truncating 10% of the entries would annoy scrapers a lot more than regular readers, at least relative to truncating 100% of them. Maybe that’s not true.

Posted by dWj | Report as abusive

Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/