It began with a hashtag — #fitn. On the eve of January’s Republican presidential primary, it seemed that every member of the political press, election observer, and New Hampshirite had adopted #fitn as a sort of quasi-official tag. It was a reference to “First in the Nation,” a long-used political phrase that dates back to the 1920s. As I watched those tweets fly by, it struck me how ubiquitous its shorthand version had become online. Where did the hashtag come from? Who first injected it into the tweet stream? Twitter’s internal search engine, as it turns out, only goes back so far. I fired up Topsy.com, by general consensus the best tweet search tool going today. But I hit the outer limits of Topsy’s archive far before I uncovered my proto-tweet. I asked Twitter HQ. No go. A smallish company, it lacks the resources, they said, to track a hashtag back to its starting point.
My struggle to find the origins of #fitn is not unique. We’re tweeting more than 340 million times a day, conducting a robust public conversation on Twitter. Yet, even on Twitter’s sixth birthday today, we still can’t track it, can’t search it, can’t access our archives. There is no public record. Is that really so much to ask?
Maybe, yes. Consider the technological constraints. Brewster Kahle, who runs the Internet Archive, a non-profit online repository for 150 billion Web pages, told me startups have a hard time being “archive aware.” For them there are more pressing concerns, like integrating servers and avoiding “fail whales.”
Twitter’s internal search tool only reaches back a week or so before you get a note saying that older tweets are not available. Twitter does, to its credit, publish an interface that allows others to pull information from its services. But there’s a built-in cap on how many tweets can be accessed that way. (It changes, but at one recent point it was in the couple-thousand-tweet range.) And so, we’re left with our current status quo: tweets that seem to fall into a black hole. (Twitter declined to speak on record for this piece.)
Who cares, right? These are tweets, after all. Somehow we’ve survived as a culture without recording, say, every phone call we made in the ’80s. But Twitter’s centrality to the political conversation from the U.S. to Egypt has already made it more than mere ephemera. It’s still the early days of the social-media era, and our vantage point is not a particularly good one to decide what’s worth saving and what’s not.