There was a quick discussion recently over at Google Groups which mentioned how to take advantage of subscription-only news articles from a search engine standpoint by allowing Google to crawl and index them.
Google’s recommendation for ensuring content is indexed on subscription sites is to detect whether or not a request for a particular article is coming from one of their bots, then allow the bot to bypass the registration page. This will allow access to the article which, in turn, will give it a chance to rank in Google News.
To create a positive user experience, Google also recommends allowing users to view the article one time without subscribing. You have to do so by detecting that the user is coming from a Google domain, then have that user bypass the registration page as well. It seems that a one time view could turn into multiple views with a simple clear of your browser cookies.
If this solution isn’t feasible, Google’s policy is to slap your publication name with a “subscription” tag to alert users that they may be prompted to subscribe or register on your site before viewing the requested article. It may just be me, but I would consider my user experience to be less than excellent if I was prompted to subscribe to an article I clicked on in a Google results page, subscription tag or not.
With Google’s official unveiling of Universal Search, more and more News results will likely be making an appearance in Google’s main search index, pushing other sites down in the results.
This doesn’t only apply to news, though. Popular articles related to a range of topics, which were once blocked off from Google because of registration pages, could soon be updated to allow for indexing. The result being even more results with a Google “subscription” tag, as many site owners won’t likely appeal to the idea of following the “free first view” approach as a simple copy and paste could result in multiple free views.
On the other side of the coin, I understand the desire of subscription sites to have articles indexed and gain more coverage throughout the search engines. The point here, however, is providing users with the best experience possible, as Google claims to want to do, and refraining from bumping relevant pages with accessible content further down in the results for subscription-designated articles.
If you have any thoughts on the issue, I’d be happy to hear them.