Posts Tagged ‘seo’

How to exclude posts from LinkWithin with the robots.txt file

January 24th, 2010

The LinkWithin plug-in has become very popular among bloggers because of its ability to link to related posts by displaying thumbnail images from those posts. We are a highly visual species, so the inclusion of images next to the links makes them far more appealing and increases the likelihood that readers will click on them. Linking to related posts on a blog encourages readers to spend more time on a site and each link helps increase search engine optimization.

This feature is particularly popular among food bloggers who frequently post pictures in their entries. Here is an example screenshot of the plug-in taken from a client’s site, Simply Sugar and Gluten Free:

LinkWithin plug-in

The downside? Bloggers don’t necessarily want all of their posts to appear in the LinkWithin widget. One client of mine manages events, sponsored reviews, and site pages in WordPress that she doesn’t want to promote at the bottom of her posts. Looking around the LinkWithin site, no solution to this problem is listed, nor did they respond to my request for help.

However, I was able to find my own solution after I realized LinkWithin operates similarly to a search engine. It indexes your site to generate a listing of posts and has to update this index regularly to include your latest posts. You can control what parts of your site a search engine indexes by creating a robots.txt file on your site. You can get more complete information on robots.txt here, but for our purposes, all you have to do is create a file titled robots.txt in the root directory of your web server with this information in it:

User-agent: *
Allow: /

User-agent: LinkWithin
Disallow: /directory-or-pages-you-want-to-hide-from-LinkWithin/
Allow: /

The first block of code says that it’s ok for any search engine to index your site. The second block then tells the LinkWithin spider not to index anything in the page or directory listed after the “Disallow:” text. You can add as many Disallow lines as you want, either to hide specific pages, or to hide any pages within a directory. If you want to disallow a directory, but want to include one specific page in that directory, first disallow the directory and then add the specific page to the Allow: line.

I hope this helps anyone who’s run into this problem with the LinkWithin plug-in. Happy blogging!