- Send us a note
- info@addicottweb.com
- (773) 633-7078
Avoid Duplicate Content on WordPress Websites
Are you new here?
If this is your first time visiting our website, welcome! Learn more about us and what we do, or look through our complete blog archives for more good ideas.
Posted on May 12, 2009 under SEO, Wordpress
WordPress is a great tool to build a website in, but when it comes to search engine optimization, there are a few areas where it needs some improvement. The issue of duplicate content is one of them, so I’d like to talk about what duplicate content is before presenting 5 easy ways to fix that problem in WordPress.
What is duplicate content?
Simply put, duplicate content is any text on your website that either completely matches, or is similar to, content elsewhere on your website. While there are acceptable kinds of duplicate content – print-only versions of a web page, for example – in other cases people intentionally use duplicate content across multiple domains in an attempt to get more traffic to their website from search engine results.
It’s the latter usage of duplicate content that is the reason Google and the other major search engines penalize you for having it on your website. There’s no way for them to understand the intent of why the content might be duplicate – even if you aren’t duplicating content maliciously – so they just penalize it altogether.
That being the case then, optimizing your website so that it avoids duplicate content is something you need to do if you’re interested in your website’s placement in search engine results.
(For more tips and explanations from Google about duplicate content, read through their page on the subject in the webmasters/site owners guide.)
5 ways to avoid duplicate content
When you install WordPress out of the box, it’s not duplicate content proof – and that especially pertains to your blog posts and how they’re displayed. For example, if you have your category, archive, and home pages all set up so that they display the full text of your blog posts, guess what you have? Duplicate content.
Here are 5 simple changes you can make to avoid duplicate content on your WordPress website:
- Display the full text once and only once – My rule of thumb is that the full text of a blog post should only be displayed on the actual page of the blog post itself. Everywhere else your recent blog posts are listed, you should either have the excerpt appear, or just the name of the post and a link the full text of it. To display the excerpt only, you can either update your theme files, find a plugin to do it for you, or just use the <!–more–> tag when writing your content.
- Fix your page header – You should insert the following code into your theme’s header file to make sure that certain pages (such as the homepage, posts, pages and category pages) are indexed by search engines spiders, while certain others (feeds, archives, etc.) are excluded :
<?php if((is_home() && ($paged < 2 )) || is_single() || is_page() || is_category()){
echo '<meta name="robots" content="index,follow" />';
} else {
echo '<meta name="robots" content="noindex,follow" />';
} ?> - Be aware of comment pagination – In WordPress 2.7, you have the option of separating your comments onto multiple pages rather than lengthening the actual post page. The only problem with this is that for every page of comments, you’re duplicating the content that people are commenting on. This function is enabled by default in WordPress 2.7, so if you don’t have a need for your comments to be paginated, go to the “Discussion” area under settings and uncheck that option.
- Add unique META descriptions to each post – I’ve written about META tag issues in WordPress previously, but the most important META tag to consider here is your description. If you have the same META description on all of your blog posts or pages, that’s duplicate content. I recommend the All-in-One SEO Pack plugin (what I use on this website) because it lets you use your excerpt (or whatever text you want) as the META description, thus avoiding duplicate content.
- Update your robots.txt file – If you don’t want search engine spiders to find unintentional duplicate content on your website, put some instructions in your robots.txt file that tells them what they shouldn’t crawl. In WordPress, that means making sure you exclude your feeds and any other auxiliary pages that duplicate content you have elsewhere. The following code will do the trick – just copy and paste it into your robots.txt file:
User-agent: *
Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$
Thoughts?
If you’re a web designer or developer and have a WordPress website (or have built WordPress websites for your clients), how do you help them avoid duplicate content on their websites? Are there any tips or suggestions that I didn’t mention that you feel would be useful to others? Share your thoughts by leaving a comment below!
Similar Posts
- SEO for WordPress: The META Tag Problem (January 30, 2009)
- Create Single Post Pages for Different Categories (July 14, 2009)
- Essential Plugins for Your WordPress Website or Blog (February 25, 2009)
11 Comments
Trackbacks/Pingbacks
-
[...] from similar problems. It is a well known problem and you can find an excellent article on how to Avoid Duplicate Content on WordPress Websites, which gives the appropriate steps to take. The most important step of all is to have the right [...]
Scott Million wrote on May 12, 2009:
Thanks for the heads up. Now as you know Google has issues with duplicate content on most blog site. Google wants to show unique content. That means that you need to make sure that each page you create is optimized for different keywords. As said, content really plays a huge part with your search engine marketing campaign and if your site offers relevant content for the search queries your site could really benefit from this.
Posted at 10:05 pm
RSS2MYSQL wrote on May 25, 2009:
Well one thing I think is that as this time has passed Google has become more efficient at weeding out the abusing duplicate content and the non-abusive duplicate content. It’s easy for a new developer to get confused and scared by these rules, and wonder things like, will my header menu be considered duplicated content because it appears on each of my pages? Should I rearrange it for each section so avoid being penalized? Seems silly but I and others have thought it and worried over it before.
No no, don’t worry over that. Google should not bully you into being so detail paranoid. And they don’t.
I’ve heard Matt Cutts explain before that there, in truth, are no penalties unless Google thinks you are malicious. The penalty is the showing of favor to whomever Google things holds the original content, and completely ignoring the contender that holds that same content. That is the penalty – the non indexing of the page in question. This does not mean that your entire website is duplicated… and Google will not treat it as such, but only the page in question.
Posted at 4:20 pm
KeHoeff wrote on May 28, 2009:
hey this is a very interesting article!
Posted at 3:17 pm
Redouane wrote on July 4, 2009:
Thank you for an interesting article. However, I tend to always check that bloggers providing advice are actually using it themselves.
I checked the source of your home page and there are no mention of the header fix you mentioned. eg. no noindex entry…etc
Also, looking at
http://www.addicottweb.com/robots.txt
I found
User-Agent: *
Allow: /
Any reason why you are using your recommendations on your site?
Posted at 4:17 pm
Addicott Web wrote on July 6, 2009:
Actually there was no reason for not updating my robots.txt file with what I had suggested other than I didn’t have a chance to at the time, and then quickly forgot about it!
Re: the header fix, if you look at the page source you’ll see it does have a META tag that reads:
That’s the header fix I mentioned in action. What the fix really is is a conditional statement, so you won’t actually see the full text of it appearing in the source code of an actual page on this website.
Posted at 11:25 am
Suzanne of New Affiliate Discoveries wrote on September 16, 2009:
Thanks for this post, you’ve summarized a lot of basic wordpress SEO issues here. I found the same or similar advice elsewhere, but was looking for a recent post to be sure nothing has changed. You’d think google would figure out teh wordpress issues. Do you think SEOs would be out of work of google managed ot stop duplicate indexing wordpress sites?
One thing…I ran across a plugin for the header issue so you don’t have to change the code yourself. I’ts called “duplicate Content Cure” what do you think of it?
Posted at 7:55 pm
Addicott Web wrote on September 17, 2009:
Thanks for your comments Suzanne! I actually think that while there’s some definitely SEO concerns about duplicate content, it’s also more of a usability issue. Too much duplicate content can make it awful confusing for visitors when looking through a website. I’ve seen too many blogs where they have the full posts displayed both on the home page, on their blog page, etc.
I also think that it’s not necessarily Google’s problem to figure out the SEO issues of WordPress. I think the creators of WordPress websites should know what to do to make sure that their websites are not only SEO-friendly, which includes knowing how to structure the site not to include duplicate content – especially coming from a usability perspective, as I mentioned.
Regarding the plugin you mentioned, I think whether you use it or not depends on your familiarity with inserting some code into the header.php file in your WordPress theme. All that the plugin does really is something that can be simply – but for those who aren’t experience with it, or don’t want to bother delving into the code, then yes, it’s absolutely worth trying.
Posted at 11:14 am
Svetoslav Spasov wrote on November 2, 2009:
Hey there!
Great article. I just added your recommendations to my robots.txt. You forgot one important string, though- /author. Because, if it’s a one author blog, the front page and the author page will be pretty much the same. Also, don’t you think that disallowing /wp- will prevent google from indexing your pictures? Just my two cents. Once again, great article.
Posted at 7:03 pm
Keith Davis wrote on November 22, 2009:
I’m in the process of setting up my first WordPress site and have started looking at the Duplicate Content issue… hence my finding this article.
I’m used to static html sites, where duplicate content is easy to avoid, you just don’t repeat yourself. But working with pages that are not pages, just made up of includes from headers footers and all the rest… is a bit confusing.
Thanks for setting things out in such a clear and easy to follow way. I will certainly follow your advice.
Posted at 6:34 am
Mike Atterlis wrote on June 16, 2010:
Great post, the reason why one should not have duplicate content is that it would confuse the search engines and basically could interfere with a pages rank.
Posted at 9:25 am