Google sometimes lists URLs that it's not allowed to spider, because it's blocked by robots.txt, because a lot of links point to a URL. A good example of this is a search for [RTL Nieuws] (disclosure: RTL is a client of mine). rtlnieuws.nl 301 redirects to the news section of rtl.nl. But... rtlnieuws.nl/robots.txt exists... And has the following content:

User-agent: *
Disallow: /

Because of that, the links towards rtlnieuws.nl don't count toward the news section on rtl.nl, and Google displays rtlnieuws.nl in the search results. This is unwanted behavior that we're trying to fix but for now it's a good example of what I wanted to explain. By blocking /wp-admin/ and /trackback/ in your robots.txt, you're not preventing them from showing up.

Unfortunately, recently the /wp-admin/ block was added to WordPress core, because of this Trac ticket. In the discussion on that ticket, I've proposed another solution in this patch. This solution involves sending an X-Robots-Tag header, which is the HTTP header equivalent of a meta name="robots" tag. This would in fact remove all wp-admin directories from Google search results.

WordPress Robots.txt blocking Search results and Feeds

There are two other sections which are blocked in the suggested robots.txt, /*?, which blocks everything with a question mark and as such all search results, and */feed/, which blocks all feeds. The first is not a good idea because if someone were to link to your search results, you wouldn't benefit from those links.

A better solution would be to add a <meta name="robots" content="noindex, follow"> tag to those search results pages, as it would prevent the search results from rankings but would allow the link "juice" to flow through to the returned posts and pages. This is what my WordPress SEO plugin does as soon as you enable it. It also does this for wp-admin and login and registration pages.

I'm aware that that is different from Google's guidelines on this topic at the moment, which state:

Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don't add much value for users coming from search engines.

I've reached out to Google to get clarification on whether they would say my solution is acceptable as well, or perhaps even better :) .

Blocking /feed/ is a bad idea because an RSS feed is actually a valid sitemap for Google. Blocking it would prevent Google from using that to find new content on your site. So, my suggested robots.txt for WordPress is actually a lot smaller than the Codex one. I only have this:

User-Agent: *
Disallow: /wp-content/plugins/

I block the plugins directory because some plugin developers have the annoying habit of adding index.php files to their plugin directories that link back to their websites. For all other parts of WordPress, there are better solutions for blocking.

WordPress robots.txt Example is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Gravity Forms + Custom Post Type addon

By default, Gravity Forms allows you to create posts through a form. It doesn't have support for custom post types at the moment though, in part because a wonderful plugin was already created that allows for this. This plugin, aptly named Gravity Forms + Custom Post Types can be downloaded from WordPress.org.

Once you have both Gravity Forms and this plugin activated, you can start creating a form. The first step is to make the form fill our custom post type. We start with creating a form and dragging in a title field:

The title field can be found in the posts field section of Gravity Forms field, below the advanced fields:

Once you've added this input field and given it a name, go to the advanced section of its edit block, you'll see an option to save as post type, this has been added by the afore mentioned plugin:

You check the box and select the custom post type you want to use, in my case, WPSEO Themes. Now we start adding the form. We need a couple of different types of values:

• The title: done.
• The "description", which will just be the body text, so you can easily drag in the Body input field.
• An image, which should be saved as the featured image too, more on that below.
• Several custom fields, more below too.

Adding a featured image trough the form

This is actually pretty easy: drag in an image field and click edit, you'll see something like the screen below:

As you can see, setting the image as featured image is as easy as ticking the box. It's wise to also ask for a description if you don't know what's going to be on the image. In my case, it's a screenshot of the theme, so I won't bother and just set the alt tag automatically.

Adding custom fields through Gravity Forms

The next step is to add the several custom fields we need. In my case I had 5, but you can have as much as you want. You start by dragging a Custom Field input into your form. Once you have that, you click edit and you select the appropriate custom field type:

In this case, I'm asking for the theme URL, so I select website, but there are all sorts of options you can choose from, as you can see. Now here comes the tricky part, you need to set the name of your custom field. You should go into your Types -> Custom Fields page and check the second value below the custom field title:

That's the name of your custom field, but you should prefix it with "wpcf-", because that's the Types plugin naming convention, which prevents its custom fields from clashing with other ones.

Of course, if you created a custom field group from already existing custom fields you don't need to prefix the custom field name.

True / false or "boolean" input fields

Some of your custom values might be checkboxes, they're either on or off, true or false. That's called a boolean value in math / developers language, but for you, it's really simple. Just create a custom field type "checkboxes", and go into it's settings:

Be sure to check the "enable values" box and set the value to just "1". That way, if checked, Gravity Forms will save it as value "1" and the Types plugin will "get it".

Deciding on workflow

Now, once you've used the above info to finish your form, you need to decide on a workflow. On the post title field, the one whose advanced settings we used to save this input as a custom post type, we now go to the "normal" properties:

As you can see, you can set a default post author and a post status. Now in my case the author will be me in most cases, as nobody will be logged in. However, if you have enabled registration on your site, you can force people to be logged in before even being able to use this form, by going into your forms advanced settings and checking the "require user to be logged in" checkbox:

This allows for all sorts of workflows, find one that suits your site!

Conclusion

We still haven't written a single line of code, yet we've already created a custom post type and created a form that allows people to submit custom post types to us.

So, one more thing to check of off the to-do list:

1. Creating a custom post type + custom fields.
2. Creating a form through which people can submit themes that fills this post type.
3. Creating a browsable interface for this post type.

In my next post, I'll explain how to use the Views plugin to create "views" for this post type and unveil the finished product!

Use Gravity Forms to submit custom post types is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Database of Themes that support my WordPress SEO plugin

I've also got a project I'll use this for: I want a database of themes that support my WordPress SEO plugin, with some specific settings info, a screenshot, etc. I want to store these as a custom post type. So the first step is to determine which info I would need to store:

• Basic stuff:
  • Title of the theme
  • Short description
  • Screenshot
  • URL
  • Is this a paid theme or not?
  • Price (if applicable)
• And some more advanced stuff:
  • Does this theme have its own SEO options that "yield" to WordPress SEO?
  • Does this theme support breadcrumbs?
  • Does this theme require force rewrite titles to be on or not?

Creating a Custom Post Type

Having determined what I wanted to store, the next step was to create a Custom Post Type. That's as easy as using this interface:

I could add Taxonomies to it as well, but I'll leave that for now, although creating a taxonomy is just as easy through the Types interface. I end up with my WordPress SEO theme CPT:

Adding Custom Fields

You'll think "huh, that hasn't got any of the specific data yet": that's right. It doesn't. That's where the true power of Types comes in, you can create "Custom Field Groups" and add these to post types. So I did:

As you can see you can choose from a lot of different types of fields, and all these types have their own content checks. For instance for a URL, it'll allow you to "force" a correct URL. I've added the custom field group to my WPSEO Themes post type, and now, when I go into edit or create a new WPSEO Theme "post", I get this interface below the title and content area:

So far, no coding was required, thanks to the wonderful Types plugin! You can get that, for free, on wp-types.com or on WordPress.org.

So, what we needed to do:

1. Creating a custom post type + custom fields.
2. Creating a form through which people can submit themes that fills this post type.
3. Creating a browsable interface for this post type.

Subscribe below to make sure you won't miss the next two steps!

Types WordPress plugin – Easy Custom Post Types is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Forcing Clicky to track a link as outbound

First of all, you need to know that when you add a class clicky_log_outbound to a link, Clicky will track each click on that link as an outbound link. Now I always add a class track or aff to my affiliate links that I want to have tracked, for instance:

<a class="track" href="http://yoast.com/out/clicky/">Clicky</a>

Of course I could manually add the clicky_log_outbound class, but that's just a tad bit too much work, especially as most of my links have been "classed" already and I might want to use this class for other stuff later on.

Now, I add a tiny filter function to my sites functions.php file, this searches for links with class aff or track and adds the clicky_log_outbound class:

function clicky_outbound_filter( $content ) {
	$content = preg_replace('/<a([^>]+)?class="(aff|track)"([^>]+)?>/',
		'<a\1class="\2 clicky_log_outbound"\3>', $content);
	return $content;
}

add_filter( 'the_content', 'clicky_outbound_filter', 10, 1 );

That's it! Clicky will now track those links as outbound. Now let's set such a link up as a goal.

Goal Tracking in Clicky

For goal tracking you need to have a premium (read, paid) Clicky account, but it's well worth it. Setting up a goal is a piece of cake. You go to your site's analytics and then to Goals, Setup. You'll see this interface:

You simply name the action, set the goal URL to be your outbound URL, you could do /out/clicky/ or, when you sometimes forget the last slash, /out/clicky*. You can set up revenue and even a funnel, a required page before one can reach that goal.

Once you've done that, the goals will show, in realtime, in your Goals overview and in your "bigscreen" Clicky display.

More tracking power: campaigns

This feature becomes even more powerful when you combine it with another great feature of Clicky; campaign tracking. Clicky fully supports the _utm type variables Google Analytics uses, so you can track campaigns and terms within campaigns. That is, in fact, how I did the tracking in yesterdays post about WordPress Dashboard widgets.

Tracking Outbound / Affiliate Links with getClicky is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

When I added mine, in the beginning, it drove lots and lots of traffic. People weren't used to it yet and thought I had somehow "found my way into core". Recently, I've added more elaborate tracking to my WordPress SEO plugin links. Allowing me to see how much traffic the individual sections of my plugin were sending back to my site. Let me share that with you now (click for a larger version):

Traffic and conversion statistics for plugin links for the last 28 days

As you can see, the widget sends a bit of traffic (1800 visitors in total) but only drove 3 conversions... Conversions on my site are click outs on affiliate programs and, more importantly, sales for my website review service. Turns out, people clicking on from the plugin interface or the plugin link are far more valuable visitors than people clicking on the dashboard widget.

So, in an effort to annoy less people and focus on the traffic that matters, I've just pushed out version 1.1.5 of my WordPress SEO plugin, without the dashboard widget. I will shortly remove it from my Google Analytics plugin too. Of course other developers should do their own analysis if they want to, but for me it's clear that the widget doesn't help enough to be interesting.

If you used the dashboard widget regularly to find new posts on my site, please consider subscribing to my newsletter using the form below!

On WordPress Dashboard Widgets is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

There's been some outcry though, because Twitter and Facebook aren't "highlighted" as much as Google+ in those new social results. Danny is doing some awesome reporting on this, first in "Search Engines Should Be Like Santa From “Miracle On 34th Street”", later in an interview with Schmidt.

Google used to have access to the Twitter firehose, all the tweets coming in in realtime, enabling them to index tweets at light speed. Facebook used to show some friends of a person on a profile to visitors to that profile who aren't logged in, now look at the cache for my Facebook profile: just other people with the same name.

As I said in a reaction to a Google+ post by Jeff Jarvis: what both Twitter and Facebook are afraid of is that they're "giving" "their" social graph to Google, thereby allowing Google to easily grow its own social network because it would make it very easy for Google to suggest friends to you or say "these friends of yours already use Google+, shouldn't you use it too?". So by opening up, they'd open their books to a competitor.

This, ultimately, should be a users choice, not a platform choice. When it does become a user choice, of course Google should favor the social network the user is the most active on, so if I'm more active on Facebook than on Twitter or Google+, it should highlight that above the others. Right now, it seems to be mostly highlighting Google+, which will raise some eyebrows here and there and is food for discussion.

A while back at the first Fusion Marketing Experience in Brussels, Bas van den Beld of State of Search interviewed Olivier Blanchard and myself about search and social. We talked about how the two intertwine and can't be unraveled, in fact, as Olivier said during the interview: "it's like coffee and cream, once they mix you can't get the cream out of the coffee". See the interview here (the sound is not the best ever, I know):

The thing is: this is a done deal. There's no way back. Search and social have now officially teamed up, so you might as well live with it. It also means that not using Google+ is... Not really an option if you're a marketer, but I guess we had that one coming for a while as well.

So, what does this mean from a tactics perspective? For now, it means: share every post on Google+ too, make sure you have Google+ buttons on your posts and, most importantly: keep building relations with people! It's not like that much changed; social mentions might have become a new and maybe even important ranking factor, but even quality links are usually the result of a relation, of social interaction.

The formula to success didn't change: you have to keep building relations / followers / an audience, create great content and make sure people notice it.

Search & Social – you can’t get the cream out of the coffee is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

<embed src="http://vimeo.com/moogaloop.swf?clip_id=31100268" type="application/x-shockwave-flash" width="400" height="300"></embed>

If you're in the US, it's time to stop SOPA, so make your voice heard. If you're outside, call on your US friends to do what ever is needed to stop SOPA. As bloggers united, we can make a difference, let's make that difference. Now.

Stop SOPA, help the internet. is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

For those who do decide to yield and for those who do not integrate SEO functionality into their theme, this is the guide to follow on how to make sure your theme works with all the major SEO plugins and more in particular, my WordPress SEO plugin.

1. Basics
2. Head elements
   1. Titles
   2. Meta's
   3. Link items
3. Body elements
   1. Breadcrumbs
   2. Other body elements
4. Is your WordPress SEO plugin active?

Basics

Whether or not your theme is in the WordPress.org repository, you should strive to meet their Theme Review guidelines. With the theme check plugin you can simply check whether that's the case for most of the technical aspects. This makes sure you have all the necessary hooks for plugins to do their work. If any of those fail, fix them before even looking at the stuff below.

Head elements

Titles

The most common issue with SEO plugins and WordPress themes is with titles: theme authors have a tendency to hardcode (sections of) titles into their themes, which results in hard to fix behavior, which I recently discussed here and hasn't been fixed in WordPress core yet. In short, your title should be not even a tiny bit longer than this:

<title><?php wp_title(''); ?></title>

If you decide to embed SEO functionality in your theme, you should then put a filter function on to that wp_title functionality just like a plugin would. This allows you to "unhook" that filter when an SEO plugin is active.

Meta's

Meta tags come in a vast amount of shapes and sizes, but there are only a few that "matter". There are the meta description, the meta keywords (on which I have a strong opinion too) and the meta robots tag. In my opinion, your theme should never embed any of these meta tags, but, if you decide to embed SEO functionality into your theme, write functions for these things and hook them on to the wp_head action.

Be very, very sure to allow for site owners to fully disable any meta functionality you write. I regularly, read: daily, advise people to change themes when their theme has a lot of SEO stuff embedded that is just plain wrong or collides with other plugins. Make no mistake: a portion of your users, probably a large portion of your users, cares about their SEO. Getting it wrong is worse then not doing anything, so if I were a theme author I'd most definitely opt for the latter.

Link items

There are a lot of <link> items that matter for SEO these days. There are rel=canonical, rel=next and rel=prev and a few more. WordPress does rel=canonical on single posts and pages but nowhere else yet, though a patch for core is ready to make WordPress add rel=canonical in more cases, but your theme should not do any of these. You're, to be honest, bound to get them wrong.

If you do decide to add them, do it in the same fashion as meta elements: hook them on to wp_head and make sure site owners can disable the functionality.

Body elements

Breadcrumbs

While the above points make integration for any SEO plugin easier, this point is more specific for my own WordPress SEO plugin. Integration of my breadcrumbs is quite easy, use something like the following code:

if ( function_exists('yoast_breadcrumb') ) {
   yoast_breadcrumb('<div id="breadcrumbs">','</div>');
}

This code will not display anything unless the breadcrumbs feature of my SEO plugin is enabled.

Breadcrumbs Variables

This breadcrumb path takes the following 3 variables:

$prefix
The code that your breadcrumb should be prefixed with. Default to an empty string.

$suffix
The code that should be added on the back of your breadcrumb. Default to an empty string.

$display
If set to false, will return the breadcrumb path instead of echo-ing it. Defaults to true.

Other body elements

You should take note of my guides on HTML heading structure and HTML Sitemaps. While my SEO plugin doesn't "interfere" with those, it's good practice to make them optimal. You should also read my WordPress SEO Theme Guidelines to make sure you're doing all the other necessary things.

Is your WordPress SEO plugin active?

You should check whether a constant named WPSEO_VERSION is defined, preferably like this:

if ( defined('WPSEO_VERSION') ) {
// Disable your SEO stuff
}

WordPress SEO Plugin Theme Integration Guide is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Today I was sick and tired of getting beaten on some rankings I was working hard for, so I decided to dive a bit deeper and see why GoDaddy was ranking as well as they were. When I looked into the link profile for those high ranking pages, I found a lot of homepages linking to these landing pages with highly optimized anchor text. These were anchor texts like "ssl", "bulk email", "web hosting", "web hosting companies" etc. Stuff like that just doesn't happen by accident, so there had to be a reason for those. I was baffled when I found what they were doing.

Want a Website Tonight, anyone?

You see, GoDaddy offers a service called "Website Tonight"; this service allows you to quite easily create a website by offering you an editor and all sorts of widgets. Not exactly the power of WordPress, but nothing wrong with it from the users perspective. What is wrong is what I found when I created such a website: when you create such a website it has an image in the footer by default saying "Powered by Website Tonight". It's possible to turn this image off, but most people don't bother as in the editor it looks rather harmless, like this:

Now, if it were just that, I don't think I'd be all that bothered (not the border is because the image is selected). The issue is, that on the live test site I created, it looks like this:

That link wasn't there in the preview... That's called deceiving your customer. Note that by default, the image is black, you can switch it to white or you can switch it off, but in the editor it'll always show. This is probably the reason why some people choose to use the white version, as they think they can't disable it and want a version that's less ugly on their design.

Example time

Ok it's time I show you some real live examples of these I guess, these websites all have ugly links like that in their footer:

• http://www.motorinsurancee.com/
• http://www.universalhealthinfo.com/
• http://www.autoinsurancecoinc.com/
• http://handson3rd.com/
• http://www.onemilerunner.com/
• http://www.trophyshowroom.com/

But those links don't work, right? Wrong.

Google has been telling us for quite a while now that footer links etc. are not that important. Well guess what, that's not true if you have enough of them. Using SearchMetrics I ran a report for the top keywords they rank in the top 3 for. Each and every keyword in there that is not their brand name, from website hosting to webhosting to website builder, to domain name registration and more: all of those landing pages have exact match anchor text links pointing to them. All coming from these types of domains, thousands if not tens of thousands of clients who are paying for a service, are unknowingly also helping GoDaddy's business by helping it rank.

These links are on by default. They are not editorial. It's not the first time this happened, Hostgator has been caught adding links to their clients websites in the same way, I mention that in my WordPress hosting article. The issue is that Google rewards these kinds of practices with top rankings, which they shouldn't.

How well this works, well by my estimate they started doing this more aggressively in September / October of this year, see how their visibility according to SearchMetrics almost doubled:

This would correlate well with the Majestic SEO's historic back link data:

For hosting related terms like the ones GoDaddy targets, doubling your search engine visibility like that is worth a fortune. To show you even more how blatant these links are, this is a screenshot of the top pages report in Majestic, after doing an advanced historic report, look at the anchor texts and notice that the two with a flag on the right are reported wrongly, the anchor text for the link in fact is email marketing there as well. You can click for a larger version:

Some of these sites however already show these links a the beginning of 2011. See this archive.org example and this one to see that, they even changed the link in the meanwhile... What I think happened in September / October that made me catch them was that they started doing this for more keywords.

The long and short of it

GoDaddy is playing this game a bit too aggressively in my opinion, and Google should really start discounting those links. The right way would be for GoDaddy to ask their customers whether they're allowed to insert a link and make them choose where it points. No single customer would, by own volition, link to an email marketing page...

I am, though, disappointed in Google's filtering of these links; there are far too many spammy links pointing at those pages that:

• have a very unnatural anchor text distribution
• they're all in the footer of these sites
• are distributed over only a select number of IP's.

Those 3 things combined, I can't believe they didn't catch that.

Disclaimer: I'm not saying anything that GoDaddy does here is illegal from a legal point of view. In my opinion it's against search engines guidelines and they're not transparent towards their customers, so I'd call it bad karma.

Thanks goes out to Dixon Jones of Majestic SEO and Marcus Tober of SearchMetrics for helping me figure all this out.

GoDaddy’s spammy link building techniques is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Review snippets for everyone

For a while now, I had review snippets showing on my blog posts and pages here on yoast.com that I'd marked up appropriately. Over the last weeks I started noticing that I was seeing more rich snippets show up in more and more SERPs on sites that had not had them before. Yesterday I did a post on my Dutch blog to test this. I added a plugin called GD Star Rating, which allows people to rate posts.

This plugin has the ability to add rich snippet markup, but does it in a rather "hackish" way by hiding it:

It was good enough for testing though. I published a post about favicons, immediately gave it a 5 star rating myself so when Google spidered it, it had a rating, and low and behold, within the hour, this was showing in the Dutch SERPs:

Immediate rich snippets... You used to need to be whitelisted for this, but that doesn't seem to be the case anymore. I've tested this with both microformat hreview markup and schema.org markup and both work 100% fine, even when hidden. That last bit is "new" as Google has always stated that you should not hide your code for rich snippets, which is also what I disliked about GD star rating. In most cases though, it's absolutely unnecessary to hide the results, you'll want to show them with stars and you can still make that machine readable too.

The conclusion here is simple: if you have reviews and you haven't marked them up as a rich snippet yet, go do that immediately. I've written how to's on hreview & rich snippets for WordPress and on rich snippets for Magento in the past, use those. For some of my clients I've seen as much as 20 to 30% uplift in the past, on the same rankings, when getting rich snippets, just because more people clicked on those results.

Combined rich snippets and author highlights

If you do a lot of searching for WordPress related terms in the English Google results you'll know that I have a so called "author highlight", based on the rel=author markup (for which you do still need to be whitelisted):

These aren't live in any other languages than english yet, so that's why you're not seeing that for my yoast.nl domain yet. A while ago, the combination of an author highlight with a rich snippet for a review would look like this:

Google dropped the +1 button recently, as in, it only shows on hover now. But what also changed is how this rich snippet is displayed, the author image is made a bit smaller and now the result looks like this:

While it looks a bit less "prominent" it actually takes up a bit more space in the search results and shows the authors name twice, which seems weird to me, so I think they'll change at some point too.

Other rich snippets

It seems as though you still need to be whitelisted for other types of snippets. For instance, I implemented the new software application snippets for my WordPress SEO plugin and Google Analytics for WordPress plugin pages, which would look something like this according to the rich snippets testing tool:

But those aren't showing yet, nor are the reviews embedded in it.

Conclusion

There's no more reason to wait with adding rich snippet markup to your pages, you really should just do it right now. I very much doubt whether this will continue to be as easy as it seems to be now though. It seems to me as though this is to easy to spam and no doubt affiliates will make a lot of use of this "feature". My guess is that soon we'll be wishing for Google to go back to actually white listing websites for rich snippets for reviews as well, because are results have become too cluttered.

Check out this video from when Google introduced Rich Snippets:

Rich Snippets showing up everywhere is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

It includes a new tool built by yours truly to help you create the necessary redirects. Keep on reading!

Why change your WordPress permalink structure?

Most of the time I tell people to change their WordPress permalink structure, it's to get rid of the dates in their permalink structure. If their content is "timeless", it just shouldn't be there. In my opinion, the only type of site that should have dates in their permalink structure is a news site. All other sites should strive to write content that is "timeless".

Having a date in your permalink structure has proven to diminish the CTR from the search results for older posts. People are just not likely to click on a result that's two years old, even though it might very well be that your post has the answer they seek. With Google seeming to use that CTR more and more as a (very valid) ranking signal, that's becoming a more serious factor each day. I did a long post on WordPress SEO URL / Permalink considerations a while back, you might want to (re-)read that.

Performance considerations

Up until WordPress 3.3, there was / is indeed a quite serious performance issue when you have a lot of pages when you use just /%postname%/, luckily, that is solved in WordPress 3.3. I have debated people quite a bit saying that it's easily solved with some caching but having it fixed in core is a big step ahead, so there are no more excuses to not use /%postname%/.

Changing WordPress Permalink Structure

There are two steps in changing your WordPress permalink structure. The first is simple, go to Settings -> Permalinks and select Post name:

Change your WordPress Permalink Settings

If you don't have the post name option yet, you're not on WordPress 3.3, the release of which is imminent. You could wait a bit for the update, or you could just add /%postname%/ as a custom permalink structure.

The second step is to redirect your old permalinks to your new ones. To do that, you have to add redirects to your .htaccess file, I have created a little tool that generates these redirects for you based on your domain and your old permalink structure. To use this tool, click the button:

Generate Redirects

There you have it! If you copied the redirect into your .htaccess, you should test whether it's working. If it's not, chances are you're not allowed to use RedirectMatch, which makes changing your WordPress Permalink Structure a bit harder and not something I can easily cover in this post.

Let me know whether the tool works for you and what you've done to your permalinks!

How to change your WordPress Permalink Structure is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Adding that keyword to the title of every page is not going to help. Nor is writing 200 articles about it without one central article to link all those articles to. You need one single page that is the center of the content about that topic. One "hub" page, if you will.

That page will need to be 100% awesome in all ways. Brian Clark of Copyblogger calls this type of content "cornerstone content" and has written an awesome article about it (a few years ago, already). In fact, go and read Brian's article, he explains that way better than I can, I'll wait... ... ... You're back? Ok, read on:

Position that new Cornerstone Content within your site

That article said a lot, right? It told you about keyword research, title tags and headlines, content and why your content needs to be awesome and more. Now let's talk about where, within your site, that content is going to live. In my opinion, really important content deserves a page within your site's structure, not a news item / post. It should be easily navigated to within a few clicks.

So, you go ahead and create that page within your site. Take some time for it, this is going to be the content that's going to make you rank, but not just that, it's going to be the content that is ranking. Which means real people will read it too and you need to convert those people. So think about search engines all you want, but think even more about the visitor that will end up on that page and give him / her something worth while.

Creating Internal Links

Now, once you have that cornerstone page, it's time for the next step: creating internal links for your article. You're going to do this by figuring out which pages Google already thinks are relevant for your targeted keyword / key phrase. The easiest way to figure out which pages Google thinks are relevant for that keyword is doing a "site:" search in Google. So if I were to try and find the most important page for "website review" within yoast.com, I'd search for:

site:yoast.com website review

You will probably find more than a few pages within your site. Go into each of those pages and add a link to your new cornerstone content. If possible, use the keyword you're targeting as the anchor text for that link, but most importantly: link from within the content. Don't just add some site wide sidebar / footer links. The reason for this is simple: links from within content are way more valuable than links from sidebars.

Afterwards, when you're writing more content for your site, when you touch on a topic related to your cornerstone content, don't forget to link to it! Now, let's go on to the last and final step:

Promote your Cornerstone Content

If you've created it well, your cornerstone content is something to be proud of, something that others will easily share and thus also something that will attract links. Don't be afraid to reach out to other people who have written about related topics: show them what you created and that it might be worth while for their visitors to see that. You might even want to offer them to write a guest post about the topic, linking back to your article.

Stock photo credit: Computer Generated Concept Of Cornerstone from Shutterstock.

Using Cornerstone Content to make your Site Rank is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

After a couple of years of struggling with email I have finally found the solution to make sure each of my emails reaches the inbox of its recipient and I'm about to share it with you.

Note: this is not a paid review. In fact, since Postmark doesn't have an affiliate program, it's not even a post with some affiliate links. It's just me solving a problem for myself that I hope you will now be able to solve too.

The problem: email not reaching its recipient

I wrote about email reliability before, but let's be honest: getting all the web servers we use for sites to send email reliably is a pain. You need to setup SPF records, preferably set up DKIM too and make sure that your web servers do not get blacklisted. One of my servers, for which I had set up SPF, for some stupid reason got blacklisted a couple of weeks ago, resulting in a couple of website review emails not reaching my customers. I hated that so much that I started looking for another solution.

On this site, I had switched from Gravity Forms to Wufoo a while back. Wufoo is another awesome web forms service, with the at that time "added value" that Wufoo would take care of the email sending for me. My main gripe with using Wufoo was that I really wanted my forms and the entries of my forms in my site's install, not somewhere else. On top of that, Gravity Forms gives me a bit more programming flexibility, so I wanted it back. So, I had two email problems and started to look for a solution to both at the same time.

Postmark: reliable email

I found that solution in Postmark, which handles transactional email through a set of reasonably simple but reliable API's. Using this pre-built library, I was able to replace the email sending in my website review application for Postmark within about 10 minutes.

Yes, Postmark costs a bit of money, but if you consider that at $1,50 per thousand emails, you run a lot smaller chance of losing customers over email not reaching its destination, it seems to me that that's money well-spent.

It took me a bit longer to code a WordPress plugin that I liked for Postmark. There are a couple of WordPress plugins listed on the Postmark site, but all of them relied on CURL, which I don't have on every server I run WordPress on, and they all ignored some of the headers that plugins like Gravity Forms passed along. So I used the same pre-built library, but adapted it this time to use the WordPress HTTP libraries and added a wrapper to more reliably pick up all the headers that might get added by plugins.

The result is a plugin that is pretty easy to use, my own Postmark Email for WordPress Plugin.

WordPress email sending tips

By default, WordPress sends email from wordpress@example.com, where example.com is your domain. In 99% of cases this is a non-existing email address. More and more people, including Postmark, are saying that using a non-existent email to send your email from is not a good idea. I agree. So I created wordpress@yoast.com, which gets filtered into a tag in my email.

The great thing about actually creating that email address is that you can also assign a gravatar to it. This means that when people use email clients like Sparrow, or other email clients that support Gravatar, they'll see a nice avatar image for your email too. In my case, I made sure that image was the Yoast logo.

Go forth and email!

And please do let me know your comments about both Postmark, the plugin and your tips for reliable email in the comments!

Sending Reliable Email with Postmark is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Clicky (aff) has recently added what I call passive rank tracking to their already awesome analytics package. André wrote about stuff you can do with the rank tracking in the Google referrer data over 2 years ago here on Yoast. About 2 years ago as well, I used this same referrer data to build a passive rank tracker, independent of Google Analytics, because I wanted to do things with that SEO ranking data that I couldn't do in Google.

The one metric I wanted most of all from that big blog of SEO ranking data is the average rank from referrers. You see, because of personalized search results, rankings are never the same and there's no "good" way of tracking the impact of personalized search. Now you have a semi decent average in Clicky though, so I like that.

Because Clicky has a pretty decent API too, you could use this to do all sorts of cool stuff. I use this kind of data in combination with Authority Labs and a new toolset in my arsenal, SEscout. Authority Labs shows me all sorts of info about universal search other tools don't give me, SEscout does hourly rankings when you have a paid account, which helps me track fluctuations in "the force" more easily.

SEO Ranking Data in Aggregate

Rankings for specific keywords are very often not that interesting for my clients, as outside of the clients for my website reviews, most of my SEO clients tend to think in tens or hundreds of thousands, if not millions of keywords, not one, two or ten. So for them I track rankings in aggregate, something you should probably do too if you track more than a couple of keywords. You can then answer questions like "how did we do on this keyword group", "how did we do on that keyword group". Authority Labs allows you to tag keywords which makes this kind of analysis even easier.

I then compare that aggregate number to the SearchMetrics data for their site and see how good that number is (usually they correlate very highly), after which I can see how well their competitors did using SearchMetrics as well.

The issue is with both Authority and SEscout: they don't give you the complete view because they don't do personalized search, which is good, because we want to know our "real" ranking, but it's also bad, because it might not always correlate well with our traffic. That's where the extra layer Clicky added comes in, which allows us to see just how much personalized search impacts those real rankings. For quite a few of my own keywords I can see that without personalized search, I'd get a lot less traffic, while for others it's completely the other way around.

Always correlate SEO ranking data with Analytics!

Of course, no ranking is worth anything if you can't correlate it to a decent amount of incoming traffic. Luckily, both the Google Analytics and Clicky API allow you to easily correlate the two and see where you have a chance of gaining more traffic.

One of my favorite ways of looking at sites SEO ranking data is looking at where they rank inbetween #5 and #10 that's already sending traffic. If a keyword you rank #8 for consistently sends you traffic, that's a keyword with enough traffic to optimize for and see if you can get into the top 5 or even the top 3.

And now, it's your turn! How and where do you use SEO ranking data? Share it in the comments!

Image credit: financial chart from Shutterstock.

SEO Ranking Data: Tracking Passively and Actively is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

• If I have just made changes to my WP site, does it help to toggle the cache plugin?

  Absolutely. My SEO plugin force refreshes the cache because otherwise people start emailing me that stuff doesn't work when it works perfectly. On sites with more traffic though, you could also just leave the cache as is and wait it out a bit.

• How do I move a WordPress site, changing its permalinks but keeping the social numbers counts (post tweets/likes/shares)?

  The answer to this is an unfortunate but resounding: you don't. I've written a tutorial on moving WordPress to a new domain quite a while ago, but you simply can't keep those stats. All the more reason to think long and hard about moving domains...

• My buddy Richard thought he was funny, and asked: How much wood could a woodchuck chuck if a wouldchuck could chuck wood?

  The answer is simple, of course: a woodchuck would chuck as much wood as a woodchuck could chuck if a woodchuck could chuck wood.

• Will you be focusing on wordpress for the rest of your life? If not what do you think will be the reason why you change your focus?

  I might be smart, but I'm no sightseer. Also, while I work with WordPress a lot, a lot of my time is also spent on consulting and website reviews (which we do for all sorts of sites).

• What is the best practice to SEO a WordPress.com site? Is it even possible?

  Of course there are things you could do on a WordPress.com site, some themes there are better than others and you can do a whole lot content wise. The minute you start asking questions like that though, you should really consider getting a self-hosted WordPress.org install and taking control of your own destiny. You'll reach a point where you'll want to do more and WordPress.com doesn't allow you to do that and the longer you wait, the harder it is to move, so, move now.

• When do you think that WordPress is going to completely rewrite their code base so it's an actual CMS instead of a hacked together glorified blogging system?

  I find I get that question quite a lot and it annoys me. WordPress is being rewritten all the time. Check out the development that happens on Trac. Most of the people who ask questions like that haven't had a decent look at the codebase for ages. WordPress IS way more than a glorified blogging system already and if there are specific issues you have with the way it's coded, patches are welcome!

• If you start working on a WordPress blog for a client and there are no plugins installed, which ones do you always install?

  A couple: my own WordPress SEO & Google Analytics plugins, W3 Total Cache and usually Gravity Forms. Of course each site is different so there'll be more plugins after that depending on that site's and site owners needs and wants.

• What do you think has more value? A good domain name or good link building?

  Good link building, any day, every day. You see, domain names, especially so called "exact match domain names" are bound to be devalued at some point. Good link building will always create traffic to your site, just from those links alone, so that's always worth while. Those links also bring in rankings but in really good link building, that's often just a side effect. I recently talked about Eric Ward's mailing list, you should check out that post.

• Your SEO plugin places a canonical tag on every generated page and I have an ongoing argument with a co-worker about that. He says that this tag should only be on pages that contain duplicate content and that it is intended to 'tell' the searchbot where to look for the original content. Googling around doesn't clarify a lot. Could you please tell your reasons behind placing it on every page?

  I get this discussion a lot. The thing is, if I was 100% sure that a URL could only be accessed through that specific URL with no query parameters added, I might not add a canonical. There's nothing against it, but it'd just be a bit cleaner. However, these URLs:

  http://www.example.com/

  http://www.example.com/?campaign=email

  Are essentially the same for a WordPress install in 99% of the cases. However, they're not the same for Google and other search engines. So, if I wouldn't add canonical, the link value of the second URL would be waisted and, in fact, you'd have a competing duplicate content URL in the search results. That's why I add it to all pages.

• I noticed you don't use a comment system like disqus or intense debate. Would be nice to hear your thoughts about whether we should or should not use a comment system in WordPress.

  If I were to use a comment system, I'd use Facebook comments. The benefits of that and the fact that it gets way less spam are quite high. So far I've decided not to do that yet because a couple of my regular visitors and active commenters actually don't have Facebook accounts. Also, my comment redirect plugin doesn't work with those systems, which I think is a pity.

  I'll say one thing: the amount of work I have to do to keep this blog spam free is nothing short of ridiculous. Read this post to see what I mean.

That's it, what do you think, should I do this more often?

Questions and Answers is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Why I made this change? Simple: I wanted the site analysis, which is a report on how to improve your site in key areas such as SEO to Usability to Site Speed & more, to be more easily accessible to more people. I know that I'm a reasonably expensive consultant, yet I love helping people improve their websites. This service is aimed at exactly that and with this new price point I think it's something that most SMBs should be able to afford.

To be able to get to that lower price, I've been working on the system by which I create site analyses over the past weeks. I used to do them all by myself, but since I sell several of these a week now, sometimes even several a day, I can't keep doing that. I also noticed that I was telling the same things an awful lot, people tend to make the same "mistakes" all the time.

Getting some help

The conclusion was simple, I needed to outsource the "simpler bits" to someone who has a bit more time on his hands and only do the hard bits myself. Now, the simpler bits aren't all that simple, to be honest, which is why it took me a while to find someone who wanted to do those and had the capabilities needed to do it. I found that person in Michiel Heijmans, who, incidentally, lives in the same town as I do here in the Netherlands.

He was one of the very first bloggers in the Netherlands, starting his first blog in 2001, and has been in the online industry ever since. He co-founded about:blank, a famous Dutch e-zine about blogs & social media. Michiel knows an awful lot about all aspects of websites, from conversion to usability and more. On the last few reviews we've been working together and I noticed that because the two of us look at each site now, we actually give more feedback instead of less. And now we do that for a lower price.

Actionable Results

While the feedback I got for my site analyses has always been good, one of the remarks I got from people is "you've suggested so much, where do I start?". I fully understand that, the report needed to be more actionable, this is why all site analyses from now on will contain a to do list, ordered by a priority from 1 (high) to low (5).

At €495, which is currently less than $680, the cost of the analysis is far far lower than the value it represents in turn-over and profit. If you want my feedback on your site, because you're stuck in improving it, or haven't even started yet and want to know where to begin, you should order a website review right now!

Lowering the Price for Site Analyses is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Google & Privacy… is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!

Last friday I was telling my buddy Remkus de Vries, known as @DeFries on Twitter, that I was almost done with the internationalization support for my WordPress SEO plugin. He told me that I should be using GlotPress, which is a collaborative, web-based software translation tool maintained by some of the people behind WordPress. I concurred and asked him to set it up for me.

He did and he wrote a post about it: How to use GlotPress for your translations. After a bit of work on my side on getting a registration form going, I tweeted about my GlotPress being there on translate.yoast.com. Within 20 minutes about 10 people had registered and started translating. I was very excited about that and decided to look into it a bit better by the following morning, when 14 people in total had signed up already.

Because I could see this becoming unmanageable quite soon, I decided to create a mailing list for the contributors. As you'll see when you've read Remkus' post I listed above, we use a WordPress install on /register/ to manager the users. This allows me to use Gravity Forms, along with its User Registration add-on, to allow people to register. Because I built it that way, it was a piece of cake to add the Mailchimp add-on and make sure all new translator were connected to the mailing list as well.

Be sure that when you set it up like this too, you make the language people want to translate into a variable in MailChimp too, so later on you can easily segment on that, by emailing just the people who are translating your plugin(s) into Polish, for instance.

The amount of people working on this has surprised me. I've tweeted about it twice and so far, 31 people have signed up and are actively translating into 12 different languages, so far translating 2568 sentences!

Your turn?

Are you using WordPress in your native language? Would you want to contribute some of your time to help translate my plugins into your native language? Then register here, you'll receive instructions from there. If your language isn't listed yet, please allow me some time to enable that language and from then on your good to go, looking forward to seeing my plugins run in more languages then were speaking on the tower of Babel!

(Collaboratively) Translating Yoast Plugins is a post by Joost de Valk on Yoast - Tweaking Websites.A good WordPress blog needs good hosting, you don't want your blog to be slow, or, even worse, down, do you? Check out my thoughts on WordPress hosting!