An Indepth Look At How The Search Engines Crawl And May Choose To Ignore, Your Meta Keyword Tag In 2009

Meta Keyword Experiment

If you’ve taken a look at my meta keywords tag on the front page of my site lately, you’ve probably noticed an odd keyword phrase in there.

<meta name=”keywords” content=”spelt grain, canadian virtual, canadianvirtual”>

Spelt grain?

Why would I be using this keyword phrase for a site that’s geared toward small business owners and online search marketing?

Spamming? Some shade of grey seo tactic? You would be right to wonder, but some of you may have already had the realization that this is only a test. I chose spelt grain because it had nothing to do with my site at all. Any search I did on my site proved fruitless for this phrase, pre-this-post of course. It was also a great term because I could not only test indexing, but I could also test rank. I decided to do this test, not so much because of SEO Paranoia, but more because information on the meta keywords tag is sparse. To clarify, I should say that any real information on the usage of this tag is sparse. There’s lots of conjecture out there, and forums full of opinion.

Google Hates Keywords

Now how is that as a linkbait title?

Google doesn’t actually hate keywords, but as a search engine, it definitely ignores the keyword meta tag. Here is their Google Approved chart from their support docs, “Google understands the following meta tags (and related items)”:

What is most interesting is this snippet below the table:

“This is not an exclusive list of available meta tags, and you should feel free to use unlisted meta tags if they are important to your site. Just remember that Google will ignore meta tags it doesn’t know”.  It’s also interesting that this table renders with tbody tags, but I’ll save that for another post.

So it is quite clear that Google completely ignores the meta keyword tag. Trying to find any “official” information from MSN/Live/Kumo has been next to impossible. Ask has some meta tag information, but no specific offering, directly or indirectly on the meta keyword tag. Yahoo however offers this advice, here:

“Use a ‘keyword’ meta-tag to list key words for the document. Use a distinct list of keywords that relate to the specific page on your site instead of using one broad set of keywords for every page.”

Out of the major search engines, Yahoo is the only one that actually advocates its use, you rebel you, Yahoo!

The Meta Keyword Tag Real World Test

So why am I doing this test? Many sites don’t even bother to include the meta keyword tag anymore, and based on the above findings, I don’t blame them. There is a certain amount of freedom in not having to worry about this tag anymore. There are some sites that still use this tag however, so who is right? Let’s find out.

There is one search engine that can identify which Engines still use the meta keyword tag for indexing and discovery. Check out what MetaCrawler found out below.

Sure enough, Google and Live completely ignore the meta keyword tag, while Yahoo, Ask, and others still accept its use. I could include screencaps from all of these engines, to prove what MetaCrawler says, but for brevity of this post, MetaCrawler is all the evidence we need.

Keyword Ranking

I added “spelt grain” to my meta keywords tag on March 8th. On that day I found 363,000 results on Google, 306,000 results on Live, and 1,570,000 on Yahoo. We can safely remove Google and Live from this part of the discussion, as they clearly are not showing me any results related to spelt grain, using advanced search operators to filter searches, based on my site. However, do you think www.Canadianvirtual.ca appears anywhere in the now 1,600,000 results on Yahoo? 0 in 1000. This site doesn’t appear in the first 1000 positions on Yahoo, the maximum number of listings that the search engines allows you to see.

Meta Keyword Benefit

Having the meta keyword tag in your pages has benefit in terms of discovery for some search engines, but not all. However don’t be sold on this tag as a work horse in your SEO stable. Also, if you are a business owner looking to hire a search marketing consultant, run, don’t walk away if he or she tries to sell you on the virtues of using keywords in this meta tag to rank in the engines, because quite clearly, you won’t.

You could consider having some misspells in your meta keyword tag, if you feel so inclined to use it, based on Danny Sullivan’s post here.  The internet gods do have  sense of humour, I only found this post after I launched my experiment.

CMA

Post from: Search Engine Marketing & Optimization Blog

The Meta Keyword Tag

The Birth of a Top Level Link Farm

ICANN Tel You

To be honest, I really can’t tell you if I am completely on board with the new .tel registry, the latest sponsored Top Level Domain to go live on the lucrative domain market. One thing I do know is that this is the first TLD that has only one IP, 195.54.228.77. It is also the first TLD to have complete control over DNS, or Domain Name System, as well as layout and syntax of webpages and mobile content. Google has already indexed 643 .tel sites as of Feb 3rd. Live has 4.060 and Yahoo has not indexed any .tel domains at all. ICANN initially rejected the .tel sTLD RFP (there were in fact two proposals from different organizations), but were somehow convinced of .tel’s proof of concept in the end.

Tel Is About Communications

The idea behind .tel is to bring all of your contact information into one place: your address, email, phone, website, Facebook profile, the list really does go on. It’s one place to store, update, and provide your details to your colleagues, friends, and family, and it has that catchy three letter, one syllable suffix. Web 2.0 Badges and applications are in the works, and it is supposed to bring innovative technology to the fore with it’s all in one stored DNS, and content for web and mobile markets. The business seems sound, normal users will be charged about $20 a year for the rights to their own .tel domain and cookie cutter layout. With this subscription plan, Telnic’s single IP is probably paid for already, and so is their application fee to ICANN. This has the makings of a potential huge buyout, if the business model proves successful.

This is where my questions begin.

.Tel Is About Linkspam

It’s early in the landrush, the early registration phase of the new TLD, and there are already some “spammy” domains in the Google index. This is where Telnic, and ICANN lose me. The .tel pages are awash in links, links to websites, phones, social media. These links are all followed, I think you can see where I am going with this. I would be very shocked if this exact problem was not raised in the evaluation period. The problem is that thousands of pages are being created to potentially game the search engines through easily obtained, followed links. The easiest solution would be to nofollow the links. Telnic is going to have to act fast, or else the engines will act on their behalf. It would be a shame to see a whole registry being taken out of the search engines’ indexes, and it may not get that far, however I can see that a penalty will happen if nothing is done, sooner than later.

The Other .tel Question

I am also not convinced that a virtual address book alone can support an entire domain registry, that all the marketing plush will wear off and .tel is only a fad in the making. It’s a great idea, but not original. Routing information through DNS zones has been brought up before, in e164.arpa. This is the original idea of a web based ENUM lookup. In other words, this means being able to access telephone numbers over the internet. First out of the gate doesn’t always give way to exclusivity, and this is not necessarily a sticking point with me. My concerns are more on the marketing side, and my own personal bias toward the evolution of the TLD. Perhaps I am old school, but if I want to give someone my telephone number, website, or other details, I will tell them my telephone number, website, or email. The same is true the other way around. I don’t want to go to a page to get my contact’s information. This seems counter intuitive.

.Tel Must Be Thinking Mobile

What also seems counter intuitive is a small amount of content for a website page. Telnic must be thinking mobile, this is the only space where this formula makes any sense. Yet, how this business model made it all the way to its own TLD is a mystery. I really can’t tell.

CMA

Post from: Search Engine Marketing & Optimization Blog

Don’t Tell Dot Tel

A Look at JavaScript Code in Relation to Search

JavaScript and the Engines

It is now pretty much common knowledge amongst Search Marketers and Developers that JavaScript is ignored by Search Engine Bots, effectively keeping any code or content within JavaScript tags out of the Search Engine Indexes.

The reason behind this is that bots are, and have been, built to be simple creatures by nature, unable to crawl more advanced code, and other rich content such as images and flash. Perhaps this was partly done to save on processing and memory usage, though it can be imagined that indexing the entire Internet raises some tough logistical questions that need to be worked out, to ensure a stable environment. The “Big 3″ or Tier 1 Search Engine companies have Datacenters the size of football fields, many times over, but before we continue in such broad strokes, how about a small test.

The Document Title JavaScript Test

A few months ago I launched a page to test how JavaScript is handled by Bots. The working assumption was of course, that the JavaScript code would be ignored. I decided to test the JavaScript Document Title against the standard title tag that you would normally see in the head of any HTML formatted page. Sure enough, the result proved that the JavaScript command was completely ignored, and the text within the title tag was used.

Looking at this test meta page result in Google, the title tag is displayed.

However, the JavaScript command does overwrite the title tag in the browser title bar.

This doesn’t just prove that JavaScript is not crawled by bots, this also makes a case for content that is displayed to a user but hidden to Search Engines, in other words an Invisible Web.

Invisible Web

The term “Invisible Web” is used to describe all that is not found in the Search Engines, and sites that are coded heavily in JavaScript can be defined in this way. A post from Eric Ward’s URLWire from July 2002 gave this  announcement: A New Search Engine Tool Searches Javascript Files, a tool that still exists today. As Search Engine Algorithms become more advanced, the technology that supports these algos will become more advanced too. However almost seven years later, the major Search Companies have only begun to start digging into parts unknown, such as forms and flash. On the Internet map of 2009, there (still) be dragons.

JavaScript, the precursor to Nofollow

JavaScript has been used in the same way that the nofollow attribute is used today, to prevent sending link juice between pages. Between the two, JavaScript is still the proven way to do this, whereas nofollow may send some value between pages, but this is not yet proven. JavaScript has also been used for cloaking methods, and for more malicious purposes.  

Practical Uses for JS

The benefits of being able to display different content to a user versus the engines has merit, and there are benefits. However there are clear lines and limitations to how far this can be taken. The Search Engines prefer that you serve them content as you would a user. Do not shy away from coding in JavaScript for the sake of the Search Engines. If it makes sense, and there are benefits, and they fall within best practices, then code away!  On an application level, JavaScript is used by many companies, including the Engines for their recent software, such as MSN’s adCenter Analytics, Google Analytics, Google Chrome, and Google Optimizer to name but a few.

It is amazing to think that bots wouldn’t exist without JavaScript, and yet they are unable to see it, even though it’s all around them as they scour the web!

CMA

Post from: Search Engine Marketing & Optimization Blog

Crouching JavaScript Hidden Intelligence

Some Straight Up Search Talk for the End of 2008

Content Is King!(?) (- !@#&!)

If you’re new to Search Marketing, you are probably hearing this quite frequently. In fact, this is probably said to you over and over again, “Content is King!”. You may have heard this repeated so many times already that you might even be sick of hearing this, (and I apologize on behalf of all search marketers if that is true). If you’re not new to search marketing, I’m sure you are already sick of hearing this, even if you might still believe it.

Content Is Not King?

In regards to search optimization for the web, this is correct. Before I go any further I want to emphasize this. I’m not saying content is not important. Content adds a great (read significant) value to any website, and is consistently undervalued, (but that is fodder for another post). What I am saying is that when looking at ranking factors on web based search engines, like Google for example, content becomes secondary to one thing. What is that? That is links.

The Proof is in the Algorithm

There are millions upon millions of websites on the web, with untold numbers of spammy sites trying to rank well on the search engines. So how do you go about indexing the web with all of this spam and whatever else lies in the great unknown of cyberspace? Simple. You build an algorithm that takes into account many factors, and one good one is looking at how many links a website receives.

These days, an algo is built with other factors, and one in particular being how many links come from “trusted sources”, and how many come from “untrusted sources”. However, built right into the way search engines index the web is how many links does a site have, or in other words, what is its authority (or popularity) on the web.

More Pudding for the Proof

Links to a site being important in ranking factors is no secret. Search Engines like Google have to keep modifying their algorithm because some people have figured this out and use this information to spam the engines. You might see a common thread here, that a change to “clean up” the engines brought about more spam, which brought about yet another change to “clean up” the engines. However, removing links as a ranking factor isn’t going away anytime time soon. It’s importance may be decreased, or devalued in the future, but there are three things that prove linking is alive and well as a way to rank on the web.

Three Blind Sighted Mice

There are “three sighted mice” that point to the value of links on the web, in terms of the search engines and indexing sites. Google employs a whole web spam team to manually remove spam from the index. They realize that an algorithm alone cannot keep out the spam, and that humans are needed.

In the future, more trust will be placed on the Google user to decide the good content from the bad (see, content is important). Google has also triumphed the nofollow attribute, which even this site has adopted as fair practice. This is a special code that lets Googlebot (Google’s bot that crawls the web, indexing sites) know that you are not “voting for” a particular site, even though you are linking to it. What more proof do you need to believe the importance that Google places on links. Many sites, Wikipedia for one, was an early adopter of the nofollow policy. If links were not an important factor, why would Google be asking the community as a whole to put a value on their own links to other sites?

The last “mouse” deserves it’s own heading.

Bombs and Bowling

Google Bombs that is. Google Bombing is the act of building links to make something rank well in the search engines, unnaturally. This effort can also be referred to as Google Bowling if your competitor is the one doing the Google Bomb. Recent Google Bombs have targeted George W. Bush and The Church of Scientology. Google Bowling can be used for monetary gain, consider it the hubcap stealing and resell of the web. Google insists that Google Bombing doesn’t work anymore. On the surface this may be true, but if you read into the details, they must detect it and act on it (see web spam team, above). Technically, Google Bombing is something that still works, it’s gone stealth, or what some are calling “Google Bomb 2.0″.

Link Building Is King

Linkbuilding is an important part of optimizing your site. If done right (white hat), Google has even sanctioned this practice. When linkbuilding, it is very important to do some checks of the site that you want a link from. Is it a quality site? Would you link to it? Is it Spam free? Are the links contained on the site spam free? These are some of the questions that you should ask yourself when engaging in a link building campaign. What can you do when a shady site links to you? Stay tuned, the answer to that question is coming up in another post.

CMA

Update January 14, 2009 AKA Proof #4, Yahoo’s Point of View: Yahoo! recently made a Patent Application at the US Patent & Trademark Office entitled, “Identifying excessively reciprocal links among web entities“. This document doesn’t just touch on recips, but on web pages themselves, hosts (subdomains), domains, and clusters (read chain-gang link networks). This is further proof of the importance of links to the search engine algo. Just imagine if these proprietary engines protected their content like they do their machine learnings against link manipulation schemes.

Hey… Wanna trade a nofollow?

Post from: Search Engine Marketing & Optimization Blog

Content Is Not King

The State of the Social Media Empire in Canada

Facebook Searchers Lead the Way

According to Google Insights, Facebook is the number 1 most searched for term in Canada today. In fact, since September 2007, Facebook has been the most searched for term on Google, surpassing the previous leader by average, lyrics. Perhaps there was a day when searching for lyrics on Google may have passed the time during those long winter months, but now that day belongs to the social networking site, Facebook. 

2008 is the Year of Facebook

Facebook’s meteoric climb among Canadian searchers using Google started in September 2006, when Facebook opened up their service to everyone, and not just to students with school email addresses. It certainly looks like 2008 is the year of Facebook, and Canadian Search Marketers had better stand up and take notice, if they haven’t done so already.

Follow the Red line of Facebook, in 2008 it’s nowhere but up on Google.

New Brunswick Really Loves Facebook

Regionally Facebook is usually the second most popular term behind a local city or province. Whitehorse for the Yukon Territory, Edmonton for Alberta, Vancouver for British Columbia, PEI for Prince Edward Island, and interestingly Facebook is the most searched for term in New Brunswick, in the last 7, 30, and 90 days, as well as the last 12 months. New Brunswick, out of all the provinces and territories, really does love the social networking site, in a big way.

Facebook is big in the UK

A similar success path can be mapped out for Facebook as a search term in the UK, only it has taken until July 2008 to surpass the leading search term for all UK searches as of late, The BBC. Just so Canadians don’t feel so bad about searching for lyrics (instead of something more info-pomp like the British Broadcasting Corporation), searchers from the United Kingdom also favour searching for games, where this search term had been duking it out for top spot with The BBC.

Facebook and the US

What is extremely curious to note, and frankly a little surprising, is that Facebook hasn’t even cracked the top 5 most searched for terms in the US. In fact, it only cracked the top ten 90 days ago - and it sits at the number seven spot, over the last week of reporting. You may find this striking, with a company so rooted in the USA, as Facebook is. However, I believe this just indicates how massive the overall search volume is on Google for the United States, and points out a nation that stll relies heavily on MySpace and Yahoo for their everyday online needs. It may be too early to predict, but it is quite possible that Facebook will not dominate the American search sphere for another two, three or perhaps five years. In the meantime, Search Marketers can hone their Facebook optimization monetization on Canadians (and the Irish, Scottish, Welsh, and Brits).

Food for thought.

CMA

Post from: Search Engine Marketing & Optimization Blog

Canada Loves Facebook

The Why & the How

The Idea

With the recent launch of Google’s official SEO Guide for startups, a well thought out document detailing the basics of what (and what not) to do when optimizing your site for Google’s organic search, the idea came to me for this post. In the document, htaccess was referenced as something to use if you want to password protect your directories, ideally to prevent indexing and possible snooping of content, by visitors (or bots) to your site. For whatever reason, you may not want this content left out in the open for all to see.

Using htaccess to Secure Data, but is your htaccess Secure?

Securing your directories by using htaccess is one idea, but the issue is, your htaccess file may not be secured in of itself. This poses a problem with a potential security breach waiting to happen. If someone knows where to look to find the htaccess file on your site, your passwords could be stolen and used to obtain the data you thought was kept away from visitors to your site, in a safe place.

The Solution

Luckily there is a way to keep your htaccess file secure and safe from prying eyes, and it doesn’t involve modifying the httpd.conf or httpds.conf, which you may not have access to. First and foremost, make sure that your permissions, or CHMOD, for your htaccess file is set to 644. Absolutely do not set it to 777, which will not only make it publically viewable, but writable as well! This you want to avoid. The other step to securing htaccess, is to add some code to the top of the file, like so:

<Files .htaccess>
order allow,deny
deny from all
</Files>

Why not take the time to make sure your htaccess file is secure now. Once you’ve made these changes, your htaccess will be that much safer, and secure.

CMA

Post from: Search Engine Marketing & Optimization Blog

Securing Htaccess

version 72.14.205.104

Hello!

Greetings and welcome to my very first post! This is the “Canadianvirtual Search Marketing” Blog, and the search marketing is in quotes simply because I will be posting, and hopefully engaging discussion, on a much broader scale of topics than web search marketing alone. Although search will be the topical mainstay of this SEO blog, I will be covering SEM, social media marketing, or what is referred to as SMM, and delving into the offdeck with mobile. Perhaps this all falls under the “one web” philosophy, and I’m sure my technical curiosity regarding the semantic web will rear its ugly head from time to time, (I will try not to get too technical!), my hope is to get you, my reader, excited about every facet of search, be you novice or professional marketer.

SEO - Search Engine Optimization?

SEO, an accronym which stands for Search Engine Optimization, is a phrase that is somewhat perplexing if you really think about it. Why call it Search Engine Optimization when really there is no Optimization of any Search Engine involved, save if you are one of the employed engineers at Google, Yahoo, Live, and others. Indeed, SEO is Optimization for the Search Engines, or more specifically, Optimization of websites for Search Engines. I believe that any mystique that surrounds Search Engine Optimization begins here, and I hope that this blog begins to clarify what SEO is all about.

White Hat vs. Black Hat

There is a terminology that is unique to search marketing, and one of these is the grayscale of hats, from white to black. Essentially these differentiate between going from within the line (following rules and reguations), to being on the line, to going over the line. Being a white hat SEO means you follow the rules set out by the search engines and industry led best practices. Being black hat means SEO that strays from any known rules and regulations, and may also involve spamming, but this is not always the case.

SEO is dynamic

The knowledge base of search marketing is always changing; what may have been common practice before, is certainly not the best practices of today. SEO is a relatively new marketing discipline, and unlike other disciplines, it is a field that is constantly updating, changing, and morphing into something else. This is one of things that attracts me to this new “science”, that in a week, month, or year, new knowledge will come to the fore and yet again change what is known about SEO, and ultimately the Search Engines themselves.

Stay Tuned…

Stay tuned for more search marketing info, feel free to bookmark this blog, and link to me! <a href=”http://www.canadianvirtual.ca/blog/” title=”SEO Blog”>SEO Blog</a>

CMA

Post from: Search Engine Marketing & Optimization Blog

Search Marketing for Dummies