Search Engine Marketing Shakeup

The past few months have been volatile for many Web site owners who keep track of their rankings in the major search engines.

Yahoo!'s long-anticipated switch from licensing Google search engine results to providing their visitors with results from their own in-house search technology finally occurred on February 18, 2004. Overnight, Google's share of the search engine market dropped from 85 percent (when you count Google's own market share along with their current-AOL-and previous search licensees, Yahoo), to 55 percent.

The market share for the major search engines are now much more competitive: Google is still on top, according to Nielsen//NetRatings, with 55 percent (Google itself has 39.4% but also licenses results to AOL, which has 15.5%), followed by Yahoo! with 30 percent, and MSN with 27 percent.

As if this recent shakeup weren't confusing enough, there are even more wrinkles to this story. In order to fully understand it, we need to take a little recent-history refresher course.

Yahoo!'s Search Spending Spree

In December of 2002, Yahoo! bought search technology company Inktomi. Inktomi provides search results for the MSN and HotBot search engines.

In July of 2003, Yahoo! bought the pay-per-click search advertising pioneer Overture and in the process acquired the then-Overture-owned search engines AltaVista and AllTheWeb.

The common wisdom at the time among the search engine cognoscenti was that Yahoo! would merge all of their newly-acquired search technologies into one uber-search site. After it became clear that AltaVista and AllTheWeb would remain stand-alone search engines, the thinking within the search engine marketing industry was that Yahoo would refine the Inktomi search technology and use that to provide their own in-house results in order to challenge Google's search supremacy.

And that appears to be what happened when Yahoo finally pulled the switch and dropped Google as their search results provider.

Yahoo!'s Pay For Inclusion

Yahoo!'s search spending spree also landed them the Pay for Inclusion (PFI) services for Inktomi, AltaVista and AllTheWeb. PFI services charge a flat fee per Web page to guarantee inclusion in the search engine results, but do not guarantee rankings. Search engine marketers use the program to get into the search engines quickly and optimize the pages they submit for specific keywords in order to rank well for those keyword phrases.

This time common wisdom had it wrong, though. The assumption among search engine marketers was that with Yahoo's acquisition of Inktomi's Pay for Inclusion program, Web site marketers would get included in the new Yahoo! through Inktomi's PFI program, since it was believed that Yahoo! would use Inktomi for their search engine results. But on March 2, 2003, Yahoo! announced that they would merge all of their Pay for Inclusion programs, charging a flat-fee for inclusion in addition to implementing a pay-for-click model for the service.

Bottom line: For site owners, this new service will raise the cost of ensuring that crucial Web pages are quickly included in the new Yahoo! search. And as a result of the switch from Google to the Yahoo!-owned Inktomi, many Web sites that did not rank well in the Inktomi database have seen their traffic fall accordingly.

Google's Updates

Since November of last year, Google has been tweaking their algorithm (the methods by which the search engine decides where sites should rank in their search results) in order to improve the relevance of the links they provide for a given search. As a result, many top-ranking sites have dropped precipitously or have been dropped entirely from the search engine. Many other sites, however, remained unaffected by the changes. Other Web sites have seen their rankings drop during a Google tweak only to return to much the same position they held prior to Google's changes.

It appears that Google's changes are an attempt to weed out sites that try to spam the search engine by employing deceptive practices in order to achieve top results. There is some evidence that Google is attempting to boost sites that they deem “authority†pages to the top of the search engine results. If you pay attention to such things, you'll notice more directory-type sites at the top of your search results.

It also appears that Google has been implementing their new algorithm by industry or category. I've noticed dramatic changes in search results for search engine marketing-related searches and, more recently, law-related searches. Others have noticed changes in real estate type searches.

Bottom line: Don't panic. The changes in your rankings may be temporary. If your site employs techniques that Google doesn't like, then you may have to make changes to better conform to Google's guidelines. Ultimately, Google wants to give its users the best content for which they are searching.

Enter, Stage Right: Microsoft Search

Microsoft announced last year that they would begin developing their own in-house search technology. The obvious conclusion is that they will eventually drop Inktomi as their search results provider.

At the beginning of February of this year, Microsoft launched a beta version of their new MSN search technology. Microsoft hopes to launch their home-brewed search engine by the end of this year.

Bottom line: Since Microsoft plans a launch for late this year, it may be a bit early to make any judgments on their entry into the search engine market. As of now, their beta search results look a lot like Inktomi's search results.

Search Competition & Your Web Site Traffic

The downside to this newly competitive search engine landscape is that Web site owners will have to pay attention to all the major search engines. Prior to Yahoo! rolling out their new search, many site owners were content to focus their search engine marketing on Google alone, since it drove 85 percent of their traffic. That means you'll need to expend more effort and more money-especially if you're using Pay for Inclusion services-to maintain the traffic you've become accustomed to.

The upside is that Web site traffic will no longer depend on one source. If your site ranks well in the three major search engines, your traffic shouldn't suffer as much as it has in the past when Google decides to change their algorithm and your rankings drop as a result.