Google Revisited: Comparing Search Engine Results

With the recent change in Google's ranking (and, in apologies to Mark Pilgrim, I now think Google has some real errors in the new ranking when I didn't think so before, but it's still not as bad as some are making it out to be), I thought it might be interesting to look at how Google compares for a specitic search with other engines.  And I picked a query that has relevance to me -- Scott Johnson.  No quotes, no phrases, just those two words (except for AllTheWeb which gets a special mention for automatically adding quotes).  All I'm measuring is not which page comes up first but where a page that is related to me comes up.  Sometimes it's a page from my website, sometimes it's a blog page and sometimes it's my O'Reilly book catalog entry. 

• Google -- # 1
• All the Web -- No Quotes -- #9
• All the Web -- With Auto Inserted Quotes -- #3
• HotBot -- #15
• Lycos -- #6
• Teoma -- #1
• LookSmart (inktomi powered results) -- #31
• Wisenut -- #1
• Alta Vista -- #1

What's really interesting here is that almost all of this is almost certainly related to my blog.  I didn't have anywhere near these kind of results before I was a blogger.  It's also extremely interesting to me the similarities between Google, Teoma, Wisenut and Alta Vista.  That's just plain shocking to me.  True the comparison isn't entirely valid since they result in different pages at times but these searches all give results related to me.

Lessons From All This

There are two easy lessons from all this:

1. Right now the single best search engine optimization technique?  A simple weblog.  And I know that Google seems to treat radio.weblogs.com as a highly valid source of input so I recommend Radio.  But I think it really matters that you blog regularly and somewhat consistently. 
2. Don't spend exorbitant fees on search engine optimization.  As ranking algorithms have gotten much more complex without explicit, inside knowledge of how the engines work it is very unclear to me that it works at all anymore.  I suspect that you'll get dramatically better results by becoming a blogger.

9:00:03 PM      Google It!   comment []    IM Me About This   

To Amazon, Do Not Pass Go, Do Not Collect $200, Go Stand in the Corner

It's been a few days without a real rant (flexes his fingers) so here goes.  Amazon:

WHAT ARE YOU THINKING ????

I was over at GeoCaching and noticed that there is a cache in Nahant.  Go figure.  It's like a mile from me.  This might be enough to finally get me to buy a GPS.  As always, with consumer electronics, I'm the last around to get one (I only got a DVD player a few months ago) since I can't write it off, I know it won't work right and will just be difficult.  So anyway I surfed over to Amazon and searched for GPS and then selected the 278 results in Electronics.  And got a list of results.  Hmmm.... $250 odd is more than I want to spend for a GPS.  So I'll just use "Sort by" and sort by price.  And then I saw this fine bit of blithering, pathetic idiocy that just boggles my (admittedly little) mind:

[ Click for Picture ]

You can't sort by price.  You can't sort by price!  Go figure.  One of the very real advantages of shopping on a computer is something like sorting.  Yes I know that I can look by "Best Selling" and then infer a price relationship since cheaper tends to sell more but that's just not the same.  And then I said to myself "Wait I know that Amazon's crew of largely Perl monkeys can sort a database by price" so let's see if they do it elsewhere like in books.  So I did and got this: 

[ Click for Picture ]

That's right -- search for a book and you can sort by price but search for electronics where sorting by price really matters and you can't.  Go figure.  That's a stupid revenue optimization strategy or it's bad product management.

New slogan for Amazon's programmers: "Amazon.  First we'll take away your ability to sort by price and then tomorrow we'll remove the search engine."

Disclaimer: I have no bias against Amazon.  I use them frequently and I think they absolutely rock.  Things like this frustrate the ever loving crap out of me since they can do better.  I'm a fan and I want Amazon to do well.  For all their flaws they have done a good job and I hope they succeed.  They even released an XML based webservice API.  And now this.  Sigh.

3:43:27 PM      Google It!   comment []    IM Me About This   

Thank You Dawn ! Ten Best Things to Say If You Are Caught Sleeping At Your Desk

10. They told me at the bloodbank this might happen.
9. This is just a 15 minute power nap like they raved about in that time management course you sent me to.
8. Whew! Guess I left the top off the white out. You probably got here just in time.
7. I wasn't sleeping. I was meditating on the mission statement and envisioning a new paradigm.
... [ More ]

11:06:55 AM      Google It!   comment []    IM Me About This   

Any Elance Users Out There?

At the FuzzyGroup we do web design and are always looking for new projects.  I was wondering if anyone out there has had luck getting work through Elance.  Elance is a site where you can post work to be bid on by service companies.  It now requires you to pay between $150-$500 per quarter [ GO ].  Anyone have any input on this one?

11:01:49 AM      Google It!   comment []    IM Me About This   

Ah, Hollywood...  Could You Buy a Clue ?

I see on Doc Searls article on Digital Hollywood this quote:

Tom McGrath, President of Paramount Enterprises and EVP of Viacom Entertainment Group, said

"We...are heavily dependent on first-run licensing from major TV networks to survive. If you follow through the train of logic that John Taplin started with, it's not a pretty picture.... The evidence from TiVo is basically people just don't watch commercials at all. It's not a question of watching the ones they are interested in or not interested in.... It is a transformation of the industry. As producers we rely on the fact that there is a market for good programming. Right now the penetration of these devices, of VoD (Video On Demand), of this disintermediation, is not so great that we face collapse in the near or intermediate future. But it's something that we think about all the time.

Disclaimer: I added the bold.  So the EVP of Viacom's Entertainment Group is just now realizing that people don't watch commercials?  Hello !  It's called the "clicker" or "remote control" and we've all been using it to flip past commercials for at least the past 15 years now.  And you are just figuring it out?  Even though some bloggers disagree with the convention of using IM speak, all I can say to this is ROFLOL (picture me rolling on the floor laughing out loud). 

And we wonder why these folks have confusion in the new digital age?  They apparently haven't figured out the old remote control age!  A Tivo doesn't really do a damn thing to affect commercial watching -- it's been dead for years only no one bothered to wake up and tell the entertainment industry.  Here's an exercise I'd like to see some consumer marketer try:

1. Stop all broadcast television advertising
2. See if it affects your sales at all.
3. Record much larger profits.

The bottom line here is that convention marketing is dead and has been for a long time.  It feels to me like a con game where advertising agencies tell businesses "If you don't advertise you'll die" but no one has tested the alternative.

10:39:27 AM      Google It!   comment []    IM Me About This   

Read This !  Generation Wrecked

A non blogger friend just sent me this, a great Fortune magazine article about the "Generation Wrecked":

Ten years ago grunge musicians and college-age Cassandras who had never held a day job preached that corporate America would crush their generation's soul and leave them without a pension plan. Films like Singles and Reality Bites chronicled their transition from college graduate to Gap salesclerk.

A few years later the core of Generation X--the 40 million Americans born between 1966 and 1975--found themselves riding the wildest economic bull ever. Salesclerks became programmers; coffee slingers morphed into experts in Java (computerese, that is)--all flush with stock options and eye-popping salaries. Now that the thrill ride is over, Gen X's plight seems particularly bruising. No generation since the Depression has been set up for failure like this. Everything the dot-com boom delivered has been taken away--and then some. Real wages are falling, wealth continues to shift from younger to older, and education costs are surging. Worse yet, for some Gen Xers, their peak earning years are behind them. Buried in college and credit card debt, a lot of them won't be able to catch up as they approach their prime spending years.

(I added the bold).

This is a really, really scary article.  If you have a job now and you're not happy?  My advice: STAY PUT.  This recession has been what I've called to friends for some time "The Quiet Recession".  If you weren't directly affected then you didn't realize how bad it was.  This article spells it out clearly and succinctly.

Now, if I was really, really cynical, I'd comment that America's current Iraq fervor is because a) Bush knows exactly how bad it is and b) he knows that he can't fix it and c) war is a big distraction.

But I'm not cynical.  No I'm not.  Really I'm not.

8:34:06 AM      Google It!   comment []    IM Me About This   

A Follow Up to the Google Post

Oh and just to clarify even my Google rankings changed.  Searching for Scott Johnson now gives me only the 3 top pages on the Internet.  So while I still have the top place I now have fewer of the top places and I don't think that's bad.

7:13:37 AM      Google It!   comment []    IM Me About This   

Hmmm.... Google's Different

Kasia is wondering why she's no longer the top result for some google queries.  Thanks to Scripting pointing me to Mark Pilgrim there is an answer:

Specifically, Google is now apparently cross-checking link text with the linked site, and discounting or ignoring links whose text does not appear in the linked site. This all but kills off Google bombing. Searching for "go to hell" no longer takes you to microsoft.com; searching for "talentless hack" no longer finds ohmessylife.com, although it finds a lot of people who were previously participating in the Google bombing. No definitive word yet on whether Google is actively penalizing such sites.

Unfortunately, the algorithm tweaks necessary to stop these two techniques have caused a wide range of collateral damage, apparently coming down hardest on medium-to-large sites that had previously been doing everything right (as far as page structure, link structure, accessibility, and general honest hard work putting together a usable and useful site). The Webmasterworld forums are alive with complaints and speculation: (see Mark's site for the links)

...

Regardless, Google’s search results in general appear to be significantly degraded in many key areas. The forums are full of people complaining that spam sites, doorway pages, and obvious cloaking attempts, which Google used to be so good at filtering out, are now popping up in top spots with disturbing frequency. Nobody in the forums wants to talk about which keywords they’re tracking, so I tried to find my own concrete example of crap search results. It didn’t take long.

Just as a general comment it doesn't surprise me that Google's results have been degraded with this change -- but I doubt that this is permanent.  Google has actually changed their algorithms a bunch of times and there is always a "issue".  This type of stuff is hard and it's even harder when people are actively trying to subvert you (i.e. Google bombing).  Google will (almost certainly) fix this.

I've told countless clients over the years that the best search engine optimization is this:

1. Build a good site. 
2. Create relevant content.
3. Extensively cross link it.
4. Regularly update it.
5. Worry about meta tags and optimization a little but not a lot.

You can definitely spend $$$ on search engine work and sometimes it will pay off but it's a constant battle and you may just want to focus your efforts on building a better site.  That pays off in the long run a lot more than countless tweaking.  It is, however, substantially worse for the web consultant.

Note: Google's one of the few companies that I really "trust" and it seems like we're all jumping to conclusions way too fast (some people are actually saying "Google's reign is over" with these changes).  Come on people !  Give them a chance to fix their bugs.

7:11:27 AM      Google It!   comment []    IM Me About This