Web Noir revisited. The main problem with search engines isn't with the engines themselves, or with any of their methodologies. It's with the haystack we call the Web. And the fact that it's a haystack instead of a directory.
Take every directory you can name. The yellow pages. The Sears catalog. The list of companies in the foyer of a high-rise. The library card catalog. The inventory at a book store. The student listings at a school.
The Web has nothing like any of them. Beyond DNS, it has no directory structure, and no directory. It's a haystack.
Of course, this is a virtue in many ways. The absence of a directory structure [~] a required way to organize everything to the right of the first single slash of every URL [~] is one of the graces that allows the Web to grow and persist as a wild and wooly place. Thanks to the absence of a directry, the Web has no hierarcies beyond the lengths of path names (/yada/yada/yada/etc.), which are made less hierarchical by the hyperlink. (Which subvert hierarchy, Dr. Weinberger famously says.)
[The Doc Searls Weblog]
Excellent writing by Doc as usual... I agree with what he says, but I would really prefer if people would stop considering the