What would a future search engine be like?
The age of brute force search engines may be about to be over. Or atleast the nature of search engines, are due for a drastic change. By brute force I mean, an approach to crank out meaning, information, and relevance, to the search query, by just looking in tera bytes and peta bytes of data on the Internet. And of course, things like number of back links to that page.
To those, following the areas of Search Engines, its pretty common knowledge, that the conventional Google-like Search Engines are no good for “high quality relevant discovery”. In fact they have been no good, for such information for a good 4 years or so, in my opinion. What we have now, is a situation of “relevant but random discovery”.
Let me explain, it with an example.
Try searching for “video cameras in India” and you get some results. And they all are relevant. But there’s no guarantee, that they are sorted in terms of a criteria, which you may want, e.g. sites having quality information about a camera. So I call this as a “relevant but random discovery” rather than a “high quality relevant discovery”.
As the information they get is still relevant, in some way, is the reason, why most folks are still happy to live with it. And people are used to just getting by, by tweeking their queries, and getting some satisfactory enough result. The problem is that people don’t even know what they may be missing. Remember no body actually wanted a real nice search engine, when Google silently arrived.
Another thing, why no body sees a problem is that, when using the search engine just for navigation. For e.g. finding out “Java 5 TimerTask”. Its brilliant. You get precisely what you want in the first two results (Aside: they still have 32,100 results for that. That’s why I perhaps used the phrase ‘brute force’)
Was the situation, always like this and people didn’t know?
Certainly not. After the Google arrived in 1998, for some good 5-7 years or so, its PageRank algorithm worked very well. The algorithm works very well, if people don’t game the system. But, we have full blown SEO industry around the Search Engine. And despite Google’s efforts to keep the SEO clean. It’s simply not possible to do it. In today’s Internet world, it will be increasingly difficult to identify for a clean link to a site, to a paid (SEM’d) link. The links from social media, only further add to this noise.
So what’s the solution?
On your machine, how do you find a file. You often search for it, using the file system search. But you know, what file you are looking for. Think, the conventional brute-force search engine, will move more towards navigation, in which a person knows what he wants, and tries to find it.
So something similar will happen on the Internet. Ofcourse the person, doesn’t know which website to look for often, so he will ask his friends for advise. Wait a minute, did you say, ask friends, come on who has the time to ask, and then wait for a reply?
Well the answer is that the asking and replying will happen asynchronously. On all the social media, people are continously telling what they like and what they don’t and sharing various experiences. “This rose smells nice”, “This coffee is not so great” and “this travel site is so great” Or “this site doesn’t rock”. So some body has to just assimilate all these statements of expressions and churn them as replies when a question is being asked.
Several folks are already at play in this area e.g. quora.com – the arguments against it are that its a closed service, but that’s a different story. But there is far-far from a clear winner. Its possible, but unlikely, that the likes of Google itself, will totally transform themselves into a totally different kind of a search engine. Unlikely I say, is because, its not always possible to kill a cash cow. The conventional brute-force search engine is perhaps at the peak of money making curve, by the various Ad programs – Ad words and Ad Sense. So in such a scenario, will somebody be able to shake of the inertia and start from a clean slate. Possible but unlikely.
There were some efforts, which I was very excited about like blekko.com. But they appeared to be working on a conventional ‘brute-force’ search engine (of course on a better version of it), atleast until about a couple of years back. This was around the time when cuil (remember?) failed spectacularly. It will be interesting to see how they launch, in a post twitter/facebook world.
It will be interesting to see, who get’s there first, and importantly who gets it right.
NOTE: This blog has opinions and thoughts, which are personal of Khushnood Naqvi, and do not necessarily reflect the opinion of 90di.
PS: Had written this piece a few months back, when blekko was yet to be launched. But shied away from blogging it here. Thanks to Vijay, Patrick, Shaurabh, Kiran and Abhinit for reading it. Since then the noise around search quality has only increased as seen in these blogs: Codding Horror; Blekko founder; Google.