An unofficial blog that watches Google's attempts to move your operating system online since 2005. Not affiliated with Google.

Send your tips to gostips@gmail.com.

December 23, 2006

Wikia, an Open Search Engine

Jimmy Wales, the founder of Wikipedia, plans to launch a search engine code-named Wikia in the first quarter of 2007. Following the model of Wikipedia, the new search engine will have user-editable search results.

"Google is very good at many types of search, but in many instances it produces nothing but spam and useless crap. Try searching for the term [Tampa hotels], for example, and you will not get any useful results," said Jimmy Wales. Well, maybe the example is not very good, because I see mostly useful results.

Mr. Wales did the impossible by creating an excellent resource of information with the support of a community. Now he wants to repeat the success, but this time the project will be supported by advertising.

"Essentially, if you consider one of the basic tasks of a search engine, it is to make a decision: 'this page is good, this page sucks'. Computers are notoriously bad at making such judgments, so algorithmic search has to go about it in a roundabout way. But we have a really great method for doing that ourselves. We just look at the page. It usually only takes a second to figure out if the page is good, so the key here is building a community of trust that can do that," added Wikipedia's founder.

I think the main job of a search engine is to understand how relevant a page is for a particular query. To scale, a search engine should that algorithmically. While people have a better ability to decide if a page is relevant, that doesn't mean spammers won't try to push their sites.

But the main reason for creating a search engine is that he thinks search is broken "for the same reason that proprietary software is always broken: lack of freedom, lack of community, lack accountability, lack of transparency." Google, for example, won't become open source because it uses proprietary algorithms, other search engines could copy its code and people could tweak their sites to abuse it.

It will be interesting to see if a search engine based only on human intelligence really works.

Update: Jimmy Wales explains in a Wired interview that "the core of the concept is the open-source nature of everything we're intending to do -- making all of the algorithms public, making all of the data public and trying to achieve the maximum possible transparency." Wales doesn't give a launch date: "The project to build the community to build the search engine is launching in the first quarter of 2007, not the search engine itself. We may have something up pretty quickly, maybe some sort of demo or test for people to start playing with."

10 comments:

  1. Want to hear on how Wikiasari can control or eliminate any bias/manipulation from its editors.

    ReplyDelete
  2. As you could've noticed, I didn't write the project is supported by Amazon.

    The article also contains a huge inexactitude: "The first result in any Google search is the website that has the most links pointing to it."

    ReplyDelete
  3. This is great news!

    I welcome our new overlords!

    ReplyDelete
  4. How about Wikiasari+Google? :D

    I guess at first there won't be a lot of results from Wikiasari, so how about they could take those from Google and modify them afterwards?

    ReplyDelete
  5. I don't think this type of system would do very well. As with any social site involving a ranking, a small group of users can find a way to "cheat" the system, in this case meaning inflate a links worth. I have seen it happen on social bookmarking sites to some extent. Even Wikipedia had it's problems with rogue users.

    ReplyDelete
  6. Should be an interesting experiment. The old DMOZ failed (IMHO) due to people issues. Too autocratic, or maybe just the wrong people/editing philosophies. I see this as something like applying the lessons learned managing Wikipedia (editor community) to a DMOZ-like endeavor, where the starting point is user driven search terms rather than categories.

    I would have thought human powered machines were dead for the Internet before Wikipedia, so, who knows? this might have a chance...

    ReplyDelete
  7. I think Wikiasari is going to fail in a BIG way.

    To start, the premise is absurd: (1) search isn't broken; (2) Google's algorithms already include human editorial judgments (that was incoming links are); (3) how is open source going to result in less spam?

    Also, developers aren't going to want to work on a open source search engine.

    More detail in these two blog posts:

    http://keysmun.blogspot.com/2006/12/why-wikiasari-will-fail.html
    http://keysmun.blogspot.com/2006/12/why-developers-wont-work-on-wikiasari.html

    ReplyDelete
  8. Up to 9 months ago we financially contributed funds to Wikipedia but no more, for we thought that it was a good idea and where its thinking was in unison with our own at that time - using knowledge for the good of humankind. When we as novices tried to place our Swiss charity within Wikipedia we were absolutely savaged by the editors. They in fact blocked our right of reply, which is documented by themselves.
    Thereafter we even sent our registration documents via email to the then executive director of Wikimedia, the holding organization, to prove that our international group was registered as a Swiss charity. He did nothing at all. A few months later he resigned with another top Wikimedia executive, 'Jimbo's second in command. The greatest problem with Wikipedia that we now find is that they are highly selective in who should place information and where therefore they will never really have a web-based encyclopaedia that is unbiased and totally factual. It is ultimately at the whims of the few enlightened ones who control what should be a great reference. Unfortunately we now see that it is not.

    For anyone interested further on how Wikipedia editors work, the full account including all emails will be part of our next web newsletter 'Scientific Discovery'. It will be on-line by the end of July 2007. Overall, it is time we feel that Wikipedia looked internally at itself and that they concluded that they have major problems with the way they treat new entrants. This analysis should especially be directed towards the attitude of their editors, who remove the right of reply and delete super-quick for reasons not based on evidence but only hearsay. By the way also, the Wikipedian Editor Zoe who first blocked us and the initial instigator of all the basic trouble, fell out with 'Jimbo' and where she as well left a few months later. Apparently she had made a vendetta against a certain professor according to 'Jimbo's' opinion. Thereafter she took her bat and ball home and has never been seen since. I believe she also threatened the embattled professor at the time - the web link is http://72.14.209.104/search?q=cache:dUfUXyA24wwJ:www.encyclopediadramatica.com/Zoe+zoe+wikipedia+professor+change+wikipedia&hl=en&ct=clnk&cd=3&gl=uk.


    Dr. David Hill
    Chief Executive
    World Innovation Foundation Charity (reg. no. CH-035.7.035.277-9 - 11th July 2005)
    Bern, Switzerland

    ReplyDelete
  9. I notice that Dr. Hill has spammed over 20 blogs and news sites with his comment, but I figured that I should respond anyway. :)

    For some reason, it appears that Dr. Hill believes that if he provides the Wikimedia Foundation with money, then his charity will be automatically guaranteed a spot on Wikipedia. I am happy to say that this is not the case. While I have nothing personally against World Innovation Foundation Charity, Wikipedia's neutrality and impartiality is very important.

    I have reviewed the original article. There are absolutely no references in the final revision before it was deleted. The main reason that it was deleted was because we didn't believe that the organisation was notable enough to be listed on Wikipedia. This was done through articles for deletion - I will let the reader judge whether this was a fair process. I should also note that if someone believes and can demonstrate that they are notable, then there is a deletion review process.

    Dr. Hill should also be aware of our suggested guidelines that deal with potential conlicts of interest. Though it isn't prohibited on Wikipedia, it is clearly a conflict of interest to write about your own organisation. It is thus frowned upon. I think that given the goals of Wikipedia, this is pretty reasonable.

    Personally, I don't believe that Wikipedia editors or the Wikimedia Foundation has anything against this charity. In my dealings with Wikipedia and the WMF, I have always found that they welcome contructive dialogue with organisations and individuals.

    Chris Sherlock
    User:Ta bu shi da yu
    English Wikipedia Administrator (writing in personal capacity)

    ReplyDelete
  10. I dont think so that Wikia can provide a perfect search result as expected. I do like to hear more on this.

    ReplyDelete

Note: Only a member of this blog may post a comment.