Building Wolfram alpha as a giant number cruncher seems like a good idea, but I am disappointed at its knowledge. It can’t even tell me how many people has died in the Tiananmen Square massacre, a.k.a. 64.

The search is on

May 14th 2009

From The Economist print edition

A search engine that computes answers instead of looking them up

IT IS the curse of every internet search engine to be compared to Google, master of the universe and supreme ruler over two-thirds of such searches. Since newcomers never measure up to the breadth and depth of the billions of pages that Google has indexed over the past decade, most of these comparisons end with an easy win for the incumbent. Pretenders to the throne, nevertheless, keep appearing.

The latest, to be launched on May 18th, is Wolfram Alpha. It is named after its inventor, Stephen Wolfram, a British prodigy who earned his PhD in physics at the tender age of 20 and made a fortune from a calculation and graphing software package called Mathematica—and who raised eyebrows when he proposed, in a self-published tome in 2002, that the entire universe is but a giant calculator that has been running for billions of years.

To be fair, many of the overblown expectations surrounding Alpha do not stem from Dr Wolfram himself. Indeed, he describes his invention not as a search engine but as a “computational knowledge engine”. The quirky label is not only an attempt to sidestep a confrontation with Google, but also hints at Alpha’s different approach to answering questions.

Instead of serving up a list of popular links to other sites that contain the search term picked by a user, Alpha is a more-or-less closed system. It tries to dissect a question into its components and then performs calculations, using its own source materials, to compute an answer. The results are presented as a sleek collage of tables, charts and graphics. Alpha, in short, acts like a nerdy reference librarian who is equipped with the latest calculators and endless stacks of books and encyclopedias. Hit upon the right subject and it will excel at calculating airflow around a surface, showing the distribution patterns for DNA base pairs in human genes, spitting out prime numbers or computing and comparing crucial statistics for various national economies. Yet simple queries such as “climate change” or “Gordon Brown” will yield the equivalent of an empty stare.

Omniscient, omnipotent, omnipresent?

The goal Dr Wolfram has set himself is to take all the knowledge civilisation has amassed and make it computable—in other words, render it as potential inputs to logical queries that can be crunched to produce answers. A modest task, then, made no easier by the fact that knowledge is an ever-expanding realm, with plenty of soft spots that do not lend themselves to any sensible computation. In areas such as literature and popular culture, which are not based purely on formulas and numbers, Alpha is either still “under development” or entirely out of its depth. Even when there are facts, such as data about the expected range of global warming and its abatement, Alpha falters as soon as those facts become mired in debate.

The engine actually works by applying code used in Mathematica (a set of mathematical tools used to perform elaborate calculations, solve equations and build models) to a vast store of curated data that a team of around 200 experts working under Dr Wolfram’s direction has amassed, scanned, processed and correlated over the past couple of years. So far, he says, his repository contains between 10 billion and 20 billion elements of data.

This use of curated data means that Alpha takes a different approach to the automation or “crowdsourcing” normally used to generate knowledge from the web. In that sense it is more like (and may be more threatening to) Wikipedia than Google.

Dr Wolfram reckons his blend of machines and human minds can be scaled up to handle both the millions of users he hopes for and the thousands of employees who will be needed to add new sources of data by hand, so that queries about the record sales of a rapper or the biography of a political figure do not go unanswered.

The result, according to some critics, is little more than a giant almanac (all those tables and charts). That is a bit harsh. A free service that almost instantaneously computes the answers to complex questions—such as the ratio between exports and imports in Ghana (or any other country) over the past ten years—is indeed novel, and may serve to help democratise expert knowledge. Alpha’s true value, though, may become apparent only when outside organisations such as companies and universities can apply it to their own internal data as well as to Dr Wolfram’s hoard. At that point it really will be able to get to work on all human knowledge.