1
This is an edited version of a short talk I gave last weekend at The Nantucket Project—a fascinatingly eclectic event held on an island that I happen to have been visiting every summer for the past dozen years.
Lots of things have happened in the world in the past 100 years. But I think in the long view of history one thing will end up standing out among all others: this has been the century when the idea of computation emerged. More »
49
Note added: Since this blog was written, Facebook has modified their API to make much less information available about Facebook friends. While I think adding privacy controls is a good idea, what Facebook has done reduces the richness of the results that Wolfram|Alpha Personal Analytics can give for Facebook users.
After I wrote about doing personal analytics with data I’ve collected about myself, many people asked how they could do similar things themselves.
Now of course most people haven’t been doing the kind of data collecting that I’ve been doing for the past couple of decades. But these days a lot of people do have a rich source of data about themselves: their Facebook histories. More »
11
Today marks an important milestone for Wolfram|Alpha, and for computational knowledge in general: for the first time, Wolfram|Alpha is now on average giving complete, successful responses to more than 90% of the queries entered on its website (and with “nearby” interpretations included, the fraction is closer to 95%).
I consider this an impressive achievement—the hard-won result of many years of progressively filling out the knowledge and linguistic capabilities of the system.
The picture below shows how the fraction of successful queries (in green) has increased relative to unsuccessful ones (red) since Wolfram|Alpha was launched in 2009. And from the log scale in the right-hand panel, we can see that there’s been a roughly exponential decrease in the failure rate, with a half-life of around 18 months. It seems to be a kind of Moore’s law for computational knowledge: the net effect of innumerable individual engineering achievements and new ideas is to give exponential improvement.
16
It’s a sad but true fact that most data that’s generated or collected—even with considerable effort—never gets any kind of serious analysis. But in a sense that’s not surprising. Because doing data science has always been hard. And even expert data scientists usually have to spend lots of time wrangling code and data to do any particular analysis.
I myself have been using computers to work with data for more than a third of a century. And over that time my tools and methods have gradually evolved. But this week—with the release of Wolfram|Alpha Pro—something dramatic has happened, that will forever change the way I approach data.
The key idea is automation. The concept in Wolfram|Alpha Pro is that I should just be able to take my data in whatever raw form it arrives, and throw it into Wolfram|Alpha Pro. And then Wolfram|Alpha Pro should automatically do a whole bunch of analysis, and then give me a well-organized report about my data. And if my data isn’t too large, this should all happen in a few seconds.
And what’s amazing to me is that it actually works. I’ve got all kinds of data lying around: measurements, business reports, personal analytics, whatever. And I’ve been feeding it into Wolfram|Alpha Pro. And Wolfram|Alpha Pro has been showing me visualizations and coming up with analyses that tell me all kinds of useful things about the data.
71
Today I’m excited to be able to announce the launch of Wolfram|Alpha Pro—the biggest single step in the development of Wolfram|Alpha since its original introduction.
Over the two and a half years since we first launched, Wolfram|Alpha has been growing rapidly in content and capabilities. But today’s introduction of Wolfram|Alpha Pro in effect adds a whole new model for interacting with Wolfram|Alpha—and brings all sorts of fundamentally new and remarkable capabilities.
Starting today, everyone has access to Wolfram|Alpha Pro at wolframalpha.com. Unlike the “tourist” version of Wolfram|Alpha, though, you have to log in, and, yes, to get full capabilities there’s a subscription ($4.99/month, or $2.99/month for students). (Right now, you can try it for free with a trial subscription.)
So, what does Wolfram|Alpha Pro do?
7
There’s been very little change in top-level internet domains (like .com, .org, .us, etc.) for a long time. But a number of years ago I started thinking about the possibility of having a new .data top-level domain (TLD). And starting this week, there’ll finally be a period when it’s possible to apply to create such a thing.
It’s not at all clear what’s going to happen with new TLDs—or how people will end up feeling about them. Presumably there’ll be TLDs for places and communities and professions and categories of goods and events. A .data TLD would be a slightly different kind of thing. But along with some other interested parties, I’ve been exploring the possibility of creating such a thing.
With Wolfram|Alpha and Mathematica—as well as our annual Data Summit—we’ve been deeply involved with the worldwide data community, and coordinating the creation of a .data TLD would be an extension of that activity. More »
23
I’m so sad this evening—as millions are—to hear of Steve Jobs’s death. Scattered over the last quarter century, I learned much from Steve Jobs, and was proud to consider him a friend. And indeed, he contributed in various ways to all three of my major life projects so far: Mathematica, A New Kind of Science and Wolfram|Alpha.
More »
12
Things with Wolfram|Alpha are going well. Really well. So well that I’m now incredibly keen to scale them up dramatically.
When I started the Wolfram|Alpha project, I was not even sure anything like it would be possible. But over the last two years we’ve proved that, yes, with the tower of technology we’ve created, one can in fact take large swaths of knowledge, make them computable, and deliver them for everyone to use.
From the outside, it’s easy to see that there’s been steady growth in the domains of knowledge that Wolfram|Alpha covers. And over the next few months there’ll be some big additions, notably in everyday and consumer areas. But to me what’s most dramatic is what’s happened on the inside. Because what we’ve done is to build a giant system of technology and management processes that allows us systematically to make any area of knowledge computable.
The catch is that it always takes effort. We rely on a huge tower of automation. But in every new area we tackle there are new issues, new opportunities—and new ways that resources and human effort have to be used.
I’m very pleased with how broad and deep the coverage we have already achieved is. But we have an immense to-do list, assembled not least from all the feedback we’ve received from users of Wolfram|Alpha. And the good news is that at this point it’s a straight shot: given enough effort, we can complete the to-do list. We have all the systems we need to scale the knowledge in Wolfram|Alpha up all the way. More »
4
The precursors of what we’re trying to do with computable data in Wolfram|Alpha in many ways stretch back to the very dawn of human history—and in fact their development has been fascinatingly tied to the whole progress of civilization.
Last year we invited the leaders of today’s great data repositories to our Wolfram Data Summit—and as a conversation piece we assembled a timeline of the historical development of systematic data and computable knowledge.
This year, as we approach the Wolfram Data Summit 2011, we’ve taken the comments and suggestions we got, and we’re making available a five-feet-long (1.5 meters) printed poster of the timeline—as well as having the basic content on the web.
The story the timeline tells is a fascinating one: of how, in a multitude of steps, our civilization has systematized more and more areas of knowledge—collected the data associated with them, and gradually made them amenable to automation. More »
10
Two weeks ago we made a major announcement: building on technology that we’ve been developing for more than 20 years, we released Computable Document Format (CDF). I think CDF is going to have a big effect on the way all sorts of things can be communicated. Because for the first time it makes it practical to include live computation as a routine part of a document.
There are many important applications of CDF that we’ll no doubt be seeing over the months and years to come. But today I’m pleased to announce an experimental one from us: Wolfram|Alpha with CDF.
Starting today, as soon as you have the free CDF plugin installed (or if you have Mathematica 8 on your system) you can go to the top right-hand corner of the Wolfram|Alpha website, and set CDF on, with the result that Wolfram|Alpha will generate not just a static web page, but instead full CDF output—that you can directly interact and compute with. More »
