Excitement and a Little Glitch with the Wolfram|Alpha App

It’s now been a week since the first version of the Wolfram|Alpha App for the iPhone and iPod touch was released, and we’re excited to see how people are using it.

Our excitement was heightened by the opportunity to showcase the Wolfram|Alpha App at our Homework Day event and by the surprise of discovering—right in the middle of Homework Day—that there were reports of the app crashing unexpectedly.

Our company has been focused on software quality for more than 20 years, and extensive testing of the app had shown no such problems. But we immediately set about trying to find out what was going on.

Thanks in part to user reports and perhaps to the good fortune that many of our senior technical staff were actually in the same building at the Homework Day event, we were able to locate the issue early on Wednesday evening.

Upon getting everyone together, we created a server-side patch that remediated the problem—and that was deployed around 2am that night. This patch delivered an immediate fix for our Wolfram|Alpha App users.

We have now finished creating and testing a new version of the app—which will be supplied as an automatic update as soon as Apple completes its review.

We greatly appreciate the feedback on features you are enjoying. Just as much, we appreciate your feedback regarding any problems or quirky behavior that you’ve encountered. Of course, we can’t guarantee that we’ll be able to resolve all issues this quickly, but you can be sure that we’ll listen and do our best.

You can reach us directly at support@wolframalpha.com.

For those of you with an interest in knowing the details of the “crashing” last week, here is a description of the problem.

Within the Wolfram|Alpha App is a collection of example queries. Initially, this was planned to be a static collection, updated only when the app itself was updated. But since the main Wolfram|Alpha codebase is updated every week, we were keen to make sure the examples in the app continually reflected new capabilities.

So, a few days from the end of the development process, we added a feature whereby every week, if examples were requested, the app would check for a new XML file of examples on our servers.

Unfortunately, there was a bug in this—essentially a race condition that surfaced when a particular length of time was spent downloading the XML file.

The bug was never visible in Wolfram|Alpha queries that successfully generated results. It only revealed itself when Wolfram|Alpha couldn’t generate any direct results, and so tried to pull up “related examples” to show.

But in the end the result was that exactly seven days after the final build of the app, in rare cases that build started to show crashes.

Usually, this kind of bug can’t be fixed except by updating the software that has been distributed. But our system architecture was good enough that on Wednesday evening, we were able to figure out a way to avoid exciting the bug—just by changing the data that is sent back to the app from the Wolfram|Alpha servers when a query is processed.

It isn’t a perfect solution, because it makes Wolfram|Alpha give somewhat less information in cases where it doesn’t give a direct result. But it completely avoids the crash. And it’s a very satisfactory stop-gap measure as we wait for the fully updated version of the app to be propagated through the distribution system.

We’re embarrassed that our testing didn’t catch the bug. Of course, it’s always tricky to test things that happen after only a week has elapsed. We have a very extensive automated testing system for Wolfram|Alpha (needless to say, it’s a giant Mathematica program). But right now, we can’t apply the automated system directly to the iPhone. So we have to use a much lower-throughput manual method. And we missed this bug. It’s disappointing, but it’ll be a good future case study for our software testing group.

Meanwhile, even without explicitly updating the app, it’s getting more and more powerful—just as a result of the weekly updates to Wolfram|Alpha on the server side. Of course, we’re working on the app itself as well. Stay tuned for some exciting new capabilities….