Launching the Wolfram Connected Devices Project

Connected devices are central to our long-term strategy of injecting sophisticated computation and knowledge into everything. With the Wolfram Language we now have a way to describe and compute about things in the world. Connected devices are what we need to measure and interface with those things.

In the end, we want every type of connected device to be seamlessly integrated with the Wolfram Language. And this will have all sorts of important consequences. But as we work toward this, there’s an obvious first step: we have to know what types of connected devices there actually are.

So to have a way to answer that question, today we’re launching the Wolfram Connected Devices Project—whose goal is to work with device manufacturers and the technical community to provide a definitive, curated, source of systematic knowledge about connected devices.

We have a couple of thousand devices (from about 300 companies) included as of today—and we expect this number to grow quite rapidly in the months ahead. For each device, there is a certain amount of structured information:

Whenever possible, this information is set up to be computable, so that it can for example be used in Wolfram|Alpha:

Soon you’ll be able to make all sorts of complex queries about devices, very much like the queries you can make now about consumer products:

We’re working hard to make the Wolfram Connected Devices Project an important and useful resource in its own right. But in the end our goal is not just to deal with information about devices, but actually be able to connect to the devices, and get data from them—and then do all sorts of things with that data.

But first—at least if we expect to do a good job—we must have a good way to represent all the kinds of data that can come out of a device. And, as it turns out, we have a great solution for this coming: WDF, the Wolfram Data Framework. In a sense, what WDF does is to take everything we’ve learned about representing data and the world from Wolfram|Alpha, and make it available to use on data from anywhere.

There’s a lot to say about WDF. But in terms of devices, it provides an immediate way to represent not just raw numbers from a device, but, say, images or geopositions—or actual measured physical quantities.

In Wolfram|Alpha we’ve, of necessity, assembled the world’s most complete system for handling physical quantities and their units. We’ve got a couple of thousand physical quantities built in (like length, or torque, or tensile strength, or clicks per impression), as well as nearly 10,000 units of measure (like inches, or meters per second or katals or micropascals per square root hertz). And in WDF we immediately get to use this whole setup.

So once we can get data out of a device, WDF provides a great way to represent it. And given the WDF form, there are lots of things we can do with the data.

For researchers, we’re building the Wolfram Data Repository, that lets people publish data—from devices or otherwise—using WDF in an immediately computable form.

We’re also building the Wolfram Data Science Platform, that lets people visualize and analyze data using all the sophistication of the Wolfram Language—and then generate complete interactive reports from the data, that can be deployed on the web, on mobile, offline, and so on.

But how can one actually interact with the device? Well, within the Wolfram Language we’ve been building a powerful framework for this. From a user’s point of view, there’s a symbolic representation of each device. Then there are a standard set of Wolfram Language functions like DeviceRead, DeviceExecute, DeviceReadBuffer and DeviceReadTimeSeries that perform operations related to the device.

Ultimately, this is implemented by having a Wolfram Language driver for each device. But the idea is that the end user never has to know about this. The appropriate driver is just automatically retrieved from the Wolfram Cloud when it’s needed. And then the general Wolfram Language framework goes from the low-level operations in the driver to all the various higher-level symbolic device functions. Like DeviceReadTimeSeries, which samples a series of data points from the device, then returns them in a symbolic TimeSeries object which can immediately be used for further visualization, analysis, etc.

There is another issue here, though: How does one actually make the connection to a particular device? It depends on the device. Some devices automatically connect to the cloud, perhaps through an intermediate mobile device. And in those cases, one typically just has to connect to an API exposed in the cloud.

But at least right now, many more devices connect in various kinds of wired or wireless ways to a specific local computer. Sometimes one may then want to interact with the data directly on that local computer.

But more often one either wants to have something autonomous happen with the data on the local computer. Or one wants to get the data into the cloud. For example so one can systematically have people or machines query it, generate reports from it, and so on.

And in both these cases, it’s often really convenient to have the basic device connect to some kind of small embeddable computer system. Like the Raspberry Pi $25 Linux computer, on which, conveniently enough, the Wolfram Language is bundled as part of its standard system software.

And if one’s running the Wolfram Language on the local machine connected to the device, there are mechanisms built into the language that allow both for immediate discovery, and for communication with the cloud. And more than that, with this setup there’s a symbolic representation of the device immediately accessible to the Wolfram Language in the cloud. Which means, for example, that parallel computation operations in the language can be used to aggregate data from networks of devices, and so on.

But, OK, so what are the kinds of devices one will be able to do all this with? Well, that’s what the Wolfram Connected Devices Project is intended to answer.

It’s certainly a very diverse list. Yes, there are lots of acceleration- and/or heart-rate-based health devices, and lots of GPS-based devices. But there are lots of other kinds of devices too, measuring scores of different physical quantities.

The devices range from tiny and cheap to huge and expensive. In the current list, about 2/3 of the devices are basically standalone, and 1/3 require continuous physical connectivity. The border of what counts as a “device”, as opposed to, for example, a component, is a bit fuzzy. Our operational definition for the Wolfram Connected Devices Project is that something can be considered a “connected device” if it measures some physical quantity, and can be connected to a general-purpose computer using some standard connector or connection technology.

For now, at least, we’ve excluded objects that in effect have complex custom electrical connectivity—for example, sensors that have the form factor of integrated circuits with lots of “legs” that have to be plugged into something. We’ve included, though, objects that have just a few wires coming out, that can for example immediately be plugged into GPIO ports, say on Raspberry Pi—or into analog ports on something like an Arduino connected to a Raspberry Pi.

The case of a device whose “interface” is just a few wires is usually one of the more straightforward. Things usually get more complicated when there are serial connections, USB, Bluetooth, and so on, involved. Sometimes devices make use of slightly higher-level protocols (like ANT+ or Bluetooth LE). But our experience so far is that ultimately there’s very little that’s truly standard. Each device requires custom work to create a driver, map properly to WDF, and so on.

The good news, of course, is that with the Wolfram Language we have an incredibly rich toolset for creating such drivers. Whether it’s by making use of the hundreds of import and export formats built into the language. Or all the mechanisms for calling external programs. Or the ways of handling time or place information. Or the algorithms for doing signal processing and time series analysis.

We’ve been interacting with many device manufacturers over the past year or so. And it’s been very encouraging. Because it seems as if the technology stack we’ve been building all these years is exactly what people need.

Countless times we’ve heard the same thing. “We’re building this great device; now we want to do great things with the data from it—analyzing it, delivering it to customers, and so on.” Well, that’s exactly what we’re going to be set up to do. And we have both the deep technical capabilities that are needed, and the practical infrastructure.

The first step is to get a Wolfram Language driver for the device. Once that’s done, everything flows from it. Whether it’s just storing computable versions of data in the Wolfram Data Repository. Or doing analysis or reporting through the Wolfram Data Science Platform. Or creating dashboards. Or exposing the data through an API. Or an app. Or producing alerts from the data. Or aggregating lots of data. Or, for that matter, combining data from multiple devices—for example in effect to create “synthetic sensors”.

There are lots of possibilities. One can use Wolfram SystemModeler to have a model for a device, that can be used to run a simulation in real time. Or one can use the control systems functions in the Wolfram Language to create a controller with the device. Or in a quite different direction, one can use our Wolfram|Alpha-style linguistic capabilities to let end users make natural language or voice queries about data coming from a device.

There are several common end results that manufacturers of devices typically want. One is just that it should be possible to take data from the device and flow it into the Wolfram Data Science Platform, or Mathematica, or some other Wolfram Language system, for some kind of processing. Another is that the whole user infrastructure around the device is built using our technology. Say creating a portal or dashboard on the web, or on a mobile device, for every single user of a particular type of device. That can use either our cloud, or a private cloud. And instead of a dashboard, one can have a query mechanism. Say through natural language for humans—or through some structured API for machines or programs.

In some ways the situation with connected devices right now is probably something of a transient. Because we’re mostly thinking about connecting devices to computers, and having those run the Wolfram Language. But in the future, the Wolfram Language is going to be running on increasingly small and ubiquitous embedded computers. And I expect that more and more connected devices are just going to end up having the computer power to run the Wolfram Language inside—so that they can do all sorts of Wolfram Language processing completely internally.

Of course, even in this case there is still going to have to be Wolfram Language code that reads raw data from sensors and so on. So there’s no getting around building drivers, just like for the current way most connected devices are set up.

We’ve had the experience now of building quite a few drivers. For simple devices, it’s a quick process. But as devices start to have more commands, and can generate more sophisticated data, it takes longer. In many ways, it feels like a curation task. Given all the Wolfram Language tools we have, it’s rarely about the details of manipulating data. Rather it’s about knowing what the data means, and knitting it into the whole WDF and Wolfram Language framework.

We’re going to have a service for manufacturers to work with us to connect their devices to our system. We’re also planning to run a sequence of hackathon-like events where students and others can work with devices to set up connections (and often get free devices at the end!).

The goal is to get seamless integration of as many kinds of devices as possible. And the more kinds of devices we have, the more interesting things are going to get. Because it means we can connect to more and more aspects of the physical world, and be in a position to compute more and more about it.

Within the Wolfram Language we have a rich symbolic way to represent the world. And with connected devices we have a way to attach this representation to real things in the world. And to make the Wolfram Language become a complete language for the Internet of Things.

But today we’re taking a first step. Launching the Wolfram Connected Devices Project to start the process of curating just what things exist so far in the current generation of the Internet of Things.

Visit the Wolfram Connected Devices Project »