The Open Data Revolution

This post was originally published as an article for The Kernel.

Open data is now mainstream. Spurred on by the US and UK governments vying over who could be the most open, there are now open data activities in many regions around the globe.

Under the “Government as Platform” rubric, you can find the public sector exploring social media, conducting experiments in citizen engagement and crowd-sourcing, and casting a spotlight on their own activities through various transparency initiatives. Almost all of those initiatives are being driven by the demand for and publishing of open data.

The local agendas often differ, and the political messages have evolved over time. In the pre-Cameron UK, we were originally promised open data to support business innovation. Then, we had that ugly little episode with expense claims, where transparency suddenly came to the fore. But when the promised armies of “armchair auditors” didn’t turn up, open data held the promise of making government more efficient. The agenda now seems to be coming full circle, with business innovation once more the primary theme behind open government data. It heralds a “new industrial revolution”.

For that to happen, however, the data needs to be put into the hands of people who can and will do something useful and valuable with it. Compared with where things stood a few years ago, we now have almost an embarrassment of riches when it comes to open data. Today, we have access to data about transit networks, weather reports, government statistics, right down to cycle paths and locations of public toilets. And yet there’s still so much more that should be opened.

It’s not hard to find lots of excellent examples of interesting re-uses of open government data. Early projects such as Mapnificent, or its UK peer Mapumental, lead the way in showing how different types of data can be layered together to create useful services. Mixing mapping with transit data and house price information can help us decide where to meet or where to live. Crime and school performance statistics add yet another dimension to some of those important life decisions. FixMyStreet and FixMyTransport do exactly what they say on the tin.

While exciting, many of these applications have been research projects or experiments. They are demonstrations in the art of the possible, which have supported the demand aggregation required to make Government open its databases. But hackdays and appathons build awareness, not businesses.

We’re now at the stage where we can move forward. Necessary issues such as data licensing are being resolved, and the next wave of open government data-driven start-ups are starting to emerge. Duedil or OpenCorporates are both great examples, and there are efforts to collect together more evidence of progress. The announcement of the Open Data User Group and the planned launch of the Open Data Institute in London’s technology hub later this year demonstrate continued Government momentum behind open data.

It feels increasingly like the ball is now in our court: What are we going to build? Can open data really help the economy? Can it help us lead better lives? There are several fertile areas.

Mixing together open public data with private personal data about our increasingly-quantified selves can lead to the creation of more personalised, tailored services. Save money on bills, get better healthcare, make better decisions.

Some of the more interesting open data projects, however, have been happening at a regional or city level. There are several cities, already tech hubs in their own right, that have been actively embracing open data for some time. San Francisco was an early leader; New York and London have their own “data hubs”; and other cities such as Chicago are rapidly playing catch-up.

The vision of “Smart Cities” is driving a lot of interest and massive investment. The term is often equated with improvements in infrastructure, such as ubiquitous wifi or sensor networks to improve transport and crisis management, but there’s more to it. Smarter cities are also about supporting and encouraging local talent, especially in creating new start-ups and services. Smarter cities are driven by open data.

Local technical communities are engaging at a local level and not just as an insular startup cluster. Hyperlocal activism has been associated with the open data movement for some time, but projects like sf.citi and Open Data Manchester illustrate how the activism is moving forward.

The combination of a motivated technical community – who have an understanding of what needs to be done to improve local services – coupled with access to the data they need to build on, is going to be a sweet spot for innovation. It can sustain a feedback loop that can benefit businesses and citizens alike.

What’s often overlooked is that open data is open for anyone to use. It’s as likely that a start-up or large corporate from overseas could use that open data as a local business. They may even be able to do it quicker. The value created by opening up data won’t necessarily all end up in the country of origin. Where local start-ups and developers have an advantage is in understanding the needs of their local communities, because they’re already embedded in it. There’s a long, long tail of open government data and much of that will be relevant only to local communities.

Ultimately the data, and the services that use it, that underlies where and how we live, will have the greatest and longer lasting impact. Whether that can turn around the economy or change the way we live, there’s really only one way to find out.