Adventures in geodata

I spend a lot of my professional life giving people advice. Mostly around how to publish and use open data. In order to make sure I give people the best advice I can, I try and spend a lot of time actually publishing and using open data. A mixture of research and practical work is the best way I’ve found of improving my own open data practice. This is one of the reasons I run Bath: Hacked, continue to work at the Open Data Institute, and like to stay hands-on with data projects.

Amongst my goals for this year was to spend time learning some new skills. For example, I’ve not been involved in running a crowd-sourcing project, but now have that underway with Accessible Bath.

And, while I’ve done some work with geographic data, until recently I hadn’t really spent any time contributing to OpenStreetmap or exploring its ecosystem. But I’ve spent the last couple of months fixing that by immersing myself in its community and tools. In this blog post I wanted to share some of the things I’ve learnt. It’s been really fascinating and, as I’d hoped, given me a new perspective on a number of issues.

Finding my way

To begin with I looked around for some online tutorials. While I knew that OpenStreetmap allowed anyone to contribute, I wasn’t really sure about how I could go about doing that. I had a bunch of questions such as:

• Did I need a dedicated GPS device or could I collect data I my phone? (Answer: you can use your phone)
• Did I need to go out with a clipboard and do a formal survey or are there other ways to contribute? (Answer: you can contribute in a lot of different ways)
• How do you actually go about editing the map, what tools do you need to use? (Answer: however you feel comfortable)
• How do I find useful ways to contribute? Has everything been mapped already? (Answer: there’s still a lot to do!)

To help answer my questions I started out by watching some YouTube tutorials. There’s a lot of great training material for the OpenStreetMap ecosystem that covers the basics of mapping, how to add buildings, and some nice bite size videos that introduce best elements of the tool-set.

Other people in the Bath: Hacked community had also been looking at OpenStreetmap, mainly as a potential data resource. So we held a small evening meetup to get together and share what we knew. We had two experienced local mappers who came along and also offered encouragement (thanks Neil and Dave!).

This was a great way to learn the ropes and build up the confidence to wade in. I personally found having some existing members of the OSM community on hand very helpful. Dave has been particularly supportive of reviewing my edits and offering suggested improvements.

Equipping my expedition

There’s an amazing set of tools that support the OSM community. Too many to mention in a single blog post. But here’s a few that I’ve found particularly useful:

• There are a few different OSM editors, but the new, default iD editor is really easy to use. If you plan on making some editrs, focus on learning this tool, rather than looking at the older, more complex tools (although they have their uses). It’s really nice to work with. It also has some pleasing little UX elements.
• osmtracker is an Android (and Windows mobile) application that lets you record GPS traces, upload them to OSM (where they can be viewed in the iD editor), and exported to GPX files for use in other tools. It’s in the app store so easy to install
• The OSM wiki is an essential resource. The OSM database itself is basically a wiki: you can add tags to any item on the map. While the online editor does a lot of the work for you, sometimes you need to add some additional metadata and the trick is in knowing which tags to add to which locations. The wiki provides plenty of examples. It also includes some beginner tutorials, but I found the videos to be a good starting point

My first attempt at proper mapping was walking my local high street, recording my progress and using osmtracker to take notes of the names of each shop. I later updated the building outlines, names and details of all the locations.

Into the unknown

That process of collecting data and updating a map lit up the bits of my brain that likes exploring and scavenging in video games, so I was immediately keen to do more. That’s when I starting contributing to Missing Maps, which I’d heard about from Rares during our meetup.

Missing Maps uses volunteers to trace satellite imagery of locations around the world. This data is then improved locally and used by humanitarian organisations to plan their disaster response activities. So I spent a happy evening finding and tracing Tukuls in Sudan. I thoroughly enjoyed it. It felt like doing an adult colouring book, but where I was painting the world a bit better with each stroke.

As a contributor the tooling is great: simple task allocation, clear guidance and tutorials, and making contributions is straight-forward as you’re using the standard editors. The community was also quick to provide feedback.

I also tripped over MapSwipe. This lets you identify, with a simple click, satellite images that contain buildings. This generates new tasks that go into the Missing Maps pipeline. It also has some light gamification and encouragement to keep you contributing.

Even if you’re not confident about editing the full map, you can quickly make small contributions using this mobile app. You can download tasks for use offline, so it’s also possible to map when you’re on the go. There’s a little micro-tasking app called StreetComplete which takes a similar approach towards making local contributions as easy as possible.

Between MapSwipe, MissingMaps and editing the local OSM map and updating locations on the Wheelmap app,  I’m now trying to make a small contribution to OSM every day.

The landscape

I’ve been really blown away by the range of tools and applications that fill out the OSM ecosystem. I plan on doing a lengthier post on some of this at a later date, but I’d be very surprised if this ecosystem wasn’t at least as good as, or even better than those used internally by national mapping agencies.

The ecosystem doesn’t just consist of hobbyists, there’s a growing commercial community that are contributing to, supporting and helping develop OpenStreetMap. Just look at how clearly Mapbox and Mapillary articulate how their company strategies align with making OSM a continued success.

I was also really surprised to learn that the satellite imagery that all OSM mappers are using has been donated by Microsoft. The Bing aerial imagery is free for use in OSM mapping and has been since 2010. That’s a significant contribution to an open data ecosystem.

If you’re interested in learning more about the OSM community, I’d encourage you to explore the videos from the annual State of the Map conference. There’s some really interesting work presented there including:

• introductions to new OSM tools and research
• analysis and discussion about the OSM community itself, the reasons why people contribute and how to encourage them to continue to do so
• case studies of how OSM data and tooling is used in a variety of projects

New territory

I’ve now done several street surveys of Bath and have refined my workflow. What I’ve found to be the simplest approach is to use osmtracker to record my route and uses its facility to take photos of streets and shop fronts. This gives a quick way to collect information on the go, and I can then use this update the map later.

Uploading the GPX traces to OSM, putting the photos into the public domain on Flickr, and also publishing them to Mapillary allows me to demonstrate that I’ve actually done the field work, rather than just sneakily copied from Google StreetView, whilst also making them available to other people to use when they’re mapping. Mapillary data can be added to the iD editor so you can see contributed photos as you work.

I’ve decided that the surveys are a good way to encourage me to be more active over the summer!

Trip report

This post has just been a taster of what I’ve learnt and explored over the last couple of months. If you’ve ever wondered about contributing to OSM I’d encourage you to have a go. And I’m happy to help you get started! As I’ve outlined here, there are a number of different ways you can contribute either your local knowledge, or pitch in to some humanitarian mapping.

I’m going to be writing more here about some of the ecosystem in future. The exercise has been a great insight into how the OSM community hangs together and I’ve really only scratched the surface.

To briefly summarise though I think there’s some aspects of OSM that could work well in other contexts, for example:

• the various approaches taken to ensuring quality and consistency of the map
• the effort that goes into understanding and managing the community
• the means by which commercial and volunteer efforts can both contribute to an open resource

If you’re interested in data as infrastructure then OSM is a great project to study in more detail. I think it embodies all of the key principles of a strong open data infrastructure.

Someone also needs to do a proper review of the OSM ecosystem because all of that “open data impact” people are looking to measure is right there. There’s a bit too much focus on measuring impact of government data IMHO, when there’s an existing ecosystem which can provide some great insights.