To help me wrap up my time at the ODI I asked the team for suggestions for things I could add to my list of handover documentation.
Amongst the suggestions that came back was: “Maybe also a poem about why standards are the best thing on Earth?”
So, with a nod to the meme and apologies to William Carlos Williams. I wrote this:
I have tidied
the data
in your
spreadsheet
those numbers
you were
planning
to share
Forgive me
they were so messy
but now standard
and FAIR
Close enough I think 🙂
In my earlier post on the importance of tracking updates to datasets I noted that the UK Statistics Authority Code of Practice includes a requirement that publishers of official statistics must publish a policy that describes their approach to revisions and corrections.
See 3.9 in T3: Orderly Release, which states: “Scheduled revisions or unscheduled corrections to the statistics and data should be released as soon as practicable. The changes should be handled transparently in line with a published policy.”
The Code of Practice includes definitions of both Scheduled Revisions and Unscheduled Corrections.
Scheduled Revisions are defined as: “Planned amendments to published statistics in order to improve quality by incorporating additional data that were unavailable at the point of initial publication“.
Whereas Unscheduled Corrections are: “Amendments made to published statistics in response to the identification or errors following their initial publication“
I decided to have a read through a bunch of policies to see what they include and how they compare.
Here are some observations based on a brief survey of this list of 15 different policies including those by the Office of National Statistics, the FSA, Gambling Commission, CQC, DfE, PHE, HESA and others.
The Code of Practice applies to official statistics. Some organisations publishing official statistics also publish other statistical datasets.
In some cases organisations have written policies that apply:
There’s some variation in the amount of detail provided.
Some read as basic compliance documents with basic statements of intent to follow the recommendations of the code of practice. The include, for example a note that revisions and corrections will be handled transparently, in a timely way and with general notes about how that will happen.
Others are more detailed, giving more insight into how the policy will actually be carried out in practice. From a data consumer perspective these feel a bit more useful as they often include timescales for reporting, lines of responsibility and notes about how changes are communicated.
Some policies elaborate on the definitions in the code of practice, providing a bit more breakdown on the types of scheduled revisions and sources of error.
For example some policies indicate that changes to statistics may be driven by:
Some organisations publish provisional releases of these statistics. So their policies discuss Scheduled Revisions in this light: a dataset is published in one or more provisional releases before being finalised. During those updates the organisation may have been in receipt of new or updated data that impacts how the statistics are calculated. Or may fix errors.
Other organisations do not publish provisional statistics so their datasets do not have scheduled revisions.
A few policies include a classification of the severity of errors, along the lines of:
These classifications are used to describe different approaches to handling the errors, appropriate to their severity.
The policies frequently require decision making around how specific revisions and corrections might be handled. With implications for investment of time and resources in handling and communicating the necessary revisions and corrections.
In some cases responsibility lies with a senior leader, e.g. a Head of Profession, or other senior analyst. In some cases decision making rests with the product owner with responsibility for the dataset.
Scheduled changes are, by definition, planned in advance. So the policy sections relating to these revisions are typically brief and tend to focus on the release process.
In general, the policies align around:
Several of the policies include methodological changes in their handling of scheduled revisions. These explain that changes will be consulted on and clearly communicated in advance. In some cases historical data may be revised to align with the new methodology.
Handling of corrections tends to be the larger sections of each policy. These sections frequently highlight that despite rigorous quality control errors may creep in, either because of mistakes or because of corrections to upstream data sources.
There are different approaches to how quickly errors will be handled and fixed. In some cases this depends on the severity of errors. But in others the process is based on publication schedules or organisational preference.
For example, in one case (SEPA), there is a stated preference to handle publication of unscheduled corrections once a year. In other policies corrections will be applied at the next planned (“orderly”) release of the dataset.
Several policies note that there will be an impact assessment undertaken to fully understand an error before any changes are made.
These assessments include questions like:
Major errors tend to get some special treatment. Corrections to these errors are typically made more rapidly. But there are few commitments to timeliness of publishing corrections. “As soon as possible” is a typical statement.
The two exceptions I noted are the MOD policy which notes that minor errors will be corrected within 12 months, and the CQC policy which commits to publishing corrections within 20 days of an agreement to do so. (Others may include commitments that I’ve missed.)
A couple of policies highlight that errors may be found before a fix. In these cases, the existence of the error will still be reported.
The Welsh Revenue Authority was the only policy that noted that it might even retract a dataset from publication while it fixed an error.
A couple of policies noted that minor errors that did not impact interpretation may not be fixed at all. For example one ONS policy notes that errors within the original bounds of uncertainty in the statistics may not be corrected.
Minor typographic errors might just be directly fixed on websites without recording or reporting of changes.
There seems to be general consensus on the use of “p” for provisional and “r” for revised figures in statistics. Interestingly, in the Welsh Revenue Authority policy they note that while there is an accepted welsh translation for “provisional” and “revised”, the marker symbols remain untranslated.
Some policies clarify that these markers may be applied at several levels, e.g. to individual cells as well as rows and columns in a table.
Only one policy noted a convention around adding “revised” to a dataset name.
As required by the code of practice, the policies align on providing some transparency around what has been changed and the reason for the changes. Where they differ is around how that will be communicated and how much detail is included in the policy.
In general, revisions and corrections will simply be explained in the next release of the dataset, or before if a major error is fixed. The goal being to provide users with a reason for the change, and the details of the impact on the statistics and data.
These explanations are handled by additional documentation to be included in publications, markers on individual statistics, etc. Revision logs and notices are common.
Significant changes to methodologies or major errors get special treatment. E.g. via notices on websites or announcements via twitter.
Many of the policies also explain that known users or “key users” will be informed of significant revisions or corrections. Presumably this is via email or other communications.
One policy noted that the results of their impact assessment and decision making around how to handle a problem might be shared publicly.
A few of the policies included a commitment to carry out a review of how an error occurred in order to improve internal processes, procedures and methods. This process may be extended to include data providers where appropriate.
One policy noted that the results of this review and any planned changes might be published where it would be deemed to increase confidence in the data.
I found this to be an interesting exercise. It isn’t a comprehensive review, but hopefully it provides a useful summary of approaches.
I’m going to resist the urge to write recommendations or thoughts on what might be added to these policies. Reading a policy doesn’t tell us how well its implemented, or whether users feel it is serving their needs.
I will admit to feeling a little surprised that there isn’t a more structured approach in many cases. For example, pointers to where I might find a list of recent revisions or how to sign up to get notified as an interested user of the data.
I had also expected some stronger commitments about how quickly fixes may be made. These can be difficult to make in a general policy, but are what you might expect from a data product or service.
These elements might be covered by other policies or regulations. If you know of any that are worth reviewing, then let me know.
There are lots of recent examples of researchers collecting and releasing datasets which end up raising serious ethical and legal concerns. The IBM facial recognition dataset being just one example that springs to mind.
I read an interesting post exploring how facial recognition datasets are being widely used despite being taken down due to ethical concerns.
The post highlights how these datasets, despite being retracted, are still being widely used in research. This is in part because the original datasets are still circulating via mirrors of the original files. But also because they have been incorporated into derived datasets which are still being distributed with the original contents intact.
The authors describe how just one dataset, the DukeMTMC dataset was used in more than 135 papers after being retracted, 116 of those drawing on derived datasets. Some datasets have many derivatives, one example cited has been used in 14 derived datasets.
The research raises important questions about how datasets are published, mirrored, used and licensed. There’s a lot to unpack there and I look forward to reading more about the research. The concerns around open licensing are reminiscent of similar debates in the open source community leading to a set of “ethical open source licences“.
But the issue I wanted to highlight here is the difficulty of tracking the mirroring and reuse of datasets.
Change notification is a missing piece of our data infrastructure.
If it were easier to monitor important changes to datasets, then it would be easier to:
The importance of these activities can be seen in other contexts.
For example, Retraction Watch is a project that monitors retractions of research papers. CrossMark helps to highlight major changes to published papers including corrections and retractions.
Principle T3: Orderly Release, of the UK Statistics Authority code of practice explains that scheduled revisions and unscheduled corrections to statistics should be transparent, and that organisations should have a specific policy for how they are handled.
More broadly, product recalls and safety notices are standard for consumer goods. Maybe datasets should be treated similarly?
This feels like an area that warrants further research, investment and infrastructure. At some point we need to raise our sights from setting up even more portals and endlessly refining their feature sets and think more broadly about the system and ecosystem we are building.
I was very pleased to announce my new TV series today, loosely based on real events. More details here in the official press release.
FOR IMMEDIATE RELEASE
Coming to all major streaming services in 2021 will be the exciting new series: “Turning the Tables“.
Exploring the murky corporate world of poorly formatted spreadsheets and nefarious macros each episode of this new series will explore another unique mystery.
When the cells lie empty, who can help the CSV:PI team pivot their investigation?
When things don’t add up, who can you turn to but an experienced solver?
Who else but Leigh Dodds, Consulting Spreadsheet Detective?
This smart, exciting and funny new show throws deductive reasoner Dodds into the mix with Detectives Rose Cortana and Colm Bing part of the crack new CSV:PI squad.
Rose: the gifted hacker. Quick to fire up an IDE, but slow to validate new friends.
Colm: the user researcher. Strong on empathy but with an enigmatic past that hints at time in the cells.
What can we expect from Season 1?
Episode 1: #VALUE!
In his first case, Dodds has to demonstrate his worth to a skeptical Rose and Colm, by fixing a corrupt formula in a startup valuation.
Episode 2: #NAME?
A personal data breach leaves the team in a race against time to protect the innocent. A mysterious informant known as VLOOKUP leaves Dodds a note.
Episode 3: #REF!
A light-hearted episode where Dodds is called in to resolve a mishap with a 5-a-side football team matchmaking spreadsheet. Does he stay between the lines?
Episode 4: #NUM?
A misparsed gene name leads a researcher into recommending the wrong vaccine. It’s up to Dodds to fix the formula.
Episode 5: #NULL!
Sometimes it’s not the spreadsheet that’s broken. Rose and Colm have to educate a researcher on the issue of data bias, while Dodds follow up references to the mysterious Macro corporation.
Episode 6: #DIV/0?
Chasing down an internationalisation issue Dodds, Rose and Colm race around the globe following a trail of error messages. As Dodds gets unexpectedly separated from the CSV:PI team, Rose and Colm unmask the hidden cell containing the mysterious VLOOKUP.
In addition to the six episodes in season one, a special feature length episode will air on National Spreadsheet Day 2021:
Feature Episode: #####
Colm’s past resurfaces. Can he grow enough to let the team see the problem, and help him validate his role in the team?
Having previously only anchored documentaries, like “Around with World with 80,000 Apps” and “Great Data Journeys“, taking on the eponymous role will be Dodds’ first foray into fiction. We’re sure he’ll have enough pizazz to wow even the harshest critics.
“Turning the Tables” will feature music composed by Dan Barrett.
I love a good standard. I’ve written about them a lot here.
As its #WorldStandardsDay I thought I’d write a quick post to share something that I’ve learned from leading and supporting some standards work.
I’ve already shared this with a number of people who have asked for advice on standards work, and in some recent user research interviews I’ve participated in. So it makes sense to write it down.
In the ODIHQ standards guide, we explained that at the end of your initial activity to develop a standard, you should plan to produce a range of outputs. This include a variety of tools and guidance that help people use the standard. You will need much more than just a technical specification.
To plan for the different types of documentation that you may need I recommend applying this “Grand Unified Theory of Documentation“.
That framework highlights four different types of documentation are intended to be used by different audiences to address different needs. The content designers and writers out there reading this will be be rolling their eyes at this obvious insight.
Here’s how I’ve been trying to apply it to standards documentation:
This is your primary technical specification. It’ll have all the detail about the standard, the background concepts, the conformance criteria, etc.
It’s the document of record that captures all of the hard work you’ve invested in building consensus around the standard. It fills a valuable role as the document you can point back to when you need to clarify or confirm what was agreed.
But, unless its a very simple standard, it’s going to have a limited audience. A developer looking to implement a conformant tool, API or library may need to read and digest all of the detail. But most people want something else.
Put the effort into ensuring its clear, precise and well-structured. But plan to also produce three additional categories of documentation.
Many people just want an overview of what it is designed to do. What value will it provide? What use cases was it designed to support? Why was it developed? Who is developing it?
These are higher-level introductory questions. The type of questions that business stakeholders want to answer to sign-off on implementing a standard, so it goes onto a product roadmap.
Explainers are also useful background information that are useful for a developer ahead of taking a deeper dive. If there are some key concepts that are important to understanding the design and implementation of a standard, then write an explainer.
A simple, end-to-end description of how to apply the standard. E.g. how to publish a dataset that conforms to the standard, or export data from an existing system.
A tutorial will walk you through using a specific set of tools, frameworks or programming languages. The end result being a basic implementation of the standard. Or a simple dataset that passes some basic validation checks. A tutorial won’t cover all of the detail, it’s enough to get you started.
You may need several tutorials to support different types of users. Or different languages and frameworks.
If you’ve produced a tool, like validator or a template spreadsheet to support data publication, you’ll probably need a tutorial for each of them unless they are very simple to use.
Tutorials are gold for a developer who has been told: “please implement this standard, but you only have 2 days to do it”.
Short, task oriented documentation focused on helping someone apply the standard. E.g. “How to produce a CSV file from Excel”, “Importing GeoJSON data in QGIS”, “Describing a bus stop”. Make them short and digestible.
How-Tos can help developers build from a tutorial, to a more complete implementation. Or help a non-technical user quickly apply a standard or benefit from it.
You’ll probably end up with lots of these over time. Drive creating them from the types of questions or support requests you’re getting. Been asked how to do something three times? Write a How-To.
There’s lots more that can be said about standards documentation. For example you could add Case Studies to this list. And its important to think about whether written documentation is the right format. Maybe your Explainers and How-Tos can be videos?
But I’ve found the framework to be a useful planning tools. Have a look at the documentation for more tips.
Producing extra documentation to support the launch of a standard, and then investing in improving and expanding it over time will always be time well-spent.
It’s an odd now looking at early 21st century content in the Internet Archive. So little nuance.
It feels a little like watching those old black and white movies. All that colour which was just right there. But now lost. Easy to imagine that life was just monochrome. Harder to imagine the richer colours.
Or at least hard for me. There are AIs that will imagine it all for you now, of course. There have been for a while. They’ll repaint the pictures using data they’ve gleaned from elsewhere. But it’s not the film that is difficult to look at. It’s the numbers.
How did you manage with just those bare numerals?
If I showed you, a 21st century reader, one of our numbers you wouldn’t know what it was. You wouldn’t be able to read it.
Maybe you’ve seen that film Arrival? Based on a book by Ted Chiang? Remember the alien writing that was so complex and rich in meaning? That’s what our numbers might look like to you. You’d struggle to decode them.
Oh, the rest of it is much the same. The text, emojis and memes. Everything is just that bit richer, more visual. More nuanced. It’s even taught in schools now. Standardised, tested and interpreted for all. It’d be familiar enough.
You struggle with the numbers though. They’d take much more time to learn.
Not all of them. House numbers. Your position in the queue. The cost of a coffee. Those look exactly the same. Why would we change those?
It’s the important numbers that look different. The employment figures. Your pension value. Your expected grade. The air quality. The life-changing numbers. Those all look very different now.
At some point we decided that those numbers needed to be legible in entirely different ways. We needed to be able to see (or hear, or feel) the richness and limitations in the most important numbers. It was, it turned out, the only way to build that shared literacy.
To imagine how we got there, just think about how people have always adapted and co-opted digital platforms and media for their own ends. Hashtags and memes.
Faced with the difficulty of digging behind the numbers – the need to search for sample sizes, cite the sources, highlight the bias, check the facts – we had to find a different way. It began with adding colour, toying with fonts and diacritics.
5—a NUMBER INTERPOLATED.
It took off from there. Layers of annotations becoming conventions and then standards. Whole new planes and dimensions in unicode.
42—a PROJECTION based on a SIGNIFICANT POPULATION SAMPLE.
All of the richness, all of the context made visible right there in the number.
27-30—a PREDICTED RANGE created by a BAYESIAN INTERPOLATION over a RECENT SAMPLE produced by an OFFICIAL SOURCE.
180—an INDICATOR AUTOMATICALLY SELECTED by a DEEP LEARNING system, NO HUMAN INTERVENTION.
Context expressed as colour and weight and strokes in the glyphs. You can just read it all right off the digits. There and there. See?
Things aren’t automatically better of course. Numbers aren’t suddenly to be more trusted. Why would they be?.
It’s easier to see what’s not being said. It’s easier to demand better. It’s that little bit harder to ignore what’s before your eyes. It moves us on in our debates or just helps us recognise when the reasons for them aren’t actually down to the numbers at all.
It’s no longer acceptable to elide the detail. The numbers just look wrong. Simplistic. Black and white.
Which is why it’s difficult to read the Internet Archive sometimes.
We’ve got AIs that can dream up the missing information. Mining the Archive for the necessary provenance and add it all back into the numbers. Just like adding colour to those old films, it can be breathtaking to see. But not in a good way. How could you have deluded yourselves and misled each other so easily?
I’ve got one more analogy for you.
Rorschach tests have long been consigned to history. But one of our numbers – the life-changing ones – might just remind you of a colourful inkblots. And you might accuse use of we’re just reading things into them. Imagining things that you just aren’t there.
But numbers are just inkblots. Shapes in which we choose to see different aspects of the world. They always have been. We’ve just got a better palette.
I’ve been growing vegetables in our garden for years now. I usually end up putting the garden “to bed” for the winter towards the end of September. Harvesting the last bits of produce, weeding out the vegetable patches and covering up the earth until the Spring.
I thought I’d also do a bit of a retro to help me reflect on what worked and what didn’t work so well this year. We’ve had some mixed successes, so there are some things to reflect and improve on.
This year I wanted to do a few things:
To help achieve my goals, I made the following changes this year:
The final list for this year was (new things in bold):
Basil, Butternut Squash, Carrots, Coriander, Cucumber, Lettuce, Pak Choi, Peas, Potato, Radish, Shallots, Spinach, Spring Onion, Sweetcorn
So, not as many new vegetables as I’d hoped, but I did try some different varieties.
Main thing I want to do next year is get a green house. I’ve got my eye on this one. I want to grow tomatoes, chillis and peppers. It’ll also help me acclimatise some of the seedling before properly planting out.
Having the space to grow vegetables is a privilege and I’m very glad and very lucky to have the opportunity.
Gardening can be time consuming and frustrating, but I love being able to cook with what I’ve grown myself. Getting out into the garden, doing something physical, seeing things grown is also a nice balm given everything else that is going on.
Looking forward to next year.
Vaughn Tan’s The Uncertainty Mindset is one of the most fascinating books I’ve read this year. It’s an exploration of how to build R&D teams drawing on lessons learned in high-end kitchens around the world. I love cooking and I’m interested in creative R&D and what makes high-performing teams work well. I’d strongly recommend it if you’re interested in any of these topics.
I’m also a sucker for a good intellectual framework that helps me think about things in different ways. I did that recently with the BASEDEF framework.
Tan introduces a nice framework in Chapter 4 of the book which looks at four broad types of innovation around food. These are presented as a way to help the reader understand how and where innovation creates impact in restaurants. The four categories are:
The categories are the top are more evident to the consumer, those lower down less so. But the impacts of new methods and processes are greater as they apply in a variety of contexts.
Somewhat inevitably, I found myself thinking about how these categories work in the context of data:
The breakdown isn’t perfect, but I found the exercise useful to think through the types of innovation around data. I’ve been conscious recently that I’m often using the word “innovation” without really digging into what that means, how that innovation happens and what exactly is being done differently or produced as a result.
The categories are also useful, I think, in reflecting on the possible impacts of breakthroughs of different types. Or perhaps where investment in R&D might be prioritised and where ensuring the translation of innovative approaches into the mainstream might have most impact?
What do you think?
This post is a bit of a diary entry. It’s to help me remember a fun little activity that I was involved in recently.
I’d seen little gifs and screenshots of Townscaper on twitter for months. But then suddenly it was in early access.
I bought it and started playing around. I’ve been feeling like I was in a rut recently and wanted to do something creative. After seeing Jim Rossignol mention playing with townscaper as a nightly activity, I thought I’d do similar.
Years ago I used to do lunchtime hacks and experiments as a way to be a bit more creative than I got to be in my day job. Having exactly an hour to create and build something is a nice constraint. Forces you to plan ahead and do the simplest thing to move an idea forward.
I decided to try lunchtime Townscaper builds. Each one with a different theme. I did my first one, with the theme “Bridge”, and shared it on twitter.
Chris Love liked the idea and suggested adding a hashtag so others could do the same. I hadn’t planned to share my themes and builds every day, but I thought, why not? The idea was to try doing something different after all.
So I tweeted out the first theme using the hashtag.
That tweet is the closest thing I’ve ever had to a “viral” tweet. It’s had over 53,523 impressions and over 650 interactions.
Turns out people love Townscaper. And are making lots of cool things with it.
Tweetdeck was pretty busy for the next few days. I had a few people start following me as a result, and suddenly felt a bit pressured. To help orchestra things and manage my own piece of mind, I did a bit of forward planning.
I decided to run the activity for one week. At the end I’d either hand it over to someone or just step back.
I also spent the first evening brainstorming a list of themes. More than enough for me to keep me going for the week, so I could avoid the need to come up with new themes on the fly. I tried to find a mixture of words that were within the bounds of the types of things you could create in Townscaper, but left room for creativity. In the end I revised and prioritized the initial list over the course of the week based on how people engaged.
I wanted the activity to be inclusive so came up with a few ground rules: “No prizes, no winners. It’s just for fun.”. And some brief guidance about how to participate: post screenshots, use the right hashtags).
I also wanted to help gather together submissions, but didn’t want to retweet or share all of them. So decided to finally try out creating twitter moments. One for each daily challenge. This added some work as I was always worrying I’d missed something, but it also meant I spent time looking at every build.
I ended up with two template tweets, one to introduce the challenge and one to publish the results. These were provided as a single thread to help weave everything together.
And over the course of a week, people built some amazing things. Take a look for yourself:
People played with the themes in interesting ways. They praised and commented on each others work. It was one of the most interesting, creative and fun things I’ve done on twitter.
By the end of the week, only a few people were contributing, so it was right to let it run its course. (Although I see that people are still occasionally using the hashtag).
It was a reminder than twitter can be and often is a completely different type of social space. A break from the doomscrolling was good.
It was also a reminded me how much I loved creating and making things. So I’m resolved to do more of that in the future.
I read an interesting article this week by Ana Brandusescu, Michael Canares and Silvana Fumega. Called “Open data standards design behind closed doors?” it explores issues of inclusion and equity around the development of “open data standards” (which I’m reading as “open standards for data”).
Ana, Michael and Silvana rightly highlight that standards development is often seen and carried out as a technical process, whereas their development and impacts are often political, social or economic. To ensure that standards are well designed, we need to recognise their power, choose when to wield that tool, and ensure that we use it well. The article also asks questions about how standards are currently developed and suggests a framework for creating more participatory approaches throughout their development.
I’ve been reflecting on the article this week alongside a discussion that took place in this thread started by Ana.
I agree that standards development should absolutely be more inclusive. I too often find myself in standards discussions and groups with people that look like me and whose experiences may not always reflect those who are ultimately impacted by the creation and use of a standard.
In the open standards for data guidebook we explore how and why standards are developed to help make that process more transparent to a wider group of people. We also placed an emphasis on the importance of the scoping and adoption phases of standards development because this is so often where standards fail. Not just because the wrong thing is standardised, but also because the standard is designed for the wrong audience, or its potential impacts and value are not communicated.
Sometimes we don’t even need a standard. Standards development isn’t about creating specifications or technology, those are just outputs. The intended impact is to create some wider change in the world, which might be to increase transparency, or support implementation of a policy or to create a more equitable marketplace. Other interventions or activities might achieve those same goals better or faster. Some of them might not even use data(!)
But looking back through the guidebook, while we highlight in many places the need for engagement, outreach, developing a shared understanding of goals and desired impacts and a clear set of roles and responsibilities, we don’t specifically foreground issues of inclusion and equity as much as we could have.
The language and content of the guidebook could be improved. As could some prototype tools we included like the standards canvas. How would that be changed in order to foreground issues of inclusion and equity?
I’d love to get some contributions to the guidebook to help us improve it. Drop me a message if you have suggestions about that.
Open standards for data are reusable agreements that guide the exchange of data. They shape how I collect data from you, as a data provider. And as a data provider they shape how you (re)present data you have collected and, in many cases will ultimately impact how you collect data in the future.
If we foreground standards as agreements for shaping how data is collected and shared, then to increase inclusion and equity in the design of those agreements we can look to existing work like the Toolkit for Centering Racial Equity which provides a framework for thinking about inclusion throughout the life-cycle of data. Standards development fits within that life-cycle, even if it operates at a larger scale and extends it out to different time frames.
We can also recognise existing work and best practices around good participatory design and research.
We should avoid standards development, as a process, being divorced from broader discussions and best practices around ethics, equity and engagement around data. Taking a more inclusive and equitable approach to standards development is part of the broader discussion around the need for more integration across the computing and social sciences.
We may also need to recognise that sometimes agreements are made that don’t provide equitable outcomes for everyone. We might not be able to achieve a compromise that works for everyone. Being transparent about the goals and aims of a standard, and how it was developed, can help to surface who it is designed for (or not). Sometimes we might just need different standards, optimised for different purposes.
There are many different types of standard. And standards can be applied to different types of data. The authors of the original article didn’t really touch on this within their framework, but I think its important to recognise these differences, as part of any follow-on activities.
The impacts of a poorly designed standard that classifies people or their health outcomes will be much more harmful than a poorly defined data exchange format. See all of Susan Leigh Star‘s work. Or concerns from indigenous peoples about how they are counted and represented (or not) in statistical datasets.
Increasing inclusion can help to mitigate the harmful impacts around data. So focusing on improving inclusion (or recognising existing work and best practices) around the design of standards with greater capacity for harms is important. The skills and experience required in developing a taxonomy is fundamentally different to those required to develop a data exchange format.
Recognising these differences is also helpful when planning how to engage with a wider group of people. As we can identify what help and input is needed: What skills or perspectives are lacking among those leading standards work? What help or support needs to be offered to increase inclusion. E.g. by developing skills, or choosing different collaboration tools or methods of seeking input.
Since we launched the standards guidebook I’ve been wondering whether it would be helpful to have more of a community of practice around standards development. I found myself thinking about this again after reading Ana, Michael and Silvana’s article and the subsequent discussion on twitter.
What would that look like? Does it exist already?
Perhaps supported by a set of learning or training resources that re-purposes some of the ODI guidebook material alongside other resources to help others to engage with and lead impactful, inclusive standards work?
I’m interested to see how this work and discussion unfolds.
