Every now and then I check in on the Solid project. Launched about 7 years ago, Solid is intended to help put people back in control of data about them.
As I understand it, a Solid “Pod” is intended to be a personal data store that will provide you with complete control over data about you, allowing you to authorise which products and services can access that data.
Instead of all your data being held in platforms, you authorise them to use your data which is hosted separately. This can address privacy and allow portability of data between services.
The project is lead by Tim Berners-Lee and he has a startup, Inrupt, that is aiming to bring this to market by providing an enterprise grade version of the software. Solid is mentioned in this post about the Web’s 35th birthday.
What’s confusing me is that I’m really not sure how Solid is intended to deliver on its vision, or what problems it’s actually solving for either users or developers.
On my occasional check-ins, I’ve been trying to evaluate it in different ways: as a user, and as a developer/product manager. Would Solid be a useful way to manage my own archives? Or help me develop applications quicker or more securely?
A user view
As a user, I can’t find a Pod hosting service that isn’t experimental or which will allow me to actually sign up and pay to have data securely hosted. I can’t use it right now.
Having signed up to use some of those experimental services, I was surprised to find that the default user interface looks like a website prototyped in Bootstrap over a weekend. The Inrupt version of the “Pod Browser” is slightly more polished but doesn’t offer more than a browser based tool for managing files.
Things I expected to find but couldn’t, include:
- the ability to easily and regularly input data from existing services I use. Like my blog, bank transactions, photos or social media. This is the information I care about
- a mobile application that would allow me to access, sync and share data across my devices
- proper fine-grained controls for sharing data in the Pod. I can allow applications to read and write specific resources in the Pod, but there’s no way I can see to only share certain pieces of information (e.g. just my email address) or provide time-limited access (e.g. you can read my bank details for the next 24 hours)
- some level of auditing and reporting on what applications are accessing my Pod and what they’re doing with that data
I can only conclude that a Pod isn’t meant to be a consumer facing piece of software?
By not providing me with real fine-grained control over data I’ve stored in the POD, and some immediate built-in utility, it means that a POD doesn’t give me more than I can currently get with from Dropbox or Google Drive.
As a product it’s not currently convincing me that it’s going to provide a more trustworthy or reliable way to manage data about me than those existing tools.
This Inrupt video provides an explainer on Solid Pods. What was particularly surprising was the suggestion that as a user I might have multiple Solid Pods, all hosted with different organisations?
This doesn’t feel like it’s putting me in control over data about me. I still need to trust multiple services and agree to their terms of services. I need to manage information and access in multiple places. Potentially with different user interfaces.
Developer view
So maybe Solid is meant to be more of a developer tool at the moment, and the focus is on creating a richer technical platform around which applications can be built?
But again there seem to be a lot of features that are lacking.
The API to a Solid Pod is essentially a document store that supports capturing additional metadata about the stored resources. But it doesn’t provide me with a way to query or search over the data that my application has stored?
If I need that, or to work with more complex data, or combine data from other sources and users, then I’ll still need to build a separate data store. This means that I’m still going to have to deal with security, privacy, GDPR, etc. It’s not saving me any implementation time in building a secure system.
Because the current user interface is so poor I’ll need to build my own UI. If I used Dropbox or Drive as a backend for my application, the UI would at least be familiar to users.
If the users aren’t hosting their own Pods, or using a third-party, I’ll need to deploy the software myself in addition to my existing stack.
If users want to allow third-parties to access their data, then I’m taking on the hosting and API costs for those other applications.
If I want to interoperate with other apps, I’ll need to work on that with those other developers. Solid has no built in understanding of any specific schemas or formats. Or recommended ways to structure data.
If I want to create applications in banking, health, etc then I’ll still need to go through all the required security, certification and audit procedures. It’s not clear to me that Solid helps accelerate that, not least because there don’t seem to be dedicated hosting providers.
It still all feels very much like an idea trying to find a solution. I can’t help but think that something consumer focused, with a proper hosted environment, would actually accelerate adoption. But I think the core still needs a lot of work.
However, what feels like the most fundamental problem to solve around the product at the moment is this:
- if a user can only authorise and manage access for specific resources, e.g. a blob of data stored by an application, not individual fields…
- and, if the data that is stored in those resources, is decided by individual apps…
- and, Solid doesn’t natively understand that data…
- then there’s all kinds of security & privacy issues that can arise with users accidentally sharing sensitive information because they don’t realise the extent of what they’re sharing?
I may be being harsh, and all of this is still on the roadmap . But the project appears to be moving very slowly. It’s also possible I’ve just misunderstood something fundamental. If I have then leave a comment.
For the moment though I’m going to cross it off my list of things to keep an eye on.
Lost Boy 
I think the problem with SOLID might end up being the same as the semantic web. A colleague/friend once said to me that organisations don’t want to mark up their using open data formats (specifically RDF) and publish it on the web. Equally organisations don’t want users to own their own personal data (its too valuable).
i think the actual practical issues would be resolved if there was enough intent (lets face it, the web was far from perfect in the early days)
Hi Leigh! I think it is very timely to put up some big question marks. Solid has become something very different from what I thought it would be too.
First, it is much older than you’d think. Although the name Solid was probably used at MIT around 2015, the ideas were kicking around much longer. It obviously builds on that TimBL always intended for the Web to be as much write as read, I recall stuff around 1998 at the advent of RDF talking about writing RDF documents much as HTML was dominant then. And then, there’s the Policy Aware Web project around 2004, FOAF+SSL pioneered by Toby Inkster and Henry Story around 2007, the RWW community group from 2011. It was a long evolution of ideas and technology, rather than a single launch. All the more curious that so little has actually come to fruition.
When I joined Inrupt in 2018, I thought that what I should be doing was to engage and rally a community, write one or two good open source servers while helping write a more rigorous spec, and then go on to work on the really hard decentralization problems that was central to the message and also to the ph.d. on query I wrote a few years before.
What I thought “we” would be doing was to make a really good developer experience (DevX) around Solid, then a really good UX around actually useful stuff that could be done right then, and as adoption created traction, then build a business mostly around that Solid would have most of the logic on the client side.
Initially, it seemed like that was what we did, it was a lot of focus on DevX and building community, but the DevX got half baked, and the community was scared away as Inrupt chose to send myself and a co-worker on a Death March Project to save NSS, which was written when much of Solid was still in flux. Everyone who looked at NSS understood that it was in such a state they would neither develop on it, nor host their data with it.
At some point, it was decided on the main product being a server, which made zero sense to me, and it still doesn’t. I was at that time mostly writing specs (which I think should be done while also writing code and tests, but I wasn’t allowed to do that). That also caused all important discussions to be in-house, which I strongly resented and just couldn’t go along with. It is a horrible feeling, obviously, if I did my job well of writing specs, it would mean that it would be really easy to compete with Inrupt’s main product…
This sucked the oxygen completely out of the room, living no possibility for other candles to shine, and with that, made it very, very unclear what would be in it for you unless you were an Inrupt customer. That, I believe, is the strategic background for why it is so hard to understand what Solid is today.
There’s a radical difference in design philosophy that was not possible to discuss. Let me try to explain them in terms of two of my childhood toys, LEGO and Playmobil. LEGO consists of a range of plastic brick primitives common to pretty much everything you want to build. Playmobil are plastic toys that are prettier, but where the build is much more constrained to whatever they came up with. I’m a LEGO builder, Inrupt ended up with a misunderstood customer focus that mandates a Playmobil way of working. If you can’t be bothered with Knights or whatever, then you are out of luck, because Solid became what Inrupt was sleepwalking into because of clients.
I believe that Solid is unsuited for the highly-confidential enterprise use-cases. In fact, I believe HTTP is problematic, as HTTP reveals too much metadata before authen/z. But it is fine for all the use cases that people care about that is not their annual check of health records or whatever. People want to be freed from surveillance, they don’t want to be manipulated anymore, and other than that, they need to spend less time on boring stuff. That stuff could be done in a decentralized manner, we could have re-decentralized the Web. But we did nothing, zero, on that. People were instead talking about Data Vault and stuff, which Solid is terrible for.
The funny thing is that one of the first customers of Inrupt, a Swedish agency, called out exactly that in their report. Inrupt, completely oblivous to Swedish culture and that swedes are far to nice to be blunt about things that doesn’t work out too well, didn’t understand it. Between the lines, you could read that they were unimpressed, even though they badly wanted Solid to succeed for reasons unrelated to the present project. So, Inrupt’s customer were telling them that they had the wrong idea, but they persisted going down the same wrong path, all the whilst sucking up all the oxygen in the room so that nobody else could take a different direction.
I left Inrupt, but still had the opportunity to devote as much time as I wanted to Solid in another job for a Norwegian agency on Digital Public Goods, but eventually found it to be pretty pointless. I’ve been doing semwebby things for 25 years now, and to me Solid was a last-ditch effort to make it right. I suppose I came to the point where I felt I was just having a sunk-cost fallacy.
Now, for the details. Certainly, the more fine grained control is important. The term for it that has stuck is selective attribute disclosure, and it has been on the roadmap from day 1. There are several ways to do it. Possibly, there are even ways that are implemented in the proprietary server, I don’t know. My point is, and what I tried to set up in terms of community processes, was a way that different parties could come together and agree on where we focus our attention. So, if selective attribute disclosure was what people wanted, they’d nominate that, others would nominate other things, and we’d have a process to select which one to focus on, and get it done. I didn’t want to be in the position to make the selection and it would not be under the authority of a single entity to select the roadmap. My employer simply refused to participate in any conversation that they didn’t dominate.
So, there we are, things are barely moving forward, only enterprises invested in this has access to make sense of what’s going on, Solid has no practical use for anyone else, interop is a major problem, and as a community, it experiences very little growth.
There are still people outside of Inrupt who are trying to maintain a community around it. I wish them all the luck in the world. Solid could become a really good thing if it was an actual community building truly useful stuff together. It has to become much clearer to people how their concerns can be met.
Hi kjetilk,
Sounds like you have had a rough time, sorry to hear that!
Appreciate you taking time to share your experience and to try to answer some of my technical questions.
Yeah, part of it was rought. But the first year-and-a-half was really good, despite the focus on NSS. I still believed in the mission, and believed that we could do it, and it was a very stimulating environment to be in. But business realities caught up.
Hi Leigh, thank you for this great post. Like you, I have been interested in Solid since it first emerged. I like the idea of maintaining the canonical data for myself, rather than copying it to every organisation I deal with. If nothing else, it would mean I wouldn’t have to wonder who I had forgotten to tell when I moved house. I would know who I had shared my address with and if I didn’t want to hear from some company anymore I could just withdraw their access.
More recently I have started looking at the technical details and wondering whether it would make a suitable backend for an application I have been developing.
When you start reading the Solid Protocol spec you realise it mostly describes a general document store with a simple Restful API which builds on the Linked Data Platform. The lack of a query API surprised me too but for now I can work around this by carefully structuring my data and doing some querying in memory.
A bigger challenge has been getting my head around the Solid-OIDC specification which is still in draft status and it seems not consistently implemented by Pod providers currently. This is perhaps not surprising as getting this right is crucial to the success or failure of Solid – and this stuff is hard! As you mention, it seems you can only control access to a Pod as a whole and not to the things in it, which means if you want different access to different things you have to maintain multiple Pods. Ideally you want to have very granular control over access to the things in your Pod which requires something like ODRL to facilitate. I can see there have been some papers written about this, but it doesn’t seem to be part of the spec yet.
The bottom line is, as you suggest, that Solid just isn’t ready for general adoption yet. As a developer building apps, I don’t want to be worrying about implementing security protocols, I want libraries and tools that handle all of this for me. Maybe what Solid needs is more people like me willing to have a go at building apps and discovering the pitfalls. If we feed this back to, and work with, the Solid community, hopefully things will move forward. Given all of the issues that Sir Tim Berners-Lee mentions in that post for the Web’s 35th birthday, and which I agree with, I think this is too important to walk away from. Maybe somebody will come up with something else that archives the same thing in a different way, but until then I am going to continue to keep an eye on it.