The data ecosystem for non-domestic energy consumption data in the UK

Since joining Energy Sparks I’ve started investing time in trying to understanding the UK energy system and, in particular, the parts of its data infrastructure and broader ecosystem that we work with.

It’s a big, complex system so lots to wrap my head around. I looked at one part of that data ecosystem in a previous post about smart meters.

In this post I’m going to summarise my understanding of another section of that data ecosystem. Specifically, around non-domestic energy consumption data.

Non-domestic means small, medium and large businesses, schools, public sector bodies.

And consumption data refers to the half-hourly readings that indicate how much energy has been consumed via the metered supply.

Non-domestic meters

Most non-domestic customers will have AMR meters. Smart meters aren’t suitable for larger supplies, so can’t be fitted everywhere. And, unlikely the domestic market where there’s a push to get them installed, they are not mandated more broadly.

So non-domestic users will still be using older infrastructure that relies on reporting data via radio, rather than the more centralised smart meter network.

How data moves through the system

There are a number of different organisations that are involved in collecting and processing the data that comes from these meters.

Supplier agents

The first steps in the chain are:

• Meter operators – who are responsible for installing and managing the meters. A meter operator is required to manage any meter once installed. If there are any faults with the meter, then the meter operator will have the responsibility of fixing it
• Data collectors – the data collector is responsible for actually reading the meters. That used to involve sending people out to take readings, but now involves running the infrastructure to carry out the remote readings
• Data aggregators – is responsible for summarising data before it is passed to the energy supplier and other parts of the energy network, for reporting, payments, etc.

These three separate roles are spelt out in the “Balancing and Settlement Code”. Legally, every meter must have a meter operator and there must be both a data collector and a data aggregator.

A business can choose to contract with individual companies to do all of these tasks. But in general its left up to the supplier to appoint them. Which is why they’re usually referred to as “supplier agents“.

Non-domestic energy customers will have standing charges on their bills that cover the costs of the meter operator and DC/DA functions.

In practice the same company might carry out some or all of these roles. This means that there is at least one up to three systems through which the data might pass before it gets to the energy supplier.

Suppliers and brokers

Energy suppliers are same as for the domestic market. You pay them for the energy you use.

Businesses often uses a third-party to help them choose their energy supplier and manage any contracts with meter operators, etc if they have chosen to appoint them directly.

These brokers offers a variety of value-added services.

There are some some organisations, like West Mercia Energy, who work specifically with the public sector to support them in procuring their energy supplies. They are similar to brokers.

Third-parties

The final type of organisation in the ecosystem, are third-party services like Energy Sparks who offer value-added services over the consumption data. E.g. to help schools and businesses to reduce their ills.

There are also energy management services, like Systems Link, that allow public sector organisations and businesses to access and analyse their data.

Getting access to data required

Getting access to data

As a third-party, in order to get access to data you might need to negotiate access via:

• a supplier agent, e.g. the organisation offering meter operator and DC/DA services
• the energy supplier
• the energy broker or procurement organisation
• another third-party platform like Systems Link

Access is consent based. Every organisation will have its own preferred approach for proving that consent has been granted. Commonly this involves a signed letter of intent and provision of an ongoing list of meter identifiers, rather than simply a blanket permission.

There is no central database of non-domestic consumption data. So third-parties need to integrate with a wide range of different types of systems and organisations, some of whom may be competing to offer the same or similar services over that data.

While, this is changing, access to data is not guaranteed. Access to machine-readable data is usually a chargeable extra, e.g. a price per meter per year.

So schools and businesses looking to reduce costs often opt-out of these extras.

There are no formal standards for that data. Although having seen more than 24 different variations it’s all pretty similar. It’s just different enough to be awkward to deal with.

Data is generally made available via CSV files attached to emails. These emails might be automated but in some cases the data is only available on request.

Some observations

From the perspective of a third-party perspective there are multiple overheads here:

• identifying a suitable route to access data, depending on the specific network of businesses responsible for providing the meters and energy supplier
• convincing users as to the potential benefits for opting into updated contracts and service plans that have data access costs attached to them, so that data can be obtained even for a trial
• navigating a variety of different consent and permissioning processes to onboard users
• integrating with a variety of different systems in order to receive and process data, and updating that as systems evolve
• getting access to historical data for analysis, which might have extra charges attached or just not be available due to a change in supply
• managing ongoing access to data when changes in supplier can interrupt data access
• time lag in receive data due to it having to move through different systems, possibly with different reporting frequencies. E.g. some suppliers may only supply data on a weekly basis
• …etc, etc

If non-domestic users were part of the smart meter network then at least some of these overheads would be removed. But as noted above, this is not currently mandatory or possible for some non-domestic customers.