Data Cleanse

The Code Manager produces reporting on a range of data quality issues present in the Electricity Enquiry Service (EES) to Energy Suppliers and Distributor Parties each month. The reporting requirement transitioned from the Faster Switching Programme Data Working Group (DWG) upon its closing in July 2022 to the Retail Energy Code (REC).

Full details on reports produced can be found in the Data Cleanse Reports - Guidance Document in Category 3 Documents.

The Importance of Data Cleanse in the REC

It is an obligation under the REC (Schedule 24 – Switching Data Management, Section 3) for each Party to maintain the accuracy and completeness of all data items for which it is the Data Master.

This is further defined within the Retail Risk Register where Risk Drivers describe the scenarios where failure to follow defined processes due to a lack of data quality can lead to negative outcomes and poor consumer experiences.

Lack of data quality can impact both other Parties and consumers leading to negative outcomes and poor consumer experiences. We want to coordinate within the industry to jointly address these types of problems. Data Cleanse provides a means to do this.

Applicable Parties

The current set of data cleanse reports are intended for Electricity Suppliers, Distribution Networks and MAMs. The Data Cleanse Report Guidance document provides more detail on exactly which reports are applicable to which Party types.

Code Manager Reporting

Individual reports are produced by the Code Manager in .xlsb format and uploaded to each organisation's REC Portal site (Performance Assurance, Your Files) in the form of a .zip file. Parties should download and review these reports and the summary provided as part of the Performance Assurance Dashboard each month. The .xlsb reports list all Meter Points flagged as meeting the reporting criteria defined in the guidance documentation and are required to be cleansed.

The Code Manager also refreshes the ‘Data Cleanse Summary’ tab on the Performance Assurance Dashboard on the same day. This tab provides aggregate-level information on the performance of the industry and your Party on the population of each report.

The Code Manager aims to upload the .zip files for each organisation and update the ‘Data Cleanse Summary’ dashboard tab by the fifth working day of each month.

Exemption Process

Full guidance is provided in the Data Cleanse Reports - Guidance document

The Code Manager’s default position is that all MPANs in an Active Relationship need to be cleansed. However it is acknowledged that is not possible in all cases, hence the reason for an exemption process.

Monthly Exemption Requests

Parties can raise cleanse exemptions against any report by submitting an 1810 Data Cleanse Exemption Report (the full file specification is defined in the Performance Assurance Report Catalogue, tab ‘1810’) to the Code Manager, listing the:

Applicable MPANs.
Report you wish that MPAN to be claimed against (one report per entry).
Exemption Criteria you believe the exemption should be claimed against. The Exemption Criteria used must be an approved exemption criterion listed in the Data Cleanse – Exemption Scenario List document.

Files are required to be uploaded to the 'Your Files' area of your organisation’s Portal site by the end of the first working day of the month to be processed and reflected in the above Code Manager reporting.

It is not assumed that an exemption should be applied for consecutive months, so Parties are required to submit a complete listing each month.

Requests for New Exemption Scenarios

If you believe that you are unable to cleanse any items in the reporting but that there is no appropriate exemption scenario currently available, you can raise a new exemption scenario request for consideration. It is up to industry parties to present a compelling argument to justify why any new scenario should be added to the exemptions list.

Parties should complete the template forms available in Appendix C of the guidance document and submit them to the Code Manager through the REC Service Desk.

Proposed additions to the exemption list may be discussed at the REC Issues Group (RIG) to allow for input and consideration from other industry parties.

Data Cleanse Sprints

Data cleanse sprints are the preferred means to drive data cleanse activity across the market. Data cleanse sprints are designed to prioritise the investigation and resolution of selected reports that focus on specific issues that are either deemed high-risk or urgent to the industry and are expected to be resolved within the sprint timelines.

Given the shortened timeline to resolve issues and coordinated focus across Parties, a cleanse sprint reduces the likelihood of transferring issues through switching between two Parties.

It is mandatory for all applicable Parties to engage in the Data Cleanse sprints. Failure to do so may be subject to the issuing of Performance Assurance Techniques (PATs) and escalation to the Performance Assurance Board (PAB).

Sprint 1

The first data cleanse sprint ran from 01 June - 30 November 2023.

The reports selected for the sprint and their targets were as follows:

Report	Target(s)
Invalid Electricity MAP ID	10% of current pot value i.e. 404 total meter points (based on May 2023 reports).
Report 3A Unflagged	No MPANs over 12 months old. Less than 5% of MPANs (or 1 MPAN if small population) are over 6 months old. Less than 25% (or 1 MPAN if small population) over 3 months old.
Report AR3C	No MPANs over 12 months old. Less than 5% of MPANs (or 1 MPAN if small population) over 6 months old. Less than 25% of MPANs (or 1 MPAN if small population) over 3 months old.

Sprint 2

The second sprint ran from 01 March - 31 August 2024. The Code Manager will use the reports issued on 06 September 2024 to determine whether each relevant party is compliant with the below targets.

Report	Target
Invalid Electricity MAP ID	Reduction of pot value to zero.
Energised No Meter	Reduction of pot value to zero, excluding MPANs that have been energised within one month of the close of the sprint (based on Energisation EFD).

Sprint 3

The third sprint is running from 01 October 2024 - 31 March 2025. The Code Manager will use the reports issued on 07 April 2025 to determine whether each relevant party is compliant with the below targets.

Report	Target
New MTD items (EMSE ID, Meter Location, Number of Register Digits)*	No more than 10% of meter installations (measured using the date of meter installation recorded in EES) that are less than a month old should have an exception included in the Data Cleanse Report No more than 5% of meter installations (measured using the date of meter installation recorded in EES) that are more than a month old should have an exception included in the Data Cleanse Report
Invalid Topline (invalid combination due to SSC or PC)	0 for Suppliers 80% reduction for DNOs
CDSP Metering Parties reconciliation inconsistencies in Model, Manufacturer and MSN)	60% reduction in MPRNs with all 3 data items mismatched
Energised no Meter (>1 month old) and Invalid MAP ID	0 by mid-sprint

*Due to delays in obtaining the necessary data for the new MTD items we are proposing deferring the introduction of this report by two months to allow sufficient time for analysis to determine the target and produce required reports.

Sprint 4

The fourth sprint is running from 01 April 2025- 30 September 2025. The Code Manager will use the reports issued on 07 October 2025 to determine whether each relevant party is compliant with the below targets.

Report	Target
Energised No Meter	Reduction of pot value to zero, excluding MPANs that have been energised within one month of the close of sprint (based on Energisation EFD).
DNO Invalid Topline	0 items over 2 months old. No more than 20% of the report items over one month old.
New Meter Technical Details Items	No more than 10% of meter installations (measured using the date of meter installation recorded in EES) that are less than a month old should have an exception included in the Data Cleanse Report. No more than 5% of meter installations (measured using the date of meter installation recorded in EES) that are more than a month old should have an exception included in the Data Cleanse Report.
Invalid Gas MAP ID	All MPRNs where a potential MAP ID has been provided in the data cleanse report by a MAP. 75% of MPRNs where a potential MAP ID has not been provided.

Sprint 5

The fifth sprint is running from 01 November 2025 - 30 April 2026. The Code Manager will use the reports issued on 07 May 2026 to determine whether each relevant party is compliant with the below targets.

Report	Party responsible	Target
Energised No Meter	Supplier	Reduction of pot value to zero, excluding MPANs that have been energised within one month of the close of sprint (based on date first reported on the data cleanse report).
Invalid Topline	Supplier and DNO	0 items over 2 months old. No more than 20% of the report items over one month old.
New Meter Technical Details Items	Supplier	No more than 10% of meter installations (measured using the date of meter installation recorded in EES) that are less than a month old should have an exception included in the Data Cleanse Report. No more than 5% of meter installations (measured using the date of meter installation recorded in EES) that are more than a month old should have an exception included in the Data Cleanse Report. No meter points with a date of meter installation over six months old should have an ESME ID or Meter Location not populated (however meter location ‘Unknown’ is acceptable).
Related MPAN with inconsistent Domestic Premises Indicator (DPI)	Supplier	Reduction to zero of exceptions over one month old
Unmeasured MPAN with Domestic DPI	Supplier	Reduction to zero of exceptions over one month old
Gas CDSP Reconciliation	MAM	60% reduction in the number of MPRNs with a difference in meter manufacturer, model and serial number
MEM-EES Reconciliation Differences	MOA	Zero items over two months old. No more than 20% of the report items over one month old.

Sprint 6

The sixth sprint is running from 01 June 2026 - 30 October 2026. The Code Manager will use the reports issued on 06 Novemeber 2026 to determine whether each relevant party is compliant with the below targets.

Report	Party responsible	Target
Energised No Meter	Supplier	Reduction of pot value to zero, excluding MPANs that have been energised within one month of the close of sprint (based on date first reported on the data cleanse report).
Invalid Topline	Supplier and DNO	0 items over 2 months old. No more than 20% of the report items over one month old.
New Meter Technical Details Items	Supplier	No more than 5% of meter installations (measured using the date of meter installation recorded in EES) that are less than a month old should have an exception included in the Data Cleanse Report. No more than 2.5% of meter installations (measured using the date of meter installation recorded in EES) that are between 1 and 6 months old should have an exception included in the Data Cleanse Report.
Disconnected Active MPANs	Supplier	No Meter points included in the report that are over 30 working days old (measured based on the last RMP Status effective from date).

Party Responsibilities

It is the responsibility of every participating Party to ensure that:

The required persons have appropriate REC Portal permissions to be able to access the ‘Your Files’ area of the Portal. Permission changes should be completed by the Contract Manager or Master Admin User (MAU) of each organisation. The REC Service Desk can assist with any issues.
Any exemptions to be claimed are submitted by the end of the first working day of the month in a format compliant with the specification defined in the Performance Assurance Report Catalogue (PARC ref. 1810) via the REC Portal ‘Your Files’ area.
It engages in all applicable Data Cleanse Sprints, making every effort possible to achieve the targets defined by PAB or to proactively engage with the Code Manager in the case where there are any extenuating circumstances.

REC Issues Group (RIG)

Main page: REC Issues Group (RIG)

Data Cleanse is discussed as a regular agenda item at the monthly RIG meetings. The meetings are used as an opportunity for the Code Manager and RIG members to engage and provide updates on cleanse performance and raise any issues for discussion.

FAQs and Common Issues

I submitted the 1810 exemption file, but my exemptions don't appear to have been counted. Why has this happened?

There could be a few possible reasons for this:

The submission was made late. If the exemption listing was not received by the end of the first working day of the month as required, it risks not being processed in time for the reports to be issued.
The file was submitted via the ‘Other’ dropdown option. The ‘Other’ dropdown option is intended for ad-hoc submissions by Parties. Files submitted under ‘Other’ are not automatically processed or tracked. More on this in the File Validation knowledge article.
The MPAN could not be mapped to an MPAN in the data cleanse report. The most common cause of this is MPANs in the 1810 file being submitted in scientific format.
The Exemption Scenario listed is not included in the approved list.

I've not received all of the Data Cleanse reports defined in the guidance document. Why?

The individual .xlsb reports are only produced if our analysis has flagged that there are items to cleanse in that report. If you haven’t received a particular report, it is likely that you have no items to cleanse.

You can double-check this by going to the ‘Data Cleanse Summary’ tab on the dashboard and filtering for the report and your organisation to view the aggregate results.

Is there a difference between Data Cleansing activities requested by the Programme and those requested by the REC Code Manager, regarding the population of ESME IDs, Meter Location and Number of Displayed Register Digits?

Yes. The MHHS Programme asked participants to complete a back-population of ESME IDs, Meter Location and Number of Displayed Register Digits, as these data items would not have been populated before June of last year when they became mandatory.

The activity that the Code Manager is requesting of Suppliers focuses on ensuring these data items are populated for all meters installed following this population activity. Further information can be found in a document provided by the Code Manager and the MHHS Programme - MHHS-DEL3383 REC and MHHS Programme Data Cleanse Objectives.

Menu

Contents