In this article, you will learn how to create and visualise the Detect - Data Quality widget for a chosen location or set of locations.
Introduction
If you want to check your portfolio's data quality status, you might be interested in this widget.
The Detect - Data Quality widget displays all metrics related to data quality that are validated throughout Detect's executions.
The Detect - Data Quality widget uses the Table, the Stacked Columns and the Pie chart widget types.
Requirements
You need a location with a reference meter correctly assigned with data and Detect available in your account.
Configuration
- Go to Dashboards.
- Edit or create a Dashboard.
- Choose the source of data Detect - Data Quality.
- Choose the widget type. Either Table, Stacked columns or Pie chart widgets are available.
- Choose the reference meter type. Only the Main Supply is available in Detect, right now.
- Choose the locations you would like to display. You can choose Accepted locations and Discarded locations. Discarded locations are those that have been discarded throughout Detect's execution, mainly owing to data quality issues; Accepted locations are those that have Detect results' available. For the Pie Chart, all data must be chosen for a correct visualisation.
- Save
Available widget types
For this type of data, there are three Widget types available: the Pie Chart, the Stacked Columns and the Table widget.
Click here to learn more about the different widget types and also about general modifications that you can apply to a widget, such as changing the title.
Widget in detail
The Detect - Data Quality widget displays information on the data quality status of your portfolio or location. The data quality results in Detect are always displayed for a period of 1-year. You can find more information about Detect results in the following Detect article.
For all visualisations you can expect the following terminology:
- Discarded locations: All those locations that haven't generated results in Detect, mainly owing to data quality issues. The Pie chart and the Stacked columns visualisations show how many locations have been discarded. The Table visualisation shows more detail, i.e., why the locations were discarded.
- Accepted locations: All those locations that have generated results in Detect. These locations can, however, show some data quality warnings or errors. That's why the Accepted locations are divided into 3 groups: "Accepted and OK" (not even a warning); "Accepted with warnings" and "Accepted with errors" (errors being more severe than warnings). The Pie chart visualisation shows how many locations are in each group.
- Furthermore, Detect analyses the warnings and errors for 6 categories: External data, Metadata, Geolocation, Surface, Hourly consumption and Monthly consumption. The Stacked columns visualisation shows the distribution of data quality warnings and errors for each category. The Table visualisation shows a detailed reason for each warning and error within every category.
In the following sections, there is an example available for each visualisation.
Pie Chart - Detect Data Quality summary
The Pie chart displays a summary of the data quality status of your account:
This chart displays results as a percentage, but if you hover your mouse over each of the sections you will also get the absolute values. With the example here presented it would be interesting to:
- Check the Table visualisation to know the reason why almost 25% of locations were discarded.
- Check the Stacked columns visualisation to understand the warnings distribution amongst the categories.
Stacked Columns - Detect Data Quality warning analysis
The stacked columns represent the 2nd level of aggregation of all the errors. In the graph you will be able to see the total amount of locations that have been discarded plus the distribution of all the warnings into the mentioned categories:
All columns for accepted locations have the same height as each location is represented in each category once.
Table - Detect Data Quality detailed analysis
The table is the more detailed version of the Data Quality analysis. In the table, you can see how many locations have a specific warning / error and a detailed explanation of why some of your locations have been discarded.
An example for the discarded locations shows that for this account the main reason for locations being discarded is the lack of electrical reference devices, which prevents Detect from understanding which device represents the total buildings' consumption.
An example of the detailed warnings and errors for the account, where we can see in green the number of locations that do not have that warning, in yellow those that have mild warnings and in orange those which have a severe warning (error). In the first row, we can conclude that only 34 of our locations have no issues with hourly data gaps; that 59 have some data missing and that 1 has a severe lack of data in the middle months within the 12 months result period:
As the widget follows the reference meters, it can be used to aggregate the KPIs of multiple locations depending on the Hierarchy level selected, as it happens with the Widgets by Reference Device.
NOTE: Keep in mind that the widgets work only with well-configured locations, or upper hierarchy levels containing them, not with sub-locations.
And that's it, let's start reporting Detect outcomes through the Dashboards! 🚀