Getting started with Optimise

This section gives you an overview of what Optimise is and how it helps you detect anomalies in your consumption data

Contents:

• Requirements
• How are anomalies calculated?
• How to use Optimise? 
  • 1. Enable Optimise datapoints
  • 2. Your list of detected anomalies, every morning
  • 3. Go to the detail
  • 4. Advanced configuration: Focus only in anomalies relevant for you
  • 5. Advanced configuration: Tune your Artificial Intelligence (AI) models
  • 6. Add comments and tags to an anomaly
  • 7. Working with Optimise - Task management
  • 8. Anomalies impact level
• Practical case. How did we validate the results when defining AI models?

Requirements

• Have hourly data for the parameters you are willing to find anomalies: Active Energy, Gas Energy or Volume, Thermal Energy (cooling or heating) and/or Water Volume.
• At least 6 months of historical data, 12 months would be ideal
• Zip Code configured in your locations, so that the system can retrieve weather and local holidays automatically
• [Optional] Cost electrical tariffs properly configured, in case you want to quantify and filter anomalies by cost threshold

How are the anomalies calculated?

In the animation below you can see the 3 layers used to calculate an anomaly. First the real consumption, second the AI's model estimation (baseline) and third the anomalies detected.

From the moment you activate a datapoint, the AI carries out the following in order to detect anomalies:

1. Gather training data
   We gather historical readings of up to 1 year for the datapoint being analysed, as well as weather and holiday data for this period (if you have selected them as model variables). These are the training data for the AI model, serving as a reference for the typical consumption behavior of that datapoint.
2. Train the AI model
   We analyse the training data in conjunction with their calendar features (hour, day of week, month, day of year, etc.) and optionally enrich them with holiday and weather information in order to create a statistical model that estimates the expected range of consumption values. In a pre-processing step, we adjust the importance of each value in the training dataset to reduce the effect of past anomalies on the trained AI model. Every month the AI models will be automatically re-trained.
3. Detect regions of anomalous behavior
   Each morning, we obtain yesterday's readings for a given datapoint and compare them to the AI model's estimation in order to pinpoint any differences between the actual consumption and the expected range of consumption. Not all differences are classified as anomalies! Since every device will have some variability in its consumption from day to day, we utilise an additional statistical model to assess the probability of a deviation from the expected range being an actual anomaly.
4. Calculate the energy impact & cost of each anomaly
   Once we have detected an anomaly, we use an AI-generated baseline trained on the historical data to estimate the expected consumption in order to quantify the anomaly's impact in terms of active energy (in kWh) and energy cost (based on the cost tariff configuration of your account). The "impact" is the difference between the real consumption and the expected consumption. 

How to use Optimise? 

1. Enable Optimise feature in your account

Optimise is included in your license if you have an Advaced or Ultimate license. There is no limit to the number of Optimise models that you can configure.

To enable Optimise in your account, simply follow these steps:

• Go to the left hand menu and select Optimise
• On the Optimise landing page, click on Install Optimise button. Optimise will be installed. It may take up to 30-60s to install. In case it doesn't respond, refresh the page or contact with your support provider.

Now that the Optimise feature is enabled in your account, you should enable the Optimise datapoints (AI models) to start helping us detecting anomalies. To do so:

• Once Optimise is installed, click on Devices Pending Configuration button.
• You will get a list of the available devices in your account that do not have any Optimise model yet (right now Optimise is compatible with active energy devices only).
• Select one or multiple devices to get configured (Tip: use the search box to filter the devices' list or the multi-select checkbox to configure all at once)
• Click on Enable button.

Once you've selected the datapoints, click on "Enable" green button on the top of the list.

Click on "Apply to selected models" in the pop-up if you want to run on the default settings. You are done! AI models will start to be trained and the first anomalies will be calculated for the last 24h. 

Note: The model status may take a few minutes to appear as green ticked, until the request is processed internally. 

Now that the AI models are being trained, every 24h Optimise will analyse all the data for you and show the relevant anomalies discovered with the default configuration. In case you want to modify the AI models, check this section.

2. Your list of detected anomalies, every morning

Every morning, Optimise will show you the new discovered anomalies:

You can use the date-picker to filter by specific date range (for example last weekend or so far this month). 

You can use the Grouping option to group anomalies according to any criteria you find relevant:

Anomalies will always be grouped by parameter by default, since you cannot put together water consumption and electricity consumption and calculate its total impact.

You can expand and collapse any group with the arrows that are found on the left:

You can use the Filtering option to filter the anomalies you see on the screen:

Remember to always click on Apply filters to implement the changes.

If you are in the settings section, you can move back to the anomalies list by pressing the left arrow button close to the Optimise models title.

3. Go to the detail

To view what happened, simply click to any anomaly available on your list. The anomaly detail will appear and you will see what was wrong. Remember that the baseline is already adjusted by weather and holidays (if you chose so).

4. Advanced configuration: Focus only in anomalies relevant for you

Cost threshold

The system can detect very little anomalies or very huge ones. We give you the freedom to choose which ones you want to receive. To do so, once you are enabling / editing the AI models, choose a relevant cost threshold. You can use different thresholds for different models.

For example, you can set ±100€ per anomaly in big buildings or ±10€ in retail shops. Remember this is the cost impact per anomaly. If a 10€ anomaly is repeated every day, it will ultimately be a 3.500€/year, so do not underestimate the impact of small anomalies.

Note: Anomalies will be only filtered if they have a tariff configured (so the system can calculate the cost). To learn how to configure your tariffs, visit this article

The cost threshold is configured for both positive and negative anomalies (above and below the baseline). It is expressed in generic "Currency units", meaning €, ₤, $ or any currency you've configured in your account.    

Model sensitivity

Another way to focus on relevant anomalies is by adjusting the model sensitivity. There are five sensitivity levels:

• Lowest: Will only detect anomalies that feature extreme deviation from the typical consumption.
• Low: Will detect anomalies that feature high deviation from the typical consumption.
• Medium: Standard sensitivity.
• High: Will detect small deviations from typical consumption as anomalies.
• Highest: Will detect minimal deviations from typical consumption as anomalies.

Specifying the model sensitivity is an important way of tailouring the model's decisions to your data and desired level of oversight. If your data is highly predictable and you want to be notified of every single incident of abnormal consumption, you should set the sensitivity to high.

On the other hand, if your data is unreliable, influenced by multiple hidden factors and/or has insufficient historical context that serve as a reference for normal consumption (less than 6 months), you might want to choose low sensitivity to only detect the most important anomalies.

5. Advanced configuration: Tune your Artificial Intelligence (AI) models

AI model variables

You can select or unselect which variables you want to take part in the AI model. An AI model is a "black-boxed brain" that gets trained with historical data from the datapoint itself plus a set of external variables that can explain its behaviour. The most typical ones in the energy efficiency field are weather-based variables such as heating & cooling degree days and local holidays. 

In case you are willing to detect anomalies on commercial sectors (retail stores, supermarkets, offices, education, hospitality etc.) then keep these options checked. 

In case you are working in the industry sector, then your factory consumption won't correlate with the weather but maybe with holidays, so you might want to uncheck the Heating & Cooling degree days option.

In any case, don't worry. If you've selected a variable that doesn't correlate, the AI model won't use it to calculate anomalies. Easy!

Exclude training period (e.g. due to COVID-19 lockdown)

This option is very useful to tell the AI model to exclude a specific date range. By default, the AI model is trained with the last 12 months of data. However, if your building was closed for a certain period of time or it had an abnormal consumption (for example for the building's renovation/refurbishment), this data shouldn't be included in the training. You can choose this period of time in the "Exclude training period" section. 

Force to recalculate anomalies from the past

Optimise is calculating anomalies every day, at morning, weekends included. However, the first time you are enabling datapoints you might be excited to detect anomalies from last week, last month or even last quarter. No problem. You can do so by selecting a date from where you want to get the anomalies calculated.

Note: As the default training period is 12 months, we don't recommend to calculate anomalies earlier than 3 months from the model activation day, as there would be too much overlap between the training data and the anomaly results.

6. Add comments and tags to an anomaly

You can write and delete comments on an anomaly. To add a comment, simply open an anomaly detail and scroll down to the Activity section. Here you will see a list of existing comments (if any) and you can post your own.

To tag an anomaly, simply write your tags in the comment. For example, if the anomaly regards to a HVAC disfunction during the weekend because the BMS setpoint was wrongly configured, you can simply write "#HVAC #weekend #BMS" on a comment and afterwards you will be able to search using this keywords.

7. Working with Optimise - Task management

After Optimise helps you detecting anomalies, here is how you can work in a daily basis with the provided information (adjust the process to your own organisation):

1. Focus on relevant anomalies: Use the hierarchy selector, add filters on the search box and sort the anomalies (e.g. by cost impact) to detect the ones that are relevant to you.
2. Open the anomaly detail, look at the chart:
   • If this seems to be an anomaly that requires an action from your side, you can change the status from New to Ongoing or Pending action. Write a comment on the anomaly describing what you've found 
   • If the required action has been performed, you can change the status to Solved.
   • If the identified anomaly does not require any action from your side, you can always change the status to Discarded
3. You colleagues can do the same, accessing Optimise and adding comments on the anomalies, moving the statuses and fixing them.

8. Anomalies impact level

The impact level indicates how significant an anomaly is compared to the expected behaviour. It is designed to reflect how abnormal a deviation is, not just how large it is.

The impact is calculated by:

• Measuring deviation from the expected behaviour
• Normalising the deviation using typical variability
• Focusing on the most extreme moments
• Classifying the results into LOW, MEDIUM, or HIGH impact levels

1. Normalising the deviation

For each point in an anomaly, Optimise measures how much actual consumption deviates from the expected range (baseline ± bounds). This distance is normalised by the typical variability of the device, so results are comparable across different assets.

In simple terms, this section analyses how many “expected ranges” the value deviates from normal behaviour.

2. Focusing on the most critical moments

Instead of considering the full duration, Optimise looks at the 1 to 3 most extreme points of the anomaly and averages them.

This ensures that:

• Short but intense anomalies are captured
• Long but mild deviations are not overstated

3. Converting the results obtained into the different impact levels

For anomalies that last multiple hours, considering the results obtained in previous points 1 and 2, their impact level is defined as follows:

• LOW → small deviation (severity score <1.5)
• MEDIUM → moderate deviation (severity score between 1.5 and 3)
• HIGH → strong deviation (severity score >3)

For anomalies lasting one hour, the impact level High is not considered.

Practical case. How did we validate the results when defining AI models?

In order to adjust the various steps described above, as well as to validate their performance, we carried out two parallel evaluation processes - using simulated consumption data as well as real consumption data. 

Evaluation based on simulated consumption data

With the help of our energy efficiency experts, we developed a building energy simulation software that generates realistic consumption curves for a simulated building given its characteristics (consumption predictability, climate conditions, type of heating, surface area, standby level etc.).

We then generated an exhaustive collection of simulated consumption curves featuring all permutations of simulation parameters, and then artificially introduced different types of anomalies to the simulated data such as unexpected daily peaks, accidental weekend use, nightly consumption, alterations to the operating schedule, etcetera.

Finally, we trained & evaluated our AI models on the test data in order to ensure that:

• Anomalies are correctly identified and differentiated from normal variability in the consumption data
• The impact of detected anomalies is correctly quantified
• Anomalies present in the training dataset do not have a significant negative impact on the AI model performance

Evaluation based on real consumption data

Having validated the basic functionality of the AI models under controlled conditions on simulated data, we proceeded to manually compile an evaluation dataset based on real consumption data of anonymized buildings from the platform.

We manually annotated hundreds of anomalies for different building types as well as anomaly types, and used these annotations as a test dataset for our AI models. We finally gave the results of the automatic Anomaly Detection to our in-house team of energy efficiency experts so they could validate that the AI models are capable of dealing with the unpredictability of real-world consumption data.