Anomaly detection in process data: Z-score and AI in practice

Picture the situation: the return water temperature on a production line has risen half a degree over the past hour. Not one fixed limit has tripped. The next shift starts in two hours — and the fault only surfaces at the morning meeting, by which time the temperature has climbed two degrees and the pump has run overloaded for three hours.

This is exactly the case that anomaly detection would have caught within the first half hour. The Z-score would have crossed the warning threshold, local AI would have put the observation in context, and maintenance would have been notified during the same shift. No cloud was needed — no data leaving the site, no dependency on an external service.

In this article I go through how anomaly detection is implemented in practical terms in an industrial process: why fixed limits fail, how the Z-score works, where it falls short — and how local AI completes the picture.

Why is anomaly detection industry's most critical capability in 2026?

In 2026 most Finnish industrial plants collect process data through an OPC UA interface, but actual anomaly detection is still in its infancy. The typical situation looks like this: the automation system has fixed high and low alarm limits — and beyond those, nothing.

A fixed limit is necessary but not sufficient. It catches blatant deviations such as a failed sensor or a pressure spike. It does not catch a slowly developing fault that progresses inside the normal range — but in the wrong direction and in the wrong context.

In an industrial process a deviation does not arise in a vacuum. It arises in a context:

• Load level: the same temperature is normal at full load but abnormal at idle.
• Time of day: night-time behaviour differs from daytime behaviour.
• Weather conditions: outdoor temperature, humidity and sunshine matter especially in power generation.
• Process state: start-up, steady running, shutdown and a maintenance period each produce a different normal.

When anomaly detection takes these context variables into account, it finds deviations that fixed limits never see — and it finds them in time, not weeks later in a weekly report.

Six reasons why fixed limits fail at anomaly detection

In many plants deviation detection rests solely on the fixed alarms of the SCADA system. Here is why that is not enough.

1. Context blindness

A fixed limit does not know whether the plant is at full or half load, whether production is running normally or the process is restarting. The same value can be entirely normal in one situation and a critical deviation in another.

2. Slowly developing faults stay hidden

Bearing wear, a leaking valve seal or a fouling heat exchanger develop over weeks or months. The values stay inside the limits the whole time — but the trend already tells a clear story, if you know how to read it.

3. Too many false alarms

Fixed limits are often set so loose that they do not generate too many alarms. The result is an alarm storm — so many messages that operators start to ignore them. The important deviation drowns in the noise.

4. The combined effect of several signals goes unseen

Pressure may be normal, temperature normal and flow normal — yet their combination says clearly that something in the process is wrong. Watching individual limits will never reveal this.

5. No course of action after the observation

Even when SCADA raises an alarm, it does not say whether this is serious, what it relates to, or who needs to do what. Without prioritised information in context, an alarm becomes extra work rather than a trigger for action.

6. History and root-cause information are missing

The alarm log records when a limit was exceeded, but offers no surrounding process data, no trend leading up to the deviation and no post-analysis. Root-cause investigation always starts from a blank sheet.

Anomaly detection with the Z-score: formula, logic and practice

The Z-score is a statistical tool that measures how many standard deviations a single observation sits above or below the normal level. The formula is simple:

Z = (value − mean) / standard deviation

If the Z-score is below 2, the value is within normal variation. If it exceeds 3, the value is already statistically rare — 99.7% of values stay within that range in normally distributed data. In industrial process data the thresholds commonly used are 2.5 (warning) and 3.5 (critical).

The formula alone does not make the Z-score useful. A practical implementation needs four building blocks:

1. A context-specific baseline

Calculate the mean and standard deviation separately for each process state: load class, time of day, season, start-up versus steady running. A single global baseline produces too many false positives.

2. A long enough and clean reference period

The baseline is typically calculated from a 30–90 day reference period, with known fault and maintenance periods removed. If the baseline contains deviations, the model learns them as normal.

3. A requirement for consecutive points

A single Z-score exceedance can be measurement noise. Require 3–5 consecutive exceedances before raising an alarm. This removes most false alarms without losing detection sensitivity.

4. Ownership and a path to action

A deviation without a course of action is worthless. Attach to the alarm: notification to the responsible role, reference data for root-cause analysis and a way to record the actions taken. Only then does the Z-score turn into operational benefit.

70–90%

fewer false alarms when you move from fixed limits to a context-specific Z-score with a consecutive-point requirement. Operators can trust the alarms again.

Where the Z-score is not enough — and why local AI completes the model

The Z-score is an excellent first layer. It is fast, explainable and easy to audit. But it looks at each measurement point in isolation. In a complex process a deviation often arises from the combined effect of several signals — and this is precisely where local AI adds value.

The practical model works on three levels:

• Level 1 — Z-score: fast, per-tag deviation detection. Response time in seconds.
• Level 2 — correlation analysis: AI identifies which other measurement points moved at the same time and what cause-and-effect relationships have been observed in the history.
• Level 3 — prioritisation: AI assesses whether the deviation is operationally significant or transient, and suggests possible root causes.

The essential point is that the AI runs locally. Process data in industry is often sensitive: it can reveal production volumes, recipes, capacity utilisation or energy efficiency figures. That is why the analysis has to happen on the plant's own network — not in a cloud service.

DataPortia™ uses Ollama models directly on the plant's own server. The data never leaves the plant network — and language-model-based contextualisation is still available, explaining the deviation to the operator in natural language.

Practical example: a district heating plant deviation that fixed limits missed

Consider a plant producing district heat. The system has three boilers, heat exchangers on the network return side and several pump and valve groups.

District heating plant: from fixed limits to context-specific anomaly detection

Practical example

3.5 h

earlier detection. During the time the plant ran overloaded, extra energy costs and wear built up — both avoided by context-specific anomaly detection.

How to build anomaly detection without a heavy data science project

The biggest misconception about anomaly detection is that it requires a multi-year data science project, a large budget or a cloud-based AI service. In reality a practical model can be built in stages without separate consultants.

Step 1: Pick 5–10 critical measurement points

Start with the points carrying the highest financial risk, or where a deviation can damage equipment. More is not better — a focused model beats a broad but shallow one.

Step 2: Build a context-specific baseline

Use 60–90 days of historical data. Group it by process state: load class, time of day, season. Check that the baseline period contains no known fault periods or maintenance windows.

Step 3: Set the thresholds and the consecutive-point requirement

Start with a warning threshold of Z > 2.5 and a critical threshold of Z > 3.5. Require 3–5 consecutive exceedances before an alarm. Watch for a week, measure the number of false alarms and adjust if needed.

Step 4: Add local AI to put the observations in context

Feed the anomaly observations to a local language model that explains the context of the deviation: which other measurement points moved, what that has historically meant and what action is recommended.

Step 5: Expand in stages

Once the first 5–10 measurement points work reliably, extend to the next group of the most critical points. Avoid expanding too fast — every new point needs a proper baseline.

When is anomaly detection worth it — and when does it not yet help?

Summary: anomaly detection moves from theory to operational practice

In 2026 anomaly detection no longer requires a massive data science project. It requires the right structure: a context-specific Z-score, a consecutive-point requirement and local AI that puts the observation in context — plus a course of action that turns the observation into a corrective measure.

The key lessons:

• Fixed limits are not enough — they see only blatant deviations, not slowly developing faults or context-dependent situations.
• The Z-score is the best starting point — explainable, fast and auditable.
• The context-specific baseline is decisive — a single global limit produces too much alarm noise.
• Local AI completes the picture — it puts the combined effect of several signals in context without a cloud dependency.
• Start small, expand in stages — 5–10 critical measurement points are enough to begin with.

When data acquisition, visualisation, anomaly detection and AI analysis sit in the same package, the model turns from theory into a daily way of working without an IT project.

Request a 30-day trial Get in touch