Operational KPIs and team performance

Operational Key Performance Indicators

The Operational Health Matrix uses a system of operational indicators designed for the objective assessment of the effectiveness of the processes of detecting, analyzing, resolving, and verifying operational problems.

Unlike traditional reporting, the KPIs serve not only to assess departments, but also to continuously improve the platform itself.

The system calculates all indicators automatically based on the actual lifecycle history of Operational Issues.

KPI Architecture

All indicators are divided into four independent groups.

flowchart TD
  ALL["All indicators"] --> OPS["Operational KPIs"]
  ALL --> ASSET["Asset KPIs"]
  ALL --> TEAM["Team KPIs"]
  ALL --> PLAT["Platform KPIs"]

This separation prevents mixing equipment condition, employee performance, and the quality of analytical algorithms.

Mean Time To Detect (MTTD)

MTTD characterizes the effectiveness of the analytics subsystem.

Definition:

The average time between the actual occurrence of a problem and its automatic detection by the platform.

It is used to assess:

• detector quality;
• telemetry completeness;
• processing delays;
• analytics effectiveness.

The lower the value, the faster the system identifies the problem.

Mean Time To Acknowledge (MTTA)

Shows the response speed of the operations service.

Definition:

The average time between the creation of an Operational Issue and the acknowledgement of its acceptance by the responsible department.

It makes it possible to assess:

• response speed;
• organizational discipline;
• the effectiveness of the dispatch service.

Mean Time To Resolve (MTTR)

The primary operational indicator.

Definition:

The average time between the creation of an Operational Issue and the confirmed elimination of its cause.

Only completed Issues are used in the calculation.

Separate targets are allowed for different problem classes.

Mean Time To Verify (MTTV)

MTTV assesses the effectiveness of the verification process.

The indicator characterizes:

the time between the completion of work and the confirmation of the restoration of the equipment’s normal state.

A high MTTV usually indicates:

• insufficient telemetry;
• a complex verification procedure;
• a shortage of resources;
• problems in the organization of operations.

SLA Compliance

Shows the share of Operational Issues completed without violating SLA requirements.

It is calculated separately for:

• Response;
• Acknowledgement;
• Resolution;
• Verification;
• Closure.

Separate statistics are maintained:

• by region;
• by department;
• by canon;
• by equipment type.

Mean Issue Age

Shows the average age of active Operational Issues.

It is used to assess accumulated operational debt.

Additionally, the distribution is calculated:

• 0–1 day;
• 1–3 days;
• 3–7 days;
• 7–30 days;
• more than 30 days.

Reopen Rate

Characterizes the quality of problem remediation.

Definition:

The share of Operational Issues reopened after closure.

A high value indicates:

• incomplete elimination of the cause;
• erroneous diagnostics;
• insufficient verification;
• chronic equipment defects.

Repeat Issue Rate

Shows the recurrence of identical canons for a single asset.

It is used in the analysis of:

• chronic failures;
• the need for equipment modernization;
• maintenance quality.

Issue Escape Rate

Shows the share of problems that were not detected automatically and were registered by users.

Growth in this indicator requires an analysis of:

• detector coverage;
• telemetry quality;
• analytics completeness.

Verification Success Rate

Determines the share of work successfully confirmed by the system after completion.

A high value indicates:

• correct diagnostics;
• proper execution of the work;
• high reliability of the Operational Workflow.

Operational Backlog

The Backlog shows the volume of unfinished work.

Additionally, the system analyzes:

• the total volume;
• the critical backlog;
• the overdue backlog;
• the backlog by department;
• the backlog by region.

KPI Dashboard

The main analytics dashboard contains:

• MTTD;
• MTTA;
• MTTR;
• MTTV;
• SLA Compliance;
• Reopen Rate;
• Repeat Rate;
• Detector Accuracy;
• False Positive Rate;
• Operational Backlog;
• Mean Issue Age;
• Health Trend.

All indicators are available over time and at any level of the organizational structure.

Team Performance Indicators

Equipment operational indicators are not used to evaluate employees.

A separate group of Team KPIs is used to assess the effectiveness of departments.

This prevents a situation where a deterioration in equipment condition automatically worsens the assessment of staff performance.

Workload

Shows the actual workload of engineers.

It takes into account:

• the number of active Issues;
• their criticality;
• complexity;
• the number of assets;
• the volume of the Verification Queue.

Capacity Utilization

Characterizes the degree of utilization of the department’s available resources.

It is used for:

• load balancing;
• staff planning;
• assessing the need to redistribute work.

First Time Resolution

Shows the share of problems resolved without reopening.

A high value indicates high quality of diagnostics and repair.

Assignment Balance

Assesses the evenness of the distribution of Operational Issues among engineers.

It is used in automatic assignment.

Average Verification Time

Shows the average time to confirm the results of completed work.

It makes it possible to identify bottlenecks in the operations process.

Team Dashboard

The department manager receives:

• the current workload;
• a workload forecast;
• SLA;
• backlog;
• efficiency;
• reopenings;
• team productivity;
• the distribution of competences.