Conformance Checking in Process Mining: Quantify Where Real Work Deviates From the Approved Path

Conformance Checking in Process Mining: Quantify Where Real Work Deviates From the Approved Path

Every approved workflow looks clean on paper. The event log tells a different story, and that gap is where cost, risk, and rework hide. Conformance checking in process mining conformance checking compares what the model says should happen with what the recorded data shows actually happened.

This guide is written for analysts who already have access to a transactional system and an approved process document. You will see what conformance checking reveals first, how the comparison runs step by step, a labeled example separating approved variants from recording artifacts and genuine control failures, and the criteria that decide when a deviation deserves an investigation.

Key Takeaways

  • Compare recorded cases with an approved process model.
  • Validate case IDs and timestamps before interpreting results.
  • Separate approved exceptions, data artifacts, and control gaps.
  • Prioritize findings by impact and control effectiveness.
  • Fix the model, data, or process according to the cause.

Start with the question that decides how your whole analysis will be read.

How Do You Separate Real Process Deviations From Approved Exceptions?

How Do You Separate Real Process Deviations From Approved Exceptions?

You separate them by defining the approved path before you look at the data. Conformance checking compares observed process behavior in event logs with expected behavior represented by a process model, and token replay and alignment techniques expose where the two diverge. Have the process owner approve the model and documented exceptions before interpreting results.

An approved exception is a documented alternate route with a defined trigger, an authorizer, and a recorded justification. A flagged deviation needs review against policy and source records before it can be called a control failure.

So the first thing conformance checking reveals is not a list of broken steps. It reveals how little agreement exists about the official process.

  • Missing activities: a required approval, inspection, or verification never appears in the trace.
  • Unexpected activities: steps execute that the model never authorized, such as a manual override.
  • Order violations: work starts before the gate that was supposed to release it.
  • Repetition: the same activity fires three or four times, which may indicate rework, duplicate recording, or a legitimate loop.
  • Timing gaps: recorded intervals appear shorter than an applicable review standard, requiring validation of actual work and timestamps.

Compare the model with current policy and operating instructions across teams before treating it as authoritative.

Once you accept that the reference model is the anchor, the comparison itself becomes mechanical.

Setting Up Process Mining Conformance Checking Against an Approved Workflow

Setting Up Process Mining Conformance Checking Against an Approved Workflow

A practical analysis begins with an agreed model and a usable event log. Validate the data before interpreting results.

1. Assign Ownership of the Reference Model

Identify one process owner who can approve the model as the official baseline. That person signs off on the activity list, the decision points, and the documented exception routes. Without this, your conformance metrics have no authority.

2. Confirm Valid Case IDs

A case ID must represent one complete instance of the process, such as one order, one claim, or one work package. If the ID changes mid-process when a record is reassigned, traces fragment and false deviations multiply.

3. Validate Timestamp Quality

Check granularity, time zone handling, and whether the system records the moment work happened or the moment it was entered. Identical or coarse timestamps can obscure event order; check source-system records when needed.

4. Build the Variant Table

Process variants are cases that share the same activity sequence. Compare variant frequency and attributes to see which routes deserve review, as in this insurance claims process improvement example.

5. Replay the Log Against the Model

Replay walks each trace through the model and records where the trace cannot proceed. Alignment methods find a lowest-cost match, under the chosen move costs, showing where observed and modeled behavior differ.

6. Quantify Deviation Frequency

Count how many cases contain each deviation type, and what share of total volume they represent. Conformance metrics measure model-log discrepancies, so report both raw counts and percentages.

7. Classify Before You Escalate

Label every deviation as an approved variant, a recording artifact, or a genuine control failure. Confirm each label against policy and source records before escalating it.

That classification deserves a worked example, because the labels only make sense against real traces.

A Labeled Example: Event Log Analysis in a Purchase Approval Process

Consider a purchase requisition process with a model requiring Create Requisition, Budget Check, Manager Approval, Purchase Order Issued, Goods Receipt, and Invoice Match. The following figures are hypothetical, not measured results. Imagine a log with 18,400 cases over nine months in which initial replay flags about 31 percent of cases; each flag still needs validation.

Observed Pattern Cases Classification Reasoning
Manager Approval skipped for orders under 500 USD 2,940 Approved variant Policy documents an auto-approval threshold that the model never captured.
Budget Check timestamp equals Create Requisition timestamp 1,610 Recording artifact Nightly batch writes both events at the same second, hiding true order.
Purchase Order Issued before Manager Approval, value above 50,000 USD 212 Potential control failure Verify policy, approval records, and order timing before concluding that spend preceded authorization.
Goods Receipt entered twice 684 Potential control failure Check whether entries represent distinct receipts, corrections, or duplicates before assessing payment exposure.
Invoice Match repeated three or more times 1,255 Mixed Investigate whether repeats reflect corrections, supplier data issues, or normal workflow.

The 212 cases may carry higher authorization risk if records confirm the sequence. The 2,940 cases may warrant a model update if policy supports that route.

Investigate the high-value approval cases promptly, then confirm their impact before ranking corrective work.

Use frequency, severity, and existing controls together when setting priorities. Treating a high-volume path as correct simply because it is common is how teams accidentally approve a control gap.

With findings labeled, the next decision is which ones justify a formal investigation.

Investigation and Corrective Action Criteria for Deviation Frequency and Risk

Investigation and Corrective Action Criteria for Deviation Frequency and Risk

Not every flagged trace deserves a root cause study. Set investigation criteria in advance with the process owner and relevant control teams. Allow urgent issues to bypass routine thresholds.

  • Financial exposure: any deviation that allows spend, payment, or credit without the required authorization goes to investigation regardless of count.
  • Regulatory or safety relevance: missing inspections, missing signatures, or missing verification steps in regulated work escalate immediately.
  • Frequency threshold: set a context-specific threshold based on volume, impact, and acceptable variation; five percent is an example, not a universal standard.
  • Concentration: deviations clustered in one site, shift, team, or system interface usually point to a specific, fixable cause.
  • Trend direction: a deviation rising month over month matters more than a stable one at the same level.
  • Control dependency: test the downstream control before adjusting priority, and document the rationale.

Corrective action depends on the cause. Update the model for documented practice, and fix the data pipeline for a recording artifact.

When records confirm a process failure, change the process, system configuration, or standard work. An invoice exception reduction example shows a structured investigation.

One caution. A fix that blocks a workaround without addressing why the workaround existed simply pushes the deviation somewhere the log cannot see it.

This is where analytical training matters more than tool familiarity.

Building Analyst Capability for Process Variants and Control Gaps

Conformance checking produces findings. Turning findings into sustained change requires cause analysis, risk prioritization, and disciplined standard work, which are the skills Lean Six Sigma develops. Air Academy Associates reports training over 250,000 professionals and uses a Keep It Simple Statistically approach to teach practical analysis.

Training here focuses on process and analytical skill rather than instruction in any particular mining platform. Green Belt and Black Belt programs build the structured problem-solving that follows a deviation report, and the course catalog includes short courses for analysts who need one specific capability rather than a full belt.

The improvement sequence can be adapted across sectors: define the reference process, measure the gap, prioritize by risk, and standardize the fix.

AirAcad Resources for Investigating Process Conformance

AirAcad Resources for Investigating Process Conformance

Conformance findings only become improvements when analysts can define the baseline, trace causes, read recorded data, and rank risk. The four short courses below map directly to those four tasks.

  1. VSM With IPO and SIPOC addresses the first problem in any conformance study: agreeing on the reference process and where it starts and stops. The course helps analysts map scope and flow; the process owner must still approve the actual reference model and exception rules. Use it before you replay anything, so your model reflects an approved workflow rather than one analyst's assumption.
  2. PF/CE/CNX/SOP connects observed deviations to their causes and to standard work. It introduces process flow, cause-and-effect analysis, factor classification, and standard operating procedures for investigating and sustaining a fix. For analysts, this is the bridge between a flagged trace and a documented fix that holds after the project closes.
  3. Historical Data Analysis Short Course covers how to interpret data that was recorded by a system rather than collected by design. Event logs are observational records, so inspect how events were captured and what cases may be missing. The course builds caution about what recorded behavior can and cannot prove, which keeps conformance conclusions defensible when stakeholders push back.
  4. Failure Mode and Effect Analysis (FMEA) helps you rank deviations by consequence and by the strength of existing controls. FMEA considers severity, occurrence, and detection; use these factors to examine potential failure modes associated with a deviation. It can help explain why a less frequent but more consequential issue may receive earlier attention.

Closing Thought

Measure the gap, label it honestly, and act on consequence rather than volume. If your team needs the analytical discipline behind that work, explore our training and certification options, check the public schedule, or request a consultation with our team.

Air Academy Associates helps teams investigate process deviations. Our Lean Six Sigma training teaches teams to quantify and correct variation. Learn more today.

FAQs

What Is Conformance Checking in Process Mining?

Conformance checking is a process mining method that compares what actually happened in your event logs to an approved process model to quantify deviations. It highlights where work follows the intended path, where it varies, and the impact of those variances (e.g., rework, delays, compliance risk). Teams often use it to prioritize improvements and strengthen controls using proven Lean Six Sigma-style problem solving.

How Are Observed Events Compared With a Reference Process?

Observed events from an event log (case ID, activity, timestamp, and key attributes) are aligned to a reference process model using conformance techniques such as alignments or token-based replay. The output typically includes fitness (how well reality matches the model), deviation types (skips, repeats, out-of-order steps), and performance effects (time and cost). This creates a fact-based baseline for targeted improvement and control design.

How Can Approved Exceptions Be Separated From Genuine Control Failures?

Start by defining what qualifies as an approved exception (policy rules, thresholds, roles, and required approvals) and capture those criteria as attributes in the event log. Then classify deviations by rule: exceptions with the right approval and documentation are "allowed variance," while deviations without required authorization or evidence are "control failures." This approach supports practical governance and audit readiness, and it's often strengthened by structured control plans and DMAIC discipline.

Related Articles

Posted by
Air Academy Associates
Air Academy Associates is a leader in Six Sigma training and certification. Since the beginning of Six Sigma, we’ve played a role and trained the first Black Belts from Motorola. Our proven and powerful curriculum uses a “Keep It Simple Statistically” (KISS) approach. KISS means more power, not less. We develop Lean Six Sigma methodology practitioners who can use the tools and techniques to drive improvement and rapidly deliver business results.

How can we help you?

Name

— or Call us at —

1-800-748-1277

contact us for group pricing