Process Mining: A Brief Introduction

Process mining is a method of analysis that aims to discover, monitor and improve real business processes by extracting available knowledge from event log systems in an organization.

Essentially, process mining provides a crucial connection between real-time events and operational business processes. Process mining is an approach that examines event data from logs to see what employees in an organization are doing and how they’re actually doing it. By analyzing the steps required to complete a task or project, process mining automatically constructs a process. And as this data is gathered over time, it can surface the bottlenecks and inefficiencies that create barriers to productivity and profitability.

In this blog post we'll explore why process mining is important for businesses, where and how it’s used, how it can create value for your organization and how you can get started on your own process mining implementation.

---

---

The Importance of Process Mining

Process mining leverages advanced algorithms to create transparency into current business processes, helping organizations to streamline and improve on them. It quickly uncovers valuable insights that can improve productivity, and ultimately illuminates the opportunities in your core business processes that will have the biggest impact on your customers and your bottom line.

For the opportunities impacting your business, process mining can be used to examine three major types of key performance indicators (KPIs):

• Time KPIs: How long does it take to complete a particular process?
• Cost KPIs: How much does it cost to complete a particular process?
• Quality KPIs: Does the outcome of this process meet established criteria?

Process mining has a significant advantage over more traditional “as-is” analysis — and that’s its ability to access real-time event data. What’s more, process mining also looks at historical data, with an ability to closely examine a series of event logs to achieve an in-depth understanding of what’s going on — a stark contrast to the slow and manual heavy duty data infrastructure previously used to conduct the same calculations. Rather than relying on traditional data infrastructure to analyze transactions, process mining can surface what is currently happening, leveraging tremendous amounts of event data from all your systems to:

• Discover the actual behavior of people, organizations and machines and compare it to existing models.
• Correlate millions of events to show how reality is different from perceptions, opinions and beliefs.
• Provide a foundation for continually improving and building better processes.

Above all, process mining allows you to understand the current state of your systems and processes while offering a faster, more granular way to identify any deviations and aberrations — then course correct.

Example of a process flow diagram, in this case demonstrating how a system administrator should troubleshoot slow search performance.

Examples of Process Mining

Process mining use cases are numerous, but according to Gartner, some of the most common include improving processes, business process management, improving auditing and compliance, analysis and validation, improving process automation, supporting digital transformation by linking strategy to operations and improving IT operations resource optimization.

Process mining techniques help organizations address a plethora of process pain points caused by lack of visibility, insight, staff and appropriate tools. Challenges include:

• Too many systems: Enterprises of all sizes are dependent on dozens of systems and complex infrastructure that can be exceedingly difficult to monitor. It’s far harder still to achieve a clear, end-to-end view across the entire environment. Process mining can break down the silos that separate different types of data and merge it all into one dataset.
• Too much data to handle manually: Increasing volumes of data from your internal networks, connected devices, your website, supply chain management, purchasing, quality control and dozens of other systems create new and rapidly growing challenges. A good process mining solution can automate data cleansing and preparation while analyzing ever-larger datasets.
• Not enough help: While data might be the most valuable asset in your organization, you often don’t know what to do about it or how to use it to your advantage. Process mining solutions are designed for you to use on your own, without specialized data skills or expertise. It also allows you to free yourself from manually weeding through the data so that you can actually focus on business growth and other mission-critical tasks.
• The wrong tools: Chances are you’ve spent a lot of time trying to fit data into a spreadsheet and build formulas to make sense of it, all to realize that you weren’t providing real business value. In addition to surfacing data, process mining can help you use it to drive decisions.
• Inflexible, unreliable reporting: You’ve also probably spent hours or days organizing important data that only leads to more questions when it’s presented. Process mining gives you flexible reliable reporting, lets you express process analytics in easily shared (and understood) visualizations and dashboards and customize them, no matter how many questions your boss asks.

A business process flowchart provides a clear and accurate picture into the efficiency and effectiveness of your business processes.

Process Mining Algorithms

The process mining algorithm essentially determines how the process model is inferred from raw event data. An algorithm is used to correlate event log data to identify trends, patterns and aggregate metrics contained in event logs recorded by an information system. The mined models are then compared against the original enterprise process models to check conformity, or to discover more streamlined and efficient business process models.

There isn’t one widely-accepted benchmark standard from which to evaluate and compare process mining algorithms, which might create challenges when choosing the right algorithm for one given enterprise or application domain. A high-quality algorithm, for example, is one that can easily correlate events from completely different systems and other heterogeneous data sources. However, because different process mining algorithms have different qualities, enterprises will have an advantage if they’re able to select an algorithm that produces mined models similar to, or better than, the original models.

Process mining automatically discovers actual business processes and garners insights from existing application data logs — data that can be used to automatically generate process models and calculate process metrics. By analyzing the sequence of events using their timestamps, process mining can completely reconstruct actual processes while identifying and uncovering inefficiencies, bottlenecks and other weaknesses. And thanks to the availability of data — that is, any action being executed in an application and captured in an event log — process mining has the ability to conduct this analysis agnostically.

Process Mining Use Cases

Process mining can be used for business process management and process improvement in any application in any industry, in particular, financial services, telecommunications, healthcare and retail — industries with extensive data that can be used as a basis and where deviations in processes from their intended behavior can have expensive consequences.

• Financial Services: As the volume of transactions have increased and more businesses become digitized, manual processes do not effectively catch all anomalous behavior. Faced with greater regulatory and audit requirements, process mining offers companies in the financial services the opportunity to continuously and comprehensively discover problems within high-volume processes.
• Telecommunications: As subscriber volumes continue to grow and activations become ever more automated in the world of telecommunications, the risk of failed activations becomes significant. Process mining helps telecommunications firms faced with greater volumes of orders the opportunity to discover expensive problems and customer fallout in their Order-to-Activation processes.
• Healthcare: As data about patient experiences and outcomes continues to grow, the risks around maintaining population health and individual patient journey outcomes also increases. For healthcare organizations faced with an exponential increase in data, process mining helps deliver efficient and high quality end-to-end patient journeys, from before an initial doctor consultation through treatment regimens to closed treatment cases.
• Retail: Retail organizations have experienced costly customer fallout from complex e-commerce flows due to system or process issues. Faced with increasingly high transaction volumes, process mining helps retailers ensure that customers are able to complete orders quickly and effectively without problems.

Process Mining for Digital Transformation

Process mining is regularly used as part of larger-scale digital transformation efforts because it can provide objective data-driven insights into the heart of delays and inefficiencies within business processes, while also giving you the clear insights required for process improvement that enable systems to run faster, smoother and leaner. As such, process mining can help prioritize the highest value-added opportunities for digital transformation as well as evaluate if transformation efforts have actually yielded the benefits they intended. Process mining becomes an invaluable tool to maximize the return on investments in digital transformation initiatives.

Among other things, it can:

• Provide visibility that can accelerate process times and increase productivity that impacts the bottom line.
• Identify bottlenecks in business processes, as well as detect aberrations and locate the biggest time-wasting issues.
• Deliver better and more accurate insights, which can translate into higher-quality results.
• Offer more up-to-date monitoring, enabling you to drastically reduce risk by addressing problems as they emerge.

Process Mining in Context

Understanding the Goals of Business Operations Teams

Ultimately, the goals of the business operations teams are to execute on the operations plan — which means building a workflow for the organization’s systems and major operational areas that focus on efficiency and productivity so that analysts spend less time on routine issues. Partnering with business executives, the business operations team helps improve the efficacy and efficiency of business operations performance KPIs to minimize deviations and ensure system-wide operational excellence. Among other things, they’re tasked with driving and elevating transparency into their existing processes by assessing and monitoring key performance metrics to detect bottlenecks, fallout, escalations and other problems that threaten service quality, customer satisfaction, employee productivity and overall business and profitability objectives.

How can process mining help business operations?

Operational teams can use process mining techniques in new and innovative ways to better understand business processes and more quickly find bottlenecks, aberrations and redundancies that slow down operations. Once these elements have been identified, they can be eliminated, speeding up the process — often cutting it in half.

Business Intelligence vs Process Mining

Process mining differs from traditional business intelligence in the level and depth of the analysis. Traditional BI assumes a prior knowledge of underlying processes, and as such, focuses on repeated calculation of aggregated metrics for reporting. But while these business intelligence dashboards can illuminate any potential problems in known areas, they do little to provide visibility into unknown areas.

Process mining on the other hand is based on the premise that processes don’t always go according to plan and that problems creep up in places that cannot be anticipated up front. Thus, deeper introspection into how the processes are actually performing is essential to ensuring operational excellence.

Process Mining: First Steps

How do you get started with process mining?

If you’re interested in starting a process mining initiative, you can get the ball rolling by first identifying the pain points, identifying the data and then launching a pilot project. Remember, the starting point for any process mining project is the process analysis, which closely examines the current state of the business processes, maps out shortcomings and identifies opportunities for improvements.

Here’s a time-tested method for investigating the value of process mining.

• Identify the problem: Pick a problem that matters to the business, and that you can realistically tackle with process mining. Then determine the business value of solving it and what metrics you’ll use to measure success.
• Identify the data: First, you’ll need to identify the sources of the data that you’ll need to comprehensively understand to address any glaring business process problems. Similarly, you’ll need to identify which applications and systems must provide feeds of event data for continuous transparency into the end-to-end processes.
• Pilot a discreet project: A pilot project can be vital to proving the potential value of a process mining solution. Make sure the project can be accomplished relatively quickly and can deliver specific, measurable outcomes everyone in your organization can understand.
• Accept the truth: Among other things, process mining gives you a clear picture based on facts — so be transparent about the results of your analysis. While challenging, process mining can be just the tool to validate the uncomfortable realities of your business process environment, as well as being a constructive springboard for you to ask the right questions that get the problems fixed.

Remember, process mining is more than just a tool — it’s a paradigm shift that requires skilled administrators to discover issues and remediate them. In turn, they have the ability to open up a dialogue with the rest of the organization to comprehensively and objectively address ongoing, systemic process issues that have impeded productivity and effectiveness.

Getting Value from Process Mining

Using process mining offers a myriad of benefits to businesses if you understand how to extract the most value from the solution. Focus on the value potential of process mining, and investigate how it can improve the areas in which you need the most help. Key areas of potential include:

• More time and resources freed for IT staff: Because process mining provides a clear and accurate picture of business processes reconstructed from IT data, administrators spend less time investigating process issues on their own and more time on value-add projects that help the organization realize business growth objectives. And by having a solid grasp of the current state of business processes as a starting point, time spent on other issues and projects can be more productive.
• Clear and accurate big-picture of process environment: The data provides a realistic picture of business processes, as well as a foundation for accurate analysis and diagnosis. All variables, delays and other issues are incorporated as part of that holistic view, so decision makers can take action based on fact rather than opinion.
• Improved process systems: IT administrators understand the ins and outs of process flow, including delays and frequencies. IT system logs track extensive volumes of valuable data, which can be leveraged to detect process inefficiencies, illuminate most frequent paths, and generally lay a foundation for a more streamlined process environment. Process mining can also speed up investigations by allowing analysts to immediately address pain points, which in turn increases productivity and reduces cost for the organization.
• Efficient and improved audits: Organizations can rely on process mining to help more efficient and accurate audits that avoid guesswork and opinion-based findings. This means less wasted time with uncertainty and subsequent retesting.

What to look for in Process Mining Software

To choose good process mining software, a solution should excel in three functions — process discovery, conformance checking and performance analysis and improvement.

• First, your process mining solution should include strong discovery capabilities, the ability to scour event logs to track what employees are actually doing, then create an appropriate process model by generating process maps of the entire organization’s business flow.
• In addition, you’ll want to look for robust conformance checking, which analyzes event logs to ensure that actions align with process models.
• And third, your solution will need performance analysis and enhancement capabilities, that analyze potential inefficiencies within an event log, investigate to determine whether and how it can be improved and then make improvements according to the real process data.

While the right process mining software varies depending on the size of your organization, business needs and goals, key features included in your solution should give you an ability to:

• Identify bottlenecks as well as opportunities to optimize processes.
• Gain insights into failed process steps.
• Obtain an end-to-end view of the entire business process.
• Perform real-time continuous business process monitoring.
• Monitor key performance indicators in real time.
• Conduct preparation and data cleansing.
• Perform compliance verification and gap analysis.
• Improving process models.
• Correlate data across heterogeneous systems.

An organization's ability to measure, monitor and optimize business processes has a direct bearing on its revenue and customer satisfaction — which is why you’ll need to be judicious in selecting a process mining solution that best meets all your business goals.

---

---

Wrapping Up

For organizations, the ability to analyze log data via process mining represents an enormous opportunity, especially for those that are struggling with complex and unwieldy business processes. Organizations are rapidly generating enormous amounts of data that often goes unused — and that data can often unlock new opportunities for profitability. Because of an inability to gain insights into or even fully understand their business processes, they risk expensive logjams that inevitably affect efficiency, operational performance and ultimately, their revenues.

Organizations need an approach that transforms previously complex and chaotic data into an opportunity instead of a risk or an impediment — and that’s where process mining comes in. Above all else, it represents a better way to analyze and correlate disparate and seemingly unrelated information, identify weaknesses and quickly take action. Rather than wasting hours, days or weeks of your time tackling process dysfunction on spreadsheets, adopting the right process mining tool will enable you to use the data you have more effectively and drive more business value. And while tackling the data chaos in your organization might seem like a daunting task, putting the wheels into motion now will reap a multitude of rewards down the road.

What is Splunk?

This posting does not necessarily represent Splunk's position, strategies or opinion.