Handling Out of Expectation (OOE) Results in Pharmaceuticals

Not every problem with quality in pharmaceutical production is an OOS situation. There are plenty of cases when testing gives results in compliance with specifications but at the same time differing significantly from the past records of results or from expected performance. Such contradictions are known as OOE results.

Even though OOE results may not impact batch release in any way, investigating them can provide valuable information regarding the beginning of a deviation of any process, test method, piece of equipment, or utility system from its normal working conditions. Ignoring those signs of deviations may eventually lead to deviations, OOS indications, customers’ claims, etc.

In my opinion, the way a company approaches OOE events will determine its ability to control production processes and maintain them stable since it allows understanding the importance of minor changes before they lead to a greater quality issue. Thus, it is more beneficial for firms to treat OOE as an opportunity to learn more about their processes rather than a burden imposed by regulations.

The goal of this article is to explain the meaning of OOE results, their nature in comparison with the OOS and OOT results and the ways of improvement of OOE investigation results.

Understanding Out of Expectation (OOE)

An Out of Expectation result indicates a value that is within the permissible limit but quite far away from the data from similar previous processes or actual capability trends. The result might be regarded as acceptable in itself but gives rise to problems because it shows something different than what normally occurs.

For example, if we talk about a tablet assay specification of 95.0% to 105.0%, the case may be that all previous batches have had assay values in the range of 99.2% to 100.5%, while now a value that equals 95.4% appears. Thus, it may be compliant with the specification, while the historical trend should still be investigated.

Another example would be a functioning HVAC. It can be qualified for certain temperature levels, but at some point, it may show changes that have never been observed before. Although this has nothing to do with specifications exceeding, it can appear to be the evidence of the beginning deterioration of a unit.

OOE, OOS and OOT: Understanding the Difference

Even though the expressions OOE (Out of Expectation), OOS (Out of Specification) and OOT (Out of the Trend) are generally used together, they stand for different types of quality incidents and entail different actions.

1. Out of Expectation (OOE) Results

The OOE result is the one which does meet the relevant specification but falls far beyond the historical indicator or validation data. Hence, you should carry out the trend analysis and scientific investigation to find out if the process is starting to drift.

2. Out of Specification (OOS) Results

The OOS result is the one obtained when the outcome of testing does not meet the established specification or acceptance criteria. OOS results require a formal investigation in accordance with proper OOS procedures comprising the determination of any possible consequences for the quality of the product, batch disposition and possibilities for regulatory compliance.

3. Out of Trend (OOT) Results

A result that is out of trend (OOT) indicates a change in analytical results or in process data that has taken place over time. In this case, the individual results may still comply with specification. OOT is generally detected and investigated through the trend evaluation and process evaluation.

Moreover, OOT result, being the first indication of OOE, may ultimately lead to out of specification (OOS) if investigated in the wrong way. Therefore, proper investigations of the unexpected results are essential for the pharmaceutical companies.

Common Sources of OOE Results

There are numerous areas of pharmaceutical production and quality control for the emergence of abnormal results.

Typical sources are, for example:

• Historic assay readings that differ from those of today
• Unusual dissolution test results
• Higher number of microorganisms reported in alerts
• Water conductivity levels that approach highs established historically
• Any environmental monitoring data above average figures
• Unusual yield
• Variations in compression forces at tablet production
• Unusual increase in coating weight
• Stability results differing from the ones observed earlier

Each case should be analyzed to reflect its contribution to the quality of the product and the quality control systems in place.

Why OOE Results Should Never Be Ignored

Organizations routinely make mistake in believing that OOE results that fit within acceptance criteria are not worth following up. Though specification compliance constitutes a satisfactory regulatory hurdle, the robustness of a process lies in identifying and understanding deviations from normal operation at an early date, if actual product quality is to be guaranteed.

A properly executed OOE investigation can help to:

• Spot drifting processes ahead of time
• Detect deterioration of equipment before its breakdown
• Identify variability of raw materials
• Increase process capability
• Avoid unnecessary OOS investigations
• Allow for regular process verification
• Enhance risk management

In fact, in many instances, one might be better off investigating an unsatisfactory OOE result than dealing with a failed production run.

Step 1: Confirm the Result

In this step, we can answer the following questions:

• Was the analytical procedure conducted as it should have been?
• Were the calculations confirmed with integrity?
• Was the measuring instrument calibrated?
• Have the system suitability criteria been met?
• Were the standards valid?
• Was the sample prepared in reference to the method?

If errors are found, it is necessary to investigate the problems by the quality management system of the lab before continuing with investigations of process-related reasons.

Step 2: Compare with Historical Data

Historical knowledge plays an important role in investigations of the OOE.

The following subjects could be reviewed:

• Results from previous batches
• Validation reports
• Stability data
• Annual Product Quality Reviews
• Data on Continued Product Verification
• Statistical process control charts
• Trend reports

The degree of sampling population and the significance of its investigation can be determined by comparing the present event with historical data.

Step 3: Assess Manufacturing Conditions

Once the analytical result is validated, it is essential to analyze the manufacturing process.
You will want to analyze different factors like:

• Lot of raw materials
• Changes in supplier
• Equipment used
• Calibration status
• Record of preventive maintenance
• Control of environmental conditions
• Performance of utilities
• Operation of the process

There may be small changes in several variables that can justify a surprising result.

Step 4: Perform a Risk Assessment

Not every OOE result requires the same level of investigation.

A proper risk assessment must evaluate:

• Effect on product quality
• Risk of harm to patients
• Batch-to-batch consistency
• Regulatory compliance
• Likelihood of occurrence

The tools commonly used are

• FMEA
• Risk Ranking and Filtering
• Fishbone Diagram
• 5 Whys Analysis

The depth of the investigation must be based on the risk identified.

Step 5: Determine Whether the Process Is Drifting

While one record of out of specification (OOS) result doesn't mean that there is a serious problem; a third of an OOS occurrence shows that there is most likely some deterioration of the process.

Trend analysis should be done with respect to such parameters as:

• Measurement of the assay value
• Homogeneity of the mixture
• Force amount applied in compression
• Dissolving quality
• Water quality
• Environmental conditions
• Utility specifications
• Diagnostics indicated by the equipment

In most cases, the trend analysis will diagnose this drift ahead of time and alert of possible specification failures.

Step 6: Document the Investigation

Every investigation of an OOS result should be documented in such a way that it represents logical reasoning behind the decision made.

The report should include:

• Unexpected result description
• Historical comparisons
• Analysis of investigation activities
• Root cause
• Risk assessment
• Effect on the product
• Corrective actions
• Preventive actions
• Final conclusion

The documentation should also explain the reasons for decisions made as opposed to merely recording what actions were undertaken.

When Does an OOE Become a CAPA?

Not each OOE case needs one to proceed and give an opportunity for a Corrective and Preventive Action (CAPA) to commence. In cases when the study shows presence of any systemic weaknesses, corrective actions should be implemented. These include:

• Repeated equipment failure
• Lack of proper controls in the process
• Variability caused by suppliers
• Procedure lapses
• Gaps in training
• Analytical issues

The actions that were taken must bring some results that can be defined with the help of trending techniques.

Common Mistakes During OOE Investigations

There are several common errors that compromise efficiency of OOEs.

These are:

• Ending the investigations when it is established that specifications are met
• Only taking into account last batch results
• Not considering historical process capability data
• Thinking that OOE is a laboratory problem only
• Failing to perform statistical tests
• Not considering performance trends of equipment
• Starting the investigation only after the OOS result has been obtained
• Weak justification of scientific reasoning

These errors enable minor process variations to escalate into significant quality concerns.

Building an Effective OOE Management Program

The OOE program must be operated as part of the Pharmaceutical Quality System and not as a separate entity.

Key practices are:

• Determining OOE criteria by employing historical data and statistical evaluation.
• Training employees to identify unusual results.
• Conducting trending analysis of important product quality characteristics.
• Including OOE discussions into Annual Product Quality Review.
• Connecting OOE investigation with risk management and CAPA systems.
• Periodically checking repeating OOE incidents in Management Review meetings.

A proactive approach helps companies identify any discrepancies in a timely manner and thus maintain the controlled process.

Out of Specification results can point to improvements in performance since they show differences not yet out of specifications but still worthy of attention. Examining these results will enable pharmaceutical companies to see deviations in process, deterioration of equipment, variability of raw materials and quality risks in their early development stages. Thus, a successful management of OOE is important for the process verification and lifecycle management of quality.

However, in my experience, companies that have developed their quality systems do not wait for an Out of Specification result to take action. They acknowledge the fact that maintaining consistency in the process requires prompt reaction on the unexpected trends. This can be done with the application of scientific investigation, past data analysis and decision-making based on risks.