Explainable artificial neural network as a soft sensor to predict the moisture content in a continuous granulation line

Abstract

The application of artificial neural networks (ANNs) has the potential to fundamentally change the pharmaceutical industry, making manufacturing more agile, robust, efficient and reliable. Although ANNs’ application as data-driven soft sensors has a particular potential, the black-box nature of most models creates mistrust and prevents their widespread application. Therefore, this study focuses on the development of an explainable ANN used as a soft sensor to monitor an integrated, continuous manufacturing process based on twin-screw granulation. Our goal was to estimate the moisture content, a critical quality attribute, of granules only based on the applied process parameters without any direct measurements. Two separate ANNs – a multilayer perceptron (MLP) and a Nonlinear Autoregressive with Exogenous Inputs (NARX) – were built and compared with a near-infrared (NIR) spectra-based method. The validation of the methods – carried out by performing off-line loss-on-drying measurements – revealed that the accuracy of the ANNs and the NIR models was comparable, and the moisture content could be determined with a root mean square error of prediction below 1% in all cases. Additionally, the explainability of an MLP was also investigated by SHAP analysis, revealing which parameters impacted the prediction and strength of their impact, making the technology transparent and providing valuable insight into the model. This study highlights the potential of ANNs applied as data-driven soft sensors, offering a viable, orthogonal alternative to traditional analytical methods that is cost-efficient and enhances process understanding.

Highlights

Artificial neural network-based soft sensors were developed for process monitoring

The moisture content of granules was determined with an RMSE below 1%

The moisture content was predicted indirectly, from the applied process parameters

The most influential process parameters were identified

SHAP analysis made the model transparent and the predictions explainable

Introduction

With the growing importance of digitalization, automation, and the huge amount of data collected during manufacturing, the pharmaceutical industry is on the verge of transformation. The emerge of the Pharma 4.0 is part of the current change happening in other industrial sectors called the 4th industrial revolution or Industry 4.0 (Kusiak, 2018; Xu et al., 2018). The phrase generally refers to the implementation of complex, integrated, and data-driven smart factories supported by modern sensors, communication, computing platforms, control strategies, simulation, and modelling, taking manufacturing to a new level. The implementation of Pharma 4.0 is strongly connected and supported by the advanced quality assurance strategies already spreading in the sector, most notably the Quality-by-Design approach (QbD) (Jain, 2014; Mishra et al., 2018; U. S. Food and Drug Administration, 2023, 2009a, 2009b), the application of process analytical technologies (PAT) (Simon et al., 2015; U. S. Food and Drug Administration, 2009a), and the practice of real-time release testing (RTRT) (European Medicines Agency, 2012; Markl et al., 2020).

Although these practices have already reformed pharmaceutical manufacturing and laid the foundation for future improvements, implementing Pharma 4.0 requires further effort. The biggest reasons behind the delay are the uncertainties caused by the lack of precedents and regulatory guidelines, the need to change the thinking of the experts in the field, and the high financial investment associated with the development (Arden et al., 2021). It is also important to note that the transformation from Industry 2.0 to Industry 3.0 is still in progress in the sector, further complicating the field and making the implementation of new technologies challenging. Although part of the pharmaceutical industry utilizes improvements associated with Industry 3.0, such as automatization, digitalization, continuous manufacturing (CM) technologies and advanced quality control systems (including PAT), practices of Industry 2.0 (manufacturing with conventional batch technologies and using traditional off-line analytical methods) are still applied. Despite these challenges, the implementation of Pharma 4.0 is inevitable, as it can solve numerous problems in the industry (Steinwandter et al., 2019), making pharmaceutical manufacturing more agile, robust, effective, and flexible while reducing waste and enhancing product quality (Arden et al., 2021; Barenji et al., 2019; Ding, 2018).

An essential cornerstone of the Pharma 4.0 initiative is the application of artificial intelligence (AI). Among other benefits, the use of AI can enhance efficiency and quality control and accelerate research and development through the effective analysis of big data, the development of digital twins, and advanced process optimization (Nagy et al., 2022). Machine learning (ML) is one of the most steadily advancing subdisciplines of AI, showing great potential due to its ability to learn patterns from data to make decisions or predictions without being explicitly programmed (Arden et al., 2021; Nagy et al., 2022). Although implementing AI and ML into the current good manufacturing practice still poses a challenge and the regulatory framework still requires complementation, the regulatory agencies have already recognized the importance and potential benefits of these technologies. Their support is demonstrated by the release of the US. Food and Drug Administration’s guideline (“Good Machine Learning Practice for Medical Device Development: Guiding Principles”) and discussion paper (“Using Artificial Intelligence & Machine Learning in the Development of Drug and Biological Products”), highlighting significant milestones in the regulation and, thereby, the industrial application of AI and ML (U.S. Food and Drug Administration., 2023; U.S. Food and Drug Administration, 2021; Vora et al., 2023).

Among ML algorithms, artificial neural networks (ANNs) have shown particular promise in the pharmaceutical sector by applying supervised learning to describe relationships between input and output values. ANNs consist of an interconnected web of processing units called artificial neurons, where the information is processed with mathematical calculations (Kaur, 2021; Montesinos López et al., 2022; Nagy et al., 2022). With the artificial neurons organized into layers (input layer, hidden layer(s), and output layer), ANNs are capable of effectively describing complex, non-linear relationships (Arden et al., 2021; Krogh, 2008). In contrast to traditional calibration techniques, which usually rely on linear multivariate methods, the internal parameters of the ANN (i.e. weights and biases) are determined through a training process. During training, these parameters are optimized iteratively to minimize prediction error, thereby enabling the network to learn highly non-linear mappings between the input and target properties (Agatonovic-Kustrin and Beresford, 2000; da Silva et al., 2017).

A great number of studies have shown the potential of ANN in the pharmaceutical industry. One of the most studied fields is the application of ANN for process monitoring to evaluate data collected by PAT sensors. It is most commonly used to analyze spectroscopic data (UV-VIS (Hasani and Moloudi, 2008; Hasanjani and Sohrabi, 2017), infrared (Franco et al., 2006), near-infrared (NIR) (Dou et al., 2005; Rantanen et al., 2001; Zannikos et al., 1991) or Raman spectra (Péterfi et al., 2023)); however, examples of applying ANN to complement acoustic emission (Carter and Briens, 2018; Fulek et al., 2023), focused beam reflectance measurement (Crestani et al., 2021; Leite, 2023) and image analysis (Mahdi et al., 2018) can also be found in the literature. Various process steps such as synthesis, crystallization, granulation and tableting have been studied, and in the case of non-linearity, ANN regularly outperformed conventional methods (Nagy et al., 2022). Another possible application of ANN is to enhance process understanding and optimization by exploring the relationship between the initial critical material attributes (CMAs), the manufacturing parameters (the critical process parameters (CPPs)) and final product quality (critical quality attributes (CQAs)). Numerous application examples can be found in the literature: among others, ANNs have been utilized to predict the yield (Moghaddam et al., 2010) and optimize (Zhou et al., 2017) synthesis, predict the growth rate during crystallization (Vasanth Kumar et al., 2008), explore the relationship between granule properties and process parameters (Murtoniemi et al., 1994), reduce capping tendency during tableting (Belič et al., 2009) and optimize tablet coating (Benayache et al., 2023).

Another promising yet relatively underexplored field is ANNs’ application as data-driven soft sensors. Soft sensors (also known as virtual or software sensors) are applied to estimate process variables that are otherwise hard to measure with physical sensors. When the direct measurement is limited due to technological difficulties, financial reasons or impractical setups, using ANNs as soft sensors can be a viable alternative (Kadlec et al., 2009). Although the advantages of the approach have been highlighted by some studies – among which its application for monitoring complex systems is particularly promising (Roggo et al., 2020) – its potential is not fully realized. A reason behind this is the black-box nature of ANN models, as the lack of transparency complicates gaining regulatory approval, increases mistrust in the predictions, and thereby prevents the widespread industrial application of ANNs (Fan et al., 2021; Hulsen, 2023; Rawal et al., 2022).

Realizing this issue, a growing amount of research focuses on developing methods to explain the operation of ML, AI, and, consequently, ANNs (Molnar et al., 2021; Saleem et al., 2022). Although various definitions exist (Flora et al., 2022), the explainability of ANNs generally refers to the ability to understand the predictions of the model, exploring why the model produced the given outputs (Gilpin et al., 2018; Molnar, 2022; Roscher et al., 2020). Various strategies have been established to make ANNs explainable: it can be achieved by focusing on analyzing individual model components (Murdoch et al., 2019), or through the development and investigation of surrogate models that mimic the behaviour of the original ANN (e.g., Local Interpretable Model-Agnostic Explanations method) (Ribeiro et al., 2016). Another promising approach is to study the model sensitivity by analyzing the model predictions to the perturbations of the input data. It can also be carried out either globally (e.g., through perturbation-based methods (Ivanovs et al., 2021)) or locally, explaining the individual predictions (e.g. the game theory-based SHAP analysis (Ma and Tourani, 2020)). Despite the evident need for explainable ANNs, only a few examples in the pharmaceutical field can be found in the literature (Gentiluomo et al., 2019; Ghanavati et al., 2024; Honti et al., 2024; Korteby et al., 2018; Nagy et al., 2023; Ogami et al., 2021).

Therefore, the goal of this study was to develop a transparent, explainable ANN that can be applied as a data-driven soft sensor to monitor an integrated pharmaceutical manufacturing process. Our aim was to predict the moisture content, an important CQA, after a continuous, twin-screw granulation-based process only from the applied process parameters (CPPs) without any direct measurement. Two separate ANN models – a simple multilayer perceptron (MLP) and a Nonlinear Autoregressive with Exogenous Inputs (NARX) – were developed and compared with each other as well as traditional monitoring techniques to reveal whether they were suitable for in-line monitoring. Additionally, the SHAP analysis of the MLP was also carried out to reveal which input parameters influenced the prediction and the strength of their influence, thereby providing valuable insight into the model. Our goal was to demonstrate that ANNs applied as data-driven soft sensors can be promising tools to supplement or even replace the traditional off-line or spectra-based monitoring, as these methods are orthogonal, cost-efficient and robust, and thereby can reduce expenses, improve accuracy and enhance process understanding.

Download the full article as PDF here Explainable artificial neural network as a soft sensor to predict the moisture content in a continuous granulation line

or read it here

Materials

The granulation was performed using a placebo system containing α-lactose monohydrate (Granu-Lac® 70), corn starch and polyvinylpyrrolidone (povidone K30, Kollidon® 30). The α-lactose monohydrate was provided by Meggle Pharma (Wasserburg, Germany), the corn starch was supplied by Roquette Pharma (Lestrem, France), and the Kollidon® 30 was obtained from BASF (Ludwigshafen, Germany). The α-lactose and the corn starch were mixed and homogenized manually (for 5 minutes) before the experiments, while the polyvinylpyrrolidone was solved in distilled water to form the granulation liquid (ratio: 75 g Kollidon® 30 in 200 mL distilled water).

Petra Záhonyi, Dániel Fekete, Edina Szabó, Zsombor Kristóf Nagy, Brigitta Nagy, Explainable artificial neural network as a soft sensor to predict the moisture content in a continuous granulation line, European Journal of Pharmaceutical Sciences, 2025, 107173, ISSN 0928-0987, https://doi.org/10.1016/j.ejps.2025.107173.

Read also our introduction article on Continuous manufacturing here:

A universal digital control concept for end-to-end manufacturing

excipients formulation