Multimodal convolutional neural network for tablet-level dissolution prediction using compression force and UV fluorescent imaging

Modern rotary tablet presses can produce hundreds of thousands of tablets per hour, yet the quality control of pharmaceutical tablets is still based on testing small sample sizes using on-line (e.g., NIR, hardness testing) and off-line (e.g., dissolution testing) techniques. Conventional in-process controls, such as tablet mass, height, mechanical strength, and disintegration time, are time-consuming and resource-intensive and poorly suited to support an effective transition towards continuous manufacturing. (Roggo et al., 2020) Real-time release testing (RTRT) (RTRT guideline, 2012), as defined under ICH Q8(R2) (ICH Q8(R2), 2010), represents the regulatory goal to replace this paradigm by evaluating and ensuring the quality of every intermediate and/or final drug product based on in-process measurements and process understanding, rather than end-of-line destructive testing. Notably, RTRT is based on an appropriate combination of measured material attributes and process controls that collectively support the release decision, rather than on any single measurement (RTRT guideline, 2012, ICH Q8(R2), 2010). Consequently, practical RTRT strategies integrate multiple complementary in-process and PAT-based measurements, each contributing to the overall assessment of product quality. The implementation of RTRT is based on the principles of the Quality-by-Design (QbD) paradigm, which emphasizes a systematic, science-based understanding of the relationships between critical process parameters (CPPs), critical material attributes (CMAs), and critical quality attributes (CQAs). (RTRT guideline, 2012) Within this framework, Process Analytical Technology (PAT) tools provide the measurement infrastructure required to monitor critical process variables in real time and the development of predictive models for quality attributes. (Manley et al., 2019) RTRT recognizes that under predefined conditions, in-process controls and PAT measurements can replace end-product testing (RTRT guideline, 2012)(); e.g., based on spectral data, ingredient content (Wahl et al., 2014), blending homogeneity (De Beer et al., 2008), or particle properties (Zhao et al., 2023) can be monitored and controlled often combined with chemometric approaches (Roggo et al., 2007) or machine learning models (Nagy et al., 2022) to estimate product quality attributes. (Movilla-Meza et al., 2025).

However, some CQAs, such as in vitro dissolution, cannot be measured directly by any inline probe and must instead be inferred indirectly, while being influenced by the cumulative effect of material attributes, formulation and process decisions made throughout manufacturing. (Yu et al., 2014) Therefore, making dissolution predictions from compression-stage data is a valuable but technically challenging goal and the central objective of this study.

The tablet compression stage is an information-rich unit in the tableting process, as modern tablet presses generate tablet-wise process data. Two sources of information are accessible in near real-time without interrupting production: the compression force and the tablet surface image. The compression force is a routinely registered signal of modern tableting machines and its relationship with the downstream quality attributes is well documented, e.g., as compression force increases, tablet density increases while porosity decreases, directly influencing liquid ingress during dissolution and, consequently, disintegration and drug release kinetics. (Adeleye, 2019, Sagiraju et al., 2024).

Compression force reflects the mechanical state of the tablet but provides no information on component distribution, whereas surface imaging offers complementary insights. (Dave et al., 2017) UV and VIS imaging-based machine vision systems enable the determination of active pharmaceutical ingredient (API) distribution and particle size using high-resolution images of the tablets in only a few milliseconds (Ficzere et al., 2025), with potential for in-line acquisition directly from production. (Galata et al., 2021) Together, the compression force and the UV images might provide two supplementary sources of information, one encodes global mechanical behavior, and the other encodes local physicochemical heterogeneity. This makes the combination of the two a natural candidate for multi-input predictive modelling.

Extracting meaningful information from tablet images can be achieved using hand-crafted descriptors or learned representations. Classical approaches reduce the image to engineered features such as color histograms, GLCMs, or wavelet-based texture descriptors, which have been applied to predict API content, compression force, and dissolution from UV/VIS images. (García-Muñoz and Carmody, 2010, Kim and Choi, 2025, Mészáros et al., 2020, Mészáros et al., 2024, Mollereau et al., 2013, Péterfi et al., 2026) Although these approaches are straightforward and interpretable, they have a key limitation that the features are defined in advance, thus they may miss relevant information for the target prediction that is not encoded in the chosen descriptors. UV imaging inherently provides information related to API distribution and, to some extent, tablet compaction, which can be extracted using more complex analysis methods. However, this sensitivity to feature selection is particularly challenging when subtle changes in surface appearance or tablet brightness—arising from variations in tablet properties such as API particle size, disintegrant content, or compression force—are confounded with each other.

Convolutional neural networks (CNNs) offer an alternative approach by learning hierarchical feature representations directly from image data in a task-specific manner. Combined with machine vision, they enable not only descriptive analysis but also predictive modeling based on learned relationships between inputs and outputs. (Galata et al., 2021, Islam et al., 2024, Vaghela et al., 2024) In the pharmaceutical domain, CNNs have been applied predominantly to classification tasks such as defect detection (Diószegi et al., 2024, Ma et al., 2020, Pathak et al., 2025, Freiermuth et al., 2025), tablet identification (Nguyen et al., 2025, Kavitha and Madhumathy, 2025, Vignesh et al., 2025, Banerjee et al., 2016, Kumar et al), and coating quality assessment (Ficzere et al., 2022, Kim and Choi, 2025, Pathak et al., 2025), as well as particle and granule morphology analysis influencing downstream compaction behavior (Kálnai et al., 2025, Péterfi et al., 2023, Zhang and Yin, 2021). However, their application to regression tasks remains substantially under-explored. A few studies have explored the estimation of particle size distribution (PSD) using CNN-based approaches; however, these are often based on spectroscopic data (e.g., Near-Infrared Spectroscopy) combined with one-dimensional CNN architecture rather than true image-based inputs. (Peng et al., 2024, Zhong et al., May 2025) Notably, Zhang et al. demonstrated a PSD estimation directly from images using CNNs, highlighting the potential of image-based regression in this field. (Zhang and Yin, 2021) Nevertheless, to the best of our knowledge, no studies have reported the prediction of dissolution behavior directly from image data, indicating a clear research gap and an opportunity for further development.

Previous research (Mészáros et al., 2024) has shown that combining machine vision with artificial neural networks (ANNs) enables accurate prediction of in vitro dissolution profiles from UV/VIS image data, with RMSE values below 5%. However, the model in that earlier work didn’t analyze the images directly. Rather than feeding the raw images straight into an ANN, it extracted specific features from the images (such as estimates of compression force and particle size) and used those predictions as inputs to the model. Mészáros et al. has extended UV/VIS imaging to white tablet formulations, demonstrating the practicality of machine vision for real-time quality control regardless of the API color. (Mészáros et al., 2024) More recently, Péterfi et al. estimated tablet dissolution based on UV images, using the principal components of the color histograms as input data for the ANN. (Péterfi et al., 2026) It is important to highlight, that all existing image-based dissolution prediction approaches process image information through classical pipelines. Moreover, the direct use of a CNN as an end-to-end feature extractor operating on raw tablet images for dissolution profile regression, particularly in combination with a scalar process variable, has not previously been reported.

However, a fundamental limitation of single-input convolutional neural network (SI-CNN) architectures, i.e., models that rely on a single data modality, such as images alone, is that they may fail to capture all relevant information required for accurate prediction. Fusing image data with complementary non-image inputs into a unified neural network has been explored in medical imaging (Minyilu et al., 2025), where combining clinical images with patient metadata (scalar variables such as age, sex, and lesion location) has been shown to outperform image-only models in skin liaison diagnosis. (Li et al., 2025, Pacheco and Krohling, 2021, Ou et al., 2022) Such models generally follow a dual-branch design, where image data passes through convolutional layers to generate a learned feature vector, and scalar variables are processed through a fully connected branch. The two branches are then concatenated before the output layer. Despite the advantages demonstrated by the multi-input convolutional neural network (MI-CNN) in medical applications, it has not been adopted in the pharmaceutical manufacturing domain. To date, no study has applied a dual-branch CNN that combines tablet surface images with a scalar process variable, compression force, to predict a continuous pharmaceutical quality attribute. This represents the primary architectural contribution of the present work.

This work aims to provide for the first time a MI-CNN model for a regression task in a pharmaceutical application. To achieve this, a MI-CNN was developed to predict the in vitro dissolution profiles of immediate-release tablets. The model has two complementary inputs, the UV images of individual tablets, and the corresponding tableting compression force as a scalar process variable.

To systematically evaluate the contribution of each input modality, the MI-CNN is compared to two alternative models: a SI-CNN trained only on tablet images, and a traditional image processing baseline in which the blue and green channel histograms extracted from the tablet images are combined with the compression force to serve as inputs to a multilayer perceptron (MLP) model. All models are trained and evaluated on a dataset of 15 formulation settings defined by a face-centered composite Design of Experiments (DoE) plan, in which compression force, disintegrant content, and API particle size distribution are the independent variables. Comparing these three approaches allows us to directly assess the value of learned versus hand-crafted image features, and of the benefit of integrating process variables alongside image data, for pharmaceutical dissolution prediction. Note that a classical linear/statistical benchmark model (e.g., Partial Least Squares, PLS or Multiple Linear Regression, MLR) was not included, as prior work has already established that ANN-based models consistently outperform linear regression approaches for dissolution prediction from process- and composition-derived data. (Galata et al., 2019, Lourenço et al., 2025).

Moreover, from a process development and quality control perspective, a single tablet measurement is not guaranteed to carry sufficient information to reliably predict the full dissolution profile, since it is inherently governed by the combined effect of formulation attributes and process conditions. By combining image-based information with process variables measured in-line, the proposed framework reflects a more realistic industrial setting. Additionally, the formulation space covered here is considerably more challenging than those typically considered in image-based dissolution modeling, thereby providing a more rigorous test of model robustness and applicability in real-world manufacturing environments.