# Introduction In recent decades, neural networks and machine learning models have become key tools in various fields, including computer vision, natural language processing, recommender systems, medical diagnostics and also in the field of assisted reproductive technologies (ART) (Glatstein et al., 2023). They serve as the foundation for creating algorithms capable of extracting complex dependencies from data, making predictions, and making decisions based on these dependencies. In some cases, neural networks can identify a broader spectrum of associations than other statistical methods, thanks to their ability to recognize highly nonlinear associations among input parameters. Most machine learning models applied in the field of ART, in the context of in vitro fertilization (IVF), are based on regression and logistic regression algorithms to identify relationships between the target variable and input parameters (Alizadehsani et al., 2021). The target variable typically represents the outcome metric of clinic success, such as the frequency of achieving clinical pregnancy based on fitting data from patient medical history (Hernández-González et al., 2018). However, most models used for its evaluation are based on patient data and their previous IVF protocols, often failing to track patterns in the changes of quality laboratory parameters relevant to the final transfer outcome (Fernandez et al., 2020; Li et al., 2023). The importance of the laboratory stage in IVF is detailed in consensus resolutions for quality control and the assessment of its key performance indicators (KPIs), constituting an integral part of internal quality control (QC) according to international standards (ISO) (ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, 2017). Nevertheless, determining such indicators to identify potential problem areas in laboratory work is not always straightforward, and clear recommendations on which KPIs need improvement for real increases in successful IVF protocol numbers are lacking (Fabozzi et al., 2020; Bormann et al., 2021). It is hard to disagree with the statement that laboratory and clinic efforts are directed not towards achieving KPIs above the benchmark values of consensus resolutions but towards the application and integration of this quality monitoring to increase the number of protocols resulting with embryo implantation and the birth of a healthy child. Modern infertility treatment methods involve a personalized approach to individual patients, their protocols, and to the specific embryos within them (Medenica et al., 2022). To facilitate this, many clinics have introduced medical information systems and time-lapse imaging equipment to monitor the fate of individual fertilized oocytes (Polanski et al., 2014; Racowsky et al., 2015; Jiang et al., 2023). As a result of applying artificial intelligence strategies, there is the possibility of additional embryo ranking for transfer and determining the chances of implantation (Uyar et al., 2015, Bamford et al., 2024). However, this approach entails the acquisition of additional expensive equipment: time-lapse incubators and software, as well as the allocation of additional human and time resources for the analysis of obtained images, the annotation of morphokinetic parameters, and the creation of an extensive database for each patient (Glatstein et al., 2023). The question of the feasibility of such an approach is beyond the scope of this study, but in practice it is not always fully realizable in the laboratory. Not always such algorithms can be extrapolated to patients from different clinics in various regions and countries, and for successful operation, they require additional validation in a specific laboratory with individual patient population characteristics (Blais, et al., 2021). However, there is very little open access data for performing validation assessments of decision-making systems like these. Another deal is that the majority of the studies on the effectiveness of time-lapse imaging are conducted on closed databases, which do not allow for an independent assessment of the quality of models offered as commercial solutions. To simplify the presentation of the necessary information for subsequent predicting, alternative methods have been developed, including principal component analysis for models of embryo selection, particularly those based on the analysis of previous protocols and the final morphology of the embryo before transfer (Coriani et al., 2013; Xi et al., 2021). However, most laboratory KPIs, used for quality control and competence level maintenance, are overlooked by these algorithms (Ratna et al., 2020). Additionally, their implementation still requires the availability of embryo images before transfer, which is often impossible to achieve in some clinics (Chavez-Badiola et al., 2020; Mushtaq et al., 2022; Liu et al., 2023). Taking into account the described limitations of existing systems, the aim of this study was to develop a different approach to laboratory data analysis, specifically utilizing deep neural networks (DNN) to predict the chances of clinical pregnancy occurrence in a specific protocol based on the combination of KPI and clinical data. The major reason to select DNN was that factors influencing the outcome of embryo transfer are not always obvious. Delving deeper into the methodologies of selecting the best embryo (with morphokinetic and “-omic’s” data) has a major drawback: with sufficient depth, there may be no embryos suitable for transfer left after such selection. It is evident that the interaction with the embryo outside the organism (in vitro) in laboratory conditions and its individual preimplantation development followed by implantation cannot be adequately described by conventional linear mathematical models (Cimadomo et al., 2023). Comprehensive approaches are necessary to approximate understanding, and it is potentially realizable based on deep neural networks models. DNN represents a powerful tool for data analysis, forecasting, and process automation. Their effectiveness depends on data quality, network architecture selection, and training process optimization. Neural networks consist of interconnected nodes called neurons, organized into layers: input, hidden, and output. The input layer receives data, hidden layers perform computations, and the output layer provides results or predictions. Each neuron is linked to neurons in the next layer through weights, determining the strength of the connection between neurons. Thus, the neural network finds a correct method of mathematical transformations to convert input data into output, regardless of their linear or nonlinear correlation. According to international data (ESHRE Special Interest Group of Embryology), the overall clinical probability of pregnancy after embryo transfer is approximately 30%, and this probability varies significantly in different clinics, depending on input parameters that are beyond our control. It also depends on the laboratory's activities, characterized by its quality indicators - Key Performance Indicators - applied in the quality management system for minimizing and controlling risks. --DIVIDER--# Related work Neural networks pipeline for quality management in IVF laboratory Journal of IVF-Worldwide 2024-10-23 | Journal article DOI: 10.46989/001c.124947 Part of ISSN: 2994-5488 Contributors: Sergei Sergeev; Iuliia Diakova; Lasha Nadirashvili Advanced KPI framework for IVF pregnancy prediction models in IVF protocols Scientific Reports 2024-11-27 | Journal article DOI: 10.1038/s41598-024-80759-7 Part of ISSN: 2045-2322 Contributors: Sergei Sergeev; Iuliia Diakova--DIVIDER--# Methodology Data collection: We used retrospectively collected data with known outcomes in our medical center “IVF and Genetic Center”, Moscow, Russia organized in the medical informative system Medworck 3.5 from January 2013 till January 2024 to develop the data set for DNN model: totally 8732 protocols (3856 protocols with fresh embryo transfer). Additional data from 2018 - 2024 (3500 protocols) and for PGT-A embryos (1600 protocols) was used for model internal validation. For external model validation we used data from 2 independent ART centers: “#1” Moscow, Russia with 6240 protocols and “#2” Tbilisi, Georgia with 3888 protocols. Informed consent for that study was not necessary because this study used only retrospective and fully de-identified data from embryo development protocol, fully noninvasive for patients or their embryos (no medical intervention was performed on the subject, and no biological samples from the patient were collected to develop that model). For embryo evaluation the ESHRE recommendations and Gardner blastocyst grading system was used. Good blastocysts were Bl3BB and higher grade on day 5. Statistical analysis: Statistical analysis of individual KPIs and PIs for data set description was conducted using the StatTech v. 3.0.6 software. Quantitative indicators were evaluated for normal distribution using the Shapiro-Wilk test. Quantitative indicators with a normal distribution were described using the mean and standard deviation (SD), with 95% confidence intervals. In cases where normal distribution was absent, quantitative data were described using the median (Me) and lower and upper quartiles (Q1 - Q3). Comparison of groups of embryologists based on quantitative indicators with a normal distribution was performed using one-way analysis of variance (ANOVA), with post hoc comparisons conducted using the Tukey's test and using the Kruskal-Wallis test for non-normal distribution. The prognostic model, characterizing the dependence of a quantitative variable on factors, was developed using the method of linear regression. Quantitative indicators were assessed for compliance with a normal distribution using the Shapiro-Wilk test. The direction and strength of the correlation between two quantitative variables were evaluated using the Spearman rank correlation coefficient. For statistical approach P-value <0.05 was used as a significant threshold limit in analysis of clinical features. Python 3.6, Scikit-learn 1.1 and Sklearn 1.4 were used to implement machine learning models and statistical modeling. A t-test used to determine if there is a significant difference between the means of two groups. Mann-Whitney U Test (Wilcoxon Rank Sum Test) was used to compare differences between two groups: model predicted values and real clinical pregnancy rate in deafferented patients groups, staff members and time periods. Dataset description: For the pretraining step of the neural network model for proprial neuron layers wights establishment we use our routine collected data from January 2013 till January 2024 for patients with fresh embryo transfer and knowing implantation data: totally 3856 protocols. All protocols with missing data values were discarded from the study. The mean patient age was 35.52 (SD 5.26, min 19, max 56) years; Attempt number mean 2.1; Number of follicles mean 8.86 (SD 5.95); OCC number mean 7.46 (SD 4.96); Number of oocytes used for fertilization mean 5.74 (SD 3.96); 2pN number mean 4.03 (SD 2.98), Number of cleaveged embryos mean 3.95 (SD 2.92); Total number of blastocyst D5-D6 mean 1.89 (SD 2.5); Number of Good quality blastocyst (3BB and high grade) mean 0.89 (SD 2.5); Number of transferred embryos mean 1.45 (SD 0.58); Number of cryopreserved embryos mean 1.05 (SD 1.53); Fertilization rate mean 0.73 (SD 0.25); Cleavege rate mean 0.96 (SD 0.16); Blastocyst rate mean 0.33 (SD 0.36); Good blastocyst rate mean 0.16 (SD 0.25); Oocyte retrieval rate mean 0.85 (SD 0.28); KPIScore mean 14.93 (SD 4.62) with mean Pregnancy rate per embryo transfer: 38%. The correlation between numeric parameters and KPIs is linear and understandable (Supl. material #1). But a correlation between KPIs is not so clear. We conducted a correlation analysis to assess the relationship between the "MII" indicator and the "ICSI 2pN" indicator: the strength of the association on the Chaddock scale was ρ = 0.605, p = 0.084. An increase of 1 unit in the "MII on ICSI" indicator is expected to result in a 0.525 increase in the "2pN" indicator; however, this relationship is not statistically significant. The obtained model explains 51.8% of the observed variance in the "ICSI 2pN" indicator. We also conducted a correlation analysis to assess the relationship between the "OKK" indicator and the "MII on ICSI" indicator: the strength of the association on the Chaddock scale was ρ = 0.882, p = 0.002. An increase of 1 unit in the "OKK" indicator is expected to result in a 0.852 increase in the "MII on ICSI" indicator. The obtained model explains 70.8% of the observed variance in the "MII on ICSI" indicator, and its results are statistically significant. Furthermore, we conducted a correlation analysis to assess the relationship between the "BL rate" indicator and the "Good BL rate" indicator: the strength of the association on the Chaddock scale was ρ = 0.678, p = 0.045. An increase of 1 unit in the "BL" indicator is expected to result in a 0.485 increase in the "Good BL" indicator. The obtained model explains 52.8% of the observed variance in the "Good BL" indicator, and its results are statistically significant. For the remaining features in our dataset, we were unable to identify linear correlations. Therefore, the application of standard machine learning methods to determine the probability of clinical pregnancy occurrence is complicated by the correlation of some prognostic features and the lack of linear dependence of their changes on the transfer outcome. This important observation underscores the complexities in data analysis in this field and highlights the need for employing more sophisticated methods of analysis or modeling to achieve precise prognostic outcomes. One such method is the use of deep learning neural network models. Their ability to extract intricate nonlinear dependencies from data and autonomously learn features can make them an effective tool for analyzing and predicting the probability of clinical pregnancy occurrence in the IVF protocol. Model Parameter Selection: The standard and first prediction model (Kaufmann et al., 1997) based on the following data: Age, Attempt number, Number of follicles, Number of oocytes retrieved, Number of inseminated oocytes, 2PN, Number of cleaved embryos, Number of blastocysts, was simulated in this study. Unfortunately, its accuracy and reproducibility are insufficient for predicting the likelihood of clinical pregnancy occurrence in the IVF protocol (loss: 0.6658 - accuracy: 0.5735). This led to the analysis of additional publications and studies involving the modeling of raw protocol data and its embryological component in the prediction of outcomes based on laboratory input parameters (Liu et al., 2023; Blank et al., 2019; Raef et al., 2020; Bormann et al., 2020; Hariton et al., 2021). The aim was to select optimal input parameters for predicting the outcome of embryo transfer. Our deep learning neural network model additionally incorporates data from the KPI of the Vienna Consensus, which shows a correlational dependence with the occurrence of pregnancy (Scaravelli et al., 2021). We also use fine-tuning KPI for total good blastocyst development rate (TGBDR) in our model according to publications (Zacà et al., 2022). TGBDR has positive correlation with all itralaboratory KPIs and negative correlation (-0.236) with the age of the patient. Model also includes the rankings of the sum of laboratory and clinical parameters (KPI Score), used with adaptation to our data (SART embryo grading and antral follicle on the day of trigger count) to forecast the chance of pregnancy occurrence (Franco et al., 2017) and a positive correlation (+0.785) with TGBDR. The developed neural network makes predictions based on 19 parameters (13 recorded in the laboratory database and available in the MedWork report, and 6 mathematically calculated, including the predictive KPIScore for implantation): "Age", "Attempt number", "Number of follicles", "Number of oocytes retrieved", "Number of inseminated oocytes", "2PN", "Number of cleaveged embryos", "Number of blastocysts", "Number of Good blastocysts", "Fertilization rate", "Cleavage rate", "Blastocyst formation rate", "Good blastocyst rate", "Oocyte retrieval rate", "Number of embryos on day 5", "Cryo embryos", "Transfer day", "Embryos transferred", "KPIScore". Evaluating feature importance allows identifying the most informative variables and focusing on them when building the model. Analysis of the results of feature importance evaluation for the neural network model, based on various evaluation methods was performed with: Information Gain - assesses the contribution of each feature to reducing the uncertainty of the target variable; Gini Coefficient - evaluates the heterogeneity in the data, and features with lower Gini coefficients are considered more important; Analysis of Variance (ANOVA) method - assesses the degree of variation in the target variable explained by each feature; Chi-square (χ²) was used to assess the relationship between categorical variables and ReliefF to evaluate feature importance in classification tasks. From the results of feature importance evaluation, it can be observed that some features have higher importance according to multiple methods. For instance, KPIScore, Age, and Transfer Day emerged as the most important features according to Information Gain, Gini coefficient, and ANOVA. This suggests that these features contain valuable information about the target variable and may be key for predicting the transfer outcome. Additionally, Fertilization rate is highlighted as an important feature according to the Chi-square method. Using XGBoost we confirm values correlation with pregnancy occurrence for selected parameters, considering their significance in predicting the outcome of embryo transfer. Despite the relatively low correlation of the transfer outcome with most of the laboratory KPI parameters, Spearman correlation analysis revealed the following relationships among them, which play a crucial role in selecting features for the neural network model prediction: 2 pN - this variable exhibits a high correlation with several other parameters, such as "cleavaged embryos" (correlation coefficient: +0.970), "Number of inseminated oocytes" (correlation coefficient: +0.889), and "Fertilization rate" (correlation coefficient: +0.273), it also demonstrates a relatively high correlation with the target variable "KPIScore" (correlation coefficient: +0.854). Good Bl rate - it strongly correlates with "Number of high-quality blastocysts" (correlation coefficient: +0.770) and plays a significant role in determining the quality of blastocysts formed. Number of blastocysts - shows high correlation with several other parameters, with correlation coefficients ranging from +0.821 to +0.952, and likely plays an important role in the process of embryo development. Blastocyst rate - It exhibits moderate correlation with several variables, including "Number of blastocysts" (correlation coefficient: +0.616), "cleavaged embryos" (correlation coefficient: +0.565), and "Number of inseminated oocytes" (correlation coefficient: +0.821), indicating its importance in evaluating the quality of embryo development. These correlations highlight the significance of these variables in predicting the outcome of the transfer and can be used as input parameters for our model. The analysis of linear dependencies for the parameters used in predictions makes it quite challenging to trace their influence on the outcome of embryo transfer. That's why we employ a deep neural network model in our work. However, it seems feasible to conduct a similar assessment for outgoing laboratory indicators – the frequency of forming high-quality blastocysts capable of implantation to understand what can be used as selected features for our DNN model predictions from the laboratory KPIs. To identify the parameters that have the greatest impact on blastocyst formation rate, we conducted a linear regression analysis using machine learning methods, with the target variable: the "Good blastocysts rate," depending on the number of retrieved oocytes, the number of inseminated oocytes, fertilization rate, and the total number of blastocysts. After conducting the regression analysis using the method of least squares (OLS) for the Good blastocyst rate, the following statistical data were obtained: R-squared (Coefficient of Determination): 0.393 - the model explains approximately 39.3% of the variation; The F-statistic value is 322.6, and the significance level (Prob (F-statistic)) is very low (< 0.0001). These values indicate that the model is statistically significant. Bl coefficient is 0.0670 - indicating the presumed change in the frequency of forming blastocysts when the number of blastocysts changes by one. The Durbin-Watson coefficient is close to 2, indicating the absence of significant autocorrelation in the model's residuals and the correctness of the analytical system. According to the regression analysis, negative dependencies of blastocyst formation rate on the number of oocytes retrieved and the total number of zygotes were identified, which aligns with the analytics of other IVF centers. It is worth noting that for the number of retrieved oocytes, the number of inseminated oocytes, and 2pN, model determining the correlation with blastocyst formation rate is not statistically significant (p = 0.841, p = 0.842, p = 0.880, respectively). An obvious significant linear dependency is found with the total number of blastocysts. Nevertheless, this is an important indicator of laboratory parameter stability, as an increase in the total number of blastocysts does not result in a deterioration of their quality but increases the probability (proportion) of forming high-quality blastocysts. These parameters, identified using linear regression analysis, can be considered as key and main important features for developing our neural network model for predicting the frequency of clinical pregnancy occurrence. Utilizing them as the foundation for training the neural network will allow us to more accurately and reliably forecast the probability of clinical pregnancy. Such an approach will ensure a more accurate tracing of the mutual influence of laboratory parameters and their impact on the final outcome of embryo transfer as a part of the internal quality control system. Ultimately, our neural network model will be able to provide clinic staff and patients with more precise and individualized predictions, contributing to a more effective outcome in the treatment of infertility based on laboratory quality and performance indicators. --DIVIDER--# Experiments Model Compilation: The neural network model was developed and executed in the GPU Google Colab environment using Python 3.5 programming language. The implementation utilized the Keras library, version 2.14.0. For the first model we take neural network architecture from published working models (Goyal et al., 2020). The architecture contains a total of 9 dense layers, each neuron output values In the first half of the model, neurons in each layer get increased precisely two times the previous layer. The second half of the net follows a decreasing rate of two neurons per layer, making the last layer one. Totally 32x64x128x256x512x256x128x64x32x1 neurons were used. That model proposed by the authors was fed with data from the Human Fertilisation and Embryology Authority (HFEA), comprising 25 clinical parameters, which, nevertheless, proved insufficient for describing the processes of individual embryo development in the laboratory. Despite achieving good results in predicting outcomes in our data (TEST Accuracy: 0.6749), the model proved to be highly complex and required significant optimization methods to prevent overfitting and fine-tuning. Adapting the model to our dataset posed a challenging task. The proposed neural network model is a sequential model consisting of two layers of SimpleRNN recurrent neurons and one layer of a dense neural network (Dense) with sigmoid activation, designed for solving a classification task. The model starts with an input layer that takes data with a dimension corresponding to the number of selected features. Then, hidden layers are added sequentially, each containing neurons with ReLU (Rectified Linear Activation) activation function, in addition to the final layer with a sigmoid activation function. The model structure was developed based on existing publications in the field of artificial neural networks in ART (Vogiatzi et al., 2019). However, in all these works, the authors considered input parameters as independent variables, which is unfortunately not applicable in our model. When using such models with sequentially connected neurons, their prediction accuracy does not exceed 63% (loss: 0.6797 - accuracy: 0.5764). Since KPI data and the final score prediction (KPIScore) are related to the model's input parameters, a recurrent type of RNN neurons was chosen for building the deep learning neural network to improve its accuracy (Benchaib et al., 2022). The RNN neurons exhibit higher average accuracy (0.7236) compared to Dense neurons (0.6362) (Fig.1). The RNN model shows a wider range of accuracy (0.5152 to 0.8750) compared to the Dense model (0.5325 to 0.7273), the standard deviation for accuracy is also higher in the RNN model (0.0764) compared to the Dense model (0.0410) and the RNN model achieves a higher maximum accuracy (0.8750) compared to the Dense model (0.7273). These differences are statistically significant: t-Statistics: 7.055 p-value: < 0.00001. The model architecture includes the following layers (Fig.2): Recurrent Neural Network (RNN) Layer: The first layer, SimpleRNN with 32 neurons and ReLU activation; the second layer, SimpleRNN with 16 neurons and ReLU activation; Dense Neural Network (Dense) Layer: The final layer with one neuron and Sigmoid activation, designed for binary classification. Sigmoid is used to predict the probability of a positive class (in this case, a positive outcome of the embryo transfer procedure). Added Dropout layer: this layer helps prevent model overfitting by randomly excluding neurons during training, thereby improving the model's generalization ability. L2 and L1 regularization: Applying L2 and L1 regularization to the corresponding layers of the model to control overfitting by adding penalties to the model's weights allows achieving greater accuracy and reproducibility of prediction results during training. Optimizer and loss function: This model is optimized using the gradient descent method with the Adam optimization algorithm and a specified learning rate of 0.0001. Binary cross-entropy is used as the loss function. The model is trained on a data array for 12 epochs (independently passes through the entire dataframe, splitting it into training and testing parts) with a batch size of 8. Model Training: The model was trained on the MedWork database over a 11-year period from 2013 to 2024. A total of 8732 complete protocols were available for analysis during this time. After selecting protocols containing all the necessary information for neural network training, 3858 cases of embryo transfer with known outcomes were used to form the training (80%) and validation (20%) set and 4874 protocols without transfer outcome were used for model pre-training process. We used a pre-training algorithm with partial teacher involvement for the purpose of propria weights in model forming. It was a first stage in the development of our neural network model training process. This method combines the advantages of both supervised and unsupervised learning with labeled and unlabeled data. This stage allows the model to learn complex dependencies in the data and select robust features more rapidly and effectively. Partial teacher involvement on the whole data enables the model to receive hints from the teacher during training, accelerating the convergence process and improving the quality of learning. This assists the model in better generalizing the acquired knowledge to new data and scenarios. Involving the teacher helps the model avoid overfitting traps and makes it more resilient to noise and variability in the data. After pre-training the model on the presented dataset with decision correction, sufficient accuracy and reproducibility were achieved, comparable to those of artificial intelligence models in time-lapse studies (Bormann et al., 2020; Louis et al., 2021; Salih et al., 2023): Validation accuracy: 0.67 - 0.77, SD 0.04 The obtained probability of success in predicting the developed model aligns with existing published systematic reviews and meta-analyses of data in the field of using neural networks in ART (Salih et al., 2023). Similar studies on the use of artificial intelligence models in predicting outcomes of ART programs have an average accuracy of 77.8% (range 68–90%). However, in the modeling process in this study, the average accuracy of such models for our database was only 63%. A comparison was conducted between the developed model and other machine learning approaches described in the literature on the same dataset, specifically those by Lu et al., 2023: Gradient Boosting Model Accuracy on Test Data: 0.6244 Random Forest Model Accuracy on Test Data: 0.5816 Decision Tree Model Accuracy on Test Data: 0.5479 The developed model exhibits a significant higher (U-Statistic: 230.0, p-value: 0.002) accuracy (0.67, AUC-ROC: 0.63; SD = 0.058) compared to the Gradient Boosting, Random Forest, and Decision Tree models on the provided test data. But lower than was published by Blank et al., 2018: Accuracy 0.74 ± 0.03 for the proposed RFM on their datasets that means a high difference between clinic’s data and importance of multicenter validation for all ML models before their practical implementation. On that step of training we observed comparative results with the van Loendersloot's model capacity AUC-ROC: 0.64 (CI: 0.61 -- 0.67) with the logistic regression calibration and an unconditioned logistic regression. i.e. the regression analysis between the estimated probability and the rate of pregnancy in that model (Sarais et al., 2016). For model propria training and fitting we added pregnancy results from cryo protocols with known implantation data to the previous dataset (totally all 8732 protocols were used): The mean patient age 36.02 (SD 5.53, min 19, max 56) years; Attempt number mean 2.5; Number of follicles mean 10.14 (SD 8.28); OCC number mean 8.36 (SD 6.55); Number of oocytes used for fertilization mean 6.16 (SD 5.07); 2pN number mean 4.35 (SD 3.73), Number of cleaveged embryos mean 4.07 (SD 3.69); Total number of blastocyst D5-D6 mean 2.27 (SD 2.96); Number of Good quality blastocyst (3BB and high grade) mean 1.16 (SD 1.89); Number of transferred embryos mean 2.05 (SD 2.03); Number of cryopreserved embryos mean 1.74 (SD 2.08); Fertilization rate mean 0.68 (SD 0.3); Cleavege rate mean 0.96 (SD 0.16); Blastocyst rate mean 0.37 (SD 0.36); Good blastocyst rate mean 0.19 (SD 0.27); Oocyte retrieval rate mean 0.85 (SD 0.23); KPIScore mean 14.96 (SD 5.27) with mean Pregnancy rate per embryo transfer: 38%. We split that data to train (70%), validation (20%) and test (10%) set. The stratified random sampling approach was used to ensure that all split data sets have the same distribution of pregnancy positive and negative classes. On this data a high stability of model accuracy was observed: Accuracy: 0.74 - val_accuracy: 0.73; Sensitivity: 0.6; Specificity: 0.66; PPV: 0.46; NPV: 0.78; FPR: 0.33; FNR: 0.4; Overall Accuracy (test): 0.71. The graphs presented depict the model training process and validation of prediction values using embryology laboratory KPI data and patient input parameters (Fig.3). The generated graphs demonstrate the correct model development strategy, justified efficiency assessment algorithm, and proper compilation of neural layers in it. The Model AUC (0.68; SD = 0.054) comparison with other AI models in the literature has revealed the advantage of the developed DNN in terms of time lapses for implantation probability prediction: 0.573 ( Chamayou et., al. 2013); 0.629 (Basile et al., 2015); 0.543 (Dal Canto et al., 2021); 0.638 (Bori et.al., 2022); 0.64 (Benchaib et al.,2022). Alternative Model Development: Additionally, to select the best neural network model for solving our task of predicting the likelihood of clinical pregnancy in an individual protocol, we applied an evolutionary algorithm for creating and training a neural network using genetic programming. The evolutionary algorithm with genetic programming is a powerful tool for optimizing the structure and parameters of neural networks, especially when their structure is complex and non-trivial. It allows for automatic creation and improvement of models without the need to manually adjust their architecture. The genetic approach to creating and training neural networks has several advantages compared to the classical gradient descent method: it can help combat overfitting by using selection, crossover, and mutation methods, which contribute to diversity and reduce the degree of model adaptation to training data; the absence of a predefined loss function makes it more flexible and applicable to tasks where the loss function is difficult to define or assess, particularly in medical forecasting tasks; such algorithm use random search and can explore a larger solution space than standard machine learning approaches; it allows for exploration and optimization not only of model parameters but also of its architecture, including the number of layers, number of neurons in each layer, and other structural aspects. Thus, the genetic approach to creating and training neural networks represents an alternative and powerful optimization method, which can be particularly useful in our study as an additional method for choosing the best neural network model development strategy. This approach is an optimization method where neural networks are created and improved by mimicking natural selection and genetic inheritance processes. This involves initializing a population of random neural networks, called individuals. Each individual represents a potential neural network with randomly selected architectural parameters such as the number of layers, number of neurons in each layer, activation functions, and other hyperparameters. Each model from the population is evaluated on the training dataset during neural network training. Then, individuals are selected based on the performance of each model so that the best models have a greater chance of being selected for reproduction. Selected individuals are then crossed to create new offspring. This can be done in various ways, including single-point or multi-point crossover, parameter blending, network structure modification, and so on. The goal is to create offspring that can inherit good characteristics from both parent models. In addition to crossover, mutation is applied, which randomly changes the structure of some models. This helps introduce diversity into the population and prevent getting stuck in local optimal points. The process of selection, crossover, and mutation is repeated for several generations. After completing the evolutionary process, the best individual is evaluated on the test dataset to assess its generalization ability and performance on new data. In our study, the following parameters of the evolutionary algorithm for building neural networks were used: the population size of 10 individuals. During each iteration, the best individuals are selected for crossover, but 10 individuals also remain in the new population. Thus, the number of individuals remains unchanged throughout all iterations. The number of neurons in the layers is randomly generated for each individual. For each layer, the number of neurons is randomly chosen from the range of 10 to 100. The code runs for 10 iterations. During the training of each model within each iteration, 10 epochs are used. As a result, the best model is selected based on the accuracy parameter for analysis of the test data. Unfortunately, the achieved accuracy of the described genetic approach was acceptable only during the evaluation of the validation set of our data: loss: 0.5448 - accuracy: 0.7032 - val_loss: 0.5374 - val_accuracy: 0.7287. At the same time, on new data from the test set, performance metrics were: loss: 0.6374 - accuracy: 0.6375, which is lower than the indicators for the developed classical DNN model with recurrent neurons (accuracy: 0.74 - val_accuracy: 0.73; Test accuracy: 0.71) and comparable only to the quality metrics for the process of its preliminary training (accuracy = 0.67). The obtained data indicate that the architecture of the DNN chosen by us utilysing simpleRNN neurons and regularization methods performs better in the classification task and can be considered for further validation and improvement of its generalization ability. Validation Analysis of the Model: A validation analysis of the model was conducted by running it on 12 randomly shuffled pseudo-randomly generated data frames, subjected to sequential and independent analyses. The model performance on an independent test: specificity in 96.8% of cases, with false-positive predictions occurring in 15.5% of cases and an average ability to correctly predict the occurrence of clinical pregnancy in 63% of cases. An analysis of the correlation between model accuracy and various age groups of patients (up to 29; 30-34; 35-39; over 40 years) was conducted, considering different methods of fertilization (IVF and ICSI). No significant differences were observed across the specified age groups when employing these diverse fertilization methods. In the group up to 29 years AUC = 0.59; 30-34 AUC = 0.66; 35-39 AUC = 0.68; over 40 years AUC = 0.71. This suggests that the developed model's accuracy remains consistent across age brackets and is not influenced significantly by the age of the patients. Same results were observed in time-lapse models The AUC of iDA-V2, iDA-V1 and Gardener grading model AUC = 0.736, 0.720 and 0.702, respectively (Ueno et al., 2024) from which only the iDA-V2 AUC score was significantly (p< 0.001) higher compared to our model. Additionally, the lack of observed differences in model performance based on the fertilization methods (IVF and ICSI) indicates that the model is robust and effective irrespective of the specific assisted reproductive technology used. Validation of the concordance between real and predicted values of pregnancy occurrence using this model was carried out on 1600 IVF protocols with PGT-A tested embryos at the validation dataset. An analysis was performed on how often the model correctly predicts the probability of clinical pregnancy occurrence – its accuracy is 0.7685 with an 86% sensitivity for this class (the proportion of actual positive results detected by the model out of all true positive results). The summary metric, which takes both accuracy and sensitivity into account (F1-score), is 0.71, comparable to the literature data of similar models (Liu et al., 2023). Cross-validation for the DNN model was conducted to evaluate the model on 50 different data splits (folds). We choose this fold number because real model implementation will plan to care on quartall timeline and our data set contains about 50 quarters for model training. Different combinations of training and test sets were used during cross-validation, providing a more robust and reliable assessment of model performance: Average accuracy: 0.68, Standard deviation of accuracy: 0.05, Maximum accuracy: 0.83 (Supplementary material #2). For our purposes, binary classification is used – pregnancy either occurred (1) or did not (0). To reduce the likelihood of erroneous predictions biased towards negative outcomes, a decision threshold of 0.48 was set, adjusted based on the confusion matrix so that real events corresponded to the calculated values (Fig.4). An analysis of the model's erroneous predictions was conducted through ROC analysis. The results were used to create an AUC graph, allowing an assessment of the quality of binary classification (Fig.5). We compare our neural network model to Logistic Regression, Gradient Boosting, Random Forest, and AdaBoost using leave one out (LOO) cross-validation. LOO cross-validation is a vital technique for evaluating the performance and reliability of neural network models, especially in the domain of predicting pregnancy occurrence in in vitro fertilization (IVF) protocols. Unlike traditional validation methods that partition data into training and testing sets, LOO cross-validation sequentially excludes each data point from the training set, using it as a testing instance while training the model on the remaining data. This iterative process ensures comprehensive evaluation of the model's performance on individual observations, leading to more robust results. LOO cross-validation not only enables assessment of model effectiveness across various scenarios and prediction of its performance on new data but also aids in identifying vulnerabilities and errors that might be overlooked with simple data partitioning techniques in individual protocol predictions. This is particularly crucial in medical applications like predicting pregnancy occurrence in IVF protocols, where precision and reliability of predictions are essential for informed decision-making and ensuring optimal patient outcomes. Our research demonstrates that the neural network has performed favorably across all considered metrics, notably standing out with a high MCC value. The Matthews Correlation Coefficient, unlike other metrics, accounts for class balance and enables model evaluation even with imbalanced data (Chicco et al., 2023). The high MCC value for the neural network (0.412) indicates its ability for precise and balanced classification, which is crucial in the context of predicting pregnancy occurrence in IVF protocols. This finding underscores the importance of selecting the neural network as the preferred model for this task, as it provides the highest level of accuracy and balance compared to alternative methods such as logistic regression, gradient boosting, random forest, and AdaBoost. Therefore, the neural network model represents the optimal choice for predicting pregnancy occurrence in IVF protocols, ensuring reliable and high-quality forecasts in individual treatment protocols. --DIVIDER--# Results Model Calibration After training a neural network model, we utilize the probabilities obtained from it and apply logistic regression for calibration. Calibration performed using CalibratedClassifierCV from scikit-learn, which applies logistic regression to align probabilities (Fig.6). Thus, we create a model that takes outputs from the neural network and calibrates them, improving the quality of probability predictions: Precision: 0.837 and Recall: 0.66. Validation process of that combined model on 50 epochs was used with the AUC and Precision - Recall curve (PRC) measurements for precision calibration of the threshold point for binary class classification. Choosing the threshold limit along the PRC is a critical step in the evaluation and optimization of classification models. This threshold determines the trade-off between precision and recall, two essential metrics in binary classification tasks. Selecting the threshold limit along the PRC involves finding the optimal balance between precision and recall based on the specific requirements and constraints of the problem of pregnancy ability classification. This balance depends on factors such as the relative importance of precision and recall in the context of the application, the consequences of false positives and false negatives, and the desired performance trade-offs. For us (and for QC/QA) it is much more important to understand when pregnancy did not occur than to count all positive results. False negatives are more critical for our work, so a lower threshold limit (we choose 0.48) is favored to maximize recall and minimize false negatives (Fig.7). The model was fine-tuned based on the database from 2018 to 2023 (3500 protocols with known implantation data) for all embryos with specification of oocyte source (autological or donor). During this period, a comprehensive quality control system was implemented in the laboratory, and there was no variability in approaches to OPU, embryo culture and transfer. Average Accuracy: 0.855, SD = 0.027, Maximum Accuracy: 0.914, Minimum Accuracy: 0.786, ROC-AUC: 0.86, SD = 0.061 (Fig.8). A comparative analysis of prediction errors was conducted between the developed model and the AI Vitrolife for time-lapse systems based on the characteristics of the ROC-AUC curve. During validation, the obtained model exhibited a similar ROC-AUC curve (U- Statistic: 145.0, p-value: 0.471) with an area of 0.67-0.75, comparable to the AI KIDScore™ (for cases of embryo transfer with known implantation data AUC = 0.66, AUC range of 0.60-0.75 for KID embryos across different clinics). Additionally, after fitting the model, it demonstrated a similar curve to the KIDScore™ (0.86 vs 0.89) for all time lapse embryos. This convincingly illustrates the equivalent accuracy in predicting the frequency of pregnancy occurrence for the developed model, with comparable precision to commercially utilized time-lapse systems (Fréour et al.,2015, Tran et al., 2019, Reignier et al., 2019, Berntsen et al., 2022) also fitted with the additional clinical data AUC = 0.72-0.78 (Benchaib et al.,2022, Lee et al., 2024). Possible discrepancies in the quality metrics of our developed neural network model and traditional time-lapse models can be explained by the different features used as variables for prediction outcomes. In our model, we have emphasized the importance of laboratory parameters tracked during the analysis of KPIs. Meanwhile, authors of time-lapse models utilize morphokinetic data of individual embryo development for forecasting. Essentially, both approaches describe the dynamics of embryo changes during preimplantation development but from different perspectives. Traditional embryo quality assessment, combined with KPI consideration, enables reasonably accurate predictions, akin to constant video monitoring in time-lapse. However, utilizing embryo developmental kinetics data involves analyzing a large number of their images, which, unlike calculated KPIs or static images, introduces significant noise into the primary data and its final assessment. Additionally, it's challenging to compare morphokinetic algorithms developed for assessing blastocysts and cleaving embryos when forming implantation predictions on different days of transfer. Thus, our model allows looking at the formation path of necessary features for assessing the chances of clinical pregnancy occurrence using machine learning from a different perspective—precise and individual evaluation of KPIs in contrast to often noisy images evaluated in time-lapse systems. Currently, it's difficult to confidently assert whether our chosen algorithm is better or worse than traditionally used ones—additional multicenter studies are required for this. However, in any case, it enables a more comprehensive evaluation of implantation predictors related to intra-laboratory conditions and is more easily interpretable within the framework of quality control programs compared to time-lapse assessment. Using this model with logistic regression calibration, it is possible to calculate the probability of pregnancy occurrence in the dataset. Corrections to Positive Predictive Value (PPV) and Negative Predictive Value (NPV) were applied to the calculated pregnancy occurrence frequency, determined during the model validation. This approach allows the characterization of the patient population at a specific time and performance analysis based on the correspondence between real results and the calculated pregnancy occurrence value. The stability of the model structure was verified by performing five calibration procedures on the same variables and in the same order as the final output. On 5 fold cross validation the DNN model demonstrates high specificity (86.4%) and an average ability to correctly predict clinical pregnancy (Average accuracy: 78,13%, Standard deviation of accuracy: 4%, Maximum accuracy: 87.5%). The accuracy of predicting clinical pregnancy and sensitivity in the model compares favorably with the Irvine Scientific Life Whisperer model (VerMilyea et al., 2020): accuracy for predicting clinical pregnancy 76.85%, sensitivity of 70.1% for viable embryos while maintaining a specificity of 60.5%, combined accuracy of 64.3%; and with FiTTE model: accuracy 65.2%, ROC-AUC = 0.71 (Enatsu et al., 2022). And has a significantly ( p< 0.01) better ROC-AUC = 0.86 (0.82-0.89) than traditionally used predictive models AUC = 0.6335 (0.6202 -- 0.6367) (Nelson S.M., Lawlor D.A. 2011, Ratna et al., 2023) and AI algorithm based on conventional Gardner embryo grading by experienced embryologists with accuracy 59.8% AUC = 0.62 used as control model performance indicator in FiTTE model development (Enatsu et al., 2022). --DIVIDER--# Discussion We compare our DNN model with three fine-tuned pre-trained convolutional neural network (CNN) models for pregnancy prediction in IVF protocol with enhanced ICM and TE images and different neural networks architecture utilized in ImageNet: VGG16, ResNet50, DenseNet121. The best performance of them was obtained in the ResNet50 architecture in reconstructed image data, with an AUC = 0.741, Accuracy = 0.682, Sensitivity = 0.7114, Specificity = 0.669. Two others CNN showed comparative but slightly lower results: AUC = 0.684, Accuracy = 0.654, Sensitivity = 0.760, Specificity = 0.609 for VGG16 and AUC = 0.669, Accuracy = 0.669, Sensitivity = 0.694, Specificity = 0.648 for DenseNet121 (Kim et al., 2024). Only sensitivity of them was higher than in our DNN model because of real blastocyst images utilization in the CNN models. So, those embryos that did not develop to the blastocyst stage were excluded from their observation. The obtained model can be used to analyze real patient data for any time period according to our data. For example, the model's output would be represented in the "predictions.xlsx" table, with indicators of a complete match, representing pregnancies contrary to the model's expectation, and indicating cases requiring in-depth analysis of unsuccessful transfers. To prove that statement we compare the DNN model predictions with the KPIScore regression model after its validation on our data. We used the threshold value for pregnancy occurrence established by KPIScore authors (Franco et al., 2017): a cumulative score of 15, taking into account the confidence interval for implantation in that group of patients was 16%-30%. There was a significant difference (p<0.0001) with respect to the total KPI mean score among the group of patients with clinical pregnancy (total KPIScore = 21) and the group without clinical pregnancy (total KPIsscore = 15). According to that, we obtained statistically different probabilities of pregnancy occurrence (t-test = 38.786, p < 0.0001) in those groups of our patients: for positive pregnancy it was with the mean 0.6309 (CI = 0.227) and 0.4266 (CI = 0.251) for nonpregnant. When comparing the predictions of the developed model and KPIScore along (A/B model stat test), a 10% difference was observed between them. Despite minor discrepancies in the accuracy of the overall predictions between our neural network model and logistic regression for KPIScore, the mean squared error between predicted and actual values of the latter amounts to 22% (MSE = 0.219), while the coefficient of determination value of 0.142 indicates that the model of KPIScore along explains only 14.2% of the variance in the dependent variable, thus rendering it unreliable in our situation as an independent variable apart from the rest of the dataset. Investigating the significance of predictions from the individual KPIScore model revealed the statistical significance of KPIScore (F-statistic = 462.7, p < 0.0001), enabling its use as a predictor for our neural network model and aligning with published data by KPIScore authors with total KPIScore = 22 for positive clinical pregnancy result and total KPIScore < 18 for negative on our data set. Furthermore, we compare the odds ratio of KPIScore (1.24) and iDAScore (1.11) to ongoing pregnancy probability and came to the conclusion that the KPIScore system can be applied in our model as the same feature - the “embryo score” in iDAScore v2.0 and iDAScore v1.0 time-lapse models (Ueno et al., 2024). To understand the predictive capability of our pregnancy occurrence model, we selected a reference group from our dataset. This group consisted of cycles with the transfer of a single euploid embryo following Preimplantation Genetic Testing for Aneuploidy (PGT-A). Thus, we achieved an initial balance of classes in our patient subpopulation, with an average implantation rate of 54.6%. An analysis of pregnancy occurrence frequency in those protocols revealed a 27% error in predicting clinical pregnancy rate AUC = 0.67 (0.62-0.75), Accuracy: 0.7685 (Fig.9). The same metrics (AUC = 0.67) were obtained for embryonic and clinical outcomes of blastocysts with stratified AI scores including PGT-A and the predictive ability of iDAScore with respect to euploid or mosaic embryo selection - the combination of iDAScore, type of chromosomal abnormalities, and PI level (Chun et al., 2024). Same results were obtained with other Resnet CNN models: AUC = 0.65 for fresh and frozen non-PGT transfersand 0.63 for euploid transfers (Loewke et. al., 2022) These results can provide the statement that our model accuracy is comparable with PGT-A tested embryo transfers probability prediction 60-70% and our DNN model can be used as a fully noninvasive additional method for embryo selection as other CNN models (Enatsu et al., 2022) and that its quality metrics are not linked with unbalanced data set data distribution. Finally, our model validation AUC = 0.67-0.75 is higher than in Meseguer group model IDAScore V2 for the same patient category - AUC = 0.654 (Bori, L., Meseguer, et al., 2022, Ueno et al., 2024) and comparative with the Eeva model with versus without sibling features establish that cohort properties contribute to implantation prediction - AUC: 0.698 and 0.744 (Tzukerman et al., 2022) respectively. Predictive ability of the model is higher then TL features in Eeva algorithm with AUC: 0.53 - 0.61 for individual prediction and AUC: 0.64 for combined (Sayed et al., 2020) and comparative to GERI AI overall accuracy of 67.8%, AUC: 0.61-0.65 (Diakiw et al., 2022) and MIRI AI model AUC: 0.69 (Duval et al., 2023). These values, while allowing a general comparison to be made with similar reported artificial intelligence models, e.g. Alife Health artificial intelligence model ROC-AUC: 0.62–0.64 (Erlich et al., 2022), Fairtility artificial intelligence model ROC-AUC: 68–0.70 (Loewke et al., 2022) and CHLOE-EQ model ROC-AUC: 64–0.726 (Fruchter-Goldmeier et al., 2023). In the process of validating the accuracy of pregnancy occurrence prediction, a quarterly analysis for 2022-2023 was conducted, comparing the model's predictions with the real clinical report and statistics (Fig.10). The average difference between the calculated predictions and reports was 2.56% that were not significant (U-Statistics = 20.0, p value = 0.62). These results convincingly demonstrate the effectiveness and reproducibility of the model and the prediction algorithm for the outcome of transfers embedded and have higher results of AUC = 0.86 (0.82-0.89) compare with TL models prediction with logit regression AUC = 0.632 (95% CI 0.599–0.665) for SET data and AUC = 0.624 (95% CI 0.579–0.669) for DET data (Sayed et al., 2020); and comparative with ML spatial stream model (AUC = 0.76), temporal stream model (AUC = 0.77), and ensemble model STEM (AUC = 0.82) (Liao et al., 2021). After logistic regression calibration we compare our DNN model with the Large-scale simulation model of K. Loewke reported at the Annual Meeting of the ESHRE (Cho et al., 2022). It has the same results of the average pregnancy rate of the top-ranked embryos (60% vs 59.4%) with the minimum pregnancy possibility of 18.4% in low grade embryos that can be used as a powerful tool for pregnancy rate improvement from using the AI models in routine clinic practice. The DNN model pregnancy predictions are the same as “ALIFE IVF Success Predictions model” with the mean difference of pregnancy prediction performance and life birth rate of 6.34% in a good prognosis patient group (McLernon et al., 2022). Compared to the available ALIFE online calculator for success rate cont in IVF protocol our DNN model provided lower results of pregnancy probability and they are much more reliable with our real pregnancy rate (43%) in that group of patients. Achieved results showing the real importance of laboratory KPIs to precise model prediction in individual protocols that are realized in our DNN model and it is not so optimistic than predictions based only on clinical data in ALIFE model. Ofcouse available ALIFE model trained on a different patient population than ours DNN and real model comparison may hold on only after the same training data both models are tuned. This is a future perspective for our team. External validation For external validation we used significantly different data on patient population distribution (KPIScore t-statistics -9.05378, p-value <0.0001) from independent 2 centers from different countries (Russia #1 and Georgia #2). From the protocols provided by clinics #1 and #2, those executed by the current team of embryologists present in the laboratory were selected (2 embryologists from clinic #1 and 4 from clinic #2). Criteria for excluding protocols: incomplete data, errors in information provision, and procedures performed by multiple embryologists. Protocols satisfying the Vienna consensus criteria were chosen to prepare the dataset for individual staff KPI analysis. All parameters, except the patient's age, exhibited distributions different from normal (R-test), and according to the Dickey-Fuller criterion, their time series were non-stationary (p > 0.05). Therefore, non-parametric statistical methods (ANOVA) were utilized to identify statistically significant differences among staff performance results, particularly using the Mann-Whitney test. Analysis of the workload distribution in the laboratory among staff based on input (number of oocytes used in procedures) and output (number of obtained blastocysts) parameters with a significance level of p-value = 0.05 revealed no significant differences in pairwise comparisons between all embryologists in all clinics. Quarterly analysis (for the years 2013-2023 for clinic #1 and 2022-2023 for clinic #2) of input parameters for the laboratory, specifically the main criterion of properly selected stimulation - MII rate, showed stability in the obtained mature oocytes (med: 87%, Q1-Q3: 0.855-0.893) above the Maribor consensus competency threshold (lower competency threshold ≥74%, target value ≥90%), and their invariance across years. Statistically significant differences in the main clinic KPIs between IVF and ICSI procedures were not found for the fertilization rate (U statistic: 4687.0, p-value: 0.2945), blastocyst formation rate (U statistic: 6096.5, p-value: 0.0528), good-quality blastocyst formation rate (U statistic: 4920.5, p-value: 0.90306), and availability of oocytes for fertilization (U statistic: 5404.0, p-value: 0.8479), allowing the use of these datasets as reference examples for validating the training process of our model in a specific IVF centers. No statistically significant differences in achieving key quality indicators were detected between the selected ART centers based on the aggregate calculation of individual embryologist indicators. When evaluating "IVF polyspermy" depending on the "Embryologist" indicator, no significant differences were found (used method: Fisher's F-test). Similarly, comparing "Good blastocysts rate" depending on the "Embryologist" indicator revealed no significant differences (used method: Fisher's F-test). Analysis of "ICSI Deg" depending on the "Embryologist" indicator also did not reveal significant differences (used method: Kruskal-Wallis test). Comparative "ICSI fertilization rate" depending on the "Embryologist" indicator, has no significant differences (used method: Fisher's F-test). Assessing "IVF fertilization rate" depending on the "Embryologist" indicator also failed to establish statistically significant differences (used method: Kruskal-Wallis test). Furthermore, analyzing the distribution of individual embryologist KPIs showed the median values of fertilization frequency above the Vienna consensus benchmark for all conducted cycles, with quarterly values (Q1-Q3) above the competency level and the rest of the KPI parameters above the competency level. External validation of the model on the dataset from an independent clinic in Russia (#1) with the majority of poor-prognosis patients (2013 - 2023 years): 49662 embryos, 6240 protocols were used. The mean patient age 38.3 (SD 5.33, min 18, max 54) years; Number of follicles mean 7.54 (SD 5.63); OCC number mean 4.92 (SD 3.82); Number of oocytes used for fertilization mean 4.0 (SD 3.0); 2pN number mean 3.47 (SD 2.86), Number of cleaveged embryos mean 3.4 (SD 3.09); Total number of blastocyst D5-D6 mean 1.25 (SD 1.16); Number of Good quality blastocyst (3BB and high grade) mean 1.2 (SD 1.2); Fertilization rate mean 0.72 (SD 0.35); Cleavege rate mean 0.99 (SD 0.13); Blastocyst rate mean 0.44 (SD 0.38); Good blastocyst rate mean 0.42 (SD 0.28); Oocyte retrieval rate mean 0.83 (SD 0.7); KPIScore mean 13.51 (SD 4.44) with mean Pregnancy rate per embryo transfer: 28.43%, Predicted PR 26.09% (p-value: 0.2185). Metrics for the model after fitting on that clinic dataset: ROC-AUC = 0.73, Test Accuracy: 76.34%, Precision: 0.47, Recall: 0.44, F1 Score: 0.45. Results for 2022-2023 years were presented on the graphs (Fig.11). External validation of the model on the dataset from another clinic in Georgia (#2) with good prognosis patients and majority of oocyte donor cycles: 3888 protocols were used. The mean patient age 34 (SD 5.7, min 18, max 49) years; Number of follicles mean 16.3 (SD 9.64); OCC number mean 13.04 (SD 7.32); Number of oocytes used for fertilization mean 11.19 (SD 6.68); 2pN number mean 8.15 (SD 5.37), Number of cleaveged embryos mean 8.12 (SD 5.36); Total number of blastocyst D5-D6 mean 3.33 (SD 3.43); Number of Good quality blastocyst (3BB and high grade) mean 3.14 (SD 3.67); Fertilization rate mean 0.74 (SD 0.31); Cleavege rate mean 0.99 (SD 0.4); Blastocyst rate mean 0.40 (SD 0.4); Good blastocyst rate mean 0.34 (SD 0.35); Oocyte retrieval rate mean 0.78 (SD 0.5); KPIScore mean 19.49 (SD 4.89) with mean Pregnancy rate per embryo transfer: 61.93%, Predicted PR 56.18% (p-value: 0.1144), Metrics for the model after fitting on that clinic dataset: ROC-AUC = 0.63, Test Accuracy: 65.55%, Precision: 0.58, Recall: 0.47, F1 Score: 0.52 (Fig.12). We performed an analysis of obtained laboratory's KPIs stability according to their prognostic influences on pregnancy rate. The oocyte retrieval rates were observed until the third quarter of 2023, followed by a decline in this parameter in August 2023. Concurrently, a significant decrease in the blastocyst formation rate was noted, indicating a reduced probability of forming high-quality blastocysts from each retrieved oocyte. From the beginning of 2022 to the second quarter of 2022, there was a clear trend of increasing oocyte fertilization rates and blastocyst formation rates. These data correlate with the increased embryo implantation frequency observed during this period. A similar trend of increased blastocyst formation frequency was also observed in 2023, although the decrease in blastocyst formation frequency in the third quarter may seem notable, when comparing quarterly averages between years. Such a decrease is not statistically significant and amounts to less than 15%. This suggests that there are no substantial changes in the laboratory conditions leading to a reduction in the overall number of high-quality blastocysts. Thus, a significant difference in pregnancy probability was noted for patients in 2021-2022 compared to 2023, ranging from a 10% decrease in the likelihood of clinical pregnancy compared to the beginning of the year to a 20% decrease compared to patients in the program after the third quarter of 2023. The theoretically calculated probabilities using the DNN model align with the actual pregnancy frequency reports in the specified time intervals. In other words, our KPI calculation and DNN model prediction analysis demonstrate that the decrease in clinical pregnancy rates from the third quarter of 2023 is a process not directly related to the quality of stimulation (patient preparation) or laboratory work but depends on the patients' initial clinical data. The external validation of our model has comparative results of AUC: 0.73 with a hybrid AI model 3D-ResNet based on several TL technologies (MIRI, GERI and EMBR-EMBR+) in cross validation report from 14 clinics (AUC: 0.73) and with video models along (AUC: 0.68) from the same data (Duval et al., 2023). The model application in 4 different clinics in different countries with different patient population KID datasets showed AUC: 0.58 - 0.80 that is comparable to TL models in external validation studies with evaluation metric AUC: 0.67-0.8 for deafferent transfer days (Blais, et al., 2021) and 0.57 - 0.78 (Johansen et al., 2023). Monitoring the performance of assisted reproductive technology staff using DNN model: Quality assurance (QA) measures are key to reliably providing high-quality care over time and among different practices (Cherouveim et al., 2023). According to Maribor consensus (ESHRE Clinic PI Working Group 2021): “Competence and benchmark values of clinical pregnancy rate should be set for a specific local context”. Unfortunately this is not a simple task for an individual IVF center. To address this QA goal, we utilized our DNN model and its predictions. In many cases, it is challenging to differentiate between patients seeking assistance from various doctors within the clinic, making the specification of competency boundaries in achieving quality targets quite blurry and uncertain. However, our developed model employs a grouped analysis of protocols, allowing their aggregation not only based on temporal criteria but also on individual staff members. We compared the actual clinical pregnancy rates achieved individually by each reproductive specialist during the year 2023. In this scenario, we raised the decision classification threshold from 0.48 to 0.5 and used the predictions of the calibrated neural network model to determine the lower competence level of the possible pregnancy occurrence rate. Through this approach, we identified three doctors (No. 1, No. 2, and No. 3) with actual clinical pregnancy rates (34.0%, 42.5%, 40.5%) higher than the theoretically calculated threshold (33.60%, 33.33%, 38.01%) in their patient groups (p >0.05). Additionally, we identified three doctors (No. 4, No. 5, and No. 6) whose transfer outcomes (9.60%, 24.82%, 16.01%) require serious monitoring and verification as their actual clinical pregnancy rates were significantly lower (p <0.01) than those predicted by the model (33.53%, 33.33%, 37.25%) for these staff members (Fig.13). The doctors whose performance falls below the defined competency level are typically young, inexperienced members of the clinic staff. Based on the analysis conducted, it can be concluded that the competency level of doctors No. 4, 5, and 6 still does not allow them to work independently, and every procedure conducted by them requires careful monitoring and mentoring from more experienced colleagues, who can be specialists within the same center under numbers 1, 2, and 3. Thus, the developed neural network model is an integral, accurate, and reproducible element of QA and can be used for both internal QA and external clinic audits, as well as for determining the individual competency of their staff. With its help, it is possible to define the boundaries of individual competency and identify staff members whose work requires increased scrutiny, as well as those who can perform such monitoring. The importance of this analysis lies in the fact that the role of a mentor can be applied to existing clinic staff without the need for additional external experts or auditors, significantly simplifying organizational matters and reducing direct financial expenditures on staff training. Especially intriguing in this context is the application of the neural network model by centers with multiple laboratories, as it would standardize the approach to assessing the quality of each lab and establish unified standards for its indicators, based on reasonable and realistic variables, for control by the head organization of the company while maintaining the individual characteristics of patient distribution both between clinics and among their doctors. DNN model application for internal QC/QA The developed neural network model can serve as a unique and comprehensive tool for internal quality control of laboratory parameters and clinic performance. For instance, during significant changes in working conditions or the implementation of new protocols. To assess this capability, we chose two periods of operation for our clinic: before relocation from January 1, 2019, to November 1, 2020 (Old), and after relocation from November 15, 2020, to January 1, 2023 (New). Before November 2020, the laboratory operated in a standard facility without adequately equipped ventilation systems and the application of comprehensive quality control measures to all procedures. After the relocation in November 2020, the laboratory operated in a new facility equipped according to cleanroom standards ISO 15189, with the implementation of a complete quality control system (TQM), audit, and traceability of biological materials and consumables. It is therefore of interest to analyze data from these time intervals to comparatively assess the effectiveness of our work and track changes associated with the laboratory's transition to a qualitatively new level. For this evaluation, we applied the capabilities of our AI model with automatic KPI calculation and embryo transfer outcome predictions. We conducted an analysis of KPI indicators based on the Vienna Consensus criteria, depending on the "Period" indicator (Old vs New). When comparing the "OCC" indicator: Me Old = 76 (Q1-Q3 = 76-77) vs Me New = 83 (Q1-Q3 = 82-84), p = 0.026; "MII": Me Old = 73 (Q1-Q3 = 72-77) vs Me New = 80 (Q1-Q3 = 80-82), p = 0.027; "Good Bl rate": Me Old = 23 (Q1-Q3 = 22-25) vs Me New = 33 (Q1-Q3 = 30-36), p = 0.027, statistically significant differences were established (using the Mann-Whitney U test). When comparing "IVF 3pN": Me Old = 7 (Q1-Q3 = 7-8) vs Me New = 8 (Q1-Q3 = 8-9), p = 0.621; "1pN IVF": Me Old = 2 (Q1-Q3 = 2-3) vs Me New = 2 (Q1-Q3 = 2-4), p = 0.521; "1pN ICSI": Me Old = 2 (Q1-Q3 = 1-2) vs Me New = 3 (Q1-Q3 = 2-3), p = 0.126; "ICSI Deg": Me Old = 4 (Q1-Q3 = 3-6) vs Me New = 8 (Q1-Q3 = 7-9), p = 0.081; "ICSI 2pN": Me Old = 70 (Q1-Q3 = 68-70) vs Me New = 67 (Q1-Q3 = 65-67), p = 0.266; "IVF 2pN": Me Old = 54 (Q1-Q3 = 52-56) vs Me New = 59 (Q1-Q3 = 54-60), p = 0.447; "0 PN": Me Old = 6 (Q1-Q3 = 9-11) vs Me New = 10 (Q1-Q3 = 4-10), p = 0.264; "Cleavage rate": Me Old = 94 (Q1-Q3 = 93-96) vs Me New = 94 (Q1-Q3 = 93-94), p = 0.434; "D2": Me Old = 53 (Q1-Q3 = 51-59) vs Me New = 59 (Q1-Q3 = 55-60), p = 0.539; "D3": Me Old = 38 (Q1-Q3 = 35-41) vs Me New = 47 (Q1-Q3 = 47-48), p = 0.317; "BL rate": Me Old = 59 (Q1-Q3 = 55-61) vs Me New = 67 (Q1-Q3 = 63-72), p = 0.142; "IR D3": Me Old = 17 (Q1-Q3 = 16-17) vs Me New = 19 (Q1-Q3 = 17-23), p = 0.319; "IR D5": Me Old = 35 (Q1-Q3 = 28-43) vs Me New = 38 (Q1-Q3 = 35-40), p = 0.623 - we failed to identify significant differences (using the Mann-Whitney U test). For each embryologist, a similar analysis of statistically significant changes in their KPIs over the selected time periods was conducted. When analyzing the "IVF 3pN" indicator (p = 0.739); "Frequency of good blastocyst formation" (p = 0.718); "ICSI Deg" (p = 0.430); "ICSI 2pN" (p = 0.064); "IVF 2pN" (p = 0.413); "Cleavage rate" (p = 0.622); "Blastocyst rate" (p = 0.575); "Utilization rate" (p = 0.271) - we failed to identify significant differences (using the Fisher's F-test). During the specified time period, the actual pregnancy rate was calculated as follows: in 2019 (623 protocols) 43.9%; in 2020 before relocation (439 protocols) 35.2%; in 2021 after relocation (542 protocols) 41.3%; and in 2022 (357 protocols) 28.9%. The rate of multiple pregnancies by year was 8.8%; 4.6%; 7.3%; 2.6% respectively. The predicted clinical pregnancy rate according to the neural network model was: 2019 - 32.74%; 2020 - 29.04%; 2021 - 33.55%; 2022 - 28.66%. Despite the stability of laboratory indicators between the periods of the old and new centers, a significant reduction in the probability of pregnancy occurrence was observed in 2022. A detailed comparative analysis of KPI within the laboratory demonstrates the stability of all working conditions and results regardless of embryologists in the specified time intervals. With the relocation to the new facility, there was an increase in the formation of high-quality blastocysts. However, having more good blastocysts in the protocols and an increased chance of their formation in individual programs did not lead to an increase in the pregnancy occurrence rate. This fact requires careful investigation, specifically comparing patient groups participating in IVF programs. Using our neural network model, we established a lower threshold for the probability of clinical pregnancy occurrence for each year of operation. This approach demonstrated that the patient subpopulation differed between 2019, 2021 (more promising cohort with a lower pregnancy occurrence threshold of 33%) and between 2020, 2022 (less promising cohort with a lower pregnancy occurrence threshold of 29%). The observed real pregnancy occurrence pattern has a similar trend: an average of 42.6% in the prospective period of 2019 and 2021, and 32.1% in 2020 and 2022. However, despite the fact that the actual clinical pregnancy rate was above the threshold value determined by the neural network model in all periods, areas for improving the effectiveness of fertility assistance in the clinic are evident. The difference between the actual and predicted clinical pregnancy rates in 2019 was 11.2%; in 2020 – 6.2%; in 2021 – 7.8%; and in 2022 only 0.24%. This fact aligns with the change in the clinic's medical staff in 2022: two highly qualified reproductive specialists terminated their collaboration with the clinic, and new doctors without experience, requiring training and additional supervision, took their place. According to the observed temporal changes in pregnancy occurrence predictions, the dynamics of this quality indicator cannot be solely explained by patient selection with unfavorable outcomes or changes in the embryology laboratory's operation. In other words, the quality control of physicians' work requires significant modification and improvement, and with the commencement of independent work by new specialists, it proved insufficient. This example is presented as a real demonstration of the possibilities of applying the developed neural network model in the quality control system. With our model, it is possible to justify the need for specific decisions in quality monitoring issues accurately. The described approach allows for the development of effective mechanisms for personnel management and tracking their competency levels, including identifying key performance indicator levels that require concern with investigating the reasons for their decrease or taking urgent measures to eliminate unfavorable factors. DNN model for external auditing, troubleshooting and identifying clinic operation issues: Data analysis was conducted using the procedure records from the embryology laboratory of the IVF Clinic #3 (Kaliningrad, Russia), spanning from November 2022 to November 2023, comprising 172 patient IVF protocols. When analyzing quarterly changes in the probability of pregnancy occurrence, the following data were obtained: 4Q 2022: 0.35; 1Q 2023: 0.36; 2Q 2023: 0.43; 3Q 2023: 0.40; 4Q 2023: 0.42. The actual clinical pregnancy rates achieved were: 1Q 2023: 0.38; 2Q 2023: 0.50; 3Q 2023: 0.26. A comparison between the actual clinical pregnancy rate and the rate calculated based on the neural network model for each quarter of 2023 was conducted using the Mann-Whitney test: P-value: 0.4857, indicating no statistically significant differences between the theoretical and actual pregnancy rates in the first three quarters of 2023. However, there was a significant decrease (p <0.05) in the percentage of successful cycles in the fourth quarter (Q4 expected clinical pregnancy rate 42%, actual 26%), raising concerns and audit necessity of the clinic with a deep analysis of clinical and laboratory indicators. An analysis of patient distribution by age groups and determination of the mean patient age 35.9 years was conducted for the selected time period. To compare the effectiveness of IVF programs between two specific time periods, a t-test statistical method was applied, revealing no statistically significant differences in indicators between the selected time periods: Age: t-Statistic: -0.612, p-value: 0.5412; AMH level: t-Statistic: 0.5919, p-value: 0.5548; Number of oocytes: t-Statistic: 0.2, p-value: 0.8413; 2PN: t-Statistic: -0.1497, p-value: 0.8812; Number of good blastocysts: t-Statistic: -0.8940, p-value: 0.3727; Fertilization rate: t-Statistic: -0.9597, p-value: 0.3387. The predictive probability of clinical pregnancy occurrence is determined based on KPIscore data: the average total KPIScore was found to be among the group of patients with clinical pregnancy (total KPIScore = 20.4±3.7) and the group without clinical pregnancy (total KPIScore = 15.9±5). The probability of clinical pregnancy occurrence can be obtained using the DNN model with the total KPIScore as the predictor variable. When comparing the overall value of the predicted probability of clinical pregnancy occurrence generated by the neural network model between the selected time periods, no statistically significant differences were found. In all 171 provided protocols, the mean expected clinical pregnancy rate was 40% (SD = 0.04). The lower limit of the confidence interval can be used as the target variable in determining the clinical pregnancy rate for this patient population, with a competency threshold of 36% as the quality indicator threshold. In the period from August 2023, the calculated median value (0.4) of the predicted clinical pregnancy probability suggests a trend towards a possible increase in this threshold, indicating a lack of improvements in implantation frequency related to primary patient data and laboratory parameters. The observed decrease in clinical pregnancy rate during this period requires close attention to the clinical aspects of the IVF process in this center. The highest proportion of patients with a mean age of 38 years and sufficient ovarian reserve based on AMH level (2 ng/ml) during this time period is notable, implying the expectation of obtaining 10-15 oocytes in the stimulation protocol and at least 2 good-quality blastocysts for subsequent transfer. With the regression model a linear correlation is observed between the number of retrieved oocytes and zygotes with normal fertilization, and between the fertilization rate and the number of good-quality blastocysts, indicating stability in intralaboratory parameters. The absence of a visible correlation between patient age and fertilization rate allows for a preliminary conclusion regarding the quality work of the embryology team regardless of patient age characteristics, with the distribution skewing towards 1 in all comparison groups confirming this observation. The distribution pattern of performance indicators remains invariant between the selected comparison time periods. The observed decrease in clinical pregnancy rates in September 2023 aligns with the theoretical data on the possible frequency of pregnancy occurrence, which was 27% in that month compared to August 2023, showing potential differences of up to 20% between these months. However, this does not indicate instability in the clinic or embryology laboratory's operations (Fig.14). Based on the median of the clinical pregnancy rate distribution over two years of operation, it is advisable to establish a target value of this parameter in this clinic at 37%. Statistically significant decreases in clinical pregnancy rates should be determined by the 95% confidence interval values for previous indicators, and investigations should commence when the clinical pregnancy rate falls below 31.1% for transfer. In the first three quarters of 2023, the actual clinical pregnancy rate exceeded that predicted by the neural network model. However, in the third quarter, it was lower than the permissible 20% decrease obtained for September patients (compared to August), necessitating a more detailed analysis of other performance indicators. Even considering that September patients with a poor prognosis and a calculated theoretical clinical pregnancy rate of 27% mostly had delayed transfers, such significant changes in pregnancy occurrence dynamics should not be evident from the presented data. Nevertheless, the previously established concern threshold at 37% is higher than the obtained real values. The fourth quarter is composed solely of October data, demonstrating a bleak picture with a decrease in the clinical pregnancy rate to 18%. According to the probability of pregnancy occurrence analysis using the neural network model, preliminary conclusions can be drawn, and the main directions for the upcoming external audit of the clinic can be identified. If the combined clinical pregnancy rate for October and November reaches 40%, the third quarter can be considered an unsuccessful period with outlier values (the number of procedures conducted in it was less than 20). However, if the combined clinical pregnancy rate for October and November falls below the 37% confidence interval threshold, immediate investigation into the underlying causes should proceed, with particular focus on the clinical aspect of embryo transfer. Thus, our neural network model provides the ability to determine the real potential (theoretically justified) probability of pregnancy occurrence in a specific patient group. Its application as a quality control tool is justified primarily in terms of accurately defining threshold values for the competency level in pregnancy occurrence frequency for an individual clinic with its patient subpopulation characteristics. The application of the model not only allows us to identify specific time frames required for analysis during audits but also assesses the significance of laboratory data for transfer outcomes and identifies areas of increased interest during external quality control assessments. Adds-on with KPI forecasting Calculating the laboratory's KPI during the data preparation procedure allows for the utilization of machine learning methods to forecast key performance indicator changes with a justified mathematical model and the determination of reference threshold values for quality indicators and the identification of growth zones for potential performance improvement. SARIMA (Seasonal AutoRegressive Integrated Moving Average) is a time series method widely used for forecasting data with seasonal fluctuations. In the context of embryology laboratory and data preparation procedure for neural network predictions, SARIMA can be employed to forecast major KPI changes over a selected period. To achieve this, we have integrated the SARIMA model into the laboratory's KPI calculation process. Data collection and preparation occur during their processing for neural network analysis, including information on the time periods they were conducted and the values of the indicators for individual protocols. Following this step is the analysis of time series to identify their structure, including seasonal fluctuations, trends, and noise. Based on such analysis, a suitable SARIMA model is chosen. This selection involves determining the orders of autoregression (p), differencing (d), and moving average (q) for the time series components, as well as SARIMA seasonal orders (P, D, Q) and seasonality period. The model is then trained on historical data to forecast future performance indicator values based on past observations. These forecasts can be used to calculate target KPIs for the individual laboratory and analyze its effectiveness. Subsequent comparison of forecasted KPIs with actual data forms the basis for a systematic analysis of laboratory efficiency. As new data becomes available, the models can be periodically updated, allowing adjustments to forecasts based on changing conditions and requirements. This approach to internal quality control provides the laboratory with a tool for more predictable process management, early detection of potential issues, and optimization of operations based on data and analytics. Explanation of the Model results The importance of explaining and interpreting the results of neural network models used in reproductive medicine is critical for ensuring transparency and trust both from patients and medical professionals. Understanding which factors influence the probability of clinical pregnancy is particularly important to ensure the most effective treatment and improve outcomes. In the context of approaches to interpreting the results of DNN models, we have utilized SHAP (SHapley Additive exPlanations) and ICE (Individual Conditional Expectation). SHAP is a method for explaining machine learning models based on the concept of Shapley values. It provides explanations for each prediction of the model, indicating the importance of each feature in this context. By randomly selecting individual protocols with positive and negative outcomes, we demonstrated the ranking of input features according to their influence on the final classification of the neural network model's predictions. The features with the highest weights were: Embryo transfer day, KPIScore, Patient age, Number of transferred and cryopreserved embryos, TGBDR. ICE (Individual Conditional Expectation) allows visualizing how model predictions change for individual observations as the values of a specific feature change, given fixed values of the other features. This helps identify nonlinear dependencies and interactions between features, which can also be useful for refining predictions and developing personalized treatment approaches. LIME (Local Interpretable Model-agnostic Explanations) method allows the creation of independent local, interpretable explanations for individual model predictions. This is particularly important in our situation, where machine learning models are applied in tasks where not only prediction accuracy matters but also understanding the reasons behind their decisions. The main idea of the LIME method is to approximate complex machine learning models with locally interpretable models, such as linear models. To do this, LIME generates a set of artificial "neighbors" around the individual observation of interest and uses them to train a local model that approximates the behavior of the main model in the vicinity of this observation. Then LIME analyzes the parameters of the local model to understand which features were most important for making the specific prediction and provides local explanations for specific observations, allowing us to understand why the model made a particular decision for a given case. Such application of the method will be especially valuable in complex cases for establishing the decision threshold based on the classification of neural network models. By combining these methods, we were able to determine the average minimum conditions of the laboratory stage of IVF for the possibility of pregnancy occurrence in the protocol. In other words, we were able to interpret the finding of the probability prediction value of our model above the accepted decision threshold and its relationship with key input data for the model. For TGBDR, the minimum number of blastocysts for successful transfer was 1, Number of follicles - 4, OCC – 3, MII – 2, 2pN – 2, Number of D3 embryos – 2, Blastocysts – 2, Good quality blastocysts – 2, Transferred embryos – 1. For KPIScore, its minimum value for pregnancy occurrence was determined as 15, and the minimum indicators were: Number of follicles - 3, OCC – 2, MII – 2, 2pN – 2, Number of D3 embryos – 2, Blastocysts – 1, Transferred embryos – 1. For the primary clinical factor - patient's age: Number of follicles - 4, OCC – 3, MII – 1, 2pN – 1, Number of D3 embryos – 1, Blastocysts – 1, Good quality blastocysts – 1, Transferred embryos – 1. For our DNN predictions the following threshold values were obtained: KPIScore - 15, Number of follicles - 4, OCC – 3, MII – 2, 2pN – 2, Number of D3 embryos – 2, Number of D5 embryos – 1, Good quality blastocysts – 1, Transferred embryos – 1 and Patient Age – 36. With the LIME algorithm we observed that for positive pregnancy result it is necessary to reach KPIScore - 17, OCC retrieval rate = 0.83, MII rate - 0.78, Fertilization rate - 0.64, Blastocyst development rate - 0.44, Totall Good blastocyst development rate - 0.35. It is remarkable that those obtained values are consistent with Vienna and Maribor consensus opinion for good IVF practice. These results are aligned with Bologna Criteria (Ferraretti et al., 2011) for poor responders (< 3 oocytes retrieved) and POSEIDON - Patient-Oriented Strategies Encompassing Individualized Oocyte Number Criteria for prognosis of the treatment cycle results (Esteves et al., 2019). In that model the ability to retrieve the number of MII oocytes needed to obtain at least one euploid blastocyst for transfer in each patient with the implantation rates in the range of 50–60% was established with AUC = 0.70. The minimum number of MII oocytes required in the POSEIDON model was 5 (95% CI: 5–6) with validation of ART calculator in multicenter study (Esteves et al., 2020). Low prognosis probability of clinical pregnancy rate with the threshold of Number of follicles - 5, MII - 4 with follicle-to-oocyte index (FOI) <50%, D3 embryos - 4 and minimum 1 good quality blastocyst for transfer was approved by other research groups (Roque et al., 2021; Lee et al., 2022). Results of Regression Analysis of neural network predictions In this section, we present the results of applying linear regression to evaluate the probability of pregnancy predicted by the neural network for its interpretability increasement with the simplest approach. We assessed the model's quality using several metrics, including mean squared error (MSE) and coefficient of determination (R^2). We chose linear regression for analyzing the probabilities predicted by the neural network for several reasons. Firstly, linear regression is a simple and interpretable method. Secondly, it allows us to assess the influence of each predictor on pregnancy probability, which is crucial in our study. Lastly, linear regression provides the opportunity to evaluate the statistical significance of predictors and test their adequacy for modeling the target variable. Based on these factors, we consider the use of linear regression to analyze pregnancy probabilities predicted by the neural network to be a suitable and justified approach. The dependent variable was the predicted probabilities of pregnancy, and the independent variables were the parameters used in the model. We obtained an MSE value of 0.00267, indicating that our model has a low mean squared error. This suggests that the model's predictions are close to the actual values of pregnancy probability. The coefficient of determination (R^2) was found to be 0.937, indicating that approximately 93.7% of the variance in pregnancy probability is explained by the independent variables in the model. This confirms the model's high predictive ability. To assess the significance of the predictors' influence on pregnancy probability, we analyzed the regression coefficients and their statistical significance. Out of the 15 regression coefficients analyzed, 13 were found to be statistically significant at the p < 0.05 level. This implies that they make a significant contribution to explaining the variation in pregnancy probability predicted by the neural network model. The most influential predictors on predicted pregnancy probability were variables such as the frequency of receiving high-quality embryos (coefficient 0.1115), KPIScore index (0.0147), blastocyst formation rate (0.0847), and cleavage rate (0.0442). The positive coefficients obtained from the regression analysis indicate that increasing the values of these predictors significantly contributes to an increase in the probability of pregnancy. The obtained results align well with the biological interpretation of factors influencing the outcome of in vitro fertilization. Analysis and Interpretation of Clusters To deepen our understanding of the predictive capabilities of the developed neural network model for IVF pregnancy probability, we conducted k-means clustering analysis on the model's predicted output values across the dataset comprising 8732 samples. This exploratory approach aimed to identify inherent patterns in the predicted pregnancy probabilities, offering insights into the diverse patient profiles represented within the dataset. The clustering process partitioned the cases into three distinct clusters, each characterized by unique combinations of predictive features: Cluster 0 - High Pregnancy Probability (55.1% of samples). Key features driving the high pregnancy probability in this cluster include: Good Quality Blastocyst Formation Rate (Weight: 0.2251) - a significant contributor to the favorable outcomes observed in this cluster is the high rate of successful blastocyst formation, indicating robust embryo development. Number of Quality Embryos Obtained (Weight: 0.0598) - the quantity and quality of embryos obtained play a pivotal role in determining the likelihood of successful implantation and pregnancy. KPIScore (Weight: 0.1554) - the score, reflecting the overall quality and developmental potential of embryos with patient clinical data combination, emerges as a crucial predictor of pregnancy success. Number of Cryo Embryos (Weight: 0.2197) - the presence of surplus high-quality embryos available for cryopreservation further augments the chances of achieving pregnancy through subsequent transfer cycles. Number of Embryos Transferred (Weight: 0.1868) - the decision to transfer multiple embryos underscores the clinician's confidence in the viability of the embryos and the patient's likelihood of achieving a successful pregnancy. These findings suggest that Cluster 0 encompasses cases characterized by favorable embryo development trajectories and abundant high-quality embryos, contributing to the elevated predicted probability of pregnancy. Cluster 1 - Intermediate Pregnancy Probability (30.9% of samples). Notable features distinguishing this cluster include: Extremely High Weight on Number of Embryos Dividing on Day 3 (Weight: 0.556) - the remarkably high emphasis on the number of embryos exhibiting successful division on day 3 highlights the critical role of early embryonic development dynamics in determining pregnancy outcomes. Lower Weights on Blastocyst Formation and Morphology: Unlike Cluster 0, where blastocyst formation rate held substantial importance, Cluster 1 exhibits relatively lower weights on indicators of blastocyst morphology and development quality. These findings suggest that Cluster 1 comprises cases with favorable early-stage embryo development but potentially less robust blastocyst formation, contributing to the intermediate predicted probability of pregnancy. Cluster 2 - Moderate Pregnancy Probability (48.7% of samples). Key features driving the pregnancy probability within this cluster include: Number of Blastocysts Obtained (Weight: 0.0833) - the quantity of blastocysts obtained serves as a significant predictor of pregnancy success, reflecting the availability of viable embryos for transfer. Blastocyst Quality (Weight: 0.065) - the quality assessment of blastocysts plays a crucial role in determining their implantation potential and subsequent pregnancy outcomes. Number of D3 Embryos (Weight: 0.1626) - the presence of high-quality embryos suitable for transfer positively influences the likelihood of achieving a successful pregnancy. These findings suggest that Cluster 2 comprises cases with a balanced combination of embryo quantity and quality, contributing to the moderate predicted probability of pregnancy observed within this cluster. In summary, the comprehensive analysis of feature weights within each predicted probability cluster provides valuable insights into the multifaceted determinants of IVF pregnancy success. Cases with good quality embryos and development are assigned higher chances, while indicators of early embryo arrest lead to lower predictions. By elucidating the differential contributions of various patient and treatment-related factors, this approach enhances the interpretability of the neural network model's predictions and informs clinical decision-making in assisted reproductive practice aligned with real clinical data. For a deeper analysis of the effectiveness of our neural network model in predicting pregnancy probability in IVF, we compared predicted clusters with clusters based on real data on pregnancy occurrence in our clinic. This allowed us to identify consistency between predicted and real results, as well as to identify differences that may indicate important aspects in the data. Both sets of clusters contain similar sets of features characterizing stimulation quality indicators and patient clinical data, such as age, number of follicles, and number of oocytes. We also observe similar trends in the distribution of feature weights characterizing pregnancy probability. However, there are distinct differences between predicted and real clusters. For example, when comparing Cluster 0, which demonstrates the highest predicted pregnancy probability, with the corresponding real cluster, we see that in the predicted cluster, higher weights are assigned to features such as "Good-quality blastocyst formation rate" (weight 0.2251), "Frequency of obtaining good-quality embryos" (weight 0.0598), and "KPIScore" (weight 0.1554), compared to the real cluster. This may indicate that our neural network model pays more attention to these features when predicting pregnancy probability than is actually observed. Nevertheless, such an approach in the model's predictions allows us to speak about the importance of considering intralaboratory parameters for predicting transfer outcomes, which is consistent with the need to consider consensus KPIs published in the Vienna and Maribor Consensus, and the model itself primarily evaluates the dynamics of such parameters. The idea of choosing parameters and subsequent training of our model was precisely aimed at tracing this relationship. This aspect convincingly demonstrates the importance of laboratory KPI monitoring, and the model itself with SARIMA KPI forecus is primarily used as an indispensable quality control tool in embryology laboratories. The comparison of predicted and real clusters is an important step in assessing the effectiveness of our neural network model. The observed differences and similarities between the clusters can help better understand which factors truly influence pregnancy probability in IVF and how they can be accounted for in building prognostic models. These results underscore the importance of further research for more accurate prediction of pregnancy outcomes in IVF. --DIVIDER--# Areas of Application of the Model The developed model allows for a detailed analysis of selected protocols with unsuccessful pregnancy outcomes while simultaneously providing a real-time working system for assessing the KPIs and PIs of the laboratory. The model monitors statistical KPIs to evaluate the efficiency of individual embryologists and culture conditions. Discrepancies detected in the model's report can be used for systematic and early identification of unfavorable results and clinically significant changes in laboratory performance indicators. The Model AUC (0.68; SD = 0.054) for multiple IVF clinics implies that the neural network model not only excels in its predictive capabilities but also demonstrates efficiency in terms of processing speed when compared to alternative AI models discussed in the literature including commercial time-lapse systems (t-Statistics: -0.74, p-value = 0.46). The use of this neural network model is beneficial in terms of accuracy and interpretability, and it also stands out in terms of time efficiency, making it a favorable choice in practical applications. The accuracy data of the model's outcome predictions surpass the analogous accuracy of predicting transfer outcomes by experts based on clinical and laboratory parameters—51% (range 43–59%) according to literature data (Fig.15). This enables its application for remote monitoring of the ART department's performance and serves as an effective mechanism for the clinic's overall quality control. The developed deep learning neural network after logistic regression calibration for all embryos has high AUC = 0.86 (0.82-0.89) that not only facilitates the necessary monitoring of internal quality control but also possesses sufficient predictive power for analyzing IVF protocols. It can be used for retrospective analysis of work results as well as for prospective evaluation of the chances of clinical pregnancy occurrence, proving to be promising in assisting medical decision-making. For instance, the model performs predictions even for protocols where embryo transfer did not take place, but cryopreserved embryos are available. This model allows assessing the success chances of a cryo cycle in a particular program and deciding on the advisability of additional stimulation and a new IVF attempt or the prospect of cryotransfer, determining the need for further storage of vitrified embryos. The results of the regression analysis confirm the correctness of selecting the most significant predictors and the adequacy of the developed neural network model for predicting pregnancy probability during IVF treatment. The significance and positive direction of influence of the selected predictors confirm their role in explaining the variation in the target variable, allowing us to confidently use the model's predictions for practical purposes, such as selecting optimal infertility treatment strategies. During the execution of the model code, the laboratory's embryology KPI is automatically calculated according to the recommendations of the Vienna Consensus. This serves as a convenient tool for internal quality control, allowing for quick response and decision-making regarding changes in the process when deviations from reference values are detected. The report is output in the form of a table "predictions.xlsx," and its data is specific to a particular IVF cycle. The next use of SARIMA (Seasonal Autoregressive Integrated Moving Average) models provides the opportunity for further forecasting of obtained KPIs and subsequent comparison with actual performance metrics. This approach allows for the development of a versatile and effective tool for precise internal quality control in the laboratory based on recorded quality performance indicators. SARIMA models are a powerful time series tool that takes into account trends and seasonal variations in data. Forecasting KPIs using SARIMA enables the laboratory to analyze expected indicators in the future, identify potential risks, and take measures to manage the quality of operations. The model can be utilized as a tool for quality control and the success of ART programs. It provides insights into cases where a transfer should have resulted in pregnancy if the expected and actual pregnancy rates do not align — there are valid reasons for identifying factors hindering implantation. From the laboratory perspective, this involves monitoring KPIs, automatically calculated by the model; from the clinical side, it involves comparing the actual pregnancy rate with the theoretically calculated one. Model limitations: The developed model primarily serves as a tool for internal quality control and allows for determining the realistically possible probability of clinical pregnancy occurrence within a specific period or in a certain patient group based on a combination of clinical and laboratory parameters from retrospective data. In situations where there are indeed issues related to decreased clinical pregnancy rates, the model performs better, enabling the establishment of a lower competence threshold akin to a KPIs tracking and analysis system for an individual clinic. Personalized fitting of the model on the clinic's database is required for a specific ART center for model validation and accurate threshold limit establishment. Conclusions: The developed deep learning neural network model successfully passed all validation tests, demonstrating comparable capabilities to AI systems used in ART for correctly predicting the chance of clinical pregnancy occurrence in a specific IVF protocol and the overall pregnancy rate over a selected period and can be personalized to clinic staff. The model allows for detailed tracking of laboratory KPIs and their impact on the transfer outcome and can serve as a comprehensive QC/QA tool for quality management programs. --DIVIDER--# References 1. Glatstein, I., Chavez-Badiola, A., & Curchoe, C. L. (2023). New frontiers in embryo selection. Journal of assisted reproduction and genetics, 40(2), 223–234. https://doi.org/10.1007/s10815-022-02708-5 2. Alizadehsani, R., Roshanzamir, M., Hussain, S., Khosravi, A., Koohestani, A., Zangooei, M. H., Abdar, M., Beykikhoshk, A., Shoeibi, A., Zare, A., Panahiazar, M., Nahavandi, S., Srinivasan, D., Atiya, A. F., & Acharya, U. R. (2021). Handling of uncertainty in medical data using machine learning and probability theory techniques: a review of 30 years (1991-2020). Annals of operations research, 1–42. Advance online publication. https://doi.org/10.1007/s10479-021-04006-2 3. Hernández-González, J., Inza, I., Crisol-Ortíz, L., Guembe, M. A., Iñarra, M. J., & Lozano, J. A. (2018). Fitting the data from embryo implantation prediction: Learning from label proportions. Statistical methods in medical research, 27(4), 1056–1066. https://doi.org/10.1177/0962280216651098 4. Fernandez, E. I., Ferreira, A. S., Cecílio, M. H. M., Chéles, D. S., de Souza, R. C. M., Nogueira, M. F. G., & Rocha, J. C. (2020). Artificial intelligence in the IVF laboratory: overview through the application of different types of algorithms for the classification of reproductive data. Journal of assisted reproduction and genetics, 37(10), 2359–2376. https://doi.org/10.1007/s10815-020-01881-9 5. Li, L., Cui, X., Yang, J., Wu, X., & Zhao, G. (2023). Using feature optimization and LightGBM algorithm to predict the clinical pregnancy outcomes after in vitro fertilization. Frontiers in endocrinology, 14, 1305473. https://doi.org/10.3389/fendo.2023.1305473 6. ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine. Electronic address: coticchio.biogenesi@grupposandonato.it (2017). The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reproductive biomedicine online, 35(5), 494–510. https://doi.org/10.1016/j.rbmo.2017.06.015 7. Fabozzi, G., Cimadomo, D., Maggiulli, R., Vaiarelli, A., Ubaldi, F. M., & Rienzi, L. (2020). Which key performance indicators are most effective in evaluating and managing an in vitro fertilization laboratory?. Fertility and sterility, 114(1), 9–15. https://doi.org/10.1016/j.fertnstert.2020.04.054 8. Bormann, C. L., Curchoe, C. L., Thirumalaraju, P., Kanakasabapathy, M. K., Gupta, R., Pooniwala, R., Kandula, H., Souter, I., Dimitriadis, I., & Shafiee, H. (2021). Deep learning early warning system for embryo culture conditions and embryologist performance in the ART laboratory. Journal of assisted reproduction and genetics, 38(7), 1641–1646. https://doi.org/10.1007/s10815-021-02198-x 9. Medenica, S., Zivanovic, D., Batkoska, L., Marinelli, S., Basile, G., Perino, A., Cucinella, G., Gullo, G., & Zaami, S. (2022). The Future Is Coming: Artificial Intelligence in the Treatment of Infertility Could Improve Assisted Reproduction Outcomes-The Value of Regulatory Frameworks. Diagnostics (Basel, Switzerland), 12(12), 2979. https://doi.org/10.3390/diagnostics12122979 10. Polanski, L. T., Coelho Neto, M. A., Nastri, C. O., Navarro, P. A., Ferriani, R. A., Raine-Fenning, N., & Martins, W. P. (2014). Time-lapse embryo imaging for improving reproductive outcomes: systematic review and meta-analysis. Ultrasound in obstetrics & gynecology : the official journal of the International Society of Ultrasound in Obstetrics and Gynecology, 44(4), 394–401. https://doi.org/10.1002/uog.13428 11. Racowsky, C., Kovacs, P., & Martins, W. P. (2015). A critical appraisal of time-lapse imaging for embryo selection: where are we and where do we need to go?. Journal of assisted reproduction and genetics, 32(7), 1025–1030. https://doi.org/10.1007/s10815-015-0510-6 12. Jiang, Y., Wang, L., Wang, S., Shen, H., Wang, B., Zheng, J., Yang, J., Ma, B., & Zhang, X. (2023). The effect of embryo selection using time-lapse monitoring on IVF/ICSI outcomes: A systematic review and meta-analysis. The journal of obstetrics and gynaecology research, 49(12), 2792–2803. https://doi.org/10.1111/jog.15797 13. Uyar, A., Bener, A., & Ciray, H. N. (2015). Predictive Modeling of Implantation Outcome in an In Vitro Fertilization Setting: An Application of Machine Learning Methods. Medical decision making : an international journal of the Society for Medical Decision Making, 35(6), 714–725. https://doi.org/10.1177/0272989X14535984 14. Bamford, T., Smith, R., Young, S., Evans, A., Lockwood, M., Easter, C., Montgomery, S., Barrie, A., Dhillon-Smith, R., Coomarasamy, A., & Campbell, A. (2024). A comparison of morphokinetic models and morphological selection for prioritizing euploid embryos: a multicentre cohort study. Human reproduction (Oxford, England), 39(1), 53–61. https://doi.org/10.1093/humrep/dead237 15. Cimadomo, D., Rienzi, L., Conforti, A., Forman, E., Canosa, S., Innocenti, F., Poli, M., Hynes, J., Gemmell, L., Vaiarelli, A., Alviggi, C., Ubaldi, F. M., & Capalbo, A. (2023). Opening the black box: why do euploid blastocysts fail to implant? A systematic review and meta-analysis. Human reproduction update, 29(5), 570–633. https://doi.org/10.1093/humupd/dmad010 16. Blais, I., Koifman, M., Feferkorn, I., Dirnfeld, M., & Lahav-Baratz, S. (2021). Improving embryo selection by the development of a laboratory-adapted time-lapse model. F&S science, 2(2), 176–197. https://doi.org/10.1016/j.xfss.2021.02.001 17. Corani, G., Magli, C., Giusti, A., Gianaroli, L., & Gambardella, L. M. (2013). A Bayesian network model for predicting pregnancy after in vitro fertilization. Computers in biology and medicine, 43(11), 1783–1792. https://doi.org/10.1016/j.compbiomed.2013.07.035 18. Xi, Q., Yang, Q., Wang, M., Huang, B., Zhang, B., Li, Z., Liu, S., Yang, L., Zhu, L., & Jin, L. (2021). Individualized embryo selection strategy developed by stacking machine learning model for better in vitro fertilization outcomes: an application study. Reproductive biology and endocrinology : RB&E, 19(1), 53. https://doi.org/10.1186/s12958-021-00734-z 19. Ratna, M. B., Bhattacharya, S., Abdulrahim, B., & McLernon, D. J. (2020). A systematic review of the quality of clinical prediction models in in vitro fertilisation. Human reproduction (Oxford, England), 35(1), 100–116. https://doi.org/10.1093/humrep/dez258 20. Chavez-Badiola, A., Flores-Saiffe Farias, A., Mendizabal-Ruiz, G., Garcia-Sanchez, R., Drakeley, A. J., & Garcia-Sandoval, J. P. (2020). Predicting pregnancy test results after embryo transfer by image feature extraction and analysis using machine learning. Scientific reports, 10(1), 4394. https://doi.org/10.1038/s41598-020-61357-9 21. Mushtaq, A., Mumtaz, M., Raza, A., Salem, N., & Yasir, M. N. (2022). Artificial Intelligence-Based Detection of Human Embryo Components for Assisted Reproduction by In Vitro Fertilization. Sensors (Basel, Switzerland), 22(19), 7418. https://doi.org/10.3390/s22197418 22. Liu, H., Zhang, Z., Gu, Y., Dai, C., Shan, G., Song, H., Li, D., Chen, W., Lin, G., & Sun, Y. (2023). Development and evaluation of a live birth prediction model for evaluating human blastocysts from a retrospective study. eLife, 12, e83662. https://doi.org/10.7554/eLife.83662 23. Kaufmann, S J et al. “The application of neural networks in predicting the outcome of in-vitro fertilization.” Human reproduction (Oxford, England) vol. 12,7 (1997): 1454-7. doi:10.1093/humrep/12.7.1454 24. Vogiatzi, Paraskevi et al. “An artificial neural network for the prediction of assisted reproduction outcome.” Journal of assisted reproduction and genetics vol. 36,7 (2019): 1441-1448. doi:10.1007/s10815-019-01498-7 25. Louis, Claudio Michael et al. “Review of computer vision application in in vitro fertilization: the application of deep learning-based computer vision technology in the world of IVF.” Journal of assisted reproduction and genetics vol. 38,7 (2021): 1627-1639. doi:10.1007/s10815-021-02123-2 26. Benchaib, M., Labrune, E., Giscard d'Estaing, S., Salle, B., & Lornage, J. (2022). Shallow artificial networks with morphokinetic time-lapse parameters coupled to ART data allow to predict live birth. Reproductive medicine and biology, 21(1), e12486. https://doi.org/10.1002/rmb2.12486 27. Lee, C. I., Huang, C. C., Lee, T. H., Chen, H. H., Cheng, E. H., Lin, P. Y., Yu, T. N., Chen, C. I., Chen, C. H., & Lee, M. S. (2024). Associations between the artificial intelligence scoring system and live birth outcomes in preimplantation genetic testing for aneuploidy cycles. Reproductive biology and endocrinology : RB&E, 22(1), 12. https://doi.org/10.1186/s12958-024-01185-y 28. Bormann, Charles L et al. “Performance of a deep learning based neural network in the selection of human blastocysts for implantation.” eLife vol. 9 e55301. 15 Sep. 2020, doi:10.7554/eLife.55301 29. Salih, M et al. “Embryo selection through artificial intelligence versus embryologists: a systematic review.” Human reproduction open vol. 2023,3 hoad031. 15 Aug. 2023, doi:10.1093/hropen/hoad031 30. Liu, Xiaoyan et al. ``Construction of the machine learning-based live birth prediction models for the first in vitro fertilization pregnant women.” BMC pregnancy and childbirth vol. 23,1 476. 27 Jun. 2023, doi:10.1186/s12884-023-05775-3 31. Chicco D, Jurman G. The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification. BioData Min. 2023 Feb 17;16(1):4. doi: 10.1186/s13040-023-00322-4. PMID: 36800973; PMCID: PMC9938573. 32. Sarais, V., Reschini, M., Busnelli, A., Biancardi, R., Paffoni, A., & Somigliana, E. (2016). Predicting the success of IVF: external validation of the van Loendersloot's model. Human reproduction (Oxford, England), 31(6), 1245–1252. https://doi.org/10.1093/humrep/dew069 33. Chamayou, S., Patrizio, P., Storaci, G., Tomaselli, V., Alecci, C., Ragolia, C., Crescenzo, C., & Guglielmino, A. (2013). The use of morphokinetic parameters to select all embryos with full capacity to implant. Journal of assisted reproduction and genetics, 30(5), 703–710. https://doi.org/10.1007/s10815-013-9992-2 34. Dal Canto, M., Bartolacci, A., Turchi, D., Pignataro, D., Lain, M., De Ponti, E., Brigante, C., Mignini Renzini, M., & Buratini, J. (2021). Faster fertilization and cleavage kinetics reflect competence to achieve a live birth after intracytoplasmic sperm injection, but this association fades with maternal age. Fertility and sterility, 115(3), 665–672. https://doi.org/10.1016/j.fertnstert.2020.06.023 35. Basile, N., Vime, P., Florensa, M., Aparicio Ruiz, B., García Velasco, J. A., Remohí, J., & Meseguer, M. (2015). The use of morphokinetics as a predictor of implantation: a multicentric study to define and validate an algorithm for embryo selection. Human reproduction (Oxford, England), 30(2), 276–283. https://doi.org/10.1093/humrep/deu331 36. Scaravelli, G., Zacà, C., Levi Setti, P. E., Livi, C., Ubaldi, F. M., Villani, M. T., Greco, E., Coccia, M. E., Revelli, A., Ricci, G., Fusi, F., Vigiliano, V., De Luca, R., Bolli, S., & Borini, A. (2021). Fertilization rate as a novel indicator for cumulative live birth rate: a multicenter retrospective cohort study of 9,394 complete in vitro fertilization cycles. Fertility and sterility, 116(3), 766–773. https://doi.org/10.1016/j.fertnstert.2021.04.006 37. Zacà, C., Coticchio, G., Vigiliano, V., Lagalla, C., Nadalini, M., Tarozzi, N., & Borini, A. (2022). Fine-tuning IVF laboratory key performance indicators of the Vienna consensus according to female age. Journal of assisted reproduction and genetics, 39(4), 945–952. https://doi.org/10.1007/s10815-022-02468-2 38. Franco, José G Jr et al. “Key performance indicators score (KPIs-score) based on clinical and laboratorial parameters can establish benchmarks for internal quality control in an ART program.” JBRA assisted reproduction vol. 21,2 61-66. 1 Jun. 2017, doi:10.5935/1518-0557.20170016 39. Goyal, A., Kuchana, M., & Ayyagari, K. P. R. (2020). Machine learning predicts live-birth occurrence before in-vitro fertilization treatment. Scientific reports, 10(1), 20925. https://doi.org/10.1038/s41598-020-76928-z 40. Raef, Behnaz et al. “Computational prediction of implantation outcome after embryo transfer.” Health informatics journal vol. 26,3 (2020): 1810-1826. doi:10.1177/1460458219892138 41. Blank, Celine et al. “Prediction of implantation after blastocyst transfer in in vitro fertilization: a machine-learning perspective.” Fertility and sterility vol. 111,2 (2019): 318-326. doi:10.1016/j.fertnstert.2018.10.030 42. Loewke, Kevin et al. “Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos.” Fertility and sterility vol. 117,3 (2022): 528-535. doi:10.1016/j.fertnstert.2021.11.022 43. Fréour, T., Le Fleuter, N., Lammers, J., Splingart, C., Reignier, A., & Barrière, P. (2015). External validation of a time-lapse prediction model. Fertility and sterility, 103(4), 917–922. https://doi.org/10.1016/j.fertnstert.2014.12.111 44. Berntsen, J., Rimestad, J., Lassen, J. T., Tran, D., & Kragh, M. F. (2022). Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences. PloS one, 17(2), e0262661. https://doi.org/10.1371/journal.pone.0262661 45. Reignier, A., Girard, J. M., Lammers, J., Chtourou, S., Lefebvre, T., Barriere, P., & Freour, T. (2019). Performance of Day 5 KIDScore™ morphokinetic prediction models of implantation and live birth after single blastocyst transfer. Journal of assisted reproduction and genetics, 36(11), 2279–2285. https://doi.org/10.1007/s10815-019-01567-x 46. Tran, D., Cooke, S., Illingworth, P. J., & Gardner, D. K. (2019). Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Human reproduction (Oxford, England), 34(6), 1011–1018. https://doi.org/10.1093/humrep/dez064 47. VerMilyea, M., Hall, J. M. M., Diakiw, S. M., Johnston, A., Nguyen, T., Perugini, D., Miller, A., Picou, A., Murphy, A. P., & Perugini, M. (2020). Development of an artificial intelligence-based assessment model for prediction of embryo viability using static images captured by optical light microscopy during IVF. Human reproduction (Oxford, England), 35(4), 770–784. https://doi.org/10.1093/humrep/deaa013 48. Loewke, K., Cho, J. H., Brumar, C. D., Maeder-York, P., Barash, O., Malmsten, J. E., Zaninovic, N., Sakkas, D., Miller, K. A., Levy, M., & VerMilyea, M. D. (2022). Characterization of an artificial intelligence model for ranking static images of blastocyst stage embryos. Fertility and sterility, 117(3), 528–535. https://doi.org/10.1016/j.fertnstert.2021.11.022 49. Fruchter-Goldmeier, Y., Kantor, B., Ben-Meir, A. et al. An artificial intelligence algorithm for automated blastocyst morphometric parameters demonstrates a positive association with implantation potential. Sci Rep 13, 14617 (2023). https://doi.org/10.1038/s41598-023-40923-x 50. Enatsu, N., Miyatsuka, I., An, L. M., Inubushi, M., Enatsu, K., Otsuki, J., Iwasaki, T., Kokeguchi, S., & Shiotani, M. (2022). A novel system based on artificial intelligence for predicting blastocyst viability and visualizing the explanation. Reproductive medicine and biology, 21(1), e12443. https://doi.org/10.1002/rmb2.12443 51. Kim, H. M., Ko, T., Kang, H., Choi, S., Park, J. H., Chung, M. K., Kim, M., Kim, N. Y., & Lee, H. J. (2024). Improved prediction of clinical pregnancy using artificial intelligence with enhanced inner cell mass and trophectoderm images. Scientific reports, 14(1), 3240. https://doi.org/10.1038/s41598-024-52241-x 52. Bori, L., Meseguer, F., Valera, M. A., Galan, A., Remohi, J., & Meseguer, M. (2022). The higher the score, the better the clinical outcome: retrospective evaluation of automatic embryo grading as a support tool for embryo selection in IVF laboratories. Human reproduction (Oxford, England), 37(6), 1148–1160. https://doi.org/10.1093/humrep/deac066 53. Ueno, S., Berntsen, J., Okimura, T., & Kato, K. (2024). Improved pregnancy prediction performance in an updated deep-learning embryo selection model: a retrospective independent validation study. Reproductive biomedicine online, 48(1), 103308. https://doi.org/10.1016/j.rbmo.2023.103308 54. Tzukerman, N., Rotem, O., Shapiro, M. T., Maor, R., Meseguer, M., Gilboa, D., Seidman, D. S., & Zaritsky, A. (2023). Using Unlabeled Information of Embryo Siblings from the Same Cohort Cycle to Enhance In Vitro Fertilization Implantation Prediction. Advanced science (Weinheim, Baden-Wurttemberg, Germany), 10(27), e2207711. https://doi.org/10.1002/advs.202207711 55. Diakiw, S. M., Hall, J. M. M., VerMilyea, M., Lim, A. Y. X., Quangkananurug, W., Chanchamroen, S., Bankowski, B., Stones, R., Storr, A., Miller, A., Adaniya, G., van Tol, R., Hanson, R., Aizpurua, J., Giardini, L., Johnston, A., Van Nguyen, T., Dakka, M. A., Perugini, D., & Perugini, M. (2022). An artificial intelligence model correlated with morphological and genetic features of blastocyst quality improves ranking of viable embryos. Reproductive biomedicine online, 45(6), 1105–1117. https://doi.org/10.1016/j.rbmo.2022.07.018 56. Duval, A., Nogueira, D., Dissler, N., Maskani Filali, M., Delestro Matos, F., Chansel-Debordeaux, L., Ferrer-Buitrago, M., Ferrer, E., Antequera, V., Ruiz-Jorro, M., Papaxanthos, A., Ouchchane, H., Keppi, B., Prima, P. Y., Regnier-Vigouroux, G., Trebesses, L., Geoffroy-Siraudin, C., Zaragoza, S., Scalici, E., Sanguinet, P., … Boussommier-Calleja, A. (2023). A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems. Human reproduction (Oxford, England), 38(4), 596–608. https://doi.org/10.1093/humrep/dead023 57. Erlich, I., Ben-Meir, A., Har-Vardi, I., Grifo, J., Wang, F., Mccaffrey, C., McCulloh, D., Or, Y., & Wolf, L. (2022). Pseudo contrastive labeling for predicting IVF embryo developmental potential. Scientific reports, 12(1), 2488. https://doi.org/10.1038/s41598-022-06336-y 58. Sayed, S., Reigstad, M. M., Petersen, B. M., Schwennicke, A., Wegner Hausken, J., & Storeng, R. (2020). Time-lapse imaging derived morphokinetic variables reveal association with implantation and live birth following in vitro fertilization: A retrospective study using data from transferred human embryos. PloS one, 15(11), e0242377. https://doi.org/10.1371/journal.pone.0242377 59. Liao, Q., Zhang, Q., Feng, X., Huang, H., Xu, H., Tian, B., Liu, J., Yu, Q., Guo, N., Liu, Q., Huang, B., Ma, D., Ai, J., Xu, S., & Li, K. (2021). Development of deep learning algorithms for predicting blastocyst formation and quality by time-lapse monitoring. Communications biology, 4(1), 415. https://doi.org/10.1038/s42003-021-0193Cho7-1 60. J.H. Cho, A. Ehlers, C. Brumar, P. Maeder-York, O. Barash, J. Malmsten, Z. Nikica, D. Sakkas, M. Levy, K. Miller, M.D. VerMilyea, K. Loewke. (2022). Large-scale simulation of pregnancy rate improvements using an AI model for embryo ranking. 38th Hybrid Annual Meeting of the ESHRE, Milan — Italy, 3–6 July 2022. 61. McLernon, D. J., Raja, E.-A., Toner, J. P., Baker, V. L., Doody, K. J., Seifer, D. B., Sparks, A. E., Wantman, E., Lin, P. C., Bhattacharya, S., & Van Voorhis, B. J. (2022). Predicting personalized cumulative live birth following in vitro fertilization. Fertility and Sterility, 117(2), 326–338. https://doi.org/10.1016/j.fertnstert.2021.09.015 62. Nelson, S. M., & Lawlor, D. A. (2011). Predicting live birth, preterm delivery, and low birth weight in infants born from in vitro fertilisation: a prospective study of 144,018 treatment cycles. PLoS medicine, 8(1), e1000386. https://doi.org/10.1371/journal.pmed.1000386 63. Ratna, M. B., Bhattacharya, S., & McLernon, D. J. (2023). External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment. Human reproduction (Oxford, England), 38(10), 1998–2010. https://doi.org/10.1093/humrep/dead165 64. Johansen, M. N., Parner, E. T., Kragh, M. F., Kato, K., Ueno, S., Palm, S., Kernbach, M., Balaban, B., Keleş, İ., Gabrielsen, A. V., Iversen, L. H., & Berntsen, J. (2023). Comparing performance between clinics of an embryo evaluation algorithm based on time-lapse images and machine learning. Journal of assisted reproduction and genetics, 40(9), 2129–2137. https://doi.org/10.1007/s10815-023-02871-3 65. Cherouveim, P., Jiang, V. S., Kanakasabapathy, M. K., Thirumalaraju, P., Souter, I., Dimitriadis, I., Bormann, C. L., & Shafiee, H. (2023). Quality assurance (QA) for monitoring the performance of assisted reproductive technology (ART) staff using artificial intelligence (AI). Journal of assisted reproduction and genetics, 40(2), 241–249. https://doi.org/10.1007/s10815-022-02649-z 66. ESHRE Clinic PI Working Group, Vlaisavljevic, V., Apter, S., Capalbo, A., D'Angelo, A., Gianaroli, L., Griesinger, G., Kolibianakis, E. M., Lainas, G., Mardesic, T., Motrenko, T., Pelkonen, S., Romualdi, D., Vermeulen, N., & Tilleman, K. (2021). The Maribor consensus: report of an expert meeting on the development of performance indicators for clinical practice in ART. Human reproduction open, 2021(3), hoab022. https://doi.org/10.1093/hropen/hoab 67. Ferraretti, A. P., La Marca, A., Fauser, B. C., Tarlatzis, B., Nargund, G., Gianaroli, L., & ESHRE working group on Poor Ovarian Response Definition (2011). ESHRE consensus on the definition of 'poor response' to ovarian stimulation for in vitro fertilization: the Bologna criteria. Human reproduction (Oxford, England), 26(7), 1616–1624. https://doi.org/10.1093/humrep/der092 68. Esteves, S. C., Alviggi, C., Humaidan, P., Fischer, R., Andersen, C. Y., Conforti, A., Bühler, K., Sunkara, S. K., Polyzos, N. P., Galliano, D., Grynberg, M., Yarali, H., Özbek, I. Y., Roque, M., Vuong, L. N., Banker, M., Rienzi, L., Vaiarelli, A., Cimadomo, D., & Ubaldi, F. M. (2019). The POSEIDON Criteria and Its Measure of Success Through the Eyes of Clinicians and Embryologists. Frontiers in endocrinology, 10, 814. https://doi.org/10.3389/fendo.2019.00814 69. Esteves, S. C., Yarali, H., Ubaldi, F. M., Carvalho, J. F., Bento, F. C., Vaiarelli, A., Cimadomo, D., Özbek, İ. Y., Polat, M., Bozdag, G., Rienzi, L., & Alviggi, C. (2020). Validation of ART Calculator for Predicting the Number of Metaphase II Oocytes Required for Obtaining at Least One Euploid Blastocyst for Transfer in Couples Undergoing in vitro Fertilization/Intracytoplasmic Sperm Injection. Frontiers in endocrinology, 10, 917. https://doi.org/10.3389/fendo.2019.00917 70. Roque, M., Haahr, T., Esteves, S. C., & Humaidan, P. (2021). The POSEIDON stratification - moving from poor ovarian response to low prognosis. JBRA assisted reproduction, 25(2), 282–292. https://doi.org/10.5935/1518-0557.20200100 71. Lee, H. J., Noh, H. K., & Joo, J. K. (2022). Comparison of ART outcome in patients with poor ovarian response according to POSEIDON criteria. Scientific reports, 12(1), 17723. https://doi.org/10.1038/s41598-022-22859-w