­Use of Artificial Intelligence for Public Health Surveillance: A case study to develop a Machine Learning-algorithm to estimate the incidence of Diabetes Mellitus in France


 Background The use of machine learning techniques is increasing in healthcare which allows to estimate and predict health outcomes from large administrative data sets more efficiently. The main objective of this study was to develop a generic machine learning (ML) algorithm to estimate the incidence of diabetes based on the number of reimbursements over the last 2 years. Methods We selected a training data set from a population-based epidemiological cohort (i.e., CONSTANCES) linked with French National Health Database (i.e., SNDS) to develop a ML-algorithm for estimating the incidence of diabetes. To develop this algorithm, we adopted a supervised ML approach. Following steps were performed: i. selection of final data set, ii. target definition, iii. coding variables for a given window of time, iv. split final data into training and test data sets, v. variables selection, vi. training model, vii. validation of model with test data set and viii. selection of the model. Results The final data set used to develop the algorithm included 44,659 participants from CONSTANCES. Out of 3,468 variables, which were similar in SNDS and CONSTANCES cohort were coded, 23 variables were selected to train different algorithms. The final algorithm to estimate the incidence of diabetes was a Linear Discriminant Analysis model based on number of reimbursements of selected variables related to biological tests, drugs, medical acts and hospitalization without a procedure over the last two years. This algorithm has a sensitivity of 62%, a specificity of 67% and an accuracy of 67% [95% CI: 0.66 – 0.68]. Conclusions Supervised ML is an innovative tool for the development of new methods to exploit large health administrative databases. In context of InfAct project, we have developed and applied the first time a generic ML-algorithm to estimate the incidence of diabetes for public health surveillance. The ML-algorithm we have developed, has a moderate performance. The next step is to apply this algorithm on SNDS to estimate the incidence of type 2 diabetes cases. More research is needed to apply various MLTs to estimate the incidence of various health conditions and to calculate the contribution of various risk factors on developing type 2 diabetes.


Background
The availability of administrative data generated from different sources is increasing and the possibility to link these data sources with other databases offers unique opportunity to answer those research questions, which require a large sample size or detailed data on hard-to-reach population [1]. French National Health Data System (i.e., SNDS [Système National de Données Santé]) is an example of a big data/large administrative linked data set, which is used for public health surveillance in France [2]. It includes most updated, individual level health information about health insurance claims, hospital discharge and mortality of whole French population (i.e., 66 million people) [2]. However, the estimation of health indicators from linked administrative data is challenging due to several reasons such as variability in data sources and data collection methods, availability of a large number of variables, lack of skills and capacity to analyze big data [3]. More e cient ways of analyzing health information using big data across European countries are required. In that context, the use of arti cial intelligence (AI) is increasing in healthcare. Indeed AI allows to handle data with a large number of dimensions (features) and units (feature vectors) e ciently with a high precision. AI techniques offer bene ts in estimation of health indicators both at individual and population levels (i.e., improving social and health policy process). Machine learning (ML) is an application of AI that provides systems the ability to learn automatically and improve from experience without being explicitly programmed [4]. Supervised learning algorithms build on a mathematical model of a set of data that contains both the inputs and the desired outputs [5]. This approach is based on the prior knowledge of what the output values for a given sample should be [6]. ML techniques have been applied for the diagnosis of certain conditions as well as outcome prediction and prognosis evaluation with high precision [7][8][9].
This study was carried out under the InfAct (Information for Action) project [10], which is a joint action of Member States aiming to develop a more sustainable European health information system through improving the availability of comparable, robust and policy-relevant health status data and health system performance information. InfAct gathers 40 national health authorities from 28 Member States. This study is part of a work package (WP9) focused on innovation in health information system (i.e., using data linkages and/or AI) to improve public health surveillance and health system performance for health policy process. As a rst step, we have explored the current usage of these innovative techniques (i.e., data linkages and/or AI) in European countries and very few countries apply AI to estimate health indicators in their public health activities [11]. Therefore, the next step was to develop a generic approach by applying these innovative techniques to estimate the health indicators of chronic conditions for improved surveillance.
We used diabetes as a case study due to several reasons. First, it is one of the leading cause of morbidity in the world [12] and its prevalence is increasing among all ages in the European region, mostly due to increase in overweight and obesity, unhealthy diet and physical inactivity [13]. Second, a training data set using CONSTANCE cohort was already developed and used to answer various research questions for diabetes. Third, as this study is part of the InfAct project with a limited period to be completed. Fourth, estimation of incidence of diabetes cases is important to develop the prevention strategies to reduce its burden. Therefore, we decided to use this training dataset to develop a generic ML-approach.
The main objective of this study was to develop for the rst time a generic ML-algorithm to estimate the incidence of diabetes based on the number of reimbursements over the last 2 years.

Development of the ML-algorithm
To develop ML-algorithm, we adopted a supervised ML approach. Following steps were performed: i. selection of nal data set, ii. target de nition, iii. coding of variables for a given window of time, iv. split nal data into training and test data sets, v. variables selection, vi. training model, vii. validation of model with test data set and viii. selection of the model.

i. Selection of nal data set
We selected a training data set from a population-based epidemiological cohort (i.e., CONSTANCES) to develop an algorithm to estimate the incidence of diabetes. The participants were recruited by CONSTANCES between January 1, 2012, and December 31, 2014. This cohort comprises after nal completion a national representative randomly selected sample of 50,954 aged between 18 and 69 years (inclusive) and living in France [14,15]. The participants are randomly selected from the bene ciaries of the National Health Insurance Fund (i.e. CNAM [Caisse Nationale d'Assurance Maladie]). In this cohort, data are collected using a self-administered questionnaire (SAQ) and a medical examination (MQ) and were used to de ne the known diabetes cases and pharmacologically-treated diabetes [16]. For known diabetes cases, in the SAQ, participants reported to have diabetes through the item: "Have you ever been told by a doctor or other health care professional that you had diabetes?" In the medical questionnaire, completed during the medical examination, the physician asked each participant if they had diabetes. For the pharmacologically-treated diabetes, two questions in the SAQ were related to diabetes treatment: "Are you currently being treated for diabetes with oral medication?" And "Are you currently being treated for diabetes with one or more insulin injections?" [16].
After ful lling a SAQ on health status, life style factors, socioeconomic and demographic characteristics, the participants attend to their related health screening center for a medical examination which includes: medical questionnaire, physical examination and blood sampling. This information previously collected was linked with the French National Health Data System (i.e., SNDS). We excluded pregnant women, women who declared being already diagnosed of gestational diabetes mellitus and participants without SNDS data.
ii. Target de nition The diabetes status was de ned according to CONSTANCES as described above. The diabetes cases treated for the rst time over the 12 months before the date of SAQ were de ned as incident cases (target 1). These diabetes cases included both type 1 and 2 diabetes. No diabetes cases treated over the 12 months before the date of SAQ, were de ned as non-diabetes cases (target 0). The rest of diabetes cases were excluded (see Figure 1).
iii. Coding of variables for a given window of time In CONSTANCES, we only coded those variables, which were also available in the SNDS to apply the potential ML-algorithm on SNDS to estimate the incidence of diabetes. A total of 3,483 continuous variables were coded and standardized (mean= 0, standard deviation=1) over the last 24 months before the date of SAQ. The rational to have a time window of 24 months before the SAQ was to provide a long duration to study changes in diagnostic procedures, hospitalizations and drug consumption that allows to estimate the incidence of diabetes with high accuracy. Following were the main categories of variables: number of medical consultations (50 variables), drug dispensed coded using the 5 th level of the Anatomical Therapeutic code [ATC 05] (461 variables), biological test (747 variables), medical acts (i.e., X-ray, surgery, etc.) (2135 variables), all hospitalizations (5 variables), hospitalizations with a procedure (i.e., dialysis, radiotherapy, etc.) (5 variables), hospitalizations without a procedure (5 variables), hospitalizations related to following associated health conditions: diabetes, heart failure, stroke, heart attack, foot ulcer, lower limb amputation, ischemic heart disease, transient ischemic attack, end-stage renal failure, diabetic coma, diabetic ketoacidosis and cancer (75 variables).
iv. Split nal data set into training and test data sets The nal data set was randomly split into 80% as a training data set and 20% as test data set. There was a signi cant imbalance of number of positive target (i.e., target 1 = diabetes treated cases) over the number of negative target (i.e., target 0 = non-diabetes cases) in the training dataset. To avoid the bias in ML-algorithm and skew in class distribution, we performed a random under sampling in the target 0 group to achieve the same number of individuals in both target groups. The selection of variables and the model was performed using the training data. The test data was used solely to test the nal model performance.

v. Variables selection
First, we removed all variables with a variance equal to zero and then the ReliefF exp score was estimated, based on the relevance of each variable, to differentiate between target 1 and target 0. The ReliefF expRank method is noise tolerant and is not affected by features interactions [17][18][19]. All the variables were ranked according to the ReliefF exp score. For continuous variables, the score values range from 0 to 1 [18]. The cutoff score was 0.01 and was selected based on the visual inspection of the ordered plot of ReliefF values for all variables, called "elbow plot" approach. The variables that had a ReliefF exp score equal or more than 0.01 were included to train different models and the variables less than 0.01 were excluded.

Steps vi to viii Model selection and validation of the model with test data set
The four following models [i.e., 1. Linear discriminant analysis (LDA), 2. Logistic regression (LR), 3.
Flexible discriminant analysis (FDA) and 4. Decision tree model (C5)] were applied to the training data set. For each model, we compared the performance in terms of Area under the Receiver Operating Characteristics (AROC) curve. The rst validation of the models was performed using k-fold [three repeats of ve-fold] cross-validation on training data set. After that, the models' performances were assessed using the testing data set. Then, we automated the model selection process by giving the computer a speci c metric including sensitivity, speci city, positive predictive value, negative predictive value, F1score and kappa. Finally, a single model was retained based on its performance and its transferability to other databases.

Final data set
The nal data set to develop the algorithm included 44,659 participants, with 81 incident diabetes cases (target 1) and 44,578 participants without diabetes (target 0) (Fig.2). The general characteristics of the nal data set is described in table 1. The incident diabetes group was included older, with a higher percentage of men, treated hypertension and dyslipidemia, former smokers, a higher body mass index and a family history of diagnosed diabetes as compared to non-diabetes group.

Variables selection
Out of 3,468 continuous variables coded, 23 variables (0.7%) had a ReliefF exp Score above 0.01 and were therefore selected (Fig.3).
The 23 selected variables were ranked based on their ReliefF exp score ( Table 2). The rst variable was the "age". The following nine were related to "number of reimbursements of biological tests performed in last 2 years" (i.e., Alkaline Phosphatase test, Gamma Glutamyle Transferase test, Transaminases (ALAT and ASAT, TGP and TGO) blood test, Uric Acid (Uricemia) blood test, glucose blood, Creatinine level blood test, Exploration of a Lipid Anomaly (ELA) blood test, HbA1c test and C-Reactive Protein test). The next seven were related to "number of reimbursements of various non-diabetes drugs in last 2 years" (i.e., Proton pump inhibitors drugs, antidiarrheal drugs, Penicillin with broad spectrum drugs, bacterial and viral vaccines, Acetic acid derivatives, Propionic acid derivatives and Anilides (Paracetamol). The following ve were related to "number of reimbursements of various medical acts" (i.e., fundus examination by biomicroscopy with contact lens, functional examination of ocular motricity, binocular vision examination, mammography and X-ray for thorax). The last one is "the total number of hospitalization without a procedure (i.e., dialysis, chemotherapy) in last 2 years".
3. Algorithm to estimate the incidence of diabetes  (Fig. 4). After that, we compared the performances of these four models using test data set to select the one based on the performance metrics (Table 3). We kept the LDA model since it showed a better performance with an accuracy of 67% with the test data set as compared to other models (Table 3).

Distribution of means of selected variables in test data set
After the selection of LDA model, the 23 selected variables were trained with the test data set (20% of nal data set 44,659 = 8,931). We compared the distribution of means of these continuous variables among two groups: incident diabetes cases (i.e., 2,921) and non-diabetes cases (i.e., 6,010) using LDA algorithm in the test data set ( Table 4). The mean distribution of all selected variables related to the number of reimbursements of biological tests, medicines not used for diabetes treatment and medical acts performed in last 2 years, was signi cantly higher in the incident diabetes group than in nondiabetes group. For example, the age was the rst ranked variable with 0.04 ReliefF exp score among 23 selected variables and was highly discriminant in the incident diabetes group. The mean age of patients in diabetes group was 57 years old as compared to 44 years old in non-diabetes group (Table 4).
Following the age variable, nine other features selected, related to the mean number of reimbursements of biological tests, were more discriminant in incident diabetes group than in non-diabetes group. These biological tests were performed to measure the normal values of certain enzymes, proteins, glucose and uric acid in the blood to check the normal functions of liver, kidney, pancreas and other organs. For example, the mean number of reimbursement of blood glucose test in last two years was 1.82 times more discriminant in diabetes group than in non-diabetes group. The following group of features was the mean number of reimbursements of drugs. There were seven drugs and their mean number of reimbursements in last 2 years was more discriminant in incident diabetes group than in non-diabetes group. In the category of medical acts, there were three following features more discriminant in incident diabetes group: mean number of reimbursements of examination of fundus by biomicroscopy with contact lens, ocular motricity and binocular vision in last 2 years.
There were seven unusual features selected by the ML-algorithm and were discriminant in incident diabetes group: mean number of reimbursements of broad-spectrum penicillin, vaccines, propionic acid, Anilides (Paracetamol), mammography, X-ray for thorax and mean number of hospitalizations without any procedure.

Discussion
We have developed an algorithm based on the supervised ML approach to estimate the incidence of diabetes using a training data set from a cohort study. This algorithm (i.e., LDA model) was built on 23 selected variables from the CONSTANCES based on the number of reimbursements over the last 2 years to estimate the incidence of diabetes. This algorithm showed a moderate performance in predicting the incidence of diabetes cases with a sensitivity of 62% and an accuracy of 67%. Among 23 selected variables, six were related to diabetes that were expected, such as age and Glucose blood test. Whereas 17 other variables were not directly related to diabetes and were more discriminant in incident diabetes group than in non-diabetes group such as Proton pump inhibitors drug.

Main limitations of the ML-algorithm
Our study has highlighted that there were two discriminant features related to diabetes in LDA model i.e., mean number of reimbursements of glucose blood and HbA1c tests, which could potentially characterize the incident diabetes cases. In France, the screening recommendations for diabetes are based on the glucose blood test. HbA1c is only recommended for the management of diabetes but not for diagnoses.
In 2009 and 2010, the WHO has introduced HbA1c as an alternative method to diagnose diabetes that has been adopted by many countries since this date. The ophthalmologic problems such as glaucoma, cataract, ocular movement disorders, etc., are the main complications of diabetes. Therefore, the increase frequency of medical acts performed as a result of diabetes related complications such as visual functions allowed to better characterize incident diabetes cases. Moreover, the increased use of nondiabetic drugs along with mentioned biological tests in incident diabetes group may explain potentially the pre-existing comorbidity of cardiovascular or gastrointestinal diseases.

Implications and perspectives for future research
This innovative approach has been applied to two further studies: i. to classify and to estimate the prevalence of type 1 and type 2 diabetes cases [21] and, ii. to identify the number of undiagnosed diabetes cases ML algorithms in the SNDS (on going). For the rst study, ML-algorithm developed has a sensitivity of 100% and speci city of 97%, and for the second study, the sensitivity is 71% and speci city is 61%.
The next step is to apply this algorithm on SNDS to estimate the incidence of type 2 diabetes cases. We recommend further research for following perspectives using ML-techniques: rst to use different time windows (for example 6 months, 12 months or 16 months) to code variables and to explore their impact on estimates, second to predict the trend of diabetes over time and third, to estimate the contribution of determinants of diabetes such as BMI, dietary habits and physical activity, on developing type 2 diabetes using ML approaches.

Conclusions
The use of MLT to analyze large administrative databases (health and non-health related data sources) is increasing across European countries in order to improve the public health surveillance and health policy process. Supervised machine learning is an innovative methodology for the development of algorithms to exploit large health administrative databases. It was the rst step that we have developed a generic MLalgorithm with a moderate performance to estimate the incidence of diabetes in a training data set.  Learning-algorithm to estimate the incidence of Diabetes Mellitus in France"