Investigation and Optimization of the C-ANN Structure in Predicting the Compressive Strength of Foamed Concrete

Development of Foamed Concrete (FC) and incessant increases in fabrication technology have paved the way for many promising civil engineering applications. Nevertheless, the design of FC requires a large number of experiments to determine the appropriate Compressive Strength (CS). Employment of machine learning algorithms to take advantage of the existing experiments database has been attempted, but model performance can still be improved. In this study, the performance of an Artificial Neural Network (ANN) was fully analyzed to predict the 28 days CS of FC. Monte Carlo simulations (MCS) were used to statistically analyze the convergence of the modeled results under the effect of random sampling strategies and the network structures selected. Various statistical measures such as Coefficient of Determination (R2), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) were used for validation of model performance. The results show that ANN is a highly efficient predictor of the CS of FC, achieving a maximum R2 value of 0.976 on the training part and an R2 of 0.972 on the testing part, using the optimized C-ANN-[3–4–5–1] structure, which compares with previous published studies. In addition, a sensitivity analysis using Partial Dependence Plots (PDP) over 1000 MCS was also performed to interpret the relationship between the input parameters and 28 days CS of FC. Dry density was found as the variable with the highest impact to predict the CS of FC. The results presented could facilitate and enhance the use of C-ANN in other civil engineering-related problems.


Introduction
Over the past decades, large-scale, super-tall, or mega-tall buildings have been increasingly constructed over the world [1,2]. This fact raises a number of fundamental engineering problems, crucial to be solved, such as self-weight, large foundation sizes, and earthquake resistance [3,4]. Apart from the structural problems, the weight of conventional concrete is also a major drawback from a material point of view [5]. Foamed Concrete (FC) appears as a potential replacement of conventional concrete to be used in specific engineering applications. Since the early 1920s, when the Swedish architect Johan Eriksson was granted the patent on FC, many further inventions have been developed and applied in practice by experts in America, Japan, and Europe [6]. Lightweight concrete, known as cellular concrete or FC, is a recently developed concrete with many advantages [7,8]. Cellular concrete is made by generating air bubbles in the cement paste or mortar. These bubbles have diameters ranging from 0.1 to 1 mm [9,10] and air is usually contained within 50% of the volume [11]. The presence of algorithms, its performance depends significantly on the selection of network structure. Thus, this study also focuses on the investigation and optimization of the C-ANN structure for better prediction of the CS of FC. To achieve this goal, previously published experimental data in the literature were gathered and randomly divided into two subsets: the training dataset (70% of data) and the testing part (30% of data). Monte Carlo simulations (MCS) were used to verify the convergence of the modeled results. In addition, Partial Dependence Plots (PDP) over 1000 MCS were also produced to interpret the relationship between input parameters and 28-day CS of FC. Various statistical measures such as Coefficient of Determination (R 2 ), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) were used for validation of model performance.

Statement of the Novelty and Significance
The 28-days CS of FC is a key value in structural engineering, controlling the quality of construction works. While various surrogate, optimized, and hybrid ML algorithms have been used, it has been proven that a C-ANN model could still accurately predict the 28-days CS of FS, which depends significantly on the selection of the appropriate network structure. Limited efforts to find out a suitable C-ANN structure have been reported in the literature [48,49], but no systematic study has accounted for random sampling of the dataset. Therefore, the current study contributes to fill these research gaps through the following points: (i) an improved prediction of the CS of FC was achieved using C-ANN algorithm; (ii) MCS was used, for the first time, to determine the most effective C-ANN structure in the presence of random dataset splitting; (iii) a comparison with existing published results was conducted to confirm the accuracy of C-ANN; (iv) a sensitivity analysis was performed using PDP to reveal the relationships between the input variables and the 28-days CS of FC; and (v) three equations aiming to interpret the dependence of the 28-days CS of FC on dry density, water-to-cement, and sand-to-cement ratios were derived from PDP investigation.

Data Used
In this study, 220 data points of experiments on FC collected from the literature (Table 1) were used, including dry density (D), water/cement ratio (W/C), and sand/cement ratio (S/C) as input variables, and the CS at 28 days as an output variable. Table 1 summarizes the information of the literature constituting the database, including the number of data points collected. Besides, a summary of the statistical analysis of the three inputs and the CS as output of this study is presented in Table 2. The distribution histograms of dry density (D), water/cement ratio (W/C), sand/cement ratio (S/C), and the CS is displayed in Figure 1.

134
In the past decades, along with the intense development of computer science, AI techniques have 135 been used to a greater extent in various areas of expertise [51]. C-ANN is the most common AI 136 algorithm used in scientific research, especially in civil engineering [52]. C-ANN is an information 137 processing computing system designed based on the operation of the human brain [35,[53][54][55][56]. The 138 very first study of ANN was proposed by McCulloch and Pitts [57] in the early 1940s. In fact, C-ANN 139 is made up of multiple nodes and links that bind these nodes, and a weight which is capable of 140 learning is associated with each link. They learn by changing the value of the weights. The training 141 algorithm's objective is to reduce the mean square error (MSE) between the target and the predicted 142 outputs.

143
The advantages of C-ANN could be various but the most important point of ANN comes from 144 the ability to handle highly nonlinear problems [58]. Thus, complex interactions, relationships 145 between inputs and outputs, can be modeled without knowing the correlation between these 146 variables [58]. Thanks to these advantages C-ANN is applied in many different fields, such as basic 147 sciences, construction, health, and environment. The implementations of C-ANN in civil engineering

Conventional Artificial Neural Network (C-ANN)
In the past decades, along with the intense development of computer science, AI techniques have been used to a greater extent in various areas of expertise [51]. C-ANN is the most common AI algorithm used in scientific research, especially in civil engineering [52]. C-ANN is an information processing computing system designed based on the operation of the human brain [35,[53][54][55][56]. The very first study of ANN was proposed by McCulloch and Pitts [57] in the early 1940s. In fact, C-ANN is made up of multiple nodes and links that bind these nodes, and a weight which is capable of learning is associated with each link. They learn by changing the value of the weights. The training algorithm's objective is to reduce the mean square error (MSE) between the target and the predicted outputs.
The advantages of C-ANN could be various but the most important point of ANN comes from the ability to handle highly nonlinear problems [58]. Thus, complex interactions, relationships between inputs and outputs, can be modeled without knowing the correlation between these variables [58]. Thanks to these advantages C-ANN is applied in many different fields, such as basic sciences, construction, health, and environment. The implementations of C-ANN in civil engineering are promising and many important results have been achieved over the past years [52]. In the study of Nehdi et al. [59], C-ANN was used to predict the cellular concrete output. The investigation results show that compared to existing parametric methods, C-ANN can much more accurately predict several properties of concrete, such as production yield, foamed density, un-foamed density, and the CS of cellular concrete mixtures. In another study, from 150 empirical results, Paulson et al. [60] used C-ANN to conduct a comprehensive survey to optimize the level of cement replacement with silica fume. This study pointed out that it was expected that CS and the values were similar to the experimental results, so it concluded that instead of conducting experiments, C-ANN can be used to predict the CS for various values of input [60].

Quality Assessment of Results
In the present study, the coefficient of determination (R 2 ), Mean Absolute Error (MAE), and Root Mean Squared Error (RMSE) are used as measures to evaluate the developed ML algorithms. Precisely, R 2 values allow to identify the statistical relationship between the experimental results and predicted values. It yields a value between 0 and 1, where 0 is no correlation and 1 is total correlation [61]. In the cases of RMSE and MAE, which have the same units as the quantity being estimated, lower values of RMSE and MAE basically indicate a high accuracy of prediction output using the ML models [29,32,62,63]. Values of R, RMSE, and MAE are estimated using the following equations [31,61,64,65]: where n is defined as the number of input data, p i and v i denoted the actual and predicted CS values, respectively, and v is the mean value of the predicted CS by C-ANN.

Monte Carlo Simulations (MCS)
The performance of any ML algorithm is highly depended on the way that the training dataset is constructed-that is, the data sampling method. Data sampling is a kind of statistical method used to select observations from the database whose main objective is to investigate a selected parameter. In applied machine learning, data sampling is usually referred to as the propagation of variability of inputs on the predicted output. The real performance of ML algorithms could only be accepted only if the prediction errors are low even with the presence of variability in the dataset. In order to generate such variability, MCS is used in this study. The Monte Carlo method is one of the well-known techniques to propagate input variability on the output results [63,[66][67][68][69]. The Monte Carlo method is notably powerful and efficient for evaluating the performance of ML models [34,70]. Moreover, Monte Carlo allows massively parallel computing to be conducted, making it an effective strategy to reduce simulation time [71][72][73][74]. The main idea of the MCS is to use random sampling to generate as many realizations as possible in the input space. The next step is dedicated to the calculation of the output through ML models [75,76]. Finally, the obtained results of performance could be assessed using several statistical criteria. In this work, the statistical convergence of MCS is examined using the equation as follows [62,70,77,78]: where K is the mean value of the considered random variable K and M is the number of Monte Carlo runs. This convergence function provides efficient information related to the computational time, reliable results for further statistical analysis.

Methodological flow chart
The methodology flowchart of the present study included five main steps, as follows: Step 1: Preparation and pre-processing of the database, including 220 data units collected from the available literature. The three input variables were dry density, water-to-cement, and sand-to-cement ratios, and the output was the CS of FC. The histograms of the distribution of the inputs and outputs were next presented. Before further processing, the database was scaled in the (0,1) range to reduce numerical bias, as in common ML problems.
Step 2: To analyze the convergence of the C-ANN under random sampling, a number of 1000 of simulations was conducted for each combination of C-ANN. The number of neurons in the first hidden layer of the C-ANN structure was varied from 1 to 20, whereas those of the second hidden layer ranged from 0 to 20. The value of 0 is denoted with the case using only one hidden layer. A total of 420 structures was constructed using one and two hidden layers, making a total of 420,000 simulations (21 neurons in the first hidden layer x 20 neurons in the second hidden layer x 1000 simulations). It is worth noting that the exact values of the error criteria are not presented in this step. It is dedicated only for the convergence analysis of the two typical C-ANN structures.
Step 3: To determine the optimized structure of C-ANN, various structures of C-ANN were developed and validated using R 2 , RMSE, and MAE. The optimized C-ANN structure was selected based on the maximum values of R 2 and minimum values of RMSE and MAE.
Step 4: Using the optimal C-ANN structure, a detailed analysis of robustness of C-ANN algorithm was carried out and presented.
Step 5: Interpretation of the relationship and dependence of the 28-days CS with the input factors (dry density, water/cement ratio, and sand/cement ratio) using C-ANN was presented using PDP.

Convergence of C-ANN under Random Sampling Effect
Convergence of C-ANN under the random sampling effect was analyzed with a total of 420,000 simulations for 420 C-ANN structure cases. However, the convergence results of only two typical C-ANN structures (C-ANN-  Figure 3, the fluctuation in terms of R 2 seemed more important than the previous case, i.e., from 70% to 110% compared with a range of 90% to 105%. The 1% level of fluctuation was reached from above 800 simulations, compared with 100 runs in the previous case. An identical number of simulations was also required for the case of RMSE and MAE, as well as a more important fluctuation that was observed in contrast with the C-ANN-[3-1-1-1] structure. Nevertheless, the standard deviation of the three error criteria converged toward a value at about 500 runs (Figure 3d,e,f), smaller than the results obtained by C-ANN-[3-1-1-1].
It can be observed that the two extracted typical C-ANN structures converged toward the statistical results. Thus, all the remaining C-ANN structures, with the number of neurons in the two hidden layers varied from 1 to 20, could also converge toward the simulation results. Thus, it can be stated that the C-ANN converged under the random sampling effect of the data used in this study.

Optimization of C-ANN Architecture
Various structures of C-ANN were investigated, as shown in Figure 4, for both the training and testing datasets. With respect to the training dataset, it was observed that higher number of neurons in both hidden layers produced a higher accuracy (R 2 > 0.95). Differently, an optimal zone was detected to achieve the highest accuracy (R 2 around 0.95) for the testing dataset (Figure 4c). This is an excellent example to demonstrate that an appropriate ANN structure must be determined before performing ML simulations. To further investigate the effect of neurons in the two hidden layers, standard deviation plots of the R 2 is presented (Figure 4b,d). The training part exhibited a small standard deviation, in general, with a low number of neurons in the second hidden layer (i.e., from 4 to 8 neurons). On the other hand, an optimal zone was observed for values of standard deviation of R 2 for the testing dataset. Such a zone was in a similar position with the average values of R 2 , ranging from 3 to 8 neurons in the first hidden layer and from 4 to 6 in the second hidden layer.

Robustness of Optimal C-ANN Structure
The robustness of the optimal C-ANN structure of C-ANN-[3-4-5-1] was analyzed using MCS (Figure 8). From a statistical point of view, Figure 8 shows the distribution of the performance indicators over 1000 realizations using the optimal C-ANN architecture, whereas the statistical analysis is highlighted in Table 3. The values of the average and standard deviation of the R 2 values are given in Figure 8a and those of the RMSE and MAE are plotted in Figure 8b for the training and testing datasets.
It can be seen that the performance indicators of the training part is better than the testing one, which could avoid overfitting problems. While the curves of R 2 , RMSE, and MAE were rather narrow, the curves of slope for the testing part were rather broad. In all the cases, the distributions of the results followed a Gaussian distribution but with different standard deviations. In addition, the values of three levels of quantiles (Q25, Q50, and Q75) with the coefficient of variation are given in Table 3. As expected, the optimal architecture provided a high performance of prediction as well as a low value of coefficient of variation (CV). In terms of CV, the model also proved as a robust predictor. Values of CV of RMSE and MAE were approximately 10% due to the small mean values. In case of R 2 and slope, CVs were lower than 3.2%. Table 3. As expected, the optimal architecture provided a high performance of prediction as well as 295 a low value of coefficient of variation (CV). In terms of CV, the model also proved as a robust 296 predictor. Values of CV of RMSE and MAE were approximately 10% due to the small mean values.

297
In case of R 2 and slope, CVs were lower than 3.2%.

306
In general, the use of PDP is effective in interpreting the relationships between the inputs and 307 the predicted output [79]. As seen in Figure 9, all considered input variables affected the 28-days CS 308 of FC; however, at different amplitudes. For instance, the CS of FC decreased when the water/cement 309 ratio (W/C) and sand/cement ratio (S/C) increased, but it increased when dry density (D) increased.

310
Thus, the water/cement ratio (W/C) and sand/cement ratio (S/C) had a negative effect, whereas dry 311 density had a positive effect on the CS of FS. It is seen that the PDP investigation for each input could 312 be approximated by a different fit (i.e., exponential, quadratic, and linear). Table 4

Interpretation of Relationship between of Inputs and Output Using PDP
In general, the use of PDP is effective in interpreting the relationships between the inputs and the predicted output [79]. As seen in Figure 9, all considered input variables affected the 28-days CS of FC; however, at different amplitudes. For instance, the CS of FC decreased when the water/cement ratio (W/C) and sand/cement ratio (S/C) increased, but it increased when dry density (D) increased. Thus, the water/cement ratio (W/C) and sand/cement ratio (S/C) had a negative effect, whereas dry density had a positive effect on the CS of FS. It is seen that the PDP investigation for each input could be approximated by a different fit (i.e., exponential, quadratic, and linear). Table 4 summarizes the overall values of the PDP investigation, including a classification of the influence of the level of inputs and the negative and positive effects of such input to the predicted output.

Discussion
approach is proved as a promising tool and technique for better performance of the model, the accuracy of this approach depends significantly on the selection of network structures [82][83][84]. In this study, a thorough investigation of ANN structures has been carried out for predicting the 28 days CS 324 of FC, which is one of the most important mechanical properties of FC. In addition, it is well-known 325 that the accuracy of the given machine learning algorithm greatly depends on the sampling strategy 326 to construct the model [85,86]. Therefore, MCS was used in this study to fully analyze the capability results in [25,47]. Similar observations with respect to criteria such as RMSE or MAE were also noticed 339 and compared with previously published works, such as the RMSE = 1.808 [47] or 1.65 [25], but these 340 studies used only 177 instances [25] or 150 samples [47].

341
In addition, it is also noticed that the performance of machine learning models such as C-ANN

Discussion
A machine learning algorithm like C-ANN has been widely and effectively applied in different fields for solving a lot of real-world problems, including material sciences [80,81]. Although this approach is proved as a promising tool and technique for better performance of the model, the accuracy of this approach depends significantly on the selection of network structures [82][83][84]. In this study, a thorough investigation of ANN structures has been carried out for predicting the 28 days CS of FC, which is one of the most important mechanical properties of FC. In addition, it is well-known that the accuracy of the given machine learning algorithm greatly depends on the sampling strategy to construct the model [85,86]. Therefore, MCS was used in this study to fully analyze the capability of all the C-ANN structures, taking into account such variability of the input space in the training phase of the model. As a result, an accuracy map was given where an optimal structure was defined by statistical analysis of many criteria, such as the maximum average values of the R 2 , RMSE, and MAE, as well as the minimum values of the standard deviation of R 2 , RMSE, and MAE.
The investigation results showed that a good result could be obtained with simple machine learning algorithms like the C-ANN, as long as the structure and the parameters were well established. The optimization results of the C-ANN structure also showed that increasing the number of neurons in the hidden layers can enhance the accuracy of the C-ANN for the training dataset, and the optimal architecture of the C-ANN was selected as C-ANN- [3-4-5-1]. It is worth noticing that the maximum value of the R2 for the testing part, in this study, could reach an accuracy of R 2 = 0.972 (R = 0.986). This value is amongst the best accuracies reported in the literature, as compared with results in [25,47]. Similar observations with respect to criteria such as RMSE or MAE were also noticed and compared with previously published works, such as the RMSE = 1.808 [47] or 1.65 [25], but these studies used only 177 instances [25] or 150 samples [47].
In addition, it is also noticed that the performance of machine learning models such as C-ANN depends on the selection of input factors used in the datasets. Therefore, a sensitivity analysis should be carried out to (i) verify the performance of the C-ANN model on the choice of features in the input space and (ii) interpret the relationships between the input variables and the 28 days FC of CS. For this, such relationships were investigated thanks to PDP. In general, PDP analysis is given as the result of one constructed machine learning model. Thus, the results are influenced by the accuracy of such model and it could vary in function of the random sampling strategies. Again, the PDP analysis in this study was taken as the average of 1000 reliable simulations and could be confident for further investigations. The results of this study are in agreement with the findings in the literature, which stated that the 28-days CS of FC increased exponentially with an increase in dry density (D) [23,47]. Besides, a previous study of Hoff [87] indicated that for a certain range of density, this relationship can be linear, as observed for the scaled density from 0 to 0.5 in this study. However, for the whole range of density, an exponential relationship is assumed. The positive effect of dry density (D) and cement content was confirmed and had a significant role in designing FC mixes [47]. Meanwhile, a negative effect on the CS of FC was confirmed to be associated with an increase in the water/cement ratio (W/C) and sand/cement ratio (S/C) [47]. Moreover, the water/cement ratio (W/C) is a controlling factor of the CS of FC [88], or, an appropriate content of water enhances the consistency and stability of FC, which increase the CS of FC [10,89].

Conclusion
Despite a number of studies on the prediction of the CS of FC using ML, the prediction performance could still benefit from more in-depth investigations. The current study showed a simple but efficient manner to use an ANN structure to predict the 28-days CS of FC. An optimal C-ANN-[3-4-5-1] was proposed, where the robustness was confirmed in the presence of a random dataset splitting over 1000 MCS. Statistical assessment of the results was derived to verify the reliability of predicted results and testify the convergence of the outcomes of C-ANN. A simple and optimal C-ANN structure was proven to produce comparative results with complex algorithms proposed in the literature, where the value of R 2 could reach R 2 = 0.972. A comprehensive interpretation of the results using PDP investigations was performed and derived several relationships, revealing the dependence of the 28-days CS of FC on the dry density (D), water/cement ratio (W/C), and sand/cement ratio (S/C). Overall, dry density (D) was found to be the most affecting factor, which had a significant role on the 28-days CS of FC. The results of the present study could facilitate and enhance the use of C-ANN in other civil engineering-related problems. Two perspectives can be envisioned: (i) gathering more data to cover larger ranges of input and output variables, and (ii) implementing optimization techniques to improve the accuracy of the C-ANN algorithm.