1 BACKGROUND

Hidden hunger (HH) refers to a condition where the body, despite adequate energy intake, lacks specific micronutrients like iron, zinc, and vitamin A [1]. The term “hidden” is used because this condition often lacks obvious physical symptoms [2]. An imbalance in nutrient intake is a major contributing factor to HH [3]. Statistics indicate that approximately 2 billion people are affected by HH worldwide, especially in economically underdeveloped regions [4]. Numerous studies have demonstrated the significant impact of HH on people's health [5]. For example, prolonged iodine deficiency can lead to cognitive impairments, and iodine deficiency in pregnant women during the gestational period can increase the risk of stillbirth [6]; long-term zinc deficiency can adversely affect growth and development as well as diminish the body's resistance to diseases [7]; and the deficiency of vitamin A similarly augments the risk of maternal and fetal mortality [8]. Accurate identification of populations experiencing micronutrient deficiencies is of paramount importance for symptom improvement. However, there are no effective assessment methods for HH due to constraints such as technology and funding.

At present, the detection of micronutrients predominantly relies on conventional physical and chemical methods. For instance, water-soluble vitamins (including vitamins B₁, B₂, B₃, B₅, B₆, B₇, B₉, B₁₂, and C) are often analyzed using liquid chromatography tandem mass spectrometry (LC-MS/MS) and high-performance liquid chromatography (HPLC) techniques [9, 10]. Nonetheless, because of the lack of effective standardization across different laboratories and the susceptibility of detection results to the influence of sample collection conditions, the accuracy of the obtained results is often compromised [11]. Regarding the detection of fat-soluble vitamins, ¹³C isotope labeling LC-MS/MS (vitamin A) [12], LC-MS/MS (vitamins D and K) [13, 14], and liquid chromatography-ultraviolet detection (vitamin E) [15] are usually used. However, because of variations between detection techniques, and the requirement for lipids and cholesterol as quantitative references, the test results are not always reliable [16]. For the quantitative analysis of minerals (including Fe, Cu, Zn, Se, I, and Mn), electrochemiluminescence immunoassay, atomic absorption spectroscopy, and inductively coupled plasma mass spectrometry are often used [17-19]. However, this approach is also susceptible to an individual's inflammatory status and lifestyle habits [20]. Moreover, the high cost and the lack of specialized detection equipment also restrict the large-scale screening for HH. Thus, there is an urgent need for an effective method to predict HH risk.

Machine learning (ML) is an ever-evolving realm encompassing computational algorithms designed to enable systems to acquire knowledge and emulate human intelligence by iteratively learning from the surrounding environment [21]. With the continuous application of ML in the field of medicine, it has demonstrated significant advantages in personalized disease treatment [22], drug discovery [23] and sensitive drug screening [24], surgical robotics [25], assistive surgical decision-making [26], medical image identification [27], and omics analysis [28]. Particularly, the application of ML in disease diagnosis has effectively facilitated early detection, thereby improving disease prognosis. For example, recently published ML-based diagnostic models have provided strong assistance in the clinical diagnosis of Parkinson's disease [29]. Regarding the risk assessment of HH, ML also holds significant potential. It can greatly enhance the efficiency of these assessments and reduce detection costs, making it particularly suitable for large-scale population screening. In a previous study, we found significant associations between individuals' living habits, diets, nutritional cognition, and their risk of HH [30]. Therefore, we speculated that establishing ML models based on these features would contribute to the auxiliary diagnosis of HH.

In this study, we use various ML algorithms to establish effective risk-prediction models for HH based on individuals' lifestyle habits, dietary patterns, and nutritional knowledge. Moreover, we conducted an in-depth analysis of the potential influencing factors of HH based on the model's interpretations. Additionally, to aid practical use, we also developed an online application. This will facilitate the widespread screening and early prevention of HH.

2 METHODS

2.1 Data collection and pre-processing

Cities in Anhui Province were initially classified into southern, northern, eastern, and western regions based on their geographical locations. This classification facilitated comprehensive sampling across diverse areas, thereby minimizing geographical bias and enhancing the robustness of the predictive model. Subsequently, they were categorized into developed and underdeveloped cities using the average gross domestic product of all cities in Anhui Province as the cut-off value. If multiple cities had similar geographical locations and economic levels, only one city was randomly selected as a representative. Additionally, cities with highly imbalanced population distribution between urban and rural areas were excluded. Based on the aforementioned selection criteria, a set of 11 cities were finally chosen: Bengbu, Chuzhou, Anqing, Fuyang, Huaibei, Xuancheng, Wuhu, Huangshan, Suzhou, Hefei, and Huainan. Then, a multistage stratified cluster sampling method was used within these cities, with three or four senior high schools randomly selected in each city. Ultimately, a total of 35 senior high schools were selected. Then, based on the inclusion criteria (1) currently enrolled in high school in Anhui Province and (2) voluntarily willing to participate in the survey, a total of 9336 high school students were finally selected. Prior to the commencement of this study, the corresponding official approval was obtained from the respective schools in addition to informed consent from the survey participants. All participants were instructed on how to effectively complete the survey and encouraged to provide truthful responses. The entire research process is illustrated in Figure 1.

For each participant, a questionnaire and HH assessment scale (HHAS) were distributed [30]. The questionnaire contained demographic characteristics (including sex, grade, birthplace, number of siblings, mother's highest education level, father's highest education level, monthly expenditure on food, and daily sun exposure time), nutritional knowledge (including understanding level of HH and essential nutrients, understanding level of the relationship between HH and chronic diseases, willingness to acquire nutrition knowledge, attitude toward HH, and sources of acquiring nutrition knowledge), and lifestyle habits (including frequency of irregular eating, reasons for irregular eating, satisfaction level with daily meals, degree of emphasis on nutritional balance in daily life, existence of picky eating and strong-flavored dietary habits, the consumption of nutritional supplements in the past year, weekly exercise duration, supplementation of micronutrients after sweating, and frequency of consuming fried food, coarse grains, fruits, and snacks) (Figure S1). The questionnaire survey results were double entered by two investigators, and cross-checking was conducted using Epidata 3.1 (EpiData Association). The HHAS consisted of 12 items, each with three options (usually, sometimes, and seldom), which were assigned scores of 5, 3, 1 or 1, 3, 5. A total score of 38 or less indicates high risk, whereas a score above 38 indicates low risk [30]. After questionnaires with incomplete information and those containing outliers were excluded, 7109 high school students' demographic characteristics, nutritional awareness, lifestyle habits, and HH risk statuses were obtained.

3 RESULTS

3.2 Performance of ML models

All the developed ML models demonstrated high predictive accuracy. Notably, the DNN model, represented by “Deep-HH,” achieved an AUROC of 0.813 in the external test cohort from Xuancheng City, which underscored its superior discriminative capability and robustness relative to other models (Table 2). The specific architecture of this model is presented in Figure 2. Based on stratified five-fold cross-validation, the average AUROC was 0.857 for the training sets and 0.833 for the internal validation sets (Figure 3a,b). In the external test set, the AUROC achieved 0.813 (Figure 3c). Because of the imbalanced dataset, PR curves were additionally plotted to further evaluate the model's predictive performance. The results indicated that the Deep-HH model achieved average precision of 0.861, 0.910, and 0.843 for the training sets, internal validation sets, and external test set, respectively, which further confirmed its high predictive accuracy (Figure 3d–f). Furthermore, the calibration curves for the training, internal validation, and external test sets demonstrated that the Deep-HH model was well-calibrated (Figure 3g–i). DCA is a method used to evaluate a model's utility. In this study, DCA was used to further assess the utility of the model, and the results indicated that the Deep-HH model had significant value (Figure 3j–l). For more performance metrics of the Deep-HH model, please refer to Table 2. To aid use, an online application was developed (http://sec.mitusml.com:9000/).

TABLE 2. Metrics of six models for the training sets, internal validation sets, and external test set.

Data sets	Metrics	DNN (95% CI)	RF (95% CI)	SVM (95% CI)	GBDT (95% CI)	XGBoost (95% CI)	KNN (95% CI)
Training sets	Accuracy	0.777 (0.769–0.785)	0.774 (0.771–0.778)	0.772 (0.769–0.775)	0.768 (0.764–0.771)	0.769 (0.767–0.771)	0.748 (0.744–0.752)
	Sensitivity	0.822 (0.797–0.847)	0.806 (0.801–0.811)	0.810 (0.804–0.816)	0.808 (0.804–0.812)	0.794 (0.789–0.799)	0.780 (0.774–0.786)
	Specificity	0.730 (0.690–0.770)	0.746 (0.738–0.754)	0.738 (0.727–0.749)	0.728 (0.717–0.739)	0.744 (0.739–0.749)	0.716 (0.711–0.721)
	PPV	0.756 (0.734–0.778)	0.758 (0.751–0.765)	0.756 (0.748–0.764)	0.748 (0.741–0.755)	0.754 (0.749–0.759)	0.732 (0.728–0.736)
	NPV	0.805 (0.789–0.821)	0.793 (0.791–0.795)	0.794 (0.791–0.797)	0.791 (0.789–0.894)	0.783 (0.780–0.786)	0.765 (0.759–0.770)
	F1 score	0.784 (0.779–0.789)	0.781 (0.779–0.784)	0.780 (0.779–0.782)	0.778 (0.774–0.782)	0.774 (0.769–0.779)	0.758 (0.754–0.762)
	AUROC	0.857 (0.849–0.863)	0.855 (0.854–0.856)	0.848 (0.847–0.849)	0.849 (0.849–0.850)	0.848 (0.846–0.850)	0.829 (0.827–0.831)
Validation sets	Accuracy	0.773 (0.767–0.779)	0.766 (0.760–0.773)	0.774 (0.767–0.782)	0.776 (0.769–0.783)	0.766 (0.755–0.777)	0.755 (0.749–0.761)
	Sensitivity	0.808 (0.773–0.843)	0.794 (0.778–0.810)	0.804 (0.791–0.817)	0.806 (0.796–0.816)	0.788 (0.774–0.802)	0.776 (0.758–0.794)
	Specificity	0.696 (0.638–0.754)	0.704 (0.682–0.726)	0.708 (0.686–0.730)	0.712 (0.691–0.733)	0.720 (0.694–0.746)	0.710 (0.680–0.740)
	PPV	0.852 (0.832–0.872)	0.852 (0.842–0.862)	0.854 (0.846–0.862)	0.856 (0.848–0.864)	0.856 (0.844–0.868)	0.852 (0.841–0.863)
	NPV	0.633 (0.609–0.657)	0.617 (0.603–0.631)	0.630 (0.618–0.642)	0.632 (0.620–0.644)	0.614 (0.597–0.631)	0.598 (0.586–0.610)
	F1 score	0.828 (0.818–0.838)	0.822 (0.815–0.829)	0.828 (0.824–0.832)	0.830 (0.824–0.836)	0.822 (0.811–0.833)	0.810 (0.804–0.816)
	AUROC	0.833 (0.829–0.839)	0.832 (0.825–0.839)	0.837 (0.832–0.842)	0.839 (0.833–0.845)	0.836 (0.829–0.843)	0.823 (0.818–0.828)
Test set	Accuracy	0.739	0.728	0.734	0.747	0.729	0.723
	Sensitivity	0.720	0.700	0.700	0.720	0.700	0.740
	Specificity	0.760	0.770	0.780	0.780	0.770	0.700
	PPV	0.810	0.810	0.820	0.820	0.810	0.770
	NPV	0.666	0.650	0.654	0.671	0.652	0.662
	F1 score	0.760	0.750	0.750	0.770	0.750	0.760
	AUROC	0.813	0.801	0.809	0.812	0.806	0.796

• Abbreviations: CI, confidence interval; DNN, deep-learning neural network; GBDT, gradient boosting decision tree; KNN, k-nearest neighbors; RF, random forest; SVM, support vector machine; XGBoost, extreme gradient boosting.

Additionally, the predictive performance of the GBDT model was second only to that of the Deep-HH model, and achieved an AUROC of 0.812 for the external test set (Table 2). Both the ROC and PR curves demonstrated its high predictive accuracy (Figure S2a–f). In comparison, the SVM (Figure S3a–f), XGBoost (Figure S4a–f), and RF (Figure S5a–f) models achieved AUROC values of 0.809, 0.806, and 0.801, respectively, for the external test set (Table 2). The KNN model exhibited poorer performance compared with the other models and achieved an AUROC of only 0.796 for the external test set (Figure S6a–f). Table 2 presents the detailed performance metrics of these predictive models for the training sets, internal validation sets, and external test set.

3.3 ML model interpretation

The “SHAP” library identified the top 20 most important features in the Deep-HH model as follows: frequency of eating fruits, paying attention to nutritional balance, frequency of eating coarse grains, satisfaction with diet, grade, monthly expenditure on food, frequency of irregular eating, father's highest education level, understanding of essential nutrients, replenishing micronutrients after sweating, daily sun exposure time, weekly exercise time, mother's highest education level, birthplace, frequency of eating snacks, understanding of HH, frequency of eating fried food, reasons for irregular eating, understanding the relationship between HH and chronic disease, and the only child in the family (Figure 4a). Among these features, frequent consumption of fruits and coarse grains is associated with a lower risk of HH, whereas excessive intake of fried foods may increase it. Furthermore, maintaining a balanced diet, timely replenishment of micronutrients after sweating, and satisfaction with daily meals are correlated with a reduced HH risk (Figure 4b).

4 DISCUSSION

In this study, we developed effective predictive models for HH risk based on the demographic characteristics, nutritional awareness, and lifestyle habits of high school students. Additionally, we used the “SHAP” Python library for model explanation to investigate potential factors that affect the risk of HH in high school students, with the aim of leveraging this insight for early intervention against HH. Furthermore, we developed a web application based on the established model.

Currently, the detection of micronutrients in the human body is typically conducted using techniques such as HPLC, ELISA, and LC-MS/MS [31]. However, these techniques still encounter several limitations in practical applications. For example, studies have shown that HH is more prevalent in low-income and middle-income countries and regions [32]. Unfortunately, in these countries and regions, the medical infrastructure is often underdeveloped, lacking the necessary human resources, physical facilities, and financial resources to support large-scale laboratory screening for HH [33]. Therefore, a low-cost and high-efficiency detection method would be beneficial for the large-scale screening of HH, particularly in these areas. ML, because of its efficiency and convenience, is particularly well-suited for addressing these types of problems. For instance, the diagnostic model for acute appendicitis, developed using ML algorithms, significantly outperforms traditional clinical scoring systems and markedly accelerates the evaluation process. This advancement facilitates earlier medical treatment and substantially reduces labor costs [34]. Our Deep-HH model also effectively addressed the limitations inherent in current HH detection methods. By inputting basic personal information, users can assess their HH risk within minutes using our model, which dramatically reduces the time required compared with traditional methods. The online deployment of this model allows users to rapidly evaluate their HH risk via electronic devices, which significantly lowers equipment costs and reduces the reliance on professional staff. This is particularly beneficial for economically underdeveloped regions with limited resources, which provides our predictive model with a distinct advantage for rapid screening of HH in large-scale populations.

Traditionally, deep learning models have often been regarded as black boxes. However, in the medical field, attempting to interpret such black boxes has become crucial [35]. Therefore, we conducted a detailed analysis using the “SHAP” library. Our findings indicate that dietary habits significantly influence the risk of HH. Individuals who regularly consume fruits and coarse grains ensure nutritional balance in their diet, and those satisfied with their meals exhibit a reduced risk of HH. Conversely, those who frequently indulge in fried foods and exhibit irregular eating habits face an increased risk of HH. Research indicates that fruits are rich in vitamin C, vitamin K, magnesium, and carotenoids [36], and grains contain ample amounts of vitamins B, vitamin E, zinc, selenium, magnesium, and copper [37]. Furthermore, coarse grains are a significant source of dietary fiber, which can help mitigate the risk of coronary heart disease, diabetes, and certain gastrointestinal disorders [38]. Thus, regularly consuming fruits and coarse grains contributes to replenishing these essential nutrients, subsequently reducing the risk of HH. However, the prolonged consumption of substantial quantities of fried foods can elevate the risk of diabetes and might even heighten the risk of throat cancer [39, 40]. Consequently, we recommend minimizing the consumption of fried foods as much as possible. Beyond diet, our research indicates that individual lifestyle habits significantly influence the risk of HH. Specifically, those who quickly replace electrolytes after exercising and sweating, spend more time in the sun each week, and engage in longer durations of weekly physical activity show a reduced risk of HH. Existing studies show that the synthesis of vitamin D in the human body is closely linked to sun exposure duration [41]. Hence, extended periods of sunlight exposure contribute to the efficient synthesis of vitamin D, thereby facilitating nutritional replenishment and consequently lowering the risk of HH. Additionally, we discovered a significant correlation between an individual's familiarity with nutritional knowledge and the individual's risk of HH. Those more knowledgeable about nutrition face a lower risk of HH, which is likely because of their proactive engagement in preventive interventions. Intriguingly, our findings indicate that the higher the educational attainment of the participants' fathers, the lower their risk of HH. This is likely to be linked to improved economic status and enhanced nutritional understanding [42, 43].

Our predictive model demonstrated high accuracy in both the internal validation and external test sets. However, it still showed a gap compared with the HHAS, potentially because of the model's limited focus on the quantitative assessment of diet. Quantitatively evaluating diet increases the complexity of implementation and requires professional guidance, which can impede the rapid screening of HH risk in large-scale populations. Despite this, certain improvements can help to mitigate this gap. Through model interpretation, we identified that dietary habits, particularly the frequency of consuming fruits and coarse grains, significantly impact the risk of HH. Therefore, when subjects demonstrate a marked reduction in the frequency of consuming fruits, coarse grains, and similar foods, it is crucial to direct increased attention toward these cases to enhance the model's detection rate.

However, there were still some limitations to this study. For instance, the groups predicted as high-risk in this study still require further validation through laboratory testing methods. Additionally, this predictive model was established based on a population of high school students and may not be applicable to other age groups. Therefore, there is a need for future research to develop predictive models based on a more diverse sample.

5 CONCLUSIONS

We developed an efficient, accurate, and cost-effective ML-based risk prediction model for HH. This model will facilitate the screening of HH in large-scale populations.

AUTHOR CONTRIBUTIONS

Yang Yang: Conceptualization (equal); data curation (equal); formal analysis (equal); investigation (equal); methodology (equal); software (equal); validation (equal); visualization (equal); writing—original draft (equal). Zheng Zhang: Conceptualization (equal); data curation (equal); formal analysis (equal); investigation (equal); methodology (equal); software (equal); validation (equal); visualization (equal); writing—original draft (equal). Huake Cao: Formal analysis (equal); investigation (equal); validation (equal); writing—original draft (equal). Yuchen Zhang: Formal analysis (equal); investigation (equal); validation (equal); writing—original draft (equal). Minao Wang: Validation (equal); visualization (equal). Ning Zhang: Funding acquisition (equal); project administration (equal); resources (equal); supervision (equal); validation (equal); writing—review & editing (equal).

CONFLICT OF INTEREST STATEMENT

The authors declare there are no competing interests.

ETHICS STATEMENT

This study was approved by the ethics committee of Anhui Medical University (20190495).

INFORMED CONSENT

All patients provided written informed consent at the time of entering this study.