Skip to main content

Development and validation of a predictive model for suicidal thoughts and behaviors among freshmen

Abstract

Background

There are fewer studies on prospective predictors of first-time suicidal thoughts and behaviors (STB) among first-year university students and fewer studies prospectively identifying and screening for those at high risk of suicide among college students. This study assessed the impact of prospective baseline variables on the risk of new STB onset among first-year university students over two years and developed a multivariate risk prediction model.

Methods

4,560 first-year university students (38.4% males, mean age:18.34) from China participated and completed this prospective cohort study over a three-year period from 2018 to 2020. LASSO regression, and logistic regression models under resilient networks, were used for risk predictor variable screening and final prediction model building. Independent validation sets were used for external validation of the models. Independent validation sets were used for external validation of the models. Area Under the Curve (AUC), accuracy, F1 scores, and Hosmer-Lemeshow test metrics were used to evaluate the model performance.

Results

The incidence rates of suicidal thoughts, suicidal behaviors, and STB within two years were 4.89%,1.03%, and 4.96%, respectively. Predictors in the final model included females, always solo activity, bigotry under pressure, socially oriented perfectionism, drinking to relieve stress, autonomy attitude, poorer parental marriage satisfaction, maternal emotional warmth, perceived others social support, and number of lifetime severe traumatic events. The predictive model had an AUC of 0.738 (95% CI: 0.697–0.780) for predictive accuracy in the training dataset as well as 0.710 (95% CI: 0.657–0.763) for predictive accuracy in the validation dataset, which represents a high degree of model discrimination.

Conclusion

Based on this predictive model of suicidal thoughts and behaviors, this study may help to assess and screen college students at risk for STB and develop suicide prevention strategies for at-risk populations.

Peer Review reports

Introduction

Globally, more than 700,000 people die by suicide each year, making it the fourth leading cause of death among 15–29-year-olds. About 88% of adolescents who died by suicide in 2019 were from developing countries [1]. The youth suicide mortality rate had declined significantly as of 2015 following the adoption of a range of suicide prevention strategies in China [2]. The suicide death rate among Chinese teenagers (Age:15–19) gradually increased from 2017 to 2021, and the suicide mortality rate in 2021 is at 3.63 per 100,000 people [3]. Suicidal thoughts are defined as thoughts of harm or suicide [4]. In recent years, adolescent suicidal thoughts in the United States have similarly shown a downward and then upward trend [5]. A survey from eight high-income countries found that the 12-month prevalence of suicidal thoughts and behaviors among university students was 10.6 per cent and 1.2 per cent between 2014 and 2017 [6]. The study showed that the 12-month prevalence of suicidal thoughts among Chinese college students from 2021 to 2023 was 3.89%, 5.81% and 4.33%, respectively, showing a changing trend [7]. The rising trend and high prevalence of suicides indicate that suicide prevention strategies continue to face significant challenges.

University freshmen entering university life and in the transition to adolescence face stressors such as exams, living away from family, and financial difficulties, which make the first year of university life stressful [8], with a wide distribution of suicidal thoughts and a lifetime prevalence of 32.7% [9]. Additionally, the mental health of college students continues to deteriorate [10, 11]. Self-reported experiences of mental illness account for approximately one in four college students [12], and college students with mental health problems, exceptionally moderate and severe symptoms of depression and anxiety, are ten times more likely to have suicidal thoughts than those with undiagnosed mental health problems [13]. More than two-thirds of college students with major depression suffer from suicidal thoughts, and depressive symptoms, psychoticism, and neuroticism directly influence suicidal thoughts [14]. The factors that influence suicidal thoughts and behaviors are complex, and no single factor is effective in predicting an individual’s level of suicidality [15]. Firstly, suicidal thoughts and behaviors at baseline have been shown to be the primary predictor of new-onset suicidal thoughts and behaviors [16]. In addition, other variables that are associated with suicidal thoughts and behaviors are sexual abuse, academic-related variables [17], female, stress, and major depression [18], adverse life events [19], family environment [20], sleep quality [21] and poor lifestyle habits such as hazardous drinking [22].

Few studies are using first-year college students as research subjects to construct suicide risk prediction. Previously, logistic regression models were mainly used to study risk factors for suicide [15]. For example, in a Belgian study with a two-year follow-up of first-year students, the strongest predictors were dating violence before the age of 17 and severe personal betrayal within one year [23]. In a model predicting suicide risk among first-year students at multiple universities in Mexico, factors such as being female, minority sexual orientation and depression, reflected strong predictive effects [24]. In addition, machine learning methods, such as Random Forest and linear support vector machines, have also been applied to predict the risk of suicidal thoughts among college students [25, 26]. The predictor variables and predictive efficacy reported in the current studies vary widely due to differences in the study population and included factors. In addition, there are apparent racial/ethnic differences in the trends of suicidal thoughts [5] and differences in suicidal thoughts between Chinese and foreign college students [27, 28]. Therefore, there is a need to develop new predictive models to identify high-risk groups for suicide among first-year university students in different populations and settings to avoid and reduce the social burden of suicidal thoughts and behaviors.

The study aimed to (1) assess the two-year incidence of new-onset suicidal thoughts and behaviors among first-year university students in China; (2) incorporate and analyze predictive factors such as socio-demographic characteristics, mental health, adverse life events, and familial factors; and (3) develop and validate a risk prediction model for suicidal thoughts and behaviors among first-year university students.

Materials and methods

Study population

The cohort study was conducted from 2018 to 2020 in Shandong Province, China, using a Cluster sampling method with all first-year students from Jining Medical College and Weifang Medical College in Shandong Province. Between April and October 2018, 9,928 freshmen from the three campuses of Jining, Rizhao and Weifang served as survey respondents, and a total of 8,079 students eventually completed the baseline survey, with a response rate of 81.4%. Participants with lifetime suicidal thoughts and behaviors were excluded (n = 958). The first follow-up of 7121 eligible participants was conducted from April 2019 to October 2019, and 6638 (93.2%) participants provided complete data. The second follow-up was conducted from April 2020 to October 2020, with a total of 4560 (68.7%) students completing the follow-up (See Appendix Fig. 1). Participants signed an informed consent form agreeing to participate in the survey and subsequent follow-up. The study was reviewed and approved by the Health Committee of Jining Medical University (2019-JS-004).

Fig. 1
figure 1

The receiver operating characteristic curve (ROC) of the training set and validation set after elastic network regularization

Data collection

Baseline and first follow-up data were collected in the libraries of the two schools. The questionnaires were in a computer-assisted self-service management system with embedded voice prompts and logical jumps. The computers were distributed in 265 units in Jining Medical College Jining Campus, 50 in Rizhao Campus, and 50 in Weifang Medical College. Influenced by COVID-19 [29], the second follow-up data was conducted through an online survey (www.wjx.cn). The surveys were timed to avoid the exam revision phase. The data collection process was anonymized, and confidentiality of the data was protected.

Sample size

Our sample size calculation is based on our primary objectives, namely the emergence of suicidal thoughts and behaviors. We require a maximum number of participants to ensure the stability of the predictive model. Following the sample size calculation method for predictive models, using the standard of 10 times the events per variable (EPV), with a final set of 10 predictor variables, we need 100–200 participants for stable estimation [30]. According to prior surveys on suicidal thoughts and behaviors among university students, the incidence rate of suicidal thoughts and behaviors (STB) in December is approximately 10.6%. If our event rate is 10.6%, we need 943 to 1886 respondents. Considering a loss rate of less than 30%, we estimated a sample size of 1347 to 2694 respondents.

Missing data

Samples with missing ending variables were excluded. The missingness of the predictor variables was calculated, where variables with missing values higher than 15 per cent were excluded. Multiple interpolation in the mice package was used to interpolate the missing values for variables below 15%, using R 4.3.2 [31]. Multiple interpolation is an effective method for interpolating various types of missing values and minimizes the bias introduced by excluding such participants.

Predictor variables

This study collected four characteristics: baseline demographic variables, mental health, traumatic events, and family factors. Demographic characteristics included gender, age, and household registration. Mental health included personality traits, the Functional Attitude Dysregulation Scale, and depressed mood. A traumatic event is a lifetime of traumatic severe events experienced throughout a lifetime. Family factors included parenting styles and the home environment. Binary (0, 1) and reverse coding of some scales.

Socio-demographic variables

Socio-demographic variables include sex (male or female), age, registration (urban or rural), and city (Jining, Rizhao, Weifang).

Mental health factors

Personality traits

Borderline personality traits and schizotypal personality traits were generalized from their respective corresponding nine domains in the Diagnostic and Statistical Manual of Mental Disorders [32]. For example, subjects were asked if they agreed that they “almost always rush to do activities alone” and “when stressed, I become paranoid, stubborn, suspicious, and even feel self-separated.” The answer to each question was either “yes” or “no.” When a subject gave five or more “yes” answers, they were considered to have borderline personality traits. The Cronbach’s α of this scale in this study is 0.838.

Dysfunctional attitudes scale

The DAS is a self-assessment questionnaire consisting of 40 items comprising several dysfunctional conditions to assess the underlying deeper cognitive structure of people, rated according to the degree of agreement of the individual with these dysfunctional conditions, ranging from complete disagreement to complete agreement, on a seven-point scale from 1 to 7 [33]. The Cronbach’s α of this scale in this study is 0.814.

Multidimensional perfectionism scale

The Multidimensional Perfectionism Scale (HMPS). The HMPS has three dimensions, each with 15 questions and a total of 45 items on a seven-point scale and requires the participant to make a choice between 1 (strongly disagree) and 7 (strongly agree) [34]. The three dimensions are self-oriented perfectionism, other-oriented perfectionism, and socially oriented perfectionism. The Cronbach’s alpha of this scale in this study is 0.656. Cronbach’s α for the three sub-dimensions were 0.862, 0.761, and 0.688, respectively.

Perceived social support scale

The PSSS is a tool to measure an individual’s self-awareness of multidimensional social support [35]. The PSSS consists of three dimensions: family support (items 3, 4, 8, and 11), friend support (items 6, 7, 9, and 12), and other support (items 1, 2, 5, and 10), with a total of twelve items on a seven-point scale (from 1 to 7) ranging from “Strongly Disagree” to “Strongly Agree”. The Cronbach’s alpha of this scale in this study is 0.951. Cronbach’s α for the three sub-dimensions were 0.908, 0.940, and 0.889.

Lifetime severe traumatic events

A self-administered questionnaire was used to understand the status of Lifetime Severe Traumatic Events. The questionnaire consists of 23 questions, each corresponding to two answers of “yes” or “no.” A complete list of questionnaires can be found in Appendix Table 1.

Parenting rearing styles

Parenting styles were assessed using the revised Chinese version of the Egna Minnen Beträffande Uppfostran (EMBU) scale [36]. The scale consists of 66 items, including 58 items in 6 dimensions for the father’s parenting style subscale and 57 entries in 5 dimensions for the mother’s parenting style subscale. Each item is rated on a four-point scale from 1 (never) to 4 (always). Cronbach’s alpha of this scale in this study is 0.967. The Cronbach’s α for the six dimensions, including father’s warmth, were 0.932, 0.906, 0.723, 0.809, 0.804, and 0.589. The Cronbach’s α for the five sub-dimensions, including mother’s warmth, were 0.921, 0.912, 0.738, 0.788, and 0.808.

Other variables

(1) Subjectively, how satisfied are you with your parents’ marital status? (answer satisfied or dissatisfied). (2) Relieve stress by drinking (answer “yes” or “no”). (3) Are you an only child? (answer “yes” or “no”). (4) Is the family a nuclear type? i.e. a family consisting of parents and children (answer “yes” or “no”).

Baseline and new onset suicidal thoughts and behaviors

Baseline suicidal thoughts and behaviors

This was assessed through the following questions: (1) Have you had suicidal thoughts in the past year? Have you engaged in suicidal behavior within the past year?’ Answer ‘yes’ or ‘no’. If either question was answered ‘yes’, the participant was considered to have STB. (2) At what age did you first experience suicidal thoughts? At what age did you first experience suicidal behavior? The presence of baseline suicidal thoughts and behaviors was determined if the answer to question (1) was ‘yes’ or if the age of the answer to question (2) was younger than the baseline age.

New onset of suicidal thoughts and behaviors

Suicidal thoughts and behaviors were assessed at the first and second follow-up visits through the following question: ‘Have you had suicidal thoughts within the past year?’ Have you engaged in suicidal thought behaviors within the past year?’ The answer was ‘yes’ or ‘no.’ If the answer to the baseline suicidal thoughts and behaviors was ‘No’, but the answer was ‘Yes’ in any of the years of follow-up, the subject was judged to have developed new suicidal thoughts and behaviors.

Statistical analysis

Feature selection

The baseline features included a total of 970 variables. According to the Modified 80% rule, if the proportion of non-missing data of a variable is less than 80% of the total sample size, it is recommended to delete the variable [37]. In this study, feature variables with more than 15% missing data (n = 70) were excluded on this basis. Firstly, the missing values of the remaining 900 feature variables were interpolated using multiple interpolation. Subsequently, 123 variables with variance close to or equal to 0 were eliminated by filtering variance. LASSO regression was then performed on the remaining 777 feature variables to reduce the coefficients of the weak predictors to 0, resulting in 54 variables (see Appendix Fig. 2). LASSO regression minimizes the correlation between variables and avoids overfitting [38]. The backward stepwise regression method using Akaike’s Information Criterion (AIC) continued to filter variables, resulting in 24 variables [39]. The remaining 24 variables were subjected to backward stepwise regression analyses, and the variables with the smallest change in ΔAUC were used as the basis for variable deletion to obtain the most appropriate predictive models for the 10 variables (see Appendix Table 3). The final 10 predictor variables were found to be free of multicollinearity (VIA < 5) and not strongly correlated with each other (r < 0.3) [40, 41]. In addition, interactions between variables in this model were excluded.

Fig. 2
figure 2

Decile probability plot of risk for suicidal thoughts and behaviors

Analysis method

The study sample was first described in general terms. Continuous variables were presented with mean ± standard error. Categorical variables were presented with proportions. All tests were two-tailed, and P < 0.05 was considered statistically significant. All statistical analyses were performed using R (version 4.3.0) [42].

Establishment of predictive models

In order to establish an external validation set, this study divided college students from different schools into training and validation sets. The training set included samples from a university in Jining and Rizhao areas (n = 2934), and the external validation set included samples from another school in Weifang area collected independently during the same period (n = 1624). We tested the potential of eigenvalues to predict suicidal thoughts and behaviors using logistic regression. A predictive model for suicidal thoughts and behaviors was developed using logistic regression for the final 10 predictor variables after screening, and the potential of the model to predict suicidal thoughts and behaviors was tested.

Assessment of model performance

The model’s performance was assessed based on its discrimination, calibration curve, clinical applicability, and generalizability. Hosmer-Lemeshow tests and elasticity networks were used to view the fitness, and the final logistic predictive model was regularized. The calibration of the model was assessed by comparing the predicted values with the observed results, and the calibration curves were visualized in this study using a 1000-times bootstrapping procedure. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), F1 scores, accuracy, and Brier score of the final model in the training and validation sets were assessed. The assessment of clinical suitability was performed by decision curve analysis (DCA), which quantifies the net benefit at different threshold probabilities [43]. Focused high-risk groups (10–30%) are necessary to priorities the allocation of intervention resources. Therefore, in this study, the predicted risk probability was decimated using 10% as the risk threshold and cross-classified with observed cases to explore more intuitively the model’s ability to identify high-risk groups under different risk stratifications. The predicted probabilities were discretized into deciles, compared to the actual observed STB risk probabilities, and subsequently visualized to compare the risk concentration of the predicted probabilities.

Internal and external validation

10-fold cross-validation was used for internal validation of model performance in the training set [44]. In addition, an independent external validation set was used in this study to assess the performance of the final model. The magnitude of the area under the experimenter operating characteristic curve (AUC) was used to assess the overall diagnostic accuracy of the final model in both the training set and the external validation set [45].

Results

Sample characteristics

Table 1 shows the distribution of demographic characteristics and risk factors between the training and validation sets. The training set had 2934 (64.3%) individuals with a mean age of 18.43 years and 1190 (40.6%) males. The validation set had 1626 (35.7%) individuals, with a mean age of 18.18 years, and 1190 (34.5%) males. New onset of suicidal thoughts and behaviors over two years among the 4560 participants totaled 226 (5.0%), with 146 (5.0%) in the training set and 80 (4.9%) in the validation set. More information is detailed in Table 1. This study assessed the distribution of baseline demographic characteristics of participants who completed and did not complete follow-up (see Appendix Table 2). The total number of people who completed the follow-up was 4560(64.0%), with a mean age of 18.38 years and 1751 (38.4%) males. The results showed significant differences in the distribution of age (t = 4.80, P < 0.001), sex (χ2 = 39.19, P < 0.001), and only child (χ2 = 6.03, P = 0.028) between the missed and completed follow-up cohorts.

Table 1 Distribution of demographic characteristics and risk factors in the training set and validation set

Incidence

The incidence of new-onset suicidal thoughts, suicidal behaviors, and STB at baseline was 13.44% (n = 957; 95% CI = 12.67-14.25%), 1.77% (n = 126; 95% CI = 1.49-2.10%) and 13.45% (n = 958; 95% CI = 12.68-14.27%), respectively. The incidence rates of suicidal thoughts, suicidal behaviors, and STB within two years were 4.89% (n = 223; 95% CI = 4.30-5.56%) and 1.03% (n = 47; 95% CI = 0.78-1.37%) and 4.96% (n = 226; 95% CI = 4.36-5.62%), respectively.

Predictor selection

Table 2 shows the final model for the ten risk predictors. Among the female (OR = 2.16, 95%CI:1.48–3.22), always solo activity (OR = 2.09, 95%CI:1.32–3.23), bigotry under pressure (OR = 2.03, 95%CI:1.37–2.98), socially oriented perfectionism (OR = 1.54,95%CI:1.07–2.20), drinking to relieve stress (OR = 2.28, 95% CI:1.27–3.91), autonomy attitude (OR = 1.50, 95%CI:1.05–2.15), parental marital satisfaction (OR = 1.58, 95%CI:1.01–2.42), mother emotional warmth (OR = 0.54, 95% CI:0.38–0.79), others social support (OR = 0.59, 95%CI:0.38–0.95), number of lifetime severe traumatic events (OR = 1.14, 95%CI:1.02–1.27). Appendix Table 4 shows the model specifications for the final prediction model for new-onset suicidal thoughts and behaviors.

Table 2 Final model for risk prediction of suicidal thoughts and behaviors among university freshmen

Model representation and construction of line charts

Table 3 shows the performance of the final model. The AUC of the final prediction model was calculated to be 0.738 and 0.710 in the training and validation sets, respectively (Fig. 1). The results also showed a good fit by decile grouping and visualizing the actual STB occurrence risk and the predicted risk in both sets (Fig. 2). This implies that the final model agrees with both the training and validation sets. The AUC (95%CI), sensitivity, specificity, positive predictive value, negative predictive value, F1 score, accuracy, Brier score, and Hosmer-Lemeshow test metrics for the training and validation sets are detailed in Table 3. In both the training and validation sets, the calibration curves for both the Bias-corrected and ideal showed a better fit, demonstrating good agreement between the predicted and observed extent of STB (see Appendix Fig. 3). The AUC (95%CI) of the ten-fold cross-validation of the final prediction model in the training set is 0.721 (0.579–0.827) (See Appendix Fig. 4). Furthermore, the DCA shows that the algorithm produces more net benefits when all predictors are considered at threshold probabilities of 2–36% (see Appendix Fig. 5).

Table 3 Identification performance of a risk prediction model for suicidal thoughts and behaviors among university freshmen

In Table 4, the study presents the sensitivity, PPV, and F1 scores of students at different proportions of predicted risk based on the final prediction model. The results show that based on the prediction model, the actual percentage of students predicted to be at the highest 10% risk of STB within two years was 35.62% of all observed cases of new onset of suicidal thoughts and behaviors. Moreover, the probability of new STB among the 10% of students with the highest predicted risk was 17.75%, much higher than the 4.95% in the whole population. In addition, the relatively high F1 scores (23.69%) in the strictly segmented case (top 10% predicted risk) suggests that the model has some potential for balancing underdetection and false positives.

Table 4 Concentration of risk for suicidal thoughts and behaviors at follow-up based on different proportions of students with the highest baseline predicted risk under the final prediction model

The ten best predictor variables obtained from the screening were used to construct the suicide nomogram, as shown in Appendix Fig. 6. In this suicide nomogram, the options for each variable corresponded to their respective scores, and the scores for all variables corresponding to the options were summed to obtain the total score. The predicted risk probabilities of suicidal thoughts and behaviors that do not correspond to the total score are given at the bottom of Appendix Fig. 6. A higher total score represents a higher probability of risk of STB.

Discussion

Identifying new students at high risk for STB will help prevent and reduce the incidence of suicide. Using a logistic regression algorithm in this sample of college students, we identified ten baseline variables that predicted new-onset suicidal thoughts and behaviors over two years: female, always solo activity, bigotry under pressure, socially oriented perfectionism, drinking to relieve stress, autonomy attitude, parental marital satisfaction, mother emotional warmth, others social support, number of lifetime severe traumatic events.

In the present study, when all significant factors in the predictive modelling algorithm were considered, the prediction of the first new STB within two years was as high as 73% for university students. The final model in this study had an accuracy of 69.7–78.0% in predicting the first occurrence of STB within two years in university students. Although the final predictors varied, this prediction estimate was like studies of new-onset STB risk prediction among university students in Belgium (accuracy of 84-91%) and Mexico (accuracy of 76-81%), which also used logistic regression algorithms [23, 24]. In addition, the French study on predicting new-onset STB in university students used a machine-learning approach to divide the population into males and females (with an accuracy of 0.74% and 0.72%) [26]. Baseline suicidal thoughts tended to be one of the strongest predictors when the study population did not exclude the baseline population with suicidal thoughts and behaviors [26, 4647] when prediction accuracy tended to be higher than in the study population without STB. Identifying college freshmen with substantial risk factors for STB by identifying them at enrollment and targeting preventive interventions is an effective strategy for preventing the occurrence of STB. And using the STB risk prediction algorithm of this study and focusing interventions on the 10 per cent of students with the highest predicted risk would approximately target close to one in five positive cases. The algorithm has the obvious disadvantage that false-positive cases will inevitably occur, making the participation of a proportion of normal students in the intervention often requiring secondary screening to reduce false positives. This also means that the algorithm still has much room for improvement, including the use of interventions that should be as non-intrusive as possible whilst still ensuring that they are strong and effective, and more often targeting general mental health problems. Nonetheless, the model performs better when focusing on high-risk groups and still improves the efficiency of resource use compared to generalized interventions and random allocation.

In this study, females are at a higher risk of developing STB than males. This is in line with the findings of several previous studies on the prediction of suicide risk in university students [9, 48]. The results of several meta-analyses have shown that suicidal thoughts, suicidal planning, and suicidal behaviors are higher in females than in males [49, 50]. Bigotry under pressure belongs to borderline personality disorder (BPD), and recurrent suicidal behavior is one of its characteristics [51]. This emotional instability has been shown to play a role in the increased risk of suicidal behavior in BPD patients [52]. Always moving alone belongs to Schizoid Personality Disorder (SPD), and this solitary lifestyle is higher in severe suicide attempters [53]. Consistent with other reports, those who drank alcohol [22] experienced a traumatic event [54]; consider that work not good enough is not recognized as perfectionism [55] and Thinking that not getting things done is not respected [56], as Poor parental marital status [57] had a higher risk of having STB. In contrast, maternal warmth [58] and Support from relatives and leaders in case of a problem [59] reduce the risk of STB.

Notably, some predictors are intervenable and controllable. First, in terms of family factors, family therapeutic interventions such as family training courses and family psychoeducational programs are effective for college students with STB [60]. Secondly, in terms of social support, individual interventions (e.g., providing support through face-to-face, phone calls, etc.) and group interventions (e.g., regular meetings, walking tours, etc.) can be used to reduce the incidence of STB [59]. Finally, early screening for borderline personality disorder and schizotypal personality disorder traits should be carried out as early as possible. Although dialectical behavior therapy’s effect on STB is insignificant, short-term interventions, such as planned generic support, are effective in ameliorating personality disorders [61, 62]. Nevertheless, it is necessary to develop targeted interventions based on the prediction model algorithm in future studies and to verify the effectiveness of the prediction model interventions.

This study has certain advantages. First, unlike cross-sectional studies, this study identified multiple risk predictors of new-onset STB through a longitudinal cohort of up to 3 years, and the predictive model built based on these risk factors had a discriminatory degree of up to 73.8%. Secondly, although overfitting could not be avoided, this study used LASSO regression and elasticity network regularization to reduce the degree of model overfitting during variable selection and model building. Finally, complex predictive equations were visualized by constructing STB risk nomogram to help school or clinical psychologists understand and guide intervention decisions. Inevitably, this study has several limitations. First, as a three-year follow-up study, this study had a high natural omission rate. Despite the use of multiple interpolation to minimize the effect of missing samples, there is still a one-third omission rate. And a higher omission rate may reduce the statistical effect and thus the internal validity of the model. Secondly, due to the limitation of the number of positive cases, this study considered suicidal thoughts and suicidal behaviors as a single outcome variable, which failed to predict and explain the risk of suicidal thoughts and suicidal behaviors separately and may have overlooked the different predictors and developmental trajectories of the two. In addition, data were collected using self-report rather than through professional clinical diagnosis, which may introduce recall bias and social desirability bias. Finally, in terms of population selection, the sample selection in this study was relatively homogenous and suffered from some sampling bias, making extrapolation of the model limited.

Based on the limitations of this study, the following suggestions are made for future research: first, to enhance the compliance of the subjects who joined the follow-up study and to reduce the dropout rate. Secondly, in future studies, suicidal thoughts and behaviors at baseline should be added as significant predictors of suicide and the severity of STB considered in defining outcome variables should be examined to explore and develop a more complete predictive model of new-onset STB in university students. Additionally, future studies should provide risk prediction and interpretation of suicidal thoughts and suicidal behaviors separately to provide more specific clinical guidance. In addition, a more rigorous validation approach for external validation should be used, such as selecting independent samples from different time periods as the validation set. Finally, in terms of population selection, college students from different cultures as well as different countries or regions were chosen as study subjects to validate and generalize the prediction model.

Conclusion

As a prospective longitudinal study, this study developed a suicide prediction model consisting of 10 risk factors, including specific personality traits such as perfectionism, psychological cognition, and family environmental factors, using new STBs of first-year college students over a two-year period as a risk outcome. A screening tool based on this predictive model can identify students at high risk for suicidal thoughts and behaviors and alert at-risk college students to contact counseling and proactively engage in mental health services. It can help school mental health care providers, educators, and psychological organizations develop effective suicide risk interventions to reduce the burden of suicide among college students.

Data availability

To protect the privacy of the participants, the datasets generated and/or analyzed in this study were not made available to the public but are available from the corresponding authors on request.

References

  1. Organization W. 2019. Suicide in the world: global health estimates, World Health Organization. Switzerland. Retrieved from https://coilink.org/20.500.12592/pk2kx2 on 06 Nov 04. COI: 0.500.159/pkkx.

  2. Jiang H, Niu L, Hahne J, Hu M, Fang J, Shen M, Xiao S. Changing of suicide rates in China, 2002–2015. J Affect Disord. 2018;240:165–70.

    Article  PubMed  Google Scholar 

  3. Tian Y, Luan M, Chen H, Zhang J. Temporal trend in suicide mortality for Chinese adolescents, 2008 to 2021. JAMA Pediatr. 2023.

  4. Zhang X, Wang H, Xia Y, Liu X, Jung E. Stress, coping and suicide ideation in Chinese college students. J Adolesc. 2012;35:683–90.

    Article  PubMed  Google Scholar 

  5. Xiao Y, Cerel J, Mann JJ. Temporal trends in suicidal ideation and attempts among US adolescents by sex and race/ethnicity, 1991–2019. JAMA Netw Open. 2021;4:e2113513.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Mortier P, Auerbach RP, Alonso J, Bantjes J, Benjet C, Cuijpers P, Ebert DD, Green JG, Hasking P, Nock MK, O’Neill S, Pinder-Amaker S, Sampson NA, Vilagut G, Zaslavsky AM, Bruffaerts R, Kessler RC. Suicidal thoughts and behaviors among first-year college students: results from the WMH-ICS project. J Am Acad Child Adolesc Psychiatry. 2018;57:263–e731.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Yao ZY, Xu XM, Li SJ, Zhou XY, Li SY, Li R, Kou CG, Yao B, Shen ZZ, Ma YB, Wang XT, Liu BP, Cheng SL, Gao JG, Wong JP, Fung KP, Jia CX. The prevalence of 12-month suicidal ideation and associated factors among university students in China: findings from a three-wave cross-sectional study from 2021 to 2023. J Affect Disord. 2024;367:668–77.

    Article  PubMed  Google Scholar 

  8. Auerbach RP, Mortier P, Bruffaerts R, Alonso J, Benjet C, Cuijpers P, Demyttenaere K, Ebert DD, Green JG, Hasking P, Murray E, Nock MK, Pinder-Amaker S, Sampson NA, Stein DJ, Vilagut G, Zaslavsky AM, Kessler RC, WHO WMH-ICS Collaborators. WHO World Mental Health Surveys International College Student Project. Prevalence and distribution of mental disorders. J Abnorm Psychol. 2018;127(7):623–38.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Mortier P, Cuijpers P, Kiekens G, Auerbach RP, Demyttenaere K, Green JG, Kessler RC, Nock MK, Bruffaerts R. The prevalence of suicidal thoughts and behaviours among college students: a meta-analysis. Psychol Med. 2018;48:554–65.

    Article  CAS  PubMed  Google Scholar 

  10. Knapstad M, Sivertsen B, Knudsen AK, Smith ORF, Aarø LE, Lønning KJ, Skogen JC. Trends in self-reported psychological distress among college and university students from 2010 to 2018. Psychol Med. 2021;51:470–8.

    Article  PubMed  Google Scholar 

  11. Lipson SK, Zhou S, Abelson S, Heinze J, Jirsa M, Morigney J, Patterson A, Singh M, Eisenberg D. Trends in college student mental health and Help-Seeking by race/ethnicity: findings from the National healthy Minds study, 2013–2021. J Affect Disord. 2022;306:138–47.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fekih-Romdhane F, Pandi-Perumal SR, Conus P, Krebs MO, Cheour M, Seeman MV, Jahrami HA. Prevalence and risk factors of self-reported psychotic experiences among high school and college students: A systematic review, meta-analysis, and meta-regression. Acta Psychiatr Scand. 2022;146:492–514.

    Article  PubMed  Google Scholar 

  13. Casey SM, Varela A, Marriott JP, Coleman CM, Harlow BL. The influence of diagnosed mental health conditions and symptoms of depression and/or anxiety on suicide ideation, plan, and attempt among college students: findings from the healthy Minds study, 2018–2019. J Affect Disord. 2022;298:464–71.

    Article  PubMed  Google Scholar 

  14. Zhou SC, Luo D, Wang XQ, Zhu J, Wu S, Sun T, Li XY, Kang L, Ma S, Lu B, Liu Q, Yang BX, Liu Z. Suicidal ideation in college students having major depressive disorder: role of childhood trauma, personality and dysfunctional attitudes. J Affect Disord. 2022;311:311–8.

    Article  CAS  PubMed  Google Scholar 

  15. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, Musacchio KM, Jaroszewski AC, Chang BP, Nock MK. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol Bull. 2017;143:187–232.

    Article  PubMed  Google Scholar 

  16. Shen Y, Zhang W, Chan BSM, Zhang Y, Meng F, Kennon EA, Wu HE, Luo X, Zhang X. Detecting risk of suicide attempts among Chinese medical college students using a machine learning algorithm. J Affect Disord. 2020;273:18–23.

    Article  PubMed  Google Scholar 

  17. Altavini CS, Asciutti APR, Santana GL, Solis ACO, Andrade LH, Oliveira LG, Andrade AG, Gorenstein C, Wang YP. Suicide ideation among Brazilian college students: relationship with academic factors, mental health, and sexual abuse. J Affect Disord. 2023;329:324–34.

    Article  PubMed  Google Scholar 

  18. Ndetei DM, Mutiso VN, Weisz JR, Okoth CA, Musyimi C, Muia EN, Osborn TL, Sourander A, Wasserman D, Mamah D. Socio-demographic, economic and mental health problems were risk factors for suicidal ideation among Kenyan students aged 15 plus. J Affect Disord. 2022;302:74–82.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Zheng H, Liu D, Cheng J, Wang DB, Liu Y, Wu Y. Negative life events increase the risk of suicidal ideation in 6653 Chinese freshmen: from a 1-year longitudinal study. J Affect Disord. 2022;299:604–9.

    Article  PubMed  Google Scholar 

  20. Wang M, Kou C, Bai W, Song Y, Liu X, Yu W, Li Y, Hua W, Li W. Prevalence and correlates of suicidal ideation among college students: A mental health survey in Jilin Province, China. J Affect Disord. 2019;246:166–73.

    Article  CAS  PubMed  Google Scholar 

  21. Xu H, Yang X, Lai X, Zhao C, Tu X, Ding N, Ruan S, Jiang Y, Lv Y, Zhang G. Longitudinal relationships among perceived stress, suicidal ideation and sleep quality in Chinese undergraduates: A cross-lagged model. J Affect Disord. 2022;299:45–51.

    Article  CAS  PubMed  Google Scholar 

  22. Paulus DJ, Capron DW, Zvolensky MJ. Understanding hazardous drinking and suicidal ideation and suicide risk among college students: anxiety sensitivity as an explanatory factor. Cogn Behav Ther. 2021;50:378–94.

    Article  PubMed  Google Scholar 

  23. Mortier P, Demyttenaere K, Auerbach RP, Cuijpers P, Green JG, Kiekens G, Kessler RC, Nock MK, Zaslavsky AM, Bruffaerts R. First onset of suicidal thoughts and behaviors in college. J Affect Disord. 2017;207:291–9.

    Article  CAS  PubMed  Google Scholar 

  24. Benjet C, Borges G, Miah S, Albor Y, Gutiérrez-García RA, Zavala Berbena A, Guzmán R, Vargas-Contreras E, Hermosillo de la Torre AE, Hernández, Uribe PC, Quevedo G, Covarrubias Díaz A, Martínez Ruiz S, Valdés-García KP, Martínez Jerez AM, Mortier P. One-year incidence, predictors, and accuracy of prediction of suicidal thoughts and behaviors from the first to second year of university. Depress Anxiety. 2022;39:727–40.

  25. Han K, Ji L, Chen C, Hou B, Ren D, Yuan F, Liu L, Bi Y, Guo Z, Wu N, Feng M, Su K, Wang C, Yang F, Wu X, Li X, Liu C, Zuo Z, Zhang R, Yi Z, Xu Y, He L, Shi Y, Yu T, He G. College students’ screening early warning factors in identification of suicide risk. Front Genet. 2022;13:977007.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Macalli M, Navarro M, Orri M, Tournier M, Thiébaut R, Côté SM, Tzourio C. A machine learning approach for predicting suicidal thoughts and behaviours among college students. Sci Rep. 2021;11:11363.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Zhang J, Liu Y, Sun L. Psychological strain and suicidal ideation: A comparison between Chinese and US college students. Psychiatry Res. 2017;255:256–62.

    Article  PubMed  Google Scholar 

  28. Lew B, Osman A, Huen JMY, Siau CS, Talib MA, Cunxian J, Chan CMH, Leung ANM. A comparison between American and Chinese college students on suicide-related behavior parameters. Int J Clin Health Psychol. 2020;20:108–17.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Liu Y, Li B, Hao F, Wang B, Zhu J, Liu D, Zhai J, Chen M, Wang J. Associations between borderline personality disorder features and the risk of first onset major depressive disorder: findings from a 2-year longitudinal study in a sample of first-year university students in China. J Affect Disord. 2021;295:5–10.

    Article  PubMed  Google Scholar 

  30. Núñez E, Steyerberg EW, Núñez J. Regression modeling strategies. Rev Esp Cardiol. 2011;64:501–7.

    Article  PubMed  Google Scholar 

  31. Buuren S, Groothuis-Oudshoorn K. Mice: multivariate imputation by chained equations in R. J Stat Softw. 2011;45:1–67.

    Article  Google Scholar 

  32. First MB. Diagnostic and Statistical Manual of Mental Disorders, 5th Edition, and, Utility C. J Nerv Ment Dis. 2013;201:727.

  33. Chen Y, Xu J, Yan S, Xian Y, Li Y, Chang X, Liang G, Ma Z. A comparative study of mental health in Adulthood-Trying out the Self-Compiled mental health questionnaire. Chin Ment Health J. 1998;10–2.

  34. Chen S, Guan Y, Zhang W, Li J. The test and revision of Hewitt multidimensional perfectionism scale for 529 Chinese college students. Chin J Health Psychol. 2008;823–5.

  35. Zhang F, Zhu S, Deng P. Evaluation of perceived social support scale used in study of social support among hospitalized patients in China. Chin Nurs Res. 2018;32:2048–52.

    Google Scholar 

  36. Yue D, Li M, Jin K, Ding B. Preliminary revision of EMBU and its application in neurotic patients. Chin Ment Health J. 1993;7:97–101.

    Google Scholar 

  37. Bijlsma S, Bobeldijk I, Verheij ER, Ramaker R, Kochhar S, Macdonald IA, van Ommen B, Smilde AK. Large-scale human metabolomics studies: a strategy for data (pre-) processing and validation. Anal Chem. 2006;78(2):567–74.

    Article  CAS  PubMed  Google Scholar 

  38. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33:1–22.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Bozdogan H. Model selection and Akaike’s information criterion (AIC): the general theory and its analytical extensions. Psychometrika. 1987;52:345–70.

    Article  Google Scholar 

  40. Craney TA, Surles JG. Model-dependent variance inflation factor cutoff values. Qual Eng. 2002;14(3):391–403.

    Article  Google Scholar 

  41. Emerson RW. Causation and Pearson’s correlation coefficient. J Visual Impair Blin. 2015;109(3):242–4.

    Article  Google Scholar 

  42. R Core Team. R: A Language and Environment for Statistical Computing (4.3.0). R Foundation for Statistical Computing. 2023. Available from: https://www.R-project.org/

  43. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Mak. 2006;26(6):565–74.

    Article  Google Scholar 

  44. Friedman JD, Reece GR, Eldor L. The utility of the posterior thigh flap for complex pelvic and perineal reconstruction. Plast Reconstr Surg. 2010;126:146.

    Article  CAS  PubMed  Google Scholar 

  45. Singh JP, Desmarais SL, Van Dorn RA. Measurement of predictive validity in violence risk assessment studies: A Second-Order systematic review. Behav Sci Law. 2013;31:55–73.

    Article  PubMed  Google Scholar 

  46. Mitsui N, Asakura S, Takanobu K, Watanabe S, Toyoshima K, Kako Y, Ito YM, Kusumi I. Prediction of major depressive episodes and suicide-related ideation over a 3-year interval among Japanese undergraduates. PLoS ONE. 2018;13:e0201047.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Shim G, Jeong B. Predicting suicidal ideation in college students with mental health screening questionnaires. Psychiatry Investig. 2018;15:1037–45.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Weller O, Sagers L, Hanson C, Barnes M, Snell Q, Tass ES. Predicting suicidal thoughts and behavior among adolescents using the risk and protective factor framework: A large-scale machine learning approach. PLoS ONE. 2021;16:e0258535.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Campisi SC, Carducci B, Akseer N, Zasowski C, Szatmari P, Bhutta ZA. Suicidal behaviours among adolescents from 90 countries: a pooled analysis of the global school-based student health survey. BMC Public Health. 2020;20:1102.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Uddin R, Burton NW, Maple M, Khan SR, Khan A. Suicidal ideation, suicide planning, and suicide attempts among adolescents in 59 low-income and middle-income countries: a population-based study. Lancet Child Adolesc Health. 2019;3:223–33.

    Article  PubMed  Google Scholar 

  51. Leichsenring F, Heim N, Leweke F, Spitzer C, Steinert C, Kernberg OF. Borderline personality disorder: A review. JAMA. 2023;329:670–9.

    Article  PubMed  Google Scholar 

  52. Reichl C, Kaess M. Self-harm in the context of borderline personality disorder. Curr Opin Psychol. 2021;37:139–44.

    Article  PubMed  Google Scholar 

  53. Levi-Belz Y, Gvion Y, Levi U, Apter A. Beyond the mental pain: A case-control study on the contribution of schizoid personality disorder symptoms to medically serious suicide attempts. Compr Psychiatry. 2019;90:102–9.

    Article  CAS  PubMed  Google Scholar 

  54. Borges G, Benjet C, Orozco R, Medina-Mora ME, Mendez E, Molnar BE. Traumatic life-events and suicidality among Mexican adolescents as they grow up: A longitudinal community survey. J Psychiatr Res. 2021;142:171–8.

    Article  PubMed  Google Scholar 

  55. Robinson A, Moscardini E, Tucker R, Calamia M. Perfectionistic Self-Presentation, socially prescribed perfectionism, Self-Oriented perfectionism, interpersonal hopelessness, and suicidal ideation in U.S. Adults: reexamining the social Disconnection model. Arch Suicide Res. 2022;26:1447–61.

    Article  PubMed  Google Scholar 

  56. Rohani F, Esmaeili M. Psychological factors of vulnerability to suicide ideation: attachment styles, coping strategies, and dysfunctional attitudes. J Educ Health Promot. 2020;9:50.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Park YS, Park EC. Association between parental marital status and types of suicidal behavior among Korean adolescents: A Cross-sectional study. J Prev Med Public Health. 2020;53:419–28.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Alvarez-Subiela X, Castellano-Tejedor C, Villar-Cabeza F, Vila-Grifoll M, Palao-Vidal D. Family factors related to suicidal behavior in adolescents. Int J Environ Res Public Health. 2022;19:9892.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Hou X, Wang J, Guo J, Zhang X, Liu J, Qi L, Zhou L. Methods and efficacy of social support interventions in preventing suicide: a systematic review and meta-analysis. Evid Based Ment Health. 2022;25:29–35.

    Article  PubMed  Google Scholar 

  60. Frey LM, Hunt QA, Russon JM, Diamond G. Review of family-based treatments from 2010 to 2019 for suicidal ideation and behavior. J Marital Fam Ther. 2022;48:154–77.

    Article  PubMed  Google Scholar 

  61. Chen SY, Cheng Y, Zhao WW, Zhang YH. Effects of dialectical behaviour therapy on reducing self-harming behaviours and negative emotions in patients with borderline personality disorder: A meta-analysis. J Psychiatr Ment Health Nurs. 2021;28:1128–39.

    Article  PubMed  Google Scholar 

  62. Spong AJ, Clare ICH, Galante J, Crawford MJ, Jones PB. Brief psychological interventions for borderline personality disorder. A systematic review and meta-analysis of randomised controlled trials. Clin Psychol Rev. 2021;83:101937.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the entire staff who participated in the study.

Funding

This study was supported by the Shandong Taishan Scholars Project (tsqn201909145), the National Natural Science Foundation of China (81901391) and the Natural Science Foundation of Shandong Province (ZR2019MH095), the High-level Scientific Research Talents Cultivation Programme of Jining Medical College (JYGC2022KJ008), as well as the Key Scientific Research Planning Project of Jining City (2023YXNS102, 2023YXNS213) were funded. None of the funders were involved in the design and implementation of this study.

Author information

Authors and Affiliations

Authors

Contributions

Yan Liu and Jianli Wang designed the study. Sifang Niu, and Xingmeng Niu participated in data mining. Yan Qin was responsible for data analysis and completion of the first draft. Jianli Wang, Ying Zhang and Min Liu did the English revision. Others were involved in project management and revision. All authors approved the final version of the manuscript.

Corresponding author

Correspondence to Yan Liu.

Ethics declarations

Ethics approval and consent to participate

This study compiled with the Declaration of Helsinki. This study was approved by the Research Ethics Committee in Jining Medical University, Jining, China (No:2019-JS-004). All participants voluntarily submitted written informed consent before participating in the study. All data were handled in a confidential manner.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

CRediT authorship contribution statement

Yan Liu and Jianli Wang designed the study. Sifang Niu, and Xingmeng Niu participated in data mining. Yan Qin was responsible for data analysis and completion of the first draft. Jianli Wang, Ying Zhang and Min Liu did the English revision. Others were involved in project management and revision. All authors approved the final version of the manuscript.

Clinical trial number

Not applicable.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qin, Y., Niu, S., Niu, X. et al. Development and validation of a predictive model for suicidal thoughts and behaviors among freshmen. BMC Psychiatry 25, 409 (2025). https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12888-025-06827-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doiorg.publicaciones.saludcastillayleon.es/10.1186/s12888-025-06827-y

Keywords