Academic Performance and Behavioral Patterns

Identifying the factors which influence academic performance is an essential part of educational research. Previous studies have documented the importance of personality traits, class attendance, and social network structure. However, because most of these analyses were based on a single behavioral aspect and/or small sample sizes, we lack a quantification of the interplay of these factors. Here, we infer the academic performance among a cohort of almost 1,000 freshmen based on data collected through smartphones, which the students used as their primary phone for two years. The availability of multi-channel data from a single population allows us to directly compare the predictive power of individual and social characteristics. We find that a student's performance is best inferred from their social ties. Network indicators out-perform models based on individual characteristics. We confirm earlier findings indicating that class attendance is the most important predictor among the individual characteristics. Finally, our results indicates potential presence of strong homophily and/or peer effects among university students.


Introduction
Since research on academic achievement began to emerge as a field in the 1960s, it has guided educational policy makers on admissions and dropout prevention [1]. Although much of the literature has focused on higher education, the knowledge obtained on behavioral phenomena observed in colleges and universities can potentially guide research on student behavior in primary and secondary schools. A number of behavioral patterns have been linked to academic performance, such as time allocation [2], active social ties [3], sleep duration and sleep quality [4], or participation in sport activity [5]. Most of the existing studies, however, suffer from the typical biases and limitations associated with surveys and self-reports [6,7]. Here we investigate student performance within a novel dataset collected as part of the Copenhagen Network Study (CNS), which involved approximately 1 000 university students, with data collection ongoing for more than 2 years [8]. Due to the scale of the CNS, and the inclusion of observational data in place of self-reports we are able to mitigate some of the limitations encountered in existing 'traditional' studies. Using smartphones, we collected data from multiple channels: person-to-person proximity (using Bluetooth scans), calls and text messages, activity on online social networks (Facebook), and mobility traces. Cell phone based studies Through a variety of methods, a large number of studies have investigated the factors determining academic performance. Vandamme et al. [9] investigated three sets of characteristics: personal history, behavior, and perception. Similarly, the StudentLife study [10] used smartphones to collect data on student activity, social behavior, personality, and mental health. Both research groups observed correlations between performance and all feature categories, indicating that factors influencing academic performance are not limited to a single aspect of an individual's life. Nghe et al. [11] reframed the problem as a prediction task: using data to predict performance in a population of undergraduate and postgraduate students at two different institutions. Using a wide range of features, they predicted GPA after third year with high accuracy. One of the features included GPA after the second year, while in this work we show that it is possible to make accurate predictions without the knowledge of past performance.
Online Social Media Only a few prior studies have investigated the impact of social media activity on academic performance, despite the growing availability of such data and undisputed presence of these media in our daily lives. The majority of existing studies found a decrease in academic performance with increasing time spent on social media [12][13][14][15][16][17][18][19]. However, not all studies confirm this result. In some studies, time spent on social media was found to be unrelated to academic performance [20,21] or had even a positive effect on performance [22,23].
Personality A large body of research at the intersection of psychology and education investigates the relationship between personality and performance, as pioneered by [45]. Finally, prior research has emphasized the positive influence of attending classes as shown in [46][47][48][49]. In particular, the study by Crede et al. [49] concludes that attendance is the most accurate known predictor of academic performance. See [50] for a more detailed analysis of the impact of class attendance on academic performance based on the CNS experiment.

Results
Our research strategy is as follows. We identified a number of individual and social indicators (features) of academic performance in the collected data. We then used these features to train machine learning models designed to predict performance and help us understand the importance of particular features. Although all feature categories are correlated with academic performance, we find that features describing the social networks of students have the highest predictive power. Based on these results, we conduct an in-depth analysis of the most correlated impact factors of each category. In particular, we elaborate on the social behavioral features which have only been considered to a limited extent in previous studies.

Performance Predictions
We report the results in two stages. First, we divide the student population in quintiles based on performance and report the correlations between individual characteristics and quintile assignment. Second, we divide the population in tertiles: low, moderate, and high performers. This division into tertiles allows us to compare our results to previous work on performance prediction while quintiles provide a more detailed insight into the behavioral differences of the individual groups. To account for the different predictive power of the individual and network aspects, we construct four feature sets, each representing a certain aspect of life and corresponding to a specific level of information: personality (personality traits only), individual (all individual features), network (network metrics) and combined (full feature set). See Table 2 for a complete list of features in each category. Figure 1 shows the feature importance for features achieving significance of p < .001. Note that in general, network properties dominate the predictions with more than half of the significant features corresponding to this category. The fraction of low performing friends as well as the mean GPA of peers contacted over text messages and calls are the most important features. Class attendance proves to be the most important individual feature. Centrality in the communication networks (call and text messages) are also found to be significant descriptors with moderate importance. Among personality traits, only self-esteem and conscientiousness have significant predictive power.
For the evaluation we investigate the performance of four different classifier, namely Logistic Regression, Linear Discriminant Analysis (LDA), Support Vector Classifier, and Random Forest Classifier. The achieved predictive accuracies of every model are visualized in Fig. 2. Each data point is the result of a 10-fold cross-validation and is based on 30 independent measurements (see Materials and Methods for more details on the experimental setup).
In the following, we report the in-class precision and recall values only for the classification problem with the LDA model and the combined feature set, which achieved the best results. Fig. 3 shows these results along with the corresponding F 1 values. As the results indicate, once the GPA class is provided, the model has high predictive power among the low and high performers (compared to that of the moderate performers) with F 1 values of .649 and .646, respectively. A corresponding accuracy of 58.7% is achieved (significantly higher than that of the baseline of 33.33%).
Identifying students in risk of failing or dropping out at an early stage, is of particular interest from a practical standpoint. In a practical scenario, these are the students that we might want to offer extra help. Modeling the question of identifying these students as a binary classification allows us to identify 77.4% of the at-risk students (bottom third) with a precision of 58.3%. This corresponds to a F 1 score of .666. On the other hand, non-risk students were correctly classified as such with a precision of 86.5% (F 1 = .788). These results underline the predictive power of the observed variables, in particular in the application of ascertaining at-risk students.

Individual behavior
Among individual effects we considered, class attendance is found to be the strongest predictor for academic performance. This observation is robust regardless of the measure used to quantify academic performance: correlation coefficients of r S = .255 for course grades, r S = .255 for term GPAs, and r S = .274 for cumulative GPAs were determined (all values are significant with p < .001). An in-depth analysis of the observed class attendance patterns along with a detailed description of measuring attendance applied to the CNS dataset is discussed in [50].
Assuming that any posting activity on Facebook is positively correlated with the total time spent on the social network site, we quantify the time devoted to the social networking site by the number of posts a student performed during the semester. Figure 4 illustrates the results after dividing the population into quintiles according to their average number of posts per week. As Fig. 4a shows, the distribution of posts among students is heavy-tailed and is described by the vast majority of the students having less than 3 posts in a typical week. Distribution of term GPA values in the different quintiles reveals that, on average, students with lower activity perform better (see Fig. 4b). The last two groups (individuals with the highest number of average posts per week) show high similarity, whereas the first quintile differs from the other to the largest extent.

Social interactions
We find that a student's performance can be accurately predicted from the achievements of their peers. We have quantified this effect using three measures: the mean GPA of friends and the fraction of low performers (bottom tertile) and high performers (top tertile) among the friends. Results from this analysis are displayed in Fig. 5, where we divide the population into deciles based on term GPA and calculated mean GPA of their contacts in various communication and interaction channels. First, regardless of the channel considered, the curves show a strong increasing trend. The general relationship between peers from each channel and performance is quantified in Table 1. Each channel displays higher correlations than any of the personality traits. The most pronounced effect is obtained with calls and text messages, which are considered a proxy for the strongest social ties because they require more effort to initiate and maintain [51].
We further assess the validity of pairwise similarity in the network by focusing exclusively on social ties based on text messages. Figure 6 shows a scatter plot of the correlation between the own GPA and mean GPA of the texting peers for every student in the dataset. Once again, we observe a clear linear trend; the trend is especially strong in the region where the majority of the students is located (GPAs in the range between 2 and 3). In Fig. 7 we divide the population in quintiles based on term GPA and calculate the fraction of text messages exchanged with members of the different groups. Beyond the correlation, we can see that students' communication in each group is dominated by members of the same or neighboring groups.
The structure of the interaction networks provide further insight into how students position in their social environment is correlated with performance. To investigate the network effects, we consider two descriptors here: number of contacts (degree) and eigenvector centrality. Eigenvector centrality is related to flows in the social network and individuals of high centrality are regarded as influencers in case of a spreading dynamics (outbreak, rumors, etc.) [52]. Here we consider networks based on various channels (person-to-person, calls, text messages, and Facebook friendship or direct interaction). The correlations between cumulative GPA and the network descriptors are summarized in Fig. 8. We find weak to moderate positive correlations in most of the networks, in agreement with the existing literature [35][36][37][38][39][40], however, proximity interactions display stronger values relative to rest of the interactions. It should be noted that the proximity networks include all connections, even those formed on campus, and during lectures, which may account for some of the observed correlations.

Discussion
For the participants of the CNS study, we found that the most efficient predictor for academic performance for an individual is the academic performance of her contacts. We observed this effect across different channels of social interactions with calls and text messages showing the strongest correlations, further emphasizing the phenomena. As mentioned in the literature review, this effect could be caused by multiple factors: peer effects, homophily or the impact from taking the same classes.
These finding contrast with previous conclusions from a meta-study [49], which found class attendance to be the strongest predictor of academic achievements. In our study, however, when network effects (such as social ties) are disregarded and only individual features are considered, class attendance is indeed the important feature. We note that our results on social ties are limited due to the limited number of interactions recorded (only contacts between participants of the CNS are considered). Because of this limitation, our results show the lower bound for the importance of social ties in inferring academic performance. The availability of more data about interactions would allow us to make even more precise inferences.
We found network centrality to have positive correlation with academic performance, in agreement with the literature [35][36][37][38][39][40]. However, from among all types of interaction networks, only proximity network exhibited strong effect. A possible limitation in measuring centrality is that the mere physical proximity of two individuals does not necessarily involve direct communication. Nevertheless, it is reasonable to expect an increased level of information exchange in a group of individuals if they are in close proximity, which was the case in our dataset. [1] Consistent with findings in existing literature, we found that class attendance showed the strongest correlation with academic performance when we consider only individual effects [46,48,49,[54][55][56][57][58]. We also found that Facebook activity had a negative relation to academic performance -also in agreement with the majority [1] The Copenhagen Networks study uses Bluetooth visibility as an indicator of person-to-person proximity.
of the studies that investigated Facebook and social media usage [12][13][14][15][16][17][18][19]. We note, however, that our the data is limited to Facebook activities such as posting a status update or uploading a picture etc, and that we have no information regarding 'passive' Facebook usage, such as scrolling and reading. It is a reasonable assumption, however, that active participation on Facebook has a positive correlation with the (passive) time spent on Facebook. Also, our data does not include direct messages which may constitute a relevant fraction of communications performed via the social network site.
The analysis of the different personality traits revealed that two aspects of personality, namely conscientiousness and self-esteem are useful for predicting academic performance. These two traits reached a correlation coefficient between 0.2 and 0.3 corresponding to the upper limit achievable for any correlation with a personality trait, according to Mischel [59]. These results also agree with existing literature .
In the prediction experiment we achieved a classification accuracy of around 25.4 percentage points above baseline, a result similar to that of Vandamme et al. [9], although they utilized nearly ten times as many features to build a model as we did. In addition the accuracy of Vandamme et al. [9] is driven by using prior achievement (grades), which is known to be a strong predictor of performance (e.g. due to persistence of skill and motivation). Our findings-together with the results in the literature-emphasize that there is a considerable dependence of academic performance on personality and social environment. However, there are also other factors that play a role which we were unable to capture. See Materials and Methods for a review of the major limitations of our approach.

Data collection
Results presented in this paper are based on the data collected in the Copenhagen Network Study (CNS) [8]. In the CNS experiment, various data types were recorded by dedicated smartphones: Bluetooth scans, call and text message meta data, Facebook activity logs, and mobility traces among around 1 000 students at the Technical University of Denmark. The study covered the academic years 2013/14 and 2014/15. Participating students responded to a survey on personality. Finally, course grades were provided by the Technical University of Denmark. Due to the possibility to exit the experiment at any given point, the number of participants varied over time.
The raw data records are cleaned and transformed to meaningful information before the analysis. Bluetooth scans are used to estimate person-to-person interactions corresponding to a physical distance of up to 10 m between participants. After constructing proximity links, data is binned in 15 minutes time bins. Facebook data was obtained by the Facebook Graph API, and contains both static friendship connections as well as various interactions on the social network. All types of interactions are treated equally. Private messages, however, are unavailable at they cannot be obtained from Facebook using the official Graph API. Locations are estimated by selecting the ones with the highest accuracy in 15 minute bins. Only courses using the Danish 7-point grading scale are considered and all grades are converted to GPAs by linear scaling, with negative grades being converted to 0. We also calculate cumulative and term average GPAs weighted by the course credit points, and only students attending at least three courses are considered. In total, 3494 term GPAs and 957 cumulative GPAs are computed.
In order to compute attendance we used the smartphone locations as well as person-to-person proximity obtained from Bluetooth scans. A detailed description of how we compute attendance can be found in a companion paper [50].
The different channels collected (proximity, Facebook, calls, and text messages) may indicate several types of social connections. Due to the nature and high usage frequency of these channels, in case of physical proximity and Facebook interactions links are weighted by the total number of encounters or actions. On the contrary, calls and text messages are reduced to unweighted relationships, as these interactions show insignificant variations over the pairs of students. To further decrease the impact of noise due to the wide distribution of weights in the proximity and Facebook interaction networks, a non-linear scaling function is applied to remove large deviations: where w i denotes the original number of interactions between students i and j.
Qualitative results and the findings are not affected by the above scaling. Finally, all interactions are considered as undirected, that is, social ties are symmetric. In order to increase the stability of the results we apply bootstrap resampling. That is, analyses are performed on 100 bootstrap samples, where each has the same size as the original sample. Results are given by the mean of the bootstrap analyses with approximated standard errors described by the Standard Error of the Mean.

Performance predictions
During performance prediction, we define four different types of feature sets based on their origin and function: personality, individual, network and, combined (see Table 2 for the different feature sets along with the features included). The personality feature set is based on solely personality traits, while the individual set contains all individual related features that do not require knowledge on the structure of the social ties. On the other hand, the network feature set includes only features that are extracted from the social interactions. Finally, the combined set contains every feature introduced in this paper. Instances with missing attribute values were excluded. At total a sample of 540 students were considered for the prediction.
As a performance measure for prediction, we use classification accuracy. Model selection is carried out by a 10-fold cross-validation, and due to variations in the different realizations of predictions, we present the average performance of 30 independent predictions.

Limitations
Although we utilize wider and more insightful data than most other studies our approach has also some drawbacks which need to be taken into account. The first limitation comes from the fact that we only observed students from a single, technical, Danish university. For this reason, the findings may not be generalizable to students at other institutions, of other academic disciplines or with other demographics. Furthermore, only a subset of all the students participated in our studyfor first year students the rate was around 40%. There was some tendency of selection into the study as the average student tend to achieve higher grades [90]. Yet, within our dataset we observe a high degree of variation with respect to behavioral and network measures as well as academic performance. Thus, we argue that our results are valid and therefore appropriate precursors of the trends present in the larger student population.
Our measures of attendance and student social networks are also subject to some noise. Our method of inferring attendance and its drawbacks is discussed in [50] where we show it is quite similar to official records where available but with some error. Also our measures of social networks is limited to phone calling/texting, meetings and public Facebook messages. Although these are probably some of the most important means of communication, some students may communicate via other smartphone apps. Furthermore, attendance does not always imply neither in class participation nor attention to the taught material.
As a final remark we note that measuring academic success is not only exam performance but also includes long term learning, skills not measured in the tests etc.        cumulative GPA mean GPA of contacts r S = .
(p < . )     . Figure 8 Correlation between network measures and cumulative GPA. Bars denote the correlation measured between network descriptors and cumulative GPA: degree (gray) and eigenvector centrality (red).