Adaptation of student behavioural routines during Covid-19: a multimodal approach

Abstract

One population group that had to significantly adapt and change their behaviour during the COVID-19 pandemic is students. While previous studies have extensively investigated the impact of the pandemic on their psychological well-being and academic performance, limited attention has been given to their activity routines. In this work, we analyze students’ behavioural changes by examining qualitative and quantitative differences in their daily routines between two distinct periods (2018 and 2020). Using an Experience Sampling Method (ESM) that captures multimodal self-reported data on students’ activity, locations and sociality, we apply Non-Negative Matrix Factorization (NMF) to extract meaningful behavioural components, and quantify the variations in behaviour between students in 2018 and 2020. Surprisingly, despite the presence of COVID-19 restrictions, we find minimal changes in the activities performed by students, and the diversity of activities also remains largely unaffected. Leveraging the richness of the data at our disposal, we discover that activities adaptation to the pandemic primarily occurred in the location and sociality dimensions.

1 Introduction

The COVID-19 pandemic has had a wide-ranging impact on various components of our daily lives. The implementation of government restrictions such as travel bans, business and school closures, stay-at-home orders and physical distancing mandates aimed at preventing the spread of the virus [11, 19, 57]. These measures resulted in substantial changes in how we live, work, and socialize with long lasting impacts, among others, on human mobility and encounters [19, 48, 68], employment [1, 6], education [7, 65, 69], mental health and well-being [13, 23, 53].

Traditionally, social scientists have relied on qualitative techniques such as direct observation and fieldwork to study various aspects of our daily lives and gain insights into these social phenomena [14, 37]. However, these methods have limitations as they can be time-consuming, resource-intensive, and challenging to replicate, making them less suitable for large-scale studies.

In the last decades, the emergence of the digital sensing revolution has provided researchers with vast amounts of quantitative data generated by, among others, social media platforms, credit cards, and mobile phones. Digital data sources collected with these technologies offer unprecedented opportunities to gain insights into individual and group behaviours with a breadth and depth that was previously inconceivable [45, 46]. This wealth of digital data has become particularly valuable during the COVID-19 pandemic, enabling researchers to uncover patterns and trends in human behaviour and to understand the changes that have occurred in our daily lives and society as a whole [10, 41, 58].

While these data sources allow us to track the behaviour of thousands or even millions of individuals, it’s important to note that the temporal resolution of the collected data is not continuous but rather tied to specific events (e.g., phone calls or traffic data for mobile phone data, app usage for GPS location data, credit card usage for spending data, etc.). Furthermore, these data sources often represent only one aspect of human behaviour at a time such as communication, spending, and mobility.

To overcome these limitations researchers have implemented “living labs” studies, which collect multiple sources of data [2, 18, 27, 34, 49, 51, 62]. In these studies, participants are equipped with mobile phones and/or sensing apps installed on their devices to collect data from diverse sources and sensors, including phone and SMS logs, Bluetooth proximity interactions, GPS location data, accelerometer data, etc. Additionally, surveys and experience sampling methodologies are employed to gather information on individual characteristics (e.g., attitudes, personality traits, etc.), on states (e.g., mood, stress levels, etc.) as well as situational and activity data. By integrating data from multiple sources, living labs deployments offer the advantage of capturing a richer representation of the individuals under study.

In our work, we leverage an Experience Sampling Method (ESM) [22] approach on smartphones to capture self-reported information on the daily activities of university students. ESM is a methodology aiming at collecting information on the behaviours and emotions of study participants throughout their daily activities and routines [55, 60, 64]. As in traditional time diary studies, ESM collects data by means of study participants’ self-reports. However, participants, unlike in diary studies, are proactively triggered at various moments during the day. In our work, we ask the study participants to report information on their activities with a prompt every 30 minutes, and we collect not only the activity of the students but also their location and sociality (i.e., with whom they were). Hence, this approach produces a unique understanding of students’ behaviour from the point of view of these three different behavioural dimensions. Moreover, the ESM-based data collection happened in two periods, in 2018 and 2020, and we leverage these data to analyse how students’ routines changed across all three behavioural dimensions during the COVID-19 pandemic.

As initial step, we delve into the shifts in students’ behaviours by examining the raw frequencies of activities. By comparing the distributions of the 2018 and 2020 samples, we identify the most noteworthy changes. Our analysis reveals that common activities, such as moving and engaging in social activities, experienced a decrease in their frequency, while others, like watching shows, saw an increase in prominence. Concurrently, there was a general decrease in location frequencies, compensated with a significant increase in the frequency of the home location. A similar trend is observed in the dimension of social interactions, where even interactions within the same household decreased, leading to an increase in alone time.

To place these results within the context of daily time allocation, we further investigate how students’ routines were impacted. To this end, we use Non-Negative Matrix Factorization (NMF) [20, 61] to extract meaningful behavioural components (i.e., routines) from self-reported data. Indeed, NMF unveils underlying behavioral patterns not readily apparent in the data while also reducing irrelevant or noisy information. This results in the extraction of more interpretable behavioral components when compared to alternative methods based on Principal Component Analysis (PCA). These alternative techniques often face issues related to interpretability due to their lack of a non-negativity constraint. As a result, they may extract less meaningful routines (see Sect. S3 in the Additional file 1 for the routines obtained using Principal Component Analysis and Robust Principal Component Analysis).

Hence, by analyzing the extracted components and their associated weights, we are able to quantify the differences in behaviour between students in 2018 and students in 2020. Our findings indicate that despite the impact of the pandemic, the activity routines of students in 2018 and 2020 remained largely similar with minimal differences observed. However, with the multimodal information collected through the time diaries, we discover that the behavioural dimensions of location and sociality exhibit stronger signals of adaptation. To delve deeper into this shift of behaviour, we first examine the relationship between activities and locations, then the relationship between activities and sociality. This approach allows us to identify the activities that were associated with diverse locations and sociality routines before and during the pandemic. Notably, activities such as studying and attending lessons, which had a social component before the pandemic, transitioned to solitary activities, done in private homes, even if students could go to the university. We find that most of the extracted routines in 2020 follow this pattern, and take place at home and alone, while in 2018 students visited more diverse locations and spent more time with friends. For example, social life is more present in 2018, with students spending time with friends, while in 2020 this activity is done with roommates, thus people residing in the same household.

2 Related work

2.1 Behaviours and routines modelling

In recent years, the field of modelling human behaviour and routines has witnessed significant growth, thanks to advancements in mobile sensing [44] and experience sampling approaches [60].

Various techniques have been employed to understand individual behaviour. Principal Component Analysis (PCA) was used to extract students’ routines (eigenbehaviours) [28], from the Reality Mining dataset [27]. Techniques typically used for Natural Language Processing tasks [71] have been used to extract regular behavioural patterns from sequential data, such as Latent Dirichlet Allocation [8], which was also used to group individual activities into routines [29, 42]. Other sequence analysis techniques have been used as well [12, 63], from modelling purchasing behaviour [26] to the analysis of individual mobility [48].

Additional methods include more complex tools, such as Hidden Markov Models [59], Conditional Random Fields [30] and Bayesian Networks [67]. These approaches are commonly used to build predictive and generative models, and can be used to detect the activities from sensor data and images [30, 67] or to model cyclic behaviour and its anomalies [59]. While these methods have their advantages, in our study, we aim to identify easily interpretable and human-readable patterns to compare and measure behavioural differences.

In recent years, Non-Negative Matrix Factorization (NMF) [61] has gained popularity, thanks to its interpretability. NMF has proven to be effective in identifying repeating patterns in the daily behaviours of people, both at an individual and a collective level. NMF has been successfully applied in various domains, such as in analyzing social interactions in networks [32], understanding of chronotypes [3], and identifying routines in urban environments [26, 70]. Inspired by these studies, we use NMF in our methodology, as it allows us to associate the resulting patterns with realistic individual behaviour, and the weights associated with these patterns provide a quantitative measure of behavioural differences. Additionally, following [73], we include in our methodology, the use of multiple modalities to analyse subjects’ behaviour, which includes (i) activity, (ii) location, and (iii) sociality dimensions.

2.2 Behavioural change and the pandemic

There has been an upsurge of scientific research in response to the COVID-19 crisis aimed at understanding and mitigating the spread of the virus [4, 19]. The various consequences and effects of COVID-19 restrictions on the population, spanning economic, social, and psychological domains have been tackled in diverse disciplines.

From an economic perspective, studies have demonstrated that the effects of restrictions differ across individuals with varying income levels, often increasing segregation. For instance, in Italy, the mobility reduction was higher in high fiscal capacity municipalities, which also display high income inequality [9]. Similar findings have emerged in other countries such as France and the United Kingdom, where the mobility network became more fragmented on a national scale but maintained more connections on a small scale [31]. In the United States, researchers have observed a decrease in encounters between citizens of different income levels in urban areas, hindering economic recovery and city growth [68], despite mobility returning to pre-pandemic levels.

Another area of interest has been the well-being of the population, with a particular focus on students, enhancing research that was present even before the pandemic [66]. The impact of the pandemic on students’ educational paths has been substantial [7], as shown in studies on academic performance and mental well-being [38]. Research has indicated a significant increase in stress and anxiety levels among younger students during lockdowns [39], particularly college and university students [52, 69].

Previous studies on students primarily rely on surveys and self-reported data, often neglecting the impact on behavioural routines. Furthermore, only a few studies consider multiple dimensions of behaviour and well-being with high-resolution data. For instance, the study conducted in [51] combines surveys with mobile sensing data to explore how students’ perspectives and concerns regarding the pandemic lead to diverse responses to restrictions and shifts in behaviour.

In contrast, our work demonstrates the utility of Experience Sampling Methods on smartphones, which provide a wealth of information on activities, locations, and social interactions in a real-world setting. These methodologies offer a level of granularity, particularly in activity definition, that cannot be achieved through passive sensor data collected by mobile phones. Moreover, we highlight how these methodologies can be harnessed to understand multiple dimensions of students’ behaviour, both independently and in conjunction with one another.

3 Data

The data utilized in this work are derived from two separate datasets collected during two distinct living lab studies: the SmartUniTn study [33, 35, 36] conducted in 2018, and the subsequent WeNet study [5, 34, 50] conducted in 2020.

The SmartUniTn study gathered both sensor data and questionnaire responses from students at the University of Trento, employing the iLog [72] smartphone application. Over a period of two weeks, with a frequency of 30 minutes, participants were requested to complete a Time Diary via the Experience Sampling Method (ESM) application. Each entry in the diary required students to report their activity (e.g., studying, eating, etc.), location (e.g., home, university, bar, etc.), and sociality, indicating whether they were alone or in the company of others (e.g., alone, friends, relatives, etc.). The response options were predefined to ensure uniformity and minimize user biases in activity descriptions (refer to [33] for the available sets of responses). This comprehensive dataset, which was collected from May 10th to May 23rd, 2018, provides a multimodal representation of student behaviour in a period when students attended classes and prepared for the summer exam session.

The subsequent WeNet study [34] expanded upon the SmartUniTn study by extending the data collection to include seven universities worldwide, including the University of Trento. To ensure consistency with the previous study, we only focus on the data from the latter university. The data collection process closely followed that of the SmartUniTn study, with refinements made to the sets of activities and locations (for detailed information, see [34]). Notably, the data collection occurred during the COVID-19 pandemic, and as such, specific government restrictions were in place at the time. The Time Diaries were collected between November 14th and November 30th, 2020, a period during which the government of the province of Trento implemented several restrictions [21]. Certain activities, such as gyms, were closed, while others, like shops, had limited access. Distancing measures were enforced in various contexts, including bars and restaurants, where in addition to physical distancing, time restrictions were imposed on the duration people could spend at these locations. A curfew was in place from 10 PM. Moreover, while schools were generally closed, first-year students participating in our study could attend lessons in person.

After completing the data collection and pre-processing phases, our dataset comprises samples of 128 students in the 2018 study and 119 students in the 2020 study, resulting in an almost balanced dataset with a total of 247 students.

4 Methods

4.1 Data processing

Since the data collection of the two studies was performed in different iterations, the student samples are different. However, the study settings remained consistent, and the population under examination was similar as both studies focused on students. To facilitate a more robust comparison between the groups of students in 2018 and 2020, we performed a mapping between the categories in the Time Diaries to ensure the comparability of activities, locations, and sociality categories. Given that the data collected in 2020 were more refined, the mapping primarily involved the alignment of certain activities and locations, where multiple entries from 2020 were mapped into a single entry from 2018. For instance, activities like Free Time Study, Arts, Hobbies, and Games in 2020 were all mapped to Hobbies as in 2018. A comprehensive list of activity and location mappings can be found in the Additional file 1, Sect. S1.

Subsequently, we organized the data to follow circadian cycles starting at 5 AM, ensuring that the daily Time Diaries of each student were appropriately aligned. We excluded days and students with insufficient reported data. Additionally, as our focus was on typical behaviour during a regular student week, we excluded weekends from our analysis.

Given that Time Diaries provide insights into three different facets of behaviour (activity, location and sociality), we created three distinct matrices, each corresponding to the type of information collected. In these matrices, the rows represent students, while the columns represent their behaviour throughout the day. To achieve this, we allocated a number of columns (or features) equal to $c = |A|*48$, where A represents the set of possible choices for each data type (e.g., the possible activities for the activity matrix), and 48 represents the number of time intervals within a day (given that the data was collected every 30 minutes). Consequently, the matrices have dimensions of $N \times c$, with N denoting the 247 students from both datasets. Each matrix contains the normalized counts of observed behaviour (activity, location and sociality) within specific time slots for each student.

4.2 Methods for analyzing differences in students’ behaviours

As a first step in understanding students’ behavioural differences between 2018 and 2020, we examined the distributions of frequencies of activities, locations, and social interactions, to identify the behavioural facets that exhibited the most significant changes. To assess the significance of these differences, we employed three statistical tests, namely t-test, Analysis of Variance (ANOVA) and Kolmogorov–Smirnov (KS) test, to evaluate the distributions’ characteristics across the 2018 and 2020 samples.

More specifically, for each behavioural dimension (i.e., activities, locations and sociality) we computed the frequencies of each of the possible choices in that particular dimension (e.g., for activities: Break, Eating, Hobbies, etc.), splitting the data into the 2018 and 2020 samples and comparing them with the aforementioned tests.

4.3 Methods for detecting students’ daily behavioural routines

The focus of our paper is to analyse and investigate the behavioural routines of students, taking into account the temporal aspect of their daily routines, while simultaneously looking at different facets of students’ behaviour (activities, location and sociality). We explored established methods for modelling and extracting behavioural routines from diary data. We tested the widely used matrix decomposition techniques, Principal Component Analysis (PCA) [43] and its extension, Robust PCA (RPCA) [16], which is more robust to the presence of outliers. These methods decompose data matrices into repeating patterns, with assigned weights for reconstructing initial observations. However, the presence of negative values in the extracted components and weights complicates their interpretability. Furthermore, routines produced by these methods often proved challenging to interpret, resulting in noisy patterns and redundant information. We thoroughly tested these methods on all the behavioural dimensions (activities, location and sociality) and we report the extracted components in the Additional file 1 Sect. S3.

As an example, both PCA and RPCA, in Component 2 of the location dimension (Figs. S2 and S5 in the Additional file 1 Sect. S3.1 and S3.2 respectively), display the redundant information that students are not at Home when they are at the University. Moreover, the larger the number of items in the original data, the more noisy and less interpretable the extracted components are (e.g., activity components are harder to interpret than location components, see Figs. S1–S2 in the Additional file 1 Sect. S3.1).

Therefore, to extract meaningful daily routines from our data, we applied Non-Negative Matrix Factorization (NMF) [20, 61], which follows the same principle of the aforementioned methods. However, NMF operates on non-negative matrices and decomposes them into two non-negative matrices, making the results easier to interpret, as previous applications in different fields, such as image analysis [47], text mining [54] and also in behavioural and routine analysis [3, 68], have shown. Moreover, NMF has been proven successful in analysing temporal data [3, 17, 40]. In our scenario, the use of this method allows for the exploitation of its clustering properties, for which the algorithm extracts coherent routines from similar users with similar behaviours.

4.3.1 Non-negative matrix factorization

The Non-Negative Matrix Factorization (NMF) algorithm,¹ identifies recurring patterns within the input data and approximates the original data by representing it as a linear combination of these patterns. In more detail, NMF decomposes each of our behavioural matrices, denoted as X, into an approximation $X \approx WK$. Here, K represents the $r \times c$ matrix of extracted components, where c denotes the number of features defining the behaviour (as defined in Sect. 4.1), and r represents the number of extracted components or rank. Consequently, the matrix K contains the weights of each feature for each component, while the matrix W has dimensions $N \times r$ and contains the weights of the components for each student. The algorithm leverages its inherent clustering property to identify common observations within X, storing the discovered repeating patterns in K.

For example, if we consider the location dimension of students’ behaviour, the $N \times c$ location matrix X, contains in each row i the normalized counts of each location at each time of the day for a single student (e.g., Home at 8.30 AM, Home at 9.00 AM, etc.). The matrix of extracted components K captures common location routines among students, while the matrix W contains the weights of the extracted components for each student. The original behaviour of a student i can thus be reconstructed as: $x_{i} = w_{i1} k_{1} + w_{i2} k_{2} + \cdots + w_{ir} k_{r}$.

The approximation $X \approx WK$ is achieved by minimizing the reconstruction error, computed using the Frobenius distance between the original matrix and its approximation:

$$ \mathrm{Err} = \Vert X - WK \Vert _{F} = \sqrt{\sum _{i,j} \biggl( x_{i,j} - \sum _{r} w_{i,r} k_{r,j} \biggr)^{2}}, $$

(1)

where $x_{i,j}$ are items of the original data matrix X, $k_{r,j}$ are the values of the components and $w_{i,r}$ the weights of the components for every observation.

The NMF algorithm takes the rank r as a parameter, indicating the number of components to extract. To determine the optimal number of components, we used the cophenetic correlation coefficient [15], a common practice in the literature [3, 68]. The coefficient measures the preservation of the distance between different observations during the NMF transformation, indicating the reliability of the components. Typically, the rank is chosen where the cophenetic coefficient is maximized and begins to decline. In our analysis, we selected the number of components by considering the cophenetic coefficient, and when the choice was not evident, we prioritized the rank r that yielded the most interpretable components.

5 Results

5.1 Investigating students’ behaviour in 2018 and 2020

The initial step in understanding the shifts in students’ behaviours between 2018 and 2020, was to investigate possible differences in the frequencies of activities, locations, and sociality behavioural dimensions.

As described in Sect. 4.2, we first computed these frequencies for each student and then compared the distributions of students in the 2018 and 2020 samples. As previously anticipated, we employed three different statistical tests namely t-test, Analysis of Variance (ANOVA) and Kolmogorov–Smirnov (KS) test to evaluate the differences in the distributions’ characteristics across the samples. Given that several distributions were skewed, we report here the significant differences measured with the KS test, while the similar results that we found using the other tests are reported in the Additional file 1 Sect. S2. The statistics and the test outcome for the three behavioural dimensions are presented in Table 1, Table 2, and Table 3 respectively.

Table 1 Activity Frequencies. Mean frequency and variance of the students’ activities during 2018 and 2020. The KS test measures whether the difference between the distributions is significant

Table 2 Location Frequencies. Mean frequency and variance of the students’ locations during 2018 and 2020. The KS test measures whether the difference between the distributions is significant

Table 3 Sociality Frequencies. Mean frequency and variance of the students’ company during 2018 and 2020. The KS test measures whether the difference between the distributions is significant

Looking at the activity behavioural dimension, displayed in Table 1, we see several activity items exhibit variations across the 2018 and 2020 samples. Activities that show noteworthy differences include Sleeping, which slightly increased in 2020, Moving and Social Life, which decreased from 2018, and Watching TV/Shows, which increased in 2020. These changes are not surprising, considering the restrictions in place during the COVID-19 pandemic. It is interesting to note that some main activities, such as Eating, Hobbies, Lesson and Study, do not show significant changes between the two years.

Table 2 illustrates differences in students’ locations. As expected, given the limitations on places, there are significant differences for all location items. The frequencies of all locations decreased, with the Home location more than doubling its frequency during the pandemic.

Similarly, social interactions underwent significant changes. As shown in Table 3, only interactions with Colleagues and Others do not exhibit a significant change, most likely because of their low frequency. It is noteworthy that interactions with Classmates, Friends, Relatives, and even Roommates and Partners all decreased from 2018 to 2020. Instead, as a result of restrictions and location changes, students spent significantly more time Alone.

It is important to note that for some of the items with significant change from 2018 to 2020 (e.g. Reading, Gym and Other Library), the fraction of students that engaged in those behaviours is not relevant. Another observation is that these significant differences pertain to activity frequency and do not provide a comprehensive understanding of students’ time allocation throughout the day. For example, the activities that increased in frequency might account for the decrease in other activities. Therefore, it is essential to conduct an analysis that considers students’ actual daily routines, which will be presented in the subsequent sections.

5.2 Students’ routines changes

Given the differences found in the previous section, it is still unclear how the actual daily routines of students changed during the pandemic. In fact, routines contextualise the raw frequency of behaviours, accounting for the temporal dimension of students’ day.

To do so, we applied NMF, as described in Sect. 4, on the matrices with students from both the 2018 and 2020 data collection campaigns. Including both samples allowed us to analyze the behaviour of the two groups of students and to extract the most dominant students’ routines in both 2018 and 2020. By using the three separate matrices that represented activity, location, and sociality behaviours, we identified 6 activity routines, 4 location routines, and 7 sociality routines.

In Fig. 1 and Fig. 2, we observe the 6 activity components (or routines) extracted by the algorithm. Note that the order of the components is not related to their significance, unlike other techniques such as Principal Component Analysis. The heatmaps (Fig. 1(a) and Fig. 2(a)) illustrate the weight of each activity at specific times of the day, with higher weights indicating a higher occurrence of the activity. As it is possible to observe, during the day, certain routine patterns are more prominent than others, with essential activities like Eating and Sleeping being influenced by other activities of the day. For example, this can be seen in Component 5, where students go to sleep later than other components (e.g., Component 6) because they watch YouTube and TV shows. Furthermore, NMF correctly extracts routines related to a student’s academic life, and activities such as attending lessons and studying exhibit high weights in three components (3, 4, and 6). However, other behavioural patterns are less clear. Components 1 and 2 (in Fig. 1(a)) include several activities that share time allocations, indicating more complex daily routines. Component 1 is the routine that mostly captures social life, with higher weights for commuting (Moving) and social activities during the late afternoon and evening (Social Life).

Figure 1

Activity Routines 1–3. In (a) we can see the components 1–3 extracted by the NMF algorithm for the activity dimension. The vectors representing each component in the matrix K have been visually rearranged in a matrix format, so that the rows represent the different possible activities, while columns are the 48 bins in which the day has been divided (i.e., every 30 minutes). Thus, the heatmaps shows how present an activity is in a specific time slot. In (b) we can see the distributions of the weights each student (divided by year) has for the components, showing how much that routine is important for the student’s behaviour. We see that only the first component has different distributions for 2018 and 2020. The Information Gain (IG) shown quantifies how much knowing the component can tell apart 2018 behaviours from 2020 behaviours, and thus quantifies the weights distributions difference

Figure 2

Activity Routines 4–6. Similar to Fig. 1, (a) shows the NMF extracted components 4–6 for the activity dimension, while (b) shows the students’ weights distribution associated with them. We see that in this case the distributions do not differ much between 2018 and 2020. In fact, the IG is very low for each component

A natural question that follows after looking at the most dominant activity routines extracted by the NMF algorithm, is whether the behaviour described by a component is more associated with typical behaviours of students in 2018 or 2020. We can investigate such association by looking at the weights extracted by the NMF algorithm present in matrix W (see Sect. 4.3.1).

The weights assigned to students on the extracted components provide a measure of the presence of each component in their behaviour. Higher weights indicate a stronger association with the respective NMF component. Figure 1(b) and Fig. 2(b) display the distributions of component weights for students in 2018 and 2020 respectively. A noticeable difference is observed in the first component, characterized by students Moving and engaging in evening social activities, which is more closely associated with a behaviour typical of 2018, as indicated by the higher values of the weight distribution. The distribution for students in 2020 is much more skewed towards 0, meaning that this kind of behaviour was not present, probably due to the COVID-19 restrictions imposed during 2020 (see Sect. 3). Also, Component 6 (Fig. 2) displays a (small) distinction between the 2018 and 2020 students groups, with typical behaviours in which students study during the day and watch YouTube, TV shows, or similar activities in the evening. Here, the weight distribution for students in 2020 exhibits a heavier tail and a second saddle point, with larger values compared to the weights of 2018 students, thus associating this more sedentary routine more with 2020 than 2018.

Examining the weight distributions of the other components, it becomes evident that there are just small differences between the two groups of students. This observation is quantitatively supported by the Information Gain (IG), computed for each of the components, which quantifies the amount of information that knowing the weight of a component gives in discriminating whether the described behaviour is typical of a student in 2018 or 2020 (higher IG values indicate a larger separation in the distributions). As reported in Fig. 1(b) and Fig. 2(b), we can see that the first component carries the most informative weight, while the remaining components do not exhibit notable differences between 2018 and 2020.

Interestingly, the extracted activity routines displayed minimal changes between 2018 and 2020. Despite the pandemic restrictions, students in Trento maintained their activity behaviours focused on lessons and studying, results which are consistent with our findings in Sect. 5.1.

As detailed in the Additional file 1 (the Additional file 1 Sect. S4), more pronounced differences were observed in the locations visited and in the social interactions between the two time periods. These disparities were expected due to the imposed restrictions, which limited mobility and reduced interpersonal interactions in 2020. Among the four location routines identified through the NMF decomposition, staying at Home throughout the day exhibited a significantly stronger association with students in 2020, while going to the University was less common in 2020 compared to 2018 (Components 1 and 4, see Fig. S7 in the Additional file 1 Sect. S4). These differences are supported by higher values of Information Gain.

Furthermore, the imposed restrictions also affected social gatherings, which, along with limitations on location visits, influenced students’ social behaviour. Our analysis revealed differences in three out of seven extracted sociality components. These three routines encompassed spending a day Alone, spending a day with Friends, and a working day with Classmates (Components 5, 6, 7, see Fig. S8 in the Additional file 1 Sect. S4). As expected, higher weights were observed for a day spent alone among 2020 students, while routines involving friends and classmates showed higher weights for 2018 students.

To support our findings, we conducted a quantitative evaluation of the diversity of behaviours both in 2018 and 2020. To measure this diversity, we calculated the Shannon’s entropy [24–26] of students’ behaviour using the activity, location, and sociality matrices. The distributions depicted in Fig. 3 represent the entropy computed for each student’s average daily behaviour (i.e., using the normalized counts of each matrix) across the three types of information captured in the Time Diaries. This results in a measure of how diverse, on average, the students’ routines are across the measurement period. Intuitively, lower values of entropy indicate a lower diversity in a specific behavioural dimension.

Figure 3

Entropy for the Types of Information. The three plots show the distribution of the entropy, i.e. diversity, computed on 2018 and 2020 students’ reported behaviour for location, sociality and activity respectively. We see that the variety of activities remains the same, but students visited less diverse locations and have less diverse social interactions during the COVID-19 pandemic

The results clearly indicate that in the activity dimension, the diversity of activities performed by students in 2018 and 2020 is similar (KS-test: stat =0.1463, p-value =0.1252; T-test: t $=-1.483$, p-value =0.139), suggesting that the variety of activities did not decrease significantly in 2020. However, in the location (KS-test: stat =0.6769, p-value $=5.551e^{-16}$; T-test: t =14.403, p-value $=1.734e^{-34}$), and sociality (KS-test: stat =0.4938, p-value $=3.220e^{-14}$; T-test: t =9.748, p-value $=3.498e^{-19}$) dimensions, the entropy for 2020 is significantly lower than the entropy of 2018. This indicates that the COVID-19 restrictions had a noticeable impact on the diversity of behaviours in these dimensions, with a reduction in the range of locations visited and the social interactions engaged by students in 2020. The lower entropy values suggest a more limited range of options and routines in terms of locations and social behaviours due to the pandemic restrictions.

5.3 Multimodal activity routines: where and with whom

In the previous sections, we observed that despite the COVID-19 pandemic’s government restrictions, there were minimal changes in the activities performed by students in 2020 compared to those in 2018. However, as expected, the location and sociality dimensions of their behaviour exhibited larger variations. This observation raised the question of how these two dimensions changed in conjunction with the activities, specifically where and with whom these activities were performed.

To gain insights into these variations, we applied the NMF decomposition once again, but this time we concatenated the behavioural matrices. Initially, to examine changes in locations for the activities, we concatenated the activity behaviour matrix with the location behaviour matrix. Subsequently, we explored the social dimension of activities by concatenating the activity and sociality behaviour matrices. Additionally, to enable a more focused analysis, we separated the groups of students in 2018 and 2020. By doing so, the NMF algorithm can extract the most dominant behaviours specific to each group. This approach allowed us to look into routines specific to each year and subsequently compare them, to potentially reveal shifts in location and sociality of the activities.

5.3.1 Where: activities and locations

We begin our exploration of the variation in multimodal students’ routines by examining the relationship between activities and their respective locations. Firstly, we observe that the optimal number of components of the NMF algorithm, determined using the cophenetic coefficient, differs between students in 2018 and 2020, with values of 7 and 5, respectively. This difference suggests that the extracted behaviour in 2018 exhibits greater diversity compared to 2020, which aligns with our previous findings regarding the entropy of students in the location behavioural matrix, where the diversity of locations is lower in 2020 (Sect. 5.1). In 2020, all activity routines predominantly occur either at Home or at Relatives’ Homes except Component 6, which includes days at the University and Component 3, which does not retain much information about the location. Conversely, in 2018, we observe that activity routines, in addition to the Home or at Relatives’ Homes, took place at University, Friends’ Home, and even Workplace (Components 3 and 6).

The differentiation in students’ behaviour can be observed in Fig. 4 (the complete components can be found in the Additional file 1 Sect. S5). When examining the activities, we find that the main routines extracted by NMF for both groups of students primarily revolve around activities such as Lesson and Study. In 2018 (Fig. 4(a)), these activities are closely associated with being at Uiversity (as seen in Components 3 and 6). When, instead, students are at Home (either alone or with their relatives), Study and Lesson are not as present, and their routine is more varied. Conversely, in 2020 students carry out the majority of their activities, including lessons and studying, at Home. It is interesting that in 2020, there is still a component associated with being at the University, namely Component 6. In fact, it was still possible for students to go to the library and follow lessons in-person (if they were of the first year). Moreover, we see that when students go to the university, their routine gets more complicated and they do more activities.

Figure 4

Location and Activity Joint Components. This figure shows the joint location and activity (separated by the white line) components that most differentiate students behaviour before and during the pandemic. (a) includes routines from 2018, while (b) includes routines from 2020. We can see the shift from conducting activities such as studying and having lesson from the university to private homes

There were also other minor differences, such as having Social Life being a unique activity in 2018, while Housework is more distinctive among students in 2020, further highlighting the impact of the pandemic (see Figs. S9–S10 in the Additional file 1 Sect. S5). These results indicate that while the main activities remained consistent between the two groups of students, the pandemic restriction forced these activities to be done in different locations.

5.3.2 With whom: activities and sociality

Turning our attention to the social component of activities, we observe that in 2020, the majority of the extracted components primarily involve activities performed Alone, see Fig. 5(b) (the complete components can be found in the Additional file 1 Sect. S5). Alternatively, these components involve a small number of individuals who likely reside in the same household, such as Relatives, Partners, or Roommates. This finding is consistent with the COVID-19 restrictions that limited the variety of social interactions during that period.

Figure 5

Sociality and Activity Joint Components. This figure shows the joint sociality and activity components that most differentiate students behaviour before and during the pandemic. (a) includes routines from 2018, while (b) includes routines from 2020. The main activities are still Lesson and Study, which are social activities during 2018, while in 2020 students do them mostly alone (or with people in the same house)

In contrast, as shown in Fig. 5(a), the social component of activities in 2018 exhibits a greater variation, which involved also interactions with Friends and Classmates (e.g., Components 3 and 6), while Study is the only activity strongly associated with being Alone (Component 1).

Although the NMF algorithm extracted six dominant components for both years, in 2018, there is a greater diversity of activities compared to students in 2020, where the weights in the heatmaps are more concentrated on specific activities. The activity patterns observed in the resulting routines are similar to those detected when considering the combined locations and activities matrices, which again involve as main activities studying and having lessons.

Notably, in 2018, there is a presence of mobility activities (e.g., Moving), which are almost absent in 2020. Additionally, in Component 5 of 2020 (Fig. 5(b)), we can observe the presence of Social Life in the evening, primarily involving interactions with Roommates. In contrast, in 2018, without government restrictions, social activities are more consistently carried out in the company of Friends.

6 Discussion and conclusions

In this work, we have analyzed the changes in the behavioural routines of students of the University of Trento due to the COVID-19 pandemic, focusing on three dimensions of behaviour: their activity, their location and their sociality. Few studies have considered students’ behaviour, and generally, they tend to examine just one facet of behaviour [51]. Moreover, existing studies are often limited to the analysis of data that comes from either surveys, where data is collected manually, or mobile phones, where data is generally passively collected when triggered by specific events (e.g., phone calls, app usage, etc.). By leveraging an Experience Sampling Method (ESM) approach on smartphones, we collected self-reported information on the daily activities of university students by means of Time Diaries, which allowed for high-resolution and multimodal experience sampling data collection in a real-world setting, with a high level of control over participants’ responses.

Our investigation first focused on understanding how specific behaviours evolved from 2018 to 2020. We accomplished this by conducting several statistical tests, including the Kolmogorov–Smirnov test, to compare the frequency distributions of each behaviour in the three dimensions, enabling us to pinpoint behaviours that exhibited significant changes. Notably, among the most prevalent activities, we observed a decrease in engagement with activities like Moving and Social Life, coupled with an increase in sedentary activities such as watching shows and videos. Interestingly, main activities like Eating, Lesson and Study did not show significant changes. This shift is mirrored in the location dimension, where the majority of places experienced a reduction in frequency, while the Home location registered a substantial increase. Furthermore, the trend towards more solitary activities and locations is reaffirmed by the overall decrease in social interactions, even with people residing in the same household, with students spending more time alone. These changes can most likely be attributed to the impact of the restrictions in place during the 2020 data collection.

These findings served as a basis for a more comprehensive analysis of our dataset, which extended to the study of daily behavioural patterns. We used Non-Negative Matrix Factorization (NMF) to incorporate the temporal dimension of daily routines, which is neglected in the analysis of activity frequencies. Thanks to NMF interpretability and clustering properties, this technique is particularly useful in identifying repeating patterns in the daily behaviours of people, both at an individual and a collective level. It has been successfully used to study behavioural routines in different contexts such as urban mobility [70], people’s chronotypes [3] and interactions [32]. Our findings suggest that despite the COVID-19 restrictions, which have significantly altered various aspects of our lives [1, 48, 53, 65, 68], there is surprisingly little change in the main routine activities performed by students between 2018 (pre-pandemic) and 2020 (during the pandemic). Additionally, as shown in our analysis, the diversity of activities did not change significantly.

However, leveraging the richness of the data at our disposal, which enables a comprehensive multimodal analysis of students’ behaviour, we discovered that their adaptation to the pandemic circumstances primarily occurred in the location and sociality dimensions. We observed differences in these dimensions for the dominant activities that constitute typical student routines, such as studying and attending classes. For these routines, our results indicate that, while the main activities remained consistent between the two groups of students, the pandemic restriction forced these activities to be done in different locations, and, thus, with different people. These results, indeed, are in line with previous works on the impact of COVID-19 on students’ lives [51], and they are also reflected in the general population. Previous studies have shown that, during the pandemic, there is a reduction in the number of places visited and the number of social encounters [48, 56].

More in detail, in 2018, students’ activities such as studying and having lessons were often located at the university. In contrast, in 2020, due to the COVID-19 pandemic, students predominantly engaged in activities, including studying and lessons, from home, although there was the possibility for them to follow lessons at the university. Notably, when students went to university, their routines became more complex and involved a broader range of activities. From a sociality point of view, in 2020, the majority of social activities were performed alone or with a small number of individuals from the same household, reflecting the impact of COVID-19 restrictions on limiting social interactions. In contrast, the sociality components of activities in 2018 exhibited greater variation, including interactions with friends and classmates, indicating a broader range of social engagements before the pandemic. This could be concerning, given that the lack and the reduction of social interactions, along with more time spent alone, has a negative effect on students’ mental health, stress levels, sleeping patterns and general well-being [39, 51, 52, 69].

While our results stem from data collected from two living lab initiatives, in which the sample of students is different for 2018 and 2020, we believe that the outcomes are still reliable. In fact, this limitation is mitigated by the fact that the samples are from similar populations (students), and the data is collected in similar settings: they attend the same university and faculty in the same city.

Concluding, in applying Non-Negative Matrix Factorization, we have shown its usefulness in modelling students’ behaviour and in extracting meaningful patterns, i.e., routines. We believe that our contribution can be valuable for further understanding the impact of the COVID-19 pandemic on students. Moreover, we show that using a multimodal approach can be useful in identifying patterns of joint behavioural dimensions, and, thus, be used to understand the correlations among these dimensions. Finally, this methodology can also help correlate the behavioural dimensions with other facets of human activity, enriching the literature on well-being [51], stress levels [69], and mood [50] of students who were significantly affected by the pandemic.