On the predictability of the popularity of online recipes

2.1 Research on popularity prediction on the web

Popularity prediction as a scientific problem is, to some extent, a response to the age-old problem of information saturation [15]. Since its development, the World Wide Web has only served to intensify Herbert Simon’s dictum that an overload of information leads to a poverty of attention [16]. Deciding which sources to attend to is a daily struggle for users [17, 18] and is open to numerous biases including subconscious human, as well as algorithmic biases [19].

Patterns in web page accesses are well known to be highly skewed and Zipfian distributed [20, 21]. Online videos, for example, which make up a significant proportion of Internet traffic, have been studied extensively [22–24], revealing that the interest generated by items shared on the web is ephemeral and complex, and is influenced by numerous, changing factors, all of which combine to make prediction difficult [22, 25].

Summarizing the literature reveals that typically moderate correlation is found between popularity metrics derived from various user interaction traces, such as likes, views, shares etc. [7, 26, 27]. This is most likely explained by the fact that metrics relate to different aspects, ranging from attractiveness to utility to quality, and are often associated with different user aims or actions (e.g., consumption, sharing, the provision of feedback, indirect communication etc. [28]).

Although popular items tend to be highly accepted as user recommendations, e.g., [29], it should be clear that popularity does not equate to quality. Past research has demonstrated that popularity is a complex, multifaceted concept, which is well known to be difficult to predict [30]. Nevertheless, by studying how diverse metrics relate to each other, e.g. [31, 32], researchers can achieve a broader and better understanding of what the popularity content actually means, as well as what can be achieved with accurate predictive models for popularity.

Typical foci of investigation for researchers studying popularity prediction are diverse information items shared on the web. These include movies (Simonoff and Sparrow 2000) or songs (Pachet and Sony 2012), online news articles [2], as well as various social network content (Facebook: [33], Twitter [34–36], Weibo [37] and YouTube [3, 4]).

The problem of predicting popularity has been formulated in various ways. Shulman [30] distinguishes between efforts to predict apriori based on item content and meta-data and modified versions of the problem where researchers are allowed to peek into early adoption activity for an item to inform the prediction. Peeking formulations have been proven to lead to predictions with greater accuracy, e.g., [33–35]. A second dimension by which formulations of the problem can be distinguished relates to the aspect being predicted. In regression formulations the exact popularity of an item measured on some scale is predicted, whereas in classification formulations popularity is discretized into distinct classes, again based on some specified criteria [38]. The literature suggests that the latter makes the problem more tractable [30].

The literature summarized thus far emphasizes that popularity prediction has been studied in various contexts, using divergent methods, with heterogeneous motivations. One context, however, which has yet to be studied in detail are popularity metrics relating to online recipes. We believe this would offer utility for a number of reasons relating to understanding food culture, eating habits and links to epidemiology research, as well as the design of technology to assist people nourish themselves more healthily. We explain these points in more detail in the following section by summarizing relevant related work.

2.2 Research on online recipes

The way people interact with recipes online can provide clues regarding user food preferences and eating habits. Kusmierczyk et al. and Trattner et al. analyzed data from the German community platform Kochbar.de and found clear seasonal and weekly trends in online food recipe production, both in terms of nutritional value (fat, proteins, carbohydrates, and calories) [39, 40] and in terms of ingredient combinations and experimentation [41]. Similar patterns were observed by Wagner et al. [42] and West et al. [43]. West and colleagues also found correlations between recipes accessed via search engines and incidence of diet-related illness, which resemble findings reported recently by Said & Bellogin [44], De Coudhury et al. [45] and Abbar et al. [12] in the context of Allrecipes.com, Instagram and Twitter respectively. Rokicki et al. [46] investigated differences in nutritional values between user recipes created by different user groups, finding, for example, that recipes from females are, on average, richer in carbohydrates. The carbohydrate content of recipes seems to decrease with increasing user age, which mirrors the advice given by most nutrition advice centers. Thus, the literature provides strong evidence for the utility of using interactions with online food items to understand eating habits and for epidemiological purposes.

These published analyses are all very related to work in the recommender systems literature, which attempts to model user food preferences to present individual users with meal recommendations that they would like to eat [47, 48] or to assist users make healthier food choices [49]. A recent review of research in this area can be found in [50]. The literature emphasizes that food choice is complex and context sensitive, with user, situational, temporal, and social factors playing a role [51].

The research reviewed in this section is relatively new and data-driven. The research has, moreover, been largely performed without being related to an extensive body of work from psychology, dating from the 1950s, which has investigated the factors influencing the food choices people make. In the following section we provide a brief review of important work in this domain, which later informs the feature engineering procedures in our work.

2.3 The psychology of food choices

People typically make around 200 food choices every day [52]. Choosing which food to eat is a complex process influenced by a number of context factors at biological, personal, situational, social and socio-economic levels [53]. Among the most commonly investigated factors in the food literature are taste or sensory appeal, health-related issues, ethical concerns, convenience, price, and weight control considerations [54]. Food choices reflect mood, with people having been shown to eat to receive emotional comfort or improved mood, evoke past experiences or experience something new when choosing food [54–56].

Individual differences can be found with respect to the importance placed on these diverse attributes and this can depend on other factors including age, gender, race, lifestyle, socioeconomic status, cultural background, and education [57, 58]. However, the evidence suggests that for most people, the driving factors seem to be the taste of food and how it appeals to one’s other senses, followed by concerns about health, weight control, nutritional value, and cost [59, 60] and the recently published studies summarized above with online recipes only seem to confirm this [48, 49]. That being said, the food choices people make can be biased in countless ways. For example, people make poor decisions when stimulated (e.g., when hungry and surrounded by the sights and smells of calorie rich food) [52] or when emotional [61] or stressed [62]. People, moreover, adapt their behaviour to the social context with obese individuals more likely to be friends with other obese individuals [63] and people consume more when they eat in groups, rather than alone [52].

Thus, popularity prediction in the domain of online recipe portals is related to food recommender systems, which is, in turn, related to studies in psychology to understand the processes of food choice. All three bodies of research point to the prediction of the popularity of recipes being a challenging task. That being said, by explaining some of the factors influencing food choices, the literature also offers hints as to features, which may be helpful for predicting popularity. We use these hints to drive our analyses, as we experiment with popularity prediction in the context of online recipes.

2.4 Building on prior research

The research presented in this article extends previous work in at least three distinct ways. Firstly, we are the first, to our knowledge, to examine the popularity prediction problem in the context of online recipes. We argue that there is a number of potential benefits to this, including the possibility of improved recommendations, user advice on how to improve their content, and an instrument for learning about societal eating habits. Secondly, although both datasets presented have been studied to some extent in the past, neither have been studied with respect to popularity prediction, nor does any publication exist, which compares the two datasets directly. This is advantageous because it offers insight into how two divergent food cultures compare and contrast. Thirdly, we collate and evaluate an extensive set of features from diverse related work including from papers in the popularity prediction literature and from papers in the online recipe literature. We also derive new features from the food psychology literature, which we believed may offer utility for the task at hand. Although many of the features have been investigated to some extent in the past, this article presents an evaluation of a comprehensive feature set, which goes beyond any previous investigation.

3.1 Allrecipes.com

The first dataset used and that shall be described is Allrecipes.com. According to Ebizmba.com² it is the most popular online food platform on the Web and has global website rank³ of 885 and 246 inside the USA (July 2017). The dataset was obtained between 20th and 24th of July 2015 by implementing a standard Web crawler [64]. It contains 60,983 recipes published between the years 2000 and 2015 on the Allrecipes.com website. Of these recipes, 58,263 contain nutrition information. The recipes were published by 25,037 users during this time period. The recipes and associated user profiles were obtained via the sitemap available in the robots.txt file.

Recipe Profile View. As presented in Fig. 1 and sub-figures (a)–(c), the recipe profile view presents the user with all relevant features about the recipe and allows other users to rate and comment on the recipe. Besides typical entities such as average rating, number of ratings, image, recipe ingredients and cooking directions, this view also features the nutritional information of the meal. This information is only available for public and reviewed recipes and not for personal recipes uploaded to the user’s private space. To estimate the nutritional information of the recipes uploaded by the community, Allrecipes.com relies on the ESHA Research’s nutrient database,⁴ which allows Allrecipes.com to provide information for 18 different nutrients including energy (kCal), protein (g), carbohydrate (g), sugar (g), sodium (g), fat (g) saturated fat (g), and fiber (g). User comments and ratings are displayed at the bottom of the view. Additionally, there is the “rate and review” button, when pressed it shows a window with the rating feature (based on 5 stars) and a text area for the comment. Important to note is that these screenshots were taken in August of 2017. However, the dataset was crawled in 2015 when providing a rating was mandatory when commenting on a recipe. This is no longer the case.

User Profile View. We do not provide screenshots for this view as although publicly available, these contain user personal data. The user view in Allrecipes.com comprises information about a user’s uploaded, favoured recipes, comments and rated recipes. Until 2015 this view also contained information about where the cook is currently living, member since information, cooking interests, hobbies and some additional text describing the user. We made use of the users’ “current home location” information, which is provided on city and county level, in our analyses.

Recipe Upload View. Sub-plot (d) in Fig. 1 presents the upload view. In order to upload a recipe to Allrecipes.com, one has to provide multiple recipe parameters. These include the recipe title, a description, the preparation steps, a list of ingredients, preparation and cooking time and the number of servings. Also, the privacy level must be defined. When submitted as a public recipe and “Kitchen Approved”, the editorial staff will review the recipe to ensure quality and calculate the nutritional information based on the ingredients. A picture of the recipe is optional. Furthermore, it is important to note that the ingredients are captured with free form text with no spellcheck or guiding feature. This is the reason for the many misspellings and word variants found in the data set (see Sect. 4.1).

A very important feature of Allrecipes.com is that the platform also provides categorical information which was extracted from the platform’s category tree which is as available through the platform’s ‘category view’.⁵

3.2 Kochbar.de

The second dataset we employ in this work is from the online food website Kochbar.de. It was crawled by [41] in 2014. According to [65] Kochbar.de attracts more than 6.6 million unique users every month, is ranked among the top-50 most popular websites in Germany and is the second most popular food website in Europe.

Kochbar.de hosts about half a million recipes by users from all over the world. In 2014 Kusmierczyk et al. [41] crawled over 400,000 recipes uploaded between the years 2008 and 2014. These recipes were uploaded by nearly 200,000 unique users. Kochbar.de is very similar to Allrecipes.com in terms of how the platform works as well as the functionalities provided to users.

As in Allrecipes.com, Kochbar.de offers two main views to users when inspecting a recipe or when uploading a recipe to the platform. These are described below along with the user profile view.

Recipe Profile View. As presented in Fig. 2 and sub-figures (A)–(C), the recipe view shows almost the same information as the corresponding view in Allrecipes.com. It displays the main recipe information such as title, description, preparation steps/time, and ingredients.

Only 4 nutrients are presented to the user, which are estimated using the German Nutrient Data Base.⁶ In particular, calories, protein, carbohydrates and fat content measured in 100 g per meal are presented. The rating and commenting functionalities are slightly different to those offered in Allrecipes.com. As mentioned above, in Allrecipes.com, users could only rate and comment on recipes together at least until 2015. In Kochbar.de these two features are separated.

User Profile View. Again we do not provide screenshots for this view as it contains personal data. As with Allrecipes.com, the user view in Kochbar.de information about the user’s uploaded recipes and gives the user the opportunity to provide information about the town/country the user is currently living in, and some specific text fields related to cooking interests. In the same way as with the Allrecipes.com data, we made use of the users’ “current home location” information in our analyses.

Recipe Upload View. Fig. 2 and sub-plots (A)–(B) presents the upload view, which is more complex than the one provided by Allrecipes.com. Besides the standard recipe parameters such as title, description and preparation time, difficulty level or price level, it also features an elaborate ingredients definition widget. The ingredients widget supports the user with ingredient suggestions based on the input provided. Along with the ingredient name, it has two separate input fields for the amount and the unit. Presumably, this improves the automatic ingredient parsing, on which the nutrient information is based on. Nevertheless, there are still many misspellings and word variants in this data set (see Sect. 4.1). This could either be explained with incorrect usage of the ingredient widget, misspelled ingredients that are already in the Kochbar.de database, or that the widget has not been always been part of the upload form. The cooking instructions are also captured in a more structured manner. The form provides separate text fields for every step. Photographs of the meal can also be uploaded. In contrast to Allrecipes.com, the Kochbar.de upload form allows the user to manually overwrite the automatically calculated nutritional information. It is also possible to assign descriptive tags and categories to the recipe, which is not possible in Allrecipes.com.

Similar to Allrecipes.com, Kochbar.de contains a dedicated sub-page with a category tree⁷ relating recipes to categories from which we obtained the category labels of a recipe.

Table 2 gives an overview of the basic statistics of the Kochbar.de dataset and Table 1 provides and overview of the available entities used later for feature extraction.

4.1 Data selection and pre-processing

For the purpose of our analyses we made use of recipes that were (1) main meals, (2) provided nutrition information and (3) had at least one image of the prepared meal available. In Allrecipes.com \(11,194\) recipes fulfill these criteria while for Kochbar.de the number was \(81,232\). We chose to focus mostly on main dishes because, as shown in Fig. 4, the frequency for which recipes in different categories are uploaded varies strongly across the platforms. Main dishes is a popular category common to both platforms, and thus provides a fair basis from which to compare the platforms. We do provide additional analyses using all of the recipes in the datasets, including the prediction experiments and we emphasize this to the reader as appropriate.

Since both datasets had been used for other purposes in the context of previous studies [41, 64], the data had already been structured and partially cleaned. That being said, the ingredient lists for both datasets were noisy due to misspellings and natural vocabulary variation that occur when recipes are uploaded by users. To tackle these issues, we made use of the Spoonacular.com Web API,⁸ which offers services for analyzing food and recipes.⁹ The API is free to use for academic purposes and allows the extraction of normalized ingredients from free-form text. As Spoonacular.com only supports English text, the Google translate API was employed to translate German to English text. Text that could not be recognized by the Spoonacular API (such as misspellings) was converted manually.

Although this pre-processing pipeline is not perfect, manual inspection of the results demonstrated to us that in the vast majority of cases, the output produced was accurate. Taking a sample of 1000 matched ingredients revealed only five results we considered inappropriate, e.g., “tomatoes dried in oil” was matched as “dried tomatoes” and “flour to roll out” was matched to “roll”. The other two mismatches would make very little nutritional difference, e.g., “China spice” was matched as “BBQ Spice”. Many of the results were impressive. For example, even though the German phrase for free-range eggs “Freilandeier” was translated as “eggs Freiland”, this still resulted in a usable match of “eggs”.

By taking this approach we were able to reduce the total number of unique ingredients from 723,911 to 3842 for Allrecipes.com and from 302,126 to 2028 for Kochbar.de.

4.2 Analyzing popularity

As indicators for popularity we chose the number of comments and the number of ratings for each recipe as these were available for both datasets. It made little sense to analyze ratings applied or the sentiment of comments on recipes as neither varies to a great extent across recipes. Over 99% of the recipes in the Kochbar.de dataset were given a 5-star rating and while in Allrecipes.com the trend is not so pronounced, most of the recipes with ratings were rated as 4 or 5.

We also plot average number of comments/ratings over time, both cumulatively and non-cumulatively. Further insight is provided by calculating mean and median popularity for each time slot, that is for 1 day, 7 days (one week), 30 days (one month) and 365 days (one year).

4.3 Feature engineering for recipe popularity prediction

To investigate which factors help explain popularity, we derived a set of features relating to aspects highlighted in the literature summarized in Sects. 2.2 and 2.3.

The features relate to a recipe’s content, presentation, nutrition and healthiness, complexity, seasonality and innovation (7 feature sets in total). We also derive features capturing aspects relating to the authoring user and the context surrounding the interaction. Although many of these features have been used in previous investigations in various diverse contexts, it is the first time they have been examined together, in exhaustive detail. It is also the first time that these features have been tested in the context of predicting popularity.

We briefly describe the main groupings below and explain how these were calculated. The numbers in the brackets denote the number of features in each feature set.

Recipe Healthiness (1). To establish the healthiness of a recipe—often cited as a motivator for food choice [59, 60]—we derive the WHO health score, reported in [64, 66], which defines healthy ranges for the daily intake of macro- and micro-nutritional components as recommended by the World Health Organization (WHO). The nutrients used to derive the score are proteins, fat and carbohydrates. As in previous studies using the metric, the score ranges from 0 if none of the ranges were met, to 3 if all were met. The feature is labeled as WHO health score throughout this paper.

Recipe Complexity (5). Complexity are concepts associated with food preferences in the food recommender systems literature [51].

Textual presentation relates to 5 features, which capture aspects of the recipe title and 6 features capturing the presentation of instruction text [74].

4.4 Comparative statistical analysis

To compare the two communities we computed standard descriptive statistics, such as mean, median, minimum and maximum, for all features on both datasets and used significance tests to establish differences. Due to the differences in the feature distributions it was necessary to employ various tests. A Brown–Forsythe test for finding statistically equal variance was utilized. The Brown–Forsythe test is used for group comparison based on median absolute deviations (MAD) and is more robust against outliers compared to the Leven’s test, which uses the mean. In the case of equal variance of the two feature distributions (\(p < 0.05\)), a Wilcoxon Rank sum test was performed. When the test rejected the equal variance hypothesis, a two-sample Kolmogorov–Smirnov (KS) test was used. We report p-values.

“While a p-value can inform whether an effect exists, the p-value will not reveal the size of the effect” [80]. As such, we also computed the effect size r for each statistical test [81]. The effect sizes were calculated as \(\frac{Z}{\sqrt{n_{x}+n_{y}}}\), where Z is the Z statistic, and \(n_{x}\) the size of sample x and \(n_{y}\) the size of sample y.

We mark effect sizes \(0.1 \leq r < 0.3\) with an ∘ symbol (small), effect sizes \(0.3 \leq r < 0.5\) with an ∘∘ symbol (medium) and effect sizes great than \(r \geq0.5\) with an ∘∘∘ symbol (large) according to Cohen’s criteria [82]. According to Cohen [82] effect sizes smaller than 0.1 are irrelevant and as such are not highlighted.

4.5 Predictive modelling

The literature review presented in Sect. 2 illustrates that the task of popularity prediction can be set up in diverse ways. In this work the prediction task is preceded by a correlation analysis, which informs on possible correlations between predictors (features) and outcome variables (popularity proxies). As a metric we chose Spearman’s rank correlation coefficient, since the metric assesses both linear and non linear relationships between two variables and can cope with continuous and discrete variables simultaneously. Subsequently, we operationalize and evaluate a prediction task. In line with recent literature, we chose a classification setup to predict apriori whether a recipe will become popular in the future. We establish binary classification variables to determine whether a recipe belongs to the popular or non popular class and attempt to validate models with test data. Reflecting this we calculated medians of the popularity metrics, following the approach in [30]. Recipes below the median are considered as negative and above as positive examples. While it is true that a median split may result in many recipes being close to the boundary between the two classes, we chose to remain with the standardized approach based on the findings of Hofman and colleagues [83], who highlight the danger of non-standardization in prediction tasks, such as ours. They show empirically that individually defensible choices can arrive at qualitatively different answers to the same question.

The experiments were performed on complete and balanced datasets. To ensure completeness, potential missing data was imputed with the help of the R library Hmisc.¹⁷ To ensure balance, the majority classes were undersampled randomly employing R’s sample procedure. The classification experiment itself was conducted with 3 different classifiers using the R library Caret.¹⁸ Besides a Random Forest (RF) classifier, Logistic Regression (LOG) and Naive Bayes (NB) were employed. RF and NB have been successfully applied in similar ‘before publication’ popularity classification studies [30]. As evaluation protocol five fold-cross validation was chosen and ‘Accuracy’ was used as the main performance metric. Furthermore, the variable importance for each feature set was reported employing Caret’s ‘VarImp’ feature, to capture the importance of the variables after the models have been built, as well as FSelector’s¹⁹ InfoGain feature to find the Top-10 features before model construction.

5.1 Popularity analysis (RQ1)

In a first step we investigate popularity across datasets, looking at how popularity of recipes differs over time and across the two platforms. In this step we look for patterns over the full datasets i.e., using recipes in all categories.

A first view on temporal patterns in popularity can be seen in Figs. 5 (A)–(I). Figures 5 (A), (D), (G) display the number of comments and ratings assigned applied non-cumulatively over time. Figures 5 (B), (E), (H) show the same information cumulatively. Finally, Figs. 5 (C), (F), (I) use violin plots to demonstrate how the same popularity aspects evolve over time for the two recipe platforms. Whereas in the case of Allrecipes.com a constant increase can be observed in the number of ratings and comments over time, a different picture is shown for Kochbar.de. Here we see that most of the comments and ratings are provided within the first 7 days after publishing and little development is seen thereafter. One can observe a noticeable decay pattern for Kochbar.de whereby after a period of 10 days after publication any popularity tends to have dissipated. A different pattern is observed for Allrecipes.com where popularity is, on the whole, a much more consistent phenomenon, in terms of how and when recipes are attended to.

Figure 6 provides a different view of recipe appreciation by showing how recipes are commented on and rated over time after the date of publication. The recipes on both platforms receive overwhelmingly positive ratings with a low standard error for ratings being found in both datasets. While Kochbar.de recipes receive on average a rating of 5 with very little spread (nearly all rated recipes are given 5 stars), Allrecipes.com recipes obtain a slightly lower average rating of 4.41. On the other hand, the comment sentiment is higher for Allrecipes.com recipes when compared to the Kochbar.de recipes (2 vs 1.8). The analyses show limited variation in recipe appreciation, both generally across recipes and for the same recipe over time (see Fig. 5). This means that the ratings provided for a recipe are stable and do not seem to be too heavily influenced by fashions or trends. This is unsurprising for Kochbar.de, where the ratings tend to be applied in the short period after a recipe is published, but more so for the Allrecipes.com, where recipes are discovered and evaluated for much longer periods.

Figure 4 shows the distribution of the top-20 categories in Allrecipes.com and Kochbar.de (considered together and separately), revealing variance in the popularity of different recipe categories. The most popular category in terms of number of uploaded recipes (measured in percentages),²⁰ when considering both portals, is the ‘main dish’ category (18.35% vs 20.01% (Kochbar.de)), followed by ‘desserts’ (18.90% vs 7.90% (Kochbar.de)) and ‘side dishes’ (12.37% vs 3.51% (Kochbar.de)). The dessert category is the most popular in Allrecipes.com (\(N=11\mbox{,}526\)), if no overlapping categories are considered, while the ‘intolerance’ (\(N=11\mbox{,}526\)) is the most prominent one in Kochbar.de (\(N=275\mbox{,}478\)), followed by ‘without Wheat’ and other health related categories. The popularity of recipes in these categories demonstrates dietary trends in Europe [84, 85].²¹

Thus, these initial analyses show that patterns in popularity do exist. It seems that recipe appreciation differs across platforms (recipes on Allrecipes.com continue to gain in popularity for longer than on Kochbar.de) and across categories within the two platforms (different categories are popular on the two sites with dietary trends featuring heavily on Kochbar.de, but not on Allrecipes.com). We build on these results by statistically comparing the two services with respect to the features outlined in Sect. 4.3. As inconsistent patterns were observed here across categories and because categories are disjoint across platforms, we restrict our comparison to main dishes, a popular category common to both Allrecipes.com and Kochbar.de.

5.2 Comparative statistical analysis (RQ2)

Notable exceptions are ‘Image Sharpness Variation’ (\(M = 0.29\) vs 0.29 (Kochbar.de); p < 0.1), ‘Day of Month’ when the recipe was uploaded (\(M = 15.71\) vs 15.72 \(p = 0.541\)) and the feature ‘Living in Germany/USA’ (\(M = 0.88\) vs 0.90 (Kochbar.de); \(p < 0.1\)).

Due to the frequency with which significant results are found, we additionally report effect sizes. When examining the r values in Table 3 we find the features with the highest effect sizes for ‘User Activity & Context’ and ‘Recipe Popularity & and Appreciation’ sets. A good example is that ‘Num. Ratings received within week’ shows significantly lower values for Allrecipes.com than for Kochbar.de (\(M = 0.38\) vs 2.39 (Kochbar.de); \(p < 0.001\); \(r = 0.28\)). In the User Activity & Context set the feature ‘Num. Comments Received until Upload’ showed the highest effect size (\(M = 73.36\) vs 2472.59; \(p < 0.001\); \(r = 0.44\)).

In the other feature sets, such as ‘Recipe Nutrition’, Protein (per 100 g) showed the largest effect size and was significantly higher for Allrecipes.com recipes (\(M = 9.44\) vs 6.49 (Kochbar.de); \(p < 0.001\); \(r = 0.23\)). In the ‘Recipe Complexity’ set the feature ‘Num. Categories’ showed the highest effect size with significantly more categories applied to recipes in Kochbar.de (\(M = 4.31\) vs 12.61 (Kochbar.de); \(p < 0.001\); \(r = 0.50\)). In the ‘Recipe Presentation’ set we find the highest effect size for the feature ‘Instruction: Readability Score’ with a sign higher score for Kochbar.de recipes (\(M = 30.42\) vs 48.84 (Kochbar.de); \(p < 0.001\); \(r = 0.53\)). Lastly, for the ‘Recipe Innovation’ set we find the highest effect size for the feature ‘Recipe Innovation IDF’ which is significantly larger for Allrecipes.com (\(M = 0.02\) vs 0.00 (Kochbar.de); \(p < 0.001\); \(r = 0.37\)). It is worth mentioning that for the ‘Recipe Seasonality’ and ‘Recipe Healthiness’ sets we did not find any effect sizes above \(r < 0.1\).

In this section we have demonstrated several differences in the properties of the recipes sourced from two different platforms. The Allrecipe.com are more protein rich, whereas in terms of innovation, it seems Kochbar.de is more conservative. Ratings and categorization behavior seems to differ with recipes in Kochbar.de being assigned to more categories and receiving more comments; and drop-off rates being seen in ratings on this service. In the following section we examine these properties to establish a relationship between these properties and popularity. This forms a useful precursor to the modeling work in Sect. 5.4.

5.3 Correlation analysis (RQ3)

The results presented in Sect. 5.1 demonstrate the existence of patterns in popularity with respect to platform over time and across categories. Building on this, in Sect. 5.2 we demonstrated that the two datasets differ in terms of the developed features. The next step is to discover if popularity prediction can be achieved and if so, which features offer utility for this purpose. In a first step, we perform a correlation analysis with this aim in mind. Figures 7 and 8 show the outcome of the correlation analysis for Allrecipes.com and Kochbar.de. The two figures present a correlation matrix based on Spearman’s rank correlation where non-significant features (\(p<0.05\)) are marked with black dots and significant features in respect to popularity metrics are marked with a red asterisk.

When comparing both matrices, it is obvious that more and stronger correlations exist for Kochbar.de than Allrecipes.com.

For example, high positive and significant correlations (ρ up to 0.94) are found in Kochbar.de between the user activity features (number of recipes uploaded, comments written or ratings provided) and the outcome variables (number of comments/ratings received within time-span). The fact that these features show strong positive correlations supports the assumption that the previous user activity may relate to the future popularity of recipes these users upload.

Further observations of note relate to recipe innovation. In the Kochbar.de as well as in the Allrecipes.com dataset the outcome variables significantly negatively correlate with the innovation feature ‘Recipe Innovation IDF’ (ρ up to −0.35 for Allrecipes.com and −0.15 for Kochbar.de). Hence, innovative recipes on the both platforms tend to receive more ratings and comments.²²

In terms of the presentation of recipes, the recipe instruction readability score feature (ρ up to −0.13 for Allrecipes.com and Kochbar.de) is negatively and significantly correlated with the outcome variables (lower numbers in readability means the less complex and the more readable the text). Other than that positive correlations with the outcome variables can be found in Kochbar.de also for the image feature ‘Image:Brightness’ (ρ up to 0.13) or the number of title words or characters (ρ up to 0.26).

The remaining correlations are less surprising. For example, as one would expect, the number of preparation steps is positively correlated with the size of the instruction text and correlations can be found when comparing image features are with each other.

The correlation analyses reveal some hints that prediction may be possible with Kochbar.de, in particular correlations were revealed between features and outcome variables. The findings suggest that features measuring the user activity, recipe innovation and presentation would be descriminative when predicting popularity in Kochbar.de. The analyses suggest that prediction may be more difficult with the Allrecipes.com dataset as the patterns found were not so pronounced (less significant correlations). However, they are comparable to Kochbar.de and it seems that recipe innovation and recipe presentation (instruction readability) are the most important ones. Next, we run experiments to determine the potential for popularity prediction.

5.4 Predictive modeling (RQ4)

Our analyses thus far have focused on the main dishes category for the reasons outlined above. However, to examine the generalizability of the approach, we additionally report the results of experiments predicting the future popularity of all recipes in both collections. The top-half of the Table 4 shows the outcomes for the category ‘main dishes’ for both popularity proxies employed (‘number of ratings’ and ‘number of comments’), while the bottom half shows the results for the same experiments over the whole dataset (i.e. over all categories). Table 4 reveals that regardless of the classifier employed (NB, LOG, and RF) and proxy studied (comments or ratings), the feature sets offering best performance are consistently ‘Recipe innovation’ features for Allrecipes.com and ‘User Activity & Context’ for Kochbar.de. In most cases, when predicting the future popularity of main meals, the best prediction results are achieved with Logistic Regression (LOG) for Allrecipes.com and with Random Forests (RF) for Kochbar.de.

The highest performance is achieved using number of ratings as a proxy for popularity. When combining all features (All set) we are able to accurately predict popularity in up to 60.23% of cases for Allrecipes.com data and up to 88.45% of cases in Kochbar.de. When considering only the Top-10 features, as determined using Information Gain, slightly improved performance can be attained for the Kochbar.de dataset. Given the difficulty of problem being tackled, we feel the performance is remarkably high. Bearing the stronger and more plentiful correlations found in Sect. 5.3 in mind, the fact that better results were achieved with the Kochbar.de recipes is less surprising. Overall, we observe that performance when predicting popularity after 1 week and 1 month is comparable.

When number of comments is used as a proxy for popularity, the results demonstrate similar patterns for both platforms. Overall the task becomes slightly more difficult with the accuracy values of typically 3–4% less being achieved. Taking number of comments as the popularity measure, in Allrecipes.com we achieve maximum prediction accuracy of 61.39% when predicting for the one week (7 days) and 58.62% when predicting one month in advance (30 days). In Kochbar.de, again we achieve higher accuracy. Predicting popularity one week ahead on the German dataset, we achieve a maximum accuracy of 86.04% and for one month ahead the best result was 85.04%. The second and third best feature sets using comments for Allrecipes.com are ‘User Activity & Context’ and the ‘Recipe Presentation’, with an accuracy up to 56.82% being achieved. A similar trend is observed when investigating the second and third best feature sets in Kochbar.de. Here we also find that the ‘Recipe Presentation’ feature set works remarkably well, as well as the ‘Recipe Complexity’ set with Accuracy values up to 70.16% for comments.

When predicting popularity using the full dataset (bottom-half of Table 4), in general the same trends are observed, with ‘Recipe Innovation’ and ‘User Activity & Context’ being the most important feature sets, although, the overall predictions are typically 1–3% lower in terms of accuracy, which is to be expected given the inconsistent patterns observed across categories as shown in Sect. 5.1.

To study the effects of differences in popularity across categories in more detail, we trained models using only category information (Top-20 Categories). To do this we represented recipes by a binary vector of size 20, one element for each of the twenty most popular categories for that platform, where 1 denotes that the category had been applied to the recipe and 0 denotes this not to be the case. Surprisingly, given the differences observed across categories, none of these models could outperform any of the other full models (all), although in some of the cases, model trained on ‘Recipe Nutrition’ and ‘Recipe Healthiness’ sets were outperformed. The best performance attained by a “Top-20-Categories” model is achieved Kochbar.de dataset for the proxy ‘number of ratings’ with an accuracy value of 58.77%. In Allrecipes.com this model reaches no higher than 54.22%. This indicates that there is a signal in categories but weaker as the other signals derived directly form the recipes.

As a final step, we investigate the contribution of individual features to the predictive models. Figures 9 and 10 illustrates this graphically, showing the variable importance, measured by Breiman’s method,²³ for the highest performing model for each dataset/prediction task combination, for main dishes and for all dishes, respectively.

It seems that while ‘recipe innovation’ and ‘recipe presentation’ were the most important predictive features in Allrecipes.com, for Kochbar.de, the most influential features were related to user activity, that is the recipes posted by prominent users in the community typically go on to be the most popular. In line with the prediction results obtained before the best individual feature seems to be ‘Recipe Innovation IDF’ for Allrecipes.com while it is ‘Num. Comments received until upload’ and ‘Num. Comments Written until Upload’ for Kochbar.de. Comparing Figs. 9 and 10 suggests that the overall patterns regarding the influential features for each service is similar regardless of whether “main dishes” or “all categories” are being studied. One slight difference that can be observed is that innovation factors become more important when you predict popularity over all categories on Allrecipes.com.

6.1 The primary findings

RQ1. While investigating popularity patterns we uncovered significant differences between the two platforms and across categories within platforms. Whereas for Allrecipes.com the most popular categories reflected the traditional distinction between mains, sides and desserts, on Kochbar.de popular recipes categories often relate to health and dietary preference, such as allergies and intolerance. We, moreover, observed differences in temporal patterns in popularity between the two sites with Allrecipes.com having stable patterns of appreciation whereas Kochbar.de had steep drop-off rates, most likely reflecting differences in how recipes are accessed and promoted on the two sites.

RQ2. We uncovered differences between the platforms when investigating the features outlined in Sect. 4.3. Recipes from Allrecipes.com were found to be higher in protein content than those on Kochbar.de, however, no effect sizes were observed for the WHO score, which is a measure of overall nutritional healthiness was close to zero. Thus, we cannot claim that the recipes of one service are healthier than the other. A further distinction between the recipes on the two services relates to recipe novelty. The recipes on Kochbar.de seem to be more homogeneous with the innovation scores being significantly higher for the recipes in the Allrecipes.com dataset.

RQ3. The correlation analysis presented in Sect. 5.3 revealed relationships among the predictive features and between the predictive features and the outcome variables (proxies for popularity). More and stronger correlations were found when analyzing the Kochbar.de dataset. The strongest and significant correlations included user activity features, which represent how active a user is within the community, such as the number of comments or ratings received within a given time span, and the outcome variables. Other strong correlations for both datasets were observed for recipe innovation and the complexity of a recipe.

RQ4. Mirroring the findings of the correlation analysis, the performance of the derived predictive models was consistently better for the Kochbar.de dataset. Typical accuracy achieved was \(\sim80\%\) for the Kochbar.de experiments whereas an accuracy of 56–60% was achieved in the Allrecipes.com experiments. Overall, given the difficulty of the task, we were pleasantly surprised by the predictive performance achieved across the experiments for both portals, with stable results being provided by the various feature sets studied. The stability in performance using different popularity proxies and time windows for predictions provides evidence supporting the robustness of the predictive features. ‘Recipe innovation’ and ‘recipe presentation’ were the most important classes of predictive features in Allrecipes.com but for Kochbar.de, the features with most predictive power were related to user activity. In other words, on Kochbar.de the recipes posted by prominent users in the community typically go on to be the most popular.

6.2 Discussion

The findings can be related to past and future work. Firstly, we have shown that there are signals in the recipe content and user interaction patterns with recipes, which allow popularity to be predicted to some extent and to a large extent on Kochbar.de. This is a contribution to the popularity prediction literature, which has shown prediction to be possible on many web-based items, but not online recipes. In doing so we open up the benefits that have been offered in other contexts, such as driving and maintaining user engagement [5], informing recommendations [6] and influencing caching strategies [9, 10] to online food portals. A large contribution of our work here is the feature engineering efforts. By collating and documenting features from literature from diverse fields, we provide a strong platform for researchers to continue our work and build on popularity prediction in online recipe settings, perhaps by using different modeling approaches or by studying other food portals.

In our work we studied two different platforms and by doing so we were able to identify similarities and differences across the communities. We wish to note that because of several, sometimes subtle differences in the platforms, their interfaces and contexts in which they are used, caution should be emphasized in interpreting the findings. We identified similarities, which include the healthiness of recipes, the number of preparation steps, image properties for uploaded food photos and the percentage of users of the service resident in the primary country (Germany or the United States). Differences identified include the longevity of popularity, with Kochbar.de recipes only having a short period of user interest as previously reported with social media posts [86] and Allrecipes.com recipes offering utility for longer periods, as can be observed with web pages [87]. Further differences between the services studied relate to the content of the recipes themselves with the Kochbar.de recipes being more homogeneous in terms of the ingredients used and the Allrecipes.com recipes being more innovative according to existing metrics [41]. The Allrecipes.com recipes contained more protein and because we know that posted and interacted with recipes relate to those recipes actually consumed [11–13], such comparisons are a useful complement to more traditional dietary comparisons, such as [88, 89].

In terms of predicting popularity, recipe innovation was the most important factor in Allrecipes.com with more innovative recipes tending to be more popular overall. Interestingly, opposite trend was true for Kochbar.de. The most important predictive features for Kochbar.de related to user behaviours. A strong signal common to both platforms, however, comes from attributes of the uploaded images of recipes. This aligns with the findings of past work from the food recommendation literature [68] and indeed the psychology literature, which also emphasizes the visual nature of food choice [90, 91].

Our findings suggest that further work should be done on the visual features of food images building on the efforts started by Yang and colleagues [92] and Salvador and colleagues [93]. Other future work would be to investigate the usage of features, which assist users improve their content and its presentation based on models, such as those which we have presented. Past work has shown how subtle changes in presentation can influence how online recipes are perceived [49]. It would be interesting to know how suggestions would be used and how this would influence interaction dynamics with online food portals generally.