Table 5 The mean values and standard deviations for our 5-fold cross-validation of different models, which we initialize with different random seeds. We show the best results for each column in bold. The names of our models are also in bold
From: Detecting political biases of named entities and hashtags on Twitter
Model | Tweet-Level Results (accuracy; \(F_{1}\)) | Account-Level Results (accuracy; \(F_{1}\)) |
|---|---|---|
Politicians’ Accounts (Mean Value ± Standard Deviation) | ||
Skip-Gram | 0.7700 ± 0.0026; 0.7707 ± 0.0029 | 0.8833 ± 0.0113; 0.8996 ± 0.0100 |
GloVe | 0.7231 ± 0.0039; 0.7319 ± 0.0035 | 0.8575 ± 0.0205; 0.8798 ± 0.0161 |
BERTbase | 0.8586 ± 0.0006; 0.8587 ± 0.0006 | 0.9963 ± 0.0034; 0.9963 ± 0.0034 |
BERTweet | 0.8337 ± 0.0010; 0.8327 ± 0.0010 | 0.9828 ± 0.0077; 0.9826 ± 0.0077 |
Polarized PEM\(_{ \mathrm{no\ attn} }\) | 0.7691 ± 0.0011; 0.7665 ± 0.0011 | 0.9721 ± 0.0244; 0.9723 ± 0.0243 |
Complete PEM\(_{ \mathrm{no\ attn} }\) | 0.7955 ± 0.0009; 0.7937 ± 0.0009 | 0.9805 ± 0.0169; 0.9811 ± 0.0167 |
Polarized PEM | 0.8338 ± 0.0007; 0.8336 ± 0.0007 | 0.9841 ± 0.0030; 0.9845 ± 0.0030 |
Complete PEM | 0.8332 ± 0.0006; 0.8327 ± 0.0006 | 0.9915 ± 0.0026; 0.9927 ± 0.0026 |
Unobserved Accounts (Mean Value ± Standard Deviation) | ||
Skip-Gram | 0.5822 ± 0.0007; 0.5635 ± 0.0008 | 0.6561 ± 0.0053; 0.6324 ± 0.0074 |
GloVe | 0.5764 ± 0.0009; 0.5574 ± 0.0009 | 0.6387 ± 0.0073; 0.6222 ± 0.0099 |
BERTbase | 0.6348 ± 0.0007; 0.6231 ± 0.0006 | 0.7182 ± 0.0078; 0.7149 ± 0.0072 |
BERTweet | 0.6282 ± 0.0006; 0.6280 ± 0.0005 | 0.7752 ± 0.0176; 0.7695 ± 0.0173 |
Polarized PEM\(_{ \mathrm{no\ attn} }\) | 0.6245 ± 0.0011; 0.6067 ± 0.0011 | 0.8062 ± 0.0191; 0.8105 ± 0.0182 |
Complete PEM\(_{ \mathrm{no\ attn} }\) | 0.6259 ± 0.0014; 0.6063 ± 0.0015 | 0.8467 ± 0.0177; 0.8450 ± 0.0178 |
Polarized PEM | 0.6284 ± 0.0023; 0.6865 ± 0.0020 | 0.8463 ± 0.0063; 0.8666 ± 0.0059 |
Complete PEM | 0.6472 ± 0.0030; 0.6907 ± 0.0028 | 0.8550 ± 0.0075; 0.8814 ± 0.0072 |