Skip to main content

Table 7 3-classes experiments results with 4 datasets

From: SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods

Dataset Method Accur. Posit. sentiment Negat. sentiment Neut. sentiment Macro- F 1
P R F 1 P R F 1 P R F 1
Tweets_Semeval AFINN 62.36 61.10 70.09 65.28 44.08 55.56 49.15 71.43 58.57 64.37 59.60
ANEW_SUB 39.51 38.79 96.31 55.31 43.50 23.18 30.24 57.38 2.31 4.45 30.00
Emolex 48.74 48.15 62.71 54.47 31.27 38.59 34.55 57.90 41.30 48.21 45.74
Emoticons 52.88 72.83 11.34 19.62 55.56 5.38 9.80 34.05 96.53 50.34 26.59
Emoticons DS 36.59 36.55 100.00 53.53 75.00 0.36 0.71 100.00 0.03 0.07 18.10
NRC Hashtag 36.95 42.04 75.03 53.88 24.57 56.03 34.16 53.33 3.70 6.92 31.65
LIWC07 39.54 36.52 42.33 39.21 15.14 13.02 14.00 48.64 44.83 46.66 33.29
LIWC15 62.56 59.77 71.03 64.91 49.04 42.65 45.62 68.90 61.84 65.18 58.57
Opinion Finder 57.63 67.57 27.94 39.53 40.75 33.69 36.89 58.20 86.06 69.44 48.62
Opinion Lexicon 60.37 62.09 62.71 62.40 41.19 52.81 46.28 66.41 60.75 63.46 57.38
PANAS-t 53.08 90.95 9.04 16.45 51.56 3.94 7.33 51.65 99.01 67.89 30.55
Pattern.en 57.99 57.97 68.74 62.89 34.83 35.24 35.04 65.55 56.39 60.63 52.85
SASA 50.63 46.34 47.77 47.04 33.07 20.31 25.17 56.39 61.12 58.66 43.62
Semantria 61.54 67.28 57.35 61.92 39.57 52.81 45.24 65.98 67.03 66.50 57.89
SenticNet 49.68 51.85 1.26 2.46 29.79 1.67 3.17 49.82 98.51 66.17 23.93
Sentiment140 60.42 63.87 51.37 56.94 50.96 37.87 43.45 60.35 73.31 66.20 55.53
Sentiment140_L 39.44 43.52 74.72 55.00 27.67 65.35 38.88 65.87 6.38 11.63 35.17
SentiStrength 57.83 78.01 27.13 40.25 47.80 23.42 31.44 55.49 89.89 68.62 46.77
SentiWordNet 48.33 55.54 53.44 54.47 19.67 37.51 25.81 61.22 47.57 53.54 44.61
SO-CAL 58.83 58.89 59.02 58.95 40.39 54.24 46.30 39.89 59.96 47.91 51.05
Stanford DM 22.54 72.14 18.17 29.03 14.92 90.56 25.61 47.19 6.94 12.10 22.25
Umigon 65.88 75.18 56.14 64.28 39.66 55.91 46.41 70.65 75.78 73.13 61.27
USent 52.13 49.86 32.88 39.63 39.96 22.82 29.05 54.33 74.36 62.79 43.82
VADER 60.21 56.46 79.04 65.87 44.30 59.02 50.61 76.02 46.71 57.87 58.12
Tweets_RND_III AFINN 64.41 40.81 72.12 52.13 49.67 62.50 55.35 85.95 62.54 72.40 59.96
ANEW_SUB 28.03 21.89 92.29 35.38 44.30 34.22 38.61 74.82 8.18 14.74 29.58
Emolex 54.76 31.67 59.95 41.44 40.14 47.54 43.53 77.48 54.64 64.08 49.68
Emoticons 70.22 70.06 16.78 27.07 65.62 8.61 15.22 41.29 97.56 58.02 33.44
Emoticons DS 20.34 19.78 99.46 33.00 62.07 3.69 6.96 53.85 0.55 1.09 13.68
NRC Hashtag 30.47 28.25 77.40 41.39 24.18 72.54 36.27 79.08 8.77 15.78 31.15
LIWC 46.88 21.85 38.43 27.86 19.18 18.24 18.70 69.51 54.83 61.31 35.95
LIWC15 67.75 44.78 78.35 56.99 57.49 57.38 57.44 85.18 66.67 74.80 63.07
Opinion Finder 71.55 57.48 32.75 41.72 49.85 34.63 40.87 75.95 89.90 82.34 54.98
Opinion Lexicon 63.86 40.65 66.17 50.36 48.84 56.15 52.24 81.96 64.66 72.29 58.30
PANAS-t 68.79 79.49 8.39 15.18 48.57 3.48 6.50 68.75 98.86 81.10 34.26
Pattern.en 59.56 36.20 77.00 49.24 52.87 45.29 48.79 81.75 57.23 67.33 55.12
SASA 55.37 29.42 54.53 38.22 42.46 47.34 44.77 78.30 57.15 66.08 49.69
Semantria 68.89 48.86 63.73 55.31 49.82 55.53 52.52 82.02 72.96 77.22 61.68
SenticNet 29.97 31.08 74.83 43.92 20.98 73.98 32.68 79.70 8.49 15.35 30.65
Sentiment140 76.40 64.42 51.69 57.36 74.75 45.49 56.56 79.04 89.50 83.94 65.95
Sentiment140_L 31.32 25.83 77.13 38.70 30.05 78.69 43.49 79.37 8.92 16.04 32.74
SentiStrength 73.80 70.94 41.95 52.72 57.53 25.82 35.64 75.35 92.26 82.95 57.10
SentiWordNet 55.85 37.42 58.19 45.55 24.04 35.86 28.78 79.25 59.00 67.64 47.33
SO-CAL 66.51 43.06 68.88 52.99 51.84 60.66 55.90 45.77 66.94 54.37 54.42
Stanford DM 31.90 64.48 38.57 48.26 15.58 85.04 26.33 75.64 19.77 31.35 35.32
Umigon 74.12 57.67 70.23 63.33 48.83 68.44 57.00 88.80 76.34 82.10 67.47
USent 66.06 40.60 36.81 38.61 44.87 28.69 35.00 74.54 81.72 77.97 50.53
VADER 60.14 37.69 81.60 51.56 48.56 65.57 55.80 88.96 52.87 66.32 57.89
Comments_BBC AFINN 50.10 16.22 60.61 25.59 82.62 56.05 66.79 40.11 30.24 34.48 42.29
ANEW_SUB 24.30 11.38 91.92 20.24 84.15 21.13 33.78 38.89 5.65 9.86 21.30
Emolex 44.10 15.51 65.66 25.10 83.19 45.48 58.81 35.27 31.85 33.47 39.13
Emoticons 24.60 0.00 0.00 0.00 33.33 0.15 0.30 19.77 98.79 32.95 11.09
Emoticons DS 10.00 9.85 98.99 17.92 66.67 0.31 0.61 0.00 0.00 0.00 6.18
NRC Hashtag 64.00 20.72 23.23 21.90 70.20 91.27 79.36 52.50 8.47 14.58 38.62
LIWC07 33.00 11.11 42.42 17.61 67.69 33.69 44.99 22.90 27.42 24.95 29.18
LIWC15 43.70 17.94 68.69 28.45 85.06 42.73 56.88 30.72 36.29 33.27 39.53
Opinion Finder 51.80 14.96 35.35 21.02 78.76 60.18 68.23 33.71 36.29 34.95 41.40
Opinion Lexicon 55.00 20.67 62.63 31.08 85.27 59.42 70.04 40.82 40.32 40.57 47.23
PANAS-t 27.10 16.67 6.06 8.89 75.61 4.75 8.93 25.35 94.35 39.97 19.26
Pattern.en 28.70 14.25 58.59 22.92 82.61 17.46 28.82 25.27 46.37 32.72 28.16
SASA 38.20 17.03 47.47 25.07 70.75 36.29 47.98 25.19 39.52 30.77 34.60
Semantria 56.00 28.90 50.51 36.76 83.82 57.12 67.94 35.86 55.24 43.49 49.40
SenticNet 47.10 17.74 66.67 28.03 72.87 57.58 64.33 25.89 11.69 16.11 36.16
Sentiment140 40.00 17.75 30.30 22.39 79.77 31.39 45.05 28.75 66.53 40.15 35.86
Sentiment140_L 43.10 13.32 65.66 22.15 73.84 53.60 62.11 42.11 6.45 11.19 31.82
SentiStrength 44.20 47.37 18.18 26.28 86.64 32.77 47.56 29.37 84.68 43.61 39.15
SentiWordNet 42.40 14.90 59.60 23.84 81.63 41.50 55.03 34.56 37.90 36.15 38.34
SO-CAL 55.50 20.88 57.58 30.65 80.47 63.09 70.73 28.57 34.68 31.33 44.23
Stanford DM 65.50 43.37 36.36 39.56 71.01 89.28 79.10 37.50 14.52 20.93 46.53
Umigon 45.70 28.35 36.36 31.86 76.35 41.04 53.39 29.31 61.69 39.74 41.66
USent 33.80 13.75 33.33 19.47 82.25 21.29 33.82 28.09 66.94 39.57 30.95
VADER 49.40 16.36 71.72 26.64 83.02 54.67 65.93 48.53 26.61 34.38 42.31
Comments_NYT AFINN 42.45 64.81 41.79 50.81 80.29 39.82 53.24 7.89 77.87 14.32 39.46
ANEW_SUB 51.12 48.35 88.57 62.55 79.65 24.69 37.69 7.92 9.84 8.78 36.34
Emolex 42.97 55.12 53.72 54.41 75.35 33.33 46.22 7.22 54.10 12.74 37.79
Emoticons 4.68 0.00 0.00 0.00 0.00 0.00 0.00 4.47 99.59 8.56 2.85
Emoticons DS 42.58 42.55 99.77 59.66 78.57 0.40 0.80 0.00 0.00 0.00 20.15
NRC Hashtag 54.84 55.38 45.74 50.10 61.55 65.68 63.55 8.33 15.16 10.76 41.47
LIWC07 24.35 42.88 27.72 33.67 53.42 19.07 28.11 4.67 53.28 8.58 23.45
LIWC15 36.49 65.29 40.29 49.83 81.50 29.25 43.05 7.17 83.61 13.20 35.36
Opinion Finder 29.38 68.77 18.78 29.51 76.52 32.68 45.80 6.29 88.11 11.75 29.02
Opinion Lexicon 44.57 65.95 43.15 52.17 79.81 43.11 55.98 7.94 73.77 14.34 40.83
PANAS-t 5.88 69.23 1.23 2.41 62.07 1.31 2.57 4.75 99.18 9.07 4.68
Pattern.en 31.60 55.23 45.05 49.63 72.80 17.76 28.55 5.88 65.57 10.79 29.66
SASA 30.04 49.92 30.13 37.58 59.11 27.21 37.26 5.74 61.07 10.49 28.44
Semantria 44.59 70.60 41.83 52.54 80.54 44.24 57.11 7.53 73.36 13.65 41.10
SenticNet 61.85 58.19 59.48 58.83 65.01 69.26 67.07 0.00 0.00 0.00 41.97
Sentiment140 13.58 77.32 6.81 12.51 75.40 11.96 20.65 4.98 93.03 9.45 14.20
Sentiment140_L 54.61 54.72 59.12 56.84 67.00 54.41 60.05 6.70 15.98 9.44 42.11
SentiStrength 18.17 78.51 8.62 15.54 81.12 18.96 30.74 5.41 95.49 10.24 18.84
SentiWordNet 32.20 57.35 34.53 43.10 70.31 26.95 38.97 6.08 70.08 11.19 31.09
SO-CAL 50.79 64.36 51.13 56.99 77.25 49.16 60.08 8.68 65.98 15.34 44.14
Stanford DM 51.93 73.39 21.14 32.83 59.48 77.90 67.46 9.65 38.11 15.40 38.56
Umigon 24.08 68.76 16.38 26.46 68.78 24.51 36.14 5.88 88.93 11.04 24.54
USent 27.44 56.61 28.95 38.31 77.69 21.59 33.79 5.88 79.51 10.94 27.68
VADER 48.03 62.67 51.63 56.62 79.91 43.07 55.97 9.18 71.31 16.26 42.95