Skip to main content

Table 6 2-classes experiments results with 4 datasets

From: SentiBench - a benchmark comparison of state-of-the-practice sentiment analysis methods

Dataset

Method

Accur.

Posit. sentiment

Negat. sentiment

Macro- F 1

Coverage

P

R

F 1

P

R

F 1

Tweets_RND_II

AFINN

96.37

97.66

96.94

97.30

93.75

95.19

94.47

95.88

80.77

ANEW_SUB

81.36

80.52

96.38

87.74

85.44

47.64

61.17

74.45

93.35

Emolex

86.06

89.82

89.11

89.47

78.77

80.00

79.38

84.42

63.58

Emoticons

97.75

97.90

99.42

98.65

96.97

89.72

93.20

95.93

14.82

Emoticons DS

71.04

70.61

99.90

82.74

95.83

5.43

10.28

46.51

99.09

NRC Hashtag

67.37

83.76

65.43

73.47

48.17

71.69

57.62

65.55

91.94

LIWC07

66.47

74.46

78.81

76.58

44.20

38.31

41.04

58.81

73.93

LIWC15

96.44

97.09

98.04

97.56

94.68

92.23

93.44

95.50

77.05

Opinion Finder

78.32

93.86

71.11

80.92

63.42

91.50

74.92

77.92

41.23

Opinion Lexicon

93.45

97.03

93.14

95.04

86.93

94.11

90.38

92.71

70.64

PANAS-t

90.71

96.95

88.19

92.36

82.11

95.12

88.14

90.25

5.39

Pattern.en

91.76

92.94

96.19

94.54

87.86

79.06

83.23

88.88

70.85

SASA

70.06

82.81

72.81

77.49

49.05

63.39

55.30

66.40

63.04

Semantria

91.61

96.94

90.55

93.64

82.25

93.88

87.68

90.66

63.61

SenticNet

73.64

90.74

68.45

78.03

55.41

84.88

67.05

72.54

82.82

Sentiment140

94.75

97.10

95.71

96.40

88.64

92.13

90.35

93.37

49.95

Sentiment140_L

78.05

88.68

78.31

83.17

61.32

77.47

68.45

75.81

93.28

SentiStrength

96.97

98.92

96.43

97.66

93.54

98.01

95.72

96.69

34.65

SentiWordNet

78.57

87.88

80.91

84.25

61.09

72.87

66.46

75.36

61.49

SO-CAL

87.76

94.25

86.99

90.47

77.34

89.32

82.90

86.68

67.18

Stanford DM

60.46

94.48

44.87

60.84

44.06

94.30

60.06

60.45

88.89

Umigon

88.63

97.73

85.92

91.45

73.64

95.17

83.03

87.24

70.83

USent

84.46

89.28

87.67

88.47

74.77

77.63

76.17

82.32

38.94

VADER

99.04

99.16

99.45

99.31

98.77

98.12

98.45

98.88

94.40

Tweets_STF

AFINN

84.42

80.62

91.49

85.71

89.66

77.04

82.87

84.29

76.88

ANEW_SUB

68.05

63.08

93.18

75.23

84.62

40.74

55.00

65.11

94.15

Emolex

79.65

76.09

88.98

82.03

85.23

69.44

76.53

79.28

62.95

Emoticons

85.42

80.65

96.15

87.72

94.12

72.73

82.05

84.89

13.37

Emoticons DS

51.96

51.41

100.00

67.91

100.00

2.27

4.44

36.18

99.72

NRC Hashtag

71.30

73.05

70.93

71.98

69.51

71.70

70.59

71.28

92.20

LIWC07

64.29

63.75

76.12

69.39

65.22

50.85

57.14

63.27

70.39

LIWC15

89.22

84.18

97.08

90.17

96.40

81.06

88.07

89.12

74.93

Opinion Finder

80.77

81.16

76.71

78.87

80.46

84.34

82.35

80.61

43.45

Opinion Lexicon

86.10

83.67

91.11

87.23

89.29

80.65

84.75

85.99

72.14

PANAS-t

94.12

88.89

100.00

94.12

100.00

88.89

94.12

94.12

4.74

Pattern.en

79.55

74.86

94.48

83.54

90.12

61.34

73.00

78.27

73.54

SASA

68.52

65.65

78.90

71.67

72.94

57.94

64.58

68.12

60.17

Semantria

88.45

89.15

88.46

88.80

87.70

88.43

88.07

88.43

69.92

SenticNet

70.49

71.31

63.50

67.18

69.88

76.82

73.19

70.18

80.22

Sentiment140

93.29

91.36

94.87

93.08

95.18

91.86

93.49

93.29

45.68

Sentiment140_L

79.12

81.48

76.30

78.81

76.97

82.04

79.42

79.11

94.71

SentiStrength

95.33

95.18

96.34

95.76

95.52

94.12

94.81

95.29

41.78

SentiWordNet

72.99

73.17

78.95

75.95

72.73

65.98

69.19

72.57

58.77

SO-CAL

87.36

82.89

93.33

87.80

92.80

81.69

86.89

87.35

77.16

Stanford DM

66.56

87.69

36.31

51.35

61.24

95.18

74.53

62.94

89.97

Umigon

86.99

91.73

81.88

86.52

83.02

92.31

87.42

86.97

81.34

USent

73.21

69.35

82.69

75.44

78.82

63.81

70.53

72.98

58.22

VADER

84.44

80.23

92.21

85.80

90.40

76.35

82.78

84.29

84.12

Comments_Digg

AFINN

70.94

47.01

81.82

59.72

91.17

67.05

77.27

68.49

74.81

ANEW_SUB

43.25

30.98

92.31

46.39

90.13

25.46

39.71

43.05

93.73

Emolex

61.71

34.60

75.83

47.52

88.93

57.53

69.87

58.69

67.14

Emoticons

73.08

72.22

86.67

78.79

75.00

54.55

63.16

70.97

3.32

Emoticons DS

28.24

27.30

100.00

42.89

100.00

1.77

3.48

23.19

98.72

NRC Hashtag

74.69

51.01

40.64

45.24

80.80

86.48

83.54

64.39

92.97

LIWC07

46.15

27.44

58.40

37.34

72.49

41.52

52.79

45.07

58.18

LIWC15

70.67

49.81

90.91

64.36

94.35

62.36

75.09

69.72

62.79

Opinion Finder

71.14

43.04

64.76

51.71

86.88

73.13

79.42

65.56

56.27

Opinion Lexicon

71.82

47.45

86.43

61.27

93.40

66.75

77.86

69.56

69.44

PANAS-t

68.00

12.50

50.00

20.00

94.12

69.57

80.00

50.00

3.20

Pattern.en

60.05

43.73

92.14

59.31

92.57

45.21

60.75

60.03

56.65

SASA

65.54

40.26

66.91

50.27

84.82

65.06

73.64

61.95

68.29

Semantria

82.46

62.72

88.33

73.36

94.81

80.25

86.93

80.14

56.14

SenticNet

69.40

46.30

72.46

56.50

86.77

68.25

76.40

66.45

96.55

Sentiment140

85.06

62.50

78.95

69.77

93.65

86.76

90.08

79.92

33.38

Sentiment140_L

67.76

42.07

73.45

53.50

88.01

65.84

75.33

64.41

89.64

SentiStrength

92.09

78.69

92.31

84.96

97.40

92.02

94.64

89.80

27.49

SentiWordNet

62.17

36.86

77.68

50.00

88.84

57.18

69.58

59.79

58.82

SO-CAL

76.55

52.86

77.08

62.71

90.65

76.37

82.90

72.81

71.99

Stanford DM

69.16

35.29

20.27

25.75

75.21

86.68

80.54

53.15

78.90

Umigon

83.37

66.22

75.38

70.50

90.72

86.23

88.42

79.46

63.04

USent

55.98

36.06

80.65

49.83

86.67

46.80

60.78

55.31

43.86

VADER

69.05

45.48

85.88

59.47

92.55

63.00

74.97

67.22

82.23

Comments_BBC

AFINN

66.56

23.08

81.08

35.93

96.32

64.66

77.38

56.65

85.11

ANEW_SUB

31.37

15.48

95.79

26.65

97.18

21.73

35.52

31.08

97.07

Emolex

59.64

21.52

89.04

34.67

97.38

55.62

70.80

52.73

80.72

Emoticons

33.33

0.00

0.00

0.00

100.00

33.33

50.00

25.00

0.40

Emoticons DS

13.33

13.10

100.00

23.17

100.00

0.31

0.61

11.89

99.73

NRC Hashtag

84.45

33.33

25.27

28.75

89.76

92.83

91.27

60.01

97.47

LIWC07

50.10

15.38

58.33

24.35

88.00

48.78

62.77

43.56

69.55

LIWC15

63.21

25.86

90.67

40.24

97.55

58.86

73.42

56.83

73.01

Opinion Finder

74.43

21.74

62.50

32.26

94.93

75.72

84.24

58.25

76.46

Opinion Lexicon

74.14

29.81

84.93

44.13

97.24

72.66

83.17

63.65

80.72

PANAS-t

58.73

20.00

75.00

31.58

93.94

56.36

70.45

51.02

8.38

Pattern.en

41.75

19.73

93.55

32.58

96.61

32.57

48.72

40.65

54.79

SASA

61.61

23.50

66.20

34.69

90.80

60.77

72.81

53.75

61.30

Semantria

83.43

40.00

84.75

54.35

97.64

83.26

89.88

72.11

67.42

SenticNet

66.07

24.44

74.16

36.77

94.24

64.83

76.81

56.79

88.96

Sentiment140

68.51

24.00

69.77

35.71

94.04

68.33

79.15

57.43

45.61

Sentiment140_L

56.85

18.52

69.15

29.21

92.35

55.03

68.97

49.09

97.07

SentiStrength

93.93

64.29

78.26

70.59

97.72

95.54

96.61

83.60

32.85

SentiWordNet

57.49

20.00

88.06

32.60

97.13

53.45

68.96

50.78

76.33

SO-CAL

75.28

28.93

80.28

42.54

96.71

74.64

84.25

63.40

82.85

Stanford DM

89.45

63.16

40.91

49.66

91.81

96.52

94.11

71.88

92.02

Umigon

79.37

39.13

61.02

47.68

92.10

82.72

87.15

67.42

50.93

USent

52.60

18.33

80.49

29.86

94.56

48.60

64.20

47.03

43.48

VADER

62.76

22.68

85.54

35.86

96.75

59.60

73.76

54.81

90.69