Skip to main content

Table 1 Machine classification performance for cyber hate based on disability, race and sexual orientation (results rounded to 2dp)

From: Us and them: identifying cyber hate on Twitter across multiple protected characteristics

  Religion (baseline) Disability Race Sexual orientation
P R F P R F P R F P R F
n-Gram words 1 to 5 with 2,000 features 0.80 FP = 38 0.69 FN = 69 0.74 0.969 FP = 1 0.608 FN = 20 0.73 0.72 FP = 15 0.54 FN = 32 0.62 0.53 FP = 67 0.42 FN = 107 0.47
n-Gram hateful terms 0.89 FP = 19 0.66 FN = 75 0.76 0.00 0.00 0.00 0.93 FP = 3 0.53 FN = 33 0.67 1.00 FP = 0 0.098 FN = 165 0.18
n-Gram words (1-5) with 2,000 features + hateful terms 0.74 FP = 58 0.65 FN = 78 0.69 0.89 FP = 4 0.61 FN = 20 0.72 0.79 FP = 13 0.71 FP = 20 0.75 0.57 FP = 60 0.44 FN = 105 0.49
n-Gram typed dependencies 0.53 FP = 48 0.24 FP = 168 0.33 0.97 FP = 1 0.61 FP = 20 0.75 0.87 FP = 3 0.29 FN = 50 0.43 0.95 FP = 2 0.22 FN = 142 0.36
n-Gram typed dependencies+hateful terms 0.89 FP = 19 0.69 FN = 70 0.77 0.97 FP = 1 0.61 FP = 20 0.75 0.91 FP = 4 0.59 FN = 29 0.71 0.96 FP = 2 0.27 FN = 134 0.42
n-Gram words (1-5) with 2,000 features + n-Gram typed dependencies+hateful terms 0.89 FP = 19 0.69 FN = 70 0.77 0.97 FP = 1 0.61 FN = 20 0.75 0.87 FP = 7 0.66 FN = 24 0.75 0.72 FP = 25 0.35 FN = 119 0.47