Table 1 Machine classification performance for cyber hate based on disability, race and sexual orientation (results rounded to 2dp)
From: Us and them: identifying cyber hate on Twitter across multiple protected characteristics
Religion (baseline) | Disability | Race | Sexual orientation | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
P | R | F | P | R | F | P | R | F | P | R | F | |
n-Gram words 1 to 5 with 2,000 features | 0.80 FP = 38 | 0.69 FN = 69 | 0.74 | 0.969 FP = 1 | 0.608 FN = 20 | 0.73 | 0.72 FP = 15 | 0.54 FN = 32 | 0.62 | 0.53 FP = 67 | 0.42 FN = 107 | 0.47 |
n-Gram hateful terms | 0.89 FP = 19 | 0.66 FN = 75 | 0.76 | 0.00 | 0.00 | 0.00 | 0.93 FP = 3 | 0.53 FN = 33 | 0.67 | 1.00 FP = 0 | 0.098 FN = 165 | 0.18 |
n-Gram words (1-5) with 2,000 features + hateful terms | 0.74 FP = 58 | 0.65 FN = 78 | 0.69 | 0.89 FP = 4 | 0.61 FN = 20 | 0.72 | 0.79 FP = 13 | 0.71 FP = 20 | 0.75 | 0.57 FP = 60 | 0.44 FN = 105 | 0.49 |
n-Gram typed dependencies | 0.53 FP = 48 | 0.24 FP = 168 | 0.33 | 0.97 FP = 1 | 0.61 FP = 20 | 0.75 | 0.87 FP = 3 | 0.29 FN = 50 | 0.43 | 0.95 FP = 2 | 0.22 FN = 142 | 0.36 |
n-Gram typed dependencies+hateful terms | 0.89 FP = 19 | 0.69 FN = 70 | 0.77 | 0.97 FP = 1 | 0.61 FP = 20 | 0.75 | 0.91 FP = 4 | 0.59 FN = 29 | 0.71 | 0.96 FP = 2 | 0.27 FN = 134 | 0.42 |
n-Gram words (1-5) with 2,000 features + n-Gram typed dependencies+hateful terms | 0.89 FP = 19 | 0.69 FN = 70 | 0.77 | 0.97 FP = 1 | 0.61 FN = 20 | 0.75 | 0.87 FP = 7 | 0.66 FN = 24 | 0.75 | 0.72 FP = 25 | 0.35 FN = 119 | 0.47 |