Table 8 Influence of ChatGPT generated claims. The highest scores for each dataset and metric are in bold
From: UTDRM: unsupervised method for training debunked-narrative retrieval models
Datasets | Metrics | ChatGPT Generated Claims | |||
|---|---|---|---|---|---|
N = 1 | N = 2 | N = 6 | N = 10 | ||
Snopes | MAP@1 | 0.811 | 0.826 | 0.813 | 0.811 |
MRR | 0.869 | 0.882 | 0.880 | 0.879 | |
CLEF 22 2A | MAP@1 | 0.904 | 0.909 | 0.919 | 0.923 |
MRR | 0.926 | 0.929 | 0.937 | 0.940 | |
CLEF 21 2A | MAP@1 | 0.876 | 0.901 | 0.901 | 0.896 |
MRR | 0.915 | 0.931 | 0.931 | 0.929 | |
CLEF 20 2A | MAP@1 | 0.940 | 0.935 | 0.945 | 0.945 |
MRR | 0.956 | 0.955 | 0.962 | 0.962 | |
Average | MAP@1 | 0.883 | 0.893 | 0.894 | 0.894 |
Twitter-based | MRR | 0.917 | 0.924 | 0.928 | 0.928 |
Politifact | MAP@1 | 0.461 | 0.477 | 0.512 | 0.496 |
MRR | 0.572 | 0.593 | 0.612 | 0.606 | |
CLEF 22 2B | MAP@1 | 0.346 | 0.346 | 0.377 | 0.385 |
MRR | 0.423 | 0.424 | 0.448 | 0.458 | |
CLEF 21 2B | MAP@1 | 0.310 | 0.310 | 0.335 | 0.342 |
MRR | 0.379 | 0.381 | 0.403 | 0.411 | |
Average | MAP@1 | 0.372 | 0.378 | 0.408 | 0.407 |
Political-based | MRR | 0.458 | 0.466 | 0.488 | 0.492 |