Table 9 Influence of T5 generated claims. The highest scores for each dataset and metric are in bold
From: UTDRM: unsupervised method for training debunked-narrative retrieval models
Datasets | Metrics | T5 Generated Claims | |||
|---|---|---|---|---|---|
N = 1 | N = 2 | N = 6 | N = 10 | ||
Snopes | MAP@1 | 0.811 | 0.821 | 0.846 | 0.851 |
MRR | 0.870 | 0.879 | 0.898 | 0.900 | |
CLEF 22 2A | MAP@1 | 0.900 | 0.909 | 0.928 | 0.928 |
MRR | 0.923 | 0.930 | 0.946 | 0.945 | |
CLEF 21 2A | MAP@1 | 0.886 | 0.901 | 0.916 | 0.906 |
MRR | 0.923 | 0.933 | 0.938 | 0.937 | |
CLEF 20 2A | MAP@1 | 0.935 | 0.935 | 0.935 | 0.935 |
MRR | 0.954 | 0.955 | 0.957 | 0.955 | |
Average | MAP@1 | 0.883 | 0.891 | 0.906 | 0.905 |
Twitter-based | MRR | 0.917 | 0.924 | 0.935 | 0.934 |
Politifact | MAP@1 | 0.484 | 0.516 | 0.516 | 0.500 |
MRR | 0.598 | 0.628 | 0.637 | 0.627 | |
CLEF 22 2B | MAP@1 | 0.392 | 0.408 | 0.415 | 0.415 |
MRR | 0.451 | 0.473 | 0.487 | 0.490 | |
CLEF 21 2B | MAP@1 | 0.348 | 0.361 | 0.354 | 0.367 |
MRR | 0.403 | 0.420 | 0.431 | 0.439 | |
Average | MAP@1 | 0.408 | 0.428 | 0.428 | 0.427 |
Political-based | MRR | 0.484 | 0.507 | 0.518 | 0.519 |