The scorer currently evaluates TA1 the same as TA2, using the same denominator for recall (GT) regardless of whether the run was submitted to task 1 (single-document) or task 2 (cross-document). A future release of the scores for task 1 submissions will restrict GT for each QID to be only the mentions in the query document.
The scorer currently evaluates TA1 the same as TA2, using the same denominator for recall (GT) regardless of whether the run was submitted to task 1 (single-document) or task 2 (cross-document). A future release of the scores for task 1 submissions will restrict GT for each QID to be only the mentions in the query document.