nordlys.core.eval.query_diff module

Query Differences

Computes query-level differences between two runs.

Authors:Shuo Zhang, Krisztian Balog, Dario Garigliotti
class nordlys.core.eval.query_diff.QueryDiff(run1_file, run2_file, qrels, metric)[source]

Bases: object

Parameters:
  • run1_file – name of run1 file (baseline)
  • run2_file – name of run2 file (new method)
  • qrels – name of qrels file
  • metric – metric
Returns:
dump_differences(output_file)[source]

Outputs query-level differences between two methods into a tab-separated file.

The first method is considered the baseline, the differences are with respect to that. Output format: queryID res1 res2 diff(res2-res1)