nordlys.core.eval.trec_eval module

Trec Evaluation

Wrapper for trec_eval.

Authors:Dario Garigliotti, Shuo Zhang
class nordlys.core.eval.trec_eval.TrecEval[source]

Bases: object

Holds evaluation results obtained using trec_eval.

evaluate(qrels_file, run_file, eval_file=None)[source]

Evaluates a runfile using trec_eval. Optionally writes evaluation output to file.

Parameters:
  • qrels_file – name of qrels file
  • run_file – name of run file
  • eval_file – name of evaluation output file
get_query_ids()[source]

Returns the set of queryIDs for which we have results.

get_score(query_id, metric)[source]

Returns the score for a given queryID and metric.

Parameters:
  • query_id – queryID
  • metric – metric
Returns:

score (or None if not found)
load_results(eval_file)[source]

Loads results from an existing evaluation file.

Parameters:eval_file – name of evaluation file