nordlys.core.eval.trec_qrels module

Trec Qrels

Utility module for working with TREC qrels files.

Usage

Get statistics about a qrels file
trec_qrels <qrels_file> -o stat
Filter qrels to contain only documents from a given set
trec_qrels <qrels_file> -o filter_docs -d <doc_ids_file> -f <output_file>
Filter qrels to contain only queries from a given set
trec_qrels <qrels_file> -o filter_qs -q <query_ids_file> -f <output_file>
Author:Krisztian Balog
class nordlys.core.eval.trec_qrels.TrecQrels(file_name=None)[source]

Bases: object

Represents relevance judments (TREC qrels).

filter_by_doc_ids(doc_ids_file, output_file)[source]

Filters qrels for a set of selected docIDs and outputs the results to a file.

Parameters:
  • doc_ids_file – File with one docID per line
  • output_file – Output file name
filter_by_query_ids(query_ids_file, output_file)[source]

Filters qrels for a set of selected queryIDs and outputs the results to a file.

Parameters:
  • query_ids_file – File with one queryID per line
  • output_file – Output file name
get_queries()[source]

Returns the set of queries.

get_rel(query_id)[source]

Returns relevance level for a given query.

Parameters:query_id – queryID
Returns:dict (docID as key and relevance as value) or None
load(file_name)[source]

Loads qrels from file.

Parameters:file_name – name of qrels file
num_rel(query_id, min_rel=1)[source]

Returns the number of relevant results for a given query.

Parameters:
  • query_id – queryID
  • min_rel – minimum relevance level
Returns:

number of relevant results
print_stat()[source]

Prints simple statistics.

nordlys.core.eval.trec_qrels.arg_parser()[source]
nordlys.core.eval.trec_qrels.main(args)[source]