RankBasedEvaluator

class mowl.evaluation.RankBasedEvaluator(class_index_emb, relation_index_emb, testing_set, eval_method, training_set, head_entities, tail_entities, device)

Bases: Evaluator

This class corresponds to evaluation based on ranking. That is, for each testing triple \((h,r,t)\), scores are computed for triples \((h,r,t')\) for all possible \(t'\). After that, the ranking of the testing triple \((h,r,t)\) score is obtained.

Parameters:

• class_index_emb (dict(str, np.array)) – dictionary of classes and their embeddings

• relation_index_emb (dict(str, np.array)) – dictionary of relations and their embeddings

• testing_set (list(mowl.projection.edge.Edge)) – Set of triples that are true positives.

• eval_method (function) – evaluation method score the triples

• training_set (list(mowl.projection.edge.Edge)) – Set of triples that are true positives but exist in the training set. This is used to compute filtered metrics.

• head_entities (list(str)) – List of entities that are used as head entities in the testing set.

• tail_entities (list(str)) – List of entities that are used as tail entities in the testing set.

• device (str) – Use cpu or cuda

Methods Summary

evaluate([activation, show])
filter_head_tail_data()
load_training_scores()

Methods Documentation

evaluate(activation=None, show=False)

filter_head_tail_data()

load_training_scores()