AxiomsRankBasedEvaluator

class mowl.evaluation.AxiomsRankBasedEvaluator(eval_method, axioms_to_filter=None, device='cpu')

Bases: object

Abstract method for evaluating axioms in a rank-based manner. To inherit from this class, 3 methods must be defined (dee the corresponding docstrings for each of them).

Parameters:

• eval_method (function) – The evaluation method for the axioms.

• axioms_to_filter (any, optional) – Axioms to be put at the bottom of the rankings. If the axioms are empty, filtered metrics will not be computed. The input type of this parameter will depend on the signature of the _init_axioms_to_filter method. Defaults to None.

• device (str, optional) – Device to run the evaluation. Defaults to “cpu”.

Attributes Summary

fmetrics	Filtered metrics as a dictionary with string metric names as keys and metrics as values.
metrics	Metrics as a dictionary with string metric names as keys and metrics as values.

Methods Summary

__call__(axioms)	Call self as a function.
compute_axiom_rank(axiom)	This function should compute the rank of a single axiom.
print_metrics()

Attributes Documentation

fmetrics

Filtered metrics as a dictionary with string metric names as keys and metrics as values.

Return type:

dict

metrics

Metrics as a dictionary with string metric names as keys and metrics as values.

Return type:

dict

Methods Documentation

__call__(axioms)

Call self as a function.

compute_axiom_rank(axiom)

This function should compute the rank of a single axiom. This method will be used iteratively by the __call__ method. This method returns a 3-tuple: rank of the axiom, frank of the axiom and the possible achievable worst rank.

Parameters:

axiom – Axiom of the type congruent to the eval_method signature.

Return type:

(int, int, int)

print_metrics()