Evaluate

Metrics

relax.evaluate.BaseEvalMetrics

[source]

class relax.evaluate.BaseEvalMetrics (name=None)

Base evaluation metrics class.

relax.evaluate.PredictiveAccuracy

[source]

class relax.evaluate.PredictiveAccuracy (name=‘accuracy’)

Compute the accuracy of the predict function.

acc = PredictiveAccuracy()
exp = fake_explanation(3)
acc(exp)
Array(0.98300004, dtype=float32)

relax.evaluate.compute_validity

[source]

relax.evaluate.compute_validity (xs, cfs, pred_fn)

Parameters:

  • xs (<class 'jax.Array'>) – (n, d)
  • cfs (<class 'jax.Array'>) – (n, d) or (n, b, d)
  • pred_fn (typing.Callable[[jax.Array], jax.Array])

Returns:

    (<class 'float'>)

relax.evaluate.compute_single_validity

[source]

relax.evaluate.compute_single_validity (xs, cfs, pred_fn)

Parameters:

  • xs (<class 'jax.Array'>) – (n, d)
  • cfs (<class 'jax.Array'>) – (n, d)
  • pred_fn (typing.Callable[[jax.Array], jax.Array])

relax.evaluate.Validity

[source]

class relax.evaluate.Validity (name=‘validity’)

Compute fraction of input instances on which CF explanation methods output valid CF examples. Support binary case only.

val = Validity()
assert val(exp) == 0.

relax.evaluate.compute_proximity

[source]

relax.evaluate.compute_proximity (xs, cfs)

relax.evaluate.compute_single_proximity

[source]

relax.evaluate.compute_single_proximity (xs, cfs)

assert jnp.isclose(
    compute_proximity(xs, cfs, ), 0.
)
assert jnp.isclose(
    compute_proximity(xs, cfs[:, 0, :], ), 0.
)

relax.evaluate.Proximity

[source]

class relax.evaluate.Proximity (name=‘proximity’)

Compute L1 norm distance between input datasets and CF examples divided by the number of features.

prox = Proximity()
assert prox(exp) == 0.

relax.evaluate.compute_sparsity

[source]

relax.evaluate.compute_sparsity (xs, cfs, feature_indices)

relax.evaluate.compute_single_sparsity

[source]

relax.evaluate.compute_single_sparsity (xs, cfs, feature_indices)

relax.evaluate.Sparsity

[source]

class relax.evaluate.Sparsity (name=‘sparsity’)

Compute the number of feature changes between input datasets and CF examples.

spar = Sparsity()
assert spar(exp) == 0.

relax.evaluate.ManifoldDist

[source]

class relax.evaluate.ManifoldDist (n_neighbors=1, name=‘manifold_dist’)

Compute the L1 distance to the n-nearest neighbor for all CF examples.

man = ManifoldDist()
man(exp)
Array(6.905339e-07, dtype=float32)

relax.evaluate.Runtime

[source]

class relax.evaluate.Runtime (name=‘runtime’)

Compute the runtime of the CF explanation method.

run = Runtime()
run(exp)
0.0

Benchmarking

relax.evaluate.evaluate_cfs

[source]

relax.evaluate.evaluate_cfs (cf_exp, metrics=None, return_dict=True, return_df=False)

Parameters:

  • cf_exp (<class 'relax.explain.Explanation'>) – CF Explanations
  • metrics (typing.Iterable[typing.Union[str, __main__.BaseEvalMetrics]], default=None) – A list of Metrics. Can be str or a subclass of BaseEvalMetrics
  • return_dict (<class 'bool'>, default=True) – return a dictionary or not (default: True)
  • return_df (<class 'bool'>, default=False) – return a pandas Dataframe or not (default: False)

relax.evaluate.benchmark_cfs

[source]

relax.evaluate.benchmark_cfs (cf_results_list, metrics=None)