qxmt.evaluation.metrics package

qxmt.evaluation.metrics package#

Submodules#

Module contents#

class qxmt.evaluation.metrics.Accuracy(name='accuracy')#

Bases: BaseMetric

Accuracy metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import Accuracy
>>> metric = Accuracy()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
accuracy: 0.67
__init__(name='accuracy')#

Initialize the accuracy metric.

Parameters:

name (str, optional) – name of accuracy metric. Defaults to “accuracy”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the accuracy score.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • kwargs (Any)

Returns:

accuracy score

Return type:

float

class qxmt.evaluation.metrics.BaseMetric(name)#

Bases: ABC

Base class for evaluation metric. This class is used to define the evaluation metric for the model and visualization. Provide a common interface within the QXMT library by absorbing differences between metrics.

Examples

>>> import numpy as np
>>> from qxmt.evaluation.base import BaseMetric
>>> class CustomMetric(BaseMetric):
...     def __init__(self, name: str) -> None:
...         super().__init__(name)
...
...     @staticmethod
...     def evaluate(actual: np.ndarray, predicted: np.ndarray, **kwargs: Any) -> float:
...         return np.mean(np.abs(actual - predicted))
...
>>> metric = CustomMetric("mean_absolute_error")
>>> metric.set_score(np.array([1, 3, 3]), np.array([1, 2, 3]))
>>> metric.output_score()
mean_absolute_error: 0.33
Parameters:

name (str)

__init__(name)#

Base class for evaluation metric.

Parameters:

name (str) – name of the metric. It is used for the column name of the output and DataFrame.

Return type:

None

abstract static evaluate(actual, predicted, **kwargs)#

define evaluation method for each metric.

Parameters:

  • actual (np.ndarray) – array of actual value

  • predicted (np.ndarray) – array of predicted value

  • **kwargs (dict) – additional arguments

Returns:

evaluated score

Return type:

float

output_score(logger=<Logger qxmt.evaluation.metrics.base (INFO)>)#

Output the evaluated score on standard output.

Parameters:

logger (Logger, optional) – logger object. Defaults to LOGGER.

Raises:

ValueError – if the score is not evaluated yet

Return type:

None

set_score(actual, predicted, **kwargs)#

Evaluated the score and set it to the score attribute.

Parameters:

  • actual (np.ndarray) – array of actual value

  • predicted (np.ndarray) – array of predicted value

  • **kwargs (dict) – additional arguments

Return type:

None

class qxmt.evaluation.metrics.F1Score(name='f1_score')#

Bases: BaseMetric

F1 score metric

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import F1Score
>>> metric = F1Score()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
f1_score: 0.8
__init__(name='f1_score')#

Initialize the F1 score metric.

Parameters:

name (str, optional) – name of f1-score metric. Defaults to “f1_score”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the F1 score.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • **kwargs (dict) – additional keyword arguments. The following options are supported: - average (str): define averaging method - pos_label (int): positive label for binary classification (default: max value)

Returns:

F1 score

Return type:

float

class qxmt.evaluation.metrics.MeanAbsoluteError(name='mean_absolute_error')#

Bases: BaseMetric

Mean absolute error metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import MeanAbsoluteError
>>> metric = MeanAbsoluteError()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
mean_absolute_error: 0.33
__init__(name='mean_absolute_error')#

Initialize the mean absolute error metric.

Parameters:

name (str, optional) – name of mean absolute error metric. Defaults to “mean_absolute_error”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the mean absolute error.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • kwargs (Any)

Returns:

mean absolute error score

Return type:

float

class qxmt.evaluation.metrics.Precision(name='precision')#

Bases: BaseMetric

Precision metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import Precision
>>> metric = Precision()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
precision: 0.67
__init__(name='precision')#

Initialize the precision metric.

Parameters:

name (str, optional) – name of precision metric. Defaults to “precision”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the precision score.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • **kwargs (dict) – additional keyword arguments. The following options are supported: - average (str): define averaging method - pos_label (int): positive label for binary classification (default: max value)

Returns:

precision score

Return type:

float

class qxmt.evaluation.metrics.R2Score(name='r2_score')#

Bases: BaseMetric

R2 score metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import R2Score
>>> metric = R2Score()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
r2_score: -0.5
__init__(name='r2_score')#

Initialize the R2 score metric.

Parameters:

name (str, optional) – name of R2 score metric. Defaults to “r2_score”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the R2 score.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • kwargs (Any)

Returns:

R2 score

Return type:

float

class qxmt.evaluation.metrics.Recall(name='recall')#

Bases: BaseMetric

Recall metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import Recall
>>> metric = Recall()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
recall: 1.0
__init__(name='recall')#

Initialize the recall metric.

Parameters:

name (str, optional) – name of recall metric. Defaults to “recall”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the recall score.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • **kwargs (dict) – additional keyword arguments. The following options are supported: - average (str): define averaging method - pos_label (int): positive label for binary classification (default: max value)

Returns:

recall score

Return type:

float

class qxmt.evaluation.metrics.RootMeanSquaredError(name='root_mean_squared_error')#

Bases: BaseMetric

Root mean squared error metric class.

Parameters:

  • BaseMetric (_type_) – Base metric class

  • name (str)

Examples

>>> import numpy as np
>>> from qxmt.evaluation.defaults import RootMeanSquaredError
>>> metric = RootMeanSquaredError()
>>> metric.set_score(np.array([1, 0, 1]), np.array([1, 1, 1]))
>>> metric.output_score()
root_mean_squared_error: 0.58
__init__(name='root_mean_squared_error')#

Initialize the root mean squared error metric.

Parameters:

name (str, optional) – name of root mean squared error metric. Defaults to “root_mean_squared_error”.

Return type:

None

static evaluate(actual, predicted, **kwargs)#

Evaluate the root mean squared error.

Parameters:

  • actual (np.ndarray) – numpy array of actual values

  • predicted (np.ndarray) – numpy array of predicted values

  • kwargs (Any)

Returns:

root mean squared error score

Return type:

float