qxmt.evaluation.evaluation module#

class qxmt.evaluation.evaluation.ClassificationEvaluation(actual, predicted, default_metrics_name=None, custom_metrics=None)

Bases: Evaluation

Parameters:

actual (ndarray)
predicted (ndarray)
default_metrics_name (list[str] | None)
custom_metrics (list[dict[str, Any]] | None)

DEFAULT_METRICS_NAME: list[str] = ['accuracy', 'precision', 'recall', 'f1_score']

NAME2METRIC: dict[str, Type[BaseMetric]] = {'accuracy': <class 'qxmt.evaluation.metrics.defaults_classification.Accuracy'>, 'f1_score': <class 'qxmt.evaluation.metrics.defaults_classification.F1Score'>, 'precision': <class 'qxmt.evaluation.metrics.defaults_classification.Precision'>, 'recall': <class 'qxmt.evaluation.metrics.defaults_classification.Recall'>}

class qxmt.evaluation.evaluation.Evaluation(actual, predicted, default_metrics_name=None, custom_metrics=None)

Bases: object

Evaluation class for model evaluation. This class is used to evaluate the model performance by comparing the actual and predicted values. Evaluation metrics are defined by default and custom metrics. Furthermore, the result can be accessed as a dictionary or DataFrame.

Examples

>>> from qxmt.evaluation.evaluation import Evaluation
>>> actual = np.array([1, 0, 1])
>>> predicted = np.array([1, 1, 1])
>>> evaluation = Evaluation(actual, predicted)
>>> evaluation.evaluate()
>>> evaluation.to_dict()
{'accuracy': 0.6666666666666666, 'precision': 0.6666666666666666, 'recall': 1.0, 'f1_score': 0.8}
>>> evaluation.to_dataframe()
   accuracy  precision  recall  f1_score
0  0.666667   0.666667     1.0       0.8

Parameters:

actual (ndarray)
predicted (ndarray)
default_metrics_name (list[str] | None)
custom_metrics (list[dict[str, Any]] | None)

DEFAULT_METRICS_NAME: list[str] = []

NAME2METRIC: dict[str, Type[BaseMetric]] = {}

__init__(actual, predicted, default_metrics_name=None, custom_metrics=None)

Initialize the evaluation class.

Parameters:

actual (np.ndarray) – numpy array of actual values
predicted (np.ndarray) – numpy array of predicted values
default_metrics_name (Optional[list[str]], optional) – metrics name list defined by default. Defaults to None.
custom_metrics (Optional[list[dict[str, Any]]], optional) – metrics name list defined by user custom. Defaults to None.

Return type:

None

evaluate()

Evaluate default and custom metrics.

Return type:: None

init_custom_metrics(custom_metrics)

Initialize and validate custom metrics.

Parameters:: custom_metrics (Optional[list[dict[str, Any]]]) – list of custom metrics. Defaults to None.
Raises:: ValueError – if the metric is not subclass of BaseMetric
Return type:: None

init_default_metrics(default_metrics_name)

Initialize and validate default metrics.

Raises:: ValueError – if the metric is not implemented
Parameters:: default_metrics_name (list[str] | None)
Return type:: None

set_evaluation_result(metrics)

Evaluate default metrics.

Raises:: ValueError – if the metric is not implemented
Parameters:: metrics (list[BaseMetric])
Return type:: None

to_dataframe(id=None, id_columns_name=None)

Convert evaluation metrics to DataFrame.

Parameters:

id (Optional[str], optional) – id of the evaluation (ex: run_id). Defaults to None.
id_columns_name (Optional[str], optional) – name of the id column. Defaults to None.

Returns:

DataFrame of evaluation metrics

Return type:

pd.DataFrame

to_dict()

Convert evaluation metrics to dictionary.

Raises:: ValueError – if the metrics are not evaluated yet
Returns:: dictionary of evaluation metrics
Return type:: dict

class qxmt.evaluation.evaluation.RegressionEvaluation(actual, predicted, default_metrics_name=None, custom_metrics=None)

Bases: Evaluation

Parameters:

actual (ndarray)
predicted (ndarray)
default_metrics_name (list[str])
custom_metrics (list[BaseMetric])

DEFAULT_METRICS_NAME: list[str] = ['mean_absolute_error', 'root_mean_squared_error', 'r2_score']

NAME2METRIC: dict[str, Type[BaseMetric]] = {'mean_absolute_error': <class 'qxmt.evaluation.metrics.defaults_regression.MeanAbsoluteError'>, 'r2_score': <class 'qxmt.evaluation.metrics.defaults_regression.R2Score'>, 'root_mean_squared_error': <class 'qxmt.evaluation.metrics.defaults_regression.RootMeanSquaredError'>}