qxmt.experiment.experiment module

qxmt.experiment.experiment module#

class qxmt.experiment.experiment.Experiment(name=None, desc=None, auto_gen_mode=False, root_experiment_dirc=PosixPath('/home/runner/work/qxmt/qxmt/experiments'), llm_model_path='microsoft/Phi-3-mini-128k-instruct', logger=<Logger qxmt.experiment.experiment (INFO)>)

Bases: object

Manage the life-cycle of quantum machine-learning experiments.

The Experiment class is the centerpiece of the Quantum eXperiment Management Tool (QXMT). It orchestrates creation and loading of experiment directories, execution of runs, persistence and aggregation of results, and ensures reproducibility. Every side-effect that touches the file-system is delegated to qxmt.experiment.repository.ExperimentRepository, so this class can focus on business logic.

Responsibilities:

  1. Initialization / Loading

    • init() - create a new experiment directory and an empty ExperimentDB.

    • load() - load an existing experiment from a JSON file.

  2. Run management

    • run() - execute an experiment run with QKernel or VQE models (supports both config-based and instance-based workflows).

    • _run_setup() / _run_backfill() - create and rollback run directories.

  3. Result handling

    • runs_to_dataframe() - convert RunRecord`s into a `pandas.DataFrame for easy analysis.

    • save_experiment() - persist the ExperimentDB to disk.

    • get_run_record() - retrieve a single RunRecord by run_id.

  4. Reproducibility

    • reproduce() - re-execute a past run and validate that results match.

Example

from qxmt.experiment import Experiment

exp = Experiment(name="my_exp").init()
artifact, record = exp.run(
    model_type="qkernel",
    task_type="classification",
    dataset=dataset_instance,
    model=model_instance,
)

# Aggregate results
df = exp.runs_to_dataframe()
Parameters:

  • name (str | None)

  • desc (str | None)

  • auto_gen_mode (bool)

  • root_experiment_dirc (str | Path)

  • llm_model_path (str)

  • logger (Logger)

name

Experiment name. If None, a timestamped name is generated.

Type:

str | None

desc

Human-readable description. When auto_gen_mode is enabled and the value is an empty string, an LLM can generate it automatically.

Type:

str | None

auto_gen_mode

Whether to generate run descriptions with an LLM.

Type:

bool

root_experiment_dirc

Root directory where all experiments are stored.

Type:

pathlib.Path

experiment_dirc

Directory assigned to this particular experiment.

Type:

pathlib.Path

current_run_id

ID of the next run to be executed (zero-based).

Type:

int

exp_db

In-memory database object holding experiment meta-data.

Type:

ExperimentDB | None

logger

Logger instance to report progress and warnings.

Type:

logging.Logger

_repo

Internal repository that encapsulates all filesystem & persistence operations.

Type:

ExperimentRepository

__init__(name=None, desc=None, auto_gen_mode=False, root_experiment_dirc=PosixPath('/home/runner/work/qxmt/qxmt/experiments'), llm_model_path='microsoft/Phi-3-mini-128k-instruct', logger=<Logger qxmt.experiment.experiment (INFO)>)

Initialize the Experiment class with experiment settings and configuration.

This method sets up the basic configuration for the experiment, including: - Experiment name and description - Auto-generation mode for descriptions - Experiment directory structure - LLM model path (if using auto-generation) - Logger configuration

Parameters:

  • name (Optional[str]) – Name of the experiment. If None, a default name will be generated using the current timestamp. Defaults to None.

  • desc (Optional[str]) – Description of the experiment. Used for documentation and search purposes. If None and auto_gen_mode is True, a description will be generated. Defaults to None.

  • auto_gen_mode (bool) – Whether to use the DescriptionGenerator for automatic description generation. Requires the “USE_LLM” environment variable to be set to True. Defaults to USE_LLM.

  • root_experiment_dirc (str | Path) – Root directory where experiment data will be stored. Defaults to DEFAULT_EXP_DIRC.

  • llm_model_path (str) – Path to the LLM model used for description generation. Defaults to LLM_MODEL_PATH.

  • logger (Logger) – Logger instance for handling warning and error messages. Defaults to LOGGER.

Return type:

None

Note

If auto_gen_mode is True but USE_LLM is False, a warning will be logged and auto_gen_mode will be automatically set to False.

get_run_record(runs, run_id)

Get the run record of the target run_id.

Parameters:

  • runs (list[RunRecord]) – List of run records to search.

  • run_id (int) – Target run_id.

Returns:

Target run record.

Return type:

RunRecord

Raises:

ValueError – If the run record does not exist.

init()

Initialize the experiment directory and DB.

Creates a new experiment directory and initializes an empty ExperimentDB. The directory will be created under root_experiment_dirc with the experiment name.

Returns:

Initialized experiment instance.

Return type:

Experiment

Raises:

ExperimentSettingError – If the experiment directory already exists.

load(exp_dirc, exp_file_name=PosixPath('experiment.json'))

Load existing experiment data from a json file.

Parameters:

  • exp_dirc (str | Path) – Path to the experiment directory.

  • exp_file_name (str | Path) – Name of the experiment file. Defaults to DEFAULT_EXP_DB_FILE.

Returns:

Loaded experiment instance.

Return type:

Experiment

Raises:

  • FileNotFoundError – If the experiment file does not exist.

  • ExperimentSettingError – If the experiment directory does not exist.

reproduce(run_id, check_commit_id=False, tol=1e-06)

Reproduce a previous experiment run using its configuration.

Parameters:

  • run_id (int) – ID of the run to reproduce.

  • check_commit_id (bool) – Whether to verify that the current git commit matches the original run. Defaults to False.

  • tol (float) – Tolerance for comparing evaluation metrics between original and reproduced runs. Defaults to 1e-6.

Returns:

A tuple containing the reproduced run’s artifact and record.

Return type:

tuple[RunArtifact, RunRecord]

Raises:

  • ExperimentNotInitializedError – If the experiment has not been initialized.

  • ReproductionError – If the run was executed in instance-based mode (no config file available).

  • ValueError – If the reproduced results differ significantly from the original run.

Note

  • Only runs executed in config-based mode can be reproduced.

  • The method will raise an error if the reproduced results differ from the original beyond the specified tolerance.

  • If check_commit_id is True and the current commit differs from the original, a warning will be logged but the reproduction will continue.

run(model_type=None, task_type=None, dataset=None, model=None, config_source=None, default_metrics_name=None, custom_metrics=None, n_jobs=2, show_progress=True, desc='', repo_path=None, add_results=True)

Execute a new experiment run with the specified configuration.

This method supports two main modes of operation: 1. Config-based mode: Using a configuration file or ExperimentConfig instance 2. Instance-based mode: Directly providing dataset and model instances

Parameters:

  • model_type (Optional[str]) – Type of model to use (‘qkernel’ or ‘vqe’). Required for instance-based mode. Defaults to None.

  • task_type (Optional[str]) – Type of task for QKernel models (‘classification’ or ‘regression’). Required for QKernel models. Defaults to None.

  • dataset (Optional[Dataset]) – Dataset instance for instance-based mode. Defaults to None.

  • model (Optional[BaseMLModel | BaseVQE]) – Model instance for instance-based mode. Defaults to None.

  • config_source (Optional[ExperimentConfig | str | Path]) – Configuration source for config-based mode. Can be an ExperimentConfig instance or path to config file. Defaults to None.

  • default_metrics_name (Optional[list[str]]) – List of default metrics to evaluate. Defaults to None.

  • custom_metrics (Optional[list[dict[str, Any]]]) – List of custom metric configurations. Defaults to None.

  • n_jobs (int) – Number of parallel jobs for processing. Defaults to DEFAULT_N_JOBS.

  • show_progress (bool) – Whether to display progress bars. Defaults to True.

  • desc (str) – Description of the run. Defaults to “”.

  • repo_path (Optional[str]) – Path to git repository for version tracking. Defaults to None.

  • add_results (bool) – Whether to save run results and artifacts. Defaults to True.

Returns:

A tuple containing the run artifact and record.

Return type:

tuple[RunArtifact, RunRecord]

Raises:

  • ExperimentNotInitializedError – If the experiment has not been initialized.

  • ExperimentRunSettingError – If required parameters are missing or invalid.

  • ValueError – If model_type is invalid.

Note

  • For config-based mode, the configuration file will be saved in the run directory.

  • For instance-based mode, model_type must be specified.

  • If git is available, the commit ID will be tracked.

runs_to_dataframe(include_validation=False)

Convert experiment run records into a pandas DataFrame.

Parameters:

include_validation (bool) – Whether to include validation metrics in the DataFrame. Defaults to False.

Returns:

A DataFrame containing run results with the following columns:

  • run_id: The ID of each run

  • [metric_name]: Evaluation metrics for test data

  • [metric_name]_validation: Validation metrics (if include_validation is True)

Return type:

pd.DataFrame

Raises:

  • ExperimentNotInitializedError – If the experiment has not been initialized.

  • ValueError – If the run records contain invalid evaluation types.

Note

  • The DataFrame will be empty if no runs have been recorded.

  • Validation metrics are only included if they were computed during the run.

  • The method automatically handles both QKernel and VQE evaluation types.

save_experiment(exp_file=PosixPath('experiment.json'))

Save the experiment data to a json file.

Parameters:

exp_file (str | Path) – Name of the file to save the experiment data. Defaults to DEFAULT_EXP_DB_FILE.

Raises:

ExperimentNotInitializedError – If the experiment is not initialized.

Return type:

None