resume model search?

[yamanogotoshi (yamanogotoshi)]

It seems that optuna can resume hyperparameter search using RDB.
https://optuna.readthedocs.io/en/v2.0.0/tutorial/rdb.html
Is it possible to do something similar with flaml?

Thanks

[Chi Wang (sonichi)]

Currently, flaml can do warm start in the following ways:

1. In flaml.AutoML: https://microsoft.github.io/FLAML/docs/Use-Cases/Task-Oriented-AutoML#warm-start
2. In flaml.tune: https://microsoft.github.io/FLAML/docs/Use-Cases/Tune-User-Defined-Function#warm-start

Do these work for you?

[yamanogotoshi (yamanogotoshi)]

Thank you for answering.

From the information on the link, what I want can be achieved by saving and decompressing the AutoML object in pickle format as follows.

from __future__ import annotations
from flaml import AutoML
from sklearn.datasets import load_iris
from pathlib import PurePath
import pickle


def load_pickle(filepath: str | PurePath):
    with open(filepath, "rb") as f:
        data = pickle.load(f)
    return data


def dump_pickle(data, filepath: str | PurePath):
    with open(filepath, "wb") as f:
        pickle.dump(data, f)


automl = AutoML()
# Specify automl goal and constraint
automl_settings = {
    "time_budget": 1,  # in seconds
    "metric": 'accuracy',
    "task": 'classification',
    "log_file_name": "iris.log",
    "verbose": 0,
}
X_train, y_train = load_iris(return_X_y=True)
# Train with labeled input data
automl.fit(X_train=X_train, y_train=y_train, **automl_settings)

dump_pickle(automl, "automl.pickle")
automl = load_pickle("automl.pickle")

automl2 = AutoML()
automl2.fit(
    X_train,
    y_train,
    time_budget=1,
    starting_points=automl.best_config_per_estimator,
)
print(automl2.best_config_per_estimator)
print(type(automl2.best_config_per_estimator))

For example, when learning fails in the middle, it is worrisome that the hyperparameter search cannot be restarted from the middle, but I just decided not to pursue it any further.

Thanks.

[Chi Wang (sonichi)]

Would you like to chat more about it on discord? I want to make sure that your concern is addressed properly.