Skip to content

fixes max_eval_time_mins and max_eval_time_mins to allow None and inf - also allows steady state estimator to use template search spaces #1375

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

perib
Copy link
Contributor

@perib perib commented Apr 17, 2025

[please review the Contribution Guidelines prior to submitting your pull request. go ahead and delete this line if you've already reviewed said guidelines.]

What does this PR do?

As described in this issue: EpistasisLab/tpot2#169

max_eval_time_mins does should work with inf or None, but actually throws an error. This update fixes that issue. I also renamed some variables and moved unit conversions for better clarity. Also updated the doctrings to match the default values. (What the default values should be can be up for discussion as well. Perhaps early_stop should be set by default? I can imagine users running tpot and not realizing it doesn't have a stopping criteria by default other than waiting an hour).

I also updated TPOTEstimatorSteadyState to allow the template search spaces to be used ('linear', 'graph', 'linear-light', etc).

## Where should the reviewer start? How should this PR be tested?

from tpot import TPOTClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
import tpot
# Load the Iris dataset
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)

for max_time_mins in [3, None, float('inf')]:
    for max_eval_time_mins in [2, None, float('inf')]:
        try:
            spot = tpot.TPOTEstimator(
                    classification=True,
                    search_space = "linear-light",
                    scorers=['neg_mean_squared_error'], 
                    scorers_weights=[1],
                    population_size=10,
                    generations=3,
                    n_jobs=20,
                    max_time_mins=max_time_mins,
                    max_eval_time_mins=max_eval_time_mins,
                    verbose=0,
                    )

            spot.fit(X_train, y_train)

        except:
            print("FAILED ON")
            print("max_time_mins ", max_time_mins )
            print("max_eval_time_mins", max_eval_time_mins)


X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, test_size=0.2, random_state=42)

for max_time_mins in [3, None, float('inf')]:
    for max_eval_time_mins in [2, None, float('inf')]:
        try:
            spot = tpot.TPOTEstimatorSteadyState(
                    classification=True,
                    search_space = "linear-light",
                    scorers=['neg_mean_squared_error'], 
                    scorers_weights=[1],
                    population_size=10,
                    n_jobs=20,
                    max_time_mins=max_time_mins,
                    max_eval_time_mins=max_eval_time_mins,
                    verbose=2,
                    
                    max_evaluated_individuals=10,
                    
                    )

            spot.fit(X_train, y_train)

        except:
            print("FAILED ON")
            print("max_time_mins ", max_time_mins )
            print("max_eval_time_mins", max_eval_time_mins)

Any background context you want to provide?

What are the relevant issues?

Originally brought up in the TPOT2 repo here:
EpistasisLab/tpot2#169

@perib
Copy link
Contributor Author

perib commented Apr 17, 2025

I also realized that this was branched off of my previous PR, so you may want to review those first before this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant