Limit the maximum number of trials to be added for the quicker first response

[nabenabe0928]

Contributor License Agreement

This repository (optuna-dashboard) and Goptuna share common code.
This pull request may therefore be ported to Goptuna.
Make sure that you understand the consequences concerning licenses and check the box below if you accept the term before creating this pull request.

• I agree this patch may be ported to Goptuna by other Goptuna contributors.

Reference Issues/PRs

As the first response becomes significantly slower as the number of trials increases, this PR limits the number of trials to be added at the first response.
More specifically, each GET can take only n=limit trials.

What does this implement/fix? Explain your changes.

I added a query parameter for GET of trials.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (522b6ff) 68.96% compared to head (f838ef5) 69.07%.
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #719      +/-   ##
==========================================
+ Coverage   68.96%   69.07%   +0.10%     
==========================================
  Files          35       35              
  Lines        2388     2393       +5     
==========================================
+ Hits         1647     1653       +6     
+ Misses        741      740       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nabenabe0928]

@c-bata
This PR will be ready after PR#727 is merged.

However, I think it would be better to separate this PR into smaller PRs as this PR has multiple modifications:

1. Limiting the maximum number of trials to be used for one update,
2. Updating studyDetail only if the fetched trials has a positive length to avoid unnecessary renderings,
3. Detecting the leftovers in the cache and quickly picking them up with a shorter interval, and
4. Replacing the heuristic for the check interval of the storage with a new algorithm to avoid unnecessary checks.

I will count on you for the decision, but if you believe it is better to split this PR, please tell me which modifications should be bundled.

[nabenabe0928]

@c-bata @keisuke-umezawa The PR is ready for reviews as the PR#727 was merged:)

[keisuke-umezawa]

I left some nits and questions. Overall it seems good to me, but how did you assure the quality of code? If possible, you can add some unit test to guarantee it, or at least I would like to check it with some scripts.

[nabenabe0928]

@keisuke-umezawa I added some tests")

[keisuke-umezawa]

Thank you for working on it! LGTM.

[keisuke-umezawa]

@c-bata
Do you want to review it? If not, I will merge it.

[nabenabe0928]

import optuna

def objective_many_trials_imbalance_importances(trial: optuna.Trial) -> float:
    if (trial.number + 1) % 500 == 0:
        print(f"Evaluate Trial#{trial.number}")

    x1 = trial.suggest_float("x1", 0, 10)
    x2 = trial.suggest_float("x2", 0, 10)
    x3 = trial.suggest_categorical("x3", ["foo", "bar"])
    return (x1 - 2) ** 2 + (x2 - 5) ** 2

storage = optuna.storages.InMemoryStorage()
study = optuna.create_study(study_name="test", storage=storage, sampler=optuna.samplers.RandomSampler())
study.optimize(objective_single, n_trials=10000)