Comments (7)
Maybe we can do this
num_trials = max(get_cpu_count() // num_cpus, get_cpu_count() // num_gpus) if time_limits < THRESHOLD and num_gpus > 0 else num_trials
from autogluon.
I think we should try to primarily control everything based on time_limits (so assume most users do not specify num_trials). Something like this:
def task.fit(..., time_limits = None, num_trials=None):
if num_trials is None:
if time_limits is None:
time_limits = 10 * 60 # run for 10min by default
elif time_limits <= THRESHOLD: # THRESHOLD = say 1 or 10 sec, too little time to do much
num_trials = 1
else:
num_trials = np.inf # run as many trials as you can within the given time_limits
if time_limits is None:
time_limits = np.inf # in this case, user only specified num_trials, so just run all of them regardless of time-limits
from autogluon.
Currently the user has to explicitly be aware of both time_limits and num_trials. When I don't specify num_trials, it seems to default to 2 beneath the hood. This is very confusing for me, since when I set time_limits = big number, I expect task.fit() to run lots of trials. Currently task.fit() is basically running for min{ num_trials, time_limits} which is not very intuitive behavior. I think the default behavior outlined in my code above would be much more natural, where whichever of num_trials or time_limits is specified is the one that dictates the behavior of task.fit()
from autogluon.
Why not leveraging soft time limits as previously implemented?
from autogluon.
can you explain soft time limits?
My point is that most users will not be aware of both time_limits and num_trials, and they should not have to specify both of these things to control the overall runtime of autogluon. When they only specify one of them, the other one still heavily affects runtimes underneath the hood, which may be very confusing for users (it was for me)
from autogluon.
Ok, I agree that the users only need to specify what they want and the remaining should stay as the default. Soft time limit is different, it means that for the trials already scheduled, we let them run to the end, similar to what we have.
from autogluon.
Resolved in current version
from autogluon.
Related Issues (20)
- [BUG] Pip install with python 3.9 and 3.10 fails on windows at ray download
- Documentation Update Request: Issues with Specifying Hyperparameters and Tuning Them Example
- [BUG] Object Detection doesn't handle purely background negative images with no annotations HOT 1
- [BUG] Setting `ag_args_ensemble` `num_folds` to 0 results in error when `num_bag_folds >= 2` is set in the `.fit()`.
- Update on 2023 Roadmap Tasks and Request for 2024 Roadmap
- [BUG] Unable to work with Autogluon Object Detection
- Tabular: Test LightGBM `use_quantized_grad=True`
- [BUG] Kaggle Autogluon Installation Not Working for MultiModalPredictor
- Not able install Autogluon in my windows system. It is not able to find numpy==1.21.3 HOT 1
- Conda does not install `ray` by default HOT 1
- During inference, multi-GPUs are not used with DDP strategy. Only single GPU is used.
- Unable to Affect Model Training with Different random_seed Values in Time-series Module HOT 1
- Request: Implement Feature Importance Explainability for Time-Series Module HOT 1
- [timeseries] DirectTabular & RecursiveTabular models fail if eval_metric = "WAPE" HOT 7
- `estimate_memory_usage` throws `TypeError` when using HPO for `GBM` model in `TabularPredictor` HOT 2
- [BUG] export_onnx not working for some Multimodal models
- [BUG] refit_full does not expand memory allowance when `use_bag_holdout=True` (`good_quality` preset) HOT 2
- Disable dynamic_stacking when `use_bag_holdout=True`
- [BUG] [Errno 2] No such file or directory: c:\\...\\prediction-net-state.pt HOT 5
- [BUG] Missing example code automm/Conv-Lora HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from autogluon.