Regression_Stratified_Sampling
Regression-Stratified Sampling for Optimized Algorithm Selection in Time-Constrained Tabular AutoML
Paper | Poster | SPIGM @ ICML 2024
RSS Evaluation Artifacts
The following are the artifacts for the RSS paper:
- All Evaluations on Benchmark #1: re_v2_all_files.pkl
- Jupyter Notebook on the comparison between PDF and KL - Benchmark #2 (Figure 2)
- Dataset for the summary of data (Table 1-Avg) - Benchmark #2
- Dataset for the summary of data (Table 1-Std) - Benchmark #2
- All AutoML evaluations for CD plot: all_cd_data.pickle
- Jupyter Notebook for AutoML CD Plot which produces a TeX file
- All AutoML evaluations on extra time budgets for 10 datasets with 10 random seeds: 10_dataset_10_seed_extra_time.pkl
- All other plots for Benchmark #1: Source files come from $R^2$ evaluations as
R2_B.picklewhere $B=\beta$ and $\beta \in [0, 10, 100, 1k]$ based on the number of strata. Similarly, for RMSE evaluation, useRMSE_B.pickle. A compiled version of Jupyter Notebook #1 for all other plots (i.e., Figure 3, 7, 8, 9).