What to do with data for testing

[tomicapretto]

We had many test failures because for 403 HTTP errors, like here https://github.com/bambinos/bambi/actions/runs/18083187337/job/51449650074

Relevant output chunk

=========================== short test summary info ============================
FAILED tests/test_models.py::test_predict_new_groups[data_sleepstudy-Reaction ~ 1 + Days + (1 + Days | Subject)-gaussian-df_new0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_glmlss.py::test_normal_with_splines - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_glmlss.py::test_gamma_with_splines - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_no_effect[defaults] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_no_effect[defaults_and_user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_no_effect[user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_no_effect_kwargs - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_comparisons[defaults] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_comparisons[user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_comparisons[defaults_and_user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_slopes[defaults] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_slopes[user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_slopes[defaults_and_user_passed] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_eps[comparisons] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_data_grid_eps[slopes] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_select_draws_no_effect[1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_select_draws_no_effect[2] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret.py::test_select_draws_no_effect[3] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret_messages.py::test_predictions_list - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret_messages.py::test_predictions_list_unspecified - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret_messages.py::test_predictions_dict_unspecified - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret_messages.py::test_comparisons_contrast_default - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_interpret_messages.py::test_slopes_wrt_default - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_models.py::TestCensoredResponses::test_model_with_intercept - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_models.py::TestCensoredResponses::test_model_without_intercept - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_models.py::TestCensoredResponses::test_model_with_group_specific_effects - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_models.py::test_predict_new_groups_fail - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_use_hdi[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_use_hdi[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_hdi_prob[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_hdi_prob[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_legend[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_legend[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_ax[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestCommon::test_ax[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[hp-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[hp-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[gear-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[gear-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[covariates2-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[covariates2-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[covariates3-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_basic[covariates3-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates0-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates0-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates1-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates1-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates2-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates2-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates3-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_groups[covariates3-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates0-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates0-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates1-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates1-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates2-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_group_and_panel[covariates2-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_user_values[conditional0-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_user_values[conditional0-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_user_values[conditional1-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_with_user_values[conditional1-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_average_by[am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_average_by[drat] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_average_by[average_by2] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_fig_kwargs[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_fig_kwargs[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_subplot_kwargs[False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_subplot_kwargs[True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms0-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms0-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms1-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms1-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms2-False] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_transforms[transforms2-True] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_group_effects - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestPredictions::test_same_variable_conditional_and_group - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_basic[hp-am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_basic[am-hp] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_with_groups[hp-conditional0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_with_groups[hp-conditional1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_with_user_values[contrast0-conditional0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_with_user_values[contrast1-conditional1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_subplot_kwargs[drat-conditional0-subplot_kwargs0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_transforms[drat-conditional0-transforms0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_transforms[drat-conditional1-transforms1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_average_by[am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_average_by[drat] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_average_by[average_by2] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestComparison::test_group_effects - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_basic[hp-am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_basic[am-hp] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_groups[hp-conditional0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_groups[hp-conditional1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_user_values[wrt0-conditional0-None] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_user_values[wrt1-conditional1-None] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_user_values[wrt2-conditional2-am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_with_user_values[wrt3-conditional3-drat] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_elasticity[dydx] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_elasticity[dyex] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_elasticity[eyex] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_elasticity[eydx] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_subplot_kwargs[drat-conditional0-subplot_kwargs0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_transforms[drat-conditional0-transforms0] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_transforms[drat-conditional1-transforms1] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_average_by[am] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_average_by[drat] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_average_by[average_by2] - urllib.error.HTTPError: HTTP Error 403: Forbidden
ERROR tests/test_plots.py::TestSlopes::test_group_effects - urllib.error.HTTPError: HTTP Error 403: Forbidden
======= 1 failed, 181 passed, 1 skipped, 107 errors in 664.47s (0:11:04) =======

I think this occurs because Bambi tries to download some data for the tests. That is expected, because that is how bmb.load_data works...
But I'm a bit frustrated with these test failures. I'll explore if there's something we can do to prevent this from happening, otherwise we'll need to look into other ways to access the data.

[tomicapretto]

Related: scikit-learn/scikit-learn#28297

[aloctavodia]

I think this is the same issue we observed with ArviZ. There we have most, if not all, the datasets defined in situ in a helper module or similar.

[tomicapretto]

Thanks @aloctavodia. The datasets we're using right now in our tests are:

• "adults"
• "ANES"
• "bikes"
• "carclaims"
• "kidney"
• "mtcars"
• "sleepstudy"

We should double check whether we really need all of them in tests before adding them to the repository (also, can we do it?)

[tomicapretto]

Perhaps this is not an issue anymore since updated the links?