Species distribution models are commonly applied to predict species responses to environmental conditions. A wide variety of models with different properties exist that vary in complexity, which affects their predictive performance and interpretability. Machine learning algorithms are increasingly used because they are capable to capture complex relationships and are often better in prediction. However, to inform environmental management, it is important that a model predicts well for the right reasons. It remains a challenge to select a model with a reasonable level of complexity that captures the true relationship between the response and explanatory variables as good as possible rather than fitting to the noise in the data.
In this study we ask: 1) how much predictive performance can we gain by using increasingly complex models, 2) how does model complexity affect the degree of overfitting, and 3) do the inferred responses differ among models and what can we learn from them? To address these questions, we applied eight models with different complexity to predict the probability of occurrence of freshwater macroinvertebrate taxa based on 2729 Swiss monitoring samples. We compared the models in terms of predictive performance during cross-validation and for generalization out of the calibration domain ("extrapolation" or transferability). We applied model agnostic tools to shed light on model interpretability.
Contrary to our expectation, all models predicted similarly well during cross-validation, while no model predicted better than the null model during out-of-domain generalization on average over all taxa. Performance was best for taxa with intermediate prevalence. More complex models predicted slightly better than standard statistical models but were prone to overfitting.
Overfitting indicates that a model describes not only the signal in the data but also part of the noise. This impedes the interpretation of response shapes learned by the model, because one cannot distinguish the signal from the noise. Furthermore, the strongly overfitting models learned irregular relationships and strong interactions that are ecologically not plausible. Thus, in this study, the minor gain in predictive performance from more complex models was outweighed by the overfitting.
Ecological field data that is used as model input or for calibration is typically prone to different sources of variability, from sampling, the measurement process and stochasticity. We therefore call for caution when using complex data-driven models to learn about species responses or to inform environmental management. In such cases, we recommend to compare a range of models regarding their predictive performance, overfitting and response shapes to better understand the robustness of inferred responses.

Increasing temperatures caused by anthropogenic climate change are leading to changes in the composition of local communities across biomes. This has implications for ecological assessment methods that rely on macroinvertebrates as bioindicators of water quality. To investigate the influence of changing water temperature on these assessment methods, we analysed macroinvertebrate data from Swiss national monitoring programs. We used a species distribution model to simulate temperature change effects on macroinvertebrate communities and estimated the resulting changes on three biological indices commonly used in Switzerland, namely the species richness of Ephemeroptera, Plecoptera and Trichoptera (EPT), the Swiss biological (IBCH) index along with its components, as well as the species at risk pesticides (SPEAR_pesticides) index. While results vary by temperature scenario and index, our model results for the most realistic water temperature increase scenario of + 2 °C across most sites in Switzerland suggest no, or only a minor, influence of temperature (not accounting for other hydrological changes). Our model projection predicted only a small increase in the probability of occurrence for 70 % of the studied families. The sensitivity to temperature as captured in our model is generally not very high and varies among the biological indices: on average across all sites, a + 2 °C increase in temperature resulted in a 7 % increase in EPT species richness, a 4 % increase in the IBCH index, and a less than 1 % increase in the SPEAR_pesticides index. Our study suggests the robustness of these biological indices to moderate warming and points towards the usefulness of these biological indices for the next few decades as tools for water quality assessment. Despite some limitations of statistical species distribution models (e.g., not accounting for dispersal limitation or biotic interactions, predictive performance varying by taxon), the study provides valuable insights into the complex relationships between environmental factors and macroinvertebrate communities, and the potential impacts of future temperature change. These findings can inform conservation and management efforts for these important ecological systems.