Source mixing models are applied broadly to ecological and hydrological studies, but their accuracy in calculating source contribution (pi) has never been evaluated due to the unknowable nature of the actual source mixing ratios. Moreover, the effect of the external influence originated from the characteristics of the user-provided data has never been quantified, hampering the establishment of the model selection framework suitable for diverse study backgrounds. Therefore, we (1) evaluated the model accuracy in estimating pi of an iterative model (IsoSource) and three Bayesian models (MixSIR, SIMMR, and MixSIAR) under 500 mixing scenarios with predefined mixing ratios, and (2) analyzed the influences of external factors on the model performance. We aimed to build a model-assessment framework and provided a guideline for model selection. Our results from the mixing scenarios of unprecedented size demonstrate that the Bayesian models, particularly SIMMR, exhibited significantly improved performance in estimating the actual pi. As the external influences, less uniform pi vectors and higher underdetermination of pi distributions results in decreased accuracy of model estimation. The effect of the number of sources is negligible when it was less than seven in a dual-tracer occasion. We identify the key parameters and predict the bias of the source mixing model through machine learning approach. Our study is the first evaluation of source mixing model accuracy in pi estimation, and thus crucial for ecohydrological research. We also propose a model selection guideline for various research environments and sampling regimes.