The Impact of Bootstrap on PCR and PLS Predictive Accuracy under Multicollinearity and Small Sample Sizes: A Simulation Study
DOI:
https://doi.org/10.65405/fbmtrv02Keywords:
Bootstrap resampling; principal component regression (PCR); partial least squares regression( PLS); multicollinearity; small sample size.Abstract
In this paper, we study how resampling affects the predictive accuracy of principal component regression (PCR) and partial least squares (PLS) models in the presence of small samples and multicollinearity. Monte Carlo simulations were carried out with 500 iterations by drawing 500 bootstrap samples per iteration. For the analysis, three different sample sizes (n = 30, 50, 100), two values for the predictive variables (p = 5, 10) and three degrees of multicollinearity (r = 0.80, 0.90, 0.99) were considered while comparing the following techniques: standard PCR, bootstrap-PCR, standard PLS, and bootstrap-PLS. The performance of these techniques was further assessed based on the average mean squared error (AMSE) and mean variance (AVAR). According to the simulation results, it can be summarized that resample testing is advantageous both for PCR and PLS in statistical estimators. It is important to mention that the improvement is more remarkable as multicollinearity becomes stronger and the sample sizes (n) get smaller. The Bootstrap-PLS approach had the best performance over all other estimators.
Downloads
References
1.Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics, 7(1), 1–26.
2. Efron, B., & Tibshirani, R. J. (1994). An introduction to the bootstrap. CRC Press.
3. Efron, B., & Hastie, T. (2016). Computer age statistical inference: Algorithms, evidence,and data science. Cambridge University Press.
4.Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Pearson.
5. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An introduction to statistical learning: With applications in R (2nd ed.). Springer.
6. Jolliffe, I. T. (2002). Principal component analysis (2nd ed.). Springer.
7. Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear statistical models (5th ed.). McGraw-Hill Irwin.
8. Krichene, E., Hmadi, M. S. A., & Al-Gajamiya, S. K. (2026). A Fair Comparative Framework for Time-Series Forecasting Using ARIMAX, XGBoost, and LSTM: Evidence from Libya. Al-Farooq Journal of Sciences, 2(3), 69-85.
9. Montgomery, D. C., Peck, E. A., & Vining, G. G. (2012). Introduction to linear regression analysis (5th ed.). Wiley.
10. Montgomery, D. C., Peck, E. A., & Vining, G. G. (2021). Introduction to linear regression analysis (6th ed.). Wiley.
11. Wold, S., Sjöström, M., & Eriksson, L. (2001). PLS-regression: A basic tool of chemometrics. Chemometrics and Intelligent Laboratory Systems, 58(2), 109–130.. https://doi.org/10.1016/S0169-7439(01)00155-1.
