A Comparison Study of Principle Component Regression, Partial Least Square Regression and Ridge Regression with Application to FTIR Data

Abstract: Least squares estimator may fail when the number of explanatory vari-able is relatively large in comparison to the sample or if the variablesare almost collinear. In such a situation, principle component regres-sion, partial least squares regression and ridge regression are oftenproposed methods and widely used in many practical data analysis,especially in chemometrics. They provide biased coecient estima-tors with the relatively smaller variation than the variance of the leastsquares estimator. In this paper, a brief literature review of PCR,PLS and RR is made from a theoretical perspective. Moreover, a dataset is used, in order to examine their performance on prediction. Theconclusion is that for prediction PCR, PLS and RR provide similarresults. It requires substantial verication for any claims as to thesuperiority of any of the three biased regression methods.