One of the influential factors in statistical data analysis is the presence of outliers. Methods that are not affected by outliers are called robust statistical methods. In addition to outliers, the presence of linear dependence between predictor variables, which is referred to as multiple collinearity, and the large number of variables in the presence of a small sample size, especially in high-dimensional sparse models, are other problems that reduce the efficiency of inferences obtained from classical regression methods.
In this article, we first examine the disadvantages of the classical least squares regression method in the presence of outliers, multiple collinearity, and sparse models. Then, we introduce and examine robust regression and compensated regression methods as solutions to these problems. We also examine robust regression methods by simultaneously considering outliers and multiple collinearity or sparse models.
Finally, in order to compare the performance of the different estimators proposed in this paper, we first conduct three simulation studies and then analyze a real dataset using robust regression methods.