 # Regression Topics: Old to Modern

This post is dedicated to the Regression Topics from basic to the present days. This is done to write the topics together in a concise form and then expand each of them with corresponding data to make it concrete.

• Regression as a Projection
• Full Rank Model
• Multiple Linear Regression
• Non – Full Rank Model
• Multiple Hypothesis Testing (FDR, etc) [Controlling # False Positives]
• Residual Analysis
• Box-Cox Transformation
• $$R^2$$, Adjusted $$R^2$$, Predictive $$R^2$$
• Influential Observations, Measures of Influence (diffs, Cook’s Distance, etc)
• Variable Selection
• Full Model; Reduced Model
• Partial F – Test
• Mallow’s $$C_p$$
• AIC, BIC, Forward Selection, Backward Elimination
• Multicollinearity, VIF
• Two-Stage Regression (Handling endogeneity)

Penalized Regression
• Ridge, Lasso, Bayesian Lasso, Fused Lasso, Elastic Net, Group Lasso, Dirichlet Lasso
• Subset Selection
• Methods to deal non-constant variance (WLS, GLS)

Longitudinal ( or Temporal) Dependence
• Univariate Longitudinal ( AR1, Compound Symmetry)
• Bivariate Longitudinal
• Repeates Measure ANOVA
• Random Effects Model

Non- Parametric Regression
• KNN
• Kernel Smoothing
• Local Polynomials
• Basis Functions
• Spline
• Varying Coefficients Model
• Partially VCM
• Bootstrap Regression

GLM
• Logit Regression
• Probit Model
• Count Data
• Semi Continuous Data – Two Step Model
• Zero Inflated Poisson Model
• Hurdle Model
• Non – Parametric GLM
• Proportional Odds Cumulative Logit Model
• Generalized Linear Mixed Model

Missing Data Mechanism
• MCAR
• MAR
• MNAR

Imputation Techniques
• Case Deletion
• Available Data Usage
• Imputing the unconditional means
• Imputing the unconditional distribution
• Imputing the conditional means
• Maximum Likelihood Estimation
• Multiple Imputation Methods
• Propensity Score Based Methods
• CCMV, ACMV

Go Top