# Regression Topics: Old to Modern

This post is dedicated to the Regression Topics from basic to the present days. This is done to write the topics together in a concise form and then expand each of them with corresponding data to make it concrete.

- Regression as a Projection
- Full Rank Model
- Multiple Linear Regression
- Non – Full Rank Model
- Multiple Hypothesis Testing (FDR, etc) [Controlling # False Positives]
- Residual Analysis
- Box-Cox Transformation
- \(R^2\), Adjusted \(R^2\), Predictive \(R^2\)
- Influential Observations, Measures of Influence (diffs, Cook’s Distance, etc)
- Variable Selection
- Full Model; Reduced Model
- Partial F – Test
- Mallow’s \(C_p\)
- AIC, BIC, Forward Selection, Backward Elimination
- Multicollinearity, VIF
- Two-Stage Regression (Handling endogeneity)
**Penalized Regression** - Ridge, Lasso, Bayesian Lasso, Fused Lasso, Elastic Net, Group Lasso, Dirichlet Lasso
- Subset Selection
- Methods to deal non-constant variance (WLS, GLS)
**Longitudinal ( or Temporal) Dependence** - Univariate Longitudinal ( AR1, Compound Symmetry)
- Bivariate Longitudinal
- Pohramadi
- Repeates Measure ANOVA
- Random Effects Model
**Non- Parametric Regression** - KNN
- Kernel Smoothing
- Local Polynomials
- Linear Additive Model
- Basis Functions
- Spline

- Varying Coefficients Model
- Partially VCM
- Bootstrap Regression
**GLM** - Logit Regression
- Probit Model
- Count Data
- Semi Continuous Data – Two Step Model
- Zero Inflated Poisson Model
- Hurdle Model
- Non – Parametric GLM
- Proportional Odds Cumulative Logit Model
- Generalized Linear Mixed Model
**Missing Data Mechanism** - MCAR
- MAR
- MNAR
**Imputation Techniques** - Case Deletion
- Available Data Usage
- Imputing the unconditional means
- Imputing the unconditional distribution
- Imputing the conditional means
- Maximum Likelihood Estimation
- Multiple Imputation Methods
- Propensity Score Based Methods
- CCMV, ACMV