Regression Topics: Old to Modern

This post is dedicated to the Regression Topics from basic to the present days. This is done to write the topics together in a concise form and then expand each of them with corresponding data to make it concrete.

  • Regression as a Projection
  • Full Rank Model
  • Multiple Linear Regression
  • Non – Full Rank Model
  • Multiple Hypothesis Testing (FDR, etc) [Controlling # False Positives]
  • Residual Analysis
  • Box-Cox Transformation
  • \(R^2\), Adjusted \(R^2\), Predictive \(R^2\)
  • Influential Observations, Measures of Influence (diffs, Cook’s Distance, etc)
  • Variable Selection
  • Full Model; Reduced Model
  • Partial F – Test
  • Mallow’s \(C_p\)
  • AIC, BIC, Forward Selection, Backward Elimination
  • Multicollinearity, VIF
  • Two-Stage Regression (Handling endogeneity)

    Penalized Regression
  • Ridge, Lasso, Bayesian Lasso, Fused Lasso, Elastic Net, Group Lasso, Dirichlet Lasso
  • Subset Selection
  • Methods to deal non-constant variance (WLS, GLS)

    Longitudinal ( or Temporal) Dependence
  • Univariate Longitudinal ( AR1, Compound Symmetry)
  • Bivariate Longitudinal
  • Pohramadi
  • Repeates Measure ANOVA
  • Random Effects Model

    Non- Parametric Regression
  • KNN
  • Kernel Smoothing
  • Local Polynomials
  • Linear Additive Model
  • Basis Functions
  • Spline
  • Varying Coefficients Model
  • Partially VCM
  • Bootstrap Regression

  • Logit Regression
  • Probit Model
  • Count Data
  • Semi Continuous Data – Two Step Model
  • Zero Inflated Poisson Model
  • Hurdle Model
  • Non – Parametric GLM
  • Proportional Odds Cumulative Logit Model
  • Generalized Linear Mixed Model

    Missing Data Mechanism
  • MCAR
  • MAR
  • MNAR

    Imputation Techniques
  • Case Deletion
  • Available Data Usage
  • Imputing the unconditional means
  • Imputing the unconditional distribution
  • Imputing the conditional means
  • Maximum Likelihood Estimation
  • Multiple Imputation Methods
  • Propensity Score Based Methods

Leave a Reply

Your email address will not be published. Required fields are marked *

Go Top