Models under Regression Analysis list

by Roshan   Last Updated October 15, 2018 00:19 AM

I am compiling a list of models under Regression analysis(Whatever I think is useful for Machine learning) which is divided into two models i.e Parametric and Non-parametric Regression. Got most of the models from wikepedia but it's so confusing as there are different models and approaches all over the place . Hence I wanted to make a list that can help others and myself, so that we can refer all possible models under Regression.

PARAMETRIC REGRESSION  
1)Linear Regression
  1.Simple linear Regression
    >Ordinary Least Squares(OLS)
    >Deming Regression
    >Least Absolute Deviations Regression(LAD)
    >Theil–Sen Estimator
  2.Multiple Linear Regression
    >Polynomial Regression
    >Ordinary Least Squares(OLS)
    >Weighted least squares
    >Partial least squares
  3.Multivariate/General Linear Regression
  4.Gradient Descent
    >Batch Gradient Descent
    >Mini-Batch Gradient Descent
    >Stochastic Gradient Descent
2)Generalized Linear Model (GLM)
3)Logistic Regression
  1.Generalized linear model
  2.Iteratively reweighted least squares (IRLS)
  3.Maximum likelihood estimation
  4.Evaluating Goodness of fit  
    >Deviance and Likelihood ratio test
    >Hosmer-Lemeshow test
    >Pseudo R squared
    >R squared
  5.Assess significance of Regression coefficients
    >Case-control sampling
    >Likelihood ratio test
    >Wald statistics 
  6.Optimization
    >Batch Gradient Descent
    >Mini-Batch Gradient Descent
    >Stochastic Gradient Descent
    >Conjugate Gradient
    >BFGS
    >L-BFGS
4)Regularization
  1.Ridge Regression
  2.Lasso Regression
  3.ElasticNet Regression
5)Non-Linear Regression
  >Ordinary Least Squares(OLS)
  >Weighted least squares
6)Quantile Regression 
7)Segmented Regression
8)Percentage Regression
NON-PARAMETRIC REGRESSION
1)Gaussian process regression or Kriging
2)Kernel Regression
3)Nonparametric multiplicative regression (NPMR)

Can you guys tell me if I have put the models in the correct list and also tell me where I should put the models that I haven't placed under any header yet like:

Principal Component Regression  
Partial Least Square Regression  
Support Vector Regression  
Ordinal Regression  
Poisson Regression  
Negative Binomial Regression  
Quasi-Poisson Regression  
Cox Regression  

Also can you let me know which of these are necessary for machine learning(as I am interested in statistical analysis and applied machine learning).



Related Questions


Examples of Non-Linear Time Series?

Updated June 07, 2015 04:08 AM

Multiple regression, full and restricted model

Updated March 12, 2017 19:19 PM

Multivariate multiple non linear regression

Updated March 02, 2017 18:19 PM

Fitting polynomial equation and combined effect

Updated November 14, 2018 13:19 PM