View source for General linear model

{{distinguish|text=[[Multiple linear regression]], [[Generalized linear model]] or [[General linear methods]]}}
{{Regression bar}}
The '''general linear model''' or '''multivariate regression model''' is a statistical [[linear model]]. It may be written as<ref name="MardiaK1979Multivariate">{{Cite book | author = [[K. V. Mardia]], J. T. Kent and J. M. Bibby | title = Multivariate Analysis | publisher = [[Academic Press]] | year = 1979 | isbn = 0-12-471252-5}}</ref>
: <math>\mathbf{Y} = \mathbf{X}\mathbf{B} + \mathbf{U},</math>

where '''Y''' is a [[matrix (mathematics)|matrix]] with series of multivariate measurements (each column being a set of measurements on one of the [[dependent variable]]s), '''X''' is a matrix of observations on [[independent variable]]s that might be a [[design matrix]] (each column being a set of observations on one of the independent variables), '''B''' is a matrix containing parameters that are usually to be estimated and '''U''' is a matrix containing [[errors and residuals in statistics|errors]] (noise).
The errors are usually assumed to be uncorrelated across measurements, and follow a [[multivariate normal distribution]].  If the errors do not follow a multivariate normal distribution, [[generalized linear model]]s may be used to relax assumptions about '''Y''' and '''U'''.

The general linear model incorporates a number of different statistical models: [[ANOVA]], [[ANCOVA]], [[MANOVA]], [[MANCOVA]], ordinary [[linear regression]], [[t-test|''t''-test]] and [[F-test|''F''-test]]. The general linear model is a generalization of multiple linear regression to the case of more than one dependent variable. If '''Y''', '''B''', and '''U''' were [[column vector]]s, the matrix equation above would represent multiple linear regression.

Hypothesis tests with the general linear model can be made in two ways: [[multivariate statistics|multivariate]] or as several independent [[univariate]] tests. In multivariate tests the columns of '''Y''' are tested together, whereas in univariate tests the columns of '''Y''' are tested independently, i.e., as multiple univariate tests with the same design matrix.

== Comparison to multiple linear regression ==
{{further|Multiple linear regression}}
Multiple linear regression is a generalization of [[simple linear regression]] to the case of more than one independent variable, and a [[special case]] of general linear models, restricted to one dependent variable. The basic model for multiple linear regression is

:<math> Y_i = \beta_0 + \beta_1 X_{i1} + \beta_2 X_{i2} + \ldots + \beta_p X_{ip} + \epsilon_i</math>

for each observation ''i'' = 1, ... , ''n''.

In the formula above we consider ''n'' observations of one dependent variable and ''p'' independent variables. Thus, ''Y''<sub>''i''</sub> is the ''i''<sup>th</sup> observation of the dependent variable, ''X''<sub>''ij''</sub> is ''i''<sup>th</sup> observation of the ''j''<sup>th</sup> independent variable, ''j'' = 1, 2, ..., ''p''. The values ''β''<sub>''j''</sub> represent parameters to be estimated, and ''ε''<sub>''i''</sub> is the ''i''<sup>th</sup> independent identically distributed normal error.

In the more general multivariate linear regression, there is one equation of the above form for each of ''m'' > 1 dependent variables that share the same set of explanatory variables and hence are estimated simultaneously with each other:

:<math> Y_{ij} = \beta_{0j} + \beta_{1j} X_{i1} + \beta_{2j}X_{i2} + \ldots + \beta_{pj} X_{ip} + \epsilon_{ij}</math>

for all observations indexed as ''i'' = 1, ... , ''n'' and for all dependent variables indexed as ''j = 1, ... , ''m''.

== Comparison to generalized linear model ==

The general linear model (GLM)<ref>Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). ''Applied linear statistical models'' (Vol. 4, p. 318). Chicago: Irwin.</ref><ref name=":1">Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences.</ref> and the [[Generalized linear model|generalized linear model (GLiM)]]<ref name=":0">{{Citation|last=McCullagh|first=P.|title=An outline of generalized linear models|date=1989|work=Generalized Linear Models|pages=21–47|publisher=Springer US|isbn=9780412317606|last2=Nelder|first2=J. A.|doi=10.1007/978-1-4899-3242-6_2}}</ref><ref>Fox, J. (2015). ''Applied regression analysis and generalized linear models''. Sage Publications.</ref> are two commonly used families of [[Statistics|statistical methods]] to relate some number of continuous and/or categorical [[Dependent and independent variables|predictors]] to a single [[Dependent and independent variables|outcome variable]]. 

The main difference between the two approaches is that the GLM strictly assumes that the [[Errors and residuals|residuals]] will follow a [[Conditional probability distribution|conditionally]] [[normal distribution]]<ref name=":1" />, while the GLiM loosens this assumption and allows for a variety of other [[Distribution (mathematics)|distributions]] from the [[Exponential family|exponential]] family for the residuals<ref name=":0" />. Of note, the GLM is a special case of the GLiM in which the distribution of the residuals follow a conditionally normal distribution. 

The distribution of the residuals largely depends on the type and distribution of the outcome variable; different types of outcome variables lead to the variety of models within the GLiM family. Commonly used models in the GLiM family include [[Logistic regression|binary logistic regression]]<ref>Hosmer Jr, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). ''Applied logistic regression'' (Vol. 398). John Wiley & Sons.</ref> for binary or dichotomous outcomes, [[Poisson regression]]<ref>Gardner, W., Mulvey, E. P., & Shaw, E. C. (1995). Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. ''Psychological bulletin'', ''118''(3), 392.</ref> for count outcomes, and [[linear regression]] for continuous, normally distributed outcomes. This means that GLiM may be spoken of as a general family of statistical models or as specific models for specific outcome types.   
{| class="wikitable"
!
![[General linear model]]
![[Generalized linear model]]
|-
|Typical estimation method
|[[Least squares]], [[best linear unbiased prediction]]
|[[Maximum likelihood]] or [[Bayesian probability|Bayesian]]
|-
|Examples
|[[ANOVA]], [[ANCOVA]], [[linear regression]]
|[[linear regression]], [[logistic regression]], [[Poisson regression]], gamma regression,<ref name=":02">{{cite book|title=Generalized Linear Models, Second Edition|last=McCullagh|first=Peter|author2=Nelder, John|publisher=Boca Raton: Chapman and Hall/CRC|year=1989|isbn=978-0-412-31760-6|ref=McCullagh1989|authorlink=Peter McCullagh|authorlink2=John Nelder}}</ref> [[general linear model]]
|-
|Extensions and related methods
|[[Multivariate analysis of variance|MANOVA]], [[Multivariate analysis of covariance|MANCOVA]], [[Mixed model|linear mixed model]]
|[[generalized linear mixed model]] (GLMM), [[Generalized estimating equation|generalized estimating equations]] (GEE)
|-
|[[R (programming language)|R]] package and function
|[https://stat.ethz.ch/R-manual/R-devel/library/stats/html/lm.html lm()] in stats package (base R)
|[https://stat.ethz.ch/R-manual/R-devel/library/stats/html/glm.html glm()] in stats package (base R)
|-
|[[Matlab (programming language)|Matlab]] function
|mvregress()
|glmfit()
|-
|[[SAS System|SAS]] procedures
|[https://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#glm_toc.htm PROC GLM], [https://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#reg_toc.htm PROC REG]
|[https://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#genmod_toc.htm PROC GENMOD], [https://support.sas.com/documentation/cdl/en/statug/63962/HTML/default/viewer.htm#logistic_toc.htm PROC LOGISTIC] (for binary & ordered or unordered categorical outcomes)
|-
|[[Stata]] command
|regress
|glm
|-
|[[SPSS]] command
|[https://stats.idre.ucla.edu/spss/output/regression-analysis/ regression], [https://stats.idre.ucla.edu/spss/library/spss-librarymanova-and-glm-2/ glm]
|genlin, logistic
|-
|[[Wolfram Language]] & [[Mathematica]] function
|LinearModelFit[]<ref>[http://reference.wolfram.com/language/ref/LinearModelFit.html LinearModelFit], Wolfram Language Documentation Center.</ref>

|GeneralizedLinearModelFit[]<ref>[http://reference.wolfram.com/language/ref/GeneralizedLinearModelFit.html GeneralizedLinearModelFit], Wolfram Language Documentation Center.</ref>
|-
|[[EViews]] command
|ls<ref>[http://www.eviews.com/help/helpintro.html#page/content%2Fcommandcmd-ls.html ls], EViews Help.</ref>
|glm<ref>[http://www.eviews.com/help/helpintro.html#page/content%2Fcommandcmd-glm.html glm], EViews Help.</ref>
|}

== Applications ==
An application of the general linear model appears in the analysis of multiple [[brain scan]]s in scientific experiments where '''Y''' contains data from brain scanners, '''X''' contains experimental design variables and confounds. It is usually tested in a univariate way (usually referred to a ''mass-univariate'' in this setting) and is often referred to as [[statistical parametric mapping]].<ref>{{Cite journal| doi = 10.1002/hbm.460020402|author1=K.J. Friston |author2=A.P. Holmes |author3=K.J. Worsley |author4=J.-B. Poline |author5=C.D. Frith |author6=R.S.J. Frackowiak | year = 1995| title = Statistical Parametric Maps in functional imaging: A general linear approach| journal = Human Brain Mapping| volume = 2| pages = 189–210| issue = 4}}</ref>

== See also ==
* [[Bayesian multivariate linear regression]]

== Notes ==
{{Reflist}}

== References ==
* {{cite book
|title=Plane Answers to Complex Questions: The Theory of Linear Models|last=Christensen|first=Ronald|location=New York|publisher=Springer|year=2002| edition=Third|isbn=0-387-95361-2}}
* {{cite book|last=Wichura|first=Michael J.|title=The coordinate-free approach to linear models|series=Cambridge Series in Statistical and Probabilistic Mathematics|publisher=Cambridge University Press|location=Cambridge|year=2006|pages=xiv+199|isbn=978-0-521-86842-6|mr=2283455|ref=harv}}
* {{Cite journal | editor1-last = Rawlings | editor1-first = John O. | editor2-first = Sastry G. | editor3-first = David A. | doi = 10.1007/b98890 | title = Applied Regression Analysis | series = Springer Texts in Statistics | year = 1998 | isbn = 0-387-98454-2 | pmid =  | pmc = | editor2-last = Pantula | editor3-last = Dickey }}

{{statistics}}

[[Category:Regression models]]