View source for Logit

{{Hatnote|This article discusses the binary logit function only. See [[discrete choice]] for a discussion of [[multinomial logit]], conditional logit, nested logit, [[mixed logit]], exploded logit, and [[ordered logit]]. For the basic regression technique that uses the logit function, see [[logistic regression]].}}

[[Image:Logit.svg|thumbnail|right|350px|Plot of logit(''p'') in the domain of 0 to 1, where the base of logarithm is ''e'']]

In statistics, the '''logit''' ({{IPAc-en|ˈ|l|oʊ|dʒ|ɪ|t}} {{respell|LOH|jit}}) function or the '''log-odds''' is the [[logarithm]] of the [[odds]] <math>\frac{p}{1-p}</math> where {{mvar|p}} is probability.<ref>{{cite web|url=http://itl.nist.gov/div898/software/dataplot/refman2/auxillar/logoddra.htm|title=LOG ODDS RATIO|work=nist.gov}}</ref> It is a type of function that creates a map of probability values from <math>[0, 1]</math> to <math>(-\infty, +\infty)</math><ref>{{Cite web|url=http://www.columbia.edu/~so33/SusDev/Lecture_9.pdf|title=Logit/Probit}}</ref>. It is the [[inverse function|inverse]] of the [[sigmoid function|sigmoidal]] [[logistic function|"logistic" function or logistic transform]] used in [[mathematics]], especially in [[Data transformation (statistics)|statistics]].

In [[deep learning]], the term '''logits layer''' is popularly used for the last neuron layer of neural networks used for classification tasks, which produce raw prediction values as real numbers ranging from <math>(-\infty, +\infty)</math><ref>{{Cite web|url=https://www.tensorflow.org/tutorials/estimators/cnn#logits_layer|title=Tensorflow - Logit Layer}}</ref>.

==Definition==

If ''p'' is a [[probability]], then {{nowrap|''p''/(1 &minus; ''p'')}} is the corresponding [[odds]]; the logit of the probability is the logarithm of the odds, i.e.

:<math>\operatorname{logit}(p)=\log\left( \frac{p}{1-p} \right) =\log(p)-\log(1-p)=-\log\left( \frac{1}{p} - 1\right).</math>

The base of the [[logarithm]] function used is of little importance in the present article, as long as it is greater than 1, but the [[natural logarithm]] with base [[e (mathematical constant)|e]] is the one most often used. The choice of base corresponds to the choice of [[logarithmic unit]] for the value: base 2 corresponds to a [[shannon (unit)|shannon]], base e to a [[nat (unit)|nat]], and base 10 to a [[hartley (unit)|hartley]]; these units are particularly used in information-theoretic interpretations. For each choice of base, the logit function takes values between negative and positive infinity.

The [[logistic function|"logistic" function]] of any number <math>\alpha</math> is given by the inverse-logit:

:<math>\operatorname{logit}^{-1}(\alpha) = \operatorname{logistic}(\alpha) = \frac{1}{1 + \operatorname{exp}(-\alpha)} = \frac{\operatorname{exp}(\alpha)}{ \operatorname{exp}(\alpha) + 1}</math>

The difference between the logits of two probabilities is the logarithm of the [[odds ratio]] (''R''), thus providing a shorthand for writing the correct combination of odds ratios [[additive function|only by adding and subtracting]]:

:<math>\operatorname{log}(R)=\log\left( \frac{{p_1}/(1-p_1)}{{p_2}/(1-p_2)} \right) =\log\left( \frac{p_1}{1-p_1} \right) - \log\left(\frac{p_2}{1-p_2}\right)=\operatorname{logit}(p_1)-\operatorname{logit}(p_2).</math>

==History==
There have been several efforts to adapt linear regression methods to domain where output is probability value <math>[0, 1]</math> instead of any real number <math>[-\infty, +\infty]</math>. Many of such efforts focused on modeling this problem by somehow mapping the range <math>[0, 1]</math> to <math>(-\infty, +\infty)</math> and then running the linear regression on these transformed values. In 1934 [[Chester Ittner Bliss]] used the cumulative normal distribution function to perform this mapping and called his model [[probit]] an abbreviation for "'''prob'''ability un'''it'''";<ref name="Cramer2003">{{Cite web|url=http://www.cambridge.org/resources/0521815886/1208_default.pdf|title=The origins and development of the logit model|author=J. S. Cramer|year=2003|publisher=Cambridge UP}}</ref> . However, this is computationally more expensive. In 1944, [[Joseph Berkson]] used log of odds and called this function ''logit,'' abbreviation for "'''log'''istic un'''it'''" following the analogy for probit. Log odds was used extensively by [[Charles Sanders Peirce]] (late 19th century).<ref>{{cite book |title=The history of statistics : the measurement of uncertainty before 1900 |last=Stigler |first=Stephen M. |authorlink=Stephen M. Stigler |year=1986 |publisher=Belknap Press of Harvard University Press |location=Cambridge, Massachusetts |isbn=978-0-674-40340-6 |ref=harv |url-access=registration |url=https://archive.org/details/historyofstatist00stig }}</ref> . [[G. A. Barnard]] in 1949 coined the commonly used term ''log-odds'';<ref>{{citation|title=Logistic Regression Models|first=Joseph M.|last=Hilbe|publisher=CRC Press|year=2009|isbn=9781420075779|page=3|url=https://books.google.com/books?id=tmHMBQAAQBAJ&pg=PA3}}.</ref> the log-odds of an event is the logit of the probability of the event.<ref>{{citation|title=Logit Models from Economics and Other Fields|first=J. S.|last=Cramer|publisher=Cambridge University Press|year=2003|isbn=9781139438193|page=13|url=https://books.google.com/books?id=1Od2d72pPXUC&pg=PA13}}.</ref>

==Uses and properties==

* The '''logit''' in [[logistic regression]] is a special case of a link function in a [[generalized linear model]]: it is the canonical [[link function]] for the [[Bernoulli distribution]].
* The '''logit''' function is the negative of the [[derivative]] of the [[binary entropy function]].
* The '''logit''' is also central to the probabilistic [[Rasch model]] for [[measurement]], which has applications in psychological and educational assessment, among other areas.
* The '''inverse-logit''' function (i.e., the [[logistic function]]) is also sometimes referred to as the ''expit'' function.<ref>{{cite web |url=http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/msm/html/expit.html |title=Archived copy |accessdate=2011-02-18 |url-status=dead |archiveurl=https://web.archive.org/web/20110706132209/http://www.stat.ucl.ac.be/ISdidactique/Rhelp/library/msm/html/expit.html |archivedate=2011-07-06 }}</ref>
* In '''plant disease epidemiology''' the logit is used to fit the data to a logistic model. With the Gompertz and Monomolecular models all three are known as Richards family models.
* The log-odds function of probabilities is often used in '''state estimation''' algorithms<ref>{{Cite journal|last=Thrun|first=Sebastian|title=Learning Occupancy Grid Maps with Forward Sensor Models|journal=Autonomous Robots|language=en|volume=15|issue=2|pages=111–127|doi=10.1023/A:1025584807625|issn=0929-5593|year=2003}}</ref> because of its numerical advantages in the case of small probabilities. Instead of multiplying very small floating point numbers, log-odds probabilities can just be summed up to calculate the (log-odds) joint probability.<ref>{{Cite web|url=https://www.cs.cmu.edu/~16831-f12/notes/F12/16831_lecture05_vh.pdf|title=Statistical Techniques in Robotics|last=Styler|first=Alex|date=2012|website=|publisher=|page=2|access-date=2017-01-26}}</ref><ref>{{Cite journal|last=Dickmann|first=J.|last2=Appenrodt|first2=N.|last3=Klappstein|first3=J.|last4=Bloecher|first4=H. L.|last5=Muntzinger|first5=M.|last6=Sailer|first6=A.|last7=Hahn|first7=M.|last8=Brenk|first8=C.|date=2015-01-01|title=Making Bertha See Even More: Radar Contribution|url=https://ieeexplore.ieee.org/document/7161279/|journal=IEEE Access|volume=3|pages=1233–1247|doi=10.1109/ACCESS.2015.2454533|issn=2169-3536}}</ref>

== Comparison with probit ==
[[File:Logit-probit.svg|right|300px|thumb|Comparison of the [[logit function]] with a scaled probit (i.e. the inverse [[cumulative distribution function|CDF]] of the [[normal distribution]]), comparing <math>\operatorname{logit}(x)</math> vs. <math>\Phi^{-1}(x)/\sqrt{\frac{\pi}{8}}</math>, which makes the slopes the same at the y-origin.]]

Closely related to the logit function (and [[logit model]]) are the [[probit function]] and [[probit model]]. The logit and probit are both [[sigmoid function]]s with a domain between 0 and 1, which makes them both [[quantile function]]s—i.e., inverses of the [[cumulative distribution function]] (CDF) of a [[probability distribution]]. In fact, the logit is the quantile function of the [[logistic distribution]], while the probit is the quantile function of the [[normal distribution]].  The probit function is denoted <math>\Phi^{-1}(x)</math>, where <math>\Phi(x)</math> is the CDF of the normal distribution, as just mentioned:

:<math>\Phi(x) = \frac{1}{\sqrt{2\pi}}\int_{-\infty}^{x}  e^{-\frac{y^2}{2}} dy.</math>

As shown in the graph, the logit and probit functions are extremely similar, particularly when the probit function is scaled so that its slope at y=0 matches the slope of the logit. As a result, [[probit model]]s are sometimes used in place of [[logit model]]s because for certain applications (e.g., in [[Bayesian statistics]]) the implementation is easier.

== See also ==
* [[Discrete choice]] on binary logit, multinomial logit, conditional logit, nested logit, mixed logit, exploded logit, and ordered logit
* [[Limited dependent variable]]
* [[Daniel McFadden]], a [[Nobel Prize in Economics]] winner for development of a particular logit model used in economics<ref name=Cramer2003/>
* [[Logit analysis (in marketing)|Logit analysis in marketing]]
* [[Multinomial logit]]
* [[Ogee]], curve with similar shape
* [[Perceptron]]
* [[Probit]], another function with the same domain and range as the logit
* [[Ridit scoring]]
* [[Data transformation (statistics)]]
* [[Arcsin]] (transformation)

== References ==
{{More footnotes|date=November 2010}}

{{Reflist}}

==Further reading==
*{{cite book|last=Ashton|first=Winifred D.|title=The Logit Transformation: with special reference to its uses in Bioassay|year=1972|publisher=Charles Griffin|isbn=978-0-85264-212-2|series=Griffin's Statistical Monographs & Courses|volume= 32}}

[[Category:Logarithms]]
[[Category:Special functions]]