View source for Asymptotic theory (statistics)

{{Short description|Study of convergence properties of statistical estimators}}In [[statistics]], '''asymptotic theory''', or '''large sample theory''', is a framework for assessing properties of [[estimator]]s and [[statistical test]]s. Within this framework, it is typically assumed that the [[sample size]] {{math|''n''}} grows indefinitely; the properties of estimators and tests are then evaluated in the limit as {{math|{{nowrap|''n'' → ∞}}}}. In practice, a limit evaluation is treated as being approximately valid for large finite sample sizes, as well.

==Overview==
Most statistical problems begin with a dataset of [[sample size|size]] {{math|''n''}}. The asymptotic theory proceeds by assuming that it is possible (in principle) to keep collecting additional data, so that the sample size grows infinitely, i.e. {{math|''n'' → ∞}}. Under the assumption, many results can be obtained that are unavailable for samples of finite size. An example is the [[law of large numbers|weak law of large numbers]]. The law states that for a sequence of [[independent and identically distributed random variables|independent and identically distributed]] (IID) [[random variable]]s {{math|''X''<sub>1</sub>, ''X''<sub>2</sub>, …}}, if one value is drawn from each random variable and the average of the first {{math|''n''}} values is computed as {{math|{{overline|''X''}}<sub>''n''</sub>}}, then the {{math|{{overline|''X''}}<sub>''n''</sub>}} [[Convergence of random variables#Convergence in probability|converge in probability]] to the population mean {{math|E[''X<sub>i</sub>'']}} as {{math|{{nowrap|''n'' → ∞}}}}.

In asymptotic theory, the standard approach is {{math|''n'' → ∞}}. For some [[statistical model]]s, slightly different approaches of asymptotics may be used. For example, with [[panel data]], it is commonly assumed that one dimension in the data remains fixed, whereas the other dimension grows: {{math|{{nowrap|''T'' {{=}} constant}}}} and {{math|''N'' → ∞}}, or vice versa.

Besides the standard approach to asymptotics, other alternative approaches exist:
* Within the [[local asymptotic normality]] framework, it is assumed that the value of the "true parameter" in the model varies slightly with {{math|''n''}}, such that the {{math|''n''}}-th model corresponds to {{math| ''θ<sub>n</sub>'' {{=}} ''θ'' + ''h''/{{sqrt|''n''}} }}. This approach lets us study the [[regular estimator|regularity of estimators]].
* When [[statistical test]]s are studied for their power to distinguish against the alternatives that are close to the null hypothesis, it is done within the so-called "local alternatives" framework: the null hypothesis is {{math|''H''<sub>0</sub>: ''θ'' {{=}} ''θ''<sub>0</sub>}} and the alternative is {{math|''H''<sub>1</sub>: ''θ'' {{=}} ''θ''<sub>0</sub> + ''h''/{{sqrt|''n''}} }}. This approach is especially popular for the [[unit root test]]s.
* There are models where the dimension of the parameter space {{math|Θ<sub>''n''</sub>}} slowly expands with {{math|''n''}}, reflecting the fact that the more observations there are, the more structural effects can be feasibly incorporated in the model.
* In [[kernel density estimation]] and [[kernel regression]], an additional parameter is assumed—the bandwidth {{math|''h''}}. In those models, it is typically taken that {{math|{{nowrap|''h'' → 0}}}} as {{math|{{nowrap|''n'' → ∞}}}}. The rate of convergence must be chosen carefully, though, usually {{math|''h'' ∝ ''n''<sup>−1/5</sup>}}.

In many cases, highly accurate results for finite samples can be obtained via numerical methods (i.e. computers); even in such cases, though, asymptotic analysis can be useful. This point was made by {{Harvtxt|Small|2010|loc=§1.4}}, as follows.
{{quote|text=A primary goal of asymptotic analysis is to obtain a deeper ''qualitative'' understanding of ''quantitative'' tools. The conclusions of an asymptotic analysis often supplement the conclusions which can be obtained by numerical methods.}}

==Modes of convergence of random variables==
{{further|Convergence of random variables}}

==Asymptotic properties==

===Estimators===

====''[[Consistent estimator|Consistency]]''====
A sequence of estimates is said to be ''consistent'', if it [[converges in probability]] to the true value of the parameter being estimated:
: <math>\hat\theta_n\ \xrightarrow{\overset{}p}\ \theta_0.</math>
That is, roughly speaking with an infinite amount of data the [[estimator]] (the formula for generating the estimates) would almost surely give the correct result for the parameter being estimated.

====''[[Efficiency (statistics)#Asymptotic efficiency|Efficiency]]''====
{{expand section|date=November 2017}}

====''[[Asymptotic distribution]]''====
If it is possible to find sequences of non-random constants {{math|&#123;''a<sub>n</sub>''&#125;}}, {{math|&#123;''b<sub>n</sub>''&#125;}} (possibly depending on the value of {{math|''θ''<sub>0</sub>}}), and a non-degenerate distribution {{math|''G''}} such that
: <math>b_n(\hat\theta_n - a_n)\ \xrightarrow{d}\ G ,</math>
then the sequence of estimators <math>\textstyle\hat\theta_n</math> is said to have the ''[[asymptotic distribution]]'' ''G''.

Most often, the estimators encountered in practice are [[Estimator#Asymptotic normality|asymptotically normal]], meaning their asymptotic distribution is the [[normal distribution]], with {{math|{{nowrap|''a<sub>n</sub>'' {{=}} ''θ''<sub>0</sub>}}}}, {{math|''b<sub>n</sub>'' {{=}} {{sqrt|''n''}}}}, and {{math|{{nowrap|''G'' {{=}} [[normal distribution|''N''(0, ''V'')]]}}}}:
: <math>\sqrt{n}(\hat\theta_n - \theta_0)\ \xrightarrow{d}\ \mathcal{N}(0, V).</math>

====''Asymptotic [[confidence region]]s''====
====''[[Regular estimator|Regularity]]''====

==Asymptotic theorems==
* [[Central limit theorem]]
* [[Continuous mapping theorem]] 
* [[Glivenko–Cantelli theorem]]
* [[Law of large numbers]]
* [[Law of the iterated logarithm]]
* [[Slutsky’s theorem]]

==See also==
*[[Asymptotic analysis]] 
*[[Exact statistics]]
*[[Large deviations theory]]

==Notes==
{{reflist}}

{{no footnotes|date=April 2013}}

==References==
{{refbegin}}
* {{citation | editor1-last= Balakrishnan | editor1-first= N. | editor2-last= Ibragimov | editor2-first= I. A. V. B. | editor3-last= Nevzorov | editor3-first= V. B. | year=2001 | title= Asymptotic Methods in Probability and Statistics with Applications | publisher= [[Birkhäuser]]|url=https://books.google.com/books?id=v2ngBwAAQBAJ&printsec=frontcover#v=onepage&q&f=false}}
* {{citation | title= Asymptotic Analysis of Random Walks | last1= Borovkov | last2= Borovkov | first1= A. A. | authorlink1= Aleksandr Alekseevich Borovkov | first2= K. A. | year= 2010 | publisher= [[Cambridge University Press]]|url=https://books.google.com/books?hl=en&lr=&id=jXEwIQ4wOAwC&oi=fnd&pg=PR11&dq=%22Asymptotic+Analysis+of+Random+Walks%22&ots=LgK3yt0wgY&sig=shdEslD4ngZkeBSHGpWjjh-Fl6o}}
* {{citation | title= Asymptotic Behaviour of Linearly Transformed Sums of Random Variables | last1= Buldygin | first1= V. V. | last2= Solntsev | first2= S. | year= 1997 | publisher= Springer|url=https://books.google.com/books?hl=en&lr=&id=1UrzCAAAQBAJ&oi=fnd&pg=PR9&dq=%22Asymptotic+Behaviour+of+Linearly+Transformed+Sums+of+Random+Variables%22&ots=5ZSVeuw_S4&sig=rUIXtSgSBQK6it-NlP85XmoUaC0#v=onepage&q=%22Asymptotic%20Behaviour%20of%20Linearly%20Transformed%20Sums%20of%20Random%20Variables%22&f=false}}
* {{citation
  | last1 = Le Cam | first1 = Lucien | authorlink1 = Lucien Le Cam
  | last2 = Yang | first2 = Grace Lo | author2-link = Grace Yang
  | title = Asymptotics in Statistics 
  | year = 2000
  | edition = 2nd
  | publisher = Springer
  }}
* {{citation| last=DasGupta | first= A. | year=2008 | title=Asymptotic Theory of Statistics and Probability | publisher=Springer}}
* {{citation| title= Asymptotic Laws and Methods in Stochastics | year= 2015| publisher= [[Springer-Verlag]] | editor1-last= Dawson | editor1-first= D. | editor2-last= Kulik | editor2-first= R. | editor3-last= Ould Haye | editor3-first= M. | editor4-last= Szyszkowicz | editor4-first= B. | editor5-last= Zhao | editor5-first= Y.}}
* {{citation| last=Höpfner | first= R. | year=2014 | title=Asymptotic Statistics | publisher=[[Walter de Gruyter]]}}
* {{citation | title= Asymptotic Statistical Methods for Stochastic Processes | last= Lin'kov | first= Yu. N. | year= 2001 | publisher= [[American Mathematical Society]]}}
* {{citation | title= Asymptotics for Associated Random Variables | year= 2012 | first= P. E. | last=  Oliveira | publisher= Springer}}
* {{citation | first= V. V. | last= Petrov | title= Limit Theorems of Probability Theory| year= 1995 | publisher= [[Oxford University Press]]}}
* {{citation | last1=Sen | first1= P. K. | last2= Singer | first2= J. M. |last3=  Pedroso de Lima | first3= A. C. | year=2009 | title= From Finite Sample to Asymptotic Methods in Statistics | publisher= [[Cambridge University Press]]}}
* {{citation | last1= Shiryaev | first1= A. N. | last2=Spokoiny | first2=V. G. | year=2000 | title= Statistical Experiments and Decisions: Asymptotic theory| publisher= [[World Scientific]]}}
* {{citation| last=Small | first= C. G. | year= 2010 | title=Expansions and Asymptotics for Statistics | publisher= [[Chapman & Hall]]}}
* {{citation
  | last = van der Vaart | first = A. W.
  | title = Asymptotic Statistics
  | year = 1998
  | publisher = [[Cambridge University Press]]
  }}
{{refend}}

{{Statistics|inference|collapsed}}

{{Authority control}}

[[Category:Asymptotic theory (statistics)| ]]