Jump to content
 







Main menu
   


Navigation  



Main page
Contents
Current events
Random article
About Wikipedia
Contact us
Donate
 




Contribute  



Help
Learn to edit
Community portal
Recent changes
Upload file
 








Search  



























Create account

Log in
 









Create account
 Log in
 




Pages for logged out editors learn more  



Contributions
Talk
 



















Contents

   



(Top)
 


1 Definitions  





2 The theorem  



2.1  Mean-squared-error version  





2.2  Convex loss generalization  







3 Properties  





4 Example  





5 Idempotence  





6 Completeness and LehmannScheffé minimum variance  





7 See also  





8 References  





9 External links  














RaoBlackwell theorem






Deutsch
Español
Français
Italiano
עברית
Polski
Русский
Sunda
Українська

 

Edit links
 









Article
Talk
 

















Read
Edit
View history
 








Tools
   


Actions  



Read
Edit
View history
 




General  



What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Cite this page
Get shortened URL
Download QR code
Wikidata item
 




Print/export  



Download as PDF
Printable version
 


















From Wikipedia, the free encyclopedia
 


Instatistics, the Rao–Blackwell theorem, sometimes referred to as the Rao–Blackwell–Kolmogorov theorem, is a result that characterizes the transformation of an arbitrarily crude estimator into an estimator that is optimal by the mean-squared-error criterion or any of a variety of similar criteria.

The Rao–Blackwell theorem states that if g(X) is any kind of estimator of a parameter θ, then the conditional expectationofg(X) given T(X), where T is a sufficient statistic, is typically a better estimator of θ, and is never worse. Sometimes one can very easily construct a very crude estimator g(X), and then evaluate that conditional expected value to get an estimator that is in various senses optimal.

The theorem is named after C.R. Rao and David Blackwell. The process of transforming an estimator using the Rao–Blackwell theorem can be referred to as Rao–Blackwellization. The transformed estimator is called the Rao–Blackwell estimator.[1][2][3]

Definitions[edit]

In other words, a sufficient statistic T(X) for a parameter θ is a statistic such that the conditional probability of the data X, given T(X), does not depend on the parameter θ.

The theorem[edit]

Mean-squared-error version[edit]

One case of Rao–Blackwell theorem states:

The mean squared error of the Rao–Blackwell estimator does not exceed that of the original estimator.

In other words,

The essential tools of the proof besides the definition above are the law of total expectation and the fact that for any random variable Y, E(Y2) cannot be less than [E(Y)]2. That inequality is a case of Jensen's inequality, although it may also be shown to follow instantly from the frequently mentioned fact that

More precisely, the mean square error of the Rao-Blackwell estimator has the following decomposition[4]

Since , the Rao-Blackwell theorem immediately follows.

Convex loss generalization[edit]

The more general version of the Rao–Blackwell theorem speaks of the "expected loss" or risk function:

where the "loss function" L may be any convex function. If the loss function is twice-differentiable, as in the case for mean-squared-error, then we have the sharper inequality[4]

Properties[edit]

The improved estimator is unbiased if and only if the original estimator is unbiased, as may be seen at once by using the law of total expectation. The theorem holds regardless of whether biased or unbiased estimators are used.

The theorem seems very weak: it says only that the Rao–Blackwell estimator is no worse than the original estimator. In practice, however, the improvement is often enormous.[5]

Example[edit]

Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers X1, ..., Xn of phone calls that arrived during n successive one-minute periods are observed. It is desired to estimate the probability e−λ that the next one-minute period passes with no phone calls.

Anextremely crude estimator of the desired probability is

i.e., it estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise. Despite the apparent limitations of this estimator, the result given by its Rao–Blackwellization is a very good estimator.

The sum

can be readily shown to be a sufficient statistic for λ, i.e., the conditional distribution of the data X1, ..., Xn, depends on λ only through this sum. Therefore, we find the Rao–Blackwell estimator

After doing some algebra we have

Since the average number of calls arriving during the first n minutes is nλ, one might not be surprised if this estimator has a fairly high probability (ifn is big) of being close to

So δ1 is clearly a very much improved estimator of that last quantity. In fact, since Sniscomplete and δ0 is unbiased, δ1 is the unique minimum variance unbiased estimator by the Lehmann–Scheffé theorem.

Idempotence[edit]

Rao–Blackwellization is an idempotent operation. Using it to improve the already improved estimator does not obtain a further improvement, but merely returns as its output the same improved estimator.

Completeness and Lehmann–Scheffé minimum variance[edit]

If the conditioning statistic is both complete and sufficient, and the starting estimator is unbiased, then the Rao–Blackwell estimator is the unique "best unbiased estimator": see Lehmann–Scheffé theorem.

An example of an improvable Rao–Blackwell improvement, when using a minimal sufficient statistic that is not complete, was provided by Galili and Meilijson in 2016.[6] Let be a random sample from a scale-uniform distribution with unknown mean and known design parameter . In the search for "best" possible unbiased estimators for it is natural to consider as an initial (crude) unbiased estimator for and then try to improve it. Since is not a function of , the minimal sufficient statistic for (where and ), it may be improved using the Rao–Blackwell theorem as follows:

However, the following unbiased estimator can be shown to have lower variance:

And in fact, it could be even further improved when using the following estimator:

The model is a scale model. Optimal equivariant estimators can then be derived for loss functions that are invariant.[7]

See also[edit]

References[edit]

  1. ^ Blackwell, D. (1947). "Conditional expectation and unbiased sequential estimation". Annals of Mathematical Statistics. 18 (1): 105–110. doi:10.1214/aoms/1177730497. MR 0019903. Zbl 0033.07603.
  • ^ Kolmogorov, A. N. (1950). "Unbiased estimates". Izvestiya Akad. Nauk SSSR. Ser. Mat. 14: 303–326. MR 0036479.
  • ^ Rao, C. Radhakrishna (1945). "Information and accuracy attainable in the estimation of statistical parameters". Bulletin of the Calcutta Mathematical Society. 37 (3): 81–91.
  • ^ a b J. G. Liao; A. Berg (22 June 2018). "Sharpening Jensen's Inequality". The American Statistician. 73 (3): 278–281. arXiv:1707.08644. doi:10.1080/00031305.2017.1419145. S2CID 88515366.
  • ^ Carpenter, Bob (January 20, 2020). "Rao-Blackwellization and discrete parameters in Stan". Statistical Modeling, Causal Inference, and Social Science. Retrieved September 13, 2021. The Rao-Blackwell theorem states that the marginalization approach has variance less than or equal to the direct approach. In practice, this difference can be enormous.
  • ^ Tal Galili; Isaac Meilijson (31 Mar 2016). "An Example of an Improvable Rao–Blackwell Improvement, Inefficient Maximum Likelihood Estimator, and Unbiased Generalized Bayes Estimator". The American Statistician. 70 (1): 108–113. doi:10.1080/00031305.2015.1100683. PMC 4960505. PMID 27499547.
  • ^ Taraldsen, Gunnar (2020). "Micha Mandel (2020), "The Scaled Uniform Model Revisited," The American Statistician, 74:1, 98–100: Comment". The American Statistician. 74 (3): 315. doi:10.1080/00031305.2020.1769727. ISSN 0003-1305. S2CID 219493070.
  • External links[edit]


    Retrieved from "https://en.wikipedia.org/w/index.php?title=Rao–Blackwell_theorem&oldid=1185878360"

    Categories: 
    Theorems in statistics
    Estimation theory
    Hidden categories: 
    Articles with short description
    Short description is different from Wikidata
    Articles needing additional references from May 2014
    All articles needing additional references
     



    This page was last edited on 19 November 2023, at 15:10 (UTC).

    Text is available under the Creative Commons Attribution-ShareAlike License 4.0; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.



    Privacy policy

    About Wikipedia

    Disclaimers

    Contact Wikipedia

    Code of Conduct

    Developers

    Statistics

    Cookie statement

    Mobile view



    Wikimedia Foundation
    Powered by MediaWiki