Jump to content
 







Main menu
   


Navigation  



Main page
Contents
Current events
Random article
About Wikipedia
Contact us
Donate
 




Contribute  



Help
Learn to edit
Community portal
Recent changes
Upload file
 








Search  

































Create account

Log in
 









Create account
 Log in
 




Pages for logged out editors learn more  



Contributions
Talk
 



















Contents

   



(Top)
 


1 Introduction  





2 Definition  





3 Generalizations  





4 Examples  





5 Properties  



5.1  Continuity  





5.2  Inversion formula  





5.3  Criteria for characteristic functions  







6 Uses  



6.1  Basic manipulations of distributions  





6.2  Moments  





6.3  Data analysis  





6.4  Example  







7 Entire characteristic functions  





8 Related concepts  





9 See also  





10 Notes  





11 References  



11.1  Citations  





11.2  Sources  







12 External links  














Characteristic function (probability theory)






العربية
Български
Català
Čeština
Deutsch
Español
Esperanto
Français
Galego

Italiano
עברית
Magyar
Македонски
Nederlands

Polski
Português
Русский
Shqip
Slovenčina
Slovenščina
Sunda
Svenska
Türkçe
Українська
Tiếng Vit

 

Edit links
 









Article
Talk
 

















Read
Edit
View history
 








Tools
   


Actions  



Read
Edit
View history
 




General  



What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Cite this page
Get shortened URL
Download QR code
Wikidata item
 




Print/export  



Download as PDF
Printable version
 
















Appearance
   

 






From Wikipedia, the free encyclopedia
 


The characteristic function of a uniform U(–1,1) random variable. This function is real-valued because it corresponds to a random variable that is symmetric around the origin; however characteristic functions may generally be complex-valued.

Inprobability theory and statistics, the characteristic function of any real-valued random variable completely defines its probability distribution. If a random variable admits a probability density function, then the characteristic function is the Fourier transform (with sign reversal) of the probability density function. Thus it provides an alternative route to analytical results compared with working directly with probability density functionsorcumulative distribution functions. There are particularly simple results for the characteristic functions of distributions defined by the weighted sums of random variables.

In addition to univariate distributions, characteristic functions can be defined for vector- or matrix-valued random variables, and can also be extended to more generic cases.

The characteristic function always exists when treated as a function of a real-valued argument, unlike the moment-generating function. There are relations between the behavior of the characteristic function of a distribution and properties of the distribution, such as the existence of moments and the existence of a density function.

Introduction[edit]

The characteristic function is a way to describe a random variable. The characteristic function,

a function of t, completely determines the behavior and properties of the probability distribution of the random variable X. The characteristic function is similar to the cumulative distribution function,

(where 1{X ≤ x} is the indicator function — it is equal to 1 in the event that Xx, and zero otherwise), which also completely determines the behavior and properties of the probability distribution of the random variable X. The two approaches are equivalent in the sense that knowing one of the functions it is always possible to find the other, yet they provide different insights for understanding the features of the random variable. Moreover, in particular cases, there can be differences in whether these functions can be represented as expressions involving simple standard functions.

If a random variable admits a density function, then the characteristic function is its Fourier dual, in the sense that each of them is a Fourier transform of the other. If a random variable has a moment-generating function , then the domain of the characteristic function can be extended to the complex plane, and

[1]

Note however that the characteristic function of a distribution always exists, even when the probability density functionormoment-generating function do not.

The characteristic function approach is particularly useful in analysis of linear combinations of independent random variables: a classical proof of the Central Limit Theorem uses characteristic functions and Lévy's continuity theorem. Another important application is to the theory of the decomposability of random variables.

Definition[edit]

For a scalar random variable X the characteristic function is defined as the expected valueofeitX, where i is the imaginary unit, and tR is the argument of the characteristic function:

Here FX is the cumulative distribution functionofX, fX is the corresponding probability density function, QX(p) is the corresponding inverse cumulative distribution function also called the quantile function,[2] and the integrals are of the Riemann–Stieltjes kind. If a random variable X has a probability density function then the characteristic function is its Fourier transform with sign reversal in the complex exponential[3][page needed].[4] This convention for the constants appearing in the definition of the characteristic function differs from the usual convention for the Fourier transform.[5] For example, some authors[6] define φX(t) = E[e−2πitX], which is essentially a change of parameter. Other notation may be encountered in the literature: as the characteristic function for a probability measure p, or as the characteristic function corresponding to a density f.

Generalizations[edit]

The notion of characteristic functions generalizes to multivariate random variables and more complicated random elements. The argument of the characteristic function will always belong to the continuous dual of the space where the random variable X takes its values. For common cases such definitions are listed below:

Examples[edit]

Distribution Characteristic function
Degenerate δa
Bernoulli Bern(p)
Binomial B(n, p)
Negative binomial NB(r, p)
Poisson Pois(λ)
Uniform (continuous) U(a, b)
Uniform (discrete) DU(a, b)
Laplace L(μ, b)
Logistic Logistic(μ,s)
Normal N(μ, σ2)
Chi-squared χ2k
Noncentral chi-squared
Generalized chi-squared
Cauchy C(μ, θ)
Gamma Γ(k, θ)
Exponential Exp(λ)
Geometric Gf(p)
(number of failures)
Geometric Gt(p)
(number of trials)
Multivariate normal N(μ, Σ)
Multivariate Cauchy MultiCauchy(μ, Σ)[10]

Oberhettinger (1973) provides extensive tables of characteristic functions.

Properties[edit]

Continuity[edit]

The bijection stated above between probability distributions and characteristic functions is sequentially continuous. That is, whenever a sequence of distribution functions Fj(x) converges (weakly) to some distribution F(x), the corresponding sequence of characteristic functions φj(t) will also converge, and the limit φ(t) will correspond to the characteristic function of law F. More formally, this is stated as

Lévy’s continuity theorem: A sequence Xjofn-variate random variables converges in distribution to random variable X if and only if the sequence φXj converges pointwise to a function φ which is continuous at the origin. Where φ is the characteristic function of X.[13]

This theorem can be used to prove the law of large numbers and the central limit theorem.

Inversion formula[edit]

There is a one-to-one correspondence between cumulative distribution functions and characteristic functions, so it is possible to find one of these functions if we know the other. The formula in the definition of characteristic function allows us to compute φ when we know the distribution function F (or density f). If, on the other hand, we know the characteristic function φ and want to find the corresponding distribution function, then one of the following inversion theorems can be used.

Theorem. If the characteristic function φX of a random variable Xisintegrable, then FX is absolutely continuous, and therefore X has a probability density function. In the univariate case (i.e. when X is scalar-valued) the density function is given by

In the multivariate case it is

where is the dot product.

The density function is the Radon–Nikodym derivative of the distribution μX with respect to the Lebesgue measure λ:

Theorem (Lévy).[note 1]IfφX is characteristic function of distribution function FX, two points a < b are such that {x | a < x < b} is a continuity setofμX (in the univariate case this condition is equivalent to continuity of FX at points a and b), then

Theorem. If a is (possibly) an atom of X (in the univariate case this means a point of discontinuity of FX) then

Theorem (Gil-Pelaez).[16] For a univariate random variable X, if x is a continuity pointofFX then

where the imaginary part of a complex number is given by .

And its density function is:

The integral may be not Lebesgue-integrable; for example, when X is the discrete random variable that is always 0, it becomes the Dirichlet integral.

Inversion formulas for multivariate distributions are available.[14][17]

Criteria for characteristic functions[edit]

The set of all characteristic functions is closed under certain operations:

It is well known that any non-decreasing càdlàg function F with limits F(−∞) = 0, F(+∞) = 1 corresponds to a cumulative distribution function of some random variable. There is also interest in finding similar simple criteria for when a given function φ could be the characteristic function of some random variable. The central result here is Bochner’s theorem, although its usefulness is limited because the main condition of the theorem, non-negative definiteness, is very hard to verify. Other theorems also exist, such as Khinchine’s, Mathias’s, or Cramér’s, although their application is just as difficult. Pólya’s theorem, on the other hand, provides a very simple convexity condition which is sufficient but not necessary. Characteristic functions which satisfy this condition are called Pólya-type.[18]

Bochner’s theorem. An arbitrary function φ : RnC is the characteristic function of some random variable if and only if φispositive definite, continuous at the origin, and if φ(0) = 1.

Khinchine’s criterion. A complex-valued, absolutely continuous function φ, with φ(0) = 1, is a characteristic function if and only if it admits the representation

Mathias’ theorem. A real-valued, even, continuous, absolutely integrable function φ, with φ(0) = 1, is a characteristic function if and only if

for n = 0,1,2,..., and all p > 0. Here H2n denotes the Hermite polynomial of degree 2n.

Pólya’s theorem can be used to construct an example of two random variables whose characteristic functions coincide over a finite interval but are different elsewhere.

Pólya’s theorem. If is a real-valued, even, continuous function which satisfies the conditions

then φ(t) is the characteristic function of an absolutely continuous distribution symmetric about 0.

Uses[edit]

Because of the continuity theorem, characteristic functions are used in the most frequently seen proof of the central limit theorem. The main technique involved in making calculations with a characteristic function is recognizing the function as the characteristic function of a particular distribution.

Basic manipulations of distributions[edit]

Characteristic functions are particularly useful for dealing with linear functions of independent random variables. For example, if X1, X2, ..., Xn is a sequence of independent (and not necessarily identically distributed) random variables, and

where the ai are constants, then the characteristic function for Sn is given by

In particular, φX+Y(t) = φX(t)φY(t). To see this, write out the definition of characteristic function:

The independence of X and Y is required to establish the equality of the third and fourth expressions.

Another special case of interest for identically distributed random variables is when ai = 1 / n and then Sn is the sample mean. In this case, writing X for the mean,

Moments[edit]

Characteristic functions can also be used to find moments of a random variable. Provided that the n-th moment exists, the characteristic function can be differentiated n times:

This can be formally written using the derivatives of the Dirac delta function:which allows a formal solution to the moment problem. For example, suppose X has a standard Cauchy distribution. Then φX(t) = e−|t|. This is not differentiableatt = 0, showing that the Cauchy distribution has no expectation. Also, the characteristic function of the sample mean Xofn independent observations has characteristic function φX(t) = (e−|t|/n)n = e−|t|, using the result from the previous section. This is the characteristic function of the standard Cauchy distribution: thus, the sample mean has the same distribution as the population itself.

As a further example, suppose X follows a Gaussian distribution i.e. . Then and

A similar calculation shows and is easier to carry out than applying the definition of expectation and using integration by parts to evaluate .

The logarithm of a characteristic function is a cumulant generating function, which is useful for finding cumulants; some instead define the cumulant generating function as the logarithm of the moment-generating function, and call the logarithm of the characteristic function the second cumulant generating function.

Data analysis[edit]

Characteristic functions can be used as part of procedures for fitting probability distributions to samples of data. Cases where this provides a practicable option compared to other possibilities include fitting the stable distribution since closed form expressions for the density are not available which makes implementation of maximum likelihood estimation difficult. Estimation procedures are available which match the theoretical characteristic function to the empirical characteristic function, calculated from the data. Paulson et al. (1975)[19] and Heathcote (1977)[20] provide some theoretical background for such an estimation procedure. In addition, Yu (2004)[21] describes applications of empirical characteristic functions to fit time series models where likelihood procedures are impractical. Empirical characteristic functions have also been used by Ansari et al. (2020)[22] and Li et al. (2020)[23] for training generative adversarial networks.

Example[edit]

The gamma distribution with scale parameter θ and a shape parameter k has the characteristic function

Now suppose that we have

with X and Y independent from each other, and we wish to know what the distribution of X + Y is. The characteristic functions are

which by independence and the basic properties of characteristic function leads to

This is the characteristic function of the gamma distribution scale parameter θ and shape parameter k1 + k2, and we therefore conclude

The result can be expanded to n independent gamma distributed random variables with the same scale parameter and we get

Entire characteristic functions[edit]

As defined above, the argument of the characteristic function is treated as a real number: however, certain aspects of the theory of characteristic functions are advanced by extending the definition into the complex plane by analytic continuation, in cases where this is possible.[24]

Related concepts[edit]

Related concepts include the moment-generating function and the probability-generating function. The characteristic function exists for all probability distributions. This is not the case for the moment-generating function.

The characteristic function is closely related to the Fourier transform: the characteristic function of a probability density function p(x) is the complex conjugate of the continuous Fourier transformofp(x) (according to the usual convention; see continuous Fourier transform – other conventions).

where P(t) denotes the continuous Fourier transform of the probability density function p(x). Likewise, p(x) may be recovered from φX(t) through the inverse Fourier transform:

Indeed, even when the random variable does not have a density, the characteristic function may be seen as the Fourier transform of the measure corresponding to the random variable.

Another related concept is the representation of probability distributions as elements of a reproducing kernel Hilbert space via the kernel embedding of distributions. This framework may be viewed as a generalization of the characteristic function under specific choices of the kernel function.

See also[edit]

Notes[edit]

  1. ^ named after the French mathematician Paul Lévy

References[edit]

Citations[edit]

  1. ^ Lukacs (1970), p. 196.
  • ^ Shaw, W. T.; McCabe, J. (2009). "Monte Carlo sampling given a Characteristic Function: Quantile Mechanics in Momentum Space". arXiv:0903.1592 [q-fin.CP].
  • ^ Statistical and Adaptive Signal Processing (2005)
  • ^ Billingsley (1995).
  • ^ Pinsky (2002).
  • ^ Bochner (1955).
  • ^ Andersen et al. (1995), Definition 1.10.
  • ^ Andersen et al. (1995), Definition 1.20.
  • ^ Sobczyk (2001), p. 20.
  • ^ Kotz & Nadarajah (2004), p. 37 using 1 as the number of degree of freedom to recover the Cauchy distribution
  • ^ Lukacs (1970), Corollary 1 to Theorem 2.3.1.
  • ^ "Joint characteristic function". www.statlect.com. Retrieved 7 April 2018.
  • ^ Cuppens (1975), Theorem 2.6.9.
  • ^ a b c Shephard (1991a).
  • ^ Cuppens (1975), Theorem 2.3.2.
  • ^ Wendel (1961).
  • ^ Shephard (1991b).
  • ^ Lukacs (1970), p. 84.
  • ^ Paulson, Holcomb & Leitch (1975).
  • ^ Heathcote (1977).
  • ^ Yu (2004).
  • ^ Ansari, Scarlett & Soh (2020).
  • ^ Li et al. (2020).
  • ^ Lukacs (1970), Chapter 7.
  • Sources[edit]

    • Andersen, H.H.; Højbjerre, M.; Sørensen, D.; Eriksen, P.S. (1995). Linear and graphical models for the multivariate complex normal distribution. Lecture Notes in Statistics 101. New York: Springer-Verlag. ISBN 978-0-387-94521-7.
  • Billingsley, Patrick (1995). Probability and measure (3rd ed.). John Wiley & Sons. ISBN 978-0-471-00710-4.
  • Bisgaard, T. M.; Sasvári, Z. (2000). Characteristic functions and moment sequences. Nova Science.
  • Bochner, Salomon (1955). Harmonic analysis and the theory of probability. University of California Press.
  • Cuppens, R. (1975). Decomposition of multivariate probabilities. Academic Press. ISBN 9780121994501.
  • Heathcote, C.R. (1977). "The integrated squared error estimation of parameters". Biometrika. 64 (2): 255–264. doi:10.1093/biomet/64.2.255.
  • Lukacs, E. (1970). Characteristic functions. London: Griffin.
  • Kotz, Samuel; Nadarajah, Saralees (2004). Multivariate T Distributions and Their Applications. Cambridge University Press.
  • Manolakis, Dimitris G.; Ingle, Vinay K.; Kogon, Stephen M. (2005). Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling, Adaptive Filtering, and Array Processing. Artech House. ISBN 978-1-58053-610-3.
  • Oberhettinger, Fritz (1973). Fourier transforms of distributions and their inverses; a collection of tables. New York: Academic Press. ISBN 9780125236508.
  • Paulson, A.S.; Holcomb, E.W.; Leitch, R.A. (1975). "The estimation of the parameters of the stable laws". Biometrika. 62 (1): 163–170. doi:10.1093/biomet/62.1.163.
  • Pinsky, Mark (2002). Introduction to Fourier analysis and wavelets. Brooks/Cole. ISBN 978-0-534-37660-4.
  • Sobczyk, Kazimierz (2001). Stochastic differential equations. Kluwer Academic Publishers. ISBN 978-1-4020-0345-5.
  • Wendel, J.G. (1961). "The non-absolute convergence of Gil-Pelaez' inversion integral". The Annals of Mathematical Statistics. 32 (1): 338–339. doi:10.1214/aoms/1177705164.
  • Yu, J. (2004). "Empirical characteristic function estimation and its applications" (PDF). Econometric Reviews. 23 (2): 93–1223. doi:10.1081/ETC-120039605. S2CID 9076760.
  • Shephard, N. G. (1991a). "From characteristic function to distribution function: A simple framework for the theory". Econometric Theory. 7 (4): 519–529. doi:10.1017/s0266466600004746. S2CID 14668369.
  • Shephard, N. G. (1991b). "Numerical integration rules for multivariate inversions". Journal of Statistical Computation and Simulation. 39 (1–2): 37–46. doi:10.1080/00949659108811337.
  • Ansari, Abdul Fatir; Scarlett, Jonathan; Soh, Harold (2020). "A Characteristic Function Approach to Deep Implicit Generative Modeling". Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. pp. 7478–7487.
  • Li, Shengxi; Yu, Zeyang; Xiang, Min; Mandic, Danilo (2020). "Reciprocal Adversarial Learning via Characteristic Functions". Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
  • External links[edit]


    Retrieved from "https://en.wikipedia.org/w/index.php?title=Characteristic_function_(probability_theory)&oldid=1211040581"

    Category: 
    Functions related to probability distributions
    Hidden categories: 
    Articles with short description
    Short description is different from Wikidata
    Wikipedia articles needing page number citations from September 2022
    All articles with unsourced statements
    Articles with unsourced statements from October 2023
    Articles to be expanded from December 2009
    All articles to be expanded
    Articles using small message boxes
     



    This page was last edited on 29 February 2024, at 15:00 (UTC).

    Text is available under the Creative Commons Attribution-ShareAlike License 4.0; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.



    Privacy policy

    About Wikipedia

    Disclaimers

    Contact Wikipedia

    Code of Conduct

    Developers

    Statistics

    Cookie statement

    Mobile view



    Wikimedia Foundation
    Powered by MediaWiki