Jump to content
 







Main menu
   


Navigation  



Main page
Contents
Current events
Random article
About Wikipedia
Contact us
Donate
 




Contribute  



Help
Learn to edit
Community portal
Recent changes
Upload file
 








Search  

































Create account

Log in
 









Create account
 Log in
 




Pages for logged out editors learn more  



Contributions
Talk
 



















Contents

   



(Top)
 


1 Motivating example  





2 Specification  



2.1  Terminology and assumptions  







3 Linear model  



3.1  Simple linear model  





3.2  Multivariable linear model  







4 Non-linear models  



4.1  Instrumental variables methods  





4.2  Repeated observations  







5 References  





6 Further reading  





7 External links  














Errors-in-variables models






Deutsch
فارسی
 

Edit links
 









Article
Talk
 

















Read
Edit
View history
 








Tools
   


Actions  



Read
Edit
View history
 




General  



What links here
Related changes
Upload file
Special pages
Permanent link
Page information
Cite this page
Get shortened URL
Download QR code
Wikidata item
 




Print/export  



Download as PDF
Printable version
 
















Appearance
   

 






From Wikipedia, the free encyclopedia
 


Instatistics, errors-in-variables modelsormeasurement error models are regression models that account for measurement errors in the independent variables. In contrast, standard regression models assume that those regressors have been measured exactly, or observed without error; as such, those models account only for errors in the dependent variables, or responses.[citation needed]

Illustration of regression dilution (or attenuation bias) by a range of regression estimates in errors-in-variables models. Two regression lines (red) bound the range of linear regression possibilities. The shallow slope is obtained when the independent variable (or predictor) is on the x-axis. The steeper slope is obtained when the independent variable is on the y-axis. By convention, with the independent variable on the x-axis, the shallower slope is obtained. Green reference lines are averages within arbitrary bins along each axis. Note that the steeper green and red regression estimates are more consistent with smaller errors in the y-axis variable.

In the case when some regressors have been measured with errors, estimation based on the standard assumption leads to inconsistent estimates, meaning that the parameter estimates do not tend to the true values even in very large samples. For simple linear regression the effect is an underestimate of the coefficient, known as the attenuation bias. In non-linear models the direction of the bias is likely to be more complicated.[1][2][3]

Motivating example[edit]

Consider a simple linear regression model of the form

where denotes the true but unobserved regressor. Instead we observe this value with an error:

where the measurement error is assumed to be independent of the true value .

If the ′s are simply regressed on the ′s (see simple linear regression), then the estimator for the slope coefficient is

which converges as the sample size increases without bound:

This is in contrast to the "true" effect of , estimated using the ,:

Variances are non-negative, so that in the limit the estimated is smaller than , an effect which statisticians call attenuationorregression dilution.[4] Thus the ‘naïve’ least squares estimator is an inconsistent estimator for . However, is a consistent estimator of the parameter required for a best linear predictor of given the observed : in some applications this may be what is required, rather than an estimate of the ‘true’ regression coefficient , although that would assume that the variance of the errors in the estimation and prediction is identical. This follows directly from the result quoted immediately above, and the fact that the regression coefficient relating the ′s to the actually observed ′s, in a simple linear regression, is given by

It is this coefficient, rather than , that would be required for constructing a predictor of based on an observed which is subject to noise.

It can be argued that almost all existing data sets contain errors of different nature and magnitude, so that attenuation bias is extremely frequent (although in multivariate regression the direction of bias is ambiguous[5]). Jerry Hausman sees this as an iron law of econometrics: "The magnitude of the estimate is usually smaller than expected."[6]

Specification[edit]

Usually measurement error models are described using the latent variables approach. If is the response variable and are observed values of the regressors, then it is assumed there exist some latent variables and which follow the model's “true” functional relationship , and such that the observed quantities are their noisy observations:

where is the model's parameter and are those regressors which are assumed to be error-free (for example when linear regression contains an intercept, the regressor which corresponds to the constant certainly has no "measurement errors"). Depending on the specification these error-free regressors may or may not be treated separately; in the latter case it is simply assumed that corresponding entries in the variance matrix of 's are zero.

The variables , , are all observed, meaning that the statistician possesses a data setof statistical units which follow the data generating process described above; the latent variables , , , and are not observed however.

This specification does not encompass all the existing errors-in-variables models. For example in some of them function may be non-parametric or semi-parametric. Other approaches model the relationship between and as distributional instead of functional, that is they assume that conditionally on follows a certain (usually parametric) distribution.

Terminology and assumptions[edit]

Linear model[edit]

Linear errors-in-variables models were studied first, probably because linear models were so widely used and they are easier than non-linear ones. Unlike standard least squares regression (OLS), extending errors in variables regression (EiV) from the simple to the multivariable case is not straightforward.

Simple linear model[edit]

The simple linear errors-in-variables model was already presented in the "motivation" section:

where all variables are scalar. Here α and β are the parameters of interest, whereas σε and ση—standard deviations of the error terms—are the nuisance parameters. The "true" regressor x* is treated as a random variable (structural model), independent of the measurement error η (classic assumption).

This model is identifiable in two cases: (1) either the latent regressor x*isnot normally distributed, (2) or x* has normal distribution, but neither εt nor ηt are divisible by a normal distribution.[10] That is, the parameters α, β can be consistently estimated from the data set without any additional information, provided the latent regressor is not Gaussian.

Before this identifiability result was established, statisticians attempted to apply the maximum likelihood technique by assuming that all variables are normal, and then concluded that the model is not identified. The suggested remedy was to assume that some of the parameters of the model are known or can be estimated from the outside source. Such estimation methods include[11]

Estimation methods that do not assume knowledge of some of the parameters of the model, include

Multivariable linear model[edit]

The multivariable model looks exactly like the simple linear model, only this time β, ηt, xt and x*t are 1 vectors.

In the case when (εt,ηt) is jointly normal, the parameter β is not identified if and only if there is a non-singular k×k block matrix [a A], where a is a 1 vector such that a′x* is distributed normally and independently of A′x*. In the case when εt, ηt1,..., ηtk are mutually independent, the parameter β is not identified if and only if in addition to the conditions above some of the errors can be written as the sum of two independent variables one of which is normal.[14]

Some of the estimation methods for multivariable linear models are

Non-linear models[edit]

A generic non-linear measurement error model takes form

Here function g can be either parametric or non-parametric. When function g is parametric it will be written as g(x*, β).

For a general vector-valued regressor x* the conditions for model identifiability are not known. However in the case of scalar x* the model is identified unless the function g is of the "log-exponential" form [19]

and the latent regressor x* has density

where constants A,B,C,D,E,F may depend on a,b,c,d.

Despite this optimistic result, as of now no methods exist for estimating non-linear errors-in-variables models without any extraneous information. However there are several techniques which make use of some additional data: either the instrumental variables, or repeated observations.

Instrumental variables methods[edit]

Repeated observations[edit]

In this approach two (or maybe more) repeated observations of the regressor x* are available. Both observations contain their own measurement errors, however those errors are required to be independent:

where x*η1η2. Variables η1, η2 need not be identically distributed (although if they are efficiency of the estimator can be slightly improved). With only these two observations it is possible to consistently estimate the density function of x* using Kotlarski's deconvolution technique.[21]

References[edit]

  1. ^ Griliches, Zvi; Ringstad, Vidar (1970). "Errors-in-the-variables bias in nonlinear contexts". Econometrica. 38 (2): 368–370. doi:10.2307/1913020. JSTOR 1913020.
  • ^ Chesher, Andrew (1991). "The effect of measurement error". Biometrika. 78 (3): 451–462. doi:10.1093/biomet/78.3.451. JSTOR 2337015.
  • ^ Carroll, Raymond J.; Ruppert, David; Stefanski, Leonard A.; Crainiceanu, Ciprian (2006). Measurement Error in Nonlinear Models: A Modern Perspective (Second ed.). ISBN 978-1-58488-633-4.
  • ^ Greene, William H. (2003). Econometric Analysis (5th ed.). New Jersey: Prentice Hall. Chapter 5.6.1. ISBN 978-0-13-066189-0.
  • ^ Wansbeek, T.; Meijer, E. (2000). "Measurement Error and Latent Variables". In Baltagi, B. H. (ed.). A Companion to Theoretical Econometrics. Blackwell. pp. 162–179. doi:10.1111/b.9781405106764.2003.00013.x. ISBN 9781405106764.
  • ^ Hausman, Jerry A. (2001). "Mismeasured variables in econometric analysis: problems from the right and problems from the left". Journal of Economic Perspectives. 15 (4): 57–67 [p. 58]. doi:10.1257/jep.15.4.57. JSTOR 2696516.
  • ^ Fuller, Wayne A. (1987). Measurement Error Models. John Wiley & Sons. p. 2. ISBN 978-0-471-86187-4.
  • ^ Hayashi, Fumio (2000). Econometrics. Princeton University Press. pp. 7–8. ISBN 978-1400823833.
  • ^ Koul, Hira; Song, Weixing (2008). "Regression model checking with Berkson measurement errors". Journal of Statistical Planning and Inference. 138 (6): 1615–1628. doi:10.1016/j.jspi.2007.05.048.
  • ^ Reiersøl, Olav (1950). "Identifiability of a linear relation between variables which are subject to error". Econometrica. 18 (4): 375–389 [p. 383]. doi:10.2307/1907835. JSTOR 1907835. A somewhat more restrictive result was established earlier by Geary, R. C. (1942). "Inherent relations between random variables". Proceedings of the Royal Irish Academy. 47: 63–76. JSTOR 20488436. He showed that under the additional assumption that (ε, η) are jointly normal, the model is not identified if and only if x*s are normal.
  • ^ Fuller, Wayne A. (1987). "A Single Explanatory Variable". Measurement Error Models. John Wiley & Sons. pp. 1–99. ISBN 978-0-471-86187-4.
  • ^ Pal, Manoranjan (1980). "Consistent moment estimators of regression coefficients in the presence of errors in variables". Journal of Econometrics. 14 (3): 349–364 (pp. 360–361). doi:10.1016/0304-4076(80)90032-9.
  • ^ Xu, Shaoji (2014-10-02). "A Property of Geometric Mean Regression". The American Statistician. 68 (4): 277–281. doi:10.1080/00031305.2014.962763. ISSN 0003-1305.
  • ^ Ben-Moshe, Dan (2020). "Identification of linear regressions with errors in all variables". Econometric Theory. 37 (4): 1–31. arXiv:1404.1473. doi:10.1017/S0266466620000250. S2CID 225653359.
  • ^ Dagenais, Marcel G.; Dagenais, Denyse L. (1997). "Higher moment estimators for linear regression models with errors in the variables". Journal of Econometrics. 76 (1–2): 193–221. CiteSeerX 10.1.1.669.8286. doi:10.1016/0304-4076(95)01789-5. In the earlier paper Pal (1980) considered a simpler case when all components in vector (ε, η) are independent and symmetrically distributed.
  • ^ Fuller, Wayne A. (1987). Measurement Error Models. John Wiley & Sons. p. 184. ISBN 978-0-471-86187-4.
  • ^ Erickson, Timothy; Whited, Toni M. (2002). "Two-step GMM estimation of the errors-in-variables model using high-order moments". Econometric Theory. 18 (3): 776–799. doi:10.1017/s0266466602183101. JSTOR 3533649. S2CID 14729228.
  • ^ Tofallis, C. (2023). Fitting an Equation to Data Impartially. Mathematics, 11(18), 3957. https://ssrn.com/abstract=4556739 https://doi.org/10.3390/math11183957
  • ^ Schennach, S.; Hu, Y.; Lewbel, A. (2007). "Nonparametric identification of the classical errors-in-variables model without side information". Working Paper.
  • ^ Newey, Whitney K. (2001). "Flexible simulated moment estimation of nonlinear errors-in-variables model". Review of Economics and Statistics. 83 (4): 616–627. doi:10.1162/003465301753237704. hdl:1721.1/63613. JSTOR 3211757. S2CID 57566922.
  • ^ Li, Tong; Vuong, Quang (1998). "Nonparametric estimation of the measurement error model using multiple indicators". Journal of Multivariate Analysis. 65 (2): 139–165. doi:10.1006/jmva.1998.1741.
  • ^ Li, Tong (2002). "Robust and consistent estimation of nonlinear errors-in-variables models". Journal of Econometrics. 110 (1): 1–26. doi:10.1016/S0304-4076(02)00120-3.
  • ^ Schennach, Susanne M. (2004). "Estimation of nonlinear models with measurement error". Econometrica. 72 (1): 33–75. doi:10.1111/j.1468-0262.2004.00477.x. JSTOR 3598849.
  • ^ Schennach, Susanne M. (2004). "Nonparametric regression in the presence of measurement error". Econometric Theory. 20 (6): 1046–1093. doi:10.1017/S0266466604206028. S2CID 123036368.
  • Further reading[edit]

    External links[edit]


    Retrieved from "https://en.wikipedia.org/w/index.php?title=Errors-in-variables_models&oldid=1215691074"

    Categories: 
    Regression models
    Least squares
    Hidden categories: 
    Articles with short description
    Short description matches Wikidata
    All articles with unsourced statements
    Articles with unsourced statements from November 2015
     



    This page was last edited on 26 March 2024, at 15:59 (UTC).

    Text is available under the Creative Commons Attribution-ShareAlike License 4.0; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.



    Privacy policy

    About Wikipedia

    Disclaimers

    Contact Wikipedia

    Code of Conduct

    Developers

    Statistics

    Cookie statement

    Mobile view



    Wikimedia Foundation
    Powered by MediaWiki