Estimation of non-linear functionals of densities with confidence Public Deposited

http://ir.library.oregonstate.edu/concern/defaults/v979v4245

This is the author's peer-reviewed final manuscript, as accepted by the publisher. The published article is copyrighted by IEEE-Institute of Electrical and Electronics Engineers and can be found at:  http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=18.

Descriptions

Attribute NameValues
Alternative Title
  • Estimation of Nonlinear Functionals of Densities With Confidence
Creator
Abstract or Summary
  • This paper introduces a class of k-nearest neighbor (k-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of non-linear functions of a probability density, such as Shannon entropy and R´enyi entropy. The density is assumed to be smooth, have bounded support, and be uniformly bounded from below on this set. Unlike previous k-NN estimators of non-linear density functionals, the proposed estimator uses data-splitting and boundary correction to achieve lower mean square error. Specifically, we assume that T i.i.d. samples X[subscript i] ∈ R[superscript d] from the density are split into two pieces of cardinality M and N respectively, with M samples used for computing a k-nearest-neighbor density estimate and the remaining N samples used for empirical estimation of the integral of the density functional. By studying the statistical properties of k-NN balls, explicit rates for the bias and variance of the BPI estimator are derived in terms of the sample size, the dimension of the samples and the underlying probability distribution. Based on these results, it is possible to specify optimal choice of tuning parameters M/T, k for maximizing the rate of decrease of the mean square error (MSE). The resultant optimized BPI estimator converges faster and achieves lower mean squared error than previous k-NN entropy estimators. In addition, a central limit theorem is established for the BPI estimator that allows us to specify tight asymptotic confidence intervals.
Resource Type
DOI
Date Available
Date Issued
Citation
  • Sricharan, K., Raich, R., & Hero, A. O. (2012). Estimation of nonlinear functionals of densities with confidence. IEEE Transactions on Information Theory, 58(7), 4135-4159. doi: 10.1109/TIT.2012.2195549
Series
Keyword
Rights Statement
Funding Statement (additional comments about funding)
Publisher
Peer Reviewed
Language
Replaces
Additional Information
  • description.provenance : Made available in DSpace on 2013-01-15T01:34:51Z (GMT). No. of bitstreams: 1 RaichRavivEECSEstimationNonlinearFunctionals(AM).pdf: 375859 bytes, checksum: c43e779b689b0a81a418d947d975e484 (MD5) Previous issue date: 2012-07
  • description.provenance : Submitted by Deanne Bruner (deanne.bruner@oregonstate.edu) on 2013-01-15T01:29:49Z No. of bitstreams: 1 RaichRavivEECSEstimationNonlinearFunctionals(AM).pdf: 375859 bytes, checksum: c43e779b689b0a81a418d947d975e484 (MD5)
  • description.provenance : Approved for entry into archive by Deanne Bruner(deanne.bruner@oregonstate.edu) on 2013-01-15T01:34:51Z (GMT) No. of bitstreams: 1 RaichRavivEECSEstimationNonlinearFunctionals(AM).pdf: 375859 bytes, checksum: c43e779b689b0a81a418d947d975e484 (MD5)

Relationships

In Administrative Set:
Last modified: 07/18/2017

Downloadable Content

Download PDF
Citations:

EndNote | Zotero | Mendeley

Items