reckless intuitions of an epistemic hygienist (gustavolacerda) wrote,
reckless intuitions of an epistemic hygienist

lit review: Kullback-Leibler divergence in Statistics PhD textbooks

Bickel, Klaassen, Ritov, Wellner
* p. 227, Re: estimating cell probabilities in a two-way contingency table with known marginals.  "Iterative Proportional Fitting converges to the minimum Kullback divergence estimator"

Casella & Berger (2nd Edition) says nothing

Gelman, Carlin, Stern, Rubin (2nd Edition)
* p.107, asymptotics of estimation under mis-specified model.
* p.586-588, details about this.

Hastie, Tibshirani, Friedman
 (2nd Edition)
* p. 561, KL "distance".  Used in definition of mutual information, on a section about ICA.

Hogg, McKean, Craig (6th edition) says nothing

* p.113-114, "Kullback-Leibler information number", on chapter titled "Strong Consistency of Maximum-Likelihood Estimates"

* p.59: proof question: show that KL > 0 unless the two distributions are equal.
* p.156: consistency of MLE.
* p.466: solution to the proof question.

Lehmann & Romano
* p. 432: optimal test rejects for large values of T_n, and T_n converges to -K(P0, P1).
* p. 672: in non-parametric test for the mean, KL is used to define the most favorable distribution in H0.

van der Vaart
* p. 56: asymptotics of estimation under mis-specified model.
* p. 62: viewing MLE as an M-estimator.

* p. 125: consistency of MLE. "By LLN, M_n converges to -D(θ*, θ)"
mirror of this post
  • Post a new comment


    Anonymous comments are disabled in this journal

    default userpic

    Your reply will be screened

    Your IP address will be recorded