Convexity and robustness of the Rényi entropy
Volume 8, Issue 3 (2021), pp. 387–412
Pub. online: 26 July 2021
Type: Research Article
Open Access
Open Access
Received
27 May 2021
27 May 2021
Revised
23 June 2021
23 June 2021
Accepted
24 June 2021
24 June 2021
Published
26 July 2021
26 July 2021
Abstract
We study convexity properties of the Rényi entropy as function of $\alpha >0$ on finite alphabets. We also describe robustness of the Rényi entropy on finite alphabets, and it turns out that the rate of respective convergence depends on initial alphabet. We establish convergence of the disturbed entropy when the initial distribution is uniform but the number of events increases to ∞ and prove that the limit of Rényi entropy of the binomial distribution is equal to Rényi entropy of the Poisson distribution.
References
Acharya, J., Orlitsky, A., Suresh, A.T., Tyagi, H.: The Complexity of Estimating Rényi Entropy. Proceedings of the 2015 Annual ACM-SIAM Symposium on Discrete Algorithms (2015). Book Code:PRDA15. MR3451148. https://doi.org/10.1137/1.9781611973730.124
Banna, O., Buryak, F., Yu, M.: Distance from fractional Brownian motion with associated Hurst index $0<H<1/2$ to the subspaces of Gaussian martingales involving power integrands with an arbitrary positive exponent. Mod. Stoch. Theory Appl. 7, pp. 191–202 (2020). MR4120614. https://doi.org/10.15559/20-vmsta156
Bobkov, S.G., Marsiglietti, A., Melbourne, J.: Concentration functions and entropy bounds for discrete log-concave distributions. arXiv:2007.11030v1 [math.PR]
Erven, T., Harremoës, P.: Rényi Divergence and Kullback-Leibler Divergence. IEEE Trans. Inf. Theory 60, 3797–3820 (2014). MR3225930. https://doi.org/10.1109/TIT.2014.2320500
Gil, M., Alajaji, F., Linder, T.: Rényi divergence measures for commonly used univariate continuous distributions. Inf. Sci. 249, 124–131 (2013). MR3105467. https://doi.org/10.1016/j.ins.2013.06.018
Harremoës, P.: Binomial and Poisson Distributions as Maximum Entropy Distributions. IEEE Trans. Inf. Theory 475, 2039–2041 (2001). MR1842536. https://doi.org/10.1109/18.930936
Harremoës, P.: Interpretations of Rényi entropies and divergences. Physica A (2006). MR2223335. https://doi.org/10.1016/j.physa.2006.01.012
Ho, S.-W., Verdú, S.: Convexity/concavity of Rényi entropy and α-mutual information. In: IEEE International Symposium on Information Theory (ISIT), pp. 745–749 (2015). https://doi.org/10.1109/ISIT.2015.7282554
Lenzi, E.K., Mendes, R.S., Silva, L.R.: Statistical mechanics based on Rényi entropy. Physica A 280, 337–345 (2000). MR1758523. https://doi.org/10.1016/S0378-4371(00)00007-8
Rényi, A.: On Measures of Entropy and Information. In: Proceedings of the 4th Berkeley Symposium on Mathematics, Statistics and Probability, vol. 1, pp. 547–561. The Regents of University of the California, Berkeley, CA, USA (1961). MR0132570
Yang, Y., Li, J., Yang, Y.: Multiscale multifractal multiproperty analysis of financial time series based on Rényi entropy. Int. J. Mod. Phys. C 28(2) (2017). MR3284253. https://doi.org/10.1016/j.physa.2014.11.009
