Weighted entropy: basic inequalities

Kelbert, Mark; Stuhl, Izabella; Suhov, Yuri

doi:10.15559/17-VMSTA85

Modern Stochastics: Theory and Applications

Weighted entropy: basic inequalities

Volume 4, Issue 3 (2017), pp. 233–252

Mark Kelbert Izabella Stuhl Yuri Suhov

https://doi.org/10.15559/17-VMSTA85

Pub. online: 2 October 2017 Type: Research Article

Open Access

Received
30 August 2017

Revised
18 September 2017

Accepted
18 September 2017

Published
2 October 2017

Abstract

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb’s splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.

References

[1]

Cover, T., Thomas, J.: Elements of Information Theory. John Wiley, New York (2006). MR2239987

[2]

Dembo, A.: Simple proof of the concavity of the entropy power with respect to added Gaussian noise. IEEE Trans. Inf. Theory 35(4), 887–888 (1989). MR1013698. doi:10.1109/18.32166

[3]

Dembo, A., Cover, T., Thomas, J.: Information theoretic inequalities. IEEE Trans. Inf. Theory 37, 1501–1518 (1991). MR1134291. doi:10.1109/18.104312

[4]

Frizelle, G., Suhov, Y.: An entropic measurement of queueing behaviour in a class of manufacturing operations. Proc. R. Soc. Lond., Ser A 457, 1579–1601 (2001). doi:10.1098/rspa.2000.0731

[5]

Frizelle, G., Suhov, Y.: The measurement of complexity in production and other commercial systems. Proc. R. Soc. Lond., Ser A 464, 2649–2668 (2008). doi:10.1098/rspa.2007.0275

[6]

Guiasu, S.: Weighted entropy. Rep. Math. Phys. 2, 165–179 (1971). MR0289206. doi:10.1016/0034-4877(71)90002-4

[7]

Kelbert, M., Suhov, Y.: Information Theory and Coding by Example. Cambridge University Press, Cambridge (2013). MR3137525. doi:10.1017/CBO9781139028448

[8]

Kelbert, M., Suhov, Y.: Continuity of mutual entropy in the limiting signal-to-noise ratio regimes. In: Stochastic Analysis, pp. 281–299. Springer, Berlin (2010). MR2789089. doi:10.1007/978-3-642-15358-7_14

[9]

Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and optimal portfolios for risk-averse Kelly investments. Aequationes Mathematicae 91 (2017). in press

[10]

Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and its use in Computer Science and beyond. Lecture Notes in Computer Science. (in press)

[11]

Khan, J.F., Bhuiyan, S.M.: Weighted entropy for segmentation evaluation. Opt. Laser Technol. 57, 236–242 (2014). doi:10.1016/j.optlastec.2013.07.012

[12]

Lai, W.K., Khan, I.M., Poh, G.S.: Weighted entropy-based measure for image segmentation. Proc. Eng. 41, 1261–1267 (2012). doi:10.1016/j.proeng.2012.07.309

[13]

Lieb, E.: Proof of entropy conjecture of Wehrl. Commun. Math. Phys. 62, 35–41 (1978). MR0506364. doi:10.1007/BF01940328

[14]

Nawrockia, D.N., Harding, W.H.: State-value weighted entropy as a measure of investment risk. Appl. Econ. 18, 411–419 (1986). doi:10.1080/00036848600000038

[15]

Paksakis, C., Mermigas, S., Pirourias, S., Chondrokoukis, G.: The role of weighted entropy in security quantification. Int. Journ. Inf. Electron. Eng. 3(2), 156–159 (2013)

[16]

Rioul, O.: Information theoretic proofs of entropy power inequality. IEEE Trans. Inf. Theory 57(1), 33–55 (2011). MR2810269. doi:10.1109/TIT.2010.2090193

[17]

Shockley, K.R.: Using weighted entropy to rank chemicals in quantitative high throughput screening experiments. J. Biomol. Screen. 19, 344–353 (2014). doi:10.1177/1087057113505325

[18]

Suhov, Y.: Stuhl I. Weighted information and entropy rates (2016). arXiv:1612.09169v1

[19]

Suhov, Y., Sekeh, S.: An extension of the Ky-Fan inequality. arXiv:1504.01166

[20]

Suhov, Y., Stuhl, I., Sekeh, S., Kelbert, M.: Basic inequalities for weighted entropy. Aequ. Math. 90(4), 817–848 (2016). MR3523101. doi:10.1007/s00010-015-0396-5

[21]

Suhov, Y., Sekeh, S., Kelbert, M.: Entropy-power inequality for weighted entropy. arXiv:1502.02188

[22]

Suhov, Y., Yasaei Sekeh, S.: Stuhl I. Weighted Gaussian entropy and determinant inequalities. arXiv:1505.01753v1

[23]

Tsui, P.-H.: Ultrasound detection of scatterer concentration by weighted entropy. Entropy 17, 6598–6616 (2015). doi:10.3390/e17106598

[24]

Verdú, S., Guo, D.: A simple proof of the entropy-power inequality. IEEE Trans. Inf. Theory 52(5), 2165–2166 (2006). MR2234471. doi:10.1109/TIT.2006.872978

[25]

Villani, C.: A short proof of the “concavity of entropy power”. IEEE Trans. Inf. Theory 46, 1695–1696 (2000). MR1768665. doi:10.1109/18.850718

[26]

Yang, L., Yang, J., Peng, N., Ling, J.: Weighted information entropy: A method for estimating the complex degree of infrared images’ backgrounds. In: Kamel, M., Campilho, A. (eds.) Image Analysis and Recognition, vol. 3656, Springer, Berlin/Heidelberg, pp. 215–22 (2005). MR3157460. doi:10.1007/978-3-642-39094-4

[27]

Zamir, R.: A proof of the Fisher information inequality via a data processing argument. IEEE Trans. Inf. Theory 44(3), 1246–1250 (1998). MR1616672. doi:10.1109/18.669301

Full article Cited by

Open access article under the CC BY license.

Keywords

Weighted entropy Gibbs inequality Ky-Fan inequality Fisher information inequality entropy power inequality Lieb’s splitting inequality rates of weighted entropy and information

MSC2010

94A17

Metrics

since March 2018

876 Article info views	970 Full article views
481 PDF downloads	158 XML downloads

RSS

Authors

Abstract

References

Export citation

Copy and paste formatted citation

Download citation in file