1. Home
  2. Issues
  3. Volume 4, Issue 3 (2017)
  4. Weighted entropy: basic inequalities

Modern Stochastics: Theory and Applications


Weighted entropy: basic inequalities
Volume 4, Issue 3 (2017), pp. 233–252
Pub. online: 2 October 2017      Type: Research Article      Open accessOpen Access
Received
30 August 2017
Revised
18 September 2017
Accepted
18 September 2017
Published
2 October 2017

Abstract

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb’s splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.

References

[1] 
Cover, T., Thomas, J.: Elements of Information Theory. John Wiley, New York (2006). MR2239987
[2] 
Dembo, A.: Simple proof of the concavity of the entropy power with respect to added Gaussian noise. IEEE Trans. Inf. Theory 35(4), 887–888 (1989). MR1013698. doi:10.1109/18.32166
[3] 
Dembo, A., Cover, T., Thomas, J.: Information theoretic inequalities. IEEE Trans. Inf. Theory 37, 1501–1518 (1991). MR1134291. doi:10.1109/18.104312
[4] 
Frizelle, G., Suhov, Y.: An entropic measurement of queueing behaviour in a class of manufacturing operations. Proc. R. Soc. Lond., Ser A 457, 1579–1601 (2001). doi:10.1098/rspa.2000.0731
[5] 
Frizelle, G., Suhov, Y.: The measurement of complexity in production and other commercial systems. Proc. R. Soc. Lond., Ser A 464, 2649–2668 (2008). doi:10.1098/rspa.2007.0275
[6] 
Guiasu, S.: Weighted entropy. Rep. Math. Phys. 2, 165–179 (1971). MR0289206. doi:10.1016/0034-4877(71)90002-4
[7] 
Kelbert, M., Suhov, Y.: Information Theory and Coding by Example. Cambridge University Press, Cambridge (2013). MR3137525. doi:10.1017/CBO9781139028448
[8] 
Kelbert, M., Suhov, Y.: Continuity of mutual entropy in the limiting signal-to-noise ratio regimes. In: Stochastic Analysis, pp. 281–299. Springer, Berlin (2010). MR2789089. doi:10.1007/978-3-642-15358-7_14
[9] 
Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and optimal portfolios for risk-averse Kelly investments. Aequationes Mathematicae 91 (2017). in press
[10] 
Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and its use in Computer Science and beyond. Lecture Notes in Computer Science. (in press)
[11] 
Khan, J.F., Bhuiyan, S.M.: Weighted entropy for segmentation evaluation. Opt. Laser Technol. 57, 236–242 (2014). doi:10.1016/j.optlastec.2013.07.012
[12] 
Lai, W.K., Khan, I.M., Poh, G.S.: Weighted entropy-based measure for image segmentation. Proc. Eng. 41, 1261–1267 (2012). doi:10.1016/j.proeng.2012.07.309
[13] 
Lieb, E.: Proof of entropy conjecture of Wehrl. Commun. Math. Phys. 62, 35–41 (1978). MR0506364. doi:10.1007/BF01940328
[14] 
Nawrockia, D.N., Harding, W.H.: State-value weighted entropy as a measure of investment risk. Appl. Econ. 18, 411–419 (1986). doi:10.1080/00036848600000038
[15] 
Paksakis, C., Mermigas, S., Pirourias, S., Chondrokoukis, G.: The role of weighted entropy in security quantification. Int. Journ. Inf. Electron. Eng. 3(2), 156–159 (2013)
[16] 
Rioul, O.: Information theoretic proofs of entropy power inequality. IEEE Trans. Inf. Theory 57(1), 33–55 (2011). MR2810269. doi:10.1109/TIT.2010.2090193
[17] 
Shockley, K.R.: Using weighted entropy to rank chemicals in quantitative high throughput screening experiments. J. Biomol. Screen. 19, 344–353 (2014). doi:10.1177/1087057113505325
[18] 
Suhov, Y.: Stuhl I. Weighted information and entropy rates (2016). arXiv:1612.09169v1
[19] 
Suhov, Y., Sekeh, S.: An extension of the Ky-Fan inequality. arXiv:1504.01166
[20] 
Suhov, Y., Stuhl, I., Sekeh, S., Kelbert, M.: Basic inequalities for weighted entropy. Aequ. Math. 90(4), 817–848 (2016). MR3523101. doi:10.1007/s00010-015-0396-5
[21] 
Suhov, Y., Sekeh, S., Kelbert, M.: Entropy-power inequality for weighted entropy. arXiv:1502.02188
[22] 
Suhov, Y., Yasaei Sekeh, S.: Stuhl I. Weighted Gaussian entropy and determinant inequalities. arXiv:1505.01753v1
[23] 
Tsui, P.-H.: Ultrasound detection of scatterer concentration by weighted entropy. Entropy 17, 6598–6616 (2015). doi:10.3390/e17106598
[24] 
Verdú, S., Guo, D.: A simple proof of the entropy-power inequality. IEEE Trans. Inf. Theory 52(5), 2165–2166 (2006). MR2234471. doi:10.1109/TIT.2006.872978
[25] 
Villani, C.: A short proof of the “concavity of entropy power”. IEEE Trans. Inf. Theory 46, 1695–1696 (2000). MR1768665. doi:10.1109/18.850718
[26] 
Yang, L., Yang, J., Peng, N., Ling, J.: Weighted information entropy: A method for estimating the complex degree of infrared images’ backgrounds. In: Kamel, M., Campilho, A. (eds.) Image Analysis and Recognition, vol. 3656, Springer, Berlin/Heidelberg, pp. 215–22 (2005). MR3157460. doi:10.1007/978-3-642-39094-4
[27] 
Zamir, R.: A proof of the Fisher information inequality via a data processing argument. IEEE Trans. Inf. Theory 44(3), 1246–1250 (1998). MR1616672. doi:10.1109/18.669301

Full article Cited by PDF XML
Full article Cited by PDF XML

Copyright
© 2017 The Author(s). Published by VTeX
Open access article under the CC BY license.

Keywords
Weighted entropy Gibbs inequality Ky-Fan inequality Fisher information inequality entropy power inequality Lieb’s splitting inequality rates of weighted entropy and information

MSC2010
94A17

Metrics
since March 2018
1003

Article info
views

1110

Full article
views

542

PDF
downloads

163

XML
downloads


Share


RSS