Modern Stochastics: Theory and Applications logo


  • Help
Login Register

  1. Home
  2. Issues
  3. Volume 4, Issue 3 (2017)
  4. Weighted entropy: basic inequalities

Modern Stochastics: Theory and Applications

Submit your article Information Become a Peer-reviewer
  • Article info
  • Full article
  • Cited by
  • More
    Article info Full article Cited by

Weighted entropy: basic inequalities
Volume 4, Issue 3 (2017), pp. 233–252
Mark Kelbert   Izabella Stuhl   Yuri Suhov  

Authors

 
Placeholder
https://doi.org/10.15559/17-VMSTA85
Pub. online: 2 October 2017      Type: Research Article      Open accessOpen Access

Received
30 August 2017
Revised
18 September 2017
Accepted
18 September 2017
Published
2 October 2017

Abstract

This paper represents an extended version of an earlier note [10]. The concept of weighted entropy takes into account values of different outcomes, i.e., makes entropy context-dependent, through the weight function. We analyse analogs of the Fisher information inequality and entropy power inequality for the weighted entropy and discuss connections with weighted Lieb’s splitting inequality. The concepts of rates of the weighted entropy and information are also discussed.

References

[1] 
Cover, T., Thomas, J.: Elements of Information Theory. John Wiley, New York (2006). MR2239987
[2] 
Dembo, A.: Simple proof of the concavity of the entropy power with respect to added Gaussian noise. IEEE Trans. Inf. Theory 35(4), 887–888 (1989). MR1013698. doi:10.1109/18.32166
[3] 
Dembo, A., Cover, T., Thomas, J.: Information theoretic inequalities. IEEE Trans. Inf. Theory 37, 1501–1518 (1991). MR1134291. doi:10.1109/18.104312
[4] 
Frizelle, G., Suhov, Y.: An entropic measurement of queueing behaviour in a class of manufacturing operations. Proc. R. Soc. Lond., Ser A 457, 1579–1601 (2001). doi:10.1098/rspa.2000.0731
[5] 
Frizelle, G., Suhov, Y.: The measurement of complexity in production and other commercial systems. Proc. R. Soc. Lond., Ser A 464, 2649–2668 (2008). doi:10.1098/rspa.2007.0275
[6] 
Guiasu, S.: Weighted entropy. Rep. Math. Phys. 2, 165–179 (1971). MR0289206. doi:10.1016/0034-4877(71)90002-4
[7] 
Kelbert, M., Suhov, Y.: Information Theory and Coding by Example. Cambridge University Press, Cambridge (2013). MR3137525. doi:10.1017/CBO9781139028448
[8] 
Kelbert, M., Suhov, Y.: Continuity of mutual entropy in the limiting signal-to-noise ratio regimes. In: Stochastic Analysis, pp. 281–299. Springer, Berlin (2010). MR2789089. doi:10.1007/978-3-642-15358-7_14
[9] 
Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and optimal portfolios for risk-averse Kelly investments. Aequationes Mathematicae 91 (2017). in press
[10] 
Kelbert, M., Stuhl, I., Suhov, Y.: Weighted entropy and its use in Computer Science and beyond. Lecture Notes in Computer Science. (in press)
[11] 
Khan, J.F., Bhuiyan, S.M.: Weighted entropy for segmentation evaluation. Opt. Laser Technol. 57, 236–242 (2014). doi:10.1016/j.optlastec.2013.07.012
[12] 
Lai, W.K., Khan, I.M., Poh, G.S.: Weighted entropy-based measure for image segmentation. Proc. Eng. 41, 1261–1267 (2012). doi:10.1016/j.proeng.2012.07.309
[13] 
Lieb, E.: Proof of entropy conjecture of Wehrl. Commun. Math. Phys. 62, 35–41 (1978). MR0506364. doi:10.1007/BF01940328
[14] 
Nawrockia, D.N., Harding, W.H.: State-value weighted entropy as a measure of investment risk. Appl. Econ. 18, 411–419 (1986). doi:10.1080/00036848600000038
[15] 
Paksakis, C., Mermigas, S., Pirourias, S., Chondrokoukis, G.: The role of weighted entropy in security quantification. Int. Journ. Inf. Electron. Eng. 3(2), 156–159 (2013)
[16] 
Rioul, O.: Information theoretic proofs of entropy power inequality. IEEE Trans. Inf. Theory 57(1), 33–55 (2011). MR2810269. doi:10.1109/TIT.2010.2090193
[17] 
Shockley, K.R.: Using weighted entropy to rank chemicals in quantitative high throughput screening experiments. J. Biomol. Screen. 19, 344–353 (2014). doi:10.1177/1087057113505325
[18] 
Suhov, Y.: Stuhl I. Weighted information and entropy rates (2016). arXiv:1612.09169v1
[19] 
Suhov, Y., Sekeh, S.: An extension of the Ky-Fan inequality. arXiv:1504.01166
[20] 
Suhov, Y., Stuhl, I., Sekeh, S., Kelbert, M.: Basic inequalities for weighted entropy. Aequ. Math. 90(4), 817–848 (2016). MR3523101. doi:10.1007/s00010-015-0396-5
[21] 
Suhov, Y., Sekeh, S., Kelbert, M.: Entropy-power inequality for weighted entropy. arXiv:1502.02188
[22] 
Suhov, Y., Yasaei Sekeh, S.: Stuhl I. Weighted Gaussian entropy and determinant inequalities. arXiv:1505.01753v1
[23] 
Tsui, P.-H.: Ultrasound detection of scatterer concentration by weighted entropy. Entropy 17, 6598–6616 (2015). doi:10.3390/e17106598
[24] 
Verdú, S., Guo, D.: A simple proof of the entropy-power inequality. IEEE Trans. Inf. Theory 52(5), 2165–2166 (2006). MR2234471. doi:10.1109/TIT.2006.872978
[25] 
Villani, C.: A short proof of the “concavity of entropy power”. IEEE Trans. Inf. Theory 46, 1695–1696 (2000). MR1768665. doi:10.1109/18.850718
[26] 
Yang, L., Yang, J., Peng, N., Ling, J.: Weighted information entropy: A method for estimating the complex degree of infrared images’ backgrounds. In: Kamel, M., Campilho, A. (eds.) Image Analysis and Recognition, vol. 3656, Springer, Berlin/Heidelberg, pp. 215–22 (2005). MR3157460. doi:10.1007/978-3-642-39094-4
[27] 
Zamir, R.: A proof of the Fisher information inequality via a data processing argument. IEEE Trans. Inf. Theory 44(3), 1246–1250 (1998). MR1616672. doi:10.1109/18.669301

Full article Cited by PDF XML
Full article Cited by PDF XML

Copyright
© 2017 The Author(s). Published by VTeX
by logo by logo
Open access article under the CC BY license.

Keywords
Weighted entropy Gibbs inequality Ky-Fan inequality Fisher information inequality entropy power inequality Lieb’s splitting inequality rates of weighted entropy and information

MSC2010
94A17

Metrics
since March 2018
971

Article info
views

1094

Full article
views

527

PDF
downloads

162

XML
downloads

Export citation

Copy and paste formatted citation
Placeholder

Download citation in file


Share


RSS

MSTA

MSTA

  • Online ISSN: 2351-6054
  • Print ISSN: 2351-6046
  • Copyright © 2018 VTeX

About

  • About journal
  • Indexed in
  • Editors-in-Chief

For contributors

  • Submit
  • OA Policy
  • Become a Peer-reviewer

Contact us

  • ejournals-vmsta@vtex.lt
  • Mokslininkų 2A
  • LT-08412 Vilnius
  • Lithuania
Powered by PubliMill  •  Privacy policy