We use cookies to provide essential features and services. By using our website you agree to our use of cookies .


Warehouse Stock Clearance Sale

Grab a bargain today!

Information Measures


Product Description
Product Details

Table of Contents

Abstract.- Structure and Structuring.- 1 Introduction.- Science and information.- Man as control loop.- Information, complexity and typical sequences.- Concepts of information.- Information, its technical dimension and the meaning of a message.- Information as a central concept.- 2 Basic considerations.- 2.1 Formal derivation of information.- 2.1.1 Unit and reference scale.- 2.1.2 Information and the unit element.- 2.2 Application of the information measure (Shannon's information).- 2.2.1 Summary.- 2.3 The law of Weber and Fechner.- 2.4 Information of discrete random variables.- 3 Historic development of information theory.- 3.1 Development of information transmission.- 3.1.1 Samuel F. B. Morse 1837.- 3.1.2 Thomas Edison 1874.- 3.1.3 Nyquist 1924.- 3.1.4 Optimal number of characters of the alphabet used for the coding.- 3.2 Development of information functions.- 3.2.1 Hartley 1928.- 3.2.2 Dennis Gabor 1946.- 3.2.3 Shannon 1948.- Validity of the postulates for Shannon's Information.- Shannon's information (another possibility of a derivation).- Properties of Shannon's information, entropy.- Shannon's entropy or Shannon's information.- The Kraft inequality.- Kraft's inequality:.- Proof of Kraft's inequality:.- Limits of the optimal length of codewords.- Shannon's coding theorem.- A sequence of n symbols (elements).- Application of the previous results.- Information and utility (coding, porfolio analysis).- 4 The concept of entropy in physics.- The laws of thermodynamics:.- 4.1 Macroscopic entropy.- 4.1.1 Sadi Carnot 1824.- 4.1.2 Clausius's entropy 1850.- 4.1.3 Increase of entropy in a closed system.- 4.1.4 Prigogine's entropy.- 4.1.5 Entropy balance equation.- 4.1.6 Gibbs's free energy and the quality of the energy.- 4.1.7 Considerations on the macroscopic entropy.- Irreversible transformations.- Perpetuum mobile and transfer of heat.- 4.2 Statistical entropy.- 4.2.1 Boltzmann's entropy.- 4.2.2 Derivation of Boltzmann's entropy.- Variation, permutation and the formula of Stirling.- Special case: Two states.- Example: Lottery.- 4.2.3 The Boltzmann factor.- 4.2.4 Maximum entropy in equilibrium.- 4.2.5 Statistical interpretation of entropy.- 4.2.6 Examples regarding statistical entropy.- Energy and fluctuation.- Quantized oscillator.- 4.2.7 Brillouin-Schroedinger negentropy.- Brillouin: Precise definition of information.- Negentropy as a generalization of Carnot's principle.- Maxwell's demon.- 4.2.8 Information measures of Hartley and Boltzmann.- Examples.- 4.2.9 Shannon's entropy.- 4.3 Dynamic entropy.- 4.3.1 Eddington and the arrow of time.- 4.3.2 Kolmogorov's entropy.- 4.3.3 Renyi's entropy.- 5 Extension of Shannon's information.- 5.1 Renyi's Information 1960.- 5.1.1 Properties of Renyi's entropy.- 5.1.2 Limits in the interval 0 ? ?< ?.- 5.1.3 Nonnegativity for discrete events.- 5.1.4 Additivity and a connection to Minkowski's norm.- 5.1.5 The meaning of S?(A) for ? 1.- 5.1.6 Graphical presentations of Renyi's information.- 5.2 Another generalized entropy (logical expansion).- 5.3 Gain of information via conditional probabilities.- 5.4 Other entropy or information measures.- 5.4.1 Daroczy's entropy.- 5.4.2 Quadratic entropy.- 5.4.3 R-norm entropy.- 6 Generalized entropy measures.- 6.1 The corresponding measures of divergence.- 6.2 Weighted entropies and expectation values of entropies.- 7 Information functions and gaussian distributions.- 7.1 Renyi's information of a gaussian distributed random variable.- 7.1.1 Renyi's ?-information.- 7.1.2 Renyi's G-divergence.- 7.2 Shannon's information.- 8 Shannon's information of discrete probability distributions.- 8.1 Continuous and discrete random variables.- 8.1.1 Summary.- 8.2 Shannon's information of a gaussian distribution.- 8.3 Shannon's information as the possible gain of information in an observation.- 8.4 Limits of the information, limitations of the resolution.- 8.4.1 The resolution or the precision of the measurements.- 8.4.2 The uncertainty relation of the Fourier transformation.- 8.5 Maximization of the entropy of a continuous random variable.- 9 Information functions for gaussian distributions part II.- 9.1 Kullback's information.- 9.1.1 G1 for gaussian distribution densites.- 9.2 Kullback's divergence.- 9.2.1 Jensen's inequality for G1.- 9.3 Kolmogorov's information.- 9.4 Transformation of the coordinate system and the effects on the information.- 9.4.1 S?-information.- 9.4.2 G-divergence.- 9.4.3 S-information.- Example.- 9.4.4 Discrimination information.- 9.4.5 Kolmogorov's information.- 9.4.6 Prerequisites for the transformations.- 9.5 Transformation, discrete and continuous measures of entropy.- 9.6 Summary of the information functions.- 10 Bounds of the variance.- 10.1 Cramer-Rao bound.- 10.1.1 Fisher's information for gaussian distribution densities.- 10.1.2 Fisher's information and Kullback's information.- 10.1.3 Fisher's information and the metric tensor.- 10.1.4 Fisher's information and the stochastic observability.- Fisher's information and the Matrix-Riccati equation.- 10.1.5 Fisher's information and maximum likelihood estimation.- 10.1.6 Fisher's information and weighted least-squares estimation.- 10.1.7 The availability of the Cramer-Rao bound.- 10.1.8 Efficiency, asymptotic efficiency, consistency, bias.- Unbiased estimator.- Consistency.- Efficiency.- 10.1.9 Summary.- 10.2 Chapman-Robbins bound.- 10.2.1 Cramer-Rao bound versus Chapman-Robbins bound.- 10.3 Bhattacharrya bound.- Remark:.- Remark.- 10.3.1 Bhattacharrya bound and Cramer-Rao bound.- 10.3.2 Bhattacharrya's bound for gaussian distribution densities.- 10.4 Barankin bound.- 10.5 Other bounds.- Fraser-Guttman bound.- Kiefer bound.- Extended Fraser-Guttman bound.- 10.6 Summary.- 10.7 Biased estimator.- 10.7.1 Biased estimator versus unbiased estimator.- 11 Ambiguity function.- 11.1 The ambiguity function and Kullback's information.- 11.2 Connection between ambiguity function and Fisher's information.- 11.3 Maximum likelihood estimation and the ambiguity function.- 11.3.1 Maximum likelihood estimation = minimum Kullback estimation = maximum ambiguity estimation = minimum variance estimation.- 11.3.2 Maximum likelihood estimation.- Application: Discriminator (Demodulation).- 11.4 The ML estimation is asymptotically efficient.- 11.5 Transition to the Akaike information criterion.- 12 Akaike's information criterion.- 12.1 Akaike's information criterion and regression.- 12.1.1 Least-squares regression.- 12.1.2 Application of the results to the ambiguity function.- 12.2 BIC, SC or HQ.- 13 Channel information.- 13.1 Redundancy.- 13.1.1 Knowledge, redundancy, utility.- 13.2 Rate of transmission and equivocation.- 13.3 Hadamard's inequality and Gibbs's second theorem.- 13.4 Kolmogorov's information.- 13.5 Kullbacks divergence.- 13.6 An example of a transmission.- 13.7 Communication channel and information processing.- 13.7.1 Semantic, syntactic and pragmatic information.- 13.7.2 Information, first-time occurrence, confirmation.- 13.8 Shannon's bound.- 13.9 Example of the channel capacity.- 14 'Deterministic' and stochastic information.- 14.1 Information in state space models.- 14.2 The observation equation.- 14.3 Transmission faster than light.- 14.4 Information about state space variables.- 15 Maximum entropy estimation.- 15.1 The difference between maximum entropy and minimum variance.- 15.2 The difference from bootstrap or resampling methods.- 15.3 A maximum entropy example.- 15.4 Maximum entropy: The method.- 15.4.1 Maximum Shannon entropy.- 15.4.2 Minimum Kullback-Leibler distance.- 15.5 Maximum entropy and minimum discrimination information.- 15.6 Generation of generalized entropy measures.- 15.6.1 Example: Gaussian distribution and Shannon's information.- 16 Concluding remarks.- 16.1 Information, entropy and self-organization.- 16.2 Complexity theory.- 16.3 Data reduction.- 16.4 Cryptology.- 16.5 Concluding considerations.- 16.5.1 Information, entropy and probability.- 16.6 Information.- A.1 Inequality for Kullback's information.- A.2 The log-sum inequality.- A.3 Generalized entropy, divergence and distance measures.- A.3.1 Entropy measures.- A.3.2 Generalized measures of distance.- A.3.3 Generalized measures of the directed divergence.- A.3.4 Generalized measures of divergence.- A.3.4.1 Information radius and the J-divergence.- A.3.4.2 Generalization of the R-divergence.- A.3.4.3 Generalization of the J-divergence.- A.4 A short introduction to probability theory.- A.4.1 Axiomatic definition of probability.- A.4.1.1 Events, elementary events, sample space.- A.4.1.2 Classes of subsets, fields.- A.4.1.3 Axiomatic definition of probability according to Kolmogorov.- Probability space.- A.4.1.4 Random variables.- A.4.1.5 Probability distribution.- A.4.1.6 Probability space, sample space, realization space.- A.4.1.7 Probability distribution and distribution density function.- A.4.1.8 Probability distribution density function (PDF).- A.5 The regularity conditions.- A.6 State space description.

Promotional Information

Springer Book Archives


"Bioinformaticians are facing the challenge of how to handle immense amounts of raw data, such as are generated from genome mapping, make sense of them, and render them accessible to scientists working on a wide variety of problems. "Information Measures: Information and its Description in Science and Engineering" can be such a tool."

IEEE Engineering in Medicine and Biology

Ask a Question About this Product More...
Write your question below:
Item ships from and is sold by Fishpond World Ltd.
Back to top