The concept of entropy in information theory is closely related to thermodynamic entropy in physics. Boltzmann and Gibbs have done a lot of work on thermodynamic entropy in statistical physics. Entropy in information theory is also inspired by it. Shannon 1948+00' s paper "Mathematical Theory of Communication" published in Bell Journal of System Technology in June is generally regarded as the beginning of modern information theory research. This paper is based in part on the previous work of harry nyquist and Ralph Hartley. In this paper, Shannon gives the definition of information entropy.
Satisfied, please adopt.