Exams › GATE › Engineering Mathematics

Let H(X) denote the entropy of a discrete random variable X taking K possible distinct real values. Which of the following statements is/are necessarily true?

1. H(X) ≤ log₂ K bits
2. H(X) ≤ H(2X)
3. H(X) ≤ H(X²)
4. H(X) ≤ H(2^X)

Correct answer: H(X) ≤ log₂ K bits

Solution

The entropy H(X) measures the uncertainty in a random variable X, and it is maximized when all K outcomes are equally likely. Therefore, the maximum entropy is log₂ K bits, which means H(X) cannot exceed this value.

Related GATE Engineering Mathematics questions

• Organic fraction of municipal solid waste (OFMSW) with bulk density of 315 kg/m³ and water content of 30% is mixed with municipal sludge of bulk density 700 kg/m³ and water content of 70%, such that the water content of the mixture is 40%. The amount (in kg) of sludge to be mixed per kg of OFMSW (rounded off to 2 decimal places) and the density of the mixture (in kg/m³) (rounded off to the nearest integer) are calculated. Which of the following options is/are true:
• A probability distribution with right skew is shown in the figure. The correct statement for the probability distribution is
• The probability density function of a continuous random variable distributed uniformly between x and y (for y > x) is
• The dimension of dynamic viscosity is:
• A single-lane highway has a traffic density of 40 vehicles/km. If the time-mean speed and space-mean speed are 40 kmph and 30 kmph, respectively, the average headway (in seconds) between the vehicles is
• A fair (unbiased) coin was tossed four times in succession and resulted in the following outcomes: (i) H head, (ii) H head, (iii) H head, (iv) H head. The probability of obtaining a 'Tail' when the coin is tossed again is

⚔️ Practice GATE Engineering Mathematics free + battle 1v1 →