A Generalized Geometric Distribution from Vertically Dependent Bernoulli Random Variables

A Generalized Geometric Distribution from Vertically Dependent Bernoulli Random Variables

 For full proofs and derivations, read here.

Properties  of the Generalized Geometric Distribution

Moment Generating Function


Fact. 

The moment generating function of the generalized geometric distribution is
M_{X}(t) = pe^{t} + \frac{qp^{-}e^{2t}}{1-q^{+}e^{t}}


 

Mean


Fact. 

The mean of the generalized geometric distribution is
E[X] = \mu = \frac{1-\delta p}{p(1-\delta)}


 

The effect of dependence can be seen in the plot of E[X] below in Figure 1.. For fixed p, when \delta \to 1, E[X] \to \infty, though the rate changes with p.

Figure 1. Expectation of the Generalized Geometric Distribution.

 

Figure 2. Expected value of generalized geometric distribution with fixed p=1/2

To explore further, suppose p = 1/2, and the Bernoulli trials are thus balanced between success and failure. Figure 2 shows the effect of delta for a fixed p. Notice that the effect of \delta on the expected value become more pronounced as \delta \to 1. In particular, for \delta = 1/2 and p=1/2, E[X] = 3, but after this point, an increase of only 1/6 in \delta to \delta = 2/3 increased the expected value to 4 trials before a success. To double the expected number of trials before a success again to E[X] = 8 requires an increase of \delta by only 4/21 to 6/7.

A smaller probability of success p will yield an expected value \mu that is much more susceptible to effects of dependency \delta, and a larger p will yield an expected value more resistant to high dependency \delta. Since the geometric distribution is a count of the number of trials needed to obtain the first success, a higher p increases the probability that the first success occurs on the first trial, while a lower p decreases that probability. Therefore, the dependency \delta would have a higher effect for lower p, because a longer (dependent) sequence is expected to be generated prior to the first success, which increases the expected number of trials faster than if the Bernoulli trials were independent.

Remark. Notice that when \delta = 0, the Bernoulli trials are independent. The mean of the generalized geometric distribution when \delta = 0 is E[X] = \frac{1}{p}, the mean of the standard geometric distribution.

 

Variance


Fact. 

The variance of the generalized geometric distribution is

\text{Var}(X) = \sigma^{2} = \frac{1-p + \delta p(1-p)}{p^2(1-\delta)^2}

 

Figure 3. Variance as a function of dependency for fixed p.

Figure 3 shows the effect of \delta on the variance for different values of p. As with the mean, a smaller p induces a higher effect of \delta on the variance of the number of dependent Bernoulli trials before the first success is observed. The shape of all 4 cases is similar, but the scales are vastly different. As p increases, the scale of the variance decreases dramatically.

Remark.  Again, note that when \delta = 0, the variance of the generalized geometric distribution reduces to that of the standard geometric distribution.

 

Skew


Fact. 

The skew of the generalized geometric distribution is given by
\text{Skew}[X] = \frac{2-3p + p^2 + \delta p[q+\delta p q + p(2\delta-1-p)]}{\left(q + \delta pq\right)^{3/2}}


Figure 4. Skew of Generalized Geometric Distribution as a function of p

The skew of the generalized geometric distribution gets more complicated in its behavior as a function of both p and \delta. Figure 4 shows the skew as a function of p for the two extreme cases: complete independence (\delta = 0) and complete dependence \delta = 1. From p=0 to p \approx 0.658, the skew for the independent geometric distribution is greater than the completely dependent case. For p \gtrapprox 0.658, the skew is greater under complete dependence.

 

Entropy

The entropy of a random variable measures the average information contained in the random variable. It can also be viewed as a measure of how unpredictable or “truly random” the variable is [1] . The definition of entropy, denoted H(X), was coined by Claude Shannon [3] in 1948.

 

Definition. (Entropy)
H(X) := -\sum_{i}P(x_{i})\log_{2}(P(x_{i}))

 

For the standard geometric distribution, the entropy is given by

H_{sg}(X) = \frac{-(1-p)\log_{2}(1-p)-p\log_{2}(p)}{p}

 


Fact.
The entropy for the generalized geometric distribution is
H_{gg}(X) = \frac{-\left[pp^{-}\log_{2}(p) + qp^{-}\log_{2}(qp^{-}) + qq^{+}\log_{2}(q^{+})\right]}{p^{-}}


 

Figure 5a. Entropy of the generalized geometric distribution as a function of p
 
Figure 5b. Entropy as a function of dependency for fixed p

 

Figure 5a shows H_{gg}(X) as a function of p for fixed values of \delta. Notice that while the entropy decreases to 0 for all curves as p \to 1, the entropy curve is shifted upward for larger \delta. Figure 5b fixes p and looks at entropy as a function of \delta. Notice that for smaller p, the entropy is much higher, which aligns with intuition.