Generalizing the Negative Binomial Distribution via First-Kind Dependence

# Generalizing the Negative Binomial Distribution via First-Kind Dependence

This paper generalizes the negative binomial random variable by generating it from a sequence of first-kind dependent Bernoulli trials under the identity permutation. The PMF, MGF, and various moments are provided, and it is proven that the distribution is indeed an extension of the standard negative binomial random variable. We examine the effect of complete dependence of the Bernoulli trials on the generalized negative binomial random variable. We also show that the generalized geometric random variable is a special case of the generalized negative binomial random variable, but the generalized negative binomial random variable cannot be generated from a sum of i.i.d. generalized geometric random variables.

## Introduction

A binomial random variable $Z_n$ is constructed from a sequence of $n$ Bernoulli random variables ${\epsilon_{1}, \epsilon_{2},\ldots,\epsilon_{n}}$, and counts the number of 1s, or “successes” in the sequence. Mathematically, a binomial random variable is given by $Z_{n} = \sum_{i=1}^{n}\epsilon_{i}$. A traditional binomial random variable requires an i.i.d. sequence of Bernoulli random variables. Korzeniowski [1] developed a generalized binomial distribution under the condition that the Bernoulli sequence is first-kind dependent.

A “different perspective”, as it were, to the binomial random variable is the negative binomial random variable. With a binomial random variable, we fix the number of trials and count the number of “successes”. Suppose now we fix the number of successes as $k$ and continue to run Bernoulli trials until the $k$th success. The random number of trials necessary to get $k$ successes is a negative binomial random variable, and may be formulated mathematically as $V_k =\min_{n \geq k}(n:Z_n = k)$. The sequence is halted when the $k$th success appears, which will always be on the last trial. Thus, the event $(V_{k} = n)$ is equivalent to $(Z_{n-1} = k-1 \wedge \epsilon_{n} = 1)$. A standard negative binomial distribution is constructed from an i.i.d. sequence of Bernoulli random variables, just like the binomial random variable. The PMF is given by

$$P(V_{k} = n) = {n-1 \choose k-1}p^{k}(1-p)^{n-k},\quad n \geq k$$

We may also characterize the negative binomial distribution in a different way. Let $Y$ denote the number of additional trials beyond the minimum possible $k$ required for $k$ successes. Since the trials are Bernoulli trials, $Y$ denotes the random number of failures that will occur before the $k$th success is observed. Thus, if we denote $y$ as the number of failures, $n = k + y$, where $k$ is fixed, and thus the random variable $Y$ with support $\{0,1,2,\ldots\}$ is equivalent to the previous characterization of $V_k$. The PMF of $Y$ is easily derived and given by

$$P(Y=y) = {k+y-1 \choose y}p^{k}(1-p)^{y}$$

A first-kind dependent sequence of Bernoulli trials is identically distributed but dependent ([1], [2], [4]) (see the papers for [2] and [4] here and here.), and thus can generate generalized versions of random variables that are functions of sequences of identically distributed Bernoulli or categorical random variables ([2], [3]) (You can reach these papers here and here.). This paper generalizes the negative binomial distribution given above by allowing the Bernoulli sequence to be first-kind dependent.

## Derivation of the PMF

Theorem 1:
Let $k\in\mathbb{N}$ be fixed, and $\epsilon = \{\epsilon_{1},\epsilon_{2},\ldots\}$ be a sequence of first-kind dependent Bernoulli trials under the identity permutation with $P(\epsilon_{k} = 1) = p$ and dependency coefficient $\delta$. Define $q=1-p$, $p^{+} = p + \delta q$, $p^{-} = p-\delta p$, $q^{+} = q + \delta p$, and $q^{-} = q-\delta q$. Let $Z_{n}$ denote a generalized binomial random variable of length $n$. Let $V_{k}$ denote the random variable that counts the number of first-kind dependent Bernoulli trials until the $k$th success. That is, $V_k =\min_{n \geq k}(n:Z_n =k)$. Then the PMF of $V_k$ is given by

$$P(V_{k} = n) = p{n-2 \choose k-2}(p^{+})^{k-1}(q^{-})^{n-k}+q{n-2\choose k-1}(p^{-})^{k}(q^{+})^{n-k-1}$$

It will be more helpful to characterize the generalized negative binomial random variable our alternative way, by letting $V_{k} = Y+k$, where $Y$ is the random variable that counts the number of additional trials beyond the minimum possible $k$ necessary to achieve the $k$th success or, equivalently, the number of failures in a sequence of FK-dependent Bernoulli variables with $k$ successes.

The PMF of $Y$ is given in the following corollary

Corollary 1:

Let $Y$ be the random variable described here, with support $\{0,1,2,\ldots\}$. Then $Y$ is equivalent to $V_{k}$, and the PMF of $Y$ is given by

$$P(Y=y) = p{y+k-2 \choose y}(p^{+})^{k-1}(q^{-})^{y} + q{y+k-2 \choose y-1}(p^{-})^{k}(q^{+})^{y-1}$$

When $\delta = 0$, a FK-dependent Bernoulli sequence becomes a standard i.i.d. Bernoulli sequence. Thus, when $\delta = 0$, $Y$reverts to a standard negative binomial distribution.

Corollary 2:

Let $Y$ be a generalized negative binomial distribution constructed via FK-dependency under the identity permutation with dependency coefficient $\delta$. When $\delta = 0$, $Y$ is a standard negative binomial random variable.

## The Moment Generating Function and Various Moments

Theorem 2:

The moment generating function of the generalized negative binomial distribution is given by

$$M_{Y}(t) = \frac{p(p^{+})^{k-1}}{(1-e^{t}q^{-})^{k-1}} + \frac{q(p^{-})^{k}e^{t}}{(1-e^{t}q^{+})^{k}}$$

Remark:  This is indeed a generalization of the standard negative binomial distribution, as it is now a special case of the generalized negative binomial distribution when $\delta = 0$. To see this, we will show that the MGF of the generalized negative binomial distribution becomes the MGF for the standard negative binomial distribution.

When $\delta = 0$, a FK-dependent sequence reverts back to a standard i.i.d. sequence. That is, $p^{+}=p^{-}=p$, and $q^{+} = q^{-}=q$. So in this case,\begin{aligned}M_{Y}(t) &= \frac{p^{k}}{(1-e^{t}q)^{k-1}} + \frac{qp^{k}e^{t}}{(1-e^{t}q)^{k}}\\&= \frac{(1-e^{t}q)p^{k} + qp^{k}e^{t}}{(1-e^{t}q)^{k}}\\&=\frac{p^{k}}{(1-e^{t}q)^{k}}\end{aligned}

We may now derive the various moments of the generalized negative binomial distribution using the moment generating function.

### Mean of generalized negative binomial distribution

The mean of the generalized negative binomial distribution is given by
$$\mu_{Y} = \text{E}[Y] = \frac{kpq + k\delta q^{2}-(k-1)\delta pq(1-\delta)}{p^{2}(1-\delta)+\delta pq(1-\delta)}$$

Remark: Note the reduction of the GNB mean to that of the standard negative binomial distribution when the sequence is independent. For $\delta = 0$, $\text{E}[Y] = \frac{kq}{p}$.

### Variance of the generalized negative binomial distribution

After many attempts to distill the formula to a palatable expression, the variance of the generalized negative binomial distribution is given by

\begin{aligned}\text{Var}[Y]&=\frac{kq}{(p+q\delta)^{2}(1-\delta)^{2}}\\&\quad+\dfrac{\delta(p^{3}q + kpq^{2}-kq^{3})}{p^{2}(1-\delta^{2})(p+q\delta)^{2}}+\dfrac{\delta^{2}(k^{2}pq + kq -3kpq^{2}-2p^{3}q)}{p^{2}(1-\delta^{2})(p+q\delta)^{2}}\\&\quad\quad+\dfrac{\delta^{3}pq(p^{2}-k-2kq)}{p^{2}(1-\delta^{2})(p+q\delta)^{2}}\end{aligned}

Remark: Once again, under independence, $\text{Var}[Y] = \frac{kq}{p^{2}}$, the variance of the standard negative binomial. Other higher order moments can also be obtained from the moment generating function and copious amounts of tedious arithmetic.

## Other Considerations

### The effect of complete dependence

It’s also worth exploring the other extremes, like complete dependence. As illustrated in [2], complete dependence under FK-dependence implies that every Bernoulli trial will be identical to the outcome of the first trial. Thus, if $\epsilon_{1} = 0$, the entire sequence will be all 0s, and vice versa if $\epsilon_{1} = 1$. What does that mean for the generalized negative binomial distribution? If $\epsilon_{1} = 0$, the sequence will never end; $k$ successes will never happen. On the other hand, if $\epsilon_{1} = 1$, then you are guaranteed to reach $k$ successes in $k$ trials. This results in both an infinite mean and variance, seen by plugging in $\delta = 1$.

Exploring the PMF under $\delta = 1$, $P(Y=0) = p$, because $P(\epsilon_{1} = 1) = p$. If $\epsilon_{1}= 1$, and $\delta = 1$, then there will be only 1s in the sequence, and no 0s, and thus $P(Y=0) = P(\epsilon_{1} = 1)$. If $\epsilon_{1} = 0$, then there are only 0s in the FK-dependent Bernoulli sequence, and no 1s. Thus, $Y$ can only be $\infty$, and $P(Y = \infty) = P(\epsilon_{1} = 0) = q$.

Remark: When we say $Y = \infty$, we mean that the sequence of trials has no halting point. That is, the counting process never ends.

Thus, under complete dependence of the first kind, the support of $Y$ has two points $\{0,\infty\}$, with probabilities $p$ and $q$ respectively. This is another way to confirm that $Y$ will have infinite mean and variance in this case.

### The negative binomial random variable as a sum of geometric random variables

The standard negative binomial distribution with $k$ fixed successes can be derived as a sum of independent standard geometric random variables. One shows this by showing the moment generating function of the standard negative binomial distribution is equal to the product of $k$ i.i.d. standard geometric random variables. Moreover, it can also be shown that the standard geometric random variable is a special case of the standard negative binomial distribution when $k = 1$.

How much of this carries over to the generalized versions of both distributions? The generalized geometric distribution was introduced and detailed by Traylor in [2]. Here, we are concerned with the PMF of “Version 2” of the generalized geometric distribution, as the “shifted” generalized geometric distribution counts the number of failures prior to the first success, and is analogous to counting the number of failures in a sequence of trials before the $k$th success. We reproduce Proposition 2 from [2] here:

Proposition 2, [2]: Suppose $\epsilon = (\epsilon_{1},\epsilon_{2},\ldots,\epsilon_{n},\ldots)$ is a FK-dependent sequence of Bernoulli random variables. Let $Z = X-1$ be the count of failures prior to the first success. Then $Z$ has a shifted generalized geometric distribution with PMF

$$f_{Z}(z) = \left\{\begin{array}{lr}p, &z=0\\q(q^{+})^{z-1}p^{-}, &z\geq 1\end{array}\right.$$

We can quickly derive its MGF:

Proposition 1:
The moment generating function of the shifted generalized geometric distribution is given by

$$M_{Z}(t) = p + \frac{e^{t}qp^{-}}{1-e^{t}q^{+}}$$

#### Generalized geometric RV as a special case of generalized negative binomial RV

It is true that, for $k = 1$, the generalized negative binomial distribution under FK-dependence reduces to the generalized geometric distribution. This is given in the following theorem.

Theorem 3:

When $k=1$, the generalized negative binomial random variable reduces to a generalized geometric random variable.

(To see this, simply plug in $k=1$ to the PMF of the generalized negative binomial distribution.)

#### Sum of independent generalized geometric random variables does not yield a generalized negative binomial random variable

Unlike the standard case, a sum of i.i.d. generalized geometric random variables does not yield a generalized negative binomial random variable. First, we note what we mean by a set of i.i.d. generalized geometric random variables. Suppose we have a set of generalized geometric random variables $\{X_{1},X_{2},\ldots,X_{k}\}$, each with the same $p$, $q$, and $\delta$, and all first-kind dependent. Thus, to say that each of these geometric random variables is mutually independent of the others is to say that nothing about the other geometric random variables has any probabilistic bearing on the variable in question. That is, the dependency structure is not changed or altered, and $P(X_{i}|X_{j}) = P(X_{i})$, $i\neq j$, $i=1,2,\ldots,k$. The Bernoulli random variables that make up each geometric random variable still remain FK-dependent among themselves. That is, if $\epsilon_{i} = (\epsilon_{1}^{(i)},\epsilon_{2}^{(i)},\ldots,\epsilon_{n_{i}}^{(i)})$ is the sequence of Bernoulli trials that comprises $X_{i}$, each $\epsilon_{i}$ is FK-dependent among its elements, but independent of the other sequences $\epsilon_{j}$.

One can easily see that, if the generalized negative binomial distribution were able to be generated by the sum of i.i.d. generalized geometric random variables, then $M_{Y}(t) = \prod_{i=1}^{k}M_{X_{i}}(t)$. But

\begin{aligned}\prod_{i=1}^{k}M_{X_{i}}(t) &=\left(p + \frac{e^{t}qp^{-}}{1-e^{t}q^{+}}\right)\\&\neq \frac{p(p^{+})^{k-1}}{(1-e^{t}q^{-})^{k-1}} + \frac{q(p^{-})^{k}e^{t}}{(1-e^{t}q^{+})^{k}}\\&= M_{Y}(t)\end{aligned}

Why is this? The answer is quite intuitive. The generalized negative binomial distribution under FK-dependence is one sequence under a FK dependency structure. That is, all Bernoulli trials after the first depend directly on the first. Summing generalized geometric random variables under FK dependence is equivalent to constructing a sequence of generalized geometric random variables, one after the other. Since these are themselves comprised of FK-dependent Bernoulli trials, each time a success is observed, the dependency structure ”starts over” with the next geometric random variable.

For example, suppose the first geometric random variable has a success on the third trial. Then the fourth Bernoulli trial is starting an entirely new sequence of FK-dependent Bernoulli variables, and does not depend on the outcome of the first. This is not equivalent to the definition of a generalized negative binomial distribution.

This property held for the standard versions of the geometric and negative binomial random variables because every Bernoulli trial in each geometric sequence is i.i.d. Thus, there is no “transition” from one geometric random variable to another; it’s as if it was all one big sequence of Bernoulli trials to begin with. We lose that when we introduce dependency structures.

## Conclusion

This paper introduced the generalized negative binomial distribution built from a sequence of FK- dependent random variables. The PMF, MGF, and various moments were derived. It was also noted that the generalized geometric distribution is a special case of the generalized negative binomial distribution, but one cannot construct a generalized negative binomial random variable from the sum of i.i.d. generalized geometric random variables.

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

## References

1. Andrzej Korzeniowski. On correlated random graphs. Journal of Probability and Statistical Science, pages 43–58, 2013.
2. Rachel Traylor. A generalized geometric distribution from vertically dependent categorical random variables. Academic Advances of the CTO, 2017.
3.  Rachel Traylor. A generalized multinomial distribution from dependent categorical random variables. Academic Advances of the CTO, 2017.
4.  Rachel Traylor and Jason Hathcock. Vertical dependency in sequences of categorical random variables. Academic Advances of the CTO, 2017.

Comments are closed.