Introduction to Beta Distribution

The beta distribution is a probability distribution with a probability density function that can be described using the Beta function, and it is commonly encountered in the context of order statistics.

Some authorUpdated 10 min read

Introducing the beta distribution.

Beta distribution follows from the Beta function by parameterization. We know that beta distribution is defined in terms of Gamma function by the following:

B(X,Y)=Γ(X)Γ(Y)Γ(X+Y)\mathrm{B}(X,Y) = \frac{\Gamma(X) \Gamma(Y)}{\Gamma(X+Y)}

And so we can define naturally the beta distribution:

fX(x)=1B(a,b)xa−1(1−x)b−1f_X(x) = \frac{1}{B(a, b)}x^{a-1}(1-x)^{b-1}

with support (0,1)(0,1).

Beta distribution exists widely with order statistics, specifically, if we have nn i.i.d random variables, we want to know how likely one of the variables (after sorted) is in terms of its value. We will encounter beta distribution.

We can first visualize the beta distribution from visualization provided.

Beta distribution

If adjust the parameter a,ba,b, we will find the skewness of the shape changes.

Specifically, when a=ba = b, the skewness remains the same. However, when a>ba > b, we will have the shape being left skewed (meaning more concentration on the right). And when a<ba < b, we will have the distribution to be more right skewed.

Now, let's dive into important characteristics of the beta distribution.

We will borrow the proof provided by statproofbook

We know the definition of moment-generating function will be

MX(t)=E[etX]=∫011B(a,b)exp⁡(tx)xa−1(1−x)b−1 dxM_X(t) = \mathbb{E}[e^{tX}] = \int_0^1 \frac{1}{\mathrm{B}(a, b)}\exp(tx)x^{a-1}(1-x)^{b-1}\, dx =1B(a,b)∫01exp⁡(tx)xa−1(1−x)b−1 dx= \frac{1}{\mathrm{B}(a, b)}\int_0^1\exp(tx)x^{a-1}(1-x)^{b-1}\, dx

We know the integral representation of the confluent hypergeometric function is the following:

1F1(a,b,z)=1B(a,b−a)∫01exp⁡(zx)xa−1(1−x)b−a−1 dx{}_1F_1(a,b,z) = \frac{1}{B(a, b-a)} \int_0^1 \exp(zx) x^{a-1}(1-x)^{b-a-1}\, dx

And so we can write the mgf as

MX(t)=1F1(a,a+b,t)M_X(t) = {}_1 F_1(a, a+b, t)

We know that the series equation for confluent hypergeometric function is

1F1(a,b,z)=∑n=0∞an‾bn‾ znn!{}_1 F_1(a,b,z) = \sum_{n=0}^{\infty} \frac{a^{\overline{n}}}{b^{\overline{n}}} \, \frac{z^n}{n!}

where

mn‾=∏i=0n−1(m+i)m^{\overline{n}} = \prod_{i=0}^{n-1} (m+i)

,

So we will have

1F1(a,a+b,t)=∑n=0∞an‾(a+b)n‾tnn!=1+∑n=1∞∏i=0n−1(a+i)∏i=0n−1(a+b+i)tnn!=1+∑n=1∞∏i=0n−1a+ia+b+itnn!=1+∑n=1∞Γ(a+n)Γ(a+b)Γ(a)Γ(a+b+n)Γ(n+1)tn{}_1 F_1(a, a+b, t) = \sum_{n=0}^{\infty} \frac{a^{\overline n}}{(a+b)^{\overline n}} \frac{t^n}{n!} = 1+\sum_{n=1}^{\infty} \frac{\prod_{i=0}^{n-1}(a+i)}{\prod_{i=0}^{n-1}(a+b+i)} \frac{t^n}{n!} = 1+\sum_{n=1}^{\infty} \prod_{i=0}^{n-1} \frac{a+i}{a+b+i} \frac{t^n}{n!} = 1 + \sum_{n=1}^{\infty} \frac{\Gamma(a+n)\Gamma(a+b)}{\Gamma(a)\Gamma(a+b+n)\Gamma(n+1)} t^n

To calculate the kkth derivative of the expected value E[eXt]\mathbb{E}[e^{Xt}] with respect to tt, we need to use the properties of the moment generating function (MGF) and its derivatives.

The MGF of a random variable XX is defined as E[eXt]\mathbb{E}[e^{Xt}]. The kkth derivative of the MGF evaluated at t=0t=0 is related to the kkth raw moment of XX. In general, the kkth derivative of the MGF evaluated at t=0t=0 can be expressed as follows:

dkdtkE[eXt]∣t=0=E[Xk]\frac{d^k}{dt^k}\mathbb{E}[e^{Xt}] \bigg|_{t=0} = \mathbb{E}[X^k]

So, the kkth derivative of the expected value E[eXt]\mathbb{E}[e^{Xt}] with respect to tt evaluated at t=0t=0 is equal to the kkth raw moment of the random variable XX.

We know that the series equation for confluent hypergeometric function is

1F1(a,b,z)=∑n=0∞an‾bn‾ znn!{}_1 F_1(a,b,z) = \sum_{n=0}^{\infty} \frac{a^{\overline{n}}}{b^{\overline{n}}} \, \frac{z^n}{n!}

where

mn‾=∏i=0n−1(m+i)m^{\overline{n}} = \prod_{i=0}^{n-1} (m+i)

,

So we will have

1F1(a,a+b,t)=∑n=0∞an‾(a+b)n‾tnn!=1+∑n=1∞∏i=0n−1(a+i)∏i=0n−1(a+b+i)tnn!=1+∑n=1∞∏i=0n−1a+ia+b+itnn!=1+∑n=1∞Γ(a+n)Γ(a+b)Γ(a)Γ(a+b+n)Γ(n+1)tn{}_1 F_1(a, a+b, t) = \sum_{n=0}^{\infty} \frac{a^{\overline n}}{(a+b)^{\overline n}} \frac{t^n}{n!} = 1+\sum_{n=1}^{\infty} \frac{\prod_{i=0}^{n-1}(a+i)}{\prod_{i=0}^{n-1}(a+b+i)} \frac{t^n}{n!} = 1+\sum_{n=1}^{\infty} \prod_{i=0}^{n-1} \frac{a+i}{a+b+i} \frac{t^n}{n!} = 1 + \sum_{n=1}^{\infty} \frac{\Gamma(a+n)\Gamma(a+b)}{\Gamma(a)\Gamma(a+b+n)\Gamma(n+1)} t^n

We also know that dkdtkE[eXt]∣t=0=E[Xk]\frac{d^k}{dt^k}\mathbb{E}[e^{Xt}] \bigg|_{t=0} = \mathbb{E}[X^k], so we have

E[X]=Γ(a+1)Γ(a+b)Γ(a)Γ(a+b+1)=aa+b\mathbb{E}[X]= \frac{\Gamma(a+1)\Gamma(a+b)}{\Gamma(a)\Gamma(a+b+1)} = \frac{a}{a+b}

And

E[X2]=Γ(a+2)Γ(a+b)Γ(a)Γ(a+b+2)=a(a+1)(a+b)(a+b+1)=a2+aa2+2ab+a+b2+b\mathbb{E}[X^2] = \frac{\Gamma(a+2)\Gamma(a+b)}{\Gamma(a) \Gamma(a+b+2)} = \frac{a(a+1)}{(a+b)(a+b+1)} = \frac{a^2+a}{a^2 + 2ab + a + b^2 + b} E[X3]=a(a+1)(a+2)(a+b)(a+b+1)(a+b+2)=(a2+a)(a+2)(a2+2ab+a+b2+b)(a+b+2)=2a+3a2+a3a3+3a2b+3a2+3ab2+6ab+2a+b3+3b2+2b\mathbb{E}[X^3] = \frac{a(a+1)(a+2)}{(a+b)(a+b+1)(a+b+2)}= \frac{(a^2+a)(a+2)}{(a^2+2ab+a+b^2+b)(a+b+2)} = \frac{2 a + 3 a^2 + a^3}{a^3 + 3 a^2 b + 3 a^2 + 3 a b^2 + 6 a b + 2 a + b^3 + 3 b^2 + 2 b}

Note that the variance is calculated as

Var(X)=E[X2]−(E[X])2=a(a+1)(a+b)(a+b+1)−a2(a+b)2=a(a+1)(a+b)−a2(a+b+1)(a+b)2(a+b+1)=ab(a+b)2(a+b+1)Var(X) = \mathbb{E}[X^2]-(\mathbb{E}[X])^2 = \frac{a(a+1)}{(a+b)(a+b+1)} - \frac{a^2}{(a+b)^2}= \frac{a(a+1)(a+b)-a^2(a+b+1)}{(a+b)^2(a+b+1)} = \frac{ab}{(a+b)^2 (a+b+1)}

We now consider a generic distribution's moments, specifically, we consider a distribution with the following mgf:

MX(t)=∑n=0∞α(n)Γ(n)tnM_X(t) = \sum_{n=0}^{\infty} \frac{\alpha(n)}{\Gamma(n)} t^n

,

where α(n)\alpha(n) is nn-th moment of the distribution.

A speedy way of computing the skewness/kurtosis is by considering solely

E[exp⁡((X−μ)σt)]=exp⁡(−μt)MX(t)=(∑n=0∞(−1)nμnΓ(i+1)σntn)(∑n=0∞α(n)Γ(n)σntn)\mathbb{E}\left[\exp\left(\frac{(X-\mu)}{\sigma}t \right)\right] = \exp(-\mu t) M_X(t) = \left(\sum_{n=0}^{\infty} \frac{(-1)^n\mu^n }{\Gamma(i+1)\sigma^n}t^n\right)\left( \sum_{n=0}^{\infty} \frac{\alpha(n)}{\Gamma(n)\sigma^n} t^n\right)

Let u(k):=dkdtkE[e(X−μ)t]∣t=0u(k):=\frac{d^k}{dt^k} \mathbb{E}[e^{(X-\mu)t}]\bigg|_{t=0}

So

u(3)=6σ3×(−μ36+μ32−σ2+μ22μ+α(3)6)=α(3)−3μσ2−μ3σ3u(3) = \frac{6}{\sigma^3}\times\left( -\frac{\mu^3}{6} + \frac{\mu^3}{2} - \frac{\sigma^2 +\mu^2}{2} \mu + \frac{\alpha(3)}{6} \right) = \frac{\alpha(3) - 3\mu \sigma^2 - \mu^3}{\sigma^3}

And

u(4)=24σ4(μ424−μ46+μ2(σ2+μ2)4−μα(3)6+α(4)24)u(4) = \frac{24}{\sigma^4} \left( \frac{\mu^4}{24} - \frac{\mu^4}{6} + \frac{\mu^2 (\sigma^2 + \mu^2)}{4} - \frac{\mu \alpha(3)}{6}+ \frac{\alpha(4)}{24}\right) =6μ2σ2+3μ4−4μα(3)+α(4)σ4= \frac{ 6\mu^2\sigma^2 + 3\mu^4 - 4\mu \alpha(3)+\alpha(4)}{\sigma^4}

Now let's compute the skewness of our beta distribution

So we will focus on

α(3)−3μσ2−μ3σ3\frac{\alpha(3) - 3\mu \sigma^2 - \mu^3}{\sigma^3}

We know that μ=aa+b\mu = \frac{a}{a+b} and σ2=ab(a+b)2(a+b+1)\sigma^2 = \frac{ab}{(a+b)^2(a+b+1)} and we also have α(3)=a(a+1)(a+2)(a+b)(a+b+1)(a+b+2)\alpha(3) = \frac{a (a+1) (a+2) }{ (a+b) ( a+ b+ 1) (a+b+2)}, so we will have the following as the original:

a(a+1)(a+2)(a+b)(a+b+1)(a+b+2)−3a2b(a+b)3(a+b+1)−a3(a+b)3ab(a+b)3(a+b+1)aba+b+1\frac{\frac{a(a+1)(a+2)}{(a+b)(a+b+1) (a+b+2)} - \frac{3a^2 b}{(a+b)^3 ( a+b+1)}- \frac{a^3}{(a+b)^3}}{\frac{ab}{(a+b)^3 (a+b+1)} \sqrt{\frac{ab}{ a+b+1}}} =(a+1)(a+b+2)−b(a+1)a+b+2−3ab(a+b)2−a3+a2b+a2(a+b)2b(a+b)2aba+b+1= \frac{\frac{(a+1)(a+b+2) - b(a+1)}{a+b+2} - \frac{3a b}{(a+b)^2 }- \frac{a^3+a^2b+a^2}{(a+b)^2}}{\frac{b}{(a+b)^2 } \sqrt{\frac{ab}{ a+b+1}}} =(a+1)(a+b+2)−b(a+1)a+b+2−3ab(a+b)2−a3+a2b+a2(a+b)2b(a+b)2aba+b+1= \frac{\frac{(a+1)(a+b+2) - b(a+1)}{a+b+2} - \frac{3a b}{(a+b)^2 }- \frac{a^3+a^2b+a^2}{(a+b)^2}}{\frac{b}{(a+b)^2 } \sqrt{\frac{ab}{ a+b+1}}} =(a+1)(a+b)2−b(a+1)(a+b)2a+b+2−3ab−a3−a2b−a2baba+b+1= \frac{(a+1)(a+b)^2 - \frac{b(a+1)(a+b)^2}{a+b+2}- 3ab -a^3-a^2b-a^2}{b \sqrt{\frac{ab}{ a+b+1}}} =a(a+b)−a+b−(a+1)(a+b)2a+b+2aba+b+1= \frac{a(a + b)-a+b- \frac{(a+1)(a+b)^2}{a+b+2}}{\sqrt{\frac{ab}{ a+b+1}}} =a(a+b)(a+b+2)+(b−a)(a+b+2)−(a+1)(a+b)2(a+b+2)aba+b+1= \frac{a(a + b)(a+b+2)+ (b-a)(a+b+2)- (a+1)(a+b)^2}{(a+b+2)\sqrt{\frac{ab}{ a+b+1}}} =(a2+ab+a+b)(a+b)+2(b−a)−(a+1)(a+b)2(a+b+2)aba+b+1=2(b−a)a+b+1(a+b+2)ab= \frac{(a^2+ab + a + b)(a+b) +2(b-a) - (a+1)(a+b)^2}{(a+b+2)\sqrt{\frac{ab}{ a+b+1}}} = \frac{2(b-a) \sqrt{a+b+1}}{(a+b+2) \sqrt{ab}}

Now let's see if we can deal with kurtosis:

Recall that

u(4)=6μ2σ2+3μ4−4μα(3)+α(4)σ4u(4) = \frac{ 6\mu^2\sigma^2 + 3\mu^4 - 4\mu \alpha(3)+\alpha(4)}{\sigma^4}

We also have

u(3)=α(3)−3μσ2−μ3σ3u(3) = \frac{\alpha(3) - 3\mu \sigma^2 - \mu^3}{\sigma^3}

So iteratively, we will have α(3)=u(3)σ3+μ3+3μσ2\alpha(3) = u(3) \sigma^3 + \mu^3 + 3\mu\sigma^2

So

u(4)=6μ2σ2+3μ4−4μ(u(3)σ3+μ3+3μσ2)+α(4)σ4u(4) = \frac{ 6\mu^2\sigma^2 + 3\mu^4 - 4\mu (u(3) \sigma^3 + \mu^3 + 3\mu\sigma^2)+\alpha(4)}{\sigma^4} u(4)=α(4)−4μu(3)σ3−μ4−6μ2σ2σ4u(4) = \frac{ \alpha(4)- 4\mu u(3) \sigma^3 - \mu^4 - 6\mu^2 \sigma^2}{\sigma^4}

The above derivation is for a generic distribution, now we will focus on the beta distribution, in fact, we know that α(4)=α(2)×(a+2)(a+3)(a+b+2)(a+b+3)=(μ2+σ)(a+2)(a+3)(a+b+2)(a+b+3)\alpha(4) = \alpha(2) \times \frac{(a+2)(a+3)}{(a+b+2)(a+b+3)} = (\mu^2 + \sigma)\frac{(a+2)(a+3)}{(a+b+2)(a+b+3)}, actually, it doesn't provide too much value.

It means the following:

u(4)=a(a+1)(a+2)(a+3)(a+b)(a+b+1)(a+b+2)(a+b+3)−4aa+b(ab(a+b)3(a+b+1))2(b−a)(a+b+2)−a4(a+b)4−6a2ab(a+b)2(a+b)2(a+b+1)(ab(a+b)2(a+b+1))2u(4) = \frac{\frac{a(a+1)(a+2)(a+3)}{(a+b)(a+b+1)(a+b+2)(a+b+3)} - 4\frac{a}{a+b} (\frac{ab}{(a+b)^3 (a+b+1)}) \frac{2(b-a) }{(a+b+2) }-\frac{a^4}{(a+b)^4}-6\frac{a^2 ab}{(a+b)^2(a+b)^2 (a+b+1)}}{\left(\frac{ab}{(a+b)^2(a+b+1)}\right)^2} u(4)=a(a+1)(a+2)(a+3)(a+b)3(a+b+2)(a+b+3)−8a2b(b−a)(a+b+2)−a4(a+b+1)−6a2ab(a2b2(a+b+1))u(4) = \frac{\frac{a(a+1)(a+2)(a+3)(a+b)^3}{(a+b+2)(a+b+3)} - \frac{8a^2b(b-a)}{(a+b+2)} -a^4 (a+b+1)-6a^2 ab}{\left(\frac{a^2b^2}{(a+b+1)}\right)} u(4)=a(a+1)(a+2)(a+b+3)(a+b)3−a(a+1)(a+2)b(a+b)3(a+b+2)(a+b+3)−8a2b(b−a)(a+b+2)−a4(a+b+1)−6a2ab(a2b2(a+b+1))u(4) = \frac{\frac{a(a+1)(a+2)(a+b+3)(a+b)^3 - a(a+1)(a+2)b(a+b)^3}{(a+b+2)(a+b+3)} - \frac{8a^2b(b-a)}{(a+b+2)} -a^4 (a+b+1)-6a^2 ab}{\left(\frac{a^2b^2}{(a+b+1)}\right)} u(4)=a(a+1)(a+b+2)(a+b)3−a(a+1)b(a+b)3a+b+2−a(a+1)(a+2)b(a+b)3(a+b+2)(a+b+3)−8a2b(b−a)(a+b+2)−a4(a+b+1)−6a2ab(a2b2(a+b+1))u(4) = \frac{\frac{a(a+1)(a+b+2)(a+b)^3 - a(a+1)b(a+b)^3}{a+b+2} - \frac{a(a+1)(a+2)b(a+b)^3}{(a+b+2)(a+b+3)} - \frac{8a^2b(b-a)}{(a+b+2)} -a^4 (a+b+1)-6a^2 ab}{\left(\frac{a^2b^2}{(a+b+1)}\right)} u(4)=ab(2a3+3a2b−3a2+ab2+3ab+b2)−a(a+1)b(a+b)3a+b+2−a(a+1)(a+2)b(a+b)3(a+b+2)(a+b+3)−8a2b(b−a)(a+b+2)(a2b2(a+b+1))u(4) = \frac{a b (2 a^3 + 3 a^2 b - 3 a^2 + a b^2 + 3 a b + b^2)-\frac{ a(a+1)b(a+b)^3}{a+b+2} - \frac{a(a+1)(a+2)b(a+b)^3}{(a+b+2)(a+b+3)} - \frac{8a^2b(b-a)}{(a+b+2)} }{\left(\frac{a^2b^2}{(a+b+1)}\right)} u(4)=2a3+3a2b−3a2+ab2+3ab+b2−(a+1)(a+b)3a+b+2−(a+1)(a+2)(a+b)3(a+b+2)(a+b+3)−8a(b−a)(a+b+2)(ab(a+b+1))u(4) = \frac{2 a^3 + 3 a^2 b - 3 a^2 + a b^2 + 3 a b + b^2-\frac{ (a+1)(a+b)^3}{a+b+2} - \frac{(a+1)(a+2)(a+b)^3}{(a+b+2)(a+b+3)} - \frac{8a(b-a)}{(a+b+2)} }{\left(\frac{ab}{(a+b+1)}\right)} u(4)=2a2+a4−2ab+3a2b+2a3b+2b2+3ab2+a2b2−(a+1)(a+2)(a+b)3(a+b+3)ab(a+b+2)a+b+1u(4) = \frac{2 a^2 + a^4 - 2 a b + 3 a^2 b + 2 a^3 b + 2 b^2 + 3 a b^2 + a^2 b^2 - \frac{(a+1)(a+2)(a+b)^3}{(a+b+3)} }{\frac{ab(a+b+2)}{a+b+1}}

...

It's getting too complicated, and there might be miscalculation, please if you spot any errand, let me know

Now let's see the relation between beta function and order statistics. Let's assume that we have several i.i.d random variables from a continuous distribution with nontrival elements X1,X2,...,XnX_1, X_2, ..., X_n and they are sorted in increasing order, by further assume that these random variables are distinct, (if some of them are same, we will replace them by drawing from the distribution again) we want to ask how likely is that Xk=xX_k = x.

Then, we will have Xi<xX_i < x for i<ki < k and Xi>xX_i > x for i>ki >k by our assumption. So

gk(x)=Pr⁡[Xk≤x]=∑i=kn(ni)FX(x)i[1−FX(x)]n−ig_k(x) = \Pr[X_k \le x] = \sum_{i=k}^n {n \choose i}F_X(x)^i [1-F_X(x)]^{n-i}

So

gk(x)−gk+1(x)=(nk)FX(x)k[1−FX(x)]n−kg_k(x) - g_{k+1}(x) = { n \choose k} F_X(x)^k [1-F_X(x)]^{n-k}

Note that when k=1k = 1, we will have essentially have everything but one element not exceeding xx, so g1(x)=1−(1−FX(x))ng_1(x) = 1 - (1-F_X(x))^{n}.

And by taking the derivative, we will have:

f1(x)=g1′(x)=nfX(x)(1−FX(x))n−1f_1(x) = g_1'(x) = nf_X(x)(1-F_X(x))^{n-1}

and we also take the derivative with regard to x for gk(x)−gk+1(x)=(nk)FX(x)k[1−FX(x)]n−kg_k(x) - g_{k+1}(x) = { n \choose k} F_X(x)^k [1-F_X(x)]^{n-k}, we will get

fk+1(x)=fk(x)−(nk)(kFX(x)k−1(1−FX(x))n−k−(n−k)FX(x)k(1−FX(x))n−k−1)f(x)f_{k+1}(x) = f_k(x) - {n\choose k}(kF_X(x)^{k-1} (1-F_X(x))^{n-k} -(n-k)F_X(x)^k (1-F_X(x))^{n-k-1})f(x) fk+1(x)=fk(x)−n!k!(n−k)!(kFX(x)k−1(1−FX(x))n−k−(n−k)FX(x)k(1−FX(x))n−k−1)f(x)f_{k+1}(x) = f_k(x) - \frac{n!}{k! (n-k)!}(kF_X(x)^{k-1} (1-F_X(x))^{n-k} -(n-k)F_X(x)^k (1-F_X(x))^{n-k-1})f(x) fk+1(x)=fk(x)−n!(k−1)!(n−k)!FX(x)k−1(1−FX(x))n−kf(x)+n!k!(n−k−1)!FX(x)k(1−FX(x))n−k−1f(x)f_{k+1}(x)= f_k(x) - \frac{n!}{(k-1)! (n-k)!}F_X(x)^{k-1} (1-F_X(x))^{n-k}f(x) +\frac{n!}{k! (n-k-1)!}F_X(x)^k (1-F_X(x))^{n-k-1}f(x)

We also notice that

fk(x)=fk−1(x)−n!(k−2)!(n−k+1)!FX(x)k−2(1−FX(x))n−k+1f(x)+n!(k−1)!(n−k)!FX(x)k−1(1−FX(x))n−kf(x)f_{k}(x) = f_{k-1}(x)- \frac{n!}{(k-2)! (n-k+1)!}F_X(x)^{k-2} (1-F_X(x))^{n-k+1}f(x) +\frac{n!}{(k-1)! (n-k)!}F_X(x)^{k-1} (1-F_X(x))^{n-k}f(x)

...

So fk+1(x)=f1(x)−nfX(x)(1−FX(x))n−1+n!(k−1)!(n−k)!FX(x)k−1(1−FX(x))n−kf(x)f_{k+1}(x) = f_{1}(x) - n f_X(x) (1-F_X(x))^{n-1} +\frac{n!}{(k-1)! (n-k)!}F_X(x)^{k-1} (1-F_X(x))^{n-k}f(x)

So when k=1k=1, we will have

f2(x)=nfX(x)(1−FX(x))n−1−n((1−FX(x))n−1−(n−1)FX(x)(1−FX(x))n−2)f(x)f_2(x) = nf_X(x)(1-F_X(x))^{n-1} - n((1-F_X(x))^{n-1} -(n-1)F_X(x) (1-F_X(x))^{n-2})f(x) f2(x)=n(n−1)FX(x)(1−FX(x))n−2f(x)f_2(x) = n(n-1)F_X(x) (1-F_X(x))^{n-2}f(x)

...

So

fk+1(x)=n!(k−1)!(n−k)!FX(x)k−1(1−FX(x))n−kf(x)f_{k+1}(x) = \frac{n!}{(k-1)! (n-k)!}F_X(x)^{k-1} (1-F_X(x))^{n-k}f(x)

The derivation can be found here

A more intuitive memoization is that we have kk being fixed, so fX(x)f_X(x). and we have k−1k-1 elements and n−kn-k elements being either below or above xx, so FX(x)k−1F_X(x)^{k-1} and (1−FX(x))n−k(1-F_X(x))^{n-k} and we also need to fix these k−1k-1 elements and n−kn-k elements so we will have n!(k−1)!(n−k)!\frac{n!}{(k-1)!(n-k)!}.

We can easily see the probability density function is a beta function. When we take the underlying distribution for FX(x)=xF_X(x) = x as uniform distribution over (0,1)(0,1), specifically with parameters a=ia = i and b=n−i+1b = n-i+1

Here are some computer simulations that allow us to verify the result above:

Order statistic simulation