1. Probability

1. Computing probabilities by counting If an event \(A\) can occur in \(m\) different ways out of a total \(n\) possible ways (equally likely), the probability of occurrence is \(P(A)=\frac{m}{n}\). If an event \(B\) occurred \(h\) times in \(N\) repetitions of an experiment, \(P(B)=\frac{h}{N}\).

2. Conditional probabilities Compute conditional probabilities by: (1) the formula \(P(B|A)=\frac{P(A∩B)}{P(A)}\), and (2) treating the outcomes of event \(A\) as the new sample space where event \(B\) occurs in a number of ways. For example, the probability that a single toss of a die will result in a number less than 5 (event \(B\)) given that the toss results in an even number (event \(A\)) is by formula \(P(B|A)=\frac{2/6}{1/2}=\frac{2}{3}\), because the outcomes of the events and their probabilities are \(A=\{2,4,6\},B=\{1,2,3,4\},A∩B=\{2,4\},P(A)=\frac{1}{2},P(A∩B)=\frac{2}{6}\). By method (2), since \(A\) has occurred, we treat the outcomes {2, 4, 6} of event \(A\) as the new sample space, where there are two ways that event \(B\) can occur. Thus \(P(B|A)=\frac{2}{3}\).

3. Total probability and Bayes' rule Suppose \(A_1,\cdots,A_n\) are mutually exclusive and the sample space is \(S=∪_{i=1}^nA_i\). If \(A\) is any event, then by the law of total probability \(P(A)=\sum_{i=1}^nP(A|A_i)P(A_i)\), and by the Bayes' rule \(P(A_k|A)=\frac{P(A_k∩A)}{P(A)}=\frac{P(A|A_k)P(A_k)}{\sum_{i=1}^nP(A|A_i)P(A_i)}\). For example, a box contains two fair coins (\(F\)) and a biased coin (\(B\)) with \(P(H)=0.3\). A coin is chosen at random from the box and tossed three times, if two heads and a tail are observed (event \(A\)), the probability that the chosen coin is fair is \(P(F|A)=\frac{P(A|F)P(F)}{P(A|F)P(F)+P(A|B)P(B)}=\frac{(_2^3)0.5^2(1-0.5)^1\frac{2}{3}}{(_2^3)0.5^2(1-0.5)^1\frac{2}{3}+(_2^3)0.3^2(1-0.3)^1\frac{1}{3}}≈0.80\). We usually use combinations and permutations to count the number of ways an event or thing can be accomplished.

4. Fundamental principle of counting If one thing can be accomplished in \(n_1\) different ways and after this a second thing can be accomplished in \(n_2\) different ways, ..., and the \(k\)th thing can be accomplished in \(n_k\) different ways, then all the \(k\) things can be accomplished in that order in \(n_1n_2\cdots n_k\) different ways.

5. Combinations and permutations The number of ways for arranging (or permuting) \(r\) objects from \(n\) distinct objects (for \(r≤n\)) in a line is \(P_r^n=n(n-1)\cdots(n-r+1)=\frac{n!}{(n-r)!}\). It follows that \(P_n^n=n!\) for \(r=n\). If the \(n\) objects are not distinct and there are \(n_1,n_2,\cdots,n_k\) subsets of distinct objects for \(n=n_1+\cdots+n_k\), then the number of permutations of the objects is \(P_{(n_1,\cdots,n_k)}^n=\frac{n!}{n_1!\cdots n_k!}\). For example, the number of different permutations of 10-letter word "abbcccdddd" is \(\frac{10!}{1!2!3!4!}\) = 12600. Use "gamma(n+1)" to computer "n!", so gamma(11) = 10! = 3628800. The total number of combinations of \(r\) objects chosen from a total of \(n\) different objects is \(C_r^n=(_r^n)=\frac{n!}{(n-r)!r!}= \frac{P_r^n}{r!}\). Note that \((_r^n)=\binom n{n-r}\). For example, the number of ways in which 3 cards are randomly drawn from a deck of 52 playing cards is \((_3^{52})=\frac{52!}{49!3!}\) = 22100 by gamma(53)/(6*gamma(50)).

Usage for computing factorial, permutation and combination Use the function "gamma(n+1)" for computing the factorial \(n!\). Check 50! = gamma(51), \(P_{20}^{30}\) = gamma(31)/gamma(11) ||, and \(C_{40}^{60}=(_{40}^{60})\) = gamma(61)/(gamma(21)*gamma(41)) ||.

6. Computing probabilities for continuous r.v.'s involves definite integrals, and the probability is equivalent to the area enclosed by the density and the given intervals (click here for details).

Examples
(1) A card is randomly drawn from a deck of 52 playing card. The probability that the card is 7 or hart is \(P(7∪H)=P(7)+P(H)-P(7∩H)=\frac{4}{52}+\frac{13}{52}-\frac{1}{52}=\frac{4}{13}\). If two cards are drawn from the deck one by one without replacement, the probability that both are king is \(P(A∩B)=P(A)P(B|A)=\frac{4}{52}·\frac{3}{51}=\frac{1}{221}\).
(2) A fair die is tossed twice. The probability of getting 1, 2, or 3 on the first toss (event \(A\)) and 4, 5, or 6 on the second (event \(B\)) can be calculated by \(P(A∩B)=P(A)P(B|A)=P(A)P(B)=\frac{1}{2}·\frac{1}{2}= \frac{1}{4}\).
(3) Three balls are drawn successfully without replacement from a box that contains 3 red balls, 4 white balls and 5 blue balls. The probability that they are in the order red, white and blue is \(P(R_1∩W_2∩B_3)=\frac{3}{12}·\frac{4}{11}·\frac{5}{10}=\frac{1}{22}\).
(4) Box I contains 4 red and 5 blue marbles, and Box II contains 3 red and 6 blue marbles. A fair coin is tossed. If the coin turns up heads, a marble is chosen from Box I; if it turns up tails, a marble is chosen from Box II. The probability that a blue marble is chosen is \(P(B)=P(I)P(B|I)+P(II)P(B|II)=\frac{1}{2}·\frac{5}{9}+\frac{1}{2}·\frac{6}{9}=\frac{11}{18}\).
(5) A shelf has 10 math books and 5 physics books. The probability that 4 math books will be together is \(\frac{4!12!}{15!}=\frac{4}{455}\).
(6) Suppose a room has \(n≤365\) people. The probability that the \(n\) people have different birthday (ignore leap years) is \(P_n^{365}=\frac{365}{365}·\frac{364}{365}\cdots\frac{365-n+1}{365}\). The probability that at least two of them have a common birthday (ignore leap years) can be calculated by \(P(A)=1-P(A^c)=1-\frac{365×364×\cdots×(365-n+1)}{365^n}\). If \(n=25,P(A)=0.5687\) by 1-(gamma(366)/gamma(365-25+1))/365.0**25 ||.
(7) How many people do you need to ask to have a 50:50 chance of finding someone who has the same birthday as yours? Solve for \(n\) = 253 from the equation \(P(A)=1-P(A^c)=1-\frac{364^n}{365^n}=0.5\) by trying 1-(364.0/365)**n such as 1-(364.0/365)**100 || 1-(364.0/365)**200 || 1-(364.0/365)**253.

2. Random Variables and Distribution Functions

We use random variables to represent and evaluate probability functions for various events. The probability distribution function or cumulative density function (cdf) of a random variable \(X\) is defined as \(F_X(x)=P(X≤x)\) for \(-∞< x<∞\). It follows that \(P(X>x)=1-F_X(x)≡S_X(x)\) (called survival function) and \(P(X< x)=F_X(x^-)=\lim\limits_{a→x^-}F_X(a)\). If \(F_X(x)\) is known, most information of the random experiment described by \(X\) is determined. Note that \(\displaystyle\lim_{x→∞}F_X(x)=P(X≤∞)=P(Ω)=1,\lim_{x→-∞}F_X(x)=P(X≤-∞)=P(Ø)=0\), and that the probability of the event \(\{ω:a< X(ω)≤b\text{ for }ω∈Ω\}\) is \(P(a< X≤b)=F_X(b)-F_X(a)\).

1. Discrete random variables and probability mass functions (pdf) A r.v. \(X\) is discrete if its range contains a finite or countably infinite number of points. In this case, \(F_X(x)\) changes values in jumps, or \(F_X(x)\) is a step function whose values between steps are constant. The function \(p(x)=P(X=x)\) is called the probability mass function (pdf) of \(X\), and \(\sum\limits_kp(x_k)=1\) for \(k=1,2,\cdots\). The cdf of \(X\) can be calculated by \(F_X(x)=P(X≤x)=\sum\limits_{x_k≤x}p(x)\).

Usage for finite discrete distributions Use "drv\(((x_1,\cdots,x_k),(p_1,\cdots,p_k))\)" (discrete random variable) to construct the distribution function for a finite discrete r.v. \(X\), where \((x_1,\cdots,x_k)\) are the possible values that \(X\) can take, and \((p_1,\cdots,p_k)\) are the probabilities of \(p_i=P(X=x_i)\) for \(i=1,2,\cdots,k\). The input probability vector "(p1, p2, ..., pk)" must sum to 1. For example, stt;drv((1,2,3),(0.2,0.3,0.5));cdf(x) || stt;drv((1,2,3,4),(1/4,1/4,1/4,1/4));cdf(3.5) || stt;drv((1,2,3),(1/5,2/5,2/5));P(3) || stt;drv((1,2,3),(1/5,2/5,2/5));E(x) || stt;drv((1,2,3),(1/5,2/5,2/5));V(x) || stt;drv((1,2,3),(1/5,2/5,2/5));rvs(10) || stt;drv((1,2,3),(1/5,2/5,2/5));q(0.5) ||.

Usage for infinite discrete distributions For an infinite discrete r.v., construct its distribution function by "drv(f(x))", where "f(x)" represents the the formula for the nonnegative pdf of the r.v. \(X\). Note that the infinite sum ("ism") for the input pdf "f(x)" must be 1, \(\sum_{x=1}^∞f(x)=1\). For example, if the input pdf is \(f(x)=(\frac{1}{2})^x\), it satisfies the requirement because the geometric series \(\sum_{i=1}^∞(\frac{1}{2})^x=1\). Check ism;0.5**x;x;1;oo || stt;drv(0.5^x);E(x) || stt;drv(0.5^x);P(x>4) || stt;drv(0.5^x);V(x) || stt;drv(0.5^x);cdf(3) || stt;drv(0.5^x);P(1@2) || stt;drv(0.5^x);P(x>1) || stt;drv(0.5^x);P(x>2&x<4) || stt;drv(0.5^x);P(x<2@x>5) || stt;drv(0.5^x);P(x>6|x>2) || stt;drv(0.5^x);M(2) ||. On the other hand, if \(f(x)=(\frac{3}{4})^x\), since \(\sum_{i=1}^∞(\frac{3}{4})^x=3\), normalize \(f(x)\) by \(\frac{1}{3}(\frac{3}{4})^x\), and then use the normalized one as the input pdf. Check ism;0.75**x;x;1;oo || ism;0.75**x/3;x;1;oo || stt;drv(0.75**x/3);P(x>2) || stt;drv(0.75**x/3);P(4<=x<8) ||. Make sure your input pdf "f(x)" is positive and must sum to 1.

Note that in the above examples, the expression "P(3)" stands for the probability P(X = 3) for a discrete r.v. X, "P(1@2)" for P(X = 1 or X = 2), "P(x>1)" for P(X > 1), "P(x>2&x<4)" for P(2 < X < 4), "P(4<=x<8)" for P(4 ≤ X < 8), and "P(x<2@x>5)" for P(X < 2 ∪ X > 5). The expression "rvs(10)" simulates 10 random numbers from the distribution, and "q(0.5)" gives the 50th percentile or 0.5th quantile.

2. Continuous random variables and probability density functions (pdf) The range of a continuous r.v. \(X\) contains an interval of real numbers. The cdf \(F_X(x)=P(X≤x)=\int_{-∞}^xf(x)dx\) of \(X\) is continuous, and its derivative \(F'_X(x)=f_X(x)\), which is called the probability density function (pdf) of \(X\), exists everywhere except at possibly a finite number of points. If \(X\) is continuous, \(P(X=x)=0\) for all \(x,\int_{-∞}^∞f(x)dx=1\) and \(f(x)≥0\), and \(P(a< X< b)=P(a≤X≤b)=P(a≤x< b)=P(a< x≤b)=\int_a^bf(x)dx=F_X(b)-F_X(a)\).

Usage for continuous distributions Use "crv(f(x),a,b)" (continuous random variable) to construct the distribution function for a continuous r.v. \(X\), where "f(x)" is the nonnegative pdf of \(X\), and "a" and "b" represent the lower and upper limits of the interval [a, b] of real numbers that \(X\) can take. Use "oo" for infinity "\(∞\)" and "-oo" for "-\(∞\)". Make sure your input pdf "f(x)" is nonnegative and integrated to 1 over the given interval. Check stt;crv(exp(-x),0,oo);P(x>2) || stt;crv(exp(-x),0,oo);P(x>3|x>1) || stt;crv(exp(-x),0,oo);P(x<1@x>5) || stt;crv(2*exp(-2*x),0,oo);V(x) || stt;crv(3*x^2,0,1);cdf(x) || stt;crv(2*x/3,1,2);P(x>1.5) || stt;crv(3*x^2,0,1);P(x<0.7) || stt;crv(3*x^2,0,1);E(x) || stt;crv(2*x/3,1,2);V(x) || stt;crv(2*x/3,1,2);cdf(x) || stt;crv(1,0,1);V(x) || stt;crv(1,0,1);rvs(30) || stt;crv(1,0,1);q(0.3) ||.

Verify that the pdfs in the above examples are integrated to 1 over the given intervals by itg;exp(-x);x;0;oo || itg(2*exp(-2*x),x,0,oo) || itg(3*x^2,x,0,1) || itg;2*x/3;x;1;2 || itg;1;x;0;1 ||.

3. Joint distributions If \(X,Y\) are two discrete r.v.'s, \(P(X=x,Y=y)=f(x,y)\) is the joint pdf and \(\displaystyle\sum_x\sum_yf(x,y)=1\). The marginal pdf's are defined by \(f_X(x)=P(X=x)=\sum\limits_yf(x,y),f_Y(y)=P(Y=y)=\sum\limits_xf(x,y)\). The joint distribution function or the cdf of \(X,Y\) is \(F(x,y)=P(X≤x,Y≤y)=\displaystyle\sum_{u≤x}\sum_{v≤y}f(u,v)\).

If \(X,Y\) are two continuous r.v.'s with joint pdf \(f(x,y)≥0\), then \(\int_{-∞}^∞\int_{-∞}^∞f(x,y)dxdy=1\), and the joint distribution function is \(F(x,y)=P(X≤x,Y≤y)=\int_{-∞}^y\int_{-∞}^xf(x,y)dxdy\). The probability \(P(a< X< b,c< Y< d)=\int_a^b\int_c^df(x,y)dxdy\). The marginal distributions are defined by \(F_X(x)=P(X≤x)=\int_{-∞}^∞\int_{-∞}^xf(x,y)dxdy,F_Y(y)=P(Y≤y)=\int_{-∞}^y\int_{-∞}^∞f(x,y)dxdy\). The joint pdf can be obtained by \(f(x,y)=\frac{∂^2F(x,y)}{∂x∂y}\), and the marginal densities can be obtained by \(f_X(x)=\int_{-∞}^∞f(x,y)dy,f_Y(y)=\int_{-∞}^∞f(x,y)dx\).

4. Independent random variables If \(X,Y\) are two discrete r.v.'s and \(P(X=x,Y=y)=P(X=x)P(Y=y)\) or \(f(x,y)=f_X(x)f_Y(y)\), then \(X,Y\) are independent. If \(X,Y\) are continuous r.v.'s and \(P(X≤x,Y≤y)=P(X≤x)P(Y≤y)\), or \(F(x,y)=F_X(x)F_Y(y)\) or \(f(x,y)=f_X(x)f_Y(y)\), then \(X,Y\) are independent. The converse is also true for both discrete and continuous cases.

5. Conditional distributions If \(X,Y\) are discrete r.v.'s, \(f(y|x)=P(Y=y|X=x)=\frac{f(x,y)}{f_X(x)}\) is called the conditional pdf of \(Y\) given \(X\), and \(f(x|y)=\frac{f(x,y)}{f_Y(y)}\). Similarly, define conditional pdf as \(f(x|y)=\frac{f(x,y)}{f_Y(y)}\) and \(f(y|x)=\frac{f(x,y)}{f_X(x)}\) for continuous r.v.'s \(X,Y\).

Example 1 Suppose the joint density of \(X,Y\) is \(f(x,y)=Cx^2y\) for \(0≤X≤4,1≤Y≤5\). The constant \(C\) can be determined by \(\int_1^5\int_0^4Cx^2ydxdy=256C=1⇒C=\frac{1}{256}\) by itg;c*x^2*y;x;0;4;y;1;5 ||.
(1) Find the probability \(P(0≤X≤1,2≤Y≤3)=\int_2^3\int_0^1\frac{x^2y}{256}dxdy=\frac{5}{1536}\) by itg;x^2*y/256;x;0;1;y;2;3 ||, and the probability \(P(X>2,Y≤3)=\frac{7}{24}\) by itg;x^2*y/256;x;2;4;y;1;3 ||.
(2) The marginal density of \(X\) is \(f_X(x)=\int_1^5\int_0^4\frac{x^2y}{256}dy=\frac{3x^2}{64}\) by itg;x^2*y/256;y;1;5 for \(0≤X≤4\) and 0 otherwise. The marginal density of \(Y\) is \(f_Y(y)=\int_1^5\int_0^4\frac{x^2y}{256}dx=\frac{y}{12}\) by itg;x^2*y/256;x;0;4 for \(1≤Y≤5\) and 0 otherwise.
(3) The marginal cdf of \(X\) is \(F_X(x)=P(X≤x)=\int_0^x\frac{3t^2}{64}dt=\frac{x^3}{64}\) or \(F_X(x)=\int_0^x\int_1^5\frac{v^2u}{256}dudv\) by itg;v^2*u/256;u;1;5;v;0;x for \(0≤X≤4;F_X(x)=0\) for \(X<0;F_X(x)=1\) for \(X>4\). Similarly, the cdf of \(Y\) is \(F_Y(y)=\int_2^y\frac{t}{12}dt=\frac{y^2-1}{24}=\int_0^4\int_1^y\frac{v^2u}{256}dudv\) by itg;v^2*u/256;u;1;y;v;0;4 for \(1≤Y≤5;F_Y(y)=0\) for \(Y<1;F_Y(y)=1\) for \(Y>5\).
(4) Since \(f(x,y)=f_X(x)f_Y(y)=\frac{3x^2}{64}\frac{y}{12}=\frac{x^2y}{256},X\) and \(Y\) are independent, and the cdf of \(X,Y\) is \(F(x,y)=F_X(x)F_Y(y)\). Thus, \(F(x,y)=0\) for \(x< 0\) or \(y<1;F(x,y)=\frac{x^3}{64}\) for \(0≤x≤4,y>5;F(x,y)=\frac{y^2-1}{24}\) for \(x>4,1≤y≤5;F(x,y)=\frac{x^2}{64}\frac{y^2-1}{24}\) for \(0≤x≤4\) and \(1≤Y≤5;F(x,y)=1\) for \(x>4\) and \(y>5\).
(5) The probability \(P(X+Y≤2)=\int_A\frac{x^2y}{256}dxdy\), where the region \(A\) is a triangle enclosed by \(y=2-x\) and \(0≤x≤1,1≤y≤2-x\). Thus, \(P(X+Y≤2)=\int_0^1\int_1^{2-x}\frac{x^2y}{256}dydx=\frac{1}{2560}\) by itg;x^2*y/256;y;1;2-x;x;0;1 ||. Similarly, \(P(X+Y≤5)=\frac{3}{20}\) by itg;x^2*y/256;y;1;5-x;x;0;4

Example 2 Suppose \(X,Y\) have a joint density \(f(x,y)=\frac{3x^2+2y}{126}\) for \(0≤X≤3,1≤Y≤4\).
(1) The marginal distributions functions \(F_X(x)=\int_{u=0}^x\int_{v=1}^4\frac{3u^2+2v}{126}dvdu=\frac{3x^3+15x-18}{126}\) for \(0≤X≤3\), and \(F_X(x)=0\) for \(x<0\) and \(F_X(x)=1\) for \(X>3\). Similarly, \(F_Y(y)=\int_{v=1}^y\int_{u=0}^3\frac{3u^2+2v}{126}dvdu= \frac{3y^2+27y-30}{126}\) for \(1≤Y≤4\), and \(F_Y(y)=0\) for \(y<1\) and \(F_Y(y)=1\) for \(Y>4\). Check itg;(3*u^2+2*v)/126;v;1;4;u;1;x || itg;(3*u^2+2*v)/126;u;0;3;v;1;y || itg;(3*x^2+2*y)/126;x;0;3;y;1;4 ||.
(2) The marginal densities \(f_X(x)=F'_X(x)=\frac{9x^2+15}{126},0≤X≤3;f_Y(y)=F_Y(y)=\frac{6y+27}{126},1≤Y≤4\). Since \(f(x,y)≠f_X(x)f_Y(y),X,Y\) are not independent.
(3) \(P(0< X<1,Y>2)=\int_0^1\int_2^4\frac{3x^2+2y}{126}dydx=\frac{1}{9};P(X>1)=\int_1^3\int_1^4\frac{3x^2+2y}{126}dydx=\frac{6}{7};P(X+Y>2)=1-P(X+Y≤2)\)
\(=1-\int_{0}^1\int_{y=1}^{2-x}\frac{3x^2+2y}{126}dydx=\frac{1493}{1512}\). Check itg;(3*x^2+2*y)/126;y;2;4;x;0;1 || itg;(3*x^2+2*y)/126;y;1;4;x;1;3 || 1-itg((3*x^2+2*y)/126,y,1,2-x,x,0,1) ||.
(4) The joint distribution function is \(F(x,y)=0\) for \(X<0\) or \(Y<1;F(x,y)=1\) for \(X>3\) and \(Y>4;F(x,y)=\int_0^x\int_1^y\frac{3u^2+2v}{126}dvdu\)
\(=\frac{2x^3(y-1)+xy^2+4xy-5x}{252}\) for \(0≤X≤3,1≤Y≤4;F(x,y)=F_X(x)\) for \(0≤X≤3,y>4;F(x,y)=F_Y(y)\) for \(x>3,1≤Y≤4\). Check itg;(3*u^2+2+v)/126;v;1;y;u;0;x ||

Example 3 If \(X,Y\) have a joint density \(f(x,y)=1,0≤X≤1,0≤Y≤1\), then \(P(|X-Y|≤0.25)=P(-0.25≤X-Y≤0.25)\)
\(=1-\int_{0.25}^1\int_{y-.25}^.75dxdy-\int_{.25}^1\int_0^{x-.25}dydx=\frac{7}{16}\) by 1-itg(1,x,y-1/4,3/4,y,1/4,1)-itg(1,y,0,x-1/4,x,1/4,1) ||.

Example 4 If \(X,Y\) have a joint density \(f(x,y)=\frac{3+2xy}{48},0≤x≤2,1≤y≤4\).
(1) \(f(x|y)=\frac{f(x,y)}{f_Y(y)}=\frac{3+2xy}{4(y+72)}\) by (3+2*x*y)/(48*itg(3+2*x*y/48,x,0,2)) ||.
(2) \(f(y|x)=\frac{f(x,y)}{f_X(x)}=\frac{3+2xy}{3(5x+144)}\) by (3+2*x*y)/(48*itg(3+2*x*y/48,y,1,4)) ||.
(3) \(P(Y>2|1< X< 1+dx)=\int_2^4\frac{3+2y}{3(149)}=\frac{6}{149}\) by itg;(3+2*y)/(3*149);y;2;4 ||.

Example 5 Suppose \(X,Y\) have a joint density \(f(x,y)=\frac{2x^2y^{-2}}{27}\) for \(1≤X≤4,1≤Y≤x\). Then \(f_X(x)=\int_1^x\frac{2x^2y^{-2}}{27}dy=\frac{2x(x-1)}{27}\),
\(f_Y(y)=\int_x^4\frac{2x^2y^{-2}}{27}dx=\frac{2(64-y^3)}{81y^2}\). Check itg;2*x^2/y^2/27;y;1;x || itg;2*x^2/y^2/27;x;y;4 || and the integrals of the densities itg;2*x^2/y^2/27;y;1;x;x;1;4 || itg;2*x*(x-1)/27;x;1;4 || itg;2*(64-y^3)/(81*y^2);y;1;4 || all return 1. Since \(f(x,y)≠f_X(x)f_Y(y),X,Y\) are not independent. The condition distributions \(f_{Y|X}(y|x)=\frac{x}{(x-1)y^2}, f_{X|Y}(x|y)=\frac{3x^2}{64-y^3}\). Check that the integrals of the densities itg;x/((x-1)*y^2);y;1;x || itg;(3*x^2)/(64-y^3);x;y;4 ||.

3. Expectation, Variance, Covariance, Moment, and Moment Generating Function (MGF)

Definitions The expectation of a r.v. \(X\) is defined as \(E(X)=μ=\sum_xxP(X=x)\) (discrete r.v.) and \(E(X)=μ=\int_{-∞}^∞xf_X(x)dx\) (continuous r.v.). The variance of \(X\) is defined as \(\text{Var}(X)=σ^2=E(X-μ)^2\) and the standard deviation is \(σ=\sqrt{\text{Var}(X)}\). The expectation of the function \(g(X)\) of \(X\) is \(E[g(X)]=\sum_xg(x)P(X=x)\) (discrete r.v.) and \(E[g(X)]=\int_{-∞}^∞g(x)f_X(x)dx\) (continuous r.v.). The \(r\)th moment of \(X\) about the origin is defined as \(E(X^r)=\sum_xx^rP(X=x)\) (discrete r.v.) and \(E(X^r)=\int_{-∞}^∞x^rf_X(x)dx\) (continuous r.v.). The moment generating function is defined as \(M(t)=E(e^{tX})=\sum_xe^{tx}P(X=x)\) (discrete r.v.) and \(M(t)=E(e^{tX})=\int_{-∞}^∞e^{tx}f_X(x)dx\). The characteristic function is defined as \(E(e^{iωX})=\sum_xe^{iωx}P(X=x)\) (discrete r.v.) and \(E(e^{iωX})=\int_{-∞}^∞e^{iωx}f_X(x)dx\) (continuous r.v.).

The covariance of two r.v.'s \(X,Y\) is defined as \(σ_{XY}=E[(X-μ_X)(Y-μ_Y)]=\sum_x\sum_y(x-μ_x)(y-μ_y)f(x,y)\) (discrete r.v.) and \(σ_{XY}=E[(X-μ_X)(Y-μ_Y)]=\int_{-∞}^∞(x-μ_x)(y-μ_y)f(x,y)dxdy\) (continuous r.v.). The correlation of \(X,Y\) is \(ρ=\frac{σ_{XY}}{σ_Xσ_Y}\).

Usages for mean, variance, standard deviation, moment, and MGF Use "E(x)", "V(x)", and "std(x)" to calculate the expectation, variance and standard deviation of a user-defined r.v. "x", and use "E(g(x))", "V(g(x))", and "std(g(x))" to computer the corresponding values of "g(x)". For common distributions, simply use "m, v, std" to compute the mean, variance and standard deviation. Compute the moment generating function by "E(exp(t*x))" and the characteristic function by "E(exp(I*w*x))", where "I" represents the imaginary unit \(i\) such that \(i^2=-1\). Use "M(r)" or "E(X^r)" to compute the r-th moment \(E(X^r)\), where "r" is a positive integer.

Example 1 The density of standard normal distribution is \(f(x)=\frac{1}{\sqrt{2π}}e^{-\frac{x^2}{2}}\), and \(E(x)=\int_{-∞}^∞xf(x)dx=0\) because \(xf(x)\) is odd. Using integration by parts, \(E(x^2)=\frac{1}{\sqrt{2π}}\int_{-∞}^∞x^2e^{-\frac{x^2}{2}}dx =-\frac{1}{\sqrt{2π}}\int_{-∞}^∞xde^{-\frac{x^2}{2}} =-\frac{1}{\sqrt{2π}}xe^{-\frac{x^2}{2}}|_{-∞}^∞+\frac{1}{\sqrt{2π}} \int_{-∞}^∞e^{-\frac{x^2}{2}}dx=1\). Check itg;e^(-x^2/2)/(2*pi)^(1/2);x;-oo;oo || itg;x*e^(-x^2/2)/(2*pi)^(1/2);x;-oo;oo || itg;x^2*e^(-x^2/2)/(2*pi)^(1/2);x;-oo;oo || stt;n(0,1);E(x) || stt;n(0,1);E(X^2) || stt;n(0,1);V(x) || stt;n(0,1);std(x) || stt;n(0,1);M(3) || stt;n(0,1);M(6) || stt;n(0,1);E(exp(t*x));x || stt;n(0,1);E(exp(I*w*x));x || stt;n(0,1);V(a*x);x || stt;n(0,1);V(x+a);x || stt;n(0,1);E(2*x+3) || stt;n(0,1);E(a*x+b);x || stt;n(0,1);E(x^3) || stt;n(0,1);E(x^6) ||.

Note that if the expressions "E(.)", "V(.)", "std(.)" involve more than one variables, add a letter after the expression to indicate which is the random variable. For example, the expression "E(a*x+b);x" indicates "x" is the random variable and "a" and "b" are treated as constants.

Example 2 Suppose the density function of a r.v. \(X\) is \(f(x)=3e^{-3x}\) for \(x≥0\). Then \(E(X)=\frac{1}{3};E(X^2)=\frac{2}{9};E(X^3)=\frac{2}{9};E(X^4)=\frac{8}{27}\);
\(E(e^{tx})=\frac{3}{3-t}=\frac{1}{1-t/3}=1+\frac{t}{3}+\frac{t^2}{3^2}+\frac{t^3}{3^3}+\frac{t^4}{3^4}+\cdots\). Compute them directly by definite integrals: itg;x*3*exp(-3*x);x;0;oo || itg;x^2*3*exp(-3*x);x;0;oo || itg;x^3*3*exp(-3*x);x;0;oo || itg;x^4*3*exp(-3*x);x;0;oo || itg;exp(t*x)*3*exp(-3*x);x;0;oo ||. Note that \(X\) has a exponential distribution with parameter λ = 3, which can be entered as "epn(3)". Check stt;epn(3);M(1) || stt;epn(3);M(2) || stt;epn(3);M(3) || stt;epn(3);M(4) ||. You can use the function "epn()" to calculate more results: stt;epn(0.5);P(x>1) || stt;epn(0.5);q(0.6) || stt;epn(2);V(x) || stt;epn(10);s || stt;epn(1);k || stt;epn(4);cdf(x) || stt;epn(0.6);svf(2) || stt;epn(1);E(exp(I*w*x));x ||.

Example 3 Suppose \(X,Y\) have a joint density \(f(x,y)=\frac{xy}{648}\) for \(0≤x≤6,3≤y≤9\). Then \(E(X)=4;E(Y)=\frac{13}{2};E(XY)=26\);
\(E(X^2)=18;E(Y^2)=45;\text{Var}(X)=E(X-4)^2=2;\text{Var}(Y)=E(Y-\frac{13}{2})^2=\frac{11}{4};\text{Cov}(X,Y)=E[(X-4)(Y-\frac{13}{2})]=0;\)
\(E(3X+2Y)=3E(X)+2E(Y)=25;\text{Var}(X+Y)=2+\frac{11}{4}=\frac{15}{4};E(X|Y)=E(X)=4;E(Y|X)=E(Y)=\frac{13}{2}\). Check itg;x^2*y/648;x;0;6;y;3;9 || itg;x*y^2/648;x;0;6;y;3;9 || itg;x^2*y^2/648;x;0;6;y;3;9 || itg;x^3*y/648;x;0;6;y;3;9 || itg;x*y^3/648;x;0;6;y;3;9 || itg;(x-4)^2*x*y/648;x;0;6;y;3;9 || itg;(y-13/2)^2*x*y/648;x;0;6;y;3;9 || itg;(x-4)*(y-13/2)*x*y/648;x;0;6;y;3;9 || itg;(3*x+2*y)*x*y/648;x;0;6;y;3;9 || itg(x*x*y/(648*itg(x*y/648,x,0,6)),x,0,6) || itg(y*x*y/(648*itg(x*y/648,y,3,9)),y,3,9) ||.

Example 4 Suppose \(X,Y\) have a joint density \(f(x,y)=\frac{x^2y+xy^2}{81}\) for \(0≤x≤3,0≤y≤3\). Then \(E(X)=E(Y)=\frac{17}{8};\text{Cov}(X,Y)=\frac{-1}{64};\)
\(\text{Cov}(X)=\text{Cov}(Y)=\frac{139}{320};E(XY)=\frac{9}{2};\text{Var}(X+Y)=2(\frac{139}{320}+2(\frac{-1}{64})=\frac{67}{80};E(X|y)=\frac{4y+9}{4y+4};E(Y|x)=\frac{4x+9}{4x+4}\). Check itg;x*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg;y*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg;(x-17/8)*(y-17/8)*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg;(x-17/8)^2*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg;(y-17/8)^2*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg;x*y*(x^2*y+x*y^2)/81;x;0;3;y;0;3 || itg(x*(x^2*y+x*y^2)/(81*itg((x^2*y+x*y^2)/81,x,0,3)),x,0,3) || itg(y*(x^2*y+x*y^2)/(81*itg((x^2*y+x*y^2)/81,y,0,3)),y,0,3) ||. Find \(\text{Var}(X|y)\) by itg((x-(4*y+9)/(4*y+4))^2*(x^2*y+x*y^2)/(81*itg((x^2*y+x*y^2)/81,x,0,3)),x,0,3) || and \(\text{Var}(Y|X)\) by itg((y-(4*x+9)/(4*x+4))^2*(x^2*y+x*y^2)/(81*itg((x^2*y+x*y^2)/81,y,0,3)),y,0,3) ||.

4. Mode, Median, Quantile, Skewness, Kurtosis, and Entropy

Mean or expectation provides a measure of central tendency for the values of a distribution. Mode and median are two other measures of central tendency. The mode of a discrete r.v. has the greatest probability of occurring. If a distribution has two, three or more such mode values, we say the distribution is bimodal, trimodal and multi-modal, respectively. The mode of a continuous r.v. is the value at which its pdf has a relative maximum.

The median is the value \(x\) such that \(P(X< x)≤\frac{1}{2}\) and \(P(X>x)≤\frac{1}{2}\). If \(X\) is continuous, \(P(X>x)=P(X< x)=\frac{1}{2}\). The quantile corresponding to a probability \(α\) is the value \(x_α\) such that the area to the left of \(x_α\) is \(α\), that is, \(P(X≤x_α)=α\). We also call the \(α\)-th quantile the 100\(α\)-th percentile. Thus, the 25th percentile is \(x_{0.25}\), the 50th percentile or median is \(x_{0.5}\), and the 75th percentile is \(x_{0.75}\). The difference \(x_{0.75}-x_{0.25}\) is called the inter-quantile range (IQR).

In information theory, the entropy of a discrete r.v. \(X\) is the average level of information or uncertainty inherent to the the possible outcomes of \(X\), and it can be calculated by \(H(X)=E(-\log p(x))=-\sum_xp(x)\log p(x)\).

Skewness and kurtosis Skewness measures the degree of asymmetry (or departure from symmetry) of a distribution, and it is defined as \(α_3=\frac{E(x-μ)^3}{σ^3}\). If \(α_3>0\), we say the distribution is skewed to the right; if \(α_3<0\), it is skewed to the right. For a symmetric distribution (e.g., normal), \(α_3=0\). Kurtosis, which is defined as \(α_4=\frac{E(x-μ)^4}{σ^4}\), measures the degree of peakedness of a distribution. Large kurtosis means the distribution may have its values concentrated around its mean (so that the distribution appears a large peak near its mean). Kurtosis is usually taken relative to the kurtosis of the standard normal distribution, which is 3.

Usages for median, skewness, kurtosis, and entropy For common distribution functions such as normal, exponential, uniform, and Poisson distributions, the letters "m, v, s, k, std, med, h" represent mean, variance, skewness, kurtosis, standard deviation, median, and entropy respectively. For user-defined distributions, use "E(x), V(x), s(x), k(x), std(x), med(x), H(x)". Use "q(α)" to compute the α-th quantile or 100α-th percentile. For example, stt;n(0,1);k || stt;n(1,1);s || stt;epn(0.6);k || stt;poi(7);s || stt;uni(1,3);k || stt;n(0,1);q(0) || stt;n(0,1);q(1) || stt;n(0,1);q(0.05) || stt;n(0,1);q(0.975) || stt;crv(2*x,0,1);s(x) || stt;crv(3*x^2,0,1);k(x) || stt;poi(5);q(0.5) || stt;ber(0.5);h ||.

Example 1 Suppose \(X\) has a density \(f(x)=\frac{x(16-x^2)}{64},0≤X≤4\). The median \(X_{0.5}\) can be obtained by \(P(X>X_{0.5})=P(X< X_{0.5})=0.5\)
\(⇒\int_0^m\frac{x(16-x^2)}{64}dx=0.5⇒m^4-32m^2+128=0⇒X_{0.5}=m=2\sqrt{4-2\sqrt{2}}≈2.16\). Check slv(itg(x*(16-x^2)/64,x,0,m)-0.5,m) || slv(itg(x*(16-x^2)/64,x,0,m)-0.5,m) || stt;crv(x*(16-x^2)/64.0,0,4.0);med(x) ||. The mode is the value of \(X\) where the density \(f(x)\) has a relative maximum (continuous case). It can be found by setting \(f'(x)=\frac{16-3x^2}{64}=0⇒x=\frac{4\sqrt{3}}{3}≈2.31\). Check slv(dif(x*(16-x^2)/64,x),x) ||. The skewness is about -0.125 by stt;crv(x*(16-x^2)/64,0,4);s(x), which means the distribution is slightly skewed to the left. The kurtosis is about 2.18 by stt;crv(x*(16-x^2)/64,0,4.0);k(x), which means the distribution is less peaked than normal distribution that has kurtosis 3. The 75th percentile is about 2.83, the 25th percentile is about 1.46, and the inter-quantile range is 1.37. Check stt;crv(x*(16-x^2)/64,0,4.0);q(0.75) || stt;crv(x*(16-x^2)/64,0,4.0);q(0.25) ||. The 10th percentile is about 0.91 by stt;crv(x*(16-x^2)/64,0,4.0);q(0.1) ||. The mean absolute deviation is \(E(|X-\frac{32}{15}|)=0.74\) by stt;crv(x*(16-x^2)/64,0,4);E(abs(x-32/15)) ||, where \(E(X)=\frac{32}{15}\) by stt;crv(x*(16-x^2)/64,0,4);E(abs(x)) ||.

Example 2 Suppose \(X\) is a discrete random variable with pdf \(P(X=x)=(\frac{1}{2})^x,x=1,2,\cdots\).The mode is the value of \(X\) with the greatest probability of occurring. Thus, the mode is \(P(X=1)=\frac{1}{2}\). Since \(P(X>1)=P(X<2)=\frac{1}{2}\), the median is any value between 1 and 2, say 1.5. The mean \(μ=\sum_{x=1}^∞x(\frac{1}{2})^x⇒μ-\frac{1}{2}μ=r+r^2+r^2+\cdots⇒μ=2\), where \(r=\frac{1}{2}\). Since this is the geometric distribution with p = 0.5, check stt;geo(1/2);m || stt;geo(0.5);med || stt;geo(0.5);s || stt;geo(0.5);P(x>1) || stt;geo(0.5);P(x<2) ||.

5. Common Probability Distributions

I. Distributions for Discrete Random Variables

1. Bernoulli A r.v. X ~ Bernoulli(p) with "success" probability of an experiment P(X = 1) = p and "failure" probability P(X = 0) = 1 - p. Use "ber(p)" for a Bernoulli r.v. for 0 < p < 1. Check stt;ber(p);cdf(x) || stt;ber(p);pdf(x) || stt;ber(0.5);h ||.

2. Binomial The binomial distribution is \(P(X=x)=(_x^n)p^x(1-p)^{n-x}\) for \(x=0,1,2,\cdots,n\). It corresponds to successive terms in the binomial expansion \((p+q)^n=p^n+(_1^n)p^{n-1}q+(_2^n)p^{n-2}q^2+\cdots+q^n=\sum_{x=0}^n(_x^n)p^xq^{n-x}\) for \(q=1-p\). If \(n=1\), the distribution is Bernoulli distribution. Use "bin(n,p)" for a binomial distribution with "n" trials and the probability of success "p" (0 < p < 1) for each trial. For example, stt;bin(20,0.5);pdf(x) || stt;bin(20,0.5);m || stt;bin(20,0.5);v || stt;bin(20,0.5);std || stt;bin(20,0.5);s || stt;bin(20,0.5);k || stt;bin(15,0.6);P(x>7) || stt;bin(15,0.6);h ||.

Examples
(1) The probability distribution of boys in families with 4 children (assuming equal probabilities for boys and girls) is \(P(X=x)=(_x^4)0.5^x(1-0.5)^{4-x}=(_x^4)0.5^4\). Check stt;bin(4,0.5);pdf(x) || stt;bin(4,0.5);P(1) || stt;bin(4,0.5);P(3).
(2) For 100 tosses of a fair coin, the expectation is np = 50,variance is np(1 - p) = 25, and the probability P(X ≤ 45) ≈ 0.184. Check stt;bin(100,0.5);m || stt;bin(100,0.5);v || stt;bin(100,0.5);cdf(45) ||. The probability can be approximated by the central limit theorem by stt;n(50,5);P(x<=45) ||.

3. Geometric For a sequence of Bernoulli\((p)\) trials, the probability of observing the first "success" is \(P(X=x)=p(1-p)^{x-1}\) for \(X=1,2,\cdots\), where \(X\) is a geometric r.v.. Use "geo(p)" for a geometric distribution with a parameter 0 < p < 1. Check the pdf, cdf, mean, and variance by stt;geo(p);pdf(x) || stt;geo(p);cdf(x) || stt;geo(p);E(x) || stt;geo(p);V(x) ||.

If \(X\) is a geometric r.v., it has a memoryless property \(P(X>i+j|X>i)=P(X>j)\). The geometric distribution is the only discrete distribution that possess this property. For example, if \(X\sim\) geometric\((0.5),P(X>2+3|X>2)=P(X>3)\) = 0.125. Check stt;geo(0.5);P(x>4|x>1) || stt;geo(0.5);P(x>5|x>2) || stt;geo(0.5);P(x>7|x>4) || stt;geo(0.5);P(x>3) || stt;geo(0.5);svf(3) ||.

4. Negative binomial For a sequence of independent Bernoulli\((p)\) trials, the probability of observing the \(k\)th "success" is \(P(X=x)=\binom {x-1}{k-1}p^k(1-p)^{x-k}\) for \(X=k,k+1,\cdots\), where \(X\) is a negative binomial r.v. with parameter \(k,p\). Use "nbn(k,p)" for a negative binomial distribution with parameters k ≥ 1 and 0 < p < 1. Check the pdf, mean, and variance by stt;nbn(p,k);pdf(x) || stt;nbn(k,p);E(x) || stt;nbn(k,p);V(x) || stt;nbn(5,0.5);P(6) || stt;nbn(5,0.5);P(7) ||.

5. Poisson The pmf of a Poisson r.v. is \(P(x)=\frac{e^{-λ}λ^x}{x!}\) for \(x≥0,λ>0\). Use "poi(λ)" for a Poisson distribution with parameter λ. Check stt;poi(r);pdf(x) || stt;poi(r);cdf(x) || stt;poi(r);E(x) || stt;poi(r);V(x) || stt;poi(10);P(7) || stt;poi(12);P(x>7|x>5) || stt;poi(21);P(x>5&x<13) || stt;poi(34);P(x>15@x<8) || stt;poi(6);h ||.

Poisson distribution can be used to approximate Binomial distribution Bin\((n,p)\) when \(n\) is large and \(p\) is small. Suppose \(λ\) is an expected number of occurrences within a period of time. Divide the time interval into \(n\) equal subintervals, with each a number of \(\frac{λ}{n}\) occurrences. Assume the probability of one occurrence in a subinterval is \(\frac{λ}{n}\), which corresponds to a Bernoulli random variable \(X_i\sim\) Ber\((\frac{λ}{n})\). Then Bin\((n,p)≈\) Poisson\((np)\). To see this, \(Y_n=\displaystyle\sum_{i=1}^nX_i\sim\text{ Bin}(n,\frac{λ}{n}),\lim_{n\to ∞}P(Y_n=k)=\lim_{n\to ∞}\frac{n!}{k!(n-k)!}(\frac{λ}{n})^k(1-\frac{λ}{n})^{n-k}=\lim_{n\to ∞}\frac{λ^k}{k!}\frac{n!}{(n-k)!(n-λ)^k}(1-\frac{λ}{n})^n=e^{-λ}\frac{λ^k}{k!}\). Check stt;bin(20,0.1);P(3) || stt;poi(2);P(3) ||.

6. Hypergeometric One important application of the hypergeometric distribution is sampling without replacement. Suppose there are \(N\) objects (population size), \(K\) of which are of interest. Randomly draw a sample of \(n\) objects from \(N\), the number \(k\) objects of interest in the sample has a hypergeometric distribution with a pmf \(P(X=k)=\frac{(_k^K)(_{n-k}^{N-K})}{(_n^N)}\) for \(\max(0,n+K-N)≤k≤\min(K,n)\). Use "hyp(N, K, n)" for a hypergeometric distribution with parameters "N, K, n", where K ≤ N and n ≤ N. Check stt;hyp(20,7,10);m || stt;hyp(20,7,10);v || stt;hyp(20,7,10);pdf(x) || stt;hyp(20,7,10);cdf(x) || stt;hyp(20,7,10);P(2) || stt;hyp(20,7,10);P(x>=2&x<5) || stt;hyp(20,7,10);P(x<=5|x>2) ||.

7. Discrete uniform A r.v. \(X\) is a discrete uniform r.v. if its pdf is \(P(X=x)=\frac{1}{n}\) for an integer \(x\) satisfying \(1≤x≤n\). Note that the r.v. \(X\) takes an integer between 1 and \(n\) with equal probability. Use "die(n)" for a discrete uniform distribution with parameter "n" a positive integer. Check stt;die(5);m || stt;die(5);v || stt;die(6);P(x>3) || stt;die(6);v || stt;die(10);P(x<5) || stt;die(20);std || stt;die(8);h || stt;die(15);q(0.5) || stt;die(9);cdf(x) ||. For small "n", you can use "drv" to get the same results. Check stt;drv((1,2,3,4,5),(1/5,1/5,1/5,1/5,1/5));pdf(x) || stt;drv((0,1,2,3,4),(1/5,1/5,1/5,1/5,1/5));cdf(x) || stt;drv((2,3,4,5,6),(1/5,1/5,1/5,1/5,1/5));E(x) || stt;drv((1,3,5,7,9),(1/5,1/5,1/5,1/5,1/5));V(x) ||.

8. Multinomial The multinomial distribution is the generalization of the binomial distribution. The pmf of the \(k\) r.v.'s \(X_1,\cdots,X_k\) in a total of \(n\) trials is \(P(X_1=n_1,X_2=n_2,\cdots,X_k=n_k)=\frac{n!}{n_1!n_2!\cdots n_k!}p_1^{n_1}p_2^{n_2}\cdots p_k^{n_k}\), where \(n=n_1+\cdots+n_k\) and \(\sum_{i=1}^kp_i=1\). Note that \(n_1,\cdots,n_k\) are the counts of \(k\) mutually exclusive events (categories) represented by \(X_1,\cdots,X_k\) in the \(n\) trials. Use "mni(n, p1, p2, ..., pk)" to create a multinomial distribution with parameter "n" total number of trials and the vector of "k" probabilities "p1, p2, ..., pk" that must sum to 1. For example, stt;mni(10,0.2,0.3,0.5);cov || stt;mni(15,0.2,0.3,0.5);h || stt;mni(12,1/3,1/3,1/3);pdf(3,4,5) || stt;mni(10,0.2,0.3,0.5);rvs(20) ||. Note that only keywords "cov", "h", "rvs(n)" and "pdf(n1, n2, ..., nk)" are available for multinomial distributions. To obtain the pmf for a set of counts by "pdf(n1, n2, ..., nk)", the sum of the counts "n1, n2, ..., nk" for the k categories must be equal to n, the total trials.

II. Distributions for Continuous Random Variables

1. Normal (Gaussian) Use "n(μ, σ)" for a normal distribution with mean μ and standard deviation σ. Find the pdf, mean, variance, and standard deviation of a normal distribution N(μ, σ\(^2\)) by stt;n(u,s);pdf(x) || stt;n(u,s);E(x) || stt;n(u,s);V(x) || stt;n(u,s);std(x) ||. If X ~ N(0, 1), P(-1 < X < 1) ≈ 0.68; P(-2 < X < 2) ≈ 0.95; P(-3 < X < 3) ≈ 0.99; P(X < 1.96) = 0.975; P(X > 1.96) = 0.025. Check stt;n(0,1);P(x>-1&x<1) || stt;n(0,1);P(x>-2&x<2) || stt;n(0,1);P(x>-3&x<3) || stt;n(0,1);cdf(1.96) || stt;n(0,1);svf(1.96) ||. The moment generating function \(M(t)=e^{-\frac{t^2}{2}}\) by stt;n(0,1);E(exp(t*x));x ||, and the characteristic function \(ϕ(w)=e^{-\frac{w^2}{2}}\) by stt;n(0,1);E(exp(I*w*x));x ||. Check other examples stt;n(1,2);E(exp(t*x));x || stt;n(1,2);P(x>-2&x<1) || stt;n(2,3);E(exp(I*w*x));x || stt;n(2,3);s || stt;n(2,3);k || stt;n(3,4);q(0.2) ||.

2. Exponential The pdf of exponential distribution is \(f_X(x)=λe^{-λx}\) for \(x≥0,λ>0\). Use "epn(λ)" for an exponential distribution with parameter "λ". Check stt;epn(t);pdf(t) || stt;epn(t);cdf(t) || stt;epn(t);E(t) || stt;epn(t);V(t) || stt;epn(t);E(exp(a*t)) || stt;epn(1);P(x>1) || stt;epn(2);P(x>3|x>1) ||.

Exponential distribution has memoryless property \(P(X>s+t|x>s)=P(X>t)\) for \(s>0,t≥0\). For example, a r.v. X ~ exponential(0.8), P(X > 3| X > 1) = P(X > 2) = 0.2019 by stt;epn(0.8);P(x>3|x>1) || stt;epn(0.8);P(x>3|x>1) ||. Similarly check the results stt;epn(0.8);P(x>4|x>2) || stt;epn(0.8);P(x>2.5|x>0.5) || stt;epn(0.8);P(x>5|x>3) ||.

3. Uniform The density of a uniform r.v. \(X\) on an interval \([a,b]\) is \(f_X(x)=\frac{1}{b-a}\). Use "uni(a, b)" for a uniform distribution with lower bound "a" and upper bound "b", or a < b. Check stt;uni(0,1);pdf(x) || stt;uni(0,1);cdf(x) || stt;uni(0,1);v || stt;uni(0,1);E(exp(t*x));x || stt;uni(2,5);cdf(x) || stt;uni(2,4);P(x>3) ||.

4. Student-t Use "t(n)" for a Student-t distribution with degree of freedom "n" a positive integer. Display the pdf of the t-distribution \(t_n\) with degree of freedom \(n\) by "stt;t(n);pdf(x)". For example, "stt;t(20);pdf(x)" returns the pdf of \(t_{20}\). Check nit;gamma(21/2)/(gamma(10)*(20*pi)^(0.5))*(1+x^2/20)^(-21);x;-oo;0 || stt;t(15);m || stt;t(18);v || stt;t(23);P(x>2) | stt;t(17);P(x<-2.5) || stt;t(1);q(0.025) || stt;t(2);q(0.975) ||.

5. Chi-Square distribution \(χ^2(n)\) is a special case of Gamma distribution with parameters \(a=\frac{n}{2},b=2\), where \(n\) is the degree of freedom. The pdf is \(χ_n^2(x)=\frac{x^{\frac{n}{2}-1}e^{-\frac{x}{2}}}{2^{\frac{n}{2}}Γ(\frac{n}{2})}\) for \(x>0\). Use "chi(n)" for a Chi-square distribution with degree of freedom "n" a positive integer. Check stt;chi(n);pdf(x) || stt;gam(n/2,2);pdf(x) || stt;chi(5);m || stt;gam(5/2,2);m || stt;chi(5);v || itg;x^(n/2-1)*e^(-x/2);x;0;oo || stt;chi(7);q(0.975) || stt;chi(1);P(x>3.5) || stt;chi(2);P(x<1) ||.

6. F-Distribution Use "f(m,n)" to create an F-distribution with degrees of freedom "m" and "n". For example, compute the pdf, mean, variance of the F-distribution F(5,8) by stt;f(5,8);pdf(x) || stt;f(5,8);m || stt;f(5,8);v || stt;f(5,8);s || stt;f(5,8);med || stt;f(6,9);q(0.05) || stt;f(9,11);P(x>2) ||.

6. Gamma The Gamma function \(Γ(a)=\int_0^∞x^{a-1}e^{-x}dx\) can be evaluated by "gamma(a)" for \(a>0\) or "itg;x^(a-1)*exp(-x);x;0;oo". Check gamma(1/2) || gamma(18) ||. The density of a Gamma r.v. is \(f_X(x)=\frac{x^{a-1}e^{-\frac{x}{b}}}{b^aΓ(a)}\) for \(x>0,a>0,b>0\). Use "gam(a, b)" for a Gamma distribution with positive parameters "a" and "b". Check stt;gam(a,b);pdf(x) || stt;gam(a,b);cdf(x) || stt;gam(a,b);E(x) || stt;gam(a,b);V(x) || stt;gam(a,b);E(exp(t*x));x || stt;gam(2,3);m || stt;gam(0.5,1);v || stt;gam(1,2);P(x>2) || stt;gam(1,2);P(x<1) || stt;gam(2,6);P(x>1) || stt;gam(3,3);P(x>1&x<4) ||. Exponential distribution is a special Gamma distribution with \(a=1\) and Chi-Square distribution is a Gamma distribution with \(a=\frac{n}{2},b=\frac{1}{2}\). Check stt;gam(1,b);pdf(x) || stt;gam(n/2,1/2);pdf(x) ||.

7. Beta Let \(X_1\sim\) Gamma\((k,b),X_2\sim\) Gamma\((m,b),Y=\frac{X_1}{X_1+X_2}\). Then \(Y\) follows a Beta\((k,m)\) distribution with pdf \(f_Y(y)=\frac{Γ(k+m)}{Γ(k)Γ(m)}y^{k-1}(1-y)^{m-1}\) for \(0< y <1\). Thus, \(E(Y)=\frac{Γ(k+m)}{Γ(k)Γ(m)}\int_0^1yy^{k-1}(1-y)^{m-1}dy=\frac{Γ(k+m)}{Γ(k)Γ(m)}\frac{Γ(k+1)Γ(m)}{Γ(k+m+1)}=\frac{k}{k+m}\),
\(E(Y^2)=\frac{Γ(k+m)}{Γ(k)Γ(m)}\int_0^1y^2y^{k-1}(1-y)^{m-1}dy=\frac{(k+1)k}{(k+m+1)(k+m)}\), and \(\text{Var}(Y)=E(Y^2)-E(Y)^2=\frac{km}{(k+m+1)(k+m)^2}\). The pdf of \(1-Y=\frac{X_2}{X_1+X_2}\) is a Beta\((m,k)\) distribution.

In general, the pdf of a r.v. \(X\sim\) Beta\((a,b)\) is \(f(x)=\frac{1}{B(a,b)}x^{a-1}(1-x)^{b-1}\), where \(B(a,b)=\int_0^1x^{a-1}(1-x)^{b-1}dx=\frac{Γ(a)Γ(b)}{Γ(a+b)}\) is the Beta function for \(a>0,b>0\). Use "beta(a,b)" for evaluating the Beta function, and use "bet(a, b)" for a Beta distribution with positive parameters "a" and "b". Check beta(0.5,1) || beta(2,3) || beta(3,5)-gamma(3)*gamma(5)/gamma(8) ||.

Examples If Y ~ Beta(4, 3), P(Y ≤ 0.5) = 0.3475 by nit;gamma(7)/(gamma(4)*gamma(3))*y^3*(1-y)^2;y;0;0.5 || stt;bet(4,3);pdf(x) || stt;bet(4,3);cdf(x) || stt;bet(4,3);cdf(0.5) || stt;bet(4,3);m || stt;bet(4,3);v ||. Compute P(Y > 0.6) for Y ~ Beta(2.5, 3) by nit;gamma(11/2)/(gamma(5/2)*gamma(3))*y^(3/2)*(1-y)^2;y;0.6;1 || stt;bet(2.5,3);pdf(x) || stt;bet(2.5,3);m || stt;bet(2.5,3);svf(0.6) || stt;bet(2.5,3);P(x>0.6) ||. Compute P(Y > 0.4) for Y ~ Beta(1.5, 2) by nit;gamma(3.5)/(gamma(1.5)*gamma(2))*y^(1/2)*(1-y);y;0.4;1 ||. Check itg;x^(n/2-1)*e^(-x/2);x;0;oo || itg;x^(n/2-1)*e^(-x/3);x;0;oo || itg;x^(z-1)*e^(-x);x;0;oo || nit;x^(5-1)*e^(-x);x;0;oo || nit;x^10*e^(-x);x;0;oo || nit;x^(-1/2)*e^(-x);x;0;oo ||.

8. Cauchy The pdf of a Cauchy distribution with location parameter \(a\) and scale parameter \(b\) is \(f_X(x)=\frac{1}{πb[1+(\frac{x-a}{b})^2]}\) for \(a≥0,b>0\). Use "cau(a, b)" for a Cauchy distribution with parameters "a" and "b". Show the pdf and cdf of a Cauchy distribution by stt;cau(a,b);pdf(x) || stt;cau(a,b);cdf(x) ||. Note that Cauchy distribution does not have mean, variance and moments. Check stt;cau(0,1);P(x>0.8) || stt;cau(2,3);P(x<1.6) || stt;cau(2,3);P(x<1.6) || stt;cau(2,3);q(0.8) || stt;cau(0,1);rvs(20) ||.

9. Bivariate and multivariate normal distribution Suppose \((X_1,Y_1),\cdots,(X_n,Y_n)\) are i.i.d. sample from a bivariate normal distribution of \(X,Y\) with population parameters \(μ_X,μ_Y,σ_X^2,σ_Y^2,ρ\). The correlation between \(X\) and \(Y\) is \(ρ=\frac{σ_{XY}}{σ_Xσ_Y}\), where \(σ_{XY}=E(XY)-E(X)E(Y)\) is the covariance. The correlation between the distributions of sample means \(\overline{X_n}\) and \(\overline{Y_n}\) is \(\frac{\text{Cov}(\overline{X}_n, \overline{Y}_n)}{σ_{\overline{X}_n}σ_{\overline{Y}_n}}\). Since \(σ_{\overline{Y}_n}=\frac{σ_X}{\sqrt{n}},σ_{\overline{X}_n}=\frac{σ_Y}{\sqrt{n}}\), and \(\text{Cov}(\overline{X}_n,\overline{Y}_n) =\frac{1}{n^2}\text{Cov}(\sum_{i=1}^nX_i,\sum_{i=1}^nY_i)=\frac{1}{n}\text{Cov}(X,Y),ρ_{(\overline{X}_n,\overline{Y}_n)}= \frac{\text{Cov}(X,Y)/n}{σ_Xσ_Y/n}=ρ\), the same as the population correlation. Note that the marginal distributions are \(X\sim N(μ_X,σ_X^2),Y\sim N(μ_Y,σ_Y^2)\).

Use "mnm(μ, Σ)" for a multivariate normal distribution with mean vector μ and covariance matrix Σ. Note that the input covariance matrix must be symmetric and positive semidefinite. Keywords "cov", "h" "pdf(x1, x2, ..., xk)","cdf(x1, x2, ..., xk)", and "rvs(n)" available for k-variate normal distribution for computing the covariance, entropy, pdf, cdf, simulating a random sample of size n, respectively. For example, stt;mnm((0,0),((1,0),(0,1)));pdf(1,1) || stt;mnm((0,0),((1,0),(0,1)));cdf(1,1) || stt;mnm([3,4],[[2,1],[1,2]]);cdf(1,2) || stt;mnm([3,4],[[2,1],[1,2]]);rvs(30) || stt;mnm([3,4],[[2,1],[1,2]]);cov || stt;mnm([3,4],[[2,1],[1,2]]);h || stt;mnm((1,2,3),((1,-1,1),(-1,2,-1),(1,-1,2)));cdf(0,1,0) ||.

7. Histograms and Graphs of Distribution Functions

Histograms displays the shape of the distribution of data values. To create a histogram, the range of the data is divided into intervals or bins, and the number or proportion of observations falling in each bin is plotted. Use the keyword "hist" to produce the histograms with size 2000 for the common distribution functions described above. For example, "plt;n(0,1);hist" displays the histogram of the standard normal distribution. Check plt;epn(2);hist || plt;t(12);hist || plt;uni(0,1);hist || plt;cau(2,5);hist || plt;f(7,10);hist || plt;chi(8);hist || plt;gam(3,7);hist || plt;bet(1,2);hist || plt;ber(0.6);hist || plt;bin(20,0.5);hist || plt;poi(4);hist || plt;geo(0.5);hist || plt;nbn(14,0.6);hist || plt;hyp(50,15,16);hist ||.

Graphs of cumulative density functions Use the keyword "cdf" to produce the cdf of a r.v.. For example, "plt;n(0,1);cdf" yields the graph of the cdf of the standard normal distribution. Check plt;epn(2);cdf || plt;t(12);cdf || plt;uni(0,1);cdf || plt;cau(2,5);cdf || plt;f(7,10);cdf || plt;chi(8);cdf || plt;gam(3,7);cdf || plt;bet(1,2);cdf || plt;ber(0.6);cdf || plt;bin(20,0.5);cdf || plt;poi(4);cdf || plt;geo(0.5);cdf || plt;nbn(14,0.6);cdf ||.

Graphs of probability density functions The graph for the pdf of the standard normal distribution can be obtained by "plt;n(0,1);pdf". Use keyword "pdf" for the graph of the pdf of a continuous r.v.. Check other pdfs by plt;epn(2);pdf || plt;t(12);pdf || plt;uni(0,1);pdf || plt;cau(2,5);pdf || plt;f(7,10);pdf || plt;chi(8);pdf || plt;gam(3,7);pdf || plt;bet(1,2);pdf ||.

You can directly enter a density function for its graph. Check plt;20/(2*pi)^(1/2)*exp(-x^2/2);20*gamma(15)/(gamma(14.5) *(29*pi)^.5)*(1+x^2/29)^(-15) || plt;20/(2*pi)^(1/2)*exp(-x^2/2);20*gamma(3)/(gamma(2.5)*(5*pi)^.5)*(1+x^2/5)^(-3) || plt;(1/3)*exp(-x/3); 3*exp(-3*x);15*exp(-15*x) || plt;1-exp(-x/3);1-exp(-3*x);1-exp(-15*x) || plt;10*x^(6-1)*exp(-x/1)/(gamma(6)*1^6);10*x^(3-1)*exp(-x/1)/(gamma(3)*1^3);itv=(0,20) || plt;10*x^(3-1)*exp(-x/2)/(gamma(3)*2^3);10*x^(2-1)*exp(-x/2)/(gamma(2)*2^2);itv=(0,20) || plt;1/(pi*(1^2+(x-0)^2));2/(pi*(2^2+(x-1)^2)) ||.

8. Limit Theorems

1. Inequalities If \(X\) is a non-negative r.v. with finite mean \(E(X)\), then for any \(t>0,P(X>t)≤\frac{E(X)}{t}\) (Markov's inequality). By Markov's inequality, \(P(|X-μ|≥t)=P((X-μ)^2≥t^2)≤\frac{E(X-μ)^2}{t^2}=\frac{σ^2}{t^2}\), provided that the mean \(E(X)=μ\) and variance Var\((X)=σ^2\) of \(X\) exist. Thus, we have \(P(|X-μ|≥t)≤\frac{σ^2}{t^2}\) for any r.v. with finite mean and variance (Chebyshev's inequality). Following Chebyshev's inequality, we observe that \(P(\frac{|X-μ|}{σ}≥\frac{t}{σ})=P(|Z|≥k)≤\frac{σ^2}{t^2}=\frac{1}{k^2}\), where \(t=kσ\). It follows that \(P(|Z|>2)≤\frac{1}{4},P(|Z|>3)≤\frac{1}{9}\). Chebyshev' inequality tells us that the values of a r.v. \(X\) are unlikely to be many standard deviations away from its mean \(μ\).

We use some common distributions to check these two inequalities. If X ~ Poisson(λ), E(x) = Var(x) = λ, and by Markov's inequality P(X > 1) ≤ λ, P(X > 2) ≤ λ/2, P(X > 3) ≤ λ/3, P(X > 4) ≤ λ/4, P(X > 5) ≤ λ/5, and P(X > t) ≤ λ/t for any t > 0. Check stt;poi(0.2);P(X>1) || stt;poi(3);P(x>5) || stt;poi(20);P(x>25) ||. By Chebyshev's inequality P(Z > 2 or Z < -2) ≤ 1/4 and P(Z > 3 or Z < -3) ≤ 1/9. If λ = 4, |Z| > 2 implies X > 8 or X < 0 and |Z| > 3 implies X > 10 or X < -2. Check stt;poi(4);P(x>8) || stt;poi(4);P(x>10) ||. Similarly, check Chebyshev's inequality if X ~ N(0,1): stt;n(0,1);P(X>1) || stt;n(0,1);P(x>2) || stt;n(0,1);P(x>3) || stt;n(0,1);P(x>4) ||. If X ~ exponential(λ), E(X) = 1/λ and Var(X) = 1/λ². Check Markov's inequality by stt;epn(0.5);P(x>1) || stt;epn(0.5);P(x>2) || stt;epn(0.5);P(x>3) || stt;epn(0.5);P(x>10) || stt;epn(4);P(x>1) || stt;epn(4);P(x>0.5) || stt;epn(4);P(x>0.25) ||. Note that |Z| > 2 implies X > 3/λ and |Z| > 3 implies X > 4/λ. Check stt;epn(0.5);P(x>6) || stt;epn(0.5);P(x>8) || stt;epn(4);P(x>3/4) || stt;epn(4);P(x>1) ||.

Example Suppose \(X\sim\) exponential(4). Then \(P(|X-μ|≥1)≤σ^2=P(|X-\frac{1}{4}|≥1)≤\frac{1}{16}\) = 0.0625 by Chebyshev's inequality, where \(μ=E(X)=\frac{1}{4},σ^2=\frac{1}{16}\). We can calculate \(P(|X-\frac{1}{4}|≥1)=P(X>\frac{5}{4})\) = 0.0101 < 0.0625. Check nit;x*4*exp(-4*x);x;5/4;oo || stt;epn(4);m || stt;epn(4);v || stt;epn(4);std || stt;epn(4);P(x>5/4) ||.

2. Law of large numbers Let \(X_1,\cdots,X_n\) be i.i.d. r.v.'s with finite mean \(E(X_i)=μ\), and let \(\overline{X}_n=\frac{1}{n}\sum_{i=1}^nX_i\). Then for any \(ε>0\), the weak law of large numbers (WLLN) says \(\lim\limits_{n→∞}P(|\overline{X}_n-μ|>ε)=0\) (denote \(\overline{X}_n\overset{p}{→}μ\)), and the strong law of large numbers (SLLN) says \(P(\lim\limits_{n→∞}|\overline{X}_n-μ|>ε)=0\) (denote \(\overline{X}_n→μ\)). The WLLN tells how a sequence of probabilities converges, but the SLLN tells how the sequence of r.v.'s behaves in the limit. The WLLN is a direct result of Chebyshev's inequality because \(P(|\overline{X}_n-μ|>ε)≤\frac{σ^2}{nε^2}→0\) as \(n→∞\), where \(σ^2\) = Var\((X)\) and \(σ_{\overline{x}_n}=\frac{σ}{\sqrt{n}}\) is the standard deviation (or standard error) of the distribution of sample means.

Use "sem" to estimate the standard error (se) \(σ_{\overline{x}_n}\) of sample mean. For example, if \(X\sim N(μ,σ^2),σ_{\overline{x}_n}=\frac{2}{\sqrt{n}}\), stt;n(1,2);rvs(100);d || gives the se is about 0.2, which means if n = 100, the sample mean is about 0.2 standard deviations away from the population mean. The se from stt;n(1,2);rvs(10000);d is about 0.02. Check stt;poi(3);rvs(500);d || stt;epn(4);rvs(1000);d ||.

3. Central limit theorem (CLT) Let \(X_1,\cdots,X_n\) be i.i.d. r.v.'s with finite mean \(μ\) and variance \(σ^2\) and let \(\overline{X}_n=\frac{1}{n}\sum_{i=1}^nX_i\). Then \(Z_n=\frac{\overline{X}_n-μ}{σ/\sqrt{n}}\overset{d}{→}Z\sim N(0,1)\). In other words, \(\lim\limits_{n→∞}P(Z_n≤z)=\int_{-∞}^z\frac{1}{\sqrt{2π}}e^{-\frac{x^2}{2}}dx\). By the CLT, we can use a normal distribution to approximate probabilities about the sample mean (or random sum), regardless of the distribution of \(X_1,\cdots,X_n\).

Example Consider tossing a fair coin 200 times and denote X as the number of heads. Calculating the exact probability P(X > 98) = 0.5840 by the binomial distribution is difficult, but we can approximate it by a normal distribution with mean np = 100 and variance 50. Check stt;bin(200,0.5);P(x>98) || stt;n(100,50^(1/2));P(x>98) ||, which gives 0.6114. Similarly, calculate the exact probability P(X > 60) = 0.054 for X ~ Poisson(49) is difficult by hand. Check stt;poi(49);P(x>60) ||. However, we can approximated by N(49, 7) by stt;n(49,7);P(x>60), which gives 0.058.

9. Functions of Random Variables

1. Transformation of random variables Let \(X\sim f_X(x)\) be a discrete r.v.. If \(U=h_1(X)\) and \(h_1\) is strictly increasing \(⇒X=h_2(U)\), then the pmf of \(U\) is \(G(U)=P(U=u)=P(h_1(X)=u)=P(X=h_2(U))=f_X(h_2(U))\). Let \(X,Y\sim f_{X,Y}(x,y)\) be discrete random variables and \(U=h_1(X,Y),V=h_2(X,Y)⇒X=h_3(U,V),Y=h_4(U,V)\), then the joint pdf of \(U,V\) is given by \(G(U,V)=f_{X,Y}(h_3(U,V),h_4(U,V))\).

Let \(X\sim f_X(x)\) be a continuous random variable. If \(U=h_1(X)\) and \(h_1\) is strictly increasing \(⇒X=h_2(U)\). Then the cdf of \(U\) is \(G(U)=P(U≤u)=P(h_1(X)≤u)=P(X≤h_2(u))=\int_{-∞}^{h_2(u)}f_X(x)dx\), and the pdf is \(g(U)=\frac{d}{du}\int_{-∞}^{h_2(u)}f_X(t)dt=f(h_2(u))|h'_2(u)|\). Similarly, let \(X,Y\sim f_{X,Y}(x,y)\) be continuous random variables and \(U=h_1(X,Y),V=h_2(X,Y)⇒X=h_3(U,V),Y=h_4(U,V)\), then the density is \(g(u,v)=f_{X,Y}(h_3(u,v),h_4(u,v))|J|\), where \(J=\frac{∂(x,y)}{∂(u,v)}=\begin{vmatrix}\frac{∂x}{∂u}&\frac{∂x}{∂v}\\ \frac{∂y}{∂u}&\frac{∂y}{∂v} \end{vmatrix}\) is the Jacobian.

Example 1
(1) Suppose \(X\sim f(x)=e^{-x},Y=g(X)=X^2\). Then \(F_Y(y)=P(Y≤y)=P(X^2≤y)=P(X≤\sqrt{y})=1-e^{-\sqrt{y}}\) because the cdf of \(X\) is \(F_X(x)=1-e^{-x}\). As a result, the pdf of \(Y\) is \(f_Y(y)=F'_Y(y)=\frac{1}{2\sqrt{y}}e^{-\sqrt{y}},y≥0\). Check itg;1/(2*y^(1/2))*exp(-y^(1/2));y;0;oo ||.
(2) Let \(X\sim U(0,1),Y=g(X)=-2\ln X. F_Y(y)=P(Y≤y)=P(-2\ln X≤y)=P(X≥e^{-\frac{y}{2}})=1-F_X(e^{-\frac{y}{2}})=1-e^{-\frac{y}{2}}\) because the cdf of \(X\) is \(F_X(x)=x\). The pdf of \(Y\) is \(f_Y(y)=F'_y(y)=\frac{1}{2}e^{-\frac{y}{2}}\), which is an exponential distribution with parameter \(λ=\frac{1}{2}\).
(3) Let \(X\sim f(x)=4x^3-3x^2+2x,0< x <1,Y=g(X)=\sqrt{X}\). Then \(F_Y(y)=P(Y≤y)=P(\sqrt{X}⩽y)=P(X≤y^2)=F_X(y^2)\). Since \(F_X(x)=\int_0^x4t^3-3t^2+2tdt=x^4-x^3+x^2,F_X(y^2)=y^8-y^6+y^4\), and \(f_Y(y)=F'_Y(y)=8y^7-6y^5+4y^3\) for \(0< y <1\).
(4) Let \(Z\sim N(0,1),Y=g(Z)=Z^2,F_Y(y)=P(Y≤y)=P(Z^2≤y)=P(-\sqrt{y}≤z≤\sqrt{y})=\Phi(\sqrt{y})-\Phi(-\sqrt{y})\). Thus the pdf of \(Y\) is \(f_Y(y)=F'_Y(y)=\frac{1}{2\sqrt{y}}f_Y(\sqrt{y})+\frac{1}{2\sqrt{y}}f_Y(-\sqrt{y})=\frac{1}{\sqrt{y}}\frac{1}{\sqrt{2π}}e^{-\frac{y}{2}}=\frac{1}{Γ(\frac{1}{2})2^{\frac{1}{2}}}y^{\frac{1}{2}-1}e^{-\frac{y}{2}}=\text{Gamma}(\frac{1}{2},2)=χ^2_1,y≥0\). Note that \(Γ(\frac{1}{2})=\sqrt{π}\) and Gamma\((\frac{n}{2},2)=χ^2_n\).
(5) Let \(X_1,X_2\sim f(x)=2e^{-2x},Y=g(X_1,X_2)=\frac{X_2}{X_1}\). Then \(F_Y(y)=P(\frac{X_2}{X_1}≤y)=P(X_2≤yX_1)=\int_0^∞\int_0^{yx_1} 4e^{-2(x_1+x_2)}dx_2dx_1\)
\(=\frac{y}{y+1}\), and \(f_Y(y)=F'_Y(y)=\frac{1}{(1+y)^2},y>0\).

Example 2
(1) Suppose \(X\sim e^{-x},Y=X^2\). Then \(F_Y(y)=P(Y≤y)=P(X^2≤y)=P(X≤\sqrt{y})=1-e^{-\sqrt{y}}\), and the pdf of \(Y\) is \(f_Y(y)=F'_Y(y)=\frac{1}{2\sqrt{y}}e^{-\sqrt{y}},y≥0\).
(2) Suppose \(X\sim U(1,3),Y=X^3⇒X=Y^{\frac{1}{3}}\). Then \(\frac{dx}{dy}=\frac{1}{3}y^{-\frac{2}{3}},f_X(x(y))=\frac{1}{2}\), and \(f_Y(y)=\frac{1}{2}\frac{1}{3}y^{-\frac{2}{3}}= \frac{1}{6}y^{-\frac{2}{3}},1≤y≤27\).
(3) Suppose \(X\sim \frac{3x^2}{133}\) for \(-2≤X≤5\) and \(U=X^2\). The distribution function of \(U\) is \(G(U)=P(U≤u)=P(X^2≤u)\)
\(=P(-\sqrt{u}≤X≤\sqrt{u})=\int_{-\sqrt{u}}^{\sqrt{u}}f(x)dx=\begin{cases}\int_{-\sqrt{u}}^{\sqrt{u}}\frac{3x^2}{133}dx=\frac{2u^{3/2}}{133}&0≤u≤4\\\int_2^{\sqrt{u}}\frac{3x^2}{133}dx=\frac{u^{3/2}-8}{133}&4< u≤25 \end{cases}\). Check itg;3*x^2/133;x;-u^(1/2);u^(1/2) || itg;3*x^2/133;x;2;u^(1/2) ||. The density is \(g(u)=\begin{cases}\frac{3u^{\frac{1}{2}}}{133}&0≤u≤4\\\frac{3u^{\frac{1}{2}}}{266}&4≤u≤25\end{cases}\). Check itg(3*u^(1/2)/133,u,0,4)+itg(3*u^(1/2)/266,u,4,25) ||, which equals 1.
(4) \(X_1,X_2\sim N(0,1), Y=g(X_1,X_2)=X_1^2+X_2^2\). Then \(F_Y(y)=P(X_1^2+X_2^2≤y)=\iint_R\frac{1}{2π}e^{-\frac{x_1^2+x_2^2}{2}}dx_1x_2\)
\(=\frac{1}{2π}\int_0^{2π}\int_{r=0}^{\sqrt{y}}re^{-\frac{r^2}{2}}drdθ=1-e^{-\frac{y}{2}}\), and thus \(f_Y(y)=F'_Y(y)=\frac{1}{2}e^{-\frac{y}{2}}=\) Gamma\((1,2),y≥0\).
(5) If \(X_1,X_2\sim U(0,1),Y=X_1+X_2,f_Y(y)=\begin{cases}\int_0^y1dx_1=y&\text{ for }y< 1\\\int_{y-1}^11dx_1=2-y&\text{ for }y> 1\end{cases}\).
(6) Suppose \(X_1,X_2\sim f(x)=e^{-x},Y=\frac{X_2}{X_1}\). Let \(Y_1=X_1,Y_2=\frac{X_2}{X_1}\). Solve for \(x_1=y_1,x_2=x_1y_2=y_1y_2\). The Jacobian is \(\begin{vmatrix}1&0\\y_2&y_1\end{vmatrix}=y_1\). Substitute \(x_1,x_2\) into the joint pdf of \(X_1,X_2\) and get \(e^{-x_1-x_2}=e^{-y_1-y_1y_2}\). Thus, the pdf of \(Y_2\) is \(\int_0^∞ ze^{-y_1-y_1y_2}dy_1=\frac{1}{(1+y_2)^2}\), or \(f_Y(y)=\frac{1}{(1+y)^2}\). Check itg;z*e^(-z-z*y);z;0;oo ||.

Example 3 Let \(X,Y\sim f(x,y)=\frac{4xy}{875}\) for \(0≤X≤5,1≤Y≤6\). Find the density of \(U=X+2Y\). Let \(V=X\). Then \(X=V,Y=\frac{U-V}{2}\) and the Jacobian \(|J|=\frac{1}{2}\). Thus, \(g(u,v)=\frac{u(u-v)}{875}\) for \(0≤v≤5,2≤u-v≤12\). The marginal density \(g(U)=\int_0^{u-2}\frac{u(u-v)}{875}dv=\frac{(u-2)^2(u+4)}{5250}\) for \(2≤u≤7;g(U)=\int_0^5\frac{u(u-v)}{875}dv=\frac{u}{70}-\frac{1}{21}\) for \(7≤u≤12;g(U)=\int_{u-12}^5\frac{u(u-v)}{875}dv=\frac{507u-u^3-3706}{5250}\). Check itg;v*(u-v)/875;v;0;u-2 || itg;v*(u-v)/875;v;0;5 || itg;v*(u-v)/875;v;u-12;5 ||.

Example 4 Find the density of \(U=XY\) if \(X,Y\) have a joint density \(f(x,y)\). Let \(V=X⇒X=V,Y=\frac{U}{V}\), and the Jacobian \(J=\frac{-1}{V}\). Thus, the joint density \(g(u,v)=f(v,\frac{u}{v})|v^{-1}|\), and the marginal density is \(g(u)=\int_{-∞}^∞f(v,\frac{u}{v})|v^{-1}|dv\). We can use the definition \(G(u)=P(U≤u)=P(XY≤u)=\int_{XY≤u}f(x,y)dxdy\). If \(u≥0\), then \(X>0,Y≤\frac{u}{X}\); if \(X<0,Y≥\frac{u}{X}\). Thus, \(G(u)=\int_{-∞}^0\int_{u/x}^∞f(x,y)dydx+\int_0^∞\int_{-∞}^{u/x}f(x,y)dydx\), and \(g(u)=G'(u)=\int_{-∞}^0f(x,\frac{u}{x})\frac{-1}{x}dx+\int_0^∞f(x,\frac{u}{x})\frac{1}{x}dx\)
\(=\int_{-∞}^∞f(v,\frac{u}{x})|x^{-1}|dx\). The same result can be obtained for \(u<0\).

2. Convolution Suppose \(X,Y\) have a joint density \(f(x,y)\) and \(U=X+Y\). Then the density of \(U\) is \(g(U)=\int_{-∞}^∞f(v,u-v)dv\). To see this, let \(V=X⇒X=V,Y=U-V\) and the Jacobian is \(J=-1\). Thus, the joint density of \(U,V\) is \(g(u,v)=f(v,u-v)|J|=f(v,u-v)\)
\(⇒g(u)=\int_{-∞}^∞f(v,u-v)dv\). In case of \(X,Y\) are independent and \(f(x,y)=f_X(x)f_Y(y)\), then \(g(u)=\int_{-∞}^∞f(v,u-v)dv\)
\(=\int_{-∞}^∞f_X(v)f_Y(u-v)dv≡f_X*f_Y\). We can find \(g(U)=G'(u)\), where \(G(u)=P(X+Y≤u)=\iint_{X+Y≤u}f(x,y)dxdy\)
\(=\int_{x=-∞}^∞\int_{y=-∞}^{u-x}dydx\). Thus, \(G'u)=\int_{-∞}^∞f(x,u-x)dx\).

For example, if \(f_X(x)=2e^{-2x},f_Y(y)=3e^{-3y},U=X+Y\) for \(x≥0,y≥0\), then \(g(u)=\int_0^u2e^{-2v}3e^{-3(u-v)}dv=6(e^{-2u}-e^{-3u})\) for \(u≥v\) and \(g(u)=0\) for \(u< v\). Check itg;6*e^(-2*v)*e^(-3*(u-v));v;0;u || itg;6*(e^(-2*u)-e^(-3*u));u;0;oo ||. Note that \(f_X*f_Y=f_Y*f_X\), which means \(\int_{-∞}^∞f_X(v)f_Y(u-v)dv=\int_{-∞}^∞f_Y(v)f_X(u-v)dv\) by change of variables.

Back to top