MSc:ProbabilityStatistics

Probability and Statistics

• Course name: Probability and statistics
• Course number: BS-02

Course Characteristics

Key concepts of the class

• Probability space & probability basics
• Random variables and their characteristics
• Limit theorems
• Introduction into mathematical statistics

What is the purpose of this course?

The main idea of the course is to study mathematical basis of modelling random experiments. The course includes constructing a probability space, a model of a random experiment, and its applications to practice. After that, random variables and their properties are considered. As examples of applying this theoretical background, limit theorems of probability theory are proved (law of large numbers, central limit theorem) and some elements of mathematical statistics are studied.

- What should a student remember at the end of the course?

By the end of the course, the students should be able to:

• construct a mathematical model of a random experiment (probability space)
• calculate conditional probabilities
• use probability generating functions for discrete random variables
• find confidence intervals for parameters of a normal distribution
• estimate unknown parameters of distributions

- What should a student be able to understand at the end of the course?

By the end of the course, the students should be able to understand:

• probability function and its properties
• law of total probability and Bayes’ theorem
• independence of events and of random variables
• different continuous distributions
• multivariate distributions for discrete and continuous cases
• maximum likelihood estimator method

- What should a student be able to apply at the end of the course?

By the end of the course, the students should be able to ...

• find expected value, variance and other characteristics of a random variable
• apply limit theorems (law of large numbers and central limit theorem)
• find parameters of a simple linear regression

Course evaluation

Course grade breakdown

		Proposed points
Test 1	?	10
Midterm	?	25
Test 2	?	10
Participation	?	5
Final exam	?	50

Grades range

Course grading range

		Proposed range
A. Excellent	90-100	85-100
B. Good	75-89	65-84
C. Satisfactory	60-74	45-64
D. Poor	0-59	0-44

Resources and reference material

• Durrett Rick. (2019) Probability. Theory and Examples,
• Suhov Y, Kelbert M (2005) Probability and Statistics by Example, Cambridge University Press

Course Sections

The main sections of the course and approximate hour distribution between them is as follows:

Course Sections

Section	Section Title	Teaching Hours
1	Basics of Probability	12
2	Univariate Distributions	24
3	Multivariate distributions	24
4	Limit theorems & Introduction into Mathematical Statistics	30
hline

Section 1

Section title:

Basics of Probability

Topics covered in this section:

• Probability space. ${\textstyle \sigma }$-algebra of events. Axiomatic definition of probability
• Classical model of probability
• Independence of events
• Conditional probability
• Probability of a sum of events (of a product of events)
• Law of total probability. Bayes theorem.

What forms of evaluation were used to test students’ performance in this section?

|a|c| & Yes/No
Development of individual parts of software product code & 0
Homework and group projects & 1
Midterm evaluation & 1
Testing (written or computer based) & 1
Reports & 0
Essays & 0
Oral polls & 0
Discussions & 0

Typical questions for ongoing performance evaluation within this section

1. Give an example of three events that are pairwise independent, but not mutually independent.
2. Can disjoint events be independent?
3. A six-sided die is rolled. Construct an algebra of events for this random experiment.
4. Derive the formula for calculating the probability of a sum ${\textstyle P(A+B+C)}$.

Typical questions for seminar classes (labs) within this section

1. Each game of a match between two equal players can end with a victory of one of them with probability ${\textstyle 0{.}5}$ independently of the other games. Each victory yields one point, and the match is played until one of the players scores 6 points. Due to technical reasons the match was interrupted when the score was ${\textstyle 5:3}$ in favour of the first player. What do you think is a fair way to distribute the prize between the players?
2. Seventy numbers are chosen at random from integers ${\textstyle 1,2,3,\cdot ,100}$. What is the probability that the largest number chosen is ${\textstyle 98}$?
3. A hospital specialises in curing three types of diseases: ${\textstyle A}$, ${\textstyle B}$ and ${\textstyle C}$. On average, there are ${\textstyle 50\%}$ of patients who suffer from disease ${\textstyle A}$, ${\textstyle 30\%}$ of patients with disease ${\textstyle B}$, and ${\textstyle 20\%}$ of patients with disease ${\textstyle C}$ (each of the patients has exactly one of these diseases). The probabilities to fully recover from the diseases are equal to ${\textstyle 0{.}95}$, ${\textstyle 0{.}9}$ and ${\textstyle 0{.}85}$ respectively. A patient who came to the hospital recovered completely. What is the probability that he had disease ${\textstyle B}$?
4. A white ball is added into an urn that initially contained ${\textstyle n}$ balls. It is known that the probabilities of having ${\textstyle 0,1,2,\ldots n}$ white balls (at the start) in the urn are equal to each other. (a) One ball is taken at random from the urn. What is the probability that the ball is white? (b) The ball taken from the urn has turned out to be white. Find the most probable number of white balls that were in the urn from the start.

Test questions for final assessment in this section

1. The probabilities for three students to pass the exam are equal to ${\textstyle {\frac {11}{12}}}$, ${\textstyle {\frac {11}{14}}}$ and ${\textstyle {\frac {18}{25}}}$ respectively. Determine the probability that at least one student passes the exam given that they pass or fail independently of each other.
2. One of 10-digit numbers in which digits go in non-increasing order is chosen at random. Find the probability that exactly 4 different digits are used in this number.
3. Two persons play a game. They take turns in rolling a 10-sided fair die. The first one wins as soon as he rolls 9 or 10, whereas the second one wins as soon as he gets no more than 4. (The game goes on until one of the player’s winning conditions is met). Determine the probability for the first player to win the game.
4. Two dice are rolled simultaneously. What is the probability that the sum is even given that it is a multiple of 3?
5. There are 5 white balls and 7 green balls in the first urn; 2 white balls and 10 green balls in the second urn. The third urn, that has initially been empty, is filled with the balls: 4 balls are taken from the first urn, 6 balls are taken from the second urn, and they are placed into the third urn. After that, 2 balls are taken at random from the third urn. It turns out that both these balls are green. Determine the probability that these balls originate from different urns.

Section 2

Section title:

Univariate Distributions

Topics covered in this section:

• Z-test
• Bernoulli trials and their generalisations
• Discrete random variables and their properties.
• Continuous random variables and their properties

What forms of evaluation were used to test students’ performance in this section?

|a|c| & Yes/No
Development of individual parts of software product code & 0
Homework and group projects & 1
Midterm evaluation & 1
Testing (written or computer based) & 1
Reports & 0
Essays & 0
Oral polls & 0
Discussions & 0

Typical questions for ongoing performance evaluation within this section

1. Bernoulli trials
2. Bernoulli trials. The most probable quantity of successes
3. Bernoulli shift
4. Random variables: general definition. Cumulative distribution function
5. Discrete random variables. Expected value and variance
6. Uniform distribution on a finite set
7. Binomial distribution
8. Indicator random variables
9. Geometric distribution
10. Poisson distribution
11. Probability generating function
12. Continuous random variables. Probability density function. Expected value and variance
13. Uniform distribution on a limited interval
14. Exponential distribution
15. Normal distribution
16. Functional dependence of random variables

Typical questions for seminar classes (labs) within this section

1. On average ${\textstyle 25\%}$ students subscribe to the newsletter. Determine the most probable number of subscribers out of (a) 100 students; (b) 103 students.
2. Two players are playing a match (that consists of several games), each of the games can finish in favour of the younger player with probability ${\textstyle 0{.}6}$ and in favour of the older player with probability ${\textstyle 0{.}4}$ The younger player has won exactly five games in the first eight games. What is the probability that he started the match with a defeat?
3. Find the range of variance for random variable ${\textstyle \eta }$ if its cumulative distribution function is given by ${\textstyle F_{\eta }(x)={\begin{cases}0,&x\leq 0,\\0{.}3,&0<x\leq 2,\\b,&2<x\leq 6,\\1,&x>6,\end{cases}}}$ if ${\textstyle b}$ is a parameter that belongs to ${\textstyle (0{.}3;1)}$.
4. Six people entered the lift at the ground floor of a nine-storied house. Find the expected value for (a) the number of stops where exactly one person gets off the lift; (b) the number of stops where exactly two persons leave the lift.
5. Find the expected value and variance of ${\textstyle a^{\xi }}$ given that ${\textstyle \xi \sim Bin(n,p)}$.
6. Random variable ${\textstyle Y}$ has a uniform distribution on interval ${\textstyle (a;b)}$, and ${\textstyle EY=}$ Var ${\textstyle Y=3}$. Find ${\textstyle a}$ and ${\textstyle b}$.

Test questions for final assessment in this section

1. Is it possible for random variable ${\textstyle X}$ to have a binomial distribution if (a) ${\textstyle EX=6}$ and ${\textstyle VarX=3}$; (b) ${\textstyle EX=7}$ and ${\textstyle VarX=4}$?
2. Let ${\textstyle Y}$ be number of sixes and ${\textstyle Z}$ be number of fours one gets when rolling six dice. Find the expected value and variance of ${\textstyle Y+Z}$.
3. Let ${\textstyle Z}$ be a random variable with geometric distribution. Prove that ${\textstyle P(Z=n+k|z>n)=P(Z=k)}$ (lack of memory property of geometric distribution).
4. Let us consider a sphere of radius ${\textstyle R}$ centered at ${\textstyle O}$. Point ${\textstyle M}$ is chosen at random inside this circle. Random variable ${\textstyle \xi }$ is equal to the length of ${\textstyle OM}$. Find the cumulative distribution function, probability density, expected value and variance of ${\textstyle \xi }$.
5. Random variable ${\textstyle \theta }$ is exponentially distributed with parameter ${\textstyle \lambda }$. Calculate the probabilities that ${\textstyle \theta }$ belongs to intervals ${\textstyle (0;1),(1;2),\ldots ,(n-1;n),\ldots }$ and show that these probabilities form a geometric sequence. What is the common ratio of this sequence?
6. It is known that ${\textstyle \xi }$ is normally distributed random variable, and ${\textstyle P\{|\xi -E\xi |<1\}=0{.}3}$. Find the probability that ${\textstyle |\xi -E\xi |<2}$.

Section 3

Section title:

Multivariate Distributions

Topics covered in this section:

• Discrete multivariate distributions
• Two variate continuous distributions
• Multivariate continuous distributions
• Multivariate normal distribution

What forms of evaluation were used to test students’ performance in this section?

|a|c| & Yes/No
Development of individual parts of software product code & 0
Homework and group projects & 1
Midterm evaluation & 0
Testing (written or computer based) & 1
Reports & 0
Essays & 0
Oral polls & 0
Discussions & 0

Typical questions for ongoing performance evaluation within this section

1. The joint distribution of ${\textstyle \xi }$ and ${\textstyle \eta }$ is provided in the table below.

   ${\textstyle \eta \backslash \xi }$	${\textstyle -1}$	0	1
   ${\textstyle -2}$	Failed to parse (unknown function "\mathstrut"): {\textstyle \frac{\mathstrut3}{\mathstrut17}}	${\textstyle {\frac {4}{17}}}$	${\textstyle {\frac {1}{17}}}$
   2	Failed to parse (unknown function "\mathstrut"): {\textstyle \frac{\mathstrut1}{\mathstrut17}}	${\textstyle {\frac {5}{17}}}$	${\textstyle {\frac {3}{17}}}$

   
   

   (a) Find marginal distributions of ${\textstyle \xi }$ and ${\textstyle \eta }$; (b) find expected value and variance for ${\textstyle \xi }$ and ${\textstyle \eta }$; (c) determine if ${\textstyle \xi }$ and ${\textstyle \eta }$ are independent; (d) find correlation coefficient of ${\textstyle \xi }$ and ${\textstyle \eta }$; (d) find conditional expected values ${\textstyle E(\xi |\eta )}$ and ${\textstyle E(\eta |\xi )}$.

2. A fair coin is flipped thrice, and for every tails obtained we write a plus; for every heads obtained we write a minus. Random variable ${\textstyle \xi }$ is equal to the quantity of tails, and random variable ${\textstyle \eta }$ is equal to the quantity of sign changes in the sequence. (a) Find marginal distributions of ${\textstyle \xi }$ and ${\textstyle \eta }$; (b) find expected value and variance for ${\textstyle \xi }$ and ${\textstyle \eta }$; (c) determine if ${\textstyle \xi }$ and ${\textstyle \eta }$ are independent; (d) find correlation coefficient of ${\textstyle \xi }$ and ${\textstyle \eta }$; (d) find conditional expected values ${\textstyle E(\xi |\eta )}$ and ${\textstyle E(\eta |\xi )}$.

3. ${\textstyle n}$ letters have been written and ${\textstyle n}$ envelopes have been inscribed for these letters. An absent minded secretary places the letters into envelopes at random and sends them with the evening post. (a) What is the probability that at least one letter reaches its destination? (b) Find the average quantity of letter that reach their destination.

Typical questions for seminar classes (labs) within this section

1. How many times does one have to flip a coin to get the results “heads”, “heads” in succession? Is the result going to change if we replace the sequence with “tails”, “heads”?
2. Forty three equally strong sportsmen take part in a ski race; 18 of them belong to club ${\textstyle A}$, 10 to club ${\textstyle B}$ and 15 to club ${\textstyle C}$. What is the average place for (a) the best participant from club ${\textstyle B}$; (b) the worst participant from club ${\textstyle B}$?
3. How much rolls does one need on average to get a sequence “6”, “6” when rolling a symmetric six-sided die? And if we change this sequence to “6”, “6”, “6”?
4. ${\textstyle \zeta }$ is the quantity of threes and ${\textstyle \eta }$ is the quantity of odd digits obtained when rolling a fair die ${\textstyle K}$ times. Find correlation coefficient between ${\textstyle \eta }$ and ${\textstyle \zeta }$.

Test questions for final assessment in the course

1. A fair die is rolled until a four is obtained. Find the expected value of a sum obtained in all the rolls.
2. Find a correlation coefficient between the quantity of sixes and the quantity of fives obtained in ${\textstyle K}$ rolls of a fair die.
3. Expected value ${\textstyle \mu }$ and covariance matrix ${\textstyle {\mathcal {K}}}$ of random vector ${\textstyle \xi =(\xi _{1},\xi _{2},\xi _{3})^{T}}$ are given. Calculate the expected value and variance of ${\textstyle \eta =\xi _{1}-\xi _{3}}$.
4. ${\textstyle N}$ letters have been written and envelopes have been inscribed for these letters. An absent-minded secretary puts the letters into envelopes at random and sends them with the evening post. Find ${\textstyle E\xi }$ and ${\textstyle Var\xi }$, where ${\textstyle \xi }$ is the number of letters that reached their destination.

Section 1

Section title:

Limit Theorems and Introduction into Mathematical Statistics

Topics covered in this section:

• Chebyshev’s inequality. Law of large numbers
• Central limit theorem
• Estimating unknown distribution parameters
• Simple linear regression

What forms of evaluation were used to test students’ performance in this section?

|a|c| & Yes/No
Development of individual parts of software product code & 0
Homework and group projects & 1
Midterm evaluation & 0
Testing (written or computer based) & 1
Reports & 0
Essays & 0
Oral polls & 0
Discussions & 0

Typical questions for ongoing performance evaluation within this section

1. Is it possible that for some random variable an equality is reached in Chebyshev’s inequality?
2. Give an example of a sequence of random variables that satisfies the law of large numbers, and another sequence that does not satisfy it.
3. What is a maximum likelihood estimator? Calculate maximum likelihood estimators for parameters of a normally distributed sample.
4. Are least square estimators of parameters of a linear regression unbiased? Are they consistent?

Typical questions for seminar classes (labs) within this section

1. Let ${\textstyle \xi _{n}\sim {\begin{pmatrix}-{\sqrt {n}}&0&{\sqrt {n}}\\1/2n&1-1/n&1/2n\end{pmatrix}}}$. Does this sequence comply the law of large numbers?
2. Let ${\textstyle \xi _{n}\sim {\begin{pmatrix}-n&0&n\\2^{-n}&1-2^{-n+1}&2^{-n}\end{pmatrix}}}$. Does this sequence comply the law of large numbers?
3. Find a maximum likelihood estimator for a parameter of a sample with Poisson distribution.
4. Prove that sample variance is a biased estimator of variance for a sample of independent identically distributed random variables. Show that Bessel’s correction makes it an unbiased estimator.
5. Provide an example of an unbiased estimator that does not have the least mean square error possible.

Test questions for final assessment in this section

1. The probabilities for three students to pass the exam are equal to ${\textstyle {\frac {11}{12}}}$, ${\textstyle {\frac {11}{14}}}$ and ${\textstyle {\frac {18}{25}}}$ respectively. Determine the probability that at least one student passes the exam given that they pass or fail independently of each other.
2. Let us consider independent identically distributed random variables with uniform distribution on ${\textstyle (\theta ;\theta +3)}$. Find the maximum likelihood estimator of ${\textstyle \theta }$. Which one of these estimators is unbiased? Justify your answer.
3. Find the smallest possible value of ${\textstyle P{\bigl (}|\xi -E\xi |\leq 3{\sqrt {Var\xi }}{\bigr )}}$.
4. The probability that a new-born baby is a boy is equal to ${\textstyle 0{.}52}$. Find the interval which contains the quantity of boys out of ${\textstyle 10\,000}$ newborn babies with probability ${\textstyle 0{.}98}$.
5. Prove that for multivariate normal distribution uncorrelatedness implies independence.
6. Use characteristic functions to show that a sum of independent (and not necessarily identically distributed) random variables also has normal distribution.