View source for BSc: Probability And Statistics.s22.new vers

= Probability and Statistics =

* <span>'''Course name:'''</span> Probability and Statistics
* <span>'''Course number:'''</span> XYZ
* <span>'''Knowledge area:'''</span> Math, Computational Science

== Administrative details ==

* <span>'''Faculty:'''</span> Computer Science and Engineering
* <span>'''Year of instruction:'''</span> 2nd year of BS
* <span>'''Semester of instruction:'''</span> 2nd semester
* <span>'''No. of Credits:'''</span> 4 ECTS
* <span>'''Total workload on average:'''</span> 144 hours overall
* <span>'''Class lecture hours:'''</span> 2 per week
* <span>'''Class tutorial hours:'''</span> 2 per week
* <span>'''Lab hours:'''</span> 2 per week
* <span>'''Individual lab hours:'''</span> 0
* <span>'''Frequency:'''</span> weekly throughout the semester
* <span>'''Grading mode:'''</span> letters: A, B, C, D

== Prerequisites ==

* [https://eduwiki.innopolis.university/index.php/BSc:_Mathematical_Analysis_I CSE201 &mdash; Mathematical Analysis I]
* [https://eduwiki.innopolis.university/index.php/BSc:Logic_and_Discrete_Mathematics CSE113 &mdash; Philosophy I - (Discrete Math and Logic)]

== Course outline ==

The course is designed to provide Software Engineers and Computer Scientists by correct knowledge of basic (core) concepts, definitions, theoretical results and applied methods &amp; techniques of Probability Theory and Mathematical Statistics. The main idea of the course is to study mathematical basis of modelling random experiments. The course includes constructing a probability space, a model of a random experiment, and its applications to practice. After that, random variables and their properties are considered. As examples of applying this theoretical background, limit theorems of probability theory are proved (law of large numbers, central limit theorem) and some elements of mathematical statistics are studied.

== Expected learning outcomes ==

* Probability and sample (probability) spaces
* Discrete and continuous distribution
* Mean and variance
* Multivariate discrete &amp; continuous distributions
* Central limit theorem, law of large numbers
* Linear regression and correlation

== Expected acquired core competences ==

* Basic (core) concepts, definitions, theoretical results and applied methods &amp; techniques of Probability Theory and Mathematical Statistics

== Reference material ==

== Required computer resources ==

Any electronic spreadsheet (Excel for example) that provides data sorting and graph &amp; chart drawing.

== Course Characteristics ==

=== Key concepts of the class ===

* Probability space &amp; probability basics
* Random variables and their characteristics
* Limit theorems
* Introduction into mathematical statistics

=== What is the purpose of this course? ===

=== - What should a student remember at the end of the course? ===

By the end of the course, the students should be able to:

* construct a mathematical model of a random experiment (probability space)
* calculate conditional probabilities
* use probability generating functions for discrete random variables
* find confidence intervals for parameters of a normal distribution
* estimate unknown parameters of distributions

=== - What should a student be able to understand at the end of the course? ===

By the end of the course, the students should be able to understand:

* probability function and its properties
* law of total probability and Bayes’ theorem
* independence of events and of random variables
* different continuous distributions
* multivariate distributions for discrete and continuous cases
* maximum likelihood estimator method

=== - What should a student be able to apply at the end of the course? ===

By the end of the course, the students should be able to ...

* find expected value, variance and other characteristics of a random variable
* apply limit theorems (law of large numbers and central limit theorem)
* find parameters of a simple linear regression

=== Course evaluation ===

{|
|+ Course grade breakdown
!
!
!align="center"| '''Proposed points'''
|-
| Weekly tests
| ?
|align="center"| 25
|-
| Midterm
| ?
|align="center"| 25
|-
| Final exam
| ?
|align="center"| 50
|}

=== Grades range ===

{|
|+ Course grading range
!
!
!align="center"| '''Proposed range'''
|-
| A. Excellent
| 90-100
|align="center"| 85-100
|-
| B. Good
| 75-89
|align="center"| 70-84
|-
| C. Satisfactory
| 60-74
|align="center"| 50-69
|-
| D. Poor
| 0-59
|align="center"| 0-49
|}

=== Resources and reference material ===

''' Textbook '''

* Durrett Rick. (2019) Probability. Theory and Examples,
* Suhov Y, Kelbert M (2005) Probability and Statistics by Example, Cambridge University Press

== Course Sections ==

The main sections of the course and approximate hour distribution between them is as follows:

{|
|+ Course Sections
!align="center"| '''Section'''
! '''Section Title'''
!align="center"| '''Teaching Hours'''
|-
|align="center"| 1
| Basics of Probability
|align="center"| 12
|-
|align="center"| 2
| Univariate Distributions
|align="center"| 24
|-
|align="center"| 3
| Multivariate distributions
|align="center"| 24
|-
|align="center"| 4
| Limit theorems &amp; Introduction into Mathematical Statistics
|align="center"| 30
|-
|align="center"| hline
|
|align="center"|
|}

=== Section 1 ===

==== Section title: ====

Basics of Probability

=== Topics covered in this section: ===

* Probability space. <math display="inline">\sigma</math>-algebra of events. Axiomatic definition of probability
* Classical model of probability
* Independence of events
* Conditional probability
* Probability of a sum of events (of a product of events)
* Law of total probability. Bayes theorem.

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

<span>|a|c|</span>

&amp; '''Yes/No'''<br />


Development of individual parts of software product code &amp; 0<br />
Homework and group projects &amp; 1<br />
Midterm evaluation &amp; 1<br />
Testing (written or computer based) &amp; 1<br />
Reports &amp; 0<br />
Essays &amp; 0<br />
Oral polls &amp; 0<br />
Discussions &amp; 0<br />



</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Give an example of three events that are pairwise independent, but not mutually independent.
# Can disjoint events be independent?
# A six-sided die is rolled. Construct an algebra of events for this random experiment.
# Derive the formula for calculating the probability of a sum <math display="inline">P(A+B+C)</math>.

=== Typical questions for seminar classes (labs) within this section ===

# Each game of a match between two equal players can end with a victory of one of them with probability <math display="inline">0{.}5</math> independently of the other games. Each victory yields one point, and the match is played until one of the players scores 6 points. Due to technical reasons the match was interrupted when the score was <math display="inline">5:3</math> in favour of the first player. What do you think is a fair way to distribute the prize between the players?
# Seventy numbers are chosen at random from integers <math display="inline">1,2,3,\cdot,100</math>. What is the probability that the largest number chosen is <math display="inline">98</math>?
# A hospital specialises in curing three types of diseases: <math display="inline">A</math>, <math display="inline">B</math> and <math display="inline">C</math>. On average, there are <math display="inline">50\%</math> of patients who suffer from disease <math display="inline">A</math>, <math display="inline">30\%</math> of patients with disease <math display="inline">B</math>, and <math display="inline">20\%</math> of patients with disease <math display="inline">C</math> (each of the patients has exactly one of these diseases). The probabilities to fully recover from the diseases are equal to <math display="inline">0{.}95</math>, <math display="inline">0{.}9</math> and <math display="inline">0{.}85</math> respectively. A patient who came to the hospital recovered completely. What is the probability that he had disease <math display="inline">B</math>?
# A white ball is added into an urn that initially contained <math display="inline">n</math> balls. It is known that the probabilities of having <math display="inline">0,1,2,\ldots n</math> white balls (at the start) in the urn are equal to each other. (a) One ball is taken at random from the urn. What is the probability that the ball is white? (b) The ball taken from the urn has turned out to be white. Find the most probable number of white balls that were in the urn from the start.

=== Test questions for final assessment in this section ===

# The probabilities for three students to pass the exam are equal to <math display="inline">\frac{11}{12}</math>, <math display="inline">\frac{11}{14}</math> and <math display="inline">\frac{18}{25}</math> respectively. Determine the probability that at least one student passes the exam given that they pass or fail independently of each other.
# One of 10-digit numbers in which digits go in non-increasing order is chosen at random. Find the probability that exactly 4 different digits are used in this number.
# Two persons play a game. They take turns in rolling a 10-sided fair die. The first one wins as soon as he rolls 9 or 10, whereas the second one wins as soon as he gets no more than 4. (The game goes on until one of the player’s winning conditions is met). Determine the probability for the first player to win the game.
# Two dice are rolled simultaneously. What is the probability that the sum is even given that it is a multiple of 3?
# There are 5 white balls and 7 green balls in the first urn; 2 white balls and 10 green balls in the second urn. The third urn, that has initially been empty, is filled with the balls: 4 balls are taken from the first urn, 6 balls are taken from the second urn, and they are placed into the third urn. After that, 2 balls are taken at random from the third urn. It turns out that both these balls are green. Determine the probability that these balls originate from different urns.

=== Section 2 ===

==== Section title: ====

Univariate Distributions

=== Topics covered in this section: ===

* Z-test
* Bernoulli trials and their generalisations
* Discrete random variables and their properties.
* Continuous random variables and their properties

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

<span>|a|c|</span>

&amp; '''Yes/No'''<br />


Development of individual parts of software product code &amp; 0<br />
Homework and group projects &amp; 1<br />
Midterm evaluation &amp; 1<br />
Testing (written or computer based) &amp; 1<br />
Reports &amp; 0<br />
Essays &amp; 0<br />
Oral polls &amp; 0<br />
Discussions &amp; 0<br />



</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Bernoulli trials
# Bernoulli trials. The most probable quantity of successes
# Bernoulli shift
# Random variables: general definition. Cumulative distribution function
# Discrete random variables. Expected value and variance
# Uniform distribution on a finite set
# Binomial distribution
# Indicator random variables
# Geometric distribution
# Poisson distribution
# Probability generating function
# Continuous random variables. Probability density function. Expected value and variance
# Uniform distribution on a limited interval
# Exponential distribution
# Normal distribution
# Functional dependence of random variables

=== Typical questions for seminar classes (labs) within this section ===

# On average <math display="inline">25\%</math> students subscribe to the newsletter. Determine the most probable number of subscribers out of (a) 100 students; (b) 103 students.
# Two players are playing a match (that consists of several games), each of the games can finish in favour of the younger player with probability <math display="inline">0{.}6</math> and in favour of the older player with probability <math display="inline">0{.}4</math> The younger player has won exactly five games in the first eight games. What is the probability that he started the match with a defeat?
# Find the range of variance for random variable <math display="inline">\eta</math> if its cumulative distribution function is given by <math display="inline">F_{\eta}(x)=\begin{cases}0,&x\leq0,\\0{.}3,&0<x\leq2,\\b,&2<x\leq6,\\1,&x>6,\end{cases}</math> if <math display="inline">b</math> is a parameter that belongs to <math display="inline">(0{.}3;1)</math>.
# Six people entered the lift at the ground floor of a nine-storied house. Find the expected value for (a) the number of stops where exactly one person gets off the lift; (b) the number of stops where exactly two persons leave the lift.
# Find the expected value and variance of <math display="inline">a^\xi</math> given that <math display="inline">\xi\sim Bin(n,p)</math>.
# Random variable <math display="inline">Y</math> has a uniform distribution on interval <math display="inline">(a;b)</math>, and <math display="inline">E Y=\\Var Y=3</math>. Find <math display="inline">a</math> and <math display="inline">b</math>.

=== Test questions for final assessment in this section ===

# Is it possible for random variable <math display="inline">X</math> to have a binomial distribution if (a) <math display="inline">E X=6</math> and <math display="inline">Var X=3</math>; (b) <math display="inline">E X=7</math> and <math display="inline">Var X=4</math>?
# Let <math display="inline">Y</math> be number of sixes and <math display="inline">Z</math> be number of fours one gets when rolling six dice. Find the expected value and variance of <math display="inline">Y+Z</math>.
# Let <math display="inline">Z</math> be a random variable with geometric distribution. Prove that <math display="inline">P(Z=n+k|z>n)=P(Z=k)</math> (''lack of memory property of geometric distribution'').
# Let us consider a sphere of radius <math display="inline">R</math> centered at <math display="inline">O</math>. Point <math display="inline">M</math> is chosen at random inside this circle. Random variable <math display="inline">\xi</math> is equal to the length of <math display="inline">OM</math>. Find the cumulative distribution function, probability density, expected value and variance of <math display="inline">\xi</math>.
# Random variable <math display="inline">\theta</math> is exponentially distributed with parameter <math display="inline">\lambda</math>. Calculate the probabilities that <math display="inline">\theta</math> belongs to intervals <math display="inline">(0;1), (1;2), \ldots, (n-1;n), \ldots</math> and show that these probabilities form a geometric sequence. What is the common ratio of this sequence?
# It is known that <math display="inline">\xi</math> is normally distributed random variable, and <math display="inline">P\{|\xi-E\xi|<1\}=0{.}3</math>. Find the probability that <math display="inline">|\xi-E\xi|<2</math>.

=== Section 3 ===

==== Section title: ====

Multivariate Distributions

==== Topics covered in this section: ====

* Discrete multivariate distributions
* Two variate continuous distributions
* Multivariate continuous distributions
* Multivariate normal distribution

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

<span>|a|c|</span>

&amp; '''Yes/No'''<br />


Development of individual parts of software product code &amp; 0<br />
Homework and group projects &amp; 1<br />
Midterm evaluation &amp; 0<br />
Testing (written or computer based) &amp; 1<br />
Reports &amp; 0<br />
Essays &amp; 0<br />
Oral polls &amp; 0<br />
Discussions &amp; 0<br />



</div>
=== Typical questions for ongoing performance evaluation within this section ===

<ol>
<li><p>The joint distribution of <math display="inline">\xi</math> and <math display="inline">\eta</math> is provided in the table below.</p>
<div class="center">

{|
!align="center"| <math display="inline">\eta\backslash \xi</math>
!align="center"| <math display="inline">-1</math>
!align="center"| 0
!align="center"| 1
|-
|align="center"| <math display="inline">-2</math>
|align="center"| <math display="inline">\frac{\mathstrut3}{\mathstrut17}</math>
|align="center"| <math display="inline">\frac4{17}</math>
|align="center"| <math display="inline">\frac1{17}</math>
|-
|align="center"| 2
|align="center"| <math display="inline">\frac{\mathstrut1}{\mathstrut17}</math>
|align="center"| <math display="inline">\frac5{17}</math>
|align="center"| <math display="inline">\frac3{17}</math>
|}


</div>
<p>(a) Find marginal distributions of <math display="inline">\xi</math> and <math display="inline">\eta</math>; (b) find expected value and variance for <math display="inline">\xi</math> and <math display="inline">\eta</math>; (c) determine if <math display="inline">\xi</math> and <math display="inline">\eta</math> are independent; (d) find correlation coefficient of <math display="inline">\xi</math> and <math display="inline">\eta</math>; (d) find conditional expected values <math display="inline">E(\xi|\eta)</math> and <math display="inline">E(\eta|\xi)</math>.</p></li>
<li><p>A fair coin is flipped thrice, and for every tails obtained we write a plus; for every heads obtained we write a minus. Random variable <math display="inline">\xi</math> is equal to the quantity of tails, and random variable <math display="inline">\eta</math> is equal to the quantity of sign changes in the sequence. (a) Find marginal distributions of <math display="inline">\xi</math> and <math display="inline">\eta</math>; (b) find expected value and variance for <math display="inline">\xi</math> and <math display="inline">\eta</math>; (c) determine if <math display="inline">\xi</math> and <math display="inline">\eta</math> are independent; (d) find correlation coefficient of <math display="inline">\xi</math> and <math display="inline">\eta</math>; (d) find conditional expected values <math display="inline">E(\xi|\eta)</math> and <math display="inline">E(\eta|\xi)</math>.</p></li>
<li><p><math display="inline">n</math> letters have been written and <math display="inline">n</math> envelopes have been inscribed for these letters. An absent minded secretary places the letters into envelopes at random and sends them with the evening post. (a) What is the probability that at least one letter reaches its destination? (b) Find the average quantity of letter that reach their destination.</p></li></ol>

==== Typical questions for seminar classes (labs) within this section ====

# How many times does one have to flip a coin to get the results “heads”, “heads” in succession? Is the result going to change if we replace the sequence with “tails”, “heads”?
# Forty three equally strong sportsmen take part in a ski race; 18 of them belong to club <math display="inline">A</math>, 10 to club <math display="inline">B</math> and 15 to club <math display="inline">C</math>. What is the average place for (a) the best participant from club <math display="inline">B</math>; (b) the worst participant from club <math display="inline">B</math>?
# How much rolls does one need on average to get a sequence “6”, “6” when rolling a symmetric six-sided die? And if we change this sequence to “6”, “6”, “6”?
# <math display="inline">\zeta</math> is the quantity of threes and <math display="inline">\eta</math> is the quantity of odd digits obtained when rolling a fair die <math display="inline">K</math> times. Find correlation coefficient between <math display="inline">\eta</math> and <math display="inline">\zeta</math>.

=== Test questions for final assessment in the course ===

<ol>
<li><p>A fair die is rolled until a four is obtained. Find the expected value of a sum obtained in all the rolls.</p></li>
<li><p>Find a correlation coefficient between the quantity of sixes and the quantity of fives obtained in <math display="inline">K</math> rolls of a fair die.</p></li>
<li><p>Expected value <math display="inline">\mu</math> and covariance matrix</p>
<p><math display="inline">\mathcal{K}</math> of random vector <math display="inline">\xi=(\xi_1, \xi_2, \xi_3)^T</math> are given. Calculate the expected value and variance of <math display="inline">\eta=\xi_1-\xi_3</math>.</p></li>
<li><p><math display="inline">N</math> letters have been written and envelopes have been inscribed for</p>
<p>these letters. An absent-minded secretary puts the letters into envelopes at random</p>
<p>and sends them with the evening post. Find <math display="inline">E\xi</math> and <math display="inline">Var \xi</math>, where <math display="inline">\xi</math> is the number of</p>
<p>letters that reached their destination.</p></li></ol>

=== Section 1 ===

==== Section title: ====

Limit Theorems and Introduction into Mathematical Statistics

=== Topics covered in this section: ===

* Chebyshev’s inequality. Law of large numbers
* Central limit theorem
* Estimating unknown distribution parameters
* Simple linear regression

=== What forms of evaluation were used to test students’ performance in this section? ===

<div class="tabular">

<span>|a|c|</span>

&amp; '''Yes/No'''<br />


Development of individual parts of software product code &amp; 0<br />
Homework and group projects &amp; 1<br />
Midterm evaluation &amp; 0<br />
Testing (written or computer based) &amp; 1<br />
Reports &amp; 0<br />
Essays &amp; 0<br />
Oral polls &amp; 0<br />
Discussions &amp; 0<br />



</div>
=== Typical questions for ongoing performance evaluation within this section ===

# Is it possible that for some random variable an equality is reached in Chebyshev’s inequality?
# Give an example of a sequence of random variables that satisfies the law of large numbers, and another sequence that does not satisfy it.
# What is a maximum likelihood estimator? Calculate maximum likelihood estimators for parameters of a normally distributed sample.
# Are least square estimators of parameters of a linear regression unbiased? Are they consistent?

=== Typical questions for seminar classes (labs) within this section ===

# Let <math display="inline">\xi_n\sim\begin{pmatrix}-\sqrt n & 0 & \sqrt n\\ 1/2n & 1-1/n & 1/2n\end{pmatrix}</math>. Does this sequence comply the law of large numbers?
# Let <math display="inline">\xi_n\sim\begin{pmatrix}- n & 0 &  n\\ 2^{-n} & 1-2^{-n+1} & 2^{-n}\end{pmatrix}</math>. Does this sequence comply the law of large numbers?
# Find a maximum likelihood estimator for a parameter of a sample with Poisson distribution.
# Prove that sample variance is a biased estimator of variance for a sample of independent identically distributed random variables. Show that Bessel’s correction makes it an unbiased estimator.
# Provide an example of an unbiased estimator that does not have the least mean square error possible.

=== Test questions for final assessment in this section ===

# The probabilities for three students to pass the exam are equal to <math display="inline">\frac{11}{12}</math>, <math display="inline">\frac{11}{14}</math> and <math display="inline">\frac{18}{25}</math> respectively. Determine the probability that at least one student passes the exam given that they pass or fail independently of each other.
# Let us consider independent identically distributed random variables with uniform distribution on <math display="inline">(\theta;\theta+3)</math>. Find the maximum likelihood estimator of <math display="inline">\theta</math>. Which one of these estimators is unbiased? Justify your answer.
# Find the smallest possible value of <math display="inline">P\bigl(|\xi-E\xi|\leq3\sqrt{Var \xi}\bigr)</math>.
# The probability that a new-born baby is a boy is equal to <math display="inline">0{.}52</math>. Find the interval which contains the quantity of boys out of <math display="inline">10\,000</math> newborn babies with probability <math display="inline">0{.}98</math>.
# Prove that for multivariate normal distribution uncorrelatedness implies independence.
# Use characteristic functions to show that a sum of independent (and not necessarily identically distributed) random variables also has normal distribution.