THE CONCEPT OF PROBABILITY. Historical sketch. Classical, Frequentist, and Axiomatic definitions.

UNIFORM PROBABILITY MODELS. Principles of Combinatorial Analysis.

CONDITIONAL PROBABILITY AND INDEPENDENCE. The Bayes rule.

SEQUENCES OF INDEPENDENT TRIALS. The Binomial and Multinomial distributions. Limit Theorems of deMoivre-Laplace and Poisson. The Poisson distribution. The normal distribution.

RANDOM VARIABLES AND DISTRIBUTION FUNCTIONS. Definitions. Properties. Examples of discrete and continuous distributions. Multivariate distributions. Independence of random variables.

TRANSFORMATIONS OF RANDOM VARIABLES.

NUMERICAL CHARACTERISTICS OF RANDOM VARIABLES. Expectation. Median. Variance and covariance. The Markov and Chebyshev inequalities.

LIMIT THEOREMS OF PROBABILITY. The Law of Large Numbers. Moment-generating functions. The Central Limit Theorem. Applications. Cramer's Theorem.

CONDITIONAL DISTRIBUTIONS. The Law of Total Probability. Applications: Simple random Walk, the gambler's ruin problem.

Required Text:

        S. ROSS      "A First Course in Probability". 5^th Edition, Prentice-Hall (1998).

Recommended Texts:

        W. FELLER      "An Introduction to Probability Theory and Its Applications".   Third Edition, J. Wiley & Sons (1967).
        B. GNEDENKO      "The Theory of Probability".   4^th Printing, Mir Publishers, Moscow (1978).

Class Schedule:

• Lecture #1:Tuesday, 19 January
  The notion of Probability. Historical Background. Examples: Chevalier de Mere.
  Algebra of Events. Review of set-theoretic notions. Postulates and elementary consequences.

   

• Lecture #2:Thursday, 21 January
  Uniform Probability Models. Principles of Combinatorial Analysis: orderings, permutations, combinations.
  Examples of combinatorial problems. Relevance and use of Stirling's formula.
  The Hypergeometric distribution; examples.
  Assignment #1:
  
  
  Read Chapter 1 (optional), Chapter 2, pp. 25-35 (required).
  From Chapter 2 (sections 1,2) do
  Problems (Pbs.) # 1- 4, 8, 9.
  Theoretical Exercises (T.E.s) # 2- 4, 6, 7.
  Due Tue. 26 January.

   

• Lecture #3: Tuesday, 26 January

  Conditional probability. Independence. Law of Total Probability. Bayes's Theorem.
  Examples: the Binomial distribution.

   

• Lecture #4: Thursday, 28 January

  The Binomial Distribution and Theorem. Pascal's triangle. Law of repeated conditioning.
  Application to the "birthday problem".
  Simple Random Walk; the Gambler's ruin problem.
  Assignment #2:
  
  
  Read Chapter 2, sections 3-6.
  Problems (Pbs.) # 13(a,b), 15(c,d), 18, 28, 29, 51.
  Theoretical Exercises (T.E.s) # 11-14.
  [#1] What is the probability that: (a) The birthdays of 12 people will fall in 12 different months? (Assume equal probabilities for the different months.) (b) The birthdays of 6 people will fall in exactly 2 months?
  [#2] Player A throws 6 dice and wins, if he scores at least one ace. Player B throws 12 dice and wins, if he scores at least 2 aces. Who has the greater probability of winning?
  Due Tue, 3 February.

   

• Lecture #5: Tuesday, 2 February.
  Example of a continuous model: the exponential distribution. Conditional Independence. Example: Laplace's "law of succession". Probability of the union of n events. Example: matching problem.

   

• Lecture #6: Thursday, 4 February.
  The Normal approximation to the Binomial distribution. Local and integral limit theorems of deMoivre and Laplace.
  Assignment #3:
  
  
  Read Chapter 3.
  Problems (Pbs.) # 18, 24, 32, 62, 72, 81.
  Theoretical Exercises (T.E.s) # 6, 10, 13, 14.
  [#1] The failure-rate of a machine is given by m(t):=1+(t/(1+t)), t>0 where time is measured in years. What is the probability that the machine will live for at least two years ?
  Due Tue, 9 February.

   

• Lecture #7: Tuesday, 9 February.
  The weak law of large numbers for the binomial distribution. The Poisson approximation to the Binomial; juxtapositioin; examples.

   

• Lecture #8: Thursday, 11 February.
  Probability spaces, axiomatic foundations. Axiom of continuity.
  Random Variables and their distribution functions. Properties. Examples.
  Assignment #4:
  
  
  Read Chapter 5, pp. 212-222.
  Problems (Pbs.) # 22, 23, 25-27, 30, 33, 34 (pp. 235-236).
  Due Thu, 18 February.

   

• Lecture #9: Tuesday, 16 February.
  Continuous and discrete distributions. Exmaples. Numerical characteristics of Random Variables.
  Expectation. The expectation of a positive random variable as the integral of the function 1-F(.) over the positive half-line.

   

• Lecture #10: Thursday, 18 February.
  The Markov inequality. Expectation of a function of a Random Variable: Moments, central-moments, Variance.
  Properties of Expectation. The Jensen inequality.
  Assignment #5:
  
  
  Read Chapters 4 and 5 in their entirety.
  Chapter 4: Pbs. # 17, 19, 58. T.E.s # 6, 9, 15, 17, 25.
  Chapter 5: Pbs. # 2, 5. T.E.s # 2, 5.
  Due Thu, 25 February.

   

• Lecture #11: Tuesday, 23 February.
  Examples of absolutely continuous distributions, and of distributions of the mixed type.
  Interpretation of the variance: the Markov and Chebyshev inequalities.
  Computation of moments and variances for various distributions.

   

• Lecture #12: Thursday, 25 February.
  Joint distributions of pairs of random variables. Properties of bivariate distribution functions.
  Examples of absolutely continuous and discrete bivariate distributions. Marginals.
  Notion of Independence. Examples.
  Assignment #6:
  
  
  Read Chapter 6 in its entirety.
  Pbs. # 8, 13, 20, 41, 42.
  T.E.s # 2, 5, 14, 15, 22, 23.
  Due Thu, 12 March.

   

• Lecture #13: Thursday, 2 March.
  Review session: Examples on the Poisson approximation to the Binomial distribution.
  Inclusion-exclusion formula for the probability of the union of events. Application to the matching problem.
  Distribution of functions of several random variables, such as the sum, product, and ratio.

   

• Lecture #14: Thursday, 4 March.
  MID-TERM EXAMINATION.

   

• Lecture #15: Tuesday, 9 March.
  Solution of the Mid-term examination.
  Independent random variables and their sums; convolution.
  Examples: Poisson, normal, exponential and the Gamma distribution.

   

• Lecture #16: Thursday, 11 March.
  Expectation of a function of several random variables. Additivity of expectation. Independence. Correlation. Examples of uncorrelated but dependent random variables. Variance of the sum.
  Assignment #7:
  
  
  Read Chapter 7.
  Pbs. # 9, 26, 28, 29, 36.
  T.E.s # 1, 2, 7, 8, 24.
  Due Thu, 25 March.

   

• Lecture #17: Tuesday, 23 March.
  The multinomial coefficients and the multinomial distribution. Computation of variances and covariances for binomial and multinomial distributions. The Cauchy-Schwarz inequality. The correlation coefficient and its properties. Distribution of the maximum and minimum of I.I.D. random variables.

   

• Lecture #18: Thursday, 25 March.
  Study of the multivariate normal distribution; equivalence of "independence" and "uncorrelatedness" in its context. Conditional distributions and expectations. The Law of Total Probability.
  Assignment #8:
  
  
  Read Chapter 7, pp. 335-354.
  Pbs. # 45, 46, 52, 66, 67.
  T.E.s # 38 - 40, 42, 44.
  Due Thu, 9 April.

   

• Lecture #19: Tuesday, 30 March.

  Examples of conditional expectations and distributions. Application to random sums: means, variances.

   

• Lecture #20: Thursday, 1 April.
  Modes of convergence for sequences of random variables: relation of "convergence in probability" to "convergence with probability one". The general formulation of the Weak and Strong Laws of Large Numbers, and of the Central Limit Theorem.

   

• Lecture #21: Tuesday, 6 April.
  The lemma of Borel and Cantelli. Proof of the Markov Strong Law of large Numbers.

   

• Lecture #22: Thursday, 8 April.
  Examples of convergence with probability one, in probability, and in distribution. Moment-generating functions and their properties.
  Assignment #9:
  
  
  Read Chapter 8 in its entirety.
  Pbs. # 2, 3, 4, 10, 13, 18.
  T.E.s # 2, 5.
  Due Thu, 16 April.

   

• Lecture #23: Tuesday, 13 April.
  Proof of the Central Limit Theorem using moment-generating functions.

   

• Lecture #24: Thursday, 15 April.
  Applications of the Central Limit Theorem and of the Law of Large Numbers.
  Confidence Intervals. Weirstrass approximation theorem. Inversion formula for Laplace transforms.

   

• Lecture #25: Tuesday, 20 April.
  Strong one-sided Chebyshev inequalities. Chernoff bounds. Examples. Large fluctuations in the Weak Law of Large Numbers. Cramer's theorem; examples.

   

• Lecture #26: Thursday, 22 April.
  Conditional expectations: some additional examples. Conditional expectation as a random variable.
  Conditional expectation as least-squares predictor. Best linear least-squares prediction.

   

• Lecture #27: Tuesday, 27 April.
  Elementary approach to the simple random walk: Gambler's ruin problem, absorption probabilities, expected duration of games. Recurrence, transience.

   

• Lecture #28: Thursday, 29 April.
  Combinatorial approach to the simple random walk. Distribution of the first-return-time to the origin.