STAT 331, University of Waterloo, Fall 2026

Applied Linear Models

Notes for each lecture. They follow the slides in order and show each derivation step.

A student wrote these notes. They are not official. Compare them with the slides.

Lectures

1. 1

   Introduction and Motivation

   This lecture defines the outcome and the covariate, reviews summaries and distributions for regression, and gives the course plan (Lecture 1 · p.1–38).

   • 01 Nomenclature and data example
   • 02 Univariate summaries: mean and variance
   • 03 Bivariate summaries: covariance and correlation
   • 04 Toward linear regression: simple and multiple models
   • 05 Review: normal, chi-square, t, and F distributions
   • 06 Course overview and logistics
2. 2

   Simple Linear Regression: Estimation

   This lecture estimates \(\beta_0\), \(\beta_1\), and \(\sigma^2\) in the simple linear regression model with least squares and maximum likelihood (Lecture 2 · p.1–38).

   • 01 Recap: mean, variance, covariance, and correlation
   • 02 The simple linear regression model and its coefficients
   • 03 Least squares estimation
   • 04 Maximum likelihood estimation and its equality to least squares
   • 05 Fitted values and residuals
   • 06 Estimating \(\sigma^2\): the MLE and the unbiased estimator
3. 3

   Simple Linear Regression: Inference

   This lecture finds the distributions of \(\hat\beta_1\) and \(\hat\beta_0\) and uses them for confidence intervals and hypothesis tests (Lecture 3 · p.1–49).

   • 01 Recap: the SLR model, least squares, and the estimate of \(\sigma^2\)
   • 02 Sampling distribution and unbiasedness of \(\hat\beta_1\)
   • 03 Variance and distribution of \(\hat\beta_1\) and \(\hat\beta_0\)
   • 04 Confidence interval for \(\beta_1\): known \(\sigma\), unknown \(\sigma\) (t distribution), and standard error
   • 05 Interval estimation example and interpretation
   • 06 Hypothesis tests for \(\beta_1\): p-value, coefficient table, and one-sided tests
4. 4

   Simple Linear Regression: Prediction

   This lecture estimates the mean response at a given \(x_0\), gives a confidence interval for it, and gives a prediction interval for a new response (Lecture 4 · p.1–43).

   • 01 Recap: the model, the estimators, and inference for \(\beta_1\)
   • 02 Point estimate of the mean response and its distribution
   • 03 Confidence interval for the mean response
   • 04 Predicting a new response: the prediction interval
   • 05 Prediction intervals over a range of \(x\) values
   • 06 Practice questions Q1 and Q2
5. 5

   Random Vectors and Multivariate Normal

   This lecture writes regression in matrix form, reviews matrix rules, and defines random vectors and the multivariate normal distribution (Lecture 5 · p.1–48).

   • 01 Recap and multiple linear regression in matrix form
   • 02 Matrix review: transpose, inverse, trace
   • 03 Matrix calculus
   • 04 Random vectors: mean vector, covariance matrix, and linear transformations
   • 05 Multivariate normal distribution: definition and properties
   • 06 Exercises on the MVN, MLR as an MVN model, and practice
6. 6

   Multiple Linear Regression: Intro and Estimation

   This lecture writes multiple linear regression in matrix form, estimates \(\boldsymbol\beta\) and \(\sigma^2\), and finds the properties of the estimators (Lecture 6 · p.1–41).

   • 01 Recap: matrix calculus, random vectors, and the multivariate normal
   • 02 Data example and interpreting regression coefficients
   • 03 Estimating β: least squares and maximum likelihood
   • 04 Properties of the LS estimator, fitted values, and residuals
   • 05 Estimating σ², calculating the estimates, and practice
7. 7

   Multiple Linear Regression: Estimation and Inference I

   This lecture finds the distributions of \(\hat{\boldsymbol\beta}\), \(\boldsymbol e\), and \(\hat\sigma^2\) and uses them for t tests and confidence intervals for each \(\beta_j\) (Lecture 7 · p.1–55).

   • 01 Recap, fitted values, residuals, and the hat matrix
   • 02 Estimating \(\sigma^2\) and the joint distribution of \(\hat{\boldsymbol\beta}\) and \(\boldsymbol e\)
   • 03 The t statistic in MLR and property (3): independence
   • 04 Property (2): \(\frac{1}{\sigma^2}\boldsymbol e^T\boldsymbol e\sim\chi^2_{n-(p+1)}\)
   • 05 t tests and confidence intervals for each \(\beta_j\)
   • 06 Practice questions Q1–Q3

Use these notes with an AI tool