STAT 331 · Lecture 1 · Introduction and Motivation

# Lecture 1: Introduction and Motivation

This lecture defines the outcome and the covariate, reviews summaries and distributions for regression, and gives the course plan (Lecture 1 · p.1–38).

Contents
01 · Nomenclature and data example 02 · Univariate summaries: mean and variance 03 · Bivariate summaries: covariance and correlation 04 · Toward linear regression: simple and multiple models 05 · Review: normal, chi-square, t, and F distributions 06 · Course overview and logistics

## 01 · Nomenclature and data example

Plan · Slides p.1–6

p.1 title · p.2 outcome \(y\) · p.3 covariate \(x\) · p.4 brainhead data · p.5 variables and first rows · p.6 scatterplot.

### 01.1 Title page Slides p.1

Course: STAT 331: Applied Linear Models.

Lecture: Lecture 1: Intro & Motivation. Instructor: Glen McGee, Fall 2026.

### 01.2 The outcome \(y\) Slides p.2

**What** Slides p.2

Some Nomenclature · Lecture 1 · p.2 Outcome: \(y\)

- Main variable of interest
- Variable whose mean we want to explain in terms of other variables
- AKA: response, output, dependent variable

A variable is a quantity that you record for each subject, for example brain weight.

The mean is the average value.

We explain the mean of \(y\), not the exact \(y\) of each subject.

"AKA" means "also known as". The four names have one meaning.

**How** Added

Find \(y\) in a data set

- Read the research question.
- Find the quantity that the researcher wants to know most.
- Call its data column \(y\).

**Self-check:** "We explain the mean of ___ with other variables" must be correct. One analysis has one \(y\).

**Example** Added

1

Main quantity: brain weight

Slides p.4

2

Data column: brain.wgt (grams)

Slides p.5

3

\(y=\) brain weight

**Why** Added

1

The research question has one target quantity

2

The model gives how its mean changes with other quantities

Slides p.2, item 2

3

The target goes on the left side, as \(y\)

\(y_i=\beta_0+\beta_1x_i+\epsilon_i\), Slides p.21

### 01.3 The covariate \(x\) Slides p.3

**What** Slides p.3

Some Nomenclature · Lecture 1 · p.3 Covariate: \(x\)

- Other variables of interest
- Variables that explain/predict the outcome in some sense
- AKA: predictor, input, independent variable, feature

To predict is to estimate an unmeasured \(y\) from a known \(x\).

"In some sense": \(x\) and \(y\) have a relation. \(x\) does not necessarily cause \(y\).

The five names have one meaning.

One analysis can have more than one covariate.

Simple linear regression has one covariate. Multiple linear regression has more than one (Slides p.21–24).

**How** Added

Find \(x\) in a data set

- Find \(y\) (Section 01.2).
- Find the other columns that explain or predict \(y\).
- Call this column (or these columns) \(x\).

**Self-check:** "We use ___ to explain or predict \(y\)" must be correct. \(x\) is not the \(y\) column.

**Example** Added

Slides p.4: "Could we predict it based on size of head?" "It" is brain weight.

1

Predictor of brain weight: head size

Slides p.4

2

Data column: head.size (cm³)

Slides p.5

3

\(x=\) head size

Pages 5–6 do not use the columns gender and age.

**Why** Added

1

\(y\) is difficult to measure

You cannot weigh a live brain

2

\(x\) is easy to measure

Measure the head from outside

3

Use the known \(x\) to give the mean of \(y\)

Slides p.2–3

### 01.4 Data example: Gladstone (1905) Slides p.4

**What** Slides p.4

Data Example: Gladstone (1905) · Lecture 1 · p.4

- “A study of the relations of the brain to the size of the head.”
- Interested in brain weight

- Difficult to measure in live subjects!
- Could we predict it based on size of head?

- Observations on 236 deceased people
- Published in 1905 (Biometrika)

An observation is the set of values recorded for one subject. This data has 236.

Biometrika is a statistics journal.

### 01.5 Variables and first rows Slides p.5

**What** Slides p.5

The file brainhead.csv has one row per observation and one column per variable.

Slides p.5 sets \(x\) = head.size and \(y\) = brain.wgt.

It plots \(y\) against \(x\). Axis labels: "Head Size (cm^3)" (horizontal), "Brain Weight (grams)" (vertical).

The subscript \(i\) is the observation number. \(x_i\) and \(y_i\) belong to person \(i\).

Slides p.23

| \(i\) | gender | age | head.size \(x_i\) (cm³) | brain.wgt \(y_i\) (g)

| 1 | M | 20-46 | 4512 | 1530

| 2 | M | 20-46 | 3738 | 1297

| 3 | M | 20-46 | 4261 | 1335

| 4 | M | 20-46 | 3777 | 1282

| 5 | M | 20-46 | 4177 | 1590

| 6 | M | 20-46 | 3585 | 1300

gender: M or F. age: age group, 20-46 or 46+.

Slides p.23 Min. and Max. (full summary: Section 04.3):

head.size: Min. 2773 cm³, Max. 4747 cm³.

brain.wgt: Min. 1012 g, Max. 1635 g.

**How** Added

Read \(x_i\) and \(y_i\)

- Find row \(i\).
- Read head.size. This is \(x_i\).
- Read brain.wgt in the same row. This is \(y_i\).

**Self-check:** Same row. \(x_i\) between 2773 and 4747. \(y_i\) between 1012 and 1635.

**Example** Added

1

Row 1: M, 20-46, 4512, 1530

Slides p.23

2

\(x_1=4512\) cm³

3

\(y_1=1530\) g

4

\(2773\le 4512\le 4747\) and \(1012\le 1530\le 1635\)

Slides p.23

**Why** Added

1

Each row is one person

2

\(x_i\) and \(y_i\) of one row belong to one person

3

The pair \((x_i,y_i)\) is one observation of the \(x\)–\(y\) relation

### 01.6 The scatterplot Slides p.6

**What** Slides p.6

A scatterplot shows each observation as one point at \((x_i, y_i)\), \(x\) horizontal, \(y\) vertical.

This plot has 236 points.

**How** Added

Read a scatterplot

- Read the axis labels and units.
- Find which axis is \(x\) and which is \(y\).
- Compare the axis ranges with the data Min. and Max.
- Examine the points from left to right: up, down, or level?

**Self-check:** 236 points. Leftmost point: \(x=2773\). Highest point: \(y=1635\).

**Example** Slides p.6

Orange point: person 1, \((4512, 1530)\).

[figure]
Figure 1-1. Brainhead scatterplot (Slides p.6). One point per deceased person.

Step 1: horizontal axis \(x\), head size (cm³). Vertical axis \(y\), brain weight (grams).

Step 2: the units agree with Slides p.5.

Step 3: \(x\) goes from 2773 to 4747, \(y\) from 1012 to 1635 (Slides p.23).

Step 4: the points go up. A larger head has a larger mean brain weight.

Later units measure this trend with covariance and correlation (Slides p.17–20), then a straight line (Slides p.21–24).

**Why** Added

1

Question: what is the mean brain weight at a given head size?

Slides p.2 and p.4

2

Fix one \(x\) on the horizontal axis

3

Examine the points in a narrow vertical strip at this \(x\)

4

Their mean height is the mean of \(y\) near this \(x\)

5

\(x\) goes horizontal, \(y\) goes vertical

### 01.7 Practice Added

**Q1.** In Gladstone (1905), which variable is the outcome and which is the covariate? Give two other names for each.

Answer

Outcome \(y\): brain weight (brain.wgt). Other names: response, output, dependent variable.

Covariate \(x\): head size (head.size). Other names: predictor, input, independent variable, feature.

**Q2.** From the table (Slides p.23), give \(x_3\), \(y_3\), \(x_5\), and \(y_5\).

Answer

1

Row 3: M, 20-46, 4261, 1335

2

\(x_3=4261\) cm³, \(y_3=1335\) g

3

Row 5: M, 20-46, 4177, 1590

4

\(x_5=4177\) cm³, \(y_5=1590\) g

**Q3.** From Min. and Max. (Slides p.23), calculate the range (Max. minus Min.) of head size and of brain weight.

Answer

1

Range of \(x\) \(=4747-2773\)

2

\(=1974\) cm³

3

Range of \(y\) \(=1635-1012\)

4

\(=623\) g

## 02 · Univariate summaries: mean and variance

Plan · Slides p.7–16

p.7–9 summary list · p.10–11 \(E[Y]\), linearity · p.12 \(\bar y\) · p.13–14 \(\mathrm{Var}[Y]\), two properties · p.15–16 \(s_y^2\) · Added: derivations.

### 02.1 How can we describe the data? Slides p.7–9

**What** Slides p.7–9

How Can we Describe The Data? · Lecture 1 · p.9 You have by now seen a few ways of describing the data:
• Univariate summaries:
• Mean
• Variance/Standard Deviation
• Bivariate summaries:
• Covariance
• Correlation
Let’s review them briefly

A summary is one number that describes one property of a data column.

A univariate summary uses one variable. This unit covers it.

A bivariate summary shows how two variables change together. Unit 03 covers it.

Slides p.7–8 build the same list in parts.

### 02.2 Mean (expectation) and linearity Slides p.10–11

**What** Slides p.10–11

Review: Mean · Lecture 1 · p.10–11 Mean/Expectation: \[E[Y]=\int y f(y)\,dy\] Linearity of expectation: For random variables \(Y_1,\dots,Y_m\) and constants \(a_i,b_i\) for \(i=1,\dots,m\): \[E\left[\sum_{i=1}^m (a_iY_i+b_i)\right]=\sum_{i=1}^m a_iE[Y_i]+\sum_{i=1}^m b_i\]

A random variable \(Y\) is a quantity whose value chance sets. Small \(y\) is one possible value.

The probability density function (PDF) \(f(y)\) tells how likely \(Y\) is near \(y\). \(f(y)\ge 0\) and \(\int f(y)\,dy=1\).

The expectation \(E[Y]\) is the average of \(Y\) over many repeats. The PDF sets it, not data.

Constants \(a_i\), \(b_i\) are fixed, not random.

Linearity: a constant factor moves outside \(E\). \(E\) of a sum is the sum of the \(E\) values. \(E\) of a constant is the constant.

**How** Added

Use linearity to simplify \(E[\cdot]\)

- Write the expression as \(\sum_i (a_iY_i+b_i)\).
- Read each \(a_i\) and \(b_i\).
- Replace each \(a_iY_i\) with \(a_iE[Y_i]\).
- Keep each \(b_i\).
- Add the terms.

**Self-check:** The result contains only \(E[Y_i]\) values and constants.

**Example** Added

\(E[Z]=2\). Calculate \(E[3Z+1]\).

1

\(E[3Z+1]=3E[Z]+1\)

linearity, \(a_1=3,\ b_1=1\)

2

\(=3(2)+1\)

\(E[Z]=2\)

3

\(=6+1\)

4

\(=7\)

### 02.3 Sample mean Slides p.12

**What** Slides p.12

Review: Mean · Lecture 1 · p.12 For observations \(y_1,\dots,y_n\), the sample mean is: \[\bar y=\frac{1}{n}\sum_i^n y_i\]

\(n\) is the sample size, the number of observations.

An estimate is a number from data that replaces an unknown population value. \(\bar y\) estimates \(E[Y]\).

\(\sum_i^n\) adds the terms for \(i=1,\dots,n\). The slides also write \(\sum_i\).

**How** Added

Calculate the sample mean \(\bar y\)

- Count the observations to get \(n\).
- Add the observations.
- Divide the sum by \(n\).

**Self-check:** \(\bar y\) is between the smallest and largest observation, in the unit of \(y_i\).

**Example 1 · First four people** Added

Brain weights (g): 1530, 1297, 1335, 1282.

1

\(\bar y=\frac14(1530+1297+1335+1282)\)

\(n=4\)

2

\(=\frac14(5444)\)

3

\(=1361\) g

Self-check: \(1282\le 1361\le 1530\).

**Example 2 · All 236 people** Added

\(n=236\) (Slides p.4). The sum over the course data file is 303086 g (not on the slides). Min. 1012 g, Max. 1635 g (Slides p.23).

1

\(\bar y=\frac{1}{236}(303086)\)

\(n=236\)

2

\(=1284.263\) g

Self-check: \(1012\le 1284.263\le 1635\). Slides p.23 prints 1284.

**Example 3 · Linearity on data** Added

Grams to kilograms: \(y_i'=0.001\,y_i\).

1

\(\bar y'=0.001\,\bar y\)

linearity, \(a=0.001,\ b=0\)

2

\(=0.001(1284.263)\)

3

\(=1.284263\) kg

### 02.4 Variance and its two properties Slides p.13–14

**What** Slides p.13–14

Review: Variance · Lecture 1 · p.13–14 Variance \[\mathrm{Var}[Y]=E\left[(Y-E[Y])^2\right]=E[Y^2]-E[Y]^2\] Properties:
• \(\mathrm{Var}(aY+b)=a^2\mathrm{Var}(Y)\)
• For independent random variables \(X\) and \(Y\): \(\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{var}(Y)\)

The deviation \(Y-E[Y]\) is the difference between \(Y\) and its mean.

The variance is the expectation of the squared deviation. It is zero or more. More spread gives a larger variance.

The standard deviation is \(\sqrt{\mathrm{Var}[Y]}\). It has the unit of \(Y\). The variance has that unit squared.

Section 02.6 shows that the two forms are equal.

\(X\) and \(Y\) are independent when the value of one does not change the distribution of the other.

\(\mathrm{Var}[Y]\), \(\mathrm{Var}(Y)\), and \(\mathrm{var}(Y)\) are the same quantity.

Property 1: a shift \(b\) does not change the spread. A factor \(a\) multiplies the variance by \(a^2\).

**How** Added

Get the variance of a transformation

- Write the target as \(aY+b\) or \(X+Y\).
- For \(aY+b\): remove \(b\).
- For \(aY+b\): write \(a^2\mathrm{Var}(Y)\).
- For \(X+Y\): make sure that \(X\) and \(Y\) are independent.
- For \(X+Y\): write \(\mathrm{Var}(X)+\mathrm{var}(Y)\).

**Self-check:** The result is zero or more. The sign of \(a\) has no effect.

**Example 1 · \(\mathrm{Var}(aY+b)\)** Added

\(Z\) has standard deviation 0.25. Calculate \(\mathrm{Var}(3Z+1)\).

1

\(\mathrm{Var}(Z)=0.25^2\)

2

\(=0.0625\)

3

\(\mathrm{Var}(3Z+1)=3^2\,\mathrm{Var}(Z)\)

Property 1, \(a=3,\ b=1\)

4

\(=9(0.0625)\)

5

\(=0.5625\)

**Example 2 · \(\mathrm{Var}(X+Y)\)** Added

\(X\), \(Y\) independent, with standard deviations 0.25 and 1. Calculate \(\mathrm{Var}(X+Y)\).

1

\(\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{var}(Y)\)

Property 2

2

\(=0.25^2+1^2\)

3

\(=0.0625+1\)

4

\(=1.0625\)

### 02.5 Sample variance Slides p.15–16

Slide p.16 is the same as p.15.

**What** Slides p.15

Review: Variance · Lecture 1 · p.15 For observations \(y_1,\dots,y_n\), the sample variance is: \[s_y^2=\frac{1}{n-1}\sum_i^n (y_i-\bar y)^2\]

\(s_y^2\) estimates \(\mathrm{Var}[Y]\). The subscript \(y\) names the variable.

The sample standard deviation is \(s_y=\sqrt{s_y^2}\).

The denominator is \(n-1\), not \(n\).

**How** Added

Calculate the sample variance \(s_y^2\)

- Calculate \(\bar y\).
- Calculate each deviation \(y_i-\bar y\).
- Square each deviation.
- Add the squares.
- Divide by \(n-1\) to get \(s_y^2\).
- Take the square root to get \(s_y\).

**Self-check:** The deviations add to 0. \(s_y^2\ge 0\), in the unit of \(y\) squared.

**Example 1 · First four people** Added

\(y_i=1530,1297,1335,1282\), \(\bar y=1361\), \(n=4\). Orange lines in Figure 2-1 are the deviations.

[figure]
Figure 2-1: First four brain weights (blue), \(\bar y=1361\) (green dashed), deviations \(y_i-\bar y\) (orange). Right is positive, left is negative.

1

\(y_i-\bar y:\ 1530-1361,\ 1297-1361,\ 1335-1361,\ 1282-1361\)

2

\(=169,\ -64,\ -26,\ -79\)

3

\(\sum_i (y_i-\bar y)=169-64-26-79=0\)

self-check

4

\(\sum_i (y_i-\bar y)^2=169^2+(-64)^2+(-26)^2+(-79)^2\)

5

\(=28561+4096+676+6241\)

6

\(=39574\)

7

\(s_y^2=\frac{39574}{4-1}\)

\(n-1=3\)

8

\(=13191.33\) g\(^2\)

9

\(s_y=\sqrt{13191.33}=114.8535\) g

**Example 2 · All 236 people** Added

The course data file gives \(\sum_i (y_i-\bar y)^2=3309754\), rounded (not on the slides).

1

\(s_y^2=\frac{1}{236-1}\sum_i (y_i-\bar y)^2\)

\(n=236\)

2

\(=\frac{3309754}{235}\)

3

\(=14084.06\) g\(^2\)

4

\(s_y=\sqrt{14084.06}=118.6763\) g

A histogram cuts the number line into bins of equal width (here 50 g). Each bar height is the count of people in the bin.

[figure]
Figure 2-2: Histogram of 236 brain weights. Green: \(\bar y\). Purple dashed: \(\bar y\pm s_y\), \(s_y=118.6763\) g. 163 people are between them.

**Example 3 · Grams to kilograms: check \(\mathrm{Var}(aY+b)=a^2\mathrm{Var}(Y)\)** Added

1

\(s_{y'}^2=0.001^2\,s_y^2\)

Property 1, \(a=0.001,\ b=0\)

2

\(=0.000001(14084.06)\)

3

\(=0.01408406\) kg\(^2\)

4

\(s_{y'}=\sqrt{0.01408406}=0.1186763\) kg

With \(a=1\), \(b=100\): the mean becomes \(1284.263+100=1384.263\) g. The variance stays \(14084.06\) g\(^2\).

### 02.6 Why: the two variance formulas Added

Let \(\mu=E[Y]\), a constant.

**Derivation 1: \(\mathrm{Var}[Y]=E[Y^2]-E[Y]^2\)**

1

\(\mathrm{Var}[Y]=E\left[(Y-\mu)^2\right]\)

definition

2

\(=E\left[Y^2-2\mu Y+\mu^2\right]\)

3

\(=E[Y^2]-2\mu E[Y]+\mu^2\)

linearity

4

\(=E[Y^2]-2\mu\cdot\mu+\mu^2\)

\(E[Y]=\mu\)

5

\(=E[Y^2]-2\mu^2+\mu^2\)

6

\(=E[Y^2]-\mu^2\)

7

\(=E[Y^2]-E[Y]^2\)

\(\mu=E[Y]\)

**Derivation 2: \(\mathrm{Var}(aY+b)=a^2\mathrm{Var}(Y)\)**

1

\(E[aY+b]=aE[Y]+b\)

linearity

2

\(=a\mu+b\)

3

\(\mathrm{Var}(aY+b)=E\left[\left((aY+b)-E[aY+b]\right)^2\right]\)

definition

4

\(=E\left[\left((aY+b)-(a\mu+b)\right)^2\right]\)

line 2

5

\(=E\left[\left(aY-a\mu\right)^2\right]\)

\(b\) cancels

6

\(=E\left[\left(a(Y-\mu)\right)^2\right]\)

7

\(=E\left[a^2(Y-\mu)^2\right]\)

8

\(=a^2E\left[(Y-\mu)^2\right]\)

linearity

9

\(=a^2\mathrm{Var}(Y)\)

definition

### 02.7 Practice Added

**Q1.** \(E[Y_1]=2\), \(E[Y_2]=5\). Calculate \(E[3Y_1-2Y_2+4]\).

Answer

1

\(E[3Y_1-2Y_2+4]=3E[Y_1]-2E[Y_2]+4\)

linearity

2

\(=3(2)-2(5)+4\)

3

\(=6-10+4\)

4

\(=0\)

**Q2.** \(X\), \(Y\) independent, \(\mathrm{Var}(X)=0.0625\), \(\mathrm{Var}(Y)=1\). Calculate \(\mathrm{Var}(2X+Y+7)\).

Answer

1

\(\mathrm{Var}(2X+Y+7)=\mathrm{Var}(2X+Y)\)

Property 1, \(a=1,\ b=7\)

2

\(=\mathrm{Var}(2X)+\mathrm{var}(Y)\)

Property 2

3

\(=2^2\,\mathrm{Var}(X)+\mathrm{var}(Y)\)

Property 1, \(a=2,\ b=0\)

4

\(=4(0.0625)+1\)

5

\(=0.25+1\)

6

\(=1.25\)

**Q3.** Brain weights (g): 1530, 1297, 1335, 1282, 1590. Calculate \(\bar y\), \(s_y^2\), \(s_y\), and \(s_y^2\) in kg\(^2\).

Answer

1

\(\bar y=\frac15(1530+1297+1335+1282+1590)\)

\(n=5\)

2

\(=\frac{7034}{5}=1406.8\)

3

\(y_i-\bar y:\ 123.2,\ -109.8,\ -71.8,\ -124.8,\ 183.2\)

sum is 0

4

\((y_i-\bar y)^2:\ 15178.24,\ 12056.04,\ 5155.24,\ 15575.04,\ 33562.24\)

5

\(\sum_i (y_i-\bar y)^2=81526.8\)

6

\(s_y^2=\frac{81526.8}{5-1}=20381.7\) g\(^2\)

7

\(s_y=\sqrt{20381.7}=142.7645\) g

8

\(0.001^2\times 20381.7=0.0203817\) kg\(^2\)

Property 1, \(a=0.001\)

## 03 · Bivariate summaries: covariance and correlation

Plan · Slides p.17–20

p.17 covariance · p.18 three properties · p.19 sample covariance · p.20 \(\rho\), \(r\), \(S_{xy},S_{xx},S_{yy}\) · Added: practice.

Examples use the first 5 rows of the brainhead data (\(x\) in cm³, \(y\) in g):

\(x_i=4512,\ 3738,\ 4261,\ 3777,\ 4177\).

\(y_i=1530,\ 1297,\ 1335,\ 1282,\ 1590\).

### 03.1 Covariance Slides p.17

**What** Slides p.17

Review: Covariance · Lecture 1 · p.17 Covariance \[\mathrm{cov}[X,Y]=E[(X-E[X])(Y-E[Y])]=E[XY]-E[X]E[Y]\]

Covariance is the expectation of the product of the two deviations. It shows if \(X\) and \(Y\) are high together and low together.

\(\mathrm{cov}[X,Y]\) and \(\mathrm{cov}(X,Y)\) are the same quantity.

Same-sign deviations give a positive product. Opposite signs give a negative product.

\(\mathrm{cov}[X,Y]>0\): \(Y\) is usually high when \(X\) is high. \(\mathrm{cov}[X,Y]<0\): \(Y\) is usually low when \(X\) is high.

**How** Added

Calculate \(\mathrm{cov}[X,Y]\) with the second form

- Calculate \(E[X]\) and \(E[Y]\).
- Calculate \(E[XY]\), the probability-weighted average of the products \(xy\).
- Calculate \(E[XY]-E[X]E[Y]\).

**Self-check:** The first form gives the same result. The unit is (unit of \(X\))·(unit of \(Y\)).

**Example** Added

Take the 5 people as a population. Select 1 person at random, each with probability \(1/5\). \(X\) is head size, \(Y\) is brain weight.

Second form: How steps 1–3

1

\(E[X]=\frac15(4512+3738+4261+3777+4177)=\frac15(20465)=4093\)

2

\(E[Y]=\frac15(1530+1297+1335+1282+1590)=\frac15(7034)=1406.8\)

3

\(x_iy_i:\ 6903360,\ 4848186,\ 5688435,\ 4842114,\ 6641430\)

4

\(E[XY]=\frac15(6903360+4848186+5688435+4842114+6641430)\)

5

\(E[XY]=\frac15(28923525)=5784705\)

6

\(E[X]E[Y]=4093\times1406.8=5758032.4\)

7

\(\mathrm{cov}[X,Y]=5784705-5758032.4=26672.6\)

First form: How self-check

1

\(x_i-4093:\ 419,\ -355,\ 168,\ -316,\ 84\)

2

\(y_i-1406.8:\ 123.2,\ -109.8,\ -71.8,\ -124.8,\ 183.2\)

3

\((x_i-4093)(y_i-1406.8):\ 51620.8,\ 38979.0,\ -12062.4,\ 39436.8,\ 15388.8\)

4

\(\mathrm{cov}[X,Y]=\frac15(51620.8+38979.0-12062.4+39436.8+15388.8)\)

5

\(=\frac15(133363)\)

6

\(=26672.6\)

Both forms give 26672.6 g·cm³. Positive: a larger head usually has a heavier brain. ▲

**Why** Added

1

\(E[(X-E[X])(Y-E[Y])]\)

2

\(=E\big[XY-XE[Y]-E[X]Y+E[X]E[Y]\big]\)

3

\(=E[XY]-E[Y]E[X]-E[X]E[Y]+E[X]E[Y]\)

linearity; \(E[X],E[Y]\) constant

4

\(=E[XY]-E[X]E[Y]\)

### 03.2 Properties of covariance Slides p.18

**What** Slides p.18

Review: Covariance · Lecture 1 · p.18 Properties:
• \(\mathrm{cov}(X,X)=\mathrm{Var}(X)\)
• \(\mathrm{cov}(aY+c,bX+d)=ab\,\mathrm{cov}(X,Y)\)
• \(\mathrm{cov}(U+V,X+Y)=\mathrm{cov}(U,X)+\mathrm{cov}(U,Y)+\mathrm{cov}(V,X)+\mathrm{cov}(V,Y)\)
• Exercise: what is \(\mathrm{Var}(X+Y)\)?

\(a,b\) are constant factors. \(c,d\) are constant shifts. \(U,V,X,Y\) are any random variables.

Property 2: shifts have no effect. Factors move outside.

Property 3: each left term pairs with each right term, \(2\times2=4\) terms.

Practice Q1 (Section 03.5) answers the slide exercise.

**How** Added

Simplify \(\mathrm{cov}(\text{expression 1},\ \text{expression 2})\)

- Write each expression as a sum of "constant × random variable" terms plus a constant.
- Read each factor and each shift.
- Use property 3 to split the sums into pairs.
- Use property 2 to remove shifts and move factors outside.
- Use property 1 to replace each \(\mathrm{cov}(X,X)\) with \(\mathrm{Var}(X)\).

**Self-check:** Number of pairs = (left terms) × (right terms). The result has only variances, covariances, and constants.

**Example** Added

Same 5-person population: \(E[X]=4093\), \(E[Y]=1406.8\), \(\mathrm{cov}(X,Y)=26672.6\).

Property 1: Property 1

1

\(\mathrm{cov}(X,X)=\frac15(419^2+(-355)^2+168^2+(-316)^2+84^2)\)

deviations \(x_i-4093\)

2

\(=\frac15(175561+126025+28224+99856+7056)\)

3

\(=\frac15(436722)=87344.4\)

4

\(E[X^2]=\frac15(4512^2+3738^2+4261^2+3777^2+4177^2)=\frac15(84199967)=16839993.4\)

5

\(\mathrm{Var}(X)=E[X^2]-E[X]^2=16839993.4-4093^2\)

Slides p.13

6

\(=16839993.4-16752649\)

7

\(=87344.4=\mathrm{cov}(X,X)\)

For \(Y\): \(\mathrm{Var}(Y)=\frac15(15178.24+12056.04+5155.24+15575.04+33562.24)=\frac15(81526.8)=16305.36\).

Property 2: g to kg, cm³ to litres (1 litre = 1000 cm³). Property 2

Basis: Slides p.18 property 2 with \(a=1/1000,\ c=0,\ b=1/1000,\ d=0\).

1

\(\mathrm{cov}\left(\frac{1}{1000}Y+0,\ \frac{1}{1000}X+0\right)=\frac{1}{1000}\cdot\frac{1}{1000}\,\mathrm{cov}(X,Y)\)

2

\(=\frac{1}{1000000}\times26672.6\)

3

\(=0.0266726\)

The covariance is \(10^6\) times smaller. The sign stays.

Property 3: \(\mathrm{cov}(X+Y,\ X-Y)\), with \(X-Y=X+(-1)Y\). How steps 1–5

1

\(\mathrm{cov}(X+Y,\ X+(-1)Y)\)

2

\(=\mathrm{cov}(X,X)+\mathrm{cov}(X,(-1)Y)+\mathrm{cov}(Y,X)+\mathrm{cov}(Y,(-1)Y)\)

property 3

3

\(=\mathrm{cov}(X,X)-\mathrm{cov}(X,Y)+\mathrm{cov}(Y,X)-\mathrm{cov}(Y,Y)\)

property 2, factor \(-1\)

4

\(=\mathrm{cov}(X,X)-\mathrm{cov}(Y,Y)\)

symmetry (see Why)

5

\(=\mathrm{Var}(X)-\mathrm{Var}(Y)\)

property 1

6

\(=87344.4-16305.36\)

7

\(=71039.04\)

**Why** Added

Symmetry: the order of the two variables does not change the covariance.

1

\(\mathrm{cov}(Y,X)=E[(Y-E[Y])(X-E[X])]\)

definition

2

\(=E[(X-E[X])(Y-E[Y])]\)

3

\(=\mathrm{cov}(X,Y)\)

Property 1:

1

\(\mathrm{cov}(X,X)=E[(X-E[X])(X-E[X])]\)

definition, \(Y=X\)

2

\(=E[(X-E[X])^2]\)

3

\(=\mathrm{Var}(X)\)

Slides p.13

Property 2:

1

\((aY+c)-E[aY+c]=aY+c-(aE[Y]+c)\)

linearity

2

\(=a(Y-E[Y])\)

3

\((bX+d)-E[bX+d]=b(X-E[X])\)

same steps

4

\(\mathrm{cov}(aY+c,bX+d)=E[a(Y-E[Y])\,b(X-E[X])]\)

lines 2 and 3

5

\(=ab\,E[(Y-E[Y])(X-E[X])]\)

linearity

6

\(=ab\,\mathrm{cov}(Y,X)\)

7

\(=ab\,\mathrm{cov}(X,Y)\)

symmetry

Property 3:

1

\((U+V)-E[U+V]=(U-E[U])+(V-E[V])\)

linearity

2

\((X+Y)-E[X+Y]=(X-E[X])+(Y-E[Y])\)

linearity

3

\(\mathrm{cov}(U+V,X+Y)=E\big[\big((U-E[U])+(V-E[V])\big)\big((X-E[X])+(Y-E[Y])\big)\big]\)

lines 1 and 2

4

\(=E\big[(U-E[U])(X-E[X])+(U-E[U])(Y-E[Y])+(V-E[V])(X-E[X])+(V-E[V])(Y-E[Y])\big]\)

5

\(=E[(U-E[U])(X-E[X])]+E[(U-E[U])(Y-E[Y])]+E[(V-E[V])(X-E[X])]+E[(V-E[V])(Y-E[Y])]\)

linearity

6

\(=\mathrm{cov}(U,X)+\mathrm{cov}(U,Y)+\mathrm{cov}(V,X)+\mathrm{cov}(V,Y)\)

definition

### 03.3 Sample covariance Slides p.19

**What** Slides p.19

Review: Covariance · Lecture 1 · p.19 For observations \((y_1,x_1),\dots,(y_n,x_n)\), the sample covariance is: \[\frac{1}{n-1}\sum_i^n (y_i-\bar y)(x_i-\bar x)\]

The sample covariance estimates \(\mathrm{cov}(X,Y)\) from \(n\) pairs. Pair \(i\) comes from person \(i\).

\(E[\cdot]\) becomes "sum, then divide by \(n-1\)". \(E[X]\), \(E[Y]\) become \(\bar x\), \(\bar y\).

**How** Added

Calculate the sample covariance

- Calculate \(\bar x\) and \(\bar y\).
- Calculate the deviations \(x_i-\bar x\) and \(y_i-\bar y\).
- Multiply them for each \(i\).
- Add the \(n\) products.
- Divide by \(n-1\).

**Self-check:** Each set of deviations adds to 0. The sign gives the direction of the relation.

**Example** Added

First 5 people, \(n=5\): How steps 1–5

1

\(\bar x=4093,\quad \bar y=1406.8\)

Section 03.1

2

\(x_i-\bar x:\ 419,\ -355,\ 168,\ -316,\ 84\qquad y_i-\bar y:\ 123.2,\ -109.8,\ -71.8,\ -124.8,\ 183.2\)

3

\((y_i-\bar y)(x_i-\bar x):\ 51620.8,\ 38979.0,\ -12062.4,\ 39436.8,\ 15388.8\)

4

\(\sum_i (y_i-\bar y)(x_i-\bar x)=133363\)

5

\(\frac{1}{5-1}(133363)=33340.75\)

Self-check: \(419-355+168-316+84=0\), \(123.2-109.8-71.8-124.8+183.2=0\). The result is larger than 26672.6 because the divisor is 4, not 5.

All 236 people (course data file, not on the slides):

\(\bar x=3637.864\) cm³, \(\bar y=1284.263\) g.

Sample covariance \(=34014.61\) g·cm³.

Orange dashed lines \(\bar x\) and \(\bar y\) cut the plane into four quadrants.

Quadrants I (top right) and III (bottom left): same-sign deviations, positive product (blue).

Quadrants II (top left) and IV (bottom right): opposite signs, negative product (purple).

[figure]
Figure 3-1. Brainhead data (\(n=236\)) in four quadrants. Blue: positive product (91+97=188). Purple: negative product (24+24=48).

Product sums by quadrant: I: 4327495.5, II: −132594.1, III: 3925393.4, IV: −126861.4.

1

\(\sum_i (y_i-\bar y)(x_i-\bar x)=4327495.5-132594.1+3925393.4-126861.4\)

2

\(=7993433.4\)

3

\(\frac{1}{236-1}(7993433.4)=34014.61\)

The positive products are much larger. A larger head usually has a heavier brain. ▲

**Why** Added

1

\(\mathrm{cov}[X,Y]=E[(X-E[X])(Y-E[Y])]\)

Slides p.17

2

\(\to\ E[(Y-E[Y])(X-E[X])]\)

symmetry

3

\(\to\ \frac{1}{n-1}\sum_i (y_i-E[Y])(x_i-E[X])\)

average over data, divisor of p.16

4

\(\to\ \frac{1}{n-1}\sum_i (y_i-\bar y)(x_i-\bar x)\)

\(E[X],E[Y]\to\bar x,\bar y\)

### 03.4 Correlation Slides p.20

Covariance depends on the units. Correlation has no unit.

**What** Slides p.20

Review: Correlation · Lecture 1 · p.20 Correlation coefficient: \[\rho=\frac{\mathrm{cov}(X,Y)}{\sqrt{\mathrm{var}(X)}\sqrt{\mathrm{var}(Y)}}\] Correlation quantifies strength of linear relationship
Sample correlation: \[r=\frac{\frac{1}{n-1}\sum_{i=1}^n (y_i-\bar y)(x_i-\bar x)}{\sqrt{\frac{1}{n-1}\sum_{i=1}^n (y_i-\bar y)^2}\sqrt{\frac{1}{n-1}\sum_{i=1}^n (x_i-\bar x)^2}}=\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}\]

\(\rho\) ("rho") is the covariance divided by the two standard deviations.

A linear relationship has points near a straight line. Correlation measures only its strength.

\(r\) uses the sample covariance and the sample variances.

From the two sides of the formula:

\(S_{xy}=\sum_{i=1}^n (y_i-\bar y)(x_i-\bar x)\), the sum of deviation products.

\(S_{xx}=\sum_{i=1}^n (x_i-\bar x)^2\), the sum of squares of \(x\). A sum of squares adds the squared deviations.

\(S_{yy}=\sum_{i=1}^n (y_i-\bar y)^2\), the sum of squares of \(y\).

\(S_{xy}\) and \(\sqrt{S_{xx}S_{yy}}=\sqrt{\text{cm}^6\cdot\text{g}^2}\) both have unit g·cm³. \(r\) has no unit.

**How** Added

Calculate \(r\) with the \(S\) form

- Calculate \(\bar x\) and \(\bar y\).
- Calculate the two deviations for each \(i\).
- Add the deviation products to get \(S_{xy}\).
- Add the squared deviations to get \(S_{xx}\) and \(S_{yy}\).
- Calculate \(\sqrt{S_{xx}S_{yy}}\).
- Divide \(S_{xy}\) by it.

**Self-check:** \(S_{xx}>0\), \(S_{yy}>0\). \(r\) has the sign of \(S_{xy}\). \(S_{xy}/(n-1)\) equals the sample covariance.

**Example** Added

First 5 people: How steps 1–6

1

\(\bar x=4093,\quad \bar y=1406.8\)

Section 03.1

2

\(S_{xy}=133363\)

Section 03.3, line 4

3

\(S_{xx}=436722\)

Section 03.2, property 1, line 3

4

\(S_{yy}=15178.24+12056.04+5155.24+15575.04+33562.24=81526.8\)

5

\(\sqrt{S_{xx}S_{yy}}=\sqrt{436722\times81526.8}=\sqrt{35604547149.6}=188691.7\)

6

\(r=\frac{133363}{188691.7}=0.7068\)

All 236 people (course data file, not on the slides): How steps 1–6

1

\(\bar x=3637.864,\quad \bar y=1284.263\)

2

\(S_{xy}=7993433.4\)

quadrant sum, Section 03.3

3

\(S_{xx}=30647233.7,\quad S_{yy}=3309753.7\)

4

\(S_{xx}S_{yy}=30647233.7\times3309753.7=1.014348\times10^{14}\)

5

\(\sqrt{S_{xx}S_{yy}}=\sqrt{1.014348\times10^{14}}=10071484.3\)

6

\(r=\frac{7993433.4}{10071484.3}=0.7937\)

Self-check: \(S_{xy}/(n-1)=7993433.4/235=34014.61\).

The \(1/(n-1)\) form divides by \(s_x s_y\): Slides p.20, first equal sign

\(s_x^2=S_{xx}/235=130413.76\), \(s_x=\sqrt{130413.76}=361.1285\).

\(s_y^2=S_{yy}/235=14084.06\), \(s_y=\sqrt{14084.06}=118.6763\).

1

\(r=\frac{34014.61}{\sqrt{14084.06}\sqrt{130413.76}}\)

2

\(=\frac{34014.61}{118.6763\times361.1285}\)

3

\(=\frac{34014.61}{42857.39}\)

4

\(=0.7937\)

\(r=0.7937\): a positive linear relation. ▲

**Why** Added

1

\(r=\frac{\frac{1}{n-1}S_{xy}}{\sqrt{\frac{1}{n-1}S_{yy}}\sqrt{\frac{1}{n-1}S_{xx}}}\)

definitions of the \(S\) sums

2

\(=\frac{\frac{1}{n-1}S_{xy}}{\sqrt{\frac{1}{n-1}}\sqrt{S_{yy}}\sqrt{\frac{1}{n-1}}\sqrt{S_{xx}}}\)

3

\(=\frac{\frac{1}{n-1}S_{xy}}{\frac{1}{n-1}\sqrt{S_{yy}}\sqrt{S_{xx}}}\)

4

\(=\frac{S_{xy}}{\sqrt{S_{yy}}\sqrt{S_{xx}}}\)

\(\frac{1}{n-1}\) cancels

5

\(=\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}\)

### 03.5 Practice Added

**Q1 (Slides p.18 Exercise).** Write \(\mathrm{Var}(X+Y)\) with \(\mathrm{Var}(X)\), \(\mathrm{Var}(Y)\), and \(\mathrm{cov}(X,Y)\). Check it on the 5-person population.

Answer

1

\(\mathrm{Var}(X+Y)=\mathrm{cov}(X+Y,X+Y)\)

property 1

2

\(=\mathrm{cov}(X,X)+\mathrm{cov}(X,Y)+\mathrm{cov}(Y,X)+\mathrm{cov}(Y,Y)\)

property 3, \(U=X,\ V=Y\)

3

\(=\mathrm{Var}(X)+\mathrm{cov}(X,Y)+\mathrm{cov}(Y,X)+\mathrm{Var}(Y)\)

property 1

4

\(=\mathrm{Var}(X)+\mathrm{Var}(Y)+2\,\mathrm{cov}(X,Y)\)

symmetry

For independent \(X\), \(Y\), Slides p.14 gives \(\mathrm{Var}(X)+\mathrm{var}(Y)\): the term \(2\,\mathrm{cov}(X,Y)\) is 0.

5-person check: \(87344.4+16305.36+2\times26672.6=103649.76+53345.2=156994.96\).

All 236 people (sample values): \(130413.76+14084.06+2\times34014.61=144497.82+68029.22=212527.04\).

**Q2.** Find \(\mathrm{Var}(X-Y)\). With \(s_x^2=130413.76\), \(s_y^2=14084.06\), and sample covariance 34014.61, calculate the sample variance of \(x-y\).

Answer

1

\(\mathrm{Var}(X-Y)=\mathrm{cov}(X+(-1)Y,\ X+(-1)Y)\)

property 1

2

\(=\mathrm{cov}(X,X)+\mathrm{cov}(X,(-1)Y)+\mathrm{cov}((-1)Y,X)+\mathrm{cov}((-1)Y,(-1)Y)\)

property 3

3

\(=\mathrm{cov}(X,X)-\mathrm{cov}(X,Y)-\mathrm{cov}(Y,X)+\mathrm{cov}(Y,Y)\)

property 2, \((-1)(-1)=1\)

4

\(=\mathrm{Var}(X)+\mathrm{Var}(Y)-2\,\mathrm{cov}(X,Y)\)

property 1, symmetry

5

\(=130413.76+14084.06-2\times34014.61\)

6

\(=144497.82-68029.22\)

7

\(=76468.60\)

**Q3.** \(S_{xx}=30647233.7\), \(S_{yy}=3309753.7\), \(n=236\), sample covariance 34014.61. Find \(S_{xy}\) and \(r\).

Answer

1

\(S_{xy}=(n-1)\times34014.61\)

2

\(=235\times34014.61=7993433.35\)

3

\(r=\frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}=\frac{7993433.35}{\sqrt{30647233.7\times3309753.7}}\)

Slides p.20

4

\(=\frac{7993433.35}{\sqrt{1.014348\times10^{14}}}\)

5

\(=\frac{7993433.35}{10071484.3}\)

6

\(=0.7937\)

The difference from 7993433.4 comes from rounding.

## 04 · Toward linear regression: simple and multiple models

This unit gives only the form of each model. Lecture 2 calculates the line.

Plan · Slides p.21–24

p.21 (footer 11/27) three questions, simple model · p.22 (12/27) line on the scatterplot · p.23 (13/27) more covariates · p.24 (14/27) multiple model, course aims.

### 04.1 Three questions and the simple model Slides p.21

**What** Slides p.21

Toward Linear Regression · Lecture 1 · p.21

- How do we characterize the relationship between \(x\) and \(y\)?
- How do we predict \(y\) given \(x\)?
- How does the mean of \(y\) change when \(x\) increases by \(a\).
We can answer questions like these with simple linear regression: \[y_i=\beta_0+\beta_1x_i+\epsilon_i\]

Relationship: how \(y\) changes when \(x\) increases.

\(a\) is any given change in \(x\), for example \(a=100\) cm³.

A parameter is a fixed, unknown number in the model. \(\beta_0\) and \(\beta_1\) are parameters, the same for all \(i\).

Intercept \(\beta_0\): the height of the line \(\beta_0+\beta_1x\) at \(x=0\).

Slope \(\beta_1\): the increase in line height when \(x\) increases by 1.

Error \(\epsilon_i\): \(y_i\) minus the line height \(\beta_0+\beta_1x_i\). Each observation has its own error.

Simple linear regression: \(y\) equals a straight line in one covariate plus an error.

**How** Added

Write one data row in the model form

- Find the \(y\) and \(x\) columns.
- Select row \(i\). Read \(x_i\) and \(y_i\).
- Put them into \(y_i=\beta_0+\beta_1x_i+\epsilon_i\).
- Keep \(\beta_0\), \(\beta_1\), and \(\epsilon_i\) as letters.

**Self-check:** Only \(\beta_0\), \(\beta_1\), \(\epsilon_i\) are unknown. The index of \(\epsilon\) equals the row number.

**Example · Row 1 of the brainhead data** Added

Row 1 (Slides p.23): M, 20-46, head size 4512, brain weight 1530.

1

\(y=\) brain weight, \(x=\) head size

How step 1

2

\(i=1:\ x_1=4512,\ y_1=1530\)

How step 2

3

\(y_1=\beta_0+\beta_1x_1+\epsilon_1\)

4

\(1530=\beta_0+\beta_1\cdot 4512+\epsilon_1\)

How step 3

The model splits 1530 g into the line height at 4512 cm³ and the error \(\epsilon_1\).

**Why · The error is a vertical distance, and \(\beta_1\) is the slope** Added

1

\(1530=\beta_0+\beta_1\cdot 4512+\epsilon_1\)

Example, line 4

2

\(\epsilon_1=1530-(\beta_0+\beta_1\cdot 4512)\)

3

\(\epsilon_1=y_1-(\beta_0+\beta_1x_1)\)

Change in line height when \(x\) increases by \(a\):

1

\([\beta_0+\beta_1(x+a)]-[\beta_0+\beta_1x]\)

2

\(=\beta_0+\beta_1x+\beta_1a-\beta_0-\beta_1x\)

3

\(=\beta_1a\)

4

\(a=1:\ =\beta_1\)

the slope

Lecture 2 adds assumptions that connect the line height to the mean of \(y\).

### 04.2 A line on the scatterplot Slides p.22

**What** Slides p.22

Simple Linear Regression · Lecture 1 · p.22 Simple Linear Regression: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\)
[Scatterplot: Brain Weight (grams) versus Head Size (cm^3), with a straight line through the points.]

The line is \(\beta_0+\beta_1x\) for one set of values that Lecture 2 calculates.

**How** Added

Read a scatterplot with a line

- Find the direction of the line.
- Up from left to right: \(\beta_1\) is positive.
- Select one point. Find if it is above or below the line.
- Above: \(\epsilon_i>0\). Below: \(\epsilon_i<0\).
- Examine the spread. A point far from the line has a large \(|\epsilon_i|\).

**Self-check:** Measure the distance vertically, at the same \(x_i\). \(\epsilon_i\) compares only heights.

**Example · The line on Slides p.22** Added

[figure]
Figure 4-1. Brainhead points and the line of Slides p.22 (green). Orange: row 1 and its vertical distance to the line.

Steps 1–2: the line goes up. The slope is positive.

Steps 3–4: row 1, \((4512,1530)\), is above the line. Its vertical distance is positive.

Step 5: points are above and below the line. Each observation needs its own \(\epsilon_i\).

**Why · The picture shows a straight line** Added

1

\(y_i=\beta_0+\beta_1x_i+\epsilon_i\)

Slides p.21

2

\(y_i-\epsilon_i=\beta_0+\beta_1x_i\)

3

\(\beta_0+\beta_1x\) is a first-degree function of \(x\)

its graph is a straight line

### 04.3 More than one covariate Slides p.23

**What** Slides p.23

What if we have multiple covariates? · Lecture 1 · p.23 The slide prints the first 6 rows and a summary of each column. The values are below.

**First 6 rows (236 rows in total):** the table in Section 01.5 shows these rows.

**Summary of the two number columns:**

|  | Min. | 1st Qu. | Median | Mean | 3rd Qu. | Max.

| head.size | 2773 | 3390 | 3614 | 3638 | 3876 | 4747

| brain.wgt | 1012 | 1208 | 1280 | 1284 | 1351 | 1635

The columns gender and age contain text, not numbers. Each has length 236.

A categorical column contains group labels, not numbers. gender and age are categorical.

Median: the middle value of the sorted column.

1st Qu. and 3rd Qu. (quartiles): approximately 25% and 75% of the values are not larger than them.

**How** Added

Find the possible covariates in a data table

- Count the columns.
- Find the \(y\) column.
- Each other column is a possible covariate.
- Find if each one contains numbers or text.
- Put a number column into the equation directly.
- Change a text column into numbers first (week 6, categorical covariates, Lecture 1 p.37).
- Read the range (Min. to Max.) and center (Median, Mean) of each number column.

**Self-check:** Possible covariates = columns − 1. Min. ≤ 1st Qu. ≤ Median ≤ 3rd Qu. ≤ Max.

**Example · The brainhead data on Slides p.23** Added

1

number of columns \(=4\): gender, age, head.size, brain.wgt

How step 1

2

\(y=\) brain.wgt

How step 2

3

possible covariates: head.size, gender, age

How step 3

4

head.size: numbers; gender, age: text

How steps 4–6

5

head.size: range 2773 to 4747 cm³, median 3614, mean 3638

How step 7

6

brain.wgt: range 1012 to 1635 g, median 1280, mean 1284

How step 7

Self-check: \(4-1=3\). \(2773\le 3390\le 3614\le 3876\le 4747\). \(1012\le 1208\le 1280\le 1351\le 1635\).

**Why · The data has more than one covariate** Added

1

possible covariates \(=\) columns \(-\) outcome columns

2

\(=4-1\)

3

\(=3\)

Simple linear regression uses one of the three. A model with more than one \(x\) can use all.

### 04.4 Simple and multiple linear regression Slides p.24

**What** Slides p.24

Toward Multiple Linear Regression · Lecture 1 · p.24 Simple Linear Regression: \[y_i=\beta_0+\beta_1x_i+\epsilon_i\] Multiple Linear Regression: \[y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\epsilon_i\] This course will focus on developing multiple linear regression

- Theoretically/mathematically (derive estimators)
- Practically (how to fit these models [with software])
- How to choose and compare a model (which \(x_{ij}\) to include)
- How to evaluate the appropriateness of the model/assumptions (model diagnostics)

\(x_{ij}\): covariate \(j\) of observation \(i\).

A coefficient is the parameter that multiplies a covariate, for example \(\beta_2\) for \(x_{i2}\).

An estimator is a formula that calculates an unknown parameter from data.

Fit: calculate the parameter values from the data.

Assumptions: conditions that the model sets on the errors and other parts.

Model diagnostics: methods that check the model and its assumptions with data.

**How** Added

Write the multiple linear regression equation

- Find the \(y\) column.
- List the covariates for the model.
- Number them \(j=1,2,\dots\).
- For each \(j\), write \(\beta_jx_{ij}\).
- Write \(y_i=\beta_0+\) (sum of the terms) \(+\epsilon_i\).

**Self-check:** Parameters = covariates + 1. Each \(x\) has first index \(i\). No \(\beta\) has index \(i\).

**Example · Three covariates for the brainhead data** Added

1

\(y_i=\) brain.wgt of person \(i\)

How step 1

2

\(x_{i1}=\) head.size, \(x_{i2}=\) gender, \(x_{i3}=\) age group

How steps 2–3

3

terms: \(\beta_1x_{i1},\ \beta_2x_{i2},\ \beta_3x_{i3}\)

How step 4

4

\(y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\epsilon_i\)

How step 5

5

\(i=1:\ y_1=1530,\ x_{11}=4512\), gender \(=\) M, age \(=\) 20-46

row 1, p.23

6

\(1530=\beta_0+\beta_1\cdot 4512+\beta_2x_{12}+\beta_3x_{13}+\epsilon_1\)

\(x_{12}\), \(x_{13}\): numbers for M, 20-46

Self-check: \(3+1=4\) parameters. \(x_{12}\) and \(x_{13}\) stay letters until week 6.

**Why · Simple is a special case of multiple** Added

1

\(y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\epsilon_i\)

p.24

2

\(y_i=\beta_0+\beta_1x_{i1}+\epsilon_i\)

keep only \(j=1\)

3

\(y_i=\beta_0+\beta_1x_i+\epsilon_i\)

write \(x_{i1}\) as \(x_i\)

### 04.5 Practice Added

**Q1.** Row 3 (Slides p.23): M, 20-46, head size 4261, brain weight 1335. Write the simple model for \(i=3\). Which values are unknown?

Answer

1

\(x_3=4261,\ y_3=1335\)

2

\(y_3=\beta_0+\beta_1x_3+\epsilon_3\)

3

\(1335=\beta_0+\beta_1\cdot 4261+\epsilon_3\)

Unknown: \(\beta_0\), \(\beta_1\), \(\epsilon_3\).

**Q2.** In \(y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3}+\epsilon_i\), \(x_{i2}\) is gender. What is \(x_{52}\)? How many parameters \(\beta\) are there?

Answer

\(x_{52}\) is the gender of person 5: M (Slides p.23).

\(3+1=4\): \(\beta_0,\beta_1,\beta_2,\beta_3\).

**Q3.** From the summary on Slides p.23, give the median head size. Between which two values are the middle 50% of brain weights?

Answer

Median head size: 3614 cm³.

Between 1st Qu. and 3rd Qu.: 1208 g and 1351 g.

## 05 · Review: normal, chi-square, t, and F distributions

Later lectures use these four distributions for inference about regression.

Plan · Slides p.25–28

p.25 normal, linear combinations · p.26 chi-square = sum of squared standard normals · p.27 t = standard normal / \(\sqrt{\text{chi-square}/\nu}\) · p.28 F = ratio of two chi-squares, each over its degrees of freedom.

A distribution describes how likely each value of a random variable is. "\(Z\sim N(\mu,\sigma^2)\)": \(Z\) has the normal distribution with mean \(\mu\), variance \(\sigma^2\).

Distribution parameters (\(\mu\), \(\sigma^2\), \(\nu\)) are fixed numbers that set its shape.

Moments: the slides use this name for \(E[Z]\) and \(\mathrm{Var}(Z)\).

### 05.1 Normal distribution Slides p.25

Lecture 3 shows that \(\hat\beta_1\) is a linear combination of independent normals.

**What** Slides p.25

Review: Normal Distribution · Lecture 1 · p.25 \[Z\sim N(\mu,\sigma^2)\quad\text{with parameters } \mu \text{ and } \sigma^2\] PDF: \[f(z)=\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left[-\frac{(z-\mu)^2}{2\sigma^2}\right]\] Moments: \[E[Z]=\mu,\qquad \mathrm{Var}(Z)=\sigma^2\] For independent \(Z_i\overset{ind}{\sim}N(\mu_i,\sigma^2)\), \(U=\sum_i^n(a_iZ_i+b_i)\) is normally distributed: \[U\sim N\left(\sum_i^n(a_i\mu_i+b_i),\ \sum_i^n a_i^2\sigma_i^2\right)\]

The normal PDF is a bell curve, symmetric about \(\mu\) ("mu"). \(\sigma^2\) ("sigma squared") sets its width.

Standard normal: \(N(0,1)\).

\(\exp[t]=e^t\).

\(\overset{ind}{\sim}\): "are independent and have the distribution".

A linear combination multiplies each \(Z_i\) by \(a_i\), adds \(b_i\), and sums.

The slide writes \(\sigma^2\) in the assumption and \(\sigma_i^2\) in \(\mathrm{Var}(U)\). With all \(\sigma_i^2=\sigma^2\), \(\mathrm{Var}(U)=\sum_i^n a_i^2\sigma^2\).

[figure]
Figure 5-1 · PDF of \(N(0,1)\). Highest point \(f(0)=0.3989\) (Example).

**How** Added

Find the distribution of \(U=\sum_i^n(a_iZ_i+b_i)\)

- Make sure that each \(Z_i\) is normal.
- Make sure that the \(Z_i\) are independent.
- Read \(a_i\) and \(b_i\).
- Read \(\mu_i\) and \(\sigma^2\).
- Mean: add \(a_i\mu_i+b_i\) over all terms.
- Variance: add \(a_i^2\sigma^2\) over all terms. Do not use \(b_i\).
- Write \(U\sim N(\text{mean},\ \text{variance})\).

**Self-check:** The variance is positive.

Calculate the normal PDF \(f(z)\)

- Put \(z\), \(\mu\), \(\sigma^2\) into the formula.
- Calculate the exponent \(-(z-\mu)^2/(2\sigma^2)\).
- Calculate \(1/\sqrt{2\pi\sigma^2}\).
- Multiply it by \(\exp\) of the exponent.

**Self-check:** \(f(z)\) is largest at \(z=\mu\).

**Example** Added

\(f(0)\) for \(N(0,1)\):

How · PDF steps 1–4

1

\(f(0)=\frac{1}{\sqrt{2\pi\cdot 1}}\exp\left[-\frac{(0-0)^2}{2\cdot 1}\right]\)

2

\(=\frac{1}{\sqrt{2\pi}}\exp[0]\)

3

\(=\frac{1}{\sqrt{2\pi}}\)

\(e^0=1\)

4

\(=\frac{1}{2.5066}\)

5

\(=0.3989\)

Independent \(Z_1\sim N(1,1)\), \(Z_2\sim N(2,1)\), \(Z_3\sim N(300,1)\). \(a=(2,-1,0.5)\), \(b=(1,0,-3)\): \(U=(2Z_1+1)+(-1\cdot Z_2+0)+(0.5Z_3-3)\).

How · linear combination steps 1–4

Normal and independent: the rule on Slides p.25 applies. \(\mu_i=1,2,300\), \(\sigma^2=1\).

How · linear combination step 5

1

\(E[U]=(2\cdot1+1)+(-1\cdot2+0)+(0.5\cdot300-3)\)

2

\(=3+(-2)+147\)

3

\(=148\)

How · linear combination step 6

1

\(\mathrm{Var}(U)=2^2\cdot1+(-1)^2\cdot1+0.5^2\cdot1\)

2

\(=4+1+0.25\)

3

\(=5.25\)

How · linear combination step 7

\(U\sim N(148,\ 5.25)\).

**Why** Added

The slide states that \(U\) is normal. The mean and variance follow from linearity, \(\mathrm{Var}(aZ+b)=a^2\mathrm{Var}(Z)\), and Property 2 (Section 02.4).

1

\(E[U]=E\left[\sum_i^n(a_iZ_i+b_i)\right]\)

2

\(=\sum_i^n\left(a_iE[Z_i]+b_i\right)\)

linearity

3

\(=\sum_i^n(a_i\mu_i+b_i)\)

1

\(\mathrm{Var}(U)=\mathrm{Var}\left(\sum_i^n(a_iZ_i+b_i)\right)\)

2

\(=\sum_i^n\mathrm{Var}(a_iZ_i+b_i)\)

\(Z_i\) independent

3

\(=\sum_i^n a_i^2\,\mathrm{Var}(Z_i)\)

Property 1

4

\(=\sum_i^n a_i^2\sigma_i^2\)

### 05.2 Chi-square distribution Slides p.26

Lecture 3 uses it for inference about \(\sigma^2\).

**What** Slides p.26

Review: Chi-square Distribution · Lecture 1 · p.26 \[X\sim\chi^2_\nu\quad\text{with } \nu \text{ degrees of freedom}\] PDF: \[f(x)=\frac{1}{2^{\frac{\nu}{2}}\Gamma(\frac{\nu}{2})}x^{\frac{\nu}{2}-1}e^{-x/2},\quad X>0\] Moments: \[E[X]=\nu\] \[\mathrm{Var}(X)=2\nu\] For \(Z_i\overset{iid}{\sim}N(0,1)\): \[X=\sum_{i=1}^n Z_i^2\sim\chi^2_n\]

\(\chi^2_\nu\) ("chi") takes only positive values. A sum of squares cannot be negative.

Degrees of freedom \(\nu\) ("nu"): the number of squared standard normals in the sum.

iid (independent and identically distributed): independent, with one common distribution.

Gamma function: \(\Gamma(\alpha)=\int_0^\infty t^{\alpha-1}e^{-t}\,dt\) for \(\alpha>0\).

Three Gamma facts: Added

\(\Gamma(1)=\int_0^\infty e^{-t}\,dt=1\).

\(\Gamma(1/2)=\sqrt\pi=1.7725\) (known result).

\(\Gamma(\alpha+1)=\alpha\,\Gamma(\alpha)\). \(\Gamma(3/2)=\frac12\Gamma(1/2)=\sqrt\pi/2=0.8862\), \(\Gamma(2)=1\cdot\Gamma(1)=1\).

[figure]
Figure 5-2 · PDF of \(\chi^2_\nu\), \(\nu=2,3,5\). Larger \(\nu\): further right and wider. Orange: \(f(2)\) for \(\nu=2\).

**How** Added

Identify a chi-square variable and write its moments

- Make sure that each squared term is \(N(0,1)\).
- Make sure that the terms are iid.
- Count the \(n\) squared terms. Set \(\nu=n\).
- Write \(X\sim\chi^2_n\), \(E[X]=n\), \(\mathrm{Var}(X)=2n\).

**Self-check:** \(\mathrm{Var}(X)=2E[X]\).

Calculate the chi-square PDF at \(x\)

- Calculate \(2^{\nu/2}\) and \(\Gamma(\nu/2)\).
- Calculate \(x^{\nu/2-1}\) and \(e^{-x/2}\).
- Multiply the two numbers from step 2.
- Divide by the two numbers from step 1.

**Self-check:** \(x>0\).

**Example** Added

Example 1: \(f(2)\), \(\nu=2\).

How · PDF steps 1–4

1

\(f(2)=\frac{1}{2^{2/2}\Gamma(2/2)}\,2^{2/2-1}e^{-2/2}\)

2

\(=\frac{1}{2\,\Gamma(1)}\,2^{0}e^{-1}\)

3

\(=\frac{1}{2}e^{-1}\)

\(\Gamma(1)=1\), \(2^0=1\)

4

\(=\frac{0.36788}{2}\)

5

\(=0.1839\)

Example 2: \(Z_1,Z_2,Z_3\overset{iid}{\sim}N(0,1)\), \(X=Z_1^2+Z_2^2+Z_3^2\). Find its distribution, moments, and \(f(1)\).

How · identify steps 1–3

3 iid squared \(N(0,1)\) terms: \(\nu=3\).

How · identify step 4

\(X\sim\chi^2_3\), \(E[X]=3\), \(\mathrm{Var}(X)=2\cdot3=6\).

How · PDF steps 1–4

1

\(f(1)=\frac{1}{2^{3/2}\Gamma(3/2)}\,1^{3/2-1}e^{-1/2}\)

2

\(=\frac{1}{2^{3/2}\cdot\frac{\sqrt\pi}{2}}\,e^{-1/2}\)

\(\Gamma(3/2)=\sqrt\pi/2\)

3

\(=\frac{1}{\sqrt2\,\sqrt\pi}\,e^{-1/2}\)

\(2^{3/2}/2=2^{1/2}\)

4

\(=\frac{e^{-1/2}}{\sqrt{2\pi}}\)

5

\(=\frac{0.60653}{2.5066}\)

6

\(=0.2420\)

**Why** Added

1

\(\mathrm{Var}(Z)=E[Z^2]-E[Z]^2\)

2

\(1=E[Z^2]-0^2\)

\(N(0,1)\) moments

3

\(E[Z^2]=1\)

1

\(E[X]=E\left[\sum_{i=1}^n Z_i^2\right]\)

2

\(=\sum_{i=1}^n E[Z_i^2]\)

linearity

3

\(=n\)

Optional — not on the slides

For \(Z\sim N(0,1)\), \(E[Z^4]=3\) (known result).

1

\(\mathrm{Var}(Z^2)=E[Z^4]-E[Z^2]^2\)

2

\(=3-1^2\)

3

\(=2\)

4

\(\mathrm{Var}(X)=\sum_{i=1}^n\mathrm{Var}(Z_i^2)\)

\(Z_i^2\) independent

5

\(=2n\)

### 05.3 t distribution Slides p.27

Lecture 3 uses it for tests of the regression coefficients.

**What** Slides p.27

Review: t-Distribution · Lecture 1 · p.27 \[Y\sim t_\nu\quad\text{with } \nu \text{ degrees of freedom}\] PDF: \[f(y)=\frac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\pi\nu}\,\Gamma(\frac{\nu}{2})}\left(1+\frac{y^2}{\nu}\right)^{-\frac{\nu+1}{2}}\] Moments: \[E[Y]=0 \text{ if } \nu>1,\ \text{NA otherwise}\] \[\mathrm{Var}(Y)=\frac{\nu}{\nu-2} \text{ if } \nu>2,\ \infty \text{ otherwise}\] For independent \(Z\sim N(0,1)\) and \(X\sim\chi^2_\nu\) \[\frac{Z}{\sqrt{X/\nu}}\sim t_\nu\]

The slide prints the PDF denominator as \(\sqrt{\pi\nu}\,\frac{\nu}{2}\), with no \(\Gamma\). The total area is 1 only with \(\Gamma(\frac{\nu}{2})\). This page uses \(\Gamma(\frac{\nu}{2})\).

\(t_\nu\) is a bell curve, symmetric about 0. Its center is lower than \(N(0,1)\), and its tails are thicker.

A tail is the part of the curve far from the center. Thicker tails make extreme values more likely.

\(\nu\) is the degrees of freedom of the chi-square in the denominator.

NA (not available): for \(\nu=1\), \(\int y f(y)\,dy\) does not converge. The mean does not exist.

\(\infty\): for \(\nu\le 2\), \(\int y^2 f(y)\,dy\) does not converge.

[figure]
Figure 5-3 · PDFs of \(t_1\) (orange), \(t_5\) (blue), \(N(0,1)\) (black, dashed). As \(\nu\) increases, \(t_\nu\) comes near \(N(0,1)\).

**How** Added

Identify a t variable and write its moments

- Numerator: one \(N(0,1)\) variable \(Z\).
- Denominator: \(\sqrt{X/\nu}\), \(X\sim\chi^2_\nu\).
- Make sure that the divisor is the degrees of freedom of \(X\).
- Make sure that \(Z\) and \(X\) are independent.
- Write \(\frac{Z}{\sqrt{X/\nu}}\sim t_\nu\).
- If \(\nu>1\), \(E[Y]=0\). If \(\nu>2\), \(\mathrm{Var}(Y)=\nu/(\nu-2)\).

**Self-check:** For \(\nu>2\), \(\nu/(\nu-2)>1\): more spread than \(N(0,1)\).

Calculate the t PDF at \(y\)

- Calculate \(\Gamma(\frac{\nu+1}{2})\) and \(\Gamma(\frac{\nu}{2})\).
- Calculate \(\sqrt{\pi\nu}\).
- Calculate \(\left(1+y^2/\nu\right)^{-(\nu+1)/2}\).
- Multiply \(\Gamma(\frac{\nu+1}{2})\) by the step 3 number.
- Divide by \(\sqrt{\pi\nu}\,\Gamma(\frac{\nu}{2})\).

**Self-check:** \(f(0)<1/\sqrt{2\pi}=0.3989\).

**Example** Added

Example 1: \(f(0)\), \(\nu=3\).

How · PDF steps 1–5

1

\(f(0)=\frac{\Gamma(\frac{3+1}{2})}{\sqrt{3\pi}\,\Gamma(\frac{3}{2})}\left(1+\frac{0^2}{3}\right)^{-\frac{3+1}{2}}\)

2

\(=\frac{\Gamma(2)}{\sqrt{3\pi}\,\Gamma(\frac32)}\cdot 1^{-2}\)

3

\(=\frac{1}{\sqrt{3\pi}\cdot\frac{\sqrt\pi}{2}}\)

\(\Gamma(2)=1,\ \Gamma(\frac32)=\frac{\sqrt\pi}{2}\)

4

\(=\frac{2}{\sqrt3\,\pi}\)

5

\(=0.3676\)

Self-check: \(0.3676<0.3989\).

Example 2: independent \(Z\sim N(0,1)\), \(X\sim\chi^2_5\). Find the distribution and moments of \(Y=Z/\sqrt{X/5}\).

How · identify steps 1–4

\(Z\) is \(N(0,1)\). \(X\sim\chi^2_5\) is divided by 5. \(Z\), \(X\) independent.

How · identify steps 5–6

\(Y\sim t_5\). \(\nu=5>1\): \(E[Y]=0\). \(\nu=5>2\):

1

\(\mathrm{Var}(Y)=\frac{5}{5-2}\)

2

\(=\frac{5}{3}\)

3

\(=1.6667\)

**Why** Added

Mean: \(y\) occurs only as \(y^2\), so \(f(y)=f(-y)\). For \(\nu>1\), symmetry gives \(E[Y]=0\).

Optional — not on the slides

Variance, for \(\nu>2\):

1

\(\mathrm{Var}(Y)=E[Y^2]-E[Y]^2\)

2

\(=E[Y^2]\)

\(E[Y]=0\)

3

\(=E\left[\frac{Z^2}{X/\nu}\right]\)

4

\(=\nu\,E[Z^2]\,E\left[\frac1X\right]\)

\(Z,X\) independent

5

\(=\nu\,E\left[\frac1X\right]\)

\(E[Z^2]=1\)

1

\(E\left[\frac1X\right]=\int_0^\infty\frac1x\cdot\frac{1}{2^{\nu/2}\Gamma(\nu/2)}x^{\nu/2-1}e^{-x/2}\,dx\)

chi-square PDF

2

\(=\frac{1}{2^{\nu/2}\Gamma(\nu/2)}\int_0^\infty x^{\nu/2-2}e^{-x/2}\,dx\)

3

\(=\frac{1}{2^{\nu/2}\Gamma(\nu/2)}\cdot 2^{\nu/2-1}\Gamma(\nu/2-1)\)

\(x=2t\); Gamma definition

4

\(=\frac{\Gamma(\nu/2-1)}{2\,\Gamma(\nu/2)}\)

5

\(=\frac{\Gamma(\nu/2-1)}{2\,(\nu/2-1)\,\Gamma(\nu/2-1)}\)

\(\Gamma(\alpha+1)=\alpha\Gamma(\alpha)\)

6

\(=\frac{1}{\nu-2}\)

7

\(\mathrm{Var}(Y)=\nu\cdot\frac{1}{\nu-2}=\frac{\nu}{\nu-2}\)

Step 3 needs \(\nu/2-1>0\), that is, \(\nu>2\).

### 05.4 F distribution Slides p.28

**What** Slides p.28

Review: F Distribution · Lecture 1 · p.28 \[W\sim F(\nu_1,\nu_2)\ \text{with } \nu_1,\nu_2 \text{ degrees of freedom}\] PDF: \[f(w)=\frac{\Gamma((\nu_1+\nu_2)/2)}{\Gamma(\nu_1/2)\Gamma(\nu_2/2)}\left(\nu_1^{\nu_1}\nu_2^{\nu_2}\frac{w^{\nu_1-2}}{(\nu_2+\nu_1w)^{(\nu_1+\nu_2)}}\right)^{1/2},\quad w>0\] For independent \(X_1\sim\chi^2_{\nu_1}\) and \(X_2\sim\chi^2_{\nu_2}\), \[W=\frac{X_1/\nu_1}{X_2/\nu_2}\sim F(\nu_1,\nu_2)\]

\(F(\nu_1,\nu_2)\) takes only positive values.

\(\nu_1\) (numerator degrees of freedom) always comes first. \(\nu_2\) (denominator degrees of freedom) comes second.

The slide gives no moments for F.

[figure]
Figure 5-4 · PDFs of \(F(2,2)\) (blue) and \(F(5,10)\) (green). Orange: \(F(2,2)\) PDF at \(w=1\).

**How** Added

Identify an F variable

- Numerator: \(X_1/\nu_1\), a chi-square over its degrees of freedom.
- Denominator: \(X_2/\nu_2\), a second chi-square over its degrees of freedom.
- Make sure that \(X_1\) and \(X_2\) are independent.
- Write \(W\sim F(\nu_1,\nu_2)\), numerator first.

**Self-check:** Each divisor is the degrees of freedom of its own chi-square.

Calculate the F PDF at \(w\)

- Calculate \(\Gamma((\nu_1+\nu_2)/2)/[\Gamma(\nu_1/2)\Gamma(\nu_2/2)]\).
- Calculate \(\nu_1^{\nu_1}\nu_2^{\nu_2}w^{\nu_1-2}/(\nu_2+\nu_1w)^{(\nu_1+\nu_2)}\).
- Take its square root.
- Multiply the step 1 and step 3 numbers.

**Self-check:** \(w>0\).

**Example** Added

\(F(2,2)\) at \(w=1\):

How · PDF steps 1–4

1

\(f(1)=\frac{\Gamma((2+2)/2)}{\Gamma(2/2)\Gamma(2/2)}\left(2^2\cdot2^2\cdot\frac{1^{2-2}}{(2+2\cdot1)^{(2+2)}}\right)^{1/2}\)

2

\(=\frac{\Gamma(2)}{\Gamma(1)\Gamma(1)}\left(16\cdot\frac{1}{4^4}\right)^{1/2}\)

3

\(=1\cdot\left(\frac{16}{256}\right)^{1/2}\)

\(\Gamma(2)=\Gamma(1)=1\)

4

\(=\frac{4}{16}\)

5

\(=0.25\)

**Why** Added

The lecture does not derive the F PDF. The square of a t variable is an F variable:

Optional — not on the slides

1

\(Y=\frac{Z}{\sqrt{X/\nu}}\sim t_\nu\)

p.27

2

\(Y^2=\frac{Z^2}{X/\nu}\)

3

\(Y^2=\frac{Z^2/1}{X/\nu}\)

4

\(Z^2\sim\chi^2_1\)

p.26, \(n=1\)

5

\(Y^2\sim F(1,\nu)\)

p.28, \(\nu_1=1,\ \nu_2=\nu\)

Unit summary

Normal: \(U\sim N\left(\sum(a_i\mu_i+b_i),\ \sum a_i^2\sigma_i^2\right)\).

Chi-square: \(\sum_{i=1}^n Z_i^2\sim\chi^2_n\), mean \(\nu\), variance \(2\nu\).

t: \(Z/\sqrt{X/\nu}\sim t_\nu\), mean 0 (\(\nu>1\)), variance \(\nu/(\nu-2)\) (\(\nu>2\)).

F: \(\frac{X_1/\nu_1}{X_2/\nu_2}\sim F(\nu_1,\nu_2)\).

### 05.5 Practice Added

**Q1.** \(Z_1\overset{ind}{\sim}N(1,4)\), \(Z_2\overset{ind}{\sim}N(3,4)\). Find the distribution of \(U=3Z_1-2Z_2+5\).

Answer

\(a_1=3,\ b_1=5,\ a_2=-2,\ b_2=0\); \(\mu_1=1,\ \mu_2=3,\ \sigma^2=4\).

1

\(E[U]=(3\cdot1+5)+(-2\cdot3+0)\)

2

\(=8-6=2\)

3

\(\mathrm{Var}(U)=3^2\cdot4+(-2)^2\cdot4\)

4

\(=36+16=52\)

5

\(U\sim N(2,\ 52)\)

**Q2.** \(Z_1,Z_2,Z_3,Z_4\overset{iid}{\sim}N(0,1)\). Name the distribution of (a) \(Z_2^2+Z_3^2+Z_4^2\); (b) \(\dfrac{Z_1}{\sqrt{(Z_2^2+Z_3^2+Z_4^2)/3}}\); (c) \(\dfrac{Z_1^2/1}{(Z_2^2+Z_3^2+Z_4^2)/3}\). For (a) and (b), give the mean and variance.

Answer

(a) \(\chi^2_3\). Mean 3, variance \(2\cdot3=6\).

(b) \(Z_1\) is \(N(0,1)\), independent of \(X\sim\chi^2_3\): \(t_3\). Mean 0 (\(\nu=3>1\)). Variance \(3/(3-2)=3\).

(c) \(\chi^2_1/1\) over \(\chi^2_3/3\), independent: \(F(1,3)\).

**Q3.** With the PDF on Slides p.27, calculate \(f(0)\) for \(Y\sim t_1\). What is \(E[Y]\)?

Answer

1

\(f(0)=\frac{\Gamma(\frac{1+1}{2})}{\sqrt{\pi\cdot1}\,\Gamma(\frac12)}\left(1+\frac{0^2}{1}\right)^{-1}\)

2

\(=\frac{\Gamma(1)}{\sqrt\pi\cdot\sqrt\pi}\)

\(\Gamma(\frac12)=\sqrt\pi\)

3

\(=\frac1\pi\)

\(\Gamma(1)=1\)

4

\(=0.3183\)

\(\nu=1\) is not \(>1\): \(E[Y]\) is NA.

## 06 · Course overview and logistics

Plan · Slides p.29–38

p.29 title · p.30 content, books · p.31 grading · p.32 assignments · p.33 exams · p.34 participation · p.35 communication · p.36 tutorials · p.37 schedule · p.38 tips.

### 06.1 Section title page Slides p.29

Slide p.29: "Course Overview".

Footer numbers 19/27 to 27/27 are on PDF pages 30 to 38. This unit uses PDF page numbers.

### 06.2 Course content and reference books Slides p.30

**What** Slides p.30

Course Overview · Lecture 1 · p.30

- Review & Simple Linear Regression (2 weeks)
- Multiple Linear Regression (4 weeks)
- Model Diagnostics (3 weeks)
- Model Building (3 weeks)
- Extensions (time permitting)
Textbook: None required. If you want a different perspective:

- Introduction to Regression Modeling by Abraham and Ledolter, 2006 (available online, see course outline for link)
- Gelman, A., Hill, J., & Vehtari, A. (2020). Regression and other stories. Cambridge University Press.
- Any other introductory book on linear regression

SLR: simple linear regression. MLR: multiple linear regression.

Model building: selecting the variables of a model and their form.

Extensions: topics past linear regression, only if time is available.

**How** Added

Find the block for a topic

- Count the covariates that the topic uses.
- One covariate: block 1.
- Two or more: block 2.
- Checking a fitted model: block 3.
- Selecting variables: block 4.

**Self-check:** Blocks 1–4 add to 12 weeks.

**Example · Check the number of weeks** Added

1

\(2+4+3+3\)

blocks 1–4, Slides p.30

2

\(=6+3+3\)

3

\(=12\) weeks

Block 5 has no fixed length.

**Why · The order of the blocks** Added

1

Start with one covariate (SLR).

2

Extend to two or more (MLR).

3

Fit a model before you check it (diagnostics).

4

Check models before you select one (model building).

### 06.3 Grading Slides p.31

**What** Slides p.31

Grading · Lecture 1 · p.31

- Assignments: 15%
- Midterm 1: 15%
- Midterm 2: 15%
- Final: 50%
- Participation: 5%

Weight: the percentage of the final grade from one component.

Midterm: an exam in the middle of the term.

Participation: a grade from your Piazza posts (Slides p.34).

[figure]
Length is proportional to weight. A = Assignments, M1 = Midterm 1, M2 = Midterm 2, P = Participation.

**How** Added

Calculate a final grade

- Write the percentage score of each component.
- Multiply each score by its weight.
- Add the five products.

**Self-check:** The weights add to 100%.

**Example · A final grade** Added

Scores: Assignments 80%, Midterm 1 70%, Midterm 2 75%, Final 72%, Participation 4 of 5 (80%).

1

\(0.15(80)+0.15(70)+0.15(75)+0.50(72)+0.05(80)\)

Slides p.31

2

\(=12+10.5+11.25+36+4\)

3

\(=73.75\)

**Why · The weights add to 100%** Added

1

\(15+15+15+50+5\)

Slides p.31

2

\(=45+50+5\)

3

\(=100\)

### 06.4 Assignments Slides p.32

**What** Slides p.32

Assignments · Lecture 1 · p.32

- Due:

- Assignment 1: 5pm Friday, September 25, 2026
- Assignment 2: 5pm Friday, October 30, 2026
- Assignment 3: 5pm Friday, November 27, 2026

- Each worth 5 % of final grade
- Goal: Practice what you've learned in class, extend your knowledge independently
- Typically 4 questions with multiple parts
- Usually include a theoretical question, an applied analysis question, and a simulation-based question
- You may discuss with other students, but you must write up your own solutions
Assignments up to 24 hours late will receive a penalty of 50%. If you are unable to complete an assignment due to severe illness or extenuating circumstances, you must email the instructor before the deadline.

The slide item about software is not quoted.

Theoretical question: you answer with a mathematical derivation.

Applied analysis question: you fit a model to real data and explain the result.

Simulation-based question: you generate random data and study how a method behaves.

**How** Added

Complete an assignment

- Record the deadline: 5pm Friday.
- Discuss with other students (optional).
- Write your own solutions.
- If severe illness stops you, email the instructor before the deadline.

**Self-check:** \(3\times 5\%\) equals the 15% on Slides p.31.

**Example · Check the assignment weight** Added

1

\(3\times 5\%\)

Slides p.32

2

\(=15\%\)

Slides p.31

**Why · The due dates spread over the term** Added

1

A1 to A2: Sept 25 to Oct 30 = 35 days

2

A2 to A3: Oct 30 to Nov 27 = 28 days

3

A1 to A3: Sept 25 to Nov 27 = 63 days

### 06.5 Midterms and final exam Slides p.33

**What** Slides p.33

Midterms & Final Exam · Lecture 1 · p.33

- Two in-class midterms (1hr20min)

- October 8th & November 12th

- You must attend the correct section!

- Each worth 15% of final grade

- Final exam (2hrs30min)

- Scheduled by the registrar during final exam period
- Worth 50% of final grade

- Coverage is cumulative
- Will test theory & application
- Bring a non-programmable calculator
If you are unable to attend a midterm due to severe illness or extenuating circumstances, you must email the instructor before the deadline/midterm start time. Grades for a missed midterm will be shifted to the final exam.

The slide also says that exams ask you to read the printed output of a fitted model.

In-class: during the usual class time and room. October 8 and November 12 are Thursdays.

Registrar: the university office for exams.

Cumulative: each exam can test all material from the start of the course.

**How** Added

New weights after a missed midterm

- Set the missed midterm to 0%.
- Add its 15% to the final.
- Keep the other weights.

**Self-check:** The weights add to 100%.

**Example · A student misses Midterm 1** Added

1

Final weight \(=50\%+15\%\)

Slides p.33

2

Final weight \(=65\%\)

**Why · The sum stays at 100%** Added

1

\(15+0+15+(50+15)+5\)

2

\(=15+15+65+5\)

3

\(=100\)

### 06.6 Participation Slides p.34

**What** Slides p.34

Participation · Lecture 1 · p.34

- Worth 5% of final grade
- Based on (public) discussions on Piazza

- 0 pts: You did not post on Piazza this term
- 1 pts: You posted on Piazza once
- 2 pts: You posted on Piazza twice
- 3 pts: You posted on Piazza several times
- 4 pts: You asked and answered several questions on Piazza
- 5 pts: You regularly posted high quality questions and gave high quality answers to student questions

Piazza: the online question-and-answer forum of the course.

Only public posts (visible to all students) count.

**How** Added

Get the participation grade

- Post publicly on Piazza.
- Ask questions.
- Answer other students.
- Do this for the full term.

**Self-check:** Find your current level on Slides p.34.

### 06.7 Communication Slides p.35

**What** Slides p.35

Communication · Lecture 1 · p.35

- Slides, assignments, etc. will all be posted on LEARN
- Announcements will be made in class and on LEARN
- All questions about course content should be posted on Piazza (not email)

- If you have personal concerns, you can email the TAs/instructor directly.

LEARN: the university course website (D2L Brightspace).

TA: teaching assistant.

Where to send a question

Course content: Piazza, not email.

Personal concern: email a TA or the instructor.

### 06.8 Tutorials Slides p.36

**What** Slides p.36

Tutorials · Lecture 1 · p.36

- Scheduled for Fridays at 2:30 in this room
- There will be 4 formal tutorials: Sept 18th, Oct 2nd, Oct 23rd, Nov 6th

- Work through practice problems with TA

- In other weeks TAs will use the tutorial time slot for office hours in this room

Tutorial: a class where a TA helps you do practice problems.

Office hour: a time when a TA answers your questions.

### 06.9 Schedule Slides p.37

**What** Slides p.37

Simple Linear Regression · Slides p.37

Week 1: Introduction. TA Office hour.

Week 2: Simple linear regression, least squares estimation, inference. **Tutorial 1 (Sept 18th)**.

Week 3: Prediction & Review of matrix algebra. TA Office hour. **Assignment 1 due 5pm (Sept 25th)**.

Multiple Linear Regression · Slides p.37

Week 4: Multiple linear regression, estimation & inference. **Tutorial 2 (Oct 2nd)**.

Week 5: Inference continued, prediction. TA Office hour. **Midterm 1 (Oct 8th, in class)**.

READING WEEK (no classes).

Week 6: Categorical covariates, interactions, non-linearities. **Tutorial 3 (Oct 23rd)**.

Week 7: ANOVA, multicollinearity. TA Office hour. **Assignment 2 due 5pm (Oct 30th)**.

Model Diagnostics · Slides p.37

Week 8: Residuals, Weighted least squares. **Tutorial 4 (Nov 6th)**.

Week 9: Outliers. TA Office hour. **Midterm 2 (Nov 12th, in class)**.

Week 10: Influential observations & Goodness of fit. TA Office hour.

Model Building · Slides p.37

Week 11: Prediction error, cross-validation, variable selection. TA Office hour. **Assignment 3 due 5pm (Nov 27th)**.

Week 12: LASSO, shrinkage, confounding. TA Office hour.

Extra · Slides p.37

Week 13: Logistic regression (time permitting).

**How** Added

Find a date in the schedule

- Find the event.
- Read its week.
- Read the topic of that week.

**Self-check:** Dates agree with Slides p.32, p.33, and p.36.

### 06.10 Tips for success Slides p.38

**What** Slides p.38

Tips for Success · Lecture 1 · p.38

- Come to class
- Participate

- Answer questions in class
- Ask questions when something is unclear

- Try practice problems, compare solutions on Piazza
- Don't fall behind

- Lecture recaps
- Tutorials
- Homework

- If you do fall behind

- Review the slides, post any misunderstandings on Piazza
- Come to class
- Come to office hours

Lecture recap: a short review of earlier lectures.

**How** Slides p.38

Recover when you fall behind (Slides p.38)

- Review the slides.
- Post what you do not understand on Piazza.
- Come to class.
- Go to office hours.

**Self-check:** Recaps, tutorials, and homework are current.

### 06.11 Practice Added

**Q1.** List the five grade components and weights. Show that they add to 100%.

Answer

Assignments 15%, Midterm 1 15%, Midterm 2 15%, Final 50%, Participation 5% (Slides p.31).

1

\(15+15+15+50+5\)

2

\(=45+50+5\)

3

\(=100\)

**Q2.** A student misses Midterm 2 because of severe illness and emails the instructor before the start time. What is the new final exam weight?

Answer

1

\(50\%+15\%\)

Slides p.33

2

\(=65\%\)

Others: Assignments 15%, Midterm 1 15%, Midterm 2 0%, Participation 5%.

**Q3.** From Slides p.37, give the date of Midterm 1, its week, and the topic of that week.

Answer

Oct 8th, in class, Week 5: "Inference continued, prediction".

**Q4.** Scores: Assignments 90%, Midterm 1 60%, Midterm 2 80%, Final 70%, Participation 5 of 5. Calculate the final grade.

Answer

1

\(0.15(90)+0.15(60)+0.15(80)+0.50(70)+0.05(100)\)

5 of 5 = 100%

2

\(=13.5+9+12+35+5\)

3

\(=74.5\)