STAT 331 · Lecture 4 · Simple Linear Regression: Prediction

# Lecture 4: Simple Linear Regression: Prediction

This lecture estimates the mean response at a given \(x_0\), gives a confidence interval for it, and gives a prediction interval for a new response (Lecture 4 · p.1–43).

Contents
01 · Recap: the model, the estimators, and inference for \(\beta_1\) 02 · Point estimate of the mean response and its distribution 03 · Confidence interval for the mean response 04 · Predicting a new response: the prediction interval 05 · Prediction intervals over a range of \(x\) values 06 · Practice questions Q1 and Q2

## 01 · Recap: the model, the estimators, and inference for \(\beta_1\)

Plan · Slides p.1–7

Model (p.3), estimators (p.4), CI (p.5), test (p.6), brainhead results (p.7).

Data for this unit · Added

Small data set: \(x=(1,2,3,4,5)\), \(y=(2,4,5,4,5)\), \(n=5\).

Brainhead data: \(y\) is brain weight (g). \(x\) is head size (cm³).

### 01.1 Title page and section page Slides p.1–2

Slides p.1 title: "Lecture 4: Simple Linear Regression: Prediction".

Prediction is the use of a fitted model to give a value for a new response (Unit 04).

Slides p.2: section page "Recap" (Slides p.3–7).

### 01.2 The model and its assumptions Slides p.3

**What** Slides p.3

Simple Linear Regression · Lecture 4 · p.3 \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\] Or: \[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\] Assumptions:
1. Linearity
2. Independence
3. Normality
4. Equal variance (homoskedasticity)

In simple linear regression (SLR), the mean of the response \(y\) is a straight line in one covariate \(x\).

The response \(y_i\) is the value we explain for observation \(i=1,\dots,n\), where \(n\) is the sample size.

The covariate \(x_i\) is the known constant that explains \(y_i\).

The intercept \(\beta_0\) is the mean of \(y\) at \(x=0\).

The slope \(\beta_1\) is the change in the mean of \(y\) per 1-unit increase in \(x\).

A parameter is an unknown constant of the model, for example \(\beta_0\) and \(\beta_1\).

The error \(\epsilon_i\) is the random difference between \(y_i\) and the line \(\beta_0+\beta_1x_i\).

\(N(\mu,\sigma^2)\) is the normal distribution: bell-shaped, with mean \(\mu\) and variance \(\sigma^2\).

The expectation \(E[\cdot]\) is the long-run average of a random variable. The variance \(\mathrm{Var}[\cdot]\) is the expected squared distance from it.

The error variance \(\sigma^2\) is the third parameter. \(\sigma\) is the error standard deviation.

"iid" means independent and identically distributed.

"indep" means independent, not identically distributed, because the mean changes with \(x_i\).

1. Linearity

\(E[y_i]=\beta_0+\beta_1x_i\).

2. Independence

The errors \(\epsilon_i\) are independent.

3. Normality

Each \(\epsilon_i\) has a normal distribution.

4. Equal variance (homoskedasticity)

Each \(\epsilon_i\) has variance \(\sigma^2\), at all \(x_i\).

**How** Added

Write a data set as an SLR model

- Find the response \(y\): the variable to explain or predict.
- Find the covariate \(x\): the variable that explains \(y\).
- Count the observations to get \(n\).
- Write \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,n\).
- Write \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\). List the four assumptions.

**Self-check:** \(\beta_0\), \(\beta_1\), \(\sigma^2\) are unknown. The data give \(x_i\), \(y_i\), \(n\).

**Example 1 · The small data set** Added

Pairs: \((1,2),(2,4),(3,5),(4,4),(5,5)\).

Step 1: \(y\): the second value of each pair.

Step 2: \(x\): the first value of each pair.

Step 3: Five pairs give \(n=5\).

Step 4: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,5\).

Step 5: \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\). For observation 3, \(y_3\sim N(\beta_0+3\beta_1,\sigma^2)\).

**Example 2 · The brainhead data** Slides p.7

Step 1: \(y_i\): brain weight of person \(i\) (g).

Step 2: \(x_i\): head size of person \(i\) (cm³).

Step 3: Slides p.7 prints \(n-2=234\) degrees of freedom. \(n=234+2=236\).

Step 4: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,236\).

Step 5: \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\).

[figure]
Figure 1-1. The 236 brainhead points (blue) and the fitted line \(\hat y=335.43+0.26082\,x\) (orange). The points follow a line with almost equal spread.

**Why · The two forms are equal** Added

1

\[y_i=\beta_0+\beta_1x_i+\epsilon_i\]

2

\[E[y_i]=\beta_0+\beta_1x_i+E[\epsilon_i]\]

\(\beta_0+\beta_1x_i\) is a constant.

3

\[E[y_i]=\beta_0+\beta_1x_i+0=\beta_0+\beta_1x_i\]

\(\epsilon_i\sim N(0,\sigma^2)\) has mean 0.

4

\[\mathrm{Var}[y_i]=\mathrm{Var}[\epsilon_i]\]

A constant does not change the variance.

5

\[\mathrm{Var}[y_i]=\sigma^2\]

6

\[y_i\sim N(\beta_0+\beta_1x_i,\ \sigma^2)\]

A normal variable plus a constant is normal.

7

\[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\]

The \(\epsilon_i\) are independent. ∎

### 01.3 The least squares estimators and the distribution of \(\hat\beta_1\) Slides p.4

**What** Slides p.4

Recap: Estimates · Lecture 4 · p.4 So the least squares estimators are: \[\hat\beta_1=\frac{S_{xy}}{S_{xx}},\qquad \hat\beta_0=\bar y-\hat\beta_1\bar x.\] \[\hat\beta_1\sim N\!\left(\beta_1,\frac{\sigma^2}{S_{xx}}\right)\ \Longrightarrow\ \frac{\hat\beta_1-\beta_1}{\sigma/\sqrt{S_{xx}}}\sim N(0,1)\ \Longrightarrow\ \frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]

An estimator is a formula that gives a parameter value from the data. A hat marks it: \(\hat\beta_1\).

Least squares (LS) chooses the \(\hat\beta_0\) and \(\hat\beta_1\) that make \(\sum_{i=1}^n(y_i-\hat\beta_0-\hat\beta_1x_i)^2\) smallest.

The sample means are \(\bar x=\frac1n\sum_{i=1}^n x_i\) and \(\bar y=\frac1n\sum_{i=1}^n y_i\).

\(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2\) is the sum of squared distances of \(x_i\) from \(\bar x\).

\(S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\) is the sum of the products of the two distances.

The fitted value is \(\hat y_i=\hat\beta_0+\hat\beta_1x_i\). The residual is \(e_i=y_i-\hat y_i\).

\(\hat\sigma\) estimates \(\sigma\), with \(\hat\sigma^2=\sum_{i=1}^n e_i^2/(n-2)\).

The standard error \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\) estimates the standard deviation of \(\hat\beta_1\).

\(N(0,1)\) is the standard normal distribution: mean 0, variance 1.

\(t_{n-2}\) is the \(t\) distribution with \(n-2\) degrees of freedom: symmetric about 0, with wider tails than \(N(0,1)\).

Putting \(\hat\sigma\) for the unknown \(\sigma\) changes \(N(0,1)\) to \(t_{n-2}\).

**How** Added

Calculate \(\hat\beta_0\), \(\hat\beta_1\), \(\hat\sigma\), and \(\mathrm{SE}(\hat\beta_1)\)

- Calculate \(\bar x\) and \(\bar y\).
- Calculate \(S_{xx}\) and \(S_{xy}\).
- Calculate \(\hat\beta_1=S_{xy}/S_{xx}\).
- Calculate \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
- Calculate each fitted value \(\hat y_i\) and each residual \(e_i\).
- Calculate \(\hat\sigma^2=\sum e_i^2/(n-2)\) and \(\hat\sigma\).
- Calculate \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\).

**Self-check:** The residuals add to 0. The line contains \((\bar x,\bar y)\).

**Example 3 · The small data set** Added

Step 1: \(\bar x=(1+2+3+4+5)/5=15/5=3\). \(\bar y=(2+4+5+4+5)/5=20/5=4\).

Step 2: \(x_i-\bar x\): \(-2,-1,0,1,2\). \(y_i-\bar y\): \(-2,0,1,0,1\).

\(S_{xx}=4+1+0+1+4=10\).

\(S_{xy}=(-2)(-2)+(0)(-1)+(1)(0)+(0)(1)+(1)(2)=4+0+0+0+2=6\).

Step 3: \(\hat\beta_1=6/10=0.6\).

Step 4: \(\hat\beta_0=4-0.6\times3=4-1.8=2.2\).

Step 5: \(\hat y_i=2.2+0.6x_i\): \(2.8,\ 3.4,\ 4.0,\ 4.6,\ 5.2\).

\(e_i=y_i-\hat y_i\): \(-0.8,\ 0.6,\ 1.0,\ -0.6,\ -0.2\).

Step 6: \(\sum e_i^2=0.64+0.36+1.00+0.36+0.04=2.4\).

\(\hat\sigma^2=2.4/(5-2)=2.4/3=0.8\), and \(\hat\sigma=\sqrt{0.8}=0.894427\).

Step 7: \(\mathrm{SE}(\hat\beta_1)=0.894427/\sqrt{10}=0.894427/3.162278=0.282843\).

Self-check: \(-0.8+0.6+1.0-0.6-0.2=0\), and \(2.2+0.6\times3=4=\bar y\). With \(n-2=3\):
\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}=\frac{0.6-\beta_1}{0.282843}\sim t_{3}.\]
**Example 4 · The brainhead data** Slides p.7

\(\hat\beta_0=335.43231\) and \(\hat\beta_1=0.26082\).

\(\mathrm{SE}(\hat\beta_1)=0.01307\).

\(\hat\sigma=72.35\), with \(n-2=234\) degrees of freedom.

Each 1 cm³ more head size increases the estimated mean brain weight by 0.26082 g.
\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}=\frac{0.26082-\beta_1}{0.01307}\sim t_{234}.\]
**Why · The distribution chain** Added

1

\[\hat\beta_1=\frac{\sum_i(y_i-\bar y)(x_i-\bar x)}{S_{xx}}\]

2

\[\hat\beta_1=\frac{\sum_i y_i(x_i-\bar x)-\bar y\sum_i(x_i-\bar x)}{S_{xx}}\]

3

\[\hat\beta_1=\frac{\sum_i y_i(x_i-\bar x)-\bar y\cdot 0}{S_{xx}}\]

\(\sum_i(x_i-\bar x)=n\bar x-n\bar x=0\).

4

\[\hat\beta_1=\sum_{i=1}^n w_iy_i,\qquad w_i=\frac{x_i-\bar x}{S_{xx}}\]

\(w_i\) uses only \(x\): a constant.

5

\[\hat\beta_1\sim N\!\left(\sum_i w_i(\beta_0+\beta_1x_i),\ \sigma^2\sum_i w_i^2\right)\]

A weighted sum of independent normals is normal.

6

\[E[\hat\beta_1]=\beta_0\sum_i w_i+\beta_1\sum_i w_ix_i\]

7

\[E[\hat\beta_1]=\beta_0\cdot0+\beta_1\frac{\sum_i(x_i-\bar x)x_i}{S_{xx}}\]

\(\sum_i w_i=\sum_i(x_i-\bar x)/S_{xx}=0\).

8

\[E[\hat\beta_1]=\beta_1\frac{\sum_i(x_i-\bar x)(x_i-\bar x)}{S_{xx}}\]

\(\sum_i(x_i-\bar x)\bar x=0\).

9

\[E[\hat\beta_1]=\beta_1\frac{S_{xx}}{S_{xx}}=\beta_1\]

10

\[\mathrm{Var}[\hat\beta_1]=\sigma^2\sum_i\frac{(x_i-\bar x)^2}{S_{xx}^2}\]

11

\[\mathrm{Var}[\hat\beta_1]=\sigma^2\frac{S_{xx}}{S_{xx}^2}=\frac{\sigma^2}{S_{xx}}\]

12

\[\hat\beta_1\sim N\!\left(\beta_1,\frac{\sigma^2}{S_{xx}}\right)\]

The first result on Slides p.4.

13

\[Z=\frac{\hat\beta_1-\beta_1}{\sigma/\sqrt{S_{xx}}}\sim N(0,1)\]

Subtract the mean. Divide by the standard deviation.

14

\[V=\frac{1}{\sigma^2}\sum_{i=1}^n e_i^2\sim\chi^2_{(n-2)},\quad Z,V\text{ independent}\]

Stated in Lecture 3. \(\chi^2_\nu\): the chi-squared distribution.

15

\[\frac{Z}{\sqrt{V/(n-2)}}\sim t_{n-2}\]

Independent \(Z\sim N(0,1)\), \(V\sim\chi^2_\nu\) give \(Z/\sqrt{V/\nu}\sim t_\nu\).

16

\[\frac{(\hat\beta_1-\beta_1)\big/(\sigma/\sqrt{S_{xx}})}{\sqrt{\frac{1}{\sigma^2}\sum_i e_i^2/(n-2)}}\sim t_{n-2}\]

17

\[\frac{(\hat\beta_1-\beta_1)\big/(1/\sqrt{S_{xx}})}{\sqrt{\sum_i e_i^2/(n-2)}}\sim t_{n-2}\]

18

\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]

\(\hat\sigma=\sqrt{\sum_i e_i^2/(n-2)}\). ∎

### 01.4 The confidence interval for \(\beta_1\) Slides p.5

**What** Slides p.5

Confidence interval for \(\beta_1\) · Lecture 4 · p.5 95% CI for \(\beta_1\) is \(\hat\beta_1\pm t_{0.975,n-2}\times\sqrt{\dfrac{\hat\sigma^2}{S_{xx}}}\)
or \(\hat\beta_1\pm t_{0.975,n-2}\times \mathrm{SE}(\hat\beta_1)\).
If we were to collect a new sample, we would obtain a different estimate and a different 95% CI

- 95% of such intervals will cover the true \(\beta_1\)
More generally a \(100\times(1-\alpha)\%\) CI is: \(\hat\beta_1\pm t_{1-\alpha/2,n-2}\times \mathrm{SE}(\hat\beta_1)\)

A confidence interval (CI) is an interval calculated from the data with a fixed rule.

The confidence level \(100\times(1-\alpha)\%\) is the proportion of such intervals that cover the true parameter. For 95%, \(\alpha=0.05\).

The quantile \(t_{1-\alpha/2,n-2}\) has area \(1-\alpha/2\) to its left under the \(t_{n-2}\) curve.

For 95%, the quantile is \(t_{0.975,n-2}\), with right-tail area 0.025.

\(\sqrt{\hat\sigma^2/S_{xx}}=\hat\sigma/\sqrt{S_{xx}}=\mathrm{SE}(\hat\beta_1)\).

**How** Added

Calculate a \(100\times(1-\alpha)\%\) CI for \(\beta_1\)

- Get \(\hat\beta_1\) and \(\mathrm{SE}(\hat\beta_1)\).
- Calculate the degrees of freedom \(n-2\).
- Find \(t_{1-\alpha/2,n-2}\) in a \(t\) table or with software.
- Calculate the half-width \(t_{1-\alpha/2,n-2}\times\mathrm{SE}(\hat\beta_1)\).
- Calculate \(\hat\beta_1-\text{half-width}\) and \(\hat\beta_1+\text{half-width}\).

**Self-check:** The midpoint is \(\hat\beta_1\). A higher confidence level gives a wider interval.

**Example 5 · 95% CI for the small data set** Added

Step 1: \(\hat\beta_1=0.6\) and \(\mathrm{SE}(\hat\beta_1)=0.282843\) (Example 3).

Step 2: \(n-2=5-2=3\).

Step 3: \(t_{0.975,3}=3.182446\).

Step 4: Half-width \(=3.182446\times0.282843=0.900132\).

Step 5: \(0.6-0.900132=-0.300132\) and \(0.6+0.900132=1.500132\).

The 95% CI is \([-0.300132,\ 1.500132]\). It is wide because \(n=5\) is small and \(t_{0.975,3}\) is large.

**Example 6 · 95% CI for the brainhead data** Slides p.7

Step 1: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).

Step 2: \(n-2=236-2=234\).

Step 3: \(t_{0.975,234}=1.970154\).

Step 4: Half-width \(=1.970154\times0.01307=0.025750\).

Step 5: \(0.26082-0.025750=0.235070\) and \(0.26082+0.025750=0.286570\).

Slides p.7 prints \([0.2350725,\ 0.2865689]\) (more input digits). We are 95% confident that 1 cm³ more head size increases the mean brain weight by 0.2351 g to 0.2866 g.

[figure]
Figure 1-2. The \(t_{234}\) density: center area 0.95 (blue), each tail 0.025 (orange). Right boundary: \(t_{0.975,234}=1.970154\).

**Why · The interval formula** Added

Let \(q=t_{0.975,n-2}\) and \(T\sim t_{n-2}\).

1

\[0.95=P(-q\le T\le q)\]

\(t\) is symmetric. Each tail has area 0.025.

2

\[0.95=P\!\left(-q\le\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\le q\right)\]

The \(t_{n-2}\) result of Section 01.3.

3

\[0.95=P\!\left(-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\hat\beta_1-\beta_1\le q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]

4

\[0.95=P\!\left(-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\beta_1-\hat\beta_1\le q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]

5

\[0.95=P\!\left(\hat\beta_1-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\beta_1\le\hat\beta_1+q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]

6

\[\hat\beta_1\pm t_{0.975,n-2}\times\mathrm{SE}(\hat\beta_1)\]

Use \(1-\alpha/2\) for 0.975 in general. ∎

In line 5, \(\hat\beta_1\) and \(\hat\sigma\) are random; \(\beta_1\) is fixed. 0.95 is the probability that the interval covers \(\beta_1\).

### 01.5 The hypothesis test for \(\beta_1\) Slides p.6

**What** Slides p.6

Hypothesis Testing · Lecture 4 · p.6 Want to test \(H_0:\beta_1=\theta_0\) vs \(H_1:\beta_1\neq\theta_0\).
Under \(H_0\): \(\dfrac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\)

- P-value: What's the probability under the null of observing a test statistic as or more extreme than the one we observed?

- I.e. for \(T\sim t_{n-2}\) and our observed test statistic \(t^{obs}\): \(P(|T|\ge|t^{obs}|)\)

A hypothesis test decides if the data agree with a statement about a parameter.

The null hypothesis \(H_0:\beta_1=\theta_0\) is the tested statement. \(\theta_0\) is given, usually 0 (no linear relation).

The alternative hypothesis \(H_1:\beta_1\neq\theta_0\) is the statement if \(H_0\) is false. \(\neq\) makes the test two-sided.

The test statistic is \(\dfrac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\). \(t^{obs}\) is its value for our data.

The p-value is the probability, under \(H_0\), of a test statistic as extreme as \(t^{obs}\) or more.

A small p-value is evidence against \(H_0\).

**How** Added

Test \(H_0:\beta_1=\theta_0\) vs \(H_1:\beta_1\neq\theta_0\)

- Write \(H_0\) and \(H_1\). Find \(\theta_0\).
- Get \(\hat\beta_1\) and \(\mathrm{SE}(\hat\beta_1)\).
- Calculate \(t^{obs}=(\hat\beta_1-\theta_0)/\mathrm{SE}(\hat\beta_1)\).
- Calculate the p-value \(2P(T\ge|t^{obs}|)\), with \(T\sim t_{n-2}\).
- Compare the p-value with the significance level \(\alpha\), usually 0.05.
- If the p-value < \(\alpha\), reject \(H_0\). If not, do not reject it.

**Self-check:** For \(\theta_0=0\), the 95% CI excludes 0 exactly when the p-value < 0.05.

**Example 7 · Test \(H_0:\beta_1=0\) for the small data set** Added

Step 1: \(H_0:\beta_1=0\) vs \(H_1:\beta_1\neq0\). \(\theta_0=0\).

Step 2: \(\hat\beta_1=0.6\) and \(\mathrm{SE}(\hat\beta_1)=0.282843\).

Step 3: \(t^{obs}=(0.6-0)/0.282843=2.121320\).

Step 4: \(2P(T\ge2.121320)=0.124027\), with \(T\sim t_3\).

Step 5–6: \(0.124027>0.05\). Do not reject \(H_0\).

The 95% CI of Example 5, \([-0.300132,\ 1.500132]\), also contains 0.

**Example 8 · Test \(H_0:\beta_1=0\) for the brainhead data** Slides p.7

Step 1: \(H_0:\beta_1=0\) vs \(H_1:\beta_1\neq0\). \(\theta_0=0\).

Step 2: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).

Step 3: \(t^{obs}=(0.26082-0)/0.01307=19.956\). Slides p.7 prints 19.957 (more input digits).

Step 4: \(2P(T\ge19.957)=2.03\times10^{-52}\), \(T\sim t_{234}\). Slides p.7 prints "< \(2\times10^{-16}\)".

Step 5–6: The p-value is much less than 0.05. Reject \(H_0\).

[figure]
Figure 1-3. The \(t_{234}\) density of \(T\) under \(H_0\), almost all between \(-4\) and 4. \(t^{obs}=19.957\) is far right; its tail area is too small to see.

Head size and brain weight have a linear relation. The 95% CI \([0.2351,\ 0.2866]\) also excludes 0.

**Why · The test statistic has the \(t_{n-2}\) distribution** Added

1

\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]

Section 01.3, line 18. True for each \(\beta_1\).

2

\[\frac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\quad\text{under }H_0\]

\(H_0\) gives \(\beta_1=\theta_0\).

3

\[P(|T|\ge|t^{obs}|)=P(T\ge|t^{obs}|)+P(T\le-|t^{obs}|)\]

4

\[P(|T|\ge|t^{obs}|)=2P(T\ge|t^{obs}|)\]

\(t\) is symmetric about 0. ∎

### 01.6 The brainhead results on Slides p.7 Slides p.7

**What** Slides p.7

Intercept: estimate \(\hat\beta_0=335.43231\), standard error 47.77644, \(t\) value 7.021, p-value \(2.37\times10^{-11}\).

Slope: estimate \(\hat\beta_1=0.26082\), standard error 0.01307, \(t\) value 19.957, p-value less than \(2\times10^{-16}\).

\(\hat\sigma=72.35\) on 234 degrees of freedom.

95% CI for \(\beta_0\): \([241.3053884,\ 429.5592416]\).

95% CI for \(\beta_1\): \([0.2350725,\ 0.2865689]\).

Later lectures explain the numbers below.

Residuals: minimum \(-174.88\), first quartile \(-49.03\), median \(-2.40\), third quartile 46.72, maximum 242.41.

\(R^2=0.6299\) and adjusted \(R^2=0.6283\).

F-statistic 398.3 on 1 and 234 degrees of freedom, p-value less than \(2.2\times10^{-16}\).

**How** Added

Connect the printed numbers to the formulas

- Match each estimate to \(\hat\beta_0\) or \(\hat\beta_1\).
- Match the slope standard error to \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\).
- Divide each estimate by its standard error: \(t^{obs}\) for \(\theta_0=0\).
- Match each p-value to \(P(|T|\ge|t^{obs}|)\), \(T\sim t_{n-2}\).
- Add 2 to the degrees of freedom: \(n\).
- Compare the midpoint of each CI with the estimate.

**Self-check:** Each \(t\) value is the estimate divided by its standard error.

**Example 9 · Check each number** Slides p.7

Step 1: \(\hat\beta_0=335.43231\) and \(\hat\beta_1=0.26082\).

Step 2: \(\mathrm{SE}(\hat\beta_1)=0.01307\).

Step 3: Slope: \(0.26082/0.01307=19.956\). Slides p.7 prints 19.957 (more input digits).

Intercept: \(335.43231/47.77644=7.021\), as printed.

Step 4: \(2P(T\ge7.021)=2.37\times10^{-11}\) with \(T\sim t_{234}\), as printed.

Step 5: \(n=234+2=236\).

Step 6: Slope CI midpoint: \((0.2350725+0.2865689)/2=0.5216414/2=0.2608207\). This rounds to 0.26082.

Intercept CI midpoint: \((241.3053884+429.5592416)/2=670.8646300/2=335.432315\). This rounds to 335.43231.

### 01.7 Practice Added

**Q1.** From Slides p.7, calculate a 90% CI for \(\beta_1\). Use \(t_{0.95,234}=1.651391\). Interpret it.

Answer

Step 1: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).

Step 2: \(\alpha=0.10\) and \(1-\alpha/2=0.95\). The quantile is \(t_{0.95,234}=1.651391\).

Step 3: Half-width \(=1.651391\times0.01307=0.021584\).

Step 4: \(0.26082-0.021584=0.239236\) and \(0.26082+0.021584=0.282404\).

The 90% CI is \([0.239236,\ 0.282404]\). We are 90% confident that 1 cm³ more head size increases the mean brain weight by 0.2392 g to 0.2824 g. It is narrower than the 95% CI because 1.651391 < 1.970154.

**Q2.** Test \(H_0:\beta_1=0.25\) vs \(H_1:\beta_1\neq0.25\) at the 5% level. Report \(t^{obs}\), the p-value, and your conclusion.

Answer

Step 1: \(\theta_0=0.25\).

Step 2: \(t^{obs}=(0.26082-0.25)/0.01307=0.01082/0.01307=0.82785\).

Step 3: \(2P(T\ge0.82785)=0.4086\), with \(T\sim t_{234}\).

Step 4: \(0.4086>0.05\). Do not reject \(H_0\). The data agree with \(\beta_1=0.25\).

The 95% CI \([0.2351,\ 0.2866]\) also contains 0.25.

**Q3.** Verify the 95% CI for \(\beta_0\) on Slides p.7. Use \(t_{0.975,234}=1.970154\).

Answer

Step 1: \(\hat\beta_0=335.43231\), with standard error 47.77644.

Step 2: Half-width \(=1.970154\times47.77644=94.1269\).

Step 3: \(335.43231-94.1269=241.3054\) and \(335.43231+94.1269=429.5592\).

This agrees with \([241.3053884,\ 429.5592416]\) on Slides p.7.

## 02 · Point estimate of the mean response and its distribution

Plan · Slides p.8–18

\(\hat\mu_0\) and its second form (p.10–13), \(E[\hat\mu_0]\) (p.14–15), \(\mathrm{Var}[\hat\mu_0]\) (p.16–17), distribution (p.18).

Slides p.8 Page 8 repeats the Lecture 4 title page.

Slides p.9 Page 9 is the section page "Estimating Mean Response".

Model and notation from earlier · Slides p.3–4 (unit 01 defines each item)

Model: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), or \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\).

Estimators: \(\hat\beta_1=S_{xy}/S_{xx}\), \(\hat\beta_0=\bar y-\hat\beta_1\bar x\), with \(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2\), \(S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\).

An estimator is random. An estimate is its number for one data set.

Example data · Slides p.7 and Added

Brainhead data: \(n=236\) persons, \(x_i\) head size (cm³), \(y_i\) brain weight (g).

Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\), \(n-2=234\).

From brainhead.csv (Added): \(\bar x=3637.864\), \(\bar y=1284.263\), \(S_{xx}=30647233.66\), \(\hat\beta_1=0.2608207\).

### 02.1 The mean response \(\mu_0\) and its point estimate \(\hat\mu_0\) Slides p.10–13

**What** Slides p.10

Lecture 4 · p.10 Recall the mean response is \(E[y_i\mid x_i]=\beta_0+\beta_1x_i\).
For an arbitrary \(x_0\), the mean outcome is \(\mu_0:=E[y_0\mid x_0]=\beta_0+\beta_1x_0\).
We can estimate this as \[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]

The conditional expectation \(E[y\mid x]\) is the long-run average of \(y\) at one fixed \(x\).

The mean response \(E[y\mid x]\) is the height \(\beta_0+\beta_1x\) of the true line, because \(E[\epsilon]=0\).

\(x_0\) is a covariate value of interest, not necessarily in the data. \(y_0\) is the response at \(x_0\).

The symbol \(:=\) means "is defined as".

A point estimate is one number that estimates an unknown quantity. \(\hat\mu_0\) is the point estimate of \(\mu_0\).

\(\mu_0\) is unknown, because \(\beta_0\) and \(\beta_1\) are unknown. \(\hat\mu_0\), the height of the fitted line at \(x_0\), comes from the data.

Lecture 4 · p.11–13 \[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0=\left(\bar y-\hat\beta_1\bar x\right)+\hat\beta_1x_0=\bar y+\hat\beta_1(x_0-\bar x)\] Intuition?
• What happens at \(\bar x\)?

The second form starts at the center \((\bar x,\bar y)\). It moves along the fitted line by \(x_0-\bar x\), with height change \(\hat\beta_1\) per unit.

At \(x_0=\bar x\) (Slides p.13), the distance is 0, and \(\hat\mu_0=\bar y\).

**How** Added

Calculate \(\hat\mu_0\)

- Calculate \(n\), \(\bar x\), \(\bar y\), \(S_{xx}\) and \(S_{xy}\).
- Calculate \(\hat\beta_1=S_{xy}/S_{xx}\) and \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
- Form 1 (Slides p.10): put \(x_0\) into \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
- Form 2 (Slides p.12): calculate \(x_0-\bar x\).
- Calculate \(\bar y+\hat\beta_1(x_0-\bar x)\).

**Self-check:** ① Both forms agree, to rounding. ② At \(x_0=\bar x\), the result is \(\bar y\). ③ If \(\hat\beta_1>0\) and \(x_0>\bar x\), the result is above \(\bar y\).

**Example 1 · The brainhead data at \(x_0=4000\) cm³** Added

1

Start values How step 1

\[n=236,\quad \bar x=3637.864,\quad \bar y=1284.263,\quad S_{xx}=30647233.66\]

From brainhead.csv (Added).

2

Coefficients How step 2

\[\hat\beta_1=0.2608207,\qquad \hat\beta_0=335.4323\]

Basis: Slides p.4. Slides p.7 prints 335.43231 and 0.26082.

3

Form 1 How step 3 · Slides p.10

\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]

\[\hat\mu_0=335.4323+0.2608207\times4000\]

\[\hat\mu_0=335.4323+1043.2828=1378.7151\]

4

Form 2 How steps 4–5 · Slides p.12

\[x_0-\bar x=4000-3637.864=362.136\]

\[\hat\beta_1(x_0-\bar x)=0.2608207\times362.136=94.452\]

\[\hat\mu_0=\bar y+\hat\beta_1(x_0-\bar x)=1284.263+94.452=1378.715\]

**Self-check:** Both forms give 1378.715, above \(\bar y=1284.263\) (check ③).

5

At \(x_0=\bar x\) Slides p.13

\[\hat\mu_0=\bar y+\hat\beta_1(\bar x-\bar x)=\bar y+\hat\beta_1\times0=\bar y=1284.263\]

[figure]
Figure 2-1. Brainhead data (blue), fitted line (orange), center \((\bar x,\bar y)\) (purple), and \(\hat\mu_0=1378.715\) at \(x_0=4000\) (green). From purple, move right \(x_0-\bar x=362.1\); the height increases \(\hat\beta_1\times362.1=94.45\).

At 4000 cm³, the estimated mean brain weight is \(\hat\mu_0=1378.715\) g. At \(x_0=\bar x\), it is \(\bar y=1284.263\) g.

**Why · The two forms are equal** Slides p.10–13

1

\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]

Definition, Slides p.10.

2

\[=\left(\bar y-\hat\beta_1\bar x\right)+\hat\beta_1x_0\]

Slides p.4, p.11.

3

\[=\bar y+\hat\beta_1x_0-\hat\beta_1\bar x\]

4

\[=\bar y+\hat\beta_1(x_0-\bar x)\]

Slides p.12. ∎

At \(x_0=\bar x\) (Slides p.13):

1

\[\hat\mu_0=\bar y+\hat\beta_1(\bar x-\bar x)\]

2

\[=\bar y+\hat\beta_1\times0\]

3

\[=\bar y\]

The fitted line contains \((\bar x,\bar y)\). ∎

### 02.2 \(\hat\mu_0\) is unbiased Slides p.14–15

**What** Slides p.15

Lecture 4 · p.15 \[E[\hat\mu_0]=E[\hat\beta_0+\hat\beta_1x_0]=E[\hat\beta_0]+E[\hat\beta_1]x_0=\beta_0+\beta_1x_0\]

An estimator is unbiased when its expectation equals the parameter it estimates. \(E[\hat\mu_0]=\mu_0\).

Repeated sampling: keep \(x_1,\dots,x_n\), get new \(y_1,\dots,y_n\), and calculate \(\hat\mu_0\) again.

One \(\hat\mu_0\) can be too high or too low. On average over repeated samples, it is correct.

**How** Added

Find \(E[\hat\mu_0]\)

- Write \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
- Use linearity of expectation: \(E[aU+bV]=aE[U]+bE[V]\) for constants \(a\), \(b\).
- Treat \(x_0\) as a constant.
- Put in \(E[\hat\beta_0]=\beta_0\) and \(E[\hat\beta_1]=\beta_1\) (Lecture 3).

**Self-check:** The result has no hat. It is \(\beta_0+\beta_1x_0=\mu_0\).

**Example 2 · A small data set with known parameters** Added

Real data have unknown \(\beta_0\), \(\beta_1\). This example sets them: \(n=3\), \(x=(1,2,3)\), \(\beta_0=2\), \(\beta_1=0.5\), \(x_0=4\).

Write \(\hat\mu_0=\sum_{i=1}^n c_iy_i\), with weight \(c_i=\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\) (Section 02.3, line 6).

1

Center and spread of \(x\)

\[\bar x=\frac{1+2+3}{3}=\frac{6}{3}=2\]

\[S_{xx}=(1-2)^2+(2-2)^2+(3-2)^2=1+0+1=2\]

\[x_0-\bar x=4-2=2\]

2

Weights \(c_i\)

\[c_1=\frac13+\frac{(1-2)(2)}{2}=\frac13-1=-\frac23\]

\[c_2=\frac13+\frac{(2-2)(2)}{2}=\frac13+0=\frac13\]

\[c_3=\frac13+\frac{(3-2)(2)}{2}=\frac13+1=\frac43\]

3

Expected responses

\[E[y_1]=2+0.5\times1=2.5,\quad E[y_2]=2+0.5\times2=3,\quad E[y_3]=2+0.5\times3=3.5\]

\(E[y_i]=\beta_0+\beta_1x_i\) (Slides p.3).

4

Expectation of \(\hat\mu_0\)

\[E[\hat\mu_0]=c_1E[y_1]+c_2E[y_2]+c_3E[y_3]\]

\[=-\frac23\times2.5+\frac13\times3+\frac43\times3.5\]

\[=-\frac53+1+\frac{14}3=\frac{-5+3+14}{3}=\frac{12}{3}=4\]

5

True mean response

\[\mu_0=\beta_0+\beta_1x_0=2+0.5\times4=2+2=4\]

**Self-check:** \(E[\hat\mu_0]=4=\mu_0\).

**Why · \(\hat\mu_0\) is unbiased** Slides p.14–15

1

\[E[\hat\mu_0]=E[\hat\beta_0+\hat\beta_1x_0]\]

Slides p.10.

2

\[=E[\hat\beta_0]+E[\hat\beta_1x_0]\]

Linearity of expectation.

3

\[=E[\hat\beta_0]+E[\hat\beta_1]x_0\]

\(x_0\) is a constant.

4

\[=\beta_0+\beta_1x_0\]

Lecture 3.

5

\[=\mu_0\]

Slides p.10. ∎

### 02.3 The variance of \(\hat\mu_0\) Slides p.16–17

**What** Slides p.17

Lecture 4 · p.17 \[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

The denominator is \(S_{xx}\).

The first term, \(\sigma^2/n\), does not change with \(x_0\).

The second term, \(\sigma^2(x_0-\bar x)^2/S_{xx}\), increases as \(x_0\) moves away from \(\bar x\).

The variance is smallest, \(\sigma^2/n\), at \(x_0=\bar x\). Away from the center, the distance multiplies the error in \(\hat\beta_1\).

**How** Added

Calculate \(\mathrm{Var}[\hat\mu_0]\)

- Calculate \(1/n\).
- Calculate \((x_0-\bar x)^2/S_{xx}\).
- Add them to get the factor \(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\).
- Multiply the factor by \(\sigma^2\).
- For a number, use \(\hat\sigma\) for \(\sigma\) and take the square root.

**Self-check:** ① The factor is at least \(1/n\). ② At \(x_0=\bar x\), it is \(1/n\). ③ Equal distances from \(\bar x\) give equal variances.

Slides p.7 prints \(\hat\sigma=72.35\).

**Example 3 · The brainhead data at \(x_0=4000\) and \(x_0=\bar x\)** Added

1

\(x_0=4000\): the two terms How steps 1–2

\[\frac1n=\frac1{236}=0.004237288\]

\[(x_0-\bar x)^2=362.1356^2=131142.19\]

\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{131142.19}{30647233.66}=0.004279087\]

2

\(x_0=4000\): the factor How step 3

\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}=0.004237288+0.004279087=0.008516375\]

3

\(x_0=4000\): multiply by \(\sigma^2\) How steps 4–5

\[\mathrm{Var}[\hat\mu_0]=\sigma^2\times0.008516375\]

\[\sqrt{0.008516375}=0.09228421\]

\[\hat\sigma\sqrt{0.008516375}=72.35\times0.09228421=6.677\]

4

\(x_0=\bar x\)

\[\frac1n+\frac{(\bar x-\bar x)^2}{S_{xx}}=\frac1n+0=0.004237288\]

\[\sqrt{236}=15.36229\]

\[\hat\sigma\sqrt{1/n}=\frac{72.35}{15.36229}=4.710\]

**Self-check:** 4.710 < 6.677. The estimate is most precise at the center.

[figure]
Figure 2-2. \(\hat\sigma\sqrt{1/n+(x_0-\bar x)^2/S_{xx}}\) for the brainhead data. Minimum \(\hat\sigma/\sqrt n=4.710\) at \(x_0=\bar x\); 6.677 at \(x_0=4000\).

**Example 4 · The small data set of Example 2** Added

Calculate \(\mathrm{Var}[\hat\mu_0]\) in two ways for \(x=(1,2,3)\), \(x_0=4\).

1

Formula from Slides p.17

\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}=\frac13+\frac{2^2}{2}=\frac13+2=\frac73\]

2

Sum of squared weights

\[\sum_{i=1}^3c_i^2=\left(-\frac23\right)^2+\left(\frac13\right)^2+\left(\frac43\right)^2=\frac49+\frac19+\frac{16}9=\frac{21}9=\frac73\]

Line 7 below: \(\mathrm{Var}[\hat\mu_0]=\sigma^2\sum c_i^2\).

**Self-check:** Both give \(\mathrm{Var}[\hat\mu_0]=\frac73\sigma^2\).

**Why · The chain for the variance** Slides p.16–17

The chain follows Slides p.17, with one change per line.

Fact A (model): \(y_1,\dots,y_n\) are independent, and \(\mathrm{Var}[y_i]=\sigma^2\).

Fact B (Lecture 2): \(\mathrm{Var}(aY)=a^2\mathrm{Var}(Y)\). For independent \(X\) and \(Y\), \(\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)\).

Fact C: \(\sum_{i=1}^n(x_i-\bar x)=\sum_{i=1}^n x_i-n\bar x=n\bar x-n\bar x=0\).

Fact D (Lecture 3): \(\hat\beta_1=\sum_{i=1}^n w_iy_i\), with fixed \(w_i=\frac{x_i-\bar x}{S_{xx}}\).

1

\[\mathrm{Var}[\hat\mu_0]=\mathrm{Var}[\hat\beta_0+\hat\beta_1x_0]\]

Slides p.10.

2

\[=\mathrm{Var}[(\bar y-\hat\beta_1\bar x)+\hat\beta_1x_0]\]

Put in \(\hat\beta_0\).

3

\[=\mathrm{Var}[\bar y+\hat\beta_1(x_0-\bar x)]\]

As in Section 02.1.

4

\[=\mathrm{Var}\left[\left(\sum_{i=1}^n\frac1n y_i\right)+\hat\beta_1(x_0-\bar x)\right]\]

Put in \(\bar y=\sum_{i=1}^n\frac1n y_i\). (Added: split.)

5

\[=\mathrm{Var}\left[\left(\sum_{i=1}^n\frac1n y_i\right)+\left(\sum_{i=1}^n\frac{x_i-\bar x}{S_{xx}}y_i\right)(x_0-\bar x)\right]\]

Fact D.

6

\[=\mathrm{Var}\left[\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)y_i\right]\]

Join the two sums.

7

\[=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)^2\mathrm{Var}[y_i]\]

Facts A and B. (Added: split.)

8

\[=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)^2\sigma^2\]

Fact A.

9

\[=\sigma^2\sum_{i=1}^n\left(\frac1{n^2}+\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\,\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

\((a+b)^2=a^2+b^2+2ab\).

10

\[=\sigma^2\left(\sum_{i=1}^n\frac1{n^2}+\sum_{i=1}^n\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Split into three sums.

11

\[=\sigma^2\left(\frac1n+\sum_{i=1}^n\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

\(\sum_{i=1}^n\frac1{n^2}=n\cdot\frac1{n^2}=\frac1n\). (Added: split.)

12

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}\sum_{i=1}^n(x_i-\bar x)^2+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Move the constant out. (Added: split.)

13

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Definition of \(S_{xx}\). (Added: split.)

14

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\,\frac1n\,\frac{(x_0-\bar x)}{S_{xx}}\sum_{i=1}^n(x_i-\bar x)\right)\]

Move the constant out. Second-last line, Slides p.17.

15

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\,\frac1n\,\frac{(x_0-\bar x)}{S_{xx}}\cdot0\right)\]

Fact C. (Added: split.)

16

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\]

\(S_{xx}/S_{xx}^2=1/S_{xx}\). (Added: split.)

17

\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

Last line, Slides p.17. ∎

Number check (Added): the three sums of line 10 for \(x=(1,2,3)\), \(x_0=4\), \(\bar x=2\), \(S_{xx}=2\), \(x_0-\bar x=2\).

First: \(\sum_{i=1}^3\frac1{3^2}=3\times\frac19=\frac13\).

Second: \(\frac{(1)(4)+(0)(4)+(1)(4)}{2^2}=\frac{8}{4}=2\), equal to \((x_0-\bar x)^2/S_{xx}=4/2=2\).

Third: \(2\times\frac13\times\frac{(-1)(2)+(0)(2)+(1)(2)}{2}=2\times\frac13\times\frac{0}{2}=0\).

Total: \(\frac13+2+0=\frac73\), as in Example 4.

### 02.4 The distribution of \(\hat\mu_0\) Slides p.18

**What** Slides p.18

Lecture 4 · p.18 \[\Longrightarrow\ \hat\mu_0\sim N\!\left[\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right]\qquad\text{(why?)}\]

The symbol \(\sim\) means "has the distribution".

A linear combination is \(\sum_{i=1}^n c_iy_i\) with fixed numbers \(c_i\).

**How** Added

Write the distribution of \(\hat\mu_0\)

- Family: \(\hat\mu_0\) is a linear combination of independent normal \(y_i\). It is normal.
- Mean: \(E[\hat\mu_0]=\mu_0=\beta_0+\beta_1x_0\) (Section 02.2).
- Variance: \(\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\) (Section 02.3).
- Put in \(n\), \(\bar x\), \(S_{xx}\) and \(x_0\).

**Self-check:** The mean has no hat. The only unknown in the variance is \(\sigma^2\).

**Example 5 · The brainhead data at \(x_0=4000\)** Added

1

Family How step 1

\[\hat\mu_0=\sum_{i=1}^{236}\left(\frac1{236}+\frac{(x_i-3637.864)(362.136)}{30647233.66}\right)y_i\]

Fixed weights, independent normal \(y_i\): normal.

2

Mean How step 2

\[E[\hat\mu_0]=\mu_0=\beta_0+4000\,\beta_1\]

3

Variance How steps 3–4

\[\hat\mu_0\sim N\!\left[\beta_0+4000\,\beta_1,\ \sigma^2\times0.008516375\right]\]

Factor 0.008516375: Example 3. Data value: \(\hat\mu_0=1378.715\).

**Why · \(\hat\mu_0\) is normal (the "why?" on Slides p.18)** Slides p.18

1

\[\hat\mu_0=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)y_i\]

Section 02.3, line 6. Each weight uses only \(n\), \(x_i\), \(\bar x\), \(x_0\), \(S_{xx}\): a fixed number.

2

\[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\]

Slides p.3.

3

\[\hat\mu_0\ \text{is a linear combination of independent normals}\ \Longrightarrow\ \hat\mu_0\ \text{is normal}\]

Same rule as for \(\hat\beta_1=\sum w_iy_i\) (Lecture 3).

4

\[E[\hat\mu_0]=\mu_0\]

Section 02.2 (Slides p.15).

5

\[\hat\mu_0\sim N\!\left[\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right]\]

Variance: Section 02.3 (Slides p.17). ∎

### 02.5 Practice Added

**Q1.** Brainhead data: \(n=236\), \(\bar x=3637.864\), \(\bar y=1284.263\), \(\hat\beta_0=335.4323\), \(\hat\beta_1=0.2608207\). Calculate \(\hat\mu_0\) at \(x_0=3000\) cm³ in two ways: (a) \(\hat\beta_0+\hat\beta_1x_0\); (b) \(\bar y+\hat\beta_1(x_0-\bar x)\).

Answer

(a) \(0.2608207\times3000=782.4621\). \(\hat\mu_0=335.4323+782.4621=1117.894\).

(b) \(x_0-\bar x=3000-3637.864=-637.864\).

\(\hat\beta_1(x_0-\bar x)=0.2608207\times(-637.864)=-166.368\).

\(\hat\mu_0=1284.263-166.368=1117.895\). This agrees with (a) to rounding (1117.894).

\(x_0<\bar x\) and \(\hat\beta_1>0\): the result is below \(\bar y\).

**Q2.** Show that \(\mathrm{Var}[\hat\mu_0]\) is smallest at \(x_0=\bar x\). Give its value there in terms of \(\sigma^2\) and \(n\).

Answer

\(\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\).

\(\sigma^2/n\) does not change with \(x_0\). \((x_0-\bar x)^2\geq0\) and \(S_{xx}>0\). The second term is 0 only at \(x_0=\bar x\), and positive elsewhere.

At \(x_0=\bar x\):

\[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(\bar x-\bar x)^2}{S_{xx}}\right)=\sigma^2\left(\frac1n+0\right)=\frac{\sigma^2}{n}\]

There \(\hat\mu_0=\bar y\), and \(\mathrm{Var}[\bar y]=\sigma^2/n\) agrees.

**Q3.** Brainhead data, \(S_{xx}=30647233.66\). Calculate \(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\) at \(x_0=3000\) and \(x_0=4500\). Where is the estimate of \(\mu_0\) more precise? Use the distance to \(\bar x\).

Answer

\(x_0=3000\): \(x_0-\bar x=-637.8644\), \((x_0-\bar x)^2=406871.0\), \(406871.0/30647233.66=0.01327595\). Factor: \(0.004237288+0.01327595=0.01751323\).

\(x_0=4500\): \(x_0-\bar x=862.1356\), \((x_0-\bar x)^2=743277.8\), \(743277.8/30647233.66=0.02425269\). Factor: \(0.004237288+0.02425269=0.02848998\).

0.01751323 < 0.02848998: the estimate is more precise at \(x_0=3000\).

3000 is 637.9 from \(\bar x\); 4500 is 862.1 from \(\bar x\). The variance increases with \((x_0-\bar x)^2\).

## 03 · Confidence interval for the mean response

Start point · Slides p.3, p.7, p.10–18

Brainhead data, \(n=236\). Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\) on 234 degrees of freedom.

More digits from brainhead.csv (Added): \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\).

Start result (Slides p.18):
\[\hat\mu_0\sim N\!\left(\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right)\]

### 03.1 Standardize: from \(N(0,1)\) to \(t_{n-2}\) Slides p.19

**What** Slides p.19

Lecture 4 · p.19 So \[\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim N(0,1)\] And by the same logic as before: \[\frac{\hat\mu_0-\mu_0}{\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim t_{n-2}\]

To standardize a normal quantity, subtract its mean and divide by its standard deviation.

The first line contains the unknown \(\sigma\). We cannot calculate it from data.

The second line uses \(\hat\sigma\) for \(\sigma\). This changes \(N(0,1)\) to \(t_{n-2}\).

\(\mathrm{SE}(\hat\mu_0)=\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\) estimates the standard deviation of \(\hat\mu_0\).

**How** Added

Calculate the standard error of \(\hat\mu_0\)

- Calculate \(n\), \(\bar x\) and \(S_{xx}\).
- Calculate \(\hat\beta_0\), \(\hat\beta_1\), the residuals \(e_i\) and \(\hat\sigma\).
- Calculate \((x_0-\bar x)^2\).
- Divide by \(S_{xx}\).
- Add \(1/n\).
- Take the square root.
- Multiply by \(\hat\sigma\).

**Self-check:** ① The unit of SE is the unit of \(y\) (g). ② The step 5 sum is much less than 1. SE must be much less than \(\hat\sigma\).

**Example 1 · Standard error at \(x_0=4000\)** Added

Steps 1–2 come from the start point.

1

Distance from \(\bar x\) How step 3

\[x_0-\bar x=4000-3637.864407=362.1355932\]

\[(x_0-\bar x)^2=362.1355932^2=131142.1879\]

2

Divide by \(S_{xx}\) How step 4

\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{131142.1879}{30647233.66}=0.004279087285\]

3

Add \(1/n\) How step 5

\[\frac1n=\frac1{236}=0.004237288136\]

\[0.004237288136+0.004279087285=0.008516375421\]

4

Square root How step 6

\[\sqrt{0.008516375421}=0.09228421003\]

5

Multiply by \(\hat\sigma\) How step 7

\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.09228421003=6.676823721\]

6.676823721 g is much less than \(\hat\sigma=72.35066237\) g. ▲

**Why** Added

Same logic as Lecture 3 p.31 for \(\hat\beta_1\). Let \(c_0=\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\).

1

\[\hat\mu_0\sim N\!\left(\mu_0,\ \sigma^2c_0\right)\]

Slides p.18.

2

\[\hat\mu_0-\mu_0\sim N\!\left(0,\ \sigma^2c_0\right)\]

3

\[Z=\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{c_0}}\sim N(0,1)\]

First line of Slides p.19.

4

\[V=\frac1{\sigma^2}\sum_{i=1}^n e_i^2\sim\chi^2_{(n-2)}\]

Lecture 3 p.31: \(Z\), \(V\) independent.

5

\[\frac{Z}{\sqrt{V/(n-2)}}\sim t_{n-2}\]

Lecture 3 p.31.

6

\[\sqrt{\frac{V}{n-2}}=\sqrt{\frac{\sum_{i=1}^n e_i^2}{\sigma^2(n-2)}}=\frac{\hat\sigma}{\sigma}\]

7

\[\frac{Z}{\sqrt{V/(n-2)}}=\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{c_0}}\cdot\frac{\sigma}{\hat\sigma}\]

8

\[\frac{\hat\mu_0-\mu_0}{\hat\sigma\sqrt{c_0}}\sim t_{n-2}\]

Second line of Slides p.19. ∎

Two random quantities are independent when the value of one gives no information about the other.

### 03.2 The \(100(1-\alpha)\%\) confidence interval for \(\mu_0\) Slides p.20

**What** Slides p.20

Lecture 4 · p.20 \[\Longrightarrow\ 1-\alpha=P\!\left(-t_{1-\frac\alpha2,\,n-2}\le\frac{\hat\mu_0-\mu_0}{\hat\sigma\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)^{1/2}}\le t_{1-\frac\alpha2,\,n-2}\right)\] and a 100(1-α)% CI is: \[\hat\mu_0\pm t_{1-\frac\alpha2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

With repeated samples, \(100(1-\alpha)\%\) of these intervals contain the true \(\mu_0\).

The interval is "center ± quantile × standard error", with center \(\hat\mu_0\).

The half-width is \(t_{1-\alpha/2,\,n-2}\times\mathrm{SE}(\hat\mu_0)\).

**How** Added

Calculate the CI for \(\mu_0\) at a value \(x_0\)

- Calculate \(\mathrm{SE}(\hat\mu_0)\) (Section 03.1).
- Calculate the center \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
- Find the quantile \(t_{1-\alpha/2,\,n-2}\).
- Multiply the quantile by \(\mathrm{SE}(\hat\mu_0)\) to get the half-width.
- Lower limit \(L\): \(\hat\mu_0\) minus the half-width.
- Upper limit \(U\): \(\hat\mu_0\) plus the half-width.

**Self-check:** ① \((L+U)/2=\hat\mu_0\). ② At 95% with \(n=236\), the quantile is 1.970, near 1.96.

**Example 2 · 95% CI for the mean brain weight at \(x_0=4000\)** Added

Example 1 gives \(\mathrm{SE}(\hat\mu_0)=6.676823721\) (How step 1).

1

Center How step 2

\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0=335.432315+0.2608207153\times4000\]

\[=335.432315+1043.282861\]

\[=1378.715176\]

2

Quantile How step 3

\[\alpha=0.05,\quad 1-\frac\alpha2=0.975,\quad n-2=236-2=234\]

\[t_{0.975,\,234}=1.970153643\]

3

Half-width How step 4

\[t_{0.975,\,234}\times\mathrm{SE}(\hat\mu_0)=1.970153643\times6.676823721=13.15436858\]

Basis: Example 1.

4

Lower limit How step 5

\[L=1378.715176-13.15436858=1365.560808\]

5

Upper limit How step 6

\[U=1378.715176+13.15436858=1391.869545\]

Self-check: \((1365.560808+1391.869545)/2=1378.715176=\hat\mu_0\).

At head size 4000 cm³, the 95% CI for the mean brain weight is (1365.561, 1391.870) g. ▲

**Why** Added

Isolate \(\mu_0\). Let \(t=t_{1-\alpha/2,\,n-2}\) and \(\mathrm{SE}=\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\).

1

\[1-\alpha=P\!\left(-t\le\frac{\hat\mu_0-\mu_0}{\mathrm{SE}}\le t\right)\]

Section 03.1; symmetric, each tail \(\alpha/2\).

2

\[1-\alpha=P\!\left(-t\,\mathrm{SE}\le\hat\mu_0-\mu_0\le t\,\mathrm{SE}\right)\]

\(\mathrm{SE}>0\).

3

\[1-\alpha=P\!\left(-\hat\mu_0-t\,\mathrm{SE}\le-\mu_0\le-\hat\mu_0+t\,\mathrm{SE}\right)\]

4

\[1-\alpha=P\!\left(\hat\mu_0-t\,\mathrm{SE}\le\mu_0\le\hat\mu_0+t\,\mathrm{SE}\right)\]

Multiply by \(-1\); the signs change direction.

5

\[\text{CI}=\hat\mu_0\pm t_{1-\frac\alpha2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

∎

### 03.3 Confidence intervals at a range of \(x_0\) values Slides p.21–22

**What** Slides p.21–22

Lecture 4 · p.22 Estimating mean outcomes and report corresponding CIs at a range of x0 values

Slides p.21 shows only code. It does Section 03.2 at many \(x_0\).

A grid is a list of values with equal spaces. The \(x_0\) grid goes from the smallest to the largest \(x\), step 1.

All intervals are 95% and use \(t_{0.975,\,234}\).

Slides p.22 plots the data, the fitted line, and \(L\) and \(U\) at each grid value.

A confidence band is the pair of curves through all \(L\) and all \(U\).

**How** Added

Draw the confidence band for the mean response

- Find the smallest and the largest \(x\).
- Make the grid of \(x_0\) values between them, step 1.
- At each grid value, calculate \(L\) and \(U\) (Section 03.2).
- Plot the data and the fitted line \(\hat\beta_0+\hat\beta_1x\).
- Join all lower limits with a curve.
- Join all upper limits with a second curve.

**Self-check:** ① Grid size = (largest \(x\) − smallest \(x\)) + 1. ② At \(x_0=4000\), the limits are (1365.561, 1391.870), as in Example 2.

**Example 3 · The first grid value** Added

Brainhead data (Added): smallest \(x\) is 2773, largest \(x\) is 4747.

1

Number of grid values How step 2

\[(4747-2773)+1=1974+1=1975\]

2

Distance from \(\bar x\) 03.1 How steps 3–4

\[x_0-\bar x=2773-3637.864407=-864.8644068\]

\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{747990.4421}{30647233.66}=0.02440645868\]

3

Standard error 03.1 How steps 5–7

\[0.004237288136+0.02440645868=0.02864374682\]

\[\sqrt{0.02864374682}=0.169244636\]

\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.169244636=12.24496152\]

4

Center 03.2 How step 2

\[\hat\mu_0=335.432315+0.2608207153\times2773\]

\[=335.432315+723.2558436=1058.688159\]

5

Half-width 03.2 How step 4

\[1.970153643\times12.24496152=24.12445554\]

6

Limits 03.2 How steps 5–6

\[L=1058.688159-24.12445554=1034.563703\]

\[U=1058.688159+24.12445554=1082.812614\]

First band points: (2773, 1034.564) and (2773, 1082.813). The other 1974 grid values give the full band in Figure 3-1. ▲

[figure]
Figure 3-1 · Slides p.22, drawn again from the data. Grey: 236 observations. Blue: fitted line \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\). Orange: 95% limits \(L\), \(U\). Purple dashed: \(\bar x\), where the curves are nearest.

### 03.4 Why the intervals become wider toward the edges Slides p.23

**What** Slides p.23

Lecture 4 · p.23 Why do the CIs get wider toward the edges?

Half-width: \(t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\).

Only \((x_0-\bar x)^2\) changes with \(x_0\). All other parts are fixed for one data set.

This term increases as \(x_0\) moves from \(\bar x\). It is 0 at \(\bar x\), where the interval is narrowest.

**How** Added

Compare the interval widths at different values of \(x_0\)

- For each \(x_0\), calculate \(|x_0-\bar x|\).
- Calculate the half-width (Section 03.2).
- Multiply it by 2 to get the width.
- Sort the \(x_0\) values by \(|x_0-\bar x|\).
- Make sure that the widths increase in the same order.

**Self-check:** ① At \(x_0=\bar x\), \(\mathrm{SE}(\hat\mu_0)=\hat\sigma/\sqrt n\). ② The ratio of two widths depends only on the two sums under the square root.

**Example 4 · Widths at four values of \(x_0\)** Added

All use \(t_{0.975,\,234}=1.970153643\), \(\hat\sigma=72.35066237\), \(1/n=0.004237288136\), \(S_{xx}=30647233.66\).

\(x_0=\bar x=3637.864407\) · \(|x_0-\bar x|=0\)

\(\frac{(x_0-\bar x)^2}{S_{xx}}=0\). Sum \(=0.004237288136\). Square root \(=0.06509445549\).

\(\mathrm{SE}(\hat\mu_0)=72.35066237\times0.06509445549=4.709626971\). Half-width \(=1.970153643\times4.709626971=9.278688732\).

\(\hat\mu_0=335.432315+948.8303969=1284.262712\). CI \(=(1274.984,\ 1293.541)\). Width \(=2\times9.278688732=18.557\).

Self-check: \(\hat\sigma/\sqrt n=72.35066237\times0.06509445549=4.709626971=\mathrm{SE}(\hat\mu_0)\).

\(x_0=4000\) · \(|x_0-\bar x|=362.1355932\)

From Examples 1–2: \(\mathrm{SE}(\hat\mu_0)=6.676823721\). Half-width \(=13.15436858\).

CI \(=(1365.561,\ 1391.870)\). Width \(=2\times13.15436858=26.309\).

\(x_0=2773\) (smallest \(x\)) · \(|x_0-\bar x|=864.8644068\)

From Example 3: \(\mathrm{SE}(\hat\mu_0)=12.24496152\). Half-width \(=24.12445554\).

CI \(=(1034.564,\ 1082.813)\). Width \(=2\times24.12445554=48.249\).

\(x_0=4747\) (largest \(x\)) · \(|x_0-\bar x|=1109.135593\)

\(\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{1230181.764}{30647233.66}=0.0401400589\). Sum \(=0.04437734703\). Square root \(=0.2106593151\).

\(\mathrm{SE}(\hat\mu_0)=72.35066237\times0.2106593151=15.24134098\). Half-width \(=1.970153643\times15.24134098=30.02778345\).

\(\hat\mu_0=335.432315+1238.115936=1573.548251\). CI \(=(1543.520,\ 1603.576)\). Width \(=2\times30.02778345=60.056\).

Widths in order of \(|x_0-\bar x|\): 18.557, 26.309, 48.249, 60.056. The width at the largest \(x\) is \(\sqrt{0.04437734703/0.004237288136}=3.236209805\) times the width at \(\bar x\). ▲

[figure]
Figure 3-2 · Half-width \(t_{0.975,\,234}\times\mathrm{SE}(\hat\mu_0)\) at each \(x_0\) (Added). Lowest at \(\bar x\) (purple dashed). Blue dots: the four \(x_0\) of Example 4.

**Why** Added

The structure of the Slides p.17 result:
\[\mathrm{Var}[\hat\mu_0]=\mathrm{Var}[\bar y+\hat\beta_1(x_0-\bar x)]=\underbrace{\frac{\sigma^2}{n}}_{\text{from }\bar y}+\underbrace{\frac{\sigma^2(x_0-\bar x)^2}{S_{xx}}}_{\text{from }\hat\beta_1}\]
The first equal sign uses Slides p.12. The second is Slides p.17, where the cross term is 0 because \(\sum_{i=1}^n(x_i-\bar x)=0\). ∎

The error in \(\bar y\) is the same at each \(x_0\). It gives \(\frac1n\). The distance \(x_0-\bar x\) multiplies the slope error. It gives \(\frac{(x_0-\bar x)^2}{S_{xx}}\).

### 03.5 Why most points are outside the lines Slides p.24

**What** Slides p.24

Lecture 4 · p.24 Why do most of the points fall outside the lines?

The interval targets the mean \(\mu_0\), not one observation.

Each observation is \(y_i=\mu_i+\epsilon_i\), with \(\mu_i=\beta_0+\beta_1x_i\).

\(\epsilon_i\) has standard deviation \(\sigma\), estimated by \(\hat\sigma=72.35066237\) g.

The largest half-width of the band is 30.02778345 g, at \(x_0=4747\) (Example 4).

Most observations are farther from the line than this half-width.

**How** Added

Find if an observation is inside the band

- Use \(x_i\) as \(x_0\).
- Calculate \(L_i\) and \(U_i\) (Section 03.2).
- Compare \(y_i\) with \(L_i\) and \(U_i\).
- If \(L_i\le y_i\le U_i\), the point is inside. If not, it is outside.
- Do steps 1–4 for all \(n\) observations. Count the two groups.

**Self-check:** inside + outside \(=n=236\).

**Example 5 · Two persons of the brainhead data** Added

Person 1: \(x_1=4512\), \(y_1=1530\). Person 2: \(x_2=3738\), \(y_2=1297\).

1

Person 1: standard error How steps 1–2

\[x_1-\bar x=4512-3637.864407=874.1355932\]

\[\frac{(x_1-\bar x)^2}{S_{xx}}=\frac{764113.0353}{30647233.66}=0.02493252878\]

\[\sqrt{0.004237288136+0.02493252878}=\sqrt{0.02916981692}=0.1707917355\]

\[\mathrm{SE}=72.35066237\times0.1707917355=12.35689519\]

2

Person 1: limits How step 2

\[\hat\mu_0=335.432315+0.2608207153\times4512=335.432315+1176.823068=1512.255383\]

\[\text{half-width}=1.970153643\times12.35689519=24.34498207\]

\[L_1=1487.910401,\qquad U_1=1536.600365\]

3

Person 1: compare How steps 3–4

\[1487.910401\le1530\le1536.600365\]

Inside.

4

Person 2: standard error How steps 1–2

\[x_2-\bar x=3738-3637.864407=100.1355932\]

\[\frac{(x_2-\bar x)^2}{S_{xx}}=\frac{10027.13703}{30647233.66}=0.0003271791882\]

\[\sqrt{0.004237288136+0.0003271791882}=\sqrt{0.004564467324}=0.06756084165\]

\[\mathrm{SE}=72.35066237\times0.06756084165=4.888071643\]

5

Person 2: limits How step 2

\[\hat\mu_0=335.432315+0.2608207153\times3738=335.432315+974.947834=1310.380149\]

\[\text{half-width}=1.970153643\times4.888071643=9.630252153\]

\[L_2=1300.749897,\qquad U_2=1320.010401\]

6

Person 2: compare How steps 3–4

\[1297<1300.749897=L_2\]

Outside: residual \(1297-1310.380149=-13.38014892\) g exceeds half-width 9.630252153 g.

All 236 persons: 41 inside, 195 outside (data file, Added). \(195/236=0.8262711864\), or 82.6%. Self-check: \(41+195=236\). ▲

[figure]
Figure 3-3 · Data and 95% band (blue) of Figure 3-1 (Added). Green: inside. Orange: outside. The vertical spread of the points is much larger than the band.

**Why** Added

Compare the variance of one observation with the variance of \(\hat\mu_0\).

1

\[y_i=\mu_i+\epsilon_i,\qquad \mathrm{Var}[y_i]=\sigma^2\]

Slides p.3.

2

\[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\]

Slides p.17.

3

\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\le0.04437734703\quad\text{for }2773\le x_0\le4747\]

Example 4: largest at \(x_0=4747\).

4

\[\mathrm{Var}[\hat\mu_0]\le0.04437734703\,\sigma^2\]

Less than 5% of \(\mathrm{Var}[y_i]\). ∎

The interval shows only the error of \(\hat\mu_0\). As \(n\) increases, the interval becomes narrower. The spread \(\sigma\) of each point does not decrease with \(n\).

### 03.6 When we want to predict a new observation Slides p.25

Lecture 4 · p.25 What if we don't just care about the mean, but predictions? I.e. even if we got the mean absolutely perfect—the new points wouldn't fall directly on this line!

With exact \(\beta_0\), \(\beta_1\), \(\hat\mu_0=\mu_0\) has no estimation error.

A new observation is still \(\mu_0+\epsilon\), with \(\epsilon\sim N(0,\sigma^2)\) and \(\hat\sigma=72.35066237\) g.

An interval for a new observation must include the error of the mean estimate and the new error \(\epsilon\).

A prediction interval is an interval made to contain a new observation (Unit 04).

Summary of this unit

CI for \(\mu_0\): \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\). Narrowest at \(\bar x\). It contains the mean, not single observations.

### 03.7 Practice Added

Given: \(n=236\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\).

**Q1.** Calculate a 90% CI for the mean brain weight at \(x_0=4000\) cm³. Use \(t_{0.95,\,234}=1.651391475\). Give the half-width and the interval.

Answer

1

\[\alpha=0.10,\qquad 1-\frac\alpha2=0.95,\qquad t_{0.95,\,234}=1.651391475\]

2

\[\hat\mu_0=1378.715176,\qquad \mathrm{SE}(\hat\mu_0)=6.676823721\]

Examples 1–2.

3

\[\text{half-width}=1.651391475\times6.676823721=11.02604977\]

4

\[1378.715176\pm11.02604977=(1367.689,\ 1389.741)\]

It is narrower than the 95% interval (1365.561, 1391.870), because 1.651391475 < 1.970153643.

**Q2.** Calculate the 95% CI for the mean response at \(x_0=3500\). First predict: wider or narrower than at \(x_0=4000\)? Then compare with your numbers.

Answer

Prediction: \(|3500-\bar x|=137.8644068\) < \(|4000-\bar x|=362.1355932\). The interval is narrower.

1

\[\hat\mu_0=335.432315+0.2608207153\times3500=335.432315+912.8725037=1248.304819\]

2

\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{19006.59466}{30647233.66}=0.000620173255\]

3

\[\sqrt{0.004237288136+0.000620173255}=\sqrt{0.004857461391}=0.06969549046\]

4

\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.06969549046=5.042514899\]

5

\[\text{half-width}=1.970153643\times5.042514899=9.934529096\]

6

\[1248.304819\pm9.934529096=(1238.370,\ 1258.239)\]

Width \(2\times9.934529096=19.869\) < 26.309 at \(x_0=4000\). This agrees with the prediction.

**Q3.** At which \(x_0\) is the CI for \(\mu_0\) narrowest? How many times wider is the 95% interval at \(x_0=4747\)? Why does this ratio not depend on \(\hat\sigma\) or \(\alpha\)?

Answer

Narrowest at \(x_0=\bar x=3637.864407\), where \((x_0-\bar x)^2=0\).

1

\[\frac{\text{width at }4747}{\text{width at }\bar x}=\frac{2t\hat\sigma\sqrt{1/n+(4747-\bar x)^2/S_{xx}}}{2t\hat\sigma\sqrt{1/n}}\]

2

\[=\sqrt{\frac{0.04437734703}{0.004237288136}}\]

3

\[=3.236209805\]

Example 4: \(60.0555669/18.55737746=3.236209805\).

\(2t_{1-\alpha/2,\,n-2}\hat\sigma\) cancels. The ratio depends only on \(x_0\), \(n\) and \(S_{xx}\).

## 04 · Predicting a new response: the prediction interval

Start point · Added

Brainhead data, \(n=236\). Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\), \(n-2=234\).

From brainhead.csv (Added): \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\).

All examples use \(x_{new}=4000\), as in Units 02–03.

### 04.1 The new response and the predicted value Slides p.26–30

**What** Slides p.27–30

Lecture 4 · p.27–30 Instead of the mean, we might want to predict the response itself for an observation with covariate \(x_{new}\) (as always treating \(x_{new}\) as fixed): \[y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\] Define the predicted value \[\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\] • How does this differ from how we estimate the mean?
• But we'll need to take into account additional uncertainty (why?)

\(x_{new}\) is the fixed covariate value of the new observation.

The new response \(y_{new}\) is the unobserved response of the new observation. It is random because of its error \(\epsilon_{new}\).

The new error \(\epsilon_{new}\sim N(0,\sigma^2)\) is independent of \(\epsilon_1,\dots,\epsilon_n\).

The predicted value \(\hat y_{new}\) is the fitted line at \(x_{new}\), our guess for \(y_{new}\).

**Slide question 1: how does this differ from the estimate of the mean?** With \(x_0=x_{new}\), the formulas are the same: \(\hat y_{new}=\hat\mu_0\). The target is different:

\(\mu_0=\beta_0+\beta_1x_0\) is a fixed parameter, with no error term.

\(y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\) has the random error \(\epsilon_{new}\).

**Slide question 2: why is there additional uncertainty?** Uncertainty is the possible difference between a guess and the true value. A new response has two sources:

Source 1: \(\hat\beta_0+\hat\beta_1x_{new}\) changes from sample to sample, as for the mean.

Source 2: \(\epsilon_{new}\). We cannot guess it, even with the true \(\beta_0\) and \(\beta_1\).

**How** Added

Calculate the predicted value \(\hat y_{new}\)

- Get \(\hat\beta_1=S_{xy}/S_{xx}\) and \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
- Multiply \(\hat\beta_1\) by \(x_{new}\).
- Add \(\hat\beta_0\).

**Self-check:** \(\hat y_{new}=\hat\mu_0\) at the same \(x\).

**Example 1 · The brainhead data at \(x_{new}=4000\)** Added How steps 2–3

Predict the brain weight of a new person with head size 4000 cm³.

1

\[\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\]

2

\[\hat y_{new}=335.432315+0.2608207153\times4000\]

3

\[\hat y_{new}=335.432315+1043.282861\]

4

\[\hat y_{new}=1378.715176\ \text{g}\]

Slide p.7 values: \(335.43231+0.26082\times4000=1378.71231\).

**Compare with unit 02:** \(\hat\mu_0=1378.715\) is the same number. In this unit, it predicts one person. In unit 02, it estimates the mean of all persons with head size 4000. ▲

### 04.2 Expectation and variance of the prediction error Slides p.31–35

**What** Slides p.31–35

Lecture 4 · p.31–35 Define the prediction error: \(\hat y_{new}-y_{new}\)
Properties: \[E[\hat y_{new}-y_{new}]=E[(\hat\beta_0+\hat\beta_1x_{new})-(\beta_0+\beta_1x_{new}+\epsilon_{new})]=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new})=0\] Now note that \(\hat y_{new}\) and \(y_{new}\) are independent (why?) \[\mathrm{Var}[\hat y_{new}-y_{new}]=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

The prediction error \(\hat y_{new}-y_{new}\) is the predicted value minus the true new response. It is random.

Expectation 0: on average, \(\hat y_{new}\) is not too high and not too low.

The term \(\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\) is source 1. It equals \(\mathrm{Var}[\hat\mu_0]\) of unit 02.

The term \(\sigma^2\) is source 2, from \(\epsilon_{new}\).

Independence: \(\hat y_{new}\) uses only \(y_1,\dots,y_n\). The random part of \(y_{new}\) is \(\epsilon_{new}\), independent of the sample.

**How** Added

Calculate the estimated variance of the prediction error

- Calculate \(\frac1n\).
- Calculate \((x_{new}-\bar x)^2/S_{xx}\).
- Multiply the sum of steps 1–2 by \(\hat\sigma^2\): the line part (source 1).
- The new-error part (source 2) is \(\hat\sigma^2\).
- Add the two parts: \(\hat\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\).

**Self-check:** Step 3 equals the estimate of \(\mathrm{Var}[\hat\mu_0]\) in unit 02. The total is larger than \(\hat\sigma^2\).

**Example 2 · The brainhead data at \(x_{new}=4000\)** Added How steps 1–5

Use \(\hat\sigma\) for \(\sigma\): \(\hat\sigma^2=72.35066237^2=5234.618345\).

1

\[\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}=0.004237288+0.004279087=0.008516375\]

How steps 1–2; unit 02, Example 3.

2

\[\text{line part}=\hat\sigma^2\times0.008516375=5234.618345\times0.008516375=44.579975\]

How step 3: estimate of \(\mathrm{Var}[\hat\mu_0]\).

3

\[\text{new-error part}=\hat\sigma^2=5234.618345\]

How step 4.

4

\[\hat\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)=44.579975+5234.618345=5279.198320\]

How step 5.

[figure]
Figure 4-1 · Total estimated variance 5279.198 at \(x_{new}=4000\), parts to scale. Orange: fitted line (source 1). Blue: \(\epsilon_{new}\) (source 2), 99.16% of the total.

**Result:** \(\epsilon_{new}\) gives almost all of the variance. This term does not decrease as \(n\) increases. The prediction interval is much wider than the confidence interval. ▲

**Why · The expectation and the variance of the prediction error** Slides p.33–35

**(a) Expectation.** Use \(E[\hat\beta_0]=\beta_0\), \(E[\hat\beta_1]=\beta_1\), \(E[\epsilon_{new}]=0\).

1

\[E[\hat y_{new}-y_{new}]=E[(\hat\beta_0+\hat\beta_1x_{new})-(\beta_0+\beta_1x_{new}+\epsilon_{new})]\]

Definitions.

2

\[=E[\hat\beta_0]+E[\hat\beta_1]x_{new}-(\beta_0+\beta_1x_{new}+E[\epsilon_{new}])\]

Linearity of expectation.

3

\[=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new}+0)\]

Unbiased estimators; \(E[\epsilon_{new}]=0\).

4

\[=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new})\]

5

\[=0\]

**(b) Independence.**

\(\hat\beta_0\) and \(\hat\beta_1\) use only \(y_1,\dots,y_n\), whose random part is \(\epsilon_1,\dots,\epsilon_n\).

The random part of \(y_{new}\) is \(\epsilon_{new}\), independent of \(\epsilon_1,\dots,\epsilon_n\).

\(\hat y_{new}\) and \(y_{new}\) are independent.

**(c) Variance.**

For independent \(A\) and \(B\): \(\mathrm{Var}[A-B]=\mathrm{Var}[A]+\mathrm{Var}[B]\).

Unit 02 (Lecture 4 · p.17–18): \(\mathrm{Var}[\hat\beta_0+\hat\beta_1x_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\).

\(\mathrm{Var}[y_{new}]=\mathrm{Var}[\epsilon_{new}]=\sigma^2\).

1

\[\mathrm{Var}[\hat y_{new}-y_{new}]=\mathrm{Var}[(\hat\beta_0+\hat\beta_1x_{new})-y_{new}]\]

2

\[=\mathrm{Var}[(\hat\beta_0+\hat\beta_1x_{new})]+\mathrm{Var}[y_{new}]\]

Independence (b). Both terms are added.

3

\[=\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)+\mathrm{Var}[y_{new}]\]

Unit 02 with \(x_0=x_{new}\).

4

\[=\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)+\sigma^2\]

\(\mathrm{Var}[y_{new}]=\sigma^2\).

5

\[=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

\(\epsilon_{new}\) adds the "1", compared with \(\mathrm{Var}[\hat\mu_0]\). ∎

### 04.3 The prediction interval Slides p.36–37

**What** Slides p.36–37

Lecture 4 · p.36–37 We know \(\hat y_{new}-y_{new}\sim N\left(0,\ \sigma^2\left[1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right]\right)\) \[\Longrightarrow\ \frac{\hat y_{new}-y_{new}}{\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim N(0,1)\] Same logic as before: \[\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim t_{n-2}\] \[\Longrightarrow\ 1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\le\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\le t_{1-\frac{\alpha}{2},n-2}\right)\] So a 100(1−α)% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

A prediction interval (PI) is an interval, calculated from the data, that contains the random \(y_{new}\) with probability \(1-\alpha\).

The confidence interval (CI) of unit 03 contains the fixed \(\mu_0\) with probability \(1-\alpha\).

The only difference from the CI is the "1" under the square root: the \(\sigma^2\) from \(\epsilon_{new}\).

**How** Added

Calculate a 100(1−α)% prediction interval

- Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\) (Section 04.1).
- Calculate \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\).
- Take the square root and multiply by \(\hat\sigma\): the standard error of prediction.
- Find the quantile \(t_{1-\alpha/2,\,n-2}\).
- Multiply step 4 by step 3: the half-width.
- Subtract the half-width from \(\hat y_{new}\) and add it to \(\hat y_{new}\).

**Self-check:** ① The midpoint is \(\hat y_{new}\). ② The standard error of prediction is larger than \(\hat\sigma\). ③ The PI is wider than the CI at the same point.

**Example 3 · The brainhead data at \(x_{new}=4000\), 95% prediction interval** Added How steps 1–6

Find the 95% PI for the brain weight of a new person with head size 4000 cm³. Compare it with the 95% CI for the mean. \(\alpha=0.05\), \(n-2=234\).

1

Predicted value How step 1

\[\hat y_{new}=335.432315+0.2608207153\times4000=1378.715176\]

**Basis:** Example 1.

2

Value under the square root How step 2

\[1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}=1+0.008516375=1.008516375\]

**Basis:** Example 2, step 1.

3

Standard error of prediction How step 3

\[\hat\sigma\sqrt{1.008516375}=72.35066237\times1.004249160=72.658092\]

4

t quantile How step 4

\[t_{1-\frac{0.05}{2},\,234}=t_{0.975,\,234}=1.970154\]

5

Half-width How step 5

\[1.970154\times72.658092=143.1476\]

6

Interval How step 6

\[1378.715\pm143.148=(1378.715-143.148,\ \ 1378.715+143.148)=(1235.568,\ \ 1521.863)\]

95% CI for \(\mu_0\) (unit 03, Example 2): \((1365.561,\ 1391.870)\), width \(2\times13.154=26.31\) g.

95% PI for \(y_{new}\): \((1235.568,\ 1521.863)\), width \(2\times143.148=286.30\) g.

Same midpoint, 1378.715. The PI is \(143.1476/13.15437=10.88\) times as wide.

Slides p.39 shows both bands over a range of \(x\) (Unit 05). ▲

**Why · From the distribution to the prediction interval** Slides p.36–37

Let \(k=1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\).

Normality: \(\hat y_{new}\) is a linear combination of the normal \(y_1,\dots,y_n\).

\(y_{new}\) is normal and independent of the sample. A linear combination of independent normals is normal.

1

\[\hat y_{new}-y_{new}\sim N\left(0,\ \sigma^2k\right)\]

Section 04.2.

2

\[\frac{\hat y_{new}-y_{new}-0}{\sqrt{\sigma^2k}}\sim N(0,1)\]

Standardize.

3

\[\frac{\hat y_{new}-y_{new}}{\sigma\sqrt{k}}\sim N(0,1)\]

4

\[\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{k}}\sim t_{n-2}\]

\(\hat\sigma\) for \(\sigma\), as for \(\hat\beta_1\), \(\hat\mu_0\).

5

\[1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\le\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{k}}\le t_{1-\frac{\alpha}{2},n-2}\right)\]

Symmetric; each tail \(\alpha/2\).

6

\[1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le\hat y_{new}-y_{new}\le t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]

7

\[1-\alpha=P\left(-\hat y_{new}-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le-y_{new}\le-\hat y_{new}+t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]

8

\[1-\alpha=P\left(\hat y_{new}-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le y_{new}\le\hat y_{new}+t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]

Multiply by \(-1\); the signs change direction.

9

\[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

Put in \(k\).

Line 8: this random interval contains the random \(y_{new}\) with probability \(1-\alpha\). ∎

### 04.4 Practice Added

Given: \(n=236\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\).

**Q1.** A new person has head size 4000 cm³. Calculate a 90% PI for this person's brain weight. Use \(t_{0.95,\,234}=1.651391\).

Answer

Step 1: \(\hat y_{new}=1378.715176\) (Example 1).

Steps 2–3: \(\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}=72.658092\) (Example 3).

Step 4: \(\alpha=0.10\), \(t_{1-0.10/2,\,234}=t_{0.95,\,234}=1.651391\).

Step 5: half-width \(=1.651391\times72.658092=119.9870\).

Step 6: \(1378.715\pm119.987=(1258.728,\ 1498.702)\).

It is narrower than the 95% PI \((1235.568,\ 1521.863)\), because its quantile is smaller.

**Q2.** Calculate a 95% PI for the brain weight of a new person with head size 4500 cm³. Use \(t_{0.975,\,234}=1.970154\).

Answer

Step 1: \(\hat y_{new}=335.432315+0.2608207153\times4500=335.432315+1173.693219=1509.125534\).

Step 2: \(x_{new}-\bar x=4500-3637.864407=862.135593\). \(\frac{862.135593^2}{30647233.66}=\frac{743277.7807}{30647233.66}=0.024252688\).

\(1+\frac1{236}+0.024252688=1+0.004237288+0.024252688=1.028489976\).

Step 3: \(\hat\sigma\sqrt{1.028489976}=72.35066237\times1.014144948=73.374059\).

Step 4: \(t_{0.975,\,234}=1.970154\).

Step 5: half-width \(=1.970154\times73.374059=144.5582\).

Step 6: \(1509.126\pm144.558=(1364.567,\ 1653.684)\).

4500 is farther from \(\bar x\) than 4000. The half-width 144.558 is a little larger than 143.148.

**Q3.** (a) At which \(x_{new}\) is the PI narrowest? (b) Calculate its 95% half-width. (c) Why does this half-width not decrease to 0 as \(n\to\infty\)?

Answer

(a) Only \(\frac{(x_{new}-\bar x)^2}{S_{xx}}\) changes with \(x_{new}\). It is 0 at \(x_{new}=\bar x=3637.864407\), where the PI is narrowest.

(b) \(\hat\sigma\sqrt{1+\frac1{236}+0}=72.35066237\times\sqrt{1.004237288}=72.35066237\times1.002116404=72.503786\).

Half-width \(=1.970154\times72.503786=142.8436\).

\(\hat y_{new}=335.432315+0.2608207153\times3637.864407=1284.262712\). PI: \(1284.263\pm142.844=(1141.419,\ 1427.106)\).

(c) As \(n\to\infty\), \(\frac1n\to0\), and the third term is 0 at \(\bar x\). The "1" stays.

The "1" is \(\mathrm{Var}[\epsilon_{new}]=\sigma^2\). More data improve the fitted line but do not remove \(\epsilon_{new}\).

The half-width stays near \(t_{1-\alpha/2,\,n-2}\,\hat\sigma\) or larger.

## 05 · Prediction intervals over a range of \(x\) values

Start point · Added

Brainhead data, \(n=236\). Values from brainhead.csv, as in units 01–04:

\(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207\).

\(\hat\sigma=72.35066\), with \(n-2=234\) degrees of freedom.

\(\bar x=3637.8644\), \(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2=30647233.66\).

\(t_{0.975,\,234}=1.970154\).

\(\min x_i=2773\), \(\max x_i=4747\).

### 05.1 Calculate the prediction interval at many values of \(x_{new}\) Slides p.38

**What** Slides p.37–38

Prediction Interval · Lecture 4 · p.37 So a 100(1-α)% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},\,n-2}\,\hat\sigma\sqrt{1+\frac{1}{n}+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

Slides p.38 shows this as code. For each \(x_{new}\), the code does four things:

It finds \(t_{1-\alpha/2,\,n-2}\). The default level 0.95 gives \(t_{0.975,\,n-2}\).

It calculates the plug-in estimate \(\hat y_{new}\). A plug-in estimate puts \(\hat\beta_0,\hat\beta_1\) in place of the unknown \(\beta_0,\beta_1\).

It calculates the standard error \(\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}\).

It gives L = estimate − quantile × standard error and U = estimate + quantile × standard error.

The grid goes from 2773 to 4747, step 1: \(4747-2773+1=1975\) values. The p.39 plot joins all L and all U.

Only the PI has the term **1** under the square root.

**How** Added

Calculate a prediction interval at one value of \(x_{new}\)

- Find the quantile \(t_{1-\alpha/2,\,n-2}\).
- Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\).
- Calculate the distance term \(k=(x_{new}-\bar x)^2/S_{xx}\).
- Calculate the standard error \(\hat\sigma\sqrt{1+1/n+k}\).
- Multiply the quantile by the standard error to get the half-width.
- Subtract the half-width from \(\hat y_{new}\) to get L. Add it to get U.

For the band, do steps 2–6 at each grid value. The quantile does not change.

**Self-check:** ① \((L+U)/2=\hat y_{new}\). ② \(1+1/n+k\) is a little larger than 1. ③ At \(x_{new}=4000\), the PI is (1235.568, 1521.863), as in unit 04.

**Example 1 · 95% PI at the smallest head size, \(x_{new}=2773\)** Added

1

Quantile How step 1

\[t_{1-\alpha/2,\,n-2}=t_{1-0.05/2,\,236-2}\]

\[=t_{0.975,\,234}=1.970154\]

2

Predicted value How step 2

\[\hat y_{new}=335.432315+0.2608207\times2773\]

\[=335.432315+723.255844\]

\[=1058.688159\]

3

Distance term How step 3

\[k=\frac{(2773-3637.8644)^2}{30647233.66}\]

\[=\frac{(-864.8644)^2}{30647233.66}\]

\[=\frac{747990.44}{30647233.66}\]

\[=0.024406\]

4

Standard error How step 4

\[\hat\sigma\sqrt{1+\tfrac1n+k}=72.35066\sqrt{1+0.004237+0.024406}\]

\[=72.35066\sqrt{1.028644}\]

\[=72.35066\times1.014221\]

\[=73.37954\]

\(1/n=1/236=0.004237\).

5

Half-width How step 5

\[1.970154\times73.37954=144.56898\]

6

Limits How step 6

\[L=1058.68816-144.56898=914.11918\]

\[U=1058.68816+144.56898=1203.25714\]

95% PIs at four grid points (data file):

\(x_{new}=2773\) (smallest): (914.119, 1203.257), width 289.138.

\(x_{new}=\bar x=3637.8644\): (1141.419, 1427.106), width 285.687.

\(x_{new}=4000\): (1235.568, 1521.863), width 286.295, as in unit 04.

\(x_{new}=4747\) (largest): (1427.878, 1719.219), width 291.341.

Self-check ①: \((914.119+1203.257)/2=1058.688=\hat y_{new}\). ▲

**Why** Added

Each How step gives one part of the p.37 formula: step 1 the quantile, step 2 \(\hat y_{new}\), steps 3–4 the standard error, steps 5–6 the limits.

Unit 04 (Slides p.35): \(\mathrm{Var}[\hat y_{new}-y_{new}]=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\). The 1 is \(\mathrm{Var}[y_{new}]=\sigma^2\). ∎

### 05.2 Read the plot: the PI is much wider than the CI Slides p.39

**What** Slides p.39

The CI for \(\mu_0\) (Lecture 4 p.20) is \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\), without the 1.

Prediction Intervals · Lecture 4 · p.39 Computing prediction intervals at a range of \(x_{new}\) values:
• Note the prediction intervals are much wider than the CIs before!
• Note also the lines are not parallel (slightly wider at the edges)

Note 1 compares PI and CI widths. Note 2: the PI is a little wider near the smallest and largest \(x\).

**How** Added

Compare the widths of the PI and the CI

- Select \(\bar x\), \(\min x_i\), \(\max x_i\), and one more \(x\).
- Calculate the PI width U − L at each \(x\).
- Calculate the CI width at the same \(x\).
- Divide the PI width by the CI width.
- Compare the PI width at \(\bar x\) with the PI width at the edges.

A ratio much larger than 1 supports note 1. Edge widths a little larger than the center width support note 2.

**Self-check:** ① Both widths are smallest at \(\bar x\), where \((x-\bar x)^2/S_{xx}=0\). ② Both intervals have the midpoint \(\hat\beta_0+\hat\beta_1x\).

**Example 2 · Widths in the brainhead data** Added

[figure]
Figure 5-1. Brainhead data (\(n=236\)). Blue: 95% CI band. Orange: 95% PI band. Black points are in the PI; purple points are outside. Dashed: \(\bar x\). The CI band widens much at the edges; the PI lines widen only a little.

PI widths from Section 05.1. CI widths use the same steps without the 1.

\(x=2773\): PI width 289.138, CI width 48.249, ratio 5.99.

\(x=\bar x\): PI width 285.687, CI width 18.557, ratio 15.39.

\(x=4000\): PI width 286.295, CI width 26.309, ratio 10.88.

\(x=4747\): PI width 291.341, CI width 60.056, ratio 4.85.

Note 1: the PI is 4.85 to 15.39 times as wide as the CI, most at \(\bar x\).

Note 2: the PI width is 285.687 at \(\bar x\), and 289.138 and 291.341 at the edges: only 3.451 and 5.654 g more.

The CI width grows from 18.557 to 60.056, more than 3 times.

Count (Added, data file): 226 of 236 points are in the 95% PI at their own \(x_i\): 95.8%, near 95%. Only 41 points are in the CI band. ▲

**Why** Added

Let \(k=\frac{(x-\bar x)^2}{S_{xx}}\ge0\).

1

\[\frac{\text{PI width}}{\text{CI width}}=\frac{2\,t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{1+\frac1n+k}}{2\,t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+k}}\]

2

\[=\sqrt{\frac{1+\frac1n+k}{\frac1n+k}}\]

3

\[=\sqrt{\frac{1+\frac1n}{\frac1n}}\]

At \(x=\bar x\), \(k=0\).

4

\[=\sqrt{n+1}\]

5

\[=\sqrt{237}=15.3948\]

Agrees with 15.39 at \(\bar x\). ∎

Note 2: in the PI, the 1 is the largest term. At the right edge, \(k\) is only 0.040140. The square root grows only from \(\sqrt{1.004237}\) to \(\sqrt{1.044377}\). Without the 1, the same \(k\) has a large effect on the CI.

### 05.3 Fitted values, the CI for the mean, and the PI together Slides p.40

**What** Slides p.40

Slides p.40 is code only, with no numbers. It gives three quantities from the fitted model:

Fitted values: \(\hat y_i=\hat\beta_0+\hat\beta_1x_i\), the fitted line above each sample \(x_i\).

CIs for the means: \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\) at each grid value \(x_0\).

PIs: \(\hat y_{new}\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}\) at each grid value \(x_{new}\).

At the same \(x\), \(\hat\mu_0=\hat y_{new}\). Only the standard error differs.

**How** Added

Calculate the CI and the PI at the same \(x\)

- Calculate the center \(\hat\beta_0+\hat\beta_1x\) for both intervals.
- Calculate \(k=(x-\bar x)^2/S_{xx}\).
- Calculate the CI standard error \(\hat\sigma\sqrt{1/n+k}\).
- Calculate the PI standard error \(\hat\sigma\sqrt{1+1/n+k}\).
- Multiply each by \(t_{1-\alpha/2,\,n-2}\) to get the two half-widths.
- Subtract each half-width from the center and add it.

**Self-check:** ① Same midpoint. ② The PI contains the CI.

**Example 3 · Both intervals at \(x=3000\)** Added

1

Center How step 1

\[\hat\mu_0=\hat y_{new}=335.432315+0.2608207\times3000\]

\[=335.432315+782.462146\]

\[=1117.894461\]

2

Distance term How step 2

\[k=\frac{(3000-3637.8644)^2}{30647233.66}\]

\[=\frac{(-637.8644)^2}{30647233.66}\]

\[=\frac{406871.00}{30647233.66}\]

\[=0.013276\]

3

CI standard error How step 3

\[72.35066\sqrt{0.004237+0.013276}=72.35066\sqrt{0.017513}\]

\[=72.35066\times0.132338\]

\[=9.574711\]

4

PI standard error How step 4

\[72.35066\sqrt{1+0.004237+0.013276}=72.35066\sqrt{1.017513}\]

\[=72.35066\times1.008719\]

\[=72.98146\]

5

Half-widths How step 5

\[\text{CI: }1.970154\times9.574711=18.86365\]

\[\text{PI: }1.970154\times72.98146=143.78469\]

6

Limits How step 6

\[\text{CI: }(1117.89446-18.86365,\ 1117.89446+18.86365)=(1099.031,\ 1136.758)\]

\[\text{PI: }(1117.89446-143.78469,\ 1117.89446+143.78469)=(974.110,\ 1261.679)\]

Midpoint 1117.894 for both. CI width 37.727, PI width 287.569. The PI contains the CI. ▲

A fitted value uses step 1 only. Person 1 has \(x_1=4512\): \(\hat y_1=335.432315+0.2608207\times4512=335.432315+1176.823068=1512.255\) g (Added).

**Why** Added

Both intervals use the fitted line at \(x\) as the center. The CI targets the fixed \(\mu_0\): only the error in \(\hat\beta_0,\hat\beta_1\) counts. The PI targets \(y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\): the variance \(\sigma^2\) of \(\epsilon_{new}\) also counts. ∎

### 05.4 Practice Added

Use \(n=236\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207\), \(\hat\sigma=72.35066\), \(\bar x=3637.8644\), \(S_{xx}=30647233.66\).

**Q1.** A new person has head size 3500 cm³. Calculate a 90% PI for this person's brain weight by hand. Use \(t_{0.95,\,234}=1.651391\).

Answer

1

\[\hat y_{new}=335.432315+0.2608207\times3500=335.432315+912.872504=1248.3048\]

2

\[k=\frac{(3500-3637.8644)^2}{30647233.66}=\frac{(-137.8644)^2}{30647233.66}=\frac{19006.59}{30647233.66}=0.000620\]

3

\[72.35066\sqrt{1+0.004237+0.000620}=72.35066\sqrt{1.004857}\]

4

\[=72.35066\times1.002426=72.5262\]

5

\[1.651391\times72.5262=119.7691\]

6

\[1248.3048\pm119.7691=(1128.536,\ 1368.074)\]

**Q2.** At \(x=\bar x\), show that the 95% PI width divided by the 95% CI width equals \(\sqrt{n+1}\). Evaluate it for the brainhead data.

Answer

At \(x=\bar x\), the distance term is 0.

1

\[\frac{2\,t_{0.975,\,n-2}\,\hat\sigma\sqrt{1+\frac1n}}{2\,t_{0.975,\,n-2}\,\hat\sigma\sqrt{\frac1n}}=\sqrt{\frac{1+\frac1n}{\frac1n}}\]

2

\[=\sqrt{n+1}\]

3

\[=\sqrt{237}=15.39\]

Check: \(285.687/18.557=15.39\).

**Q3.** Two 95% intervals at \(x=3000\) are (1099.031, 1136.758) and (974.110, 1261.679). (a) Which is the CI for the mean response, and which is the PI? (b) Why do both have the same midpoint?

Answer

(a) The first (width 37.727) is the CI. The second (width 287.569) is the PI: its standard error has the extra 1 from \(\sigma^2\).

(b) Both have the center \(\hat\beta_0+\hat\beta_1x=335.432315+0.2608207\times3000=1117.894\). The mean estimate and the predicted value are the same number.

## 06 · Practice questions Q1 and Q2

Plan · Slides p.41–43

Section page (p.41), Q1: MLEs and \(\hat\sigma/\hat\sigma_{MLE}\) (p.42), Q2: two CIs and one PI (p.43), similar exercises (Added).

Slides p.41 · Section page "Practice"

The two questions use no data set, only the numbers given.

### 06.1 Practice Q1: MLEs and the ratio \(\hat\sigma/\hat\sigma_{MLE}\) Slides p.42

**What · The question** Slides p.42

Lecture 4 · p.42 · Practice Q1 Consider the simple linear regression model \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2),\qquad i=1,\dots,n.\] (a) Write down expressions of the MLEs of \(\beta_0\) and \(\beta_1\) in terms of \(\bar x,\bar y,S_{xy}\) and the sample variance of \(x\).
(b) For \(n=20\), find the ratio \(\hat\sigma/\hat\sigma_{MLE}\).
(c) Suppose \(n=50\), and \(S_{xx}=180,\ \bar x=4.8,\ S_{xy}=353,\ \sum_{i=1}^n x_iy_i=3378\). Compute the MLEs for \(\beta_0\) and \(\beta_1\).

The maximum likelihood estimator (MLE) is the parameter value that makes the observed data most probable under the model.

The sample variance of \(x\) is \(s_x^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar x)^2=S_{xx}/(n-1)\).

Earlier result · Lecture 3 · p.7 So the least squares estimators are: \[\hat\beta_1=\frac{S_{xy}}{S_{xx}},\qquad \hat\beta_0=\bar y-\hat\beta_1\bar x.\] OLS=MLE under the assumption of Normality.

With normal errors, the least squares estimators are the MLEs.

Earlier result · Lecture 2 · p.5 For observations \(y_1,\dots,y_n\), the sample variance is: \(\displaystyle s_y^2=\frac{1}{n-1}\sum_{i}^{n}(y_i-\bar y)^2\)

Earlier result · Lecture 2 · p.35 \[\hat\sigma^2_{ML}=\frac{\sum_{i=1}^n e_i^2}{n}.\qquad\text{However we typically adopt a different estimator: }\ \hat\sigma^2=\frac{\sum_{i=1}^n e_i^2}{n-2}.\]

\(\hat\sigma_{MLE}\) in Q1 is \(\hat\sigma_{ML}\). It divides by \(n\); \(\hat\sigma^2\) divides by \(n-2\). The numerator \(\sum e_i^2\) is the same.

**How · The steps** Added

Q1 · How to solve

- **(a) Write the MLEs.** ① Write \(\hat\beta_1=S_{xy}/S_{xx}\). ② Replace \(S_{xx}\) with \((n-1)s_x^2\). ③ Write \(\hat\beta_0=\bar y-\hat\beta_1\bar x\). ④ Put ② into ③.
- **(b) Find the ratio.** ① Write \(\hat\sigma^2\) and \(\hat\sigma^2_{MLE}\). ② Divide them; \(\sum e_i^2\) cancels. ③ Take the square root. ④ Put in \(n\).
- **(c) Calculate the estimates.** ① Calculate \(\hat\beta_1=S_{xy}/S_{xx}\). ② Find \(\bar y\) from \(\sum x_iy_i=S_{xy}+n\bar x\bar y\). ③ Calculate \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).

**Self-check:** (a) contains only \(\bar x,\bar y,S_{xy},s_x^2\), \(n\), not \(S_{xx}\). (b) is more than 1, because \(n-2<n\). In (c), \(S_{xy}+n\bar x\bar y=3378\).

**Example · The answers** Added

Answer to Q1(a)

1

Write the slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{S_{xy}}{S_{xx}}\]

**Basis:** OLS = MLE with normal errors.

2

Replace \(S_{xx}\) L2 p.5 · \(s_x^2=S_{xx}/(n-1)\)

\[\hat\beta_1=\frac{S_{xy}}{(n-1)s_x^2}\]

3

Write the intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=\bar y-\frac{S_{xy}}{(n-1)s_x^2}\,\bar x\]

Answer to Q1(b)

1

Write the two estimators L2 p.35 · \(\hat\sigma^2_{ML}\) and \(\hat\sigma^2\)

\[\hat\sigma^2=\frac{\sum_{i=1}^n e_i^2}{n-2},\qquad \hat\sigma^2_{MLE}=\frac{\sum_{i=1}^n e_i^2}{n}\]

2

Divide and take the square root (Why (ii))

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{n}{n-2}}\]

3

Put in \(n=20\)

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{20}{18}}\]

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{10}{9}}\]

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}\approx 1.0541\]

\(\hat\sigma\) is approximately 5.4% larger than \(\hat\sigma_{MLE}\).

Answer to Q1(c)

1

Calculate the slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{353}{180}\]

\[\hat\beta_1\approx 1.961111\]

2

Put the numbers in the identity \(\sum x_iy_i=S_{xy}+n\bar x\bar y\) (Why (iii))

\[3378=353+50\times4.8\times\bar y\]

\[3378=353+240\,\bar y\]

3

Find \(\bar y\)

\[240\,\bar y=3025\]

\[\bar y=\frac{3025}{240}\]

\[\bar y\approx 12.604167\]

4

Calculate the intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=12.604167-1.961111\times4.8\]

\[\hat\beta_0=12.604167-9.413333\]

\[\hat\beta_0\approx 3.190833\]

**Self-check:** \(353+240\times12.604167\approx3378\).

\(\hat\beta_1\approx1.9611\), \(\hat\beta_0\approx3.1908\).

**Why · The derivations** Added

**(i) \(S_{xx}\) and the sample variance**

1

\[s_x^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar x)^2\]

**Basis:** Lecture 2 · p.5

2

\[s_x^2=\frac{S_{xx}}{n-1}\]

**Basis:** definition of \(S_{xx}\)

3

\[S_{xx}=(n-1)s_x^2\]

**(ii) The ratio \(\hat\sigma/\hat\sigma_{MLE}\)**

1

\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{\sum e_i^2/(n-2)}{\sum e_i^2/n}\]

**Basis:** Lecture 2 · p.35

2

\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{\sum e_i^2}{n-2}\cdot\frac{n}{\sum e_i^2}\]

3

\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{n}{n-2}\]

4

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{n}{n-2}}\]

**Basis:** both are positive

**(iii) The identity \(\sum x_iy_i=S_{xy}+n\bar x\bar y\)**

1

\[S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\]

2

\[S_{xy}=\sum x_iy_i-\bar x\sum y_i-\bar y\sum x_i+n\bar x\bar y\]

3

\[S_{xy}=\sum x_iy_i-\bar x(n\bar y)-\bar y(n\bar x)+n\bar x\bar y\]

**Basis:** \(\sum y_i=n\bar y\), \(\sum x_i=n\bar x\)

4

\[S_{xy}=\sum x_iy_i-n\bar x\bar y\]

5

\[\sum x_iy_i=S_{xy}+n\bar x\bar y\qquad\blacksquare\]

### 06.2 Practice Q2: two confidence intervals and one prediction interval Slides p.43

**What · The question** Slides p.43

Lecture 4 · p.43 · Practice Q2 Consider the simple linear regression model \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2),\qquad i=1,\dots,n.\] Suppose \(n=10\) and \(\hat\beta_1=5,\ \hat\beta_0=-2,\ S_{xx}=16,\ \bar x=0,\ \hat\sigma=0.4\).
Also we know that if \(T\sim t_{(8)}\): \(P(T>-1.86)=0.95\), \(P(|T|>2.75)=0.025\), \(P(T<2.31)=0.975\).
(a) Calculate a two-sided 95% confidence interval for \(\beta_1\).
(b) Calculate a two-sided 95% confidence interval for \(\beta_0\).
(c) Calculate a 90% prediction interval for a new response at \(x_{new}=5\).

\(t_{(8)}\) has \(n-2=10-2=8\) degrees of freedom.

Lecture 4 · p.5 (review of Lecture 3) 95% CI for \(\beta_1\) is \(\displaystyle\hat\beta_1\pm t_{0.975,n-2}\times\sqrt{\frac{\hat\sigma^2}{S_{xx}}}\) or \(\hat\beta_1\pm t_{0.975,n-2}\times \mathrm{SE}(\hat\beta_1)\).
More generally a \(100\times(1-\alpha)\%\) CI is: \(\hat\beta_1\pm t_{1-\alpha/2,n-2}\times \mathrm{SE}(\hat\beta_1)\).

Earlier result · Lecture 3 · p.24 \[\hat\beta_0\sim N\!\left(\beta_0,\ \sigma^2\left(\frac1n+\frac{\bar x^2}{S_{xx}}\right)\right)\]

\(\hat\sigma\) for \(\sigma\) gives \(\mathrm{SE}(\hat\beta_0)=\hat\sigma\sqrt{\frac1n+\frac{\bar x^2}{S_{xx}}}\). The CI is \(\hat\beta_0\pm t_{1-\alpha/2,n-2}\,\mathrm{SE}(\hat\beta_0)\) (Lecture 3 · p.36).

Lecture 4 · p.37 So a 100(1−\(\alpha\))% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\ \hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

**How · The steps** Added

Q2 · How to solve

- **Step 0: Read the quantiles.** ① Find \(1-\alpha/2\): 95% gives 0.975, 90% gives 0.95. ② Find \(P(T<c)=1-\alpha/2\) in the question. ③ Change \(P(T>-c)\) to \(P(T<c)\) by symmetry.
- **(a) CI for \(\beta_1\).** ① Calculate \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\). ② Multiply by \(t_{0.975,8}\). ③ Write \(\hat\beta_1\) plus or minus this number.
- **(b) CI for \(\beta_0\).** ① Calculate \(\frac1n+\frac{\bar x^2}{S_{xx}}\). ② Take the square root and multiply by \(\hat\sigma\): \(\mathrm{SE}(\hat\beta_0)\). ③ Write \(\hat\beta_0\pm t_{0.975,8}\,\mathrm{SE}(\hat\beta_0)\).
- **(c) 90% PI.** ① Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\). ② Calculate \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\). ③ Take the square root and multiply by \(\hat\sigma\). ④ Multiply by \(t_{0.95,8}\). Write \(\hat y_{new}\) plus or minus this number.

**Self-check:** Degrees of freedom \(n-2=8\). Each interval has the point estimate at its center. The PI has an extra "1" under the square root.

[figure]
Figure 6-1. The \(t_{(8)}\) density. Blue tails (beyond ±2.31): 0.025 each, middle 0.95. Orange-plus-blue tails (beyond ±1.86): 0.05 each, middle 0.90.

**Example · The answers** Added

Step 0: read \(t_{0.975,8}\) and \(t_{0.95,8}\) from the question

1

A 95% interval needs \(t_{0.975,8}\) How Step 0 ①

\[1-\alpha=0.95\ \Rightarrow\ 1-\alpha/2=0.975\]

2

Third statement definition of a quantile

\[P(T<2.31)=0.975\ \Rightarrow\ t_{0.975,8}=2.31\]

3

A 90% interval needs \(t_{0.95,8}\) How Step 0 ①

\[1-\alpha=0.90\ \Rightarrow\ 1-\alpha/2=0.95\]

4

First statement, by symmetry (Why (i)) How Step 0 ③

\[P(T>-1.86)=0.95\ \Rightarrow\ P(T<1.86)=0.95\ \Rightarrow\ t_{0.95,8}=1.86\]

\(P(|T|>2.75)=0.025\) puts 0.0125 in each tail: a two-sided 97.5% interval. Q2 does not use it.

Answer to Q2(a): 95% CI for \(\beta_1\)

1

Standard error L4 p.5 · \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\)

\[\mathrm{SE}(\hat\beta_1)=\frac{0.4}{\sqrt{16}}\]

\[\mathrm{SE}(\hat\beta_1)=\frac{0.4}{4}\]

\[\mathrm{SE}(\hat\beta_1)=0.1\]

2

Half-width Step 0 · \(t_{0.975,8}=2.31\)

\[t_{0.975,8}\times \mathrm{SE}(\hat\beta_1)=2.31\times0.1\]

\[t_{0.975,8}\times \mathrm{SE}(\hat\beta_1)=0.231\]

3

Interval L4 p.5 · \(\hat\beta_1\pm t_{0.975,n-2}\mathrm{SE}(\hat\beta_1)\)

\[5\pm0.231\]

\[(5-0.231,\ 5+0.231)=(4.769,\ 5.231)\]

Answer to Q2(b): 95% CI for \(\beta_0\)

1

SE formula L3 p.24 · \(\mathrm{SE}(\hat\beta_0)\)

\[\mathrm{SE}(\hat\beta_0)=\hat\sigma\sqrt{\frac1n+\frac{\bar x^2}{S_{xx}}}\]

2

Put in \(n=10,\ \bar x=0,\ S_{xx}=16\)

\[\frac1n+\frac{\bar x^2}{S_{xx}}=\frac1{10}+\frac{0^2}{16}\]

\[\frac1n+\frac{\bar x^2}{S_{xx}}=0.1+0\]

\[\frac1n+\frac{\bar x^2}{S_{xx}}=0.1\]

3

Square root, times \(\hat\sigma\)

\[\mathrm{SE}(\hat\beta_0)=0.4\sqrt{0.1}\]

\[\mathrm{SE}(\hat\beta_0)=0.4\times0.316228\]

\[\mathrm{SE}(\hat\beta_0)\approx0.126491\]

4

Half-width Step 0 · \(t_{0.975,8}=2.31\)

\[2.31\times0.126491\approx0.292194\]

5

Interval L3 p.36 · \(\hat\beta_0\pm t\cdot \mathrm{SE}(\hat\beta_0)\)

\[-2\pm0.292194\]

\[(-2-0.292194,\ -2+0.292194)\approx(-2.292,\ -1.708)\]

Answer to Q2(c): 90% PI at \(x_{new}=5\)

1

Center L4 p.37 · \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\)

\[\hat y_{new}=-2+5\times5\]

\[\hat y_{new}=-2+25\]

\[\hat y_{new}=23\]

2

Terms under the square root L4 p.37 · \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\)

\[1+\frac1{10}+\frac{(5-0)^2}{16}=1+0.1+\frac{25}{16}\]

\[1+0.1+1.5625=2.6625\]

3

Square root, times \(\hat\sigma\)

\[\hat\sigma\sqrt{2.6625}=0.4\times1.631717\]

\[\hat\sigma\sqrt{2.6625}\approx0.652687\]

4

Half-width Step 0 · \(t_{0.95,8}=1.86\)

\[1.86\times0.652687\approx1.213997\]

5

Interval L4 p.37 · \(\hat y_{new}\pm t_{1-\alpha/2,n-2}\,\hat\sigma\sqrt{\cdots}\)

\[23\pm1.213997\]

\[(23-1.213997,\ 23+1.213997)\approx(21.786,\ 24.214)\]

[figure]
Figure 6-2. Fitted line \(\hat y=-2+5x\) (blue) and 90% prediction band (green). Orange bar: the PI \((21.786,\ 24.214)\) at \(x_{new}=5\). The band is narrowest at \(\bar x=0\) and widens as \((x_{new}-\bar x)^2/S_{xx}\) increases.

**Why · The derivations** Added

Only the two new steps of Q2.

**(i) From \(P(T>-1.86)=0.95\) to \(t_{0.95,8}=1.86\)**

1

\[P(T>-1.86)=0.95\]

**Basis:** the question

2

\[P(-T<1.86)=0.95\]

3

\[P(T<1.86)=0.95\]

**Basis:** \(-T\) has the same distribution as \(T\) (symmetry)

4

\[t_{0.95,8}=1.86\]

**Basis:** \(P(T<t_{q,8})=q\)

**(ii) A 90% interval uses \(t_{0.95,8}\)**

1

\[100(1-\alpha)\%=90\%\]

2

\[\alpha=0.10\]

3

\[1-\frac{\alpha}{2}=1-0.05\]

4

\[1-\frac{\alpha}{2}=0.95\]

5

\[t_{1-\frac{\alpha}{2},n-2}=t_{0.95,8}\qquad\blacksquare\]

**Basis:** Slides p.37, \(n-2=8\)

### 06.3 Practice Added

Exercise 6.1

Model \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\). \(n=12\), \(S_{xx}=40\), \(\bar x=3\), \(S_{xy}=60\), \(\bar y=10\). Calculate the MLEs for \(\beta_0\) and \(\beta_1\).

Answer

1

Slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{60}{40}\]

\[\hat\beta_1=1.5\]

2

Intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=10-1.5\times3\]

\[\hat\beta_0=10-4.5\]

\[\hat\beta_0=5.5\]

Exercise 6.2

Setting of Practice Q2 (\(n=10,\ \hat\beta_0=-2,\ \hat\beta_1=5,\ \hat\sigma=0.4\)). Find \(\hat\sigma_{MLE}\).

Answer

1

Ratio formula 06.1 Why (ii) · \(\hat\sigma/\hat\sigma_{MLE}=\sqrt{n/(n-2)}\)

\[\hat\sigma_{MLE}=\hat\sigma\sqrt{\frac{n-2}{n}}\]

2

Put in \(n=10\), \(\hat\sigma=0.4\)

\[\hat\sigma_{MLE}=0.4\sqrt{\frac{8}{10}}\]

\[\hat\sigma_{MLE}=0.4\times0.894427\]

\[\hat\sigma_{MLE}\approx0.3578\]

Exercise 6.3

Setting of Practice Q2 (\(n=10,\ \hat\beta_1=5,\ \hat\beta_0=-2,\ S_{xx}=16,\ \bar x=0,\ \hat\sigma=0.4\), \(P(T<2.31)=0.975\) for \(T\sim t_{(8)}\)). Calculate a 95% PI for a new response at \(x_{new}=2\).

Answer

1

Quantile How Step 0

\[95\%\Rightarrow 1-\alpha/2=0.975\Rightarrow t_{0.975,8}=2.31\]

2

Center L4 p.37 · \(\hat y_{new}\)

\[\hat y_{new}=-2+5\times2\]

\[\hat y_{new}=8\]

3

Terms under the square root L4 p.37

\[1+\frac1{10}+\frac{(2-0)^2}{16}=1+0.1+0.25\]

\[1+0.1+0.25=1.35\]

4

Times \(\hat\sigma\), then times the quantile

\[0.4\sqrt{1.35}=0.4\times1.161895\approx0.464758\]

\[2.31\times0.464758\approx1.073591\]

5

Interval

\[8\pm1.073591\approx(6.926,\ 9.074)\]