Lecture 4: Simple Linear Regression: Prediction

Lecture 4

Simple Linear Regression: Prediction

This lecture estimates the mean response at a given \(x_0\), gives a confidence interval for it, and gives a prediction interval for a new response (Lecture 4 · p.1–43).

Unit 01Recap: the model, the estimators, and inference for \(\beta_1\)

Plan · Slides p.1–7

Model (p.3), estimators (p.4), CI (p.5), test (p.6), brainhead results (p.7).

Data for this unit · Added

Small data set: \(x=(1,2,3,4,5)\), \(y=(2,4,5,4,5)\), \(n=5\).

Brainhead data: \(y\) is brain weight (g). \(x\) is head size (cm³).

01.1Title page and section page Slides p.1–2

Slides p.1 title: "Lecture 4: Simple Linear Regression: Prediction".
Prediction is the use of a fitted model to give a value for a new response (Unit 04).
Slides p.2: section page "Recap" (Slides p.3–7).

01.2The model and its assumptions Slides p.3

What Slides p.3

Simple Linear Regression · Lecture 4 · p.3 \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\] Or: \[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\] Assumptions:
1. Linearity
2. Independence
3. Normality
4. Equal variance (homoskedasticity)
In simple linear regression (SLR), the mean of the response \(y\) is a straight line in one covariate \(x\).
The response \(y_i\) is the value we explain for observation \(i=1,\dots,n\), where \(n\) is the sample size.
The covariate \(x_i\) is the known constant that explains \(y_i\).
The intercept \(\beta_0\) is the mean of \(y\) at \(x=0\).
The slope \(\beta_1\) is the change in the mean of \(y\) per 1-unit increase in \(x\).
A parameter is an unknown constant of the model, for example \(\beta_0\) and \(\beta_1\).
The error \(\epsilon_i\) is the random difference between \(y_i\) and the line \(\beta_0+\beta_1x_i\).
\(N(\mu,\sigma^2)\) is the normal distribution: bell-shaped, with mean \(\mu\) and variance \(\sigma^2\).
The expectation \(E[\cdot]\) is the long-run average of a random variable. The variance \(\mathrm{Var}[\cdot]\) is the expected squared distance from it.
The error variance \(\sigma^2\) is the third parameter. \(\sigma\) is the error standard deviation.
"iid" means independent and identically distributed.
"indep" means independent, not identically distributed, because the mean changes with \(x_i\).
1. Linearity

\(E[y_i]=\beta_0+\beta_1x_i\).

2. Independence

The errors \(\epsilon_i\) are independent.

3. Normality

Each \(\epsilon_i\) has a normal distribution.

4. Equal variance (homoskedasticity)

Each \(\epsilon_i\) has variance \(\sigma^2\), at all \(x_i\).

How Added

Write a data set as an SLR model
  1. Find the response \(y\): the variable to explain or predict.
  2. Find the covariate \(x\): the variable that explains \(y\).
  3. Count the observations to get \(n\).
  4. Write \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,n\).
  5. Write \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\). List the four assumptions.

Self-check: \(\beta_0\), \(\beta_1\), \(\sigma^2\) are unknown. The data give \(x_i\), \(y_i\), \(n\).

Example 1 · The small data set Added

Pairs: \((1,2),(2,4),(3,5),(4,4),(5,5)\).

Step 1: \(y\): the second value of each pair.
Step 2: \(x\): the first value of each pair.
Step 3: Five pairs give \(n=5\).
Step 4: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,5\).
Step 5: \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\). For observation 3, \(y_3\sim N(\beta_0+3\beta_1,\sigma^2)\).

Example 2 · The brainhead data Slides p.7

Step 1: \(y_i\): brain weight of person \(i\) (g).
Step 2: \(x_i\): head size of person \(i\) (cm³).
Step 3: Slides p.7 prints \(n-2=234\) degrees of freedom. \(n=234+2=236\).
Step 4: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), \(i=1,\dots,236\).
Step 5: \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\).
3000 3500 4000 4500 1100 1200 1300 1400 1500 1600 Head size x (cm³) Brain weight y (g)
Figure 1-1. The 236 brainhead points (blue) and the fitted line \(\hat y=335.43+0.26082\,x\) (orange). The points follow a line with almost equal spread.

Why · The two forms are equal Added

1
\[y_i=\beta_0+\beta_1x_i+\epsilon_i\]
2
\[E[y_i]=\beta_0+\beta_1x_i+E[\epsilon_i]\]
\(\beta_0+\beta_1x_i\) is a constant.
3
\[E[y_i]=\beta_0+\beta_1x_i+0=\beta_0+\beta_1x_i\]
\(\epsilon_i\sim N(0,\sigma^2)\) has mean 0.
4
\[\mathrm{Var}[y_i]=\mathrm{Var}[\epsilon_i]\]
A constant does not change the variance.
5
\[\mathrm{Var}[y_i]=\sigma^2\]
6
\[y_i\sim N(\beta_0+\beta_1x_i,\ \sigma^2)\]
A normal variable plus a constant is normal.
7
\[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\ \sigma^2)\]
The \(\epsilon_i\) are independent. ∎

01.3The least squares estimators and the distribution of \(\hat\beta_1\) Slides p.4

What Slides p.4

Recap: Estimates · Lecture 4 · p.4 So the least squares estimators are: \[\hat\beta_1=\frac{S_{xy}}{S_{xx}},\qquad \hat\beta_0=\bar y-\hat\beta_1\bar x.\] \[\hat\beta_1\sim N\!\left(\beta_1,\frac{\sigma^2}{S_{xx}}\right)\ \Longrightarrow\ \frac{\hat\beta_1-\beta_1}{\sigma/\sqrt{S_{xx}}}\sim N(0,1)\ \Longrightarrow\ \frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]
An estimator is a formula that gives a parameter value from the data. A hat marks it: \(\hat\beta_1\).
Least squares (LS) chooses the \(\hat\beta_0\) and \(\hat\beta_1\) that make \(\sum_{i=1}^n(y_i-\hat\beta_0-\hat\beta_1x_i)^2\) smallest.
The sample means are \(\bar x=\frac1n\sum_{i=1}^n x_i\) and \(\bar y=\frac1n\sum_{i=1}^n y_i\).
\(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2\) is the sum of squared distances of \(x_i\) from \(\bar x\).
\(S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\) is the sum of the products of the two distances.
The fitted value is \(\hat y_i=\hat\beta_0+\hat\beta_1x_i\). The residual is \(e_i=y_i-\hat y_i\).
\(\hat\sigma\) estimates \(\sigma\), with \(\hat\sigma^2=\sum_{i=1}^n e_i^2/(n-2)\).
The standard error \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\) estimates the standard deviation of \(\hat\beta_1\).
\(N(0,1)\) is the standard normal distribution: mean 0, variance 1.
\(t_{n-2}\) is the \(t\) distribution with \(n-2\) degrees of freedom: symmetric about 0, with wider tails than \(N(0,1)\).

Putting \(\hat\sigma\) for the unknown \(\sigma\) changes \(N(0,1)\) to \(t_{n-2}\).

How Added

Calculate \(\hat\beta_0\), \(\hat\beta_1\), \(\hat\sigma\), and \(\mathrm{SE}(\hat\beta_1)\)
  1. Calculate \(\bar x\) and \(\bar y\).
  2. Calculate \(S_{xx}\) and \(S_{xy}\).
  3. Calculate \(\hat\beta_1=S_{xy}/S_{xx}\).
  4. Calculate \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
  5. Calculate each fitted value \(\hat y_i\) and each residual \(e_i\).
  6. Calculate \(\hat\sigma^2=\sum e_i^2/(n-2)\) and \(\hat\sigma\).
  7. Calculate \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\).

Self-check: The residuals add to 0. The line contains \((\bar x,\bar y)\).

Example 3 · The small data set Added

Step 1: \(\bar x=(1+2+3+4+5)/5=15/5=3\). \(\bar y=(2+4+5+4+5)/5=20/5=4\).
Step 2: \(x_i-\bar x\): \(-2,-1,0,1,2\). \(y_i-\bar y\): \(-2,0,1,0,1\).
\(S_{xx}=4+1+0+1+4=10\).
\(S_{xy}=(-2)(-2)+(0)(-1)+(1)(0)+(0)(1)+(1)(2)=4+0+0+0+2=6\).
Step 3: \(\hat\beta_1=6/10=0.6\).
Step 4: \(\hat\beta_0=4-0.6\times3=4-1.8=2.2\).
Step 5: \(\hat y_i=2.2+0.6x_i\): \(2.8,\ 3.4,\ 4.0,\ 4.6,\ 5.2\).
\(e_i=y_i-\hat y_i\): \(-0.8,\ 0.6,\ 1.0,\ -0.6,\ -0.2\).
Step 6: \(\sum e_i^2=0.64+0.36+1.00+0.36+0.04=2.4\).
\(\hat\sigma^2=2.4/(5-2)=2.4/3=0.8\), and \(\hat\sigma=\sqrt{0.8}=0.894427\).
Step 7: \(\mathrm{SE}(\hat\beta_1)=0.894427/\sqrt{10}=0.894427/3.162278=0.282843\).

Self-check: \(-0.8+0.6+1.0-0.6-0.2=0\), and \(2.2+0.6\times3=4=\bar y\). With \(n-2=3\):

\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}=\frac{0.6-\beta_1}{0.282843}\sim t_{3}.\]

Example 4 · The brainhead data Slides p.7

\(\hat\beta_0=335.43231\) and \(\hat\beta_1=0.26082\).
\(\mathrm{SE}(\hat\beta_1)=0.01307\).
\(\hat\sigma=72.35\), with \(n-2=234\) degrees of freedom.

Each 1 cm³ more head size increases the estimated mean brain weight by 0.26082 g.

\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}=\frac{0.26082-\beta_1}{0.01307}\sim t_{234}.\]

Why · The distribution chain Added

1
\[\hat\beta_1=\frac{\sum_i(y_i-\bar y)(x_i-\bar x)}{S_{xx}}\]
2
\[\hat\beta_1=\frac{\sum_i y_i(x_i-\bar x)-\bar y\sum_i(x_i-\bar x)}{S_{xx}}\]
3
\[\hat\beta_1=\frac{\sum_i y_i(x_i-\bar x)-\bar y\cdot 0}{S_{xx}}\]
\(\sum_i(x_i-\bar x)=n\bar x-n\bar x=0\).
4
\[\hat\beta_1=\sum_{i=1}^n w_iy_i,\qquad w_i=\frac{x_i-\bar x}{S_{xx}}\]
\(w_i\) uses only \(x\): a constant.
5
\[\hat\beta_1\sim N\!\left(\sum_i w_i(\beta_0+\beta_1x_i),\ \sigma^2\sum_i w_i^2\right)\]
A weighted sum of independent normals is normal.
6
\[E[\hat\beta_1]=\beta_0\sum_i w_i+\beta_1\sum_i w_ix_i\]
7
\[E[\hat\beta_1]=\beta_0\cdot0+\beta_1\frac{\sum_i(x_i-\bar x)x_i}{S_{xx}}\]
\(\sum_i w_i=\sum_i(x_i-\bar x)/S_{xx}=0\).
8
\[E[\hat\beta_1]=\beta_1\frac{\sum_i(x_i-\bar x)(x_i-\bar x)}{S_{xx}}\]
\(\sum_i(x_i-\bar x)\bar x=0\).
9
\[E[\hat\beta_1]=\beta_1\frac{S_{xx}}{S_{xx}}=\beta_1\]
10
\[\mathrm{Var}[\hat\beta_1]=\sigma^2\sum_i\frac{(x_i-\bar x)^2}{S_{xx}^2}\]
11
\[\mathrm{Var}[\hat\beta_1]=\sigma^2\frac{S_{xx}}{S_{xx}^2}=\frac{\sigma^2}{S_{xx}}\]
12
\[\hat\beta_1\sim N\!\left(\beta_1,\frac{\sigma^2}{S_{xx}}\right)\]
The first result on Slides p.4.
13
\[Z=\frac{\hat\beta_1-\beta_1}{\sigma/\sqrt{S_{xx}}}\sim N(0,1)\]
Subtract the mean. Divide by the standard deviation.
14
\[V=\frac{1}{\sigma^2}\sum_{i=1}^n e_i^2\sim\chi^2_{(n-2)},\quad Z,V\text{ independent}\]
Stated in Lecture 3. \(\chi^2_\nu\): the chi-squared distribution.
15
\[\frac{Z}{\sqrt{V/(n-2)}}\sim t_{n-2}\]
Independent \(Z\sim N(0,1)\), \(V\sim\chi^2_\nu\) give \(Z/\sqrt{V/\nu}\sim t_\nu\).
16
\[\frac{(\hat\beta_1-\beta_1)\big/(\sigma/\sqrt{S_{xx}})}{\sqrt{\frac{1}{\sigma^2}\sum_i e_i^2/(n-2)}}\sim t_{n-2}\]
17
\[\frac{(\hat\beta_1-\beta_1)\big/(1/\sqrt{S_{xx}})}{\sqrt{\sum_i e_i^2/(n-2)}}\sim t_{n-2}\]
18
\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]
\(\hat\sigma=\sqrt{\sum_i e_i^2/(n-2)}\). ∎

01.4The confidence interval for \(\beta_1\) Slides p.5

What Slides p.5

Confidence interval for \(\beta_1\) · Lecture 4 · p.5 95% CI for \(\beta_1\) is \(\hat\beta_1\pm t_{0.975,n-2}\times\sqrt{\dfrac{\hat\sigma^2}{S_{xx}}}\)
or \(\hat\beta_1\pm t_{0.975,n-2}\times \mathrm{SE}(\hat\beta_1)\).
If we were to collect a new sample, we would obtain a different estimate and a different 95% CI
  • 95% of such intervals will cover the true \(\beta_1\)
More generally a \(100\times(1-\alpha)\%\) CI is: \(\hat\beta_1\pm t_{1-\alpha/2,n-2}\times \mathrm{SE}(\hat\beta_1)\)
A confidence interval (CI) is an interval calculated from the data with a fixed rule.
The confidence level \(100\times(1-\alpha)\%\) is the proportion of such intervals that cover the true parameter. For 95%, \(\alpha=0.05\).
The quantile \(t_{1-\alpha/2,n-2}\) has area \(1-\alpha/2\) to its left under the \(t_{n-2}\) curve.
For 95%, the quantile is \(t_{0.975,n-2}\), with right-tail area 0.025.
\(\sqrt{\hat\sigma^2/S_{xx}}=\hat\sigma/\sqrt{S_{xx}}=\mathrm{SE}(\hat\beta_1)\).

How Added

Calculate a \(100\times(1-\alpha)\%\) CI for \(\beta_1\)
  1. Get \(\hat\beta_1\) and \(\mathrm{SE}(\hat\beta_1)\).
  2. Calculate the degrees of freedom \(n-2\).
  3. Find \(t_{1-\alpha/2,n-2}\) in a \(t\) table or with software.
  4. Calculate the half-width \(t_{1-\alpha/2,n-2}\times\mathrm{SE}(\hat\beta_1)\).
  5. Calculate \(\hat\beta_1-\text{half-width}\) and \(\hat\beta_1+\text{half-width}\).

Self-check: The midpoint is \(\hat\beta_1\). A higher confidence level gives a wider interval.

Example 5 · 95% CI for the small data set Added

Step 1: \(\hat\beta_1=0.6\) and \(\mathrm{SE}(\hat\beta_1)=0.282843\) (Example 3).
Step 2: \(n-2=5-2=3\).
Step 3: \(t_{0.975,3}=3.182446\).
Step 4: Half-width \(=3.182446\times0.282843=0.900132\).
Step 5: \(0.6-0.900132=-0.300132\) and \(0.6+0.900132=1.500132\).

The 95% CI is \([-0.300132,\ 1.500132]\). It is wide because \(n=5\) is small and \(t_{0.975,3}\) is large.

Example 6 · 95% CI for the brainhead data Slides p.7

Step 1: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).
Step 2: \(n-2=236-2=234\).
Step 3: \(t_{0.975,234}=1.970154\).
Step 4: Half-width \(=1.970154\times0.01307=0.025750\).
Step 5: \(0.26082-0.025750=0.235070\) and \(0.26082+0.025750=0.286570\).

Slides p.7 prints \([0.2350725,\ 0.2865689]\) (more input digits). We are 95% confident that 1 cm³ more head size increases the mean brain weight by 0.2351 g to 0.2866 g.

−1.970 0 1.970 area 0.95 area 0.025 area 0.025 t₂₃₄ density
Figure 1-2. The \(t_{234}\) density: center area 0.95 (blue), each tail 0.025 (orange). Right boundary: \(t_{0.975,234}=1.970154\).

Why · The interval formula Added

Let \(q=t_{0.975,n-2}\) and \(T\sim t_{n-2}\).

1
\[0.95=P(-q\le T\le q)\]
\(t\) is symmetric. Each tail has area 0.025.
2
\[0.95=P\!\left(-q\le\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\le q\right)\]
The \(t_{n-2}\) result of Section 01.3.
3
\[0.95=P\!\left(-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\hat\beta_1-\beta_1\le q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]
4
\[0.95=P\!\left(-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\beta_1-\hat\beta_1\le q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]
5
\[0.95=P\!\left(\hat\beta_1-q\frac{\hat\sigma}{\sqrt{S_{xx}}}\le\beta_1\le\hat\beta_1+q\frac{\hat\sigma}{\sqrt{S_{xx}}}\right)\]
6
\[\hat\beta_1\pm t_{0.975,n-2}\times\mathrm{SE}(\hat\beta_1)\]
Use \(1-\alpha/2\) for 0.975 in general. ∎

In line 5, \(\hat\beta_1\) and \(\hat\sigma\) are random; \(\beta_1\) is fixed. 0.95 is the probability that the interval covers \(\beta_1\).

01.5The hypothesis test for \(\beta_1\) Slides p.6

What Slides p.6

Hypothesis Testing · Lecture 4 · p.6 Want to test \(H_0:\beta_1=\theta_0\) vs \(H_1:\beta_1\neq\theta_0\).
Under \(H_0\): \(\dfrac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\)
  • P-value: What's the probability under the null of observing a test statistic as or more extreme than the one we observed?
    • I.e. for \(T\sim t_{n-2}\) and our observed test statistic \(t^{obs}\): \(P(|T|\ge|t^{obs}|)\)
A hypothesis test decides if the data agree with a statement about a parameter.
The null hypothesis \(H_0:\beta_1=\theta_0\) is the tested statement. \(\theta_0\) is given, usually 0 (no linear relation).
The alternative hypothesis \(H_1:\beta_1\neq\theta_0\) is the statement if \(H_0\) is false. \(\neq\) makes the test two-sided.
The test statistic is \(\dfrac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\). \(t^{obs}\) is its value for our data.
The p-value is the probability, under \(H_0\), of a test statistic as extreme as \(t^{obs}\) or more.
A small p-value is evidence against \(H_0\).

How Added

Test \(H_0:\beta_1=\theta_0\) vs \(H_1:\beta_1\neq\theta_0\)
  1. Write \(H_0\) and \(H_1\). Find \(\theta_0\).
  2. Get \(\hat\beta_1\) and \(\mathrm{SE}(\hat\beta_1)\).
  3. Calculate \(t^{obs}=(\hat\beta_1-\theta_0)/\mathrm{SE}(\hat\beta_1)\).
  4. Calculate the p-value \(2P(T\ge|t^{obs}|)\), with \(T\sim t_{n-2}\).
  5. Compare the p-value with the significance level \(\alpha\), usually 0.05.
  6. If the p-value < \(\alpha\), reject \(H_0\). If not, do not reject it.

Self-check: For \(\theta_0=0\), the 95% CI excludes 0 exactly when the p-value < 0.05.

Example 7 · Test \(H_0:\beta_1=0\) for the small data set Added

Step 1: \(H_0:\beta_1=0\) vs \(H_1:\beta_1\neq0\). \(\theta_0=0\).
Step 2: \(\hat\beta_1=0.6\) and \(\mathrm{SE}(\hat\beta_1)=0.282843\).
Step 3: \(t^{obs}=(0.6-0)/0.282843=2.121320\).
Step 4: \(2P(T\ge2.121320)=0.124027\), with \(T\sim t_3\).
Step 5–6: \(0.124027>0.05\). Do not reject \(H_0\).

The 95% CI of Example 5, \([-0.300132,\ 1.500132]\), also contains 0.

Example 8 · Test \(H_0:\beta_1=0\) for the brainhead data Slides p.7

Step 1: \(H_0:\beta_1=0\) vs \(H_1:\beta_1\neq0\). \(\theta_0=0\).
Step 2: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).
Step 3: \(t^{obs}=(0.26082-0)/0.01307=19.956\). Slides p.7 prints 19.957 (more input digits).
Step 4: \(2P(T\ge19.957)=2.03\times10^{-52}\), \(T\sim t_{234}\). Slides p.7 prints "< \(2\times10^{-16}\)".
Step 5–6: The p-value is much less than 0.05. Reject \(H_0\).
0 −2 2 tᵒᵇˢ = 19.957 is far to the right (the axis ends near ±4) Under H₀: T ~ t₂₃₄
Figure 1-3. The \(t_{234}\) density of \(T\) under \(H_0\), almost all between \(-4\) and 4. \(t^{obs}=19.957\) is far right; its tail area is too small to see.

Head size and brain weight have a linear relation. The 95% CI \([0.2351,\ 0.2866]\) also excludes 0.

Why · The test statistic has the \(t_{n-2}\) distribution Added

1
\[\frac{\hat\beta_1-\beta_1}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\]
Section 01.3, line 18. True for each \(\beta_1\).
2
\[\frac{\hat\beta_1-\theta_0}{\hat\sigma/\sqrt{S_{xx}}}\sim t_{n-2}\quad\text{under }H_0\]
\(H_0\) gives \(\beta_1=\theta_0\).
3
\[P(|T|\ge|t^{obs}|)=P(T\ge|t^{obs}|)+P(T\le-|t^{obs}|)\]
4
\[P(|T|\ge|t^{obs}|)=2P(T\ge|t^{obs}|)\]
\(t\) is symmetric about 0. ∎

01.6The brainhead results on Slides p.7 Slides p.7

What Slides p.7

Intercept: estimate \(\hat\beta_0=335.43231\), standard error 47.77644, \(t\) value 7.021, p-value \(2.37\times10^{-11}\).
Slope: estimate \(\hat\beta_1=0.26082\), standard error 0.01307, \(t\) value 19.957, p-value less than \(2\times10^{-16}\).
\(\hat\sigma=72.35\) on 234 degrees of freedom.
95% CI for \(\beta_0\): \([241.3053884,\ 429.5592416]\).
95% CI for \(\beta_1\): \([0.2350725,\ 0.2865689]\).

Later lectures explain the numbers below.

Residuals: minimum \(-174.88\), first quartile \(-49.03\), median \(-2.40\), third quartile 46.72, maximum 242.41.
\(R^2=0.6299\) and adjusted \(R^2=0.6283\).
F-statistic 398.3 on 1 and 234 degrees of freedom, p-value less than \(2.2\times10^{-16}\).

How Added

Connect the printed numbers to the formulas
  1. Match each estimate to \(\hat\beta_0\) or \(\hat\beta_1\).
  2. Match the slope standard error to \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\).
  3. Divide each estimate by its standard error: \(t^{obs}\) for \(\theta_0=0\).
  4. Match each p-value to \(P(|T|\ge|t^{obs}|)\), \(T\sim t_{n-2}\).
  5. Add 2 to the degrees of freedom: \(n\).
  6. Compare the midpoint of each CI with the estimate.

Self-check: Each \(t\) value is the estimate divided by its standard error.

Example 9 · Check each number Slides p.7

Step 1: \(\hat\beta_0=335.43231\) and \(\hat\beta_1=0.26082\).
Step 2: \(\mathrm{SE}(\hat\beta_1)=0.01307\).
Step 3: Slope: \(0.26082/0.01307=19.956\). Slides p.7 prints 19.957 (more input digits).
Intercept: \(335.43231/47.77644=7.021\), as printed.
Step 4: \(2P(T\ge7.021)=2.37\times10^{-11}\) with \(T\sim t_{234}\), as printed.
Step 5: \(n=234+2=236\).
Step 6: Slope CI midpoint: \((0.2350725+0.2865689)/2=0.5216414/2=0.2608207\). This rounds to 0.26082.
Intercept CI midpoint: \((241.3053884+429.5592416)/2=670.8646300/2=335.432315\). This rounds to 335.43231.

01.7Practice Added

Q1. From Slides p.7, calculate a 90% CI for \(\beta_1\). Use \(t_{0.95,234}=1.651391\). Interpret it.

Answer
Step 1: \(\hat\beta_1=0.26082\) and \(\mathrm{SE}(\hat\beta_1)=0.01307\).
Step 2: \(\alpha=0.10\) and \(1-\alpha/2=0.95\). The quantile is \(t_{0.95,234}=1.651391\).
Step 3: Half-width \(=1.651391\times0.01307=0.021584\).
Step 4: \(0.26082-0.021584=0.239236\) and \(0.26082+0.021584=0.282404\).

The 90% CI is \([0.239236,\ 0.282404]\). We are 90% confident that 1 cm³ more head size increases the mean brain weight by 0.2392 g to 0.2824 g. It is narrower than the 95% CI because 1.651391 < 1.970154.

Q2. Test \(H_0:\beta_1=0.25\) vs \(H_1:\beta_1\neq0.25\) at the 5% level. Report \(t^{obs}\), the p-value, and your conclusion.

Answer
Step 1: \(\theta_0=0.25\).
Step 2: \(t^{obs}=(0.26082-0.25)/0.01307=0.01082/0.01307=0.82785\).
Step 3: \(2P(T\ge0.82785)=0.4086\), with \(T\sim t_{234}\).
Step 4: \(0.4086>0.05\). Do not reject \(H_0\). The data agree with \(\beta_1=0.25\).

The 95% CI \([0.2351,\ 0.2866]\) also contains 0.25.

Q3. Verify the 95% CI for \(\beta_0\) on Slides p.7. Use \(t_{0.975,234}=1.970154\).

Answer
Step 1: \(\hat\beta_0=335.43231\), with standard error 47.77644.
Step 2: Half-width \(=1.970154\times47.77644=94.1269\).
Step 3: \(335.43231-94.1269=241.3054\) and \(335.43231+94.1269=429.5592\).

This agrees with \([241.3053884,\ 429.5592416]\) on Slides p.7.

Unit 02Point estimate of the mean response and its distribution

Plan · Slides p.8–18

\(\hat\mu_0\) and its second form (p.10–13), \(E[\hat\mu_0]\) (p.14–15), \(\mathrm{Var}[\hat\mu_0]\) (p.16–17), distribution (p.18).

Slides p.8 Page 8 repeats the Lecture 4 title page.

Slides p.9 Page 9 is the section page "Estimating Mean Response".

Model and notation from earlier · Slides p.3–4 (unit 01 defines each item)

Model: \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\), or \(y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\).

Estimators: \(\hat\beta_1=S_{xy}/S_{xx}\), \(\hat\beta_0=\bar y-\hat\beta_1\bar x\), with \(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2\), \(S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\).

An estimator is random. An estimate is its number for one data set.

Example data · Slides p.7 and Added

Brainhead data: \(n=236\) persons, \(x_i\) head size (cm³), \(y_i\) brain weight (g).

Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\), \(n-2=234\).

From brainhead.csv (Added): \(\bar x=3637.864\), \(\bar y=1284.263\), \(S_{xx}=30647233.66\), \(\hat\beta_1=0.2608207\).

02.1The mean response \(\mu_0\) and its point estimate \(\hat\mu_0\) Slides p.10–13

What Slides p.10

Lecture 4 · p.10 Recall the mean response is \(E[y_i\mid x_i]=\beta_0+\beta_1x_i\).
For an arbitrary \(x_0\), the mean outcome is \(\mu_0:=E[y_0\mid x_0]=\beta_0+\beta_1x_0\).
We can estimate this as \[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]

The conditional expectation \(E[y\mid x]\) is the long-run average of \(y\) at one fixed \(x\).

The mean response \(E[y\mid x]\) is the height \(\beta_0+\beta_1x\) of the true line, because \(E[\epsilon]=0\).

\(x_0\) is a covariate value of interest, not necessarily in the data. \(y_0\) is the response at \(x_0\).

The symbol \(:=\) means "is defined as".

A point estimate is one number that estimates an unknown quantity. \(\hat\mu_0\) is the point estimate of \(\mu_0\).

\(\mu_0\) is unknown, because \(\beta_0\) and \(\beta_1\) are unknown. \(\hat\mu_0\), the height of the fitted line at \(x_0\), comes from the data.

Lecture 4 · p.11–13 \[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0=\left(\bar y-\hat\beta_1\bar x\right)+\hat\beta_1x_0=\bar y+\hat\beta_1(x_0-\bar x)\] Intuition?
• What happens at \(\bar x\)?

The second form starts at the center \((\bar x,\bar y)\). It moves along the fitted line by \(x_0-\bar x\), with height change \(\hat\beta_1\) per unit.

At \(x_0=\bar x\) (Slides p.13), the distance is 0, and \(\hat\mu_0=\bar y\).

How Added

Calculate \(\hat\mu_0\)
  1. Calculate \(n\), \(\bar x\), \(\bar y\), \(S_{xx}\) and \(S_{xy}\).
  2. Calculate \(\hat\beta_1=S_{xy}/S_{xx}\) and \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
  3. Form 1 (Slides p.10): put \(x_0\) into \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
  4. Form 2 (Slides p.12): calculate \(x_0-\bar x\).
  5. Calculate \(\bar y+\hat\beta_1(x_0-\bar x)\).

Self-check: ① Both forms agree, to rounding. ② At \(x_0=\bar x\), the result is \(\bar y\). ③ If \(\hat\beta_1>0\) and \(x_0>\bar x\), the result is above \(\bar y\).

Example 1 · The brainhead data at \(x_0=4000\) cm³ Added

1

Start values How step 1

\[n=236,\quad \bar x=3637.864,\quad \bar y=1284.263,\quad S_{xx}=30647233.66\]

From brainhead.csv (Added).

2

Coefficients How step 2

\[\hat\beta_1=0.2608207,\qquad \hat\beta_0=335.4323\]

Basis: Slides p.4. Slides p.7 prints 335.43231 and 0.26082.

3

Form 1 How step 3 · Slides p.10

\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]
\[\hat\mu_0=335.4323+0.2608207\times4000\]
\[\hat\mu_0=335.4323+1043.2828=1378.7151\]
4

Form 2 How steps 4–5 · Slides p.12

\[x_0-\bar x=4000-3637.864=362.136\]
\[\hat\beta_1(x_0-\bar x)=0.2608207\times362.136=94.452\]
\[\hat\mu_0=\bar y+\hat\beta_1(x_0-\bar x)=1284.263+94.452=1378.715\]

Self-check: Both forms give 1378.715, above \(\bar y=1284.263\) (check ③).

5

At \(x_0=\bar x\) Slides p.13

\[\hat\mu_0=\bar y+\hat\beta_1(\bar x-\bar x)=\bar y+\hat\beta_1\times0=\bar y=1284.263\]
3000 3500 4000 4500 Head size (cm³) 100011001200 1300140015001600 Brain weight (g) x₀ − x̄ = 362.1 (x̄, ȳ) = (3637.9, 1284.3) (4000, 1378.7) ŷ = 335.43 + 0.2608x
Figure 2-1. Brainhead data (blue), fitted line (orange), center \((\bar x,\bar y)\) (purple), and \(\hat\mu_0=1378.715\) at \(x_0=4000\) (green). From purple, move right \(x_0-\bar x=362.1\); the height increases \(\hat\beta_1\times362.1=94.45\).

At 4000 cm³, the estimated mean brain weight is \(\hat\mu_0=1378.715\) g. At \(x_0=\bar x\), it is \(\bar y=1284.263\) g.

Why · The two forms are equal Slides p.10–13

1
\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\]

Definition, Slides p.10.

2
\[=\left(\bar y-\hat\beta_1\bar x\right)+\hat\beta_1x_0\]

Slides p.4, p.11.

3
\[=\bar y+\hat\beta_1x_0-\hat\beta_1\bar x\]
4
\[=\bar y+\hat\beta_1(x_0-\bar x)\]

Slides p.12. ∎

At \(x_0=\bar x\) (Slides p.13):

1
\[\hat\mu_0=\bar y+\hat\beta_1(\bar x-\bar x)\]
2
\[=\bar y+\hat\beta_1\times0\]
3
\[=\bar y\]

The fitted line contains \((\bar x,\bar y)\). ∎

02.2\(\hat\mu_0\) is unbiased Slides p.14–15

What Slides p.15

Lecture 4 · p.15 \[E[\hat\mu_0]=E[\hat\beta_0+\hat\beta_1x_0]=E[\hat\beta_0]+E[\hat\beta_1]x_0=\beta_0+\beta_1x_0\]

An estimator is unbiased when its expectation equals the parameter it estimates. \(E[\hat\mu_0]=\mu_0\).

Repeated sampling: keep \(x_1,\dots,x_n\), get new \(y_1,\dots,y_n\), and calculate \(\hat\mu_0\) again.

One \(\hat\mu_0\) can be too high or too low. On average over repeated samples, it is correct.

How Added

Find \(E[\hat\mu_0]\)
  1. Write \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
  2. Use linearity of expectation: \(E[aU+bV]=aE[U]+bE[V]\) for constants \(a\), \(b\).
  3. Treat \(x_0\) as a constant.
  4. Put in \(E[\hat\beta_0]=\beta_0\) and \(E[\hat\beta_1]=\beta_1\) (Lecture 3).

Self-check: The result has no hat. It is \(\beta_0+\beta_1x_0=\mu_0\).

Example 2 · A small data set with known parameters Added

Real data have unknown \(\beta_0\), \(\beta_1\). This example sets them: \(n=3\), \(x=(1,2,3)\), \(\beta_0=2\), \(\beta_1=0.5\), \(x_0=4\).

Write \(\hat\mu_0=\sum_{i=1}^n c_iy_i\), with weight \(c_i=\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\) (Section 02.3, line 6).

1

Center and spread of \(x\)

\[\bar x=\frac{1+2+3}{3}=\frac{6}{3}=2\]
\[S_{xx}=(1-2)^2+(2-2)^2+(3-2)^2=1+0+1=2\]
\[x_0-\bar x=4-2=2\]
2

Weights \(c_i\)

\[c_1=\frac13+\frac{(1-2)(2)}{2}=\frac13-1=-\frac23\]
\[c_2=\frac13+\frac{(2-2)(2)}{2}=\frac13+0=\frac13\]
\[c_3=\frac13+\frac{(3-2)(2)}{2}=\frac13+1=\frac43\]
3

Expected responses

\[E[y_1]=2+0.5\times1=2.5,\quad E[y_2]=2+0.5\times2=3,\quad E[y_3]=2+0.5\times3=3.5\]

\(E[y_i]=\beta_0+\beta_1x_i\) (Slides p.3).

4

Expectation of \(\hat\mu_0\)

\[E[\hat\mu_0]=c_1E[y_1]+c_2E[y_2]+c_3E[y_3]\]
\[=-\frac23\times2.5+\frac13\times3+\frac43\times3.5\]
\[=-\frac53+1+\frac{14}3=\frac{-5+3+14}{3}=\frac{12}{3}=4\]
5

True mean response

\[\mu_0=\beta_0+\beta_1x_0=2+0.5\times4=2+2=4\]

Self-check: \(E[\hat\mu_0]=4=\mu_0\).

Why · \(\hat\mu_0\) is unbiased Slides p.14–15

1
\[E[\hat\mu_0]=E[\hat\beta_0+\hat\beta_1x_0]\]

Slides p.10.

2
\[=E[\hat\beta_0]+E[\hat\beta_1x_0]\]

Linearity of expectation.

3
\[=E[\hat\beta_0]+E[\hat\beta_1]x_0\]

\(x_0\) is a constant.

4
\[=\beta_0+\beta_1x_0\]

Lecture 3.

5
\[=\mu_0\]

Slides p.10. ∎

02.3The variance of \(\hat\mu_0\) Slides p.16–17

What Slides p.17

Lecture 4 · p.17 \[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

The denominator is \(S_{xx}\).

The first term, \(\sigma^2/n\), does not change with \(x_0\).

The second term, \(\sigma^2(x_0-\bar x)^2/S_{xx}\), increases as \(x_0\) moves away from \(\bar x\).

The variance is smallest, \(\sigma^2/n\), at \(x_0=\bar x\). Away from the center, the distance multiplies the error in \(\hat\beta_1\).

How Added

Calculate \(\mathrm{Var}[\hat\mu_0]\)
  1. Calculate \(1/n\).
  2. Calculate \((x_0-\bar x)^2/S_{xx}\).
  3. Add them to get the factor \(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\).
  4. Multiply the factor by \(\sigma^2\).
  5. For a number, use \(\hat\sigma\) for \(\sigma\) and take the square root.

Self-check: ① The factor is at least \(1/n\). ② At \(x_0=\bar x\), it is \(1/n\). ③ Equal distances from \(\bar x\) give equal variances.

Slides p.7 prints \(\hat\sigma=72.35\).

Example 3 · The brainhead data at \(x_0=4000\) and \(x_0=\bar x\) Added

1

\(x_0=4000\): the two terms How steps 1–2

\[\frac1n=\frac1{236}=0.004237288\]
\[(x_0-\bar x)^2=362.1356^2=131142.19\]
\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{131142.19}{30647233.66}=0.004279087\]
2

\(x_0=4000\): the factor How step 3

\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}=0.004237288+0.004279087=0.008516375\]
3

\(x_0=4000\): multiply by \(\sigma^2\) How steps 4–5

\[\mathrm{Var}[\hat\mu_0]=\sigma^2\times0.008516375\]
\[\sqrt{0.008516375}=0.09228421\]
\[\hat\sigma\sqrt{0.008516375}=72.35\times0.09228421=6.677\]
4

\(x_0=\bar x\)

\[\frac1n+\frac{(\bar x-\bar x)^2}{S_{xx}}=\frac1n+0=0.004237288\]
\[\sqrt{236}=15.36229\]
\[\hat\sigma\sqrt{1/n}=\frac{72.35}{15.36229}=4.710\]

Self-check: 4.710 < 6.677. The estimate is most precise at the center.

3000350040004500 x₀ (cm³) 0481216 σ̂ √(1/n + (x₀ − x̄)²/Sxx) x̄: 4.710 4000: 6.677
Figure 2-2. \(\hat\sigma\sqrt{1/n+(x_0-\bar x)^2/S_{xx}}\) for the brainhead data. Minimum \(\hat\sigma/\sqrt n=4.710\) at \(x_0=\bar x\); 6.677 at \(x_0=4000\).

Example 4 · The small data set of Example 2 Added

Calculate \(\mathrm{Var}[\hat\mu_0]\) in two ways for \(x=(1,2,3)\), \(x_0=4\).

1

Formula from Slides p.17

\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}=\frac13+\frac{2^2}{2}=\frac13+2=\frac73\]
2

Sum of squared weights

\[\sum_{i=1}^3c_i^2=\left(-\frac23\right)^2+\left(\frac13\right)^2+\left(\frac43\right)^2=\frac49+\frac19+\frac{16}9=\frac{21}9=\frac73\]

Line 7 below: \(\mathrm{Var}[\hat\mu_0]=\sigma^2\sum c_i^2\).

Self-check: Both give \(\mathrm{Var}[\hat\mu_0]=\frac73\sigma^2\).

Why · The chain for the variance Slides p.16–17

The chain follows Slides p.17, with one change per line.

Fact A (model): \(y_1,\dots,y_n\) are independent, and \(\mathrm{Var}[y_i]=\sigma^2\).

Fact B (Lecture 2): \(\mathrm{Var}(aY)=a^2\mathrm{Var}(Y)\). For independent \(X\) and \(Y\), \(\mathrm{Var}(X+Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)\).

Fact C: \(\sum_{i=1}^n(x_i-\bar x)=\sum_{i=1}^n x_i-n\bar x=n\bar x-n\bar x=0\).

Fact D (Lecture 3): \(\hat\beta_1=\sum_{i=1}^n w_iy_i\), with fixed \(w_i=\frac{x_i-\bar x}{S_{xx}}\).

1
\[\mathrm{Var}[\hat\mu_0]=\mathrm{Var}[\hat\beta_0+\hat\beta_1x_0]\]

Slides p.10.

2
\[=\mathrm{Var}[(\bar y-\hat\beta_1\bar x)+\hat\beta_1x_0]\]

Put in \(\hat\beta_0\).

3
\[=\mathrm{Var}[\bar y+\hat\beta_1(x_0-\bar x)]\]

As in Section 02.1.

4
\[=\mathrm{Var}\left[\left(\sum_{i=1}^n\frac1n y_i\right)+\hat\beta_1(x_0-\bar x)\right]\]

Put in \(\bar y=\sum_{i=1}^n\frac1n y_i\). (Added: split.)

5
\[=\mathrm{Var}\left[\left(\sum_{i=1}^n\frac1n y_i\right)+\left(\sum_{i=1}^n\frac{x_i-\bar x}{S_{xx}}y_i\right)(x_0-\bar x)\right]\]

Fact D.

6
\[=\mathrm{Var}\left[\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)y_i\right]\]

Join the two sums.

7
\[=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)^2\mathrm{Var}[y_i]\]

Facts A and B. (Added: split.)

8
\[=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)^2\sigma^2\]

Fact A.

9
\[=\sigma^2\sum_{i=1}^n\left(\frac1{n^2}+\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\,\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

\((a+b)^2=a^2+b^2+2ab\).

10
\[=\sigma^2\left(\sum_{i=1}^n\frac1{n^2}+\sum_{i=1}^n\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Split into three sums.

11
\[=\sigma^2\left(\frac1n+\sum_{i=1}^n\frac{(x_i-\bar x)^2(x_0-\bar x)^2}{S_{xx}^2}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

\(\sum_{i=1}^n\frac1{n^2}=n\cdot\frac1{n^2}=\frac1n\). (Added: split.)

12
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}\sum_{i=1}^n(x_i-\bar x)^2+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Move the constant out. (Added: split.)

13
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\sum_{i=1}^n\frac1n\,\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)\]

Definition of \(S_{xx}\). (Added: split.)

14
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\,\frac1n\,\frac{(x_0-\bar x)}{S_{xx}}\sum_{i=1}^n(x_i-\bar x)\right)\]

Move the constant out. Second-last line, Slides p.17.

15
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}^2}S_{xx}+2\,\frac1n\,\frac{(x_0-\bar x)}{S_{xx}}\cdot0\right)\]

Fact C. (Added: split.)

16
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\]

\(S_{xx}/S_{xx}^2=1/S_{xx}\). (Added: split.)

17
\[=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

Last line, Slides p.17. ∎

Number check (Added): the three sums of line 10 for \(x=(1,2,3)\), \(x_0=4\), \(\bar x=2\), \(S_{xx}=2\), \(x_0-\bar x=2\).

First: \(\sum_{i=1}^3\frac1{3^2}=3\times\frac19=\frac13\).

Second: \(\frac{(1)(4)+(0)(4)+(1)(4)}{2^2}=\frac{8}{4}=2\), equal to \((x_0-\bar x)^2/S_{xx}=4/2=2\).

Third: \(2\times\frac13\times\frac{(-1)(2)+(0)(2)+(1)(2)}{2}=2\times\frac13\times\frac{0}{2}=0\).

Total: \(\frac13+2+0=\frac73\), as in Example 4.

02.4The distribution of \(\hat\mu_0\) Slides p.18

What Slides p.18

Lecture 4 · p.18 \[\Longrightarrow\ \hat\mu_0\sim N\!\left[\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right]\qquad\text{(why?)}\]

The symbol \(\sim\) means "has the distribution".

A linear combination is \(\sum_{i=1}^n c_iy_i\) with fixed numbers \(c_i\).

How Added

Write the distribution of \(\hat\mu_0\)
  1. Family: \(\hat\mu_0\) is a linear combination of independent normal \(y_i\). It is normal.
  2. Mean: \(E[\hat\mu_0]=\mu_0=\beta_0+\beta_1x_0\) (Section 02.2).
  3. Variance: \(\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\) (Section 02.3).
  4. Put in \(n\), \(\bar x\), \(S_{xx}\) and \(x_0\).

Self-check: The mean has no hat. The only unknown in the variance is \(\sigma^2\).

Example 5 · The brainhead data at \(x_0=4000\) Added

1

Family How step 1

\[\hat\mu_0=\sum_{i=1}^{236}\left(\frac1{236}+\frac{(x_i-3637.864)(362.136)}{30647233.66}\right)y_i\]

Fixed weights, independent normal \(y_i\): normal.

2

Mean How step 2

\[E[\hat\mu_0]=\mu_0=\beta_0+4000\,\beta_1\]
3

Variance How steps 3–4

\[\hat\mu_0\sim N\!\left[\beta_0+4000\,\beta_1,\ \sigma^2\times0.008516375\right]\]

Factor 0.008516375: Example 3. Data value: \(\hat\mu_0=1378.715\).

Why · \(\hat\mu_0\) is normal (the "why?" on Slides p.18) Slides p.18

1
\[\hat\mu_0=\sum_{i=1}^n\left(\frac1n+\frac{(x_i-\bar x)(x_0-\bar x)}{S_{xx}}\right)y_i\]

Section 02.3, line 6. Each weight uses only \(n\), \(x_i\), \(\bar x\), \(x_0\), \(S_{xx}\): a fixed number.

2
\[y_i\overset{indep}{\sim}N(\beta_0+\beta_1x_i,\sigma^2)\]

Slides p.3.

3
\[\hat\mu_0\ \text{is a linear combination of independent normals}\ \Longrightarrow\ \hat\mu_0\ \text{is normal}\]

Same rule as for \(\hat\beta_1=\sum w_iy_i\) (Lecture 3).

4
\[E[\hat\mu_0]=\mu_0\]

Section 02.2 (Slides p.15).

5
\[\hat\mu_0\sim N\!\left[\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right]\]

Variance: Section 02.3 (Slides p.17). ∎

02.5Practice Added

Q1. Brainhead data: \(n=236\), \(\bar x=3637.864\), \(\bar y=1284.263\), \(\hat\beta_0=335.4323\), \(\hat\beta_1=0.2608207\). Calculate \(\hat\mu_0\) at \(x_0=3000\) cm³ in two ways: (a) \(\hat\beta_0+\hat\beta_1x_0\); (b) \(\bar y+\hat\beta_1(x_0-\bar x)\).

Answer

(a) \(0.2608207\times3000=782.4621\). \(\hat\mu_0=335.4323+782.4621=1117.894\).

(b) \(x_0-\bar x=3000-3637.864=-637.864\).

\(\hat\beta_1(x_0-\bar x)=0.2608207\times(-637.864)=-166.368\).

\(\hat\mu_0=1284.263-166.368=1117.895\). This agrees with (a) to rounding (1117.894).

\(x_0<\bar x\) and \(\hat\beta_1>0\): the result is below \(\bar y\).

Q2. Show that \(\mathrm{Var}[\hat\mu_0]\) is smallest at \(x_0=\bar x\). Give its value there in terms of \(\sigma^2\) and \(n\).

Answer

\(\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\).

\(\sigma^2/n\) does not change with \(x_0\). \((x_0-\bar x)^2\geq0\) and \(S_{xx}>0\). The second term is 0 only at \(x_0=\bar x\), and positive elsewhere.

At \(x_0=\bar x\):

\[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(\bar x-\bar x)^2}{S_{xx}}\right)=\sigma^2\left(\frac1n+0\right)=\frac{\sigma^2}{n}\]

There \(\hat\mu_0=\bar y\), and \(\mathrm{Var}[\bar y]=\sigma^2/n\) agrees.

Q3. Brainhead data, \(S_{xx}=30647233.66\). Calculate \(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\) at \(x_0=3000\) and \(x_0=4500\). Where is the estimate of \(\mu_0\) more precise? Use the distance to \(\bar x\).

Answer

\(x_0=3000\): \(x_0-\bar x=-637.8644\), \((x_0-\bar x)^2=406871.0\), \(406871.0/30647233.66=0.01327595\). Factor: \(0.004237288+0.01327595=0.01751323\).

\(x_0=4500\): \(x_0-\bar x=862.1356\), \((x_0-\bar x)^2=743277.8\), \(743277.8/30647233.66=0.02425269\). Factor: \(0.004237288+0.02425269=0.02848998\).

0.01751323 < 0.02848998: the estimate is more precise at \(x_0=3000\).

3000 is 637.9 from \(\bar x\); 4500 is 862.1 from \(\bar x\). The variance increases with \((x_0-\bar x)^2\).

Unit 03Confidence interval for the mean response

Start point · Slides p.3, p.7, p.10–18

Brainhead data, \(n=236\). Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\) on 234 degrees of freedom.

More digits from brainhead.csv (Added): \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\).

Start result (Slides p.18):

\[\hat\mu_0\sim N\!\left(\mu_0,\ \sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\right)\]

03.1Standardize: from \(N(0,1)\) to \(t_{n-2}\) Slides p.19

What Slides p.19

Lecture 4 · p.19 So \[\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim N(0,1)\] And by the same logic as before: \[\frac{\hat\mu_0-\mu_0}{\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim t_{n-2}\]

To standardize a normal quantity, subtract its mean and divide by its standard deviation.

The first line contains the unknown \(\sigma\). We cannot calculate it from data.

The second line uses \(\hat\sigma\) for \(\sigma\). This changes \(N(0,1)\) to \(t_{n-2}\).

\(\mathrm{SE}(\hat\mu_0)=\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\) estimates the standard deviation of \(\hat\mu_0\).

How Added

Calculate the standard error of \(\hat\mu_0\)
  1. Calculate \(n\), \(\bar x\) and \(S_{xx}\).
  2. Calculate \(\hat\beta_0\), \(\hat\beta_1\), the residuals \(e_i\) and \(\hat\sigma\).
  3. Calculate \((x_0-\bar x)^2\).
  4. Divide by \(S_{xx}\).
  5. Add \(1/n\).
  6. Take the square root.
  7. Multiply by \(\hat\sigma\).

Self-check: ① The unit of SE is the unit of \(y\) (g). ② The step 5 sum is much less than 1. SE must be much less than \(\hat\sigma\).

Example 1 · Standard error at \(x_0=4000\) Added

Steps 1–2 come from the start point.

1

Distance from \(\bar x\) How step 3

\[x_0-\bar x=4000-3637.864407=362.1355932\]
\[(x_0-\bar x)^2=362.1355932^2=131142.1879\]
2

Divide by \(S_{xx}\) How step 4

\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{131142.1879}{30647233.66}=0.004279087285\]
3

Add \(1/n\) How step 5

\[\frac1n=\frac1{236}=0.004237288136\]
\[0.004237288136+0.004279087285=0.008516375421\]
4

Square root How step 6

\[\sqrt{0.008516375421}=0.09228421003\]
5

Multiply by \(\hat\sigma\) How step 7

\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.09228421003=6.676823721\]

6.676823721 g is much less than \(\hat\sigma=72.35066237\) g. ▲

Why Added

Same logic as Lecture 3 p.31 for \(\hat\beta_1\). Let \(c_0=\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\).

1
\[\hat\mu_0\sim N\!\left(\mu_0,\ \sigma^2c_0\right)\]

Slides p.18.

2
\[\hat\mu_0-\mu_0\sim N\!\left(0,\ \sigma^2c_0\right)\]
3
\[Z=\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{c_0}}\sim N(0,1)\]

First line of Slides p.19.

4
\[V=\frac1{\sigma^2}\sum_{i=1}^n e_i^2\sim\chi^2_{(n-2)}\]

Lecture 3 p.31: \(Z\), \(V\) independent.

5
\[\frac{Z}{\sqrt{V/(n-2)}}\sim t_{n-2}\]

Lecture 3 p.31.

6
\[\sqrt{\frac{V}{n-2}}=\sqrt{\frac{\sum_{i=1}^n e_i^2}{\sigma^2(n-2)}}=\frac{\hat\sigma}{\sigma}\]
7
\[\frac{Z}{\sqrt{V/(n-2)}}=\frac{\hat\mu_0-\mu_0}{\sigma\sqrt{c_0}}\cdot\frac{\sigma}{\hat\sigma}\]
8
\[\frac{\hat\mu_0-\mu_0}{\hat\sigma\sqrt{c_0}}\sim t_{n-2}\]

Second line of Slides p.19. ∎

Two random quantities are independent when the value of one gives no information about the other.

03.2The \(100(1-\alpha)\%\) confidence interval for \(\mu_0\) Slides p.20

What Slides p.20

Lecture 4 · p.20 \[\Longrightarrow\ 1-\alpha=P\!\left(-t_{1-\frac\alpha2,\,n-2}\le\frac{\hat\mu_0-\mu_0}{\hat\sigma\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)^{1/2}}\le t_{1-\frac\alpha2,\,n-2}\right)\] and a 100(1-α)% CI is: \[\hat\mu_0\pm t_{1-\frac\alpha2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

With repeated samples, \(100(1-\alpha)\%\) of these intervals contain the true \(\mu_0\).

The interval is "center ± quantile × standard error", with center \(\hat\mu_0\).

The half-width is \(t_{1-\alpha/2,\,n-2}\times\mathrm{SE}(\hat\mu_0)\).

How Added

Calculate the CI for \(\mu_0\) at a value \(x_0\)
  1. Calculate \(\mathrm{SE}(\hat\mu_0)\) (Section 03.1).
  2. Calculate the center \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\).
  3. Find the quantile \(t_{1-\alpha/2,\,n-2}\).
  4. Multiply the quantile by \(\mathrm{SE}(\hat\mu_0)\) to get the half-width.
  5. Lower limit \(L\): \(\hat\mu_0\) minus the half-width.
  6. Upper limit \(U\): \(\hat\mu_0\) plus the half-width.

Self-check: ① \((L+U)/2=\hat\mu_0\). ② At 95% with \(n=236\), the quantile is 1.970, near 1.96.

Example 2 · 95% CI for the mean brain weight at \(x_0=4000\) Added

Example 1 gives \(\mathrm{SE}(\hat\mu_0)=6.676823721\) (How step 1).

1

Center How step 2

\[\hat\mu_0=\hat\beta_0+\hat\beta_1x_0=335.432315+0.2608207153\times4000\]
\[=335.432315+1043.282861\]
\[=1378.715176\]
2

Quantile How step 3

\[\alpha=0.05,\quad 1-\frac\alpha2=0.975,\quad n-2=236-2=234\]
\[t_{0.975,\,234}=1.970153643\]
3

Half-width How step 4

\[t_{0.975,\,234}\times\mathrm{SE}(\hat\mu_0)=1.970153643\times6.676823721=13.15436858\]

Basis: Example 1.

4

Lower limit How step 5

\[L=1378.715176-13.15436858=1365.560808\]
5

Upper limit How step 6

\[U=1378.715176+13.15436858=1391.869545\]

Self-check: \((1365.560808+1391.869545)/2=1378.715176=\hat\mu_0\).

At head size 4000 cm³, the 95% CI for the mean brain weight is (1365.561, 1391.870) g. ▲

Why Added

Isolate \(\mu_0\). Let \(t=t_{1-\alpha/2,\,n-2}\) and \(\mathrm{SE}=\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\).

1
\[1-\alpha=P\!\left(-t\le\frac{\hat\mu_0-\mu_0}{\mathrm{SE}}\le t\right)\]

Section 03.1; symmetric, each tail \(\alpha/2\).

2
\[1-\alpha=P\!\left(-t\,\mathrm{SE}\le\hat\mu_0-\mu_0\le t\,\mathrm{SE}\right)\]

\(\mathrm{SE}>0\).

3
\[1-\alpha=P\!\left(-\hat\mu_0-t\,\mathrm{SE}\le-\mu_0\le-\hat\mu_0+t\,\mathrm{SE}\right)\]
4
\[1-\alpha=P\!\left(\hat\mu_0-t\,\mathrm{SE}\le\mu_0\le\hat\mu_0+t\,\mathrm{SE}\right)\]

Multiply by \(-1\); the signs change direction.

5
\[\text{CI}=\hat\mu_0\pm t_{1-\frac\alpha2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

∎

03.3Confidence intervals at a range of \(x_0\) values Slides p.21–22

What Slides p.21–22

Lecture 4 · p.22 Estimating mean outcomes and report corresponding CIs at a range of x0 values

Slides p.21 shows only code. It does Section 03.2 at many \(x_0\).

A grid is a list of values with equal spaces. The \(x_0\) grid goes from the smallest to the largest \(x\), step 1.

All intervals are 95% and use \(t_{0.975,\,234}\).

Slides p.22 plots the data, the fitted line, and \(L\) and \(U\) at each grid value.

A confidence band is the pair of curves through all \(L\) and all \(U\).

How Added

Draw the confidence band for the mean response
  1. Find the smallest and the largest \(x\).
  2. Make the grid of \(x_0\) values between them, step 1.
  3. At each grid value, calculate \(L\) and \(U\) (Section 03.2).
  4. Plot the data and the fitted line \(\hat\beta_0+\hat\beta_1x\).
  5. Join all lower limits with a curve.
  6. Join all upper limits with a second curve.

Self-check: ① Grid size = (largest \(x\) − smallest \(x\)) + 1. ② At \(x_0=4000\), the limits are (1365.561, 1391.870), as in Example 2.

Example 3 · The first grid value Added

Brainhead data (Added): smallest \(x\) is 2773, largest \(x\) is 4747.

1

Number of grid values How step 2

\[(4747-2773)+1=1974+1=1975\]
2

Distance from \(\bar x\) 03.1 How steps 3–4

\[x_0-\bar x=2773-3637.864407=-864.8644068\]
\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{747990.4421}{30647233.66}=0.02440645868\]
3

Standard error 03.1 How steps 5–7

\[0.004237288136+0.02440645868=0.02864374682\]
\[\sqrt{0.02864374682}=0.169244636\]
\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.169244636=12.24496152\]
4

Center 03.2 How step 2

\[\hat\mu_0=335.432315+0.2608207153\times2773\]
\[=335.432315+723.2558436=1058.688159\]
5

Half-width 03.2 How step 4

\[1.970153643\times12.24496152=24.12445554\]
6

Limits 03.2 How steps 5–6

\[L=1058.688159-24.12445554=1034.563703\]
\[U=1058.688159+24.12445554=1082.812614\]

First band points: (2773, 1034.564) and (2773, 1082.813). The other 1974 grid values give the full band in Figure 3-1. ▲

3000350040004500 Head size (cm³) 1000120014001600 Brain weight (g) x̄ = 3637.86 fitted line 95% CI for µ0 (L, U)
Figure 3-1 · Slides p.22, drawn again from the data. Grey: 236 observations. Blue: fitted line \(\hat\mu_0=\hat\beta_0+\hat\beta_1x_0\). Orange: 95% limits \(L\), \(U\). Purple dashed: \(\bar x\), where the curves are nearest.

03.4Why the intervals become wider toward the edges Slides p.23

What Slides p.23

Lecture 4 · p.23 Why do the CIs get wider toward the edges?

Half-width: \(t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\).

Only \((x_0-\bar x)^2\) changes with \(x_0\). All other parts are fixed for one data set.

This term increases as \(x_0\) moves from \(\bar x\). It is 0 at \(\bar x\), where the interval is narrowest.

How Added

Compare the interval widths at different values of \(x_0\)
  1. For each \(x_0\), calculate \(|x_0-\bar x|\).
  2. Calculate the half-width (Section 03.2).
  3. Multiply it by 2 to get the width.
  4. Sort the \(x_0\) values by \(|x_0-\bar x|\).
  5. Make sure that the widths increase in the same order.

Self-check: ① At \(x_0=\bar x\), \(\mathrm{SE}(\hat\mu_0)=\hat\sigma/\sqrt n\). ② The ratio of two widths depends only on the two sums under the square root.

Example 4 · Widths at four values of \(x_0\) Added

All use \(t_{0.975,\,234}=1.970153643\), \(\hat\sigma=72.35066237\), \(1/n=0.004237288136\), \(S_{xx}=30647233.66\).

\(x_0=\bar x=3637.864407\) · \(|x_0-\bar x|=0\)

\(\frac{(x_0-\bar x)^2}{S_{xx}}=0\). Sum \(=0.004237288136\). Square root \(=0.06509445549\).

\(\mathrm{SE}(\hat\mu_0)=72.35066237\times0.06509445549=4.709626971\). Half-width \(=1.970153643\times4.709626971=9.278688732\).

\(\hat\mu_0=335.432315+948.8303969=1284.262712\). CI \(=(1274.984,\ 1293.541)\). Width \(=2\times9.278688732=18.557\).

Self-check: \(\hat\sigma/\sqrt n=72.35066237\times0.06509445549=4.709626971=\mathrm{SE}(\hat\mu_0)\).

\(x_0=4000\) · \(|x_0-\bar x|=362.1355932\)

From Examples 1–2: \(\mathrm{SE}(\hat\mu_0)=6.676823721\). Half-width \(=13.15436858\).

CI \(=(1365.561,\ 1391.870)\). Width \(=2\times13.15436858=26.309\).

\(x_0=2773\) (smallest \(x\)) · \(|x_0-\bar x|=864.8644068\)

From Example 3: \(\mathrm{SE}(\hat\mu_0)=12.24496152\). Half-width \(=24.12445554\).

CI \(=(1034.564,\ 1082.813)\). Width \(=2\times24.12445554=48.249\).

\(x_0=4747\) (largest \(x\)) · \(|x_0-\bar x|=1109.135593\)

\(\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{1230181.764}{30647233.66}=0.0401400589\). Sum \(=0.04437734703\). Square root \(=0.2106593151\).

\(\mathrm{SE}(\hat\mu_0)=72.35066237\times0.2106593151=15.24134098\). Half-width \(=1.970153643\times15.24134098=30.02778345\).

\(\hat\mu_0=335.432315+1238.115936=1573.548251\). CI \(=(1543.520,\ 1603.576)\). Width \(=2\times30.02778345=60.056\).

Widths in order of \(|x_0-\bar x|\): 18.557, 26.309, 48.249, 60.056. The width at the largest \(x\) is \(\sqrt{0.04437734703/0.004237288136}=3.236209805\) times the width at \(\bar x\). ▲

0102030 3000350040004500x0 (cm³) half-width (g) 2773: 24.12 x̄: 9.28 4000: 13.15 4747: 30.03
Figure 3-2 · Half-width \(t_{0.975,\,234}\times\mathrm{SE}(\hat\mu_0)\) at each \(x_0\) (Added). Lowest at \(\bar x\) (purple dashed). Blue dots: the four \(x_0\) of Example 4.

Why Added

The structure of the Slides p.17 result:

\[\mathrm{Var}[\hat\mu_0]=\mathrm{Var}[\bar y+\hat\beta_1(x_0-\bar x)]=\underbrace{\frac{\sigma^2}{n}}_{\text{from }\bar y}+\underbrace{\frac{\sigma^2(x_0-\bar x)^2}{S_{xx}}}_{\text{from }\hat\beta_1}\]

The first equal sign uses Slides p.12. The second is Slides p.17, where the cross term is 0 because \(\sum_{i=1}^n(x_i-\bar x)=0\). ∎

The error in \(\bar y\) is the same at each \(x_0\). It gives \(\frac1n\). The distance \(x_0-\bar x\) multiplies the slope error. It gives \(\frac{(x_0-\bar x)^2}{S_{xx}}\).

03.5Why most points are outside the lines Slides p.24

What Slides p.24

Lecture 4 · p.24 Why do most of the points fall outside the lines?

The interval targets the mean \(\mu_0\), not one observation.

Each observation is \(y_i=\mu_i+\epsilon_i\), with \(\mu_i=\beta_0+\beta_1x_i\).

\(\epsilon_i\) has standard deviation \(\sigma\), estimated by \(\hat\sigma=72.35066237\) g.

The largest half-width of the band is 30.02778345 g, at \(x_0=4747\) (Example 4).

Most observations are farther from the line than this half-width.

How Added

Find if an observation is inside the band
  1. Use \(x_i\) as \(x_0\).
  2. Calculate \(L_i\) and \(U_i\) (Section 03.2).
  3. Compare \(y_i\) with \(L_i\) and \(U_i\).
  4. If \(L_i\le y_i\le U_i\), the point is inside. If not, it is outside.
  5. Do steps 1–4 for all \(n\) observations. Count the two groups.

Self-check: inside + outside \(=n=236\).

Example 5 · Two persons of the brainhead data Added

Person 1: \(x_1=4512\), \(y_1=1530\). Person 2: \(x_2=3738\), \(y_2=1297\).

1

Person 1: standard error How steps 1–2

\[x_1-\bar x=4512-3637.864407=874.1355932\]
\[\frac{(x_1-\bar x)^2}{S_{xx}}=\frac{764113.0353}{30647233.66}=0.02493252878\]
\[\sqrt{0.004237288136+0.02493252878}=\sqrt{0.02916981692}=0.1707917355\]
\[\mathrm{SE}=72.35066237\times0.1707917355=12.35689519\]
2

Person 1: limits How step 2

\[\hat\mu_0=335.432315+0.2608207153\times4512=335.432315+1176.823068=1512.255383\]
\[\text{half-width}=1.970153643\times12.35689519=24.34498207\]
\[L_1=1487.910401,\qquad U_1=1536.600365\]
3

Person 1: compare How steps 3–4

\[1487.910401\le1530\le1536.600365\]

Inside.

4

Person 2: standard error How steps 1–2

\[x_2-\bar x=3738-3637.864407=100.1355932\]
\[\frac{(x_2-\bar x)^2}{S_{xx}}=\frac{10027.13703}{30647233.66}=0.0003271791882\]
\[\sqrt{0.004237288136+0.0003271791882}=\sqrt{0.004564467324}=0.06756084165\]
\[\mathrm{SE}=72.35066237\times0.06756084165=4.888071643\]
5

Person 2: limits How step 2

\[\hat\mu_0=335.432315+0.2608207153\times3738=335.432315+974.947834=1310.380149\]
\[\text{half-width}=1.970153643\times4.888071643=9.630252153\]
\[L_2=1300.749897,\qquad U_2=1320.010401\]
6

Person 2: compare How steps 3–4

\[1297<1300.749897=L_2\]

Outside: residual \(1297-1310.380149=-13.38014892\) g exceeds half-width 9.630252153 g.

All 236 persons: 41 inside, 195 outside (data file, Added). \(195/236=0.8262711864\), or 82.6%. Self-check: \(41+195=236\). ▲

3000350040004500 Head size (cm³) 1000120014001600 Brain weight (g) inside band: 41 outside band: 195 95% CI for µ0
Figure 3-3 · Data and 95% band (blue) of Figure 3-1 (Added). Green: inside. Orange: outside. The vertical spread of the points is much larger than the band.

Why Added

Compare the variance of one observation with the variance of \(\hat\mu_0\).

1
\[y_i=\mu_i+\epsilon_i,\qquad \mathrm{Var}[y_i]=\sigma^2\]

Slides p.3.

2
\[\mathrm{Var}[\hat\mu_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\right)\]

Slides p.17.

3
\[\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}\le0.04437734703\quad\text{for }2773\le x_0\le4747\]

Example 4: largest at \(x_0=4747\).

4
\[\mathrm{Var}[\hat\mu_0]\le0.04437734703\,\sigma^2\]

Less than 5% of \(\mathrm{Var}[y_i]\). ∎

The interval shows only the error of \(\hat\mu_0\). As \(n\) increases, the interval becomes narrower. The spread \(\sigma\) of each point does not decrease with \(n\).

03.6When we want to predict a new observation Slides p.25

Lecture 4 · p.25 What if we don't just care about the mean, but predictions? I.e. even if we got the mean absolutely perfect—the new points wouldn't fall directly on this line!

With exact \(\beta_0\), \(\beta_1\), \(\hat\mu_0=\mu_0\) has no estimation error.

A new observation is still \(\mu_0+\epsilon\), with \(\epsilon\sim N(0,\sigma^2)\) and \(\hat\sigma=72.35066237\) g.

An interval for a new observation must include the error of the mean estimate and the new error \(\epsilon\).

A prediction interval is an interval made to contain a new observation (Unit 04).

Summary of this unit

CI for \(\mu_0\): \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\). Narrowest at \(\bar x\). It contains the mean, not single observations.

03.7Practice Added

Given: \(n=236\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\).

Q1. Calculate a 90% CI for the mean brain weight at \(x_0=4000\) cm³. Use \(t_{0.95,\,234}=1.651391475\). Give the half-width and the interval.

Answer
1
\[\alpha=0.10,\qquad 1-\frac\alpha2=0.95,\qquad t_{0.95,\,234}=1.651391475\]
2
\[\hat\mu_0=1378.715176,\qquad \mathrm{SE}(\hat\mu_0)=6.676823721\]

Examples 1–2.

3
\[\text{half-width}=1.651391475\times6.676823721=11.02604977\]
4
\[1378.715176\pm11.02604977=(1367.689,\ 1389.741)\]

It is narrower than the 95% interval (1365.561, 1391.870), because 1.651391475 < 1.970153643.

Q2. Calculate the 95% CI for the mean response at \(x_0=3500\). First predict: wider or narrower than at \(x_0=4000\)? Then compare with your numbers.

Answer

Prediction: \(|3500-\bar x|=137.8644068\) < \(|4000-\bar x|=362.1355932\). The interval is narrower.

1
\[\hat\mu_0=335.432315+0.2608207153\times3500=335.432315+912.8725037=1248.304819\]
2
\[\frac{(x_0-\bar x)^2}{S_{xx}}=\frac{19006.59466}{30647233.66}=0.000620173255\]
3
\[\sqrt{0.004237288136+0.000620173255}=\sqrt{0.004857461391}=0.06969549046\]
4
\[\mathrm{SE}(\hat\mu_0)=72.35066237\times0.06969549046=5.042514899\]
5
\[\text{half-width}=1.970153643\times5.042514899=9.934529096\]
6
\[1248.304819\pm9.934529096=(1238.370,\ 1258.239)\]

Width \(2\times9.934529096=19.869\) < 26.309 at \(x_0=4000\). This agrees with the prediction.

Q3. At which \(x_0\) is the CI for \(\mu_0\) narrowest? How many times wider is the 95% interval at \(x_0=4747\)? Why does this ratio not depend on \(\hat\sigma\) or \(\alpha\)?

Answer

Narrowest at \(x_0=\bar x=3637.864407\), where \((x_0-\bar x)^2=0\).

1
\[\frac{\text{width at }4747}{\text{width at }\bar x}=\frac{2t\hat\sigma\sqrt{1/n+(4747-\bar x)^2/S_{xx}}}{2t\hat\sigma\sqrt{1/n}}\]
2
\[=\sqrt{\frac{0.04437734703}{0.004237288136}}\]
3
\[=3.236209805\]

Example 4: \(60.0555669/18.55737746=3.236209805\).

\(2t_{1-\alpha/2,\,n-2}\hat\sigma\) cancels. The ratio depends only on \(x_0\), \(n\) and \(S_{xx}\).

Unit 04Predicting a new response: the prediction interval

Start point · Added

Brainhead data, \(n=236\). Slides p.7: \(\hat\beta_0=335.43231\), \(\hat\beta_1=0.26082\), \(\hat\sigma=72.35\), \(n-2=234\).

From brainhead.csv (Added): \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\).

All examples use \(x_{new}=4000\), as in Units 02–03.

04.1The new response and the predicted value Slides p.26–30

What Slides p.27–30

Lecture 4 · p.27–30 Instead of the mean, we might want to predict the response itself for an observation with covariate \(x_{new}\) (as always treating \(x_{new}\) as fixed): \[y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\] Define the predicted value \[\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\] • How does this differ from how we estimate the mean?
• But we'll need to take into account additional uncertainty (why?)

\(x_{new}\) is the fixed covariate value of the new observation.

The new response \(y_{new}\) is the unobserved response of the new observation. It is random because of its error \(\epsilon_{new}\).

The new error \(\epsilon_{new}\sim N(0,\sigma^2)\) is independent of \(\epsilon_1,\dots,\epsilon_n\).

The predicted value \(\hat y_{new}\) is the fitted line at \(x_{new}\), our guess for \(y_{new}\).

Slide question 1: how does this differ from the estimate of the mean? With \(x_0=x_{new}\), the formulas are the same: \(\hat y_{new}=\hat\mu_0\). The target is different:

\(\mu_0=\beta_0+\beta_1x_0\) is a fixed parameter, with no error term.

\(y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\) has the random error \(\epsilon_{new}\).

Slide question 2: why is there additional uncertainty? Uncertainty is the possible difference between a guess and the true value. A new response has two sources:

Source 1: \(\hat\beta_0+\hat\beta_1x_{new}\) changes from sample to sample, as for the mean.

Source 2: \(\epsilon_{new}\). We cannot guess it, even with the true \(\beta_0\) and \(\beta_1\).

How Added

Calculate the predicted value \(\hat y_{new}\)
  1. Get \(\hat\beta_1=S_{xy}/S_{xx}\) and \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).
  2. Multiply \(\hat\beta_1\) by \(x_{new}\).
  3. Add \(\hat\beta_0\).

Self-check: \(\hat y_{new}=\hat\mu_0\) at the same \(x\).

Example 1 · The brainhead data at \(x_{new}=4000\) Added How steps 2–3

Predict the brain weight of a new person with head size 4000 cm³.

1
\[\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\]
2
\[\hat y_{new}=335.432315+0.2608207153\times4000\]
3
\[\hat y_{new}=335.432315+1043.282861\]
4
\[\hat y_{new}=1378.715176\ \text{g}\]
Slide p.7 values: \(335.43231+0.26082\times4000=1378.71231\).

Compare with unit 02: \(\hat\mu_0=1378.715\) is the same number. In this unit, it predicts one person. In unit 02, it estimates the mean of all persons with head size 4000. ▲

04.2Expectation and variance of the prediction error Slides p.31–35

What Slides p.31–35

Lecture 4 · p.31–35 Define the prediction error: \(\hat y_{new}-y_{new}\)
Properties: \[E[\hat y_{new}-y_{new}]=E[(\hat\beta_0+\hat\beta_1x_{new})-(\beta_0+\beta_1x_{new}+\epsilon_{new})]=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new})=0\] Now note that \(\hat y_{new}\) and \(y_{new}\) are independent (why?) \[\mathrm{Var}[\hat y_{new}-y_{new}]=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

The prediction error \(\hat y_{new}-y_{new}\) is the predicted value minus the true new response. It is random.

Expectation 0: on average, \(\hat y_{new}\) is not too high and not too low.

The term \(\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\) is source 1. It equals \(\mathrm{Var}[\hat\mu_0]\) of unit 02.

The term \(\sigma^2\) is source 2, from \(\epsilon_{new}\).

Independence: \(\hat y_{new}\) uses only \(y_1,\dots,y_n\). The random part of \(y_{new}\) is \(\epsilon_{new}\), independent of the sample.

How Added

Calculate the estimated variance of the prediction error
  1. Calculate \(\frac1n\).
  2. Calculate \((x_{new}-\bar x)^2/S_{xx}\).
  3. Multiply the sum of steps 1–2 by \(\hat\sigma^2\): the line part (source 1).
  4. The new-error part (source 2) is \(\hat\sigma^2\).
  5. Add the two parts: \(\hat\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\).

Self-check: Step 3 equals the estimate of \(\mathrm{Var}[\hat\mu_0]\) in unit 02. The total is larger than \(\hat\sigma^2\).

Example 2 · The brainhead data at \(x_{new}=4000\) Added How steps 1–5

Use \(\hat\sigma\) for \(\sigma\): \(\hat\sigma^2=72.35066237^2=5234.618345\).

1
\[\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}=0.004237288+0.004279087=0.008516375\]
How steps 1–2; unit 02, Example 3.
2
\[\text{line part}=\hat\sigma^2\times0.008516375=5234.618345\times0.008516375=44.579975\]
How step 3: estimate of \(\mathrm{Var}[\hat\mu_0]\).
3
\[\text{new-error part}=\hat\sigma^2=5234.618345\]
How step 4.
4
\[\hat\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)=44.579975+5234.618345=5279.198320\]
How step 5.
Line part σ̂²(1/n + (x_new − x̄)²/S_xx) = 44.580 (0.84%) New-error part σ̂² = 5234.618 (99.16%) 0 Total variance 5279.198
Figure 4-1 · Total estimated variance 5279.198 at \(x_{new}=4000\), parts to scale. Orange: fitted line (source 1). Blue: \(\epsilon_{new}\) (source 2), 99.16% of the total.

Result: \(\epsilon_{new}\) gives almost all of the variance. This term does not decrease as \(n\) increases. The prediction interval is much wider than the confidence interval. ▲

Why · The expectation and the variance of the prediction error Slides p.33–35

(a) Expectation. Use \(E[\hat\beta_0]=\beta_0\), \(E[\hat\beta_1]=\beta_1\), \(E[\epsilon_{new}]=0\).

1
\[E[\hat y_{new}-y_{new}]=E[(\hat\beta_0+\hat\beta_1x_{new})-(\beta_0+\beta_1x_{new}+\epsilon_{new})]\]
Definitions.
2
\[=E[\hat\beta_0]+E[\hat\beta_1]x_{new}-(\beta_0+\beta_1x_{new}+E[\epsilon_{new}])\]
Linearity of expectation.
3
\[=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new}+0)\]
Unbiased estimators; \(E[\epsilon_{new}]=0\).
4
\[=\beta_0+\beta_1x_{new}-(\beta_0+\beta_1x_{new})\]
5
\[=0\]

(b) Independence.

\(\hat\beta_0\) and \(\hat\beta_1\) use only \(y_1,\dots,y_n\), whose random part is \(\epsilon_1,\dots,\epsilon_n\).

The random part of \(y_{new}\) is \(\epsilon_{new}\), independent of \(\epsilon_1,\dots,\epsilon_n\).

\(\hat y_{new}\) and \(y_{new}\) are independent.

(c) Variance.

For independent \(A\) and \(B\): \(\mathrm{Var}[A-B]=\mathrm{Var}[A]+\mathrm{Var}[B]\).

Unit 02 (Lecture 4 · p.17–18): \(\mathrm{Var}[\hat\beta_0+\hat\beta_1x_0]=\sigma^2\left(\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\).

\(\mathrm{Var}[y_{new}]=\mathrm{Var}[\epsilon_{new}]=\sigma^2\).

1
\[\mathrm{Var}[\hat y_{new}-y_{new}]=\mathrm{Var}[(\hat\beta_0+\hat\beta_1x_{new})-y_{new}]\]
2
\[=\mathrm{Var}[(\hat\beta_0+\hat\beta_1x_{new})]+\mathrm{Var}[y_{new}]\]
Independence (b). Both terms are added.
3
\[=\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)+\mathrm{Var}[y_{new}]\]
Unit 02 with \(x_0=x_{new}\).
4
\[=\sigma^2\left(\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)+\sigma^2\]
\(\mathrm{Var}[y_{new}]=\sigma^2\).
5
\[=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right)\]

\(\epsilon_{new}\) adds the "1", compared with \(\mathrm{Var}[\hat\mu_0]\). ∎

04.3The prediction interval Slides p.36–37

What Slides p.36–37

Lecture 4 · p.36–37 We know \(\hat y_{new}-y_{new}\sim N\left(0,\ \sigma^2\left[1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\right]\right)\) \[\Longrightarrow\ \frac{\hat y_{new}-y_{new}}{\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim N(0,1)\] Same logic as before: \[\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\sim t_{n-2}\] \[\Longrightarrow\ 1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\le\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}}\le t_{1-\frac{\alpha}{2},n-2}\right)\] So a 100(1−α)% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

A prediction interval (PI) is an interval, calculated from the data, that contains the random \(y_{new}\) with probability \(1-\alpha\).

The confidence interval (CI) of unit 03 contains the fixed \(\mu_0\) with probability \(1-\alpha\).

The only difference from the CI is the "1" under the square root: the \(\sigma^2\) from \(\epsilon_{new}\).

How Added

Calculate a 100(1−α)% prediction interval
  1. Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\) (Section 04.1).
  2. Calculate \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\).
  3. Take the square root and multiply by \(\hat\sigma\): the standard error of prediction.
  4. Find the quantile \(t_{1-\alpha/2,\,n-2}\).
  5. Multiply step 4 by step 3: the half-width.
  6. Subtract the half-width from \(\hat y_{new}\) and add it to \(\hat y_{new}\).

Self-check: ① The midpoint is \(\hat y_{new}\). ② The standard error of prediction is larger than \(\hat\sigma\). ③ The PI is wider than the CI at the same point.

Example 3 · The brainhead data at \(x_{new}=4000\), 95% prediction interval Added How steps 1–6

Find the 95% PI for the brain weight of a new person with head size 4000 cm³. Compare it with the 95% CI for the mean. \(\alpha=0.05\), \(n-2=234\).

1
Predicted value How step 1
\[\hat y_{new}=335.432315+0.2608207153\times4000=1378.715176\]

Basis: Example 1.

2
Value under the square root How step 2
\[1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}=1+0.008516375=1.008516375\]

Basis: Example 2, step 1.

3
Standard error of prediction How step 3
\[\hat\sigma\sqrt{1.008516375}=72.35066237\times1.004249160=72.658092\]
4
t quantile How step 4
\[t_{1-\frac{0.05}{2},\,234}=t_{0.975,\,234}=1.970154\]
5
Half-width How step 5
\[1.970154\times72.658092=143.1476\]
6
Interval How step 6
\[1378.715\pm143.148=(1378.715-143.148,\ \ 1378.715+143.148)=(1235.568,\ \ 1521.863)\]

95% CI for \(\mu_0\) (unit 03, Example 2): \((1365.561,\ 1391.870)\), width \(2\times13.154=26.31\) g.

95% PI for \(y_{new}\): \((1235.568,\ 1521.863)\), width \(2\times143.148=286.30\) g.

Same midpoint, 1378.715. The PI is \(143.1476/13.15437=10.88\) times as wide.

Slides p.39 shows both bands over a range of \(x\) (Unit 05). ▲

Why · From the distribution to the prediction interval Slides p.36–37

Let \(k=1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}\).

Normality: \(\hat y_{new}\) is a linear combination of the normal \(y_1,\dots,y_n\).

\(y_{new}\) is normal and independent of the sample. A linear combination of independent normals is normal.

1
\[\hat y_{new}-y_{new}\sim N\left(0,\ \sigma^2k\right)\]
Section 04.2.
2
\[\frac{\hat y_{new}-y_{new}-0}{\sqrt{\sigma^2k}}\sim N(0,1)\]
Standardize.
3
\[\frac{\hat y_{new}-y_{new}}{\sigma\sqrt{k}}\sim N(0,1)\]
4
\[\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{k}}\sim t_{n-2}\]
\(\hat\sigma\) for \(\sigma\), as for \(\hat\beta_1\), \(\hat\mu_0\).
5
\[1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\le\frac{\hat y_{new}-y_{new}}{\hat\sigma\sqrt{k}}\le t_{1-\frac{\alpha}{2},n-2}\right)\]
Symmetric; each tail \(\alpha/2\).
6
\[1-\alpha=P\left(-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le\hat y_{new}-y_{new}\le t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]
7
\[1-\alpha=P\left(-\hat y_{new}-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le-y_{new}\le-\hat y_{new}+t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]
8
\[1-\alpha=P\left(\hat y_{new}-t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\le y_{new}\le\hat y_{new}+t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{k}\right)\]
Multiply by \(-1\); the signs change direction.
9
\[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]
Put in \(k\).

Line 8: this random interval contains the random \(y_{new}\) with probability \(1-\alpha\). ∎

04.4Practice Added

Given: \(n=236\), \(\bar x=3637.864407\), \(S_{xx}=30647233.66\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207153\), \(\hat\sigma=72.35066237\).

Q1. A new person has head size 4000 cm³. Calculate a 90% PI for this person's brain weight. Use \(t_{0.95,\,234}=1.651391\).

Answer

Step 1: \(\hat y_{new}=1378.715176\) (Example 1).

Steps 2–3: \(\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}=72.658092\) (Example 3).

Step 4: \(\alpha=0.10\), \(t_{1-0.10/2,\,234}=t_{0.95,\,234}=1.651391\).

Step 5: half-width \(=1.651391\times72.658092=119.9870\).

Step 6: \(1378.715\pm119.987=(1258.728,\ 1498.702)\).

It is narrower than the 95% PI \((1235.568,\ 1521.863)\), because its quantile is smaller.

Q2. Calculate a 95% PI for the brain weight of a new person with head size 4500 cm³. Use \(t_{0.975,\,234}=1.970154\).

Answer

Step 1: \(\hat y_{new}=335.432315+0.2608207153\times4500=335.432315+1173.693219=1509.125534\).

Step 2: \(x_{new}-\bar x=4500-3637.864407=862.135593\). \(\frac{862.135593^2}{30647233.66}=\frac{743277.7807}{30647233.66}=0.024252688\).

\(1+\frac1{236}+0.024252688=1+0.004237288+0.024252688=1.028489976\).

Step 3: \(\hat\sigma\sqrt{1.028489976}=72.35066237\times1.014144948=73.374059\).

Step 4: \(t_{0.975,\,234}=1.970154\).

Step 5: half-width \(=1.970154\times73.374059=144.5582\).

Step 6: \(1509.126\pm144.558=(1364.567,\ 1653.684)\).

4500 is farther from \(\bar x\) than 4000. The half-width 144.558 is a little larger than 143.148.

Q3. (a) At which \(x_{new}\) is the PI narrowest? (b) Calculate its 95% half-width. (c) Why does this half-width not decrease to 0 as \(n\to\infty\)?

Answer

(a) Only \(\frac{(x_{new}-\bar x)^2}{S_{xx}}\) changes with \(x_{new}\). It is 0 at \(x_{new}=\bar x=3637.864407\), where the PI is narrowest.

(b) \(\hat\sigma\sqrt{1+\frac1{236}+0}=72.35066237\times\sqrt{1.004237288}=72.35066237\times1.002116404=72.503786\).

Half-width \(=1.970154\times72.503786=142.8436\).

\(\hat y_{new}=335.432315+0.2608207153\times3637.864407=1284.262712\). PI: \(1284.263\pm142.844=(1141.419,\ 1427.106)\).

(c) As \(n\to\infty\), \(\frac1n\to0\), and the third term is 0 at \(\bar x\). The "1" stays.

The "1" is \(\mathrm{Var}[\epsilon_{new}]=\sigma^2\). More data improve the fitted line but do not remove \(\epsilon_{new}\).

The half-width stays near \(t_{1-\alpha/2,\,n-2}\,\hat\sigma\) or larger.

Unit 05Prediction intervals over a range of \(x\) values

Start point · Added

Brainhead data, \(n=236\). Values from brainhead.csv, as in units 01–04:

\(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207\).
\(\hat\sigma=72.35066\), with \(n-2=234\) degrees of freedom.
\(\bar x=3637.8644\), \(S_{xx}=\sum_{i=1}^n(x_i-\bar x)^2=30647233.66\).
\(t_{0.975,\,234}=1.970154\).
\(\min x_i=2773\), \(\max x_i=4747\).

05.1Calculate the prediction interval at many values of \(x_{new}\) Slides p.38

What Slides p.37–38

Prediction Interval · Lecture 4 · p.37 So a 100(1-α)% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},\,n-2}\,\hat\sigma\sqrt{1+\frac{1}{n}+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

Slides p.38 shows this as code. For each \(x_{new}\), the code does four things:

It finds \(t_{1-\alpha/2,\,n-2}\). The default level 0.95 gives \(t_{0.975,\,n-2}\).
It calculates the plug-in estimate \(\hat y_{new}\). A plug-in estimate puts \(\hat\beta_0,\hat\beta_1\) in place of the unknown \(\beta_0,\beta_1\).
It calculates the standard error \(\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}\).
It gives L = estimate − quantile × standard error and U = estimate + quantile × standard error.

The grid goes from 2773 to 4747, step 1: \(4747-2773+1=1975\) values. The p.39 plot joins all L and all U.

Only the PI has the term 1 under the square root.

How Added

Calculate a prediction interval at one value of \(x_{new}\)
  1. Find the quantile \(t_{1-\alpha/2,\,n-2}\).
  2. Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\).
  3. Calculate the distance term \(k=(x_{new}-\bar x)^2/S_{xx}\).
  4. Calculate the standard error \(\hat\sigma\sqrt{1+1/n+k}\).
  5. Multiply the quantile by the standard error to get the half-width.
  6. Subtract the half-width from \(\hat y_{new}\) to get L. Add it to get U.

For the band, do steps 2–6 at each grid value. The quantile does not change.

Self-check: ① \((L+U)/2=\hat y_{new}\). ② \(1+1/n+k\) is a little larger than 1. ③ At \(x_{new}=4000\), the PI is (1235.568, 1521.863), as in unit 04.

Example 1 · 95% PI at the smallest head size, \(x_{new}=2773\) Added

1

Quantile How step 1

\[t_{1-\alpha/2,\,n-2}=t_{1-0.05/2,\,236-2}\]
\[=t_{0.975,\,234}=1.970154\]
2

Predicted value How step 2

\[\hat y_{new}=335.432315+0.2608207\times2773\]
\[=335.432315+723.255844\]
\[=1058.688159\]
3

Distance term How step 3

\[k=\frac{(2773-3637.8644)^2}{30647233.66}\]
\[=\frac{(-864.8644)^2}{30647233.66}\]
\[=\frac{747990.44}{30647233.66}\]
\[=0.024406\]
4

Standard error How step 4

\[\hat\sigma\sqrt{1+\tfrac1n+k}=72.35066\sqrt{1+0.004237+0.024406}\]
\[=72.35066\sqrt{1.028644}\]
\[=72.35066\times1.014221\]
\[=73.37954\]

\(1/n=1/236=0.004237\).

5

Half-width How step 5

\[1.970154\times73.37954=144.56898\]
6

Limits How step 6

\[L=1058.68816-144.56898=914.11918\]
\[U=1058.68816+144.56898=1203.25714\]

95% PIs at four grid points (data file):

\(x_{new}=2773\) (smallest): (914.119, 1203.257), width 289.138.
\(x_{new}=\bar x=3637.8644\): (1141.419, 1427.106), width 285.687.
\(x_{new}=4000\): (1235.568, 1521.863), width 286.295, as in unit 04.
\(x_{new}=4747\) (largest): (1427.878, 1719.219), width 291.341.

Self-check ①: \((914.119+1203.257)/2=1058.688=\hat y_{new}\). ▲

Why Added

Each How step gives one part of the p.37 formula: step 1 the quantile, step 2 \(\hat y_{new}\), steps 3–4 the standard error, steps 5–6 the limits.

Unit 04 (Slides p.35): \(\mathrm{Var}[\hat y_{new}-y_{new}]=\sigma^2\left(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\right)\). The 1 is \(\mathrm{Var}[y_{new}]=\sigma^2\). ∎

05.2Read the plot: the PI is much wider than the CI Slides p.39

What Slides p.39

The CI for \(\mu_0\) (Lecture 4 p.20) is \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\), without the 1.

Prediction Intervals · Lecture 4 · p.39 Computing prediction intervals at a range of \(x_{new}\) values:
• Note the prediction intervals are much wider than the CIs before!
• Note also the lines are not parallel (slightly wider at the edges)

Note 1 compares PI and CI widths. Note 2: the PI is a little wider near the smallest and largest \(x\).

How Added

Compare the widths of the PI and the CI
  1. Select \(\bar x\), \(\min x_i\), \(\max x_i\), and one more \(x\).
  2. Calculate the PI width U − L at each \(x\).
  3. Calculate the CI width at the same \(x\).
  4. Divide the PI width by the CI width.
  5. Compare the PI width at \(\bar x\) with the PI width at the edges.

A ratio much larger than 1 supports note 1. Edge widths a little larger than the center width support note 2.

Self-check: ① Both widths are smallest at \(\bar x\), where \((x-\bar x)^2/S_{xx}=0\). ② Both intervals have the midpoint \(\hat\beta_0+\hat\beta_1x\).

Example 2 · Widths in the brainhead data Added

x̄ = 3637.9 3000350040004500 Head size (cm³) 1000120014001600 Brain weight (g) Fitted line 95% CI (mean response) 95% PI (new response) Points outside the PI (10)
Figure 5-1. Brainhead data (\(n=236\)). Blue: 95% CI band. Orange: 95% PI band. Black points are in the PI; purple points are outside. Dashed: \(\bar x\). The CI band widens much at the edges; the PI lines widen only a little.

PI widths from Section 05.1. CI widths use the same steps without the 1.

\(x=2773\): PI width 289.138, CI width 48.249, ratio 5.99.
\(x=\bar x\): PI width 285.687, CI width 18.557, ratio 15.39.
\(x=4000\): PI width 286.295, CI width 26.309, ratio 10.88.
\(x=4747\): PI width 291.341, CI width 60.056, ratio 4.85.

Note 1: the PI is 4.85 to 15.39 times as wide as the CI, most at \(\bar x\).

Note 2: the PI width is 285.687 at \(\bar x\), and 289.138 and 291.341 at the edges: only 3.451 and 5.654 g more.

The CI width grows from 18.557 to 60.056, more than 3 times.

Count (Added, data file): 226 of 236 points are in the 95% PI at their own \(x_i\): 95.8%, near 95%. Only 41 points are in the CI band. ▲

Why Added

Let \(k=\frac{(x-\bar x)^2}{S_{xx}}\ge0\).

1
\[\frac{\text{PI width}}{\text{CI width}}=\frac{2\,t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{1+\frac1n+k}}{2\,t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+k}}\]
2
\[=\sqrt{\frac{1+\frac1n+k}{\frac1n+k}}\]
3
\[=\sqrt{\frac{1+\frac1n}{\frac1n}}\]

At \(x=\bar x\), \(k=0\).

4
\[=\sqrt{n+1}\]
5
\[=\sqrt{237}=15.3948\]

Agrees with 15.39 at \(\bar x\). ∎

Note 2: in the PI, the 1 is the largest term. At the right edge, \(k\) is only 0.040140. The square root grows only from \(\sqrt{1.004237}\) to \(\sqrt{1.044377}\). Without the 1, the same \(k\) has a large effect on the CI.

05.3Fitted values, the CI for the mean, and the PI together Slides p.40

What Slides p.40

Slides p.40 is code only, with no numbers. It gives three quantities from the fitted model:

Fitted values: \(\hat y_i=\hat\beta_0+\hat\beta_1x_i\), the fitted line above each sample \(x_i\).
CIs for the means: \(\hat\mu_0\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{\frac1n+\frac{(x_0-\bar x)^2}{S_{xx}}}\) at each grid value \(x_0\).
PIs: \(\hat y_{new}\pm t_{1-\alpha/2,\,n-2}\,\hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}}\) at each grid value \(x_{new}\).

At the same \(x\), \(\hat\mu_0=\hat y_{new}\). Only the standard error differs.

How Added

Calculate the CI and the PI at the same \(x\)
  1. Calculate the center \(\hat\beta_0+\hat\beta_1x\) for both intervals.
  2. Calculate \(k=(x-\bar x)^2/S_{xx}\).
  3. Calculate the CI standard error \(\hat\sigma\sqrt{1/n+k}\).
  4. Calculate the PI standard error \(\hat\sigma\sqrt{1+1/n+k}\).
  5. Multiply each by \(t_{1-\alpha/2,\,n-2}\) to get the two half-widths.
  6. Subtract each half-width from the center and add it.

Self-check: ① Same midpoint. ② The PI contains the CI.

Example 3 · Both intervals at \(x=3000\) Added

1

Center How step 1

\[\hat\mu_0=\hat y_{new}=335.432315+0.2608207\times3000\]
\[=335.432315+782.462146\]
\[=1117.894461\]
2

Distance term How step 2

\[k=\frac{(3000-3637.8644)^2}{30647233.66}\]
\[=\frac{(-637.8644)^2}{30647233.66}\]
\[=\frac{406871.00}{30647233.66}\]
\[=0.013276\]
3

CI standard error How step 3

\[72.35066\sqrt{0.004237+0.013276}=72.35066\sqrt{0.017513}\]
\[=72.35066\times0.132338\]
\[=9.574711\]
4

PI standard error How step 4

\[72.35066\sqrt{1+0.004237+0.013276}=72.35066\sqrt{1.017513}\]
\[=72.35066\times1.008719\]
\[=72.98146\]
5

Half-widths How step 5

\[\text{CI: }1.970154\times9.574711=18.86365\]
\[\text{PI: }1.970154\times72.98146=143.78469\]
6

Limits How step 6

\[\text{CI: }(1117.89446-18.86365,\ 1117.89446+18.86365)=(1099.031,\ 1136.758)\]
\[\text{PI: }(1117.89446-143.78469,\ 1117.89446+143.78469)=(974.110,\ 1261.679)\]

Midpoint 1117.894 for both. CI width 37.727, PI width 287.569. The PI contains the CI. ▲

A fitted value uses step 1 only. Person 1 has \(x_1=4512\): \(\hat y_1=335.432315+0.2608207\times4512=335.432315+1176.823068=1512.255\) g (Added).

Why Added

Both intervals use the fitted line at \(x\) as the center. The CI targets the fixed \(\mu_0\): only the error in \(\hat\beta_0,\hat\beta_1\) counts. The PI targets \(y_{new}=\beta_0+\beta_1x_{new}+\epsilon_{new}\): the variance \(\sigma^2\) of \(\epsilon_{new}\) also counts. ∎

05.4Practice Added

Use \(n=236\), \(\hat\beta_0=335.432315\), \(\hat\beta_1=0.2608207\), \(\hat\sigma=72.35066\), \(\bar x=3637.8644\), \(S_{xx}=30647233.66\).

Q1. A new person has head size 3500 cm³. Calculate a 90% PI for this person's brain weight by hand. Use \(t_{0.95,\,234}=1.651391\).

Answer
1
\[\hat y_{new}=335.432315+0.2608207\times3500=335.432315+912.872504=1248.3048\]
2
\[k=\frac{(3500-3637.8644)^2}{30647233.66}=\frac{(-137.8644)^2}{30647233.66}=\frac{19006.59}{30647233.66}=0.000620\]
3
\[72.35066\sqrt{1+0.004237+0.000620}=72.35066\sqrt{1.004857}\]
4
\[=72.35066\times1.002426=72.5262\]
5
\[1.651391\times72.5262=119.7691\]
6
\[1248.3048\pm119.7691=(1128.536,\ 1368.074)\]

Q2. At \(x=\bar x\), show that the 95% PI width divided by the 95% CI width equals \(\sqrt{n+1}\). Evaluate it for the brainhead data.

Answer

At \(x=\bar x\), the distance term is 0.

1
\[\frac{2\,t_{0.975,\,n-2}\,\hat\sigma\sqrt{1+\frac1n}}{2\,t_{0.975,\,n-2}\,\hat\sigma\sqrt{\frac1n}}=\sqrt{\frac{1+\frac1n}{\frac1n}}\]
2
\[=\sqrt{n+1}\]
3
\[=\sqrt{237}=15.39\]

Check: \(285.687/18.557=15.39\).

Q3. Two 95% intervals at \(x=3000\) are (1099.031, 1136.758) and (974.110, 1261.679). (a) Which is the CI for the mean response, and which is the PI? (b) Why do both have the same midpoint?

Answer

(a) The first (width 37.727) is the CI. The second (width 287.569) is the PI: its standard error has the extra 1 from \(\sigma^2\).

(b) Both have the center \(\hat\beta_0+\hat\beta_1x=335.432315+0.2608207\times3000=1117.894\). The mean estimate and the predicted value are the same number.

Unit 06Practice questions Q1 and Q2

Plan · Slides p.41–43

Section page (p.41), Q1: MLEs and \(\hat\sigma/\hat\sigma_{MLE}\) (p.42), Q2: two CIs and one PI (p.43), similar exercises (Added).

Slides p.41 · Section page "Practice"

The two questions use no data set, only the numbers given.

06.1Practice Q1: MLEs and the ratio \(\hat\sigma/\hat\sigma_{MLE}\) Slides p.42

What · The question Slides p.42

Lecture 4 · p.42 · Practice Q1 Consider the simple linear regression model \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2),\qquad i=1,\dots,n.\] (a) Write down expressions of the MLEs of \(\beta_0\) and \(\beta_1\) in terms of \(\bar x,\bar y,S_{xy}\) and the sample variance of \(x\).
(b) For \(n=20\), find the ratio \(\hat\sigma/\hat\sigma_{MLE}\).
(c) Suppose \(n=50\), and \(S_{xx}=180,\ \bar x=4.8,\ S_{xy}=353,\ \sum_{i=1}^n x_iy_i=3378\). Compute the MLEs for \(\beta_0\) and \(\beta_1\).
The maximum likelihood estimator (MLE) is the parameter value that makes the observed data most probable under the model.
The sample variance of \(x\) is \(s_x^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar x)^2=S_{xx}/(n-1)\).
Earlier result · Lecture 3 · p.7 So the least squares estimators are: \[\hat\beta_1=\frac{S_{xy}}{S_{xx}},\qquad \hat\beta_0=\bar y-\hat\beta_1\bar x.\] OLS=MLE under the assumption of Normality.

With normal errors, the least squares estimators are the MLEs.

Earlier result · Lecture 2 · p.5 For observations \(y_1,\dots,y_n\), the sample variance is: \(\displaystyle s_y^2=\frac{1}{n-1}\sum_{i}^{n}(y_i-\bar y)^2\)
Earlier result · Lecture 2 · p.35 \[\hat\sigma^2_{ML}=\frac{\sum_{i=1}^n e_i^2}{n}.\qquad\text{However we typically adopt a different estimator: }\ \hat\sigma^2=\frac{\sum_{i=1}^n e_i^2}{n-2}.\]

\(\hat\sigma_{MLE}\) in Q1 is \(\hat\sigma_{ML}\). It divides by \(n\); \(\hat\sigma^2\) divides by \(n-2\). The numerator \(\sum e_i^2\) is the same.

How · The steps Added

Q1 · How to solve
  1. (a) Write the MLEs. ① Write \(\hat\beta_1=S_{xy}/S_{xx}\). ② Replace \(S_{xx}\) with \((n-1)s_x^2\). ③ Write \(\hat\beta_0=\bar y-\hat\beta_1\bar x\). ④ Put ② into ③.
  2. (b) Find the ratio. ① Write \(\hat\sigma^2\) and \(\hat\sigma^2_{MLE}\). ② Divide them; \(\sum e_i^2\) cancels. ③ Take the square root. ④ Put in \(n\).
  3. (c) Calculate the estimates. ① Calculate \(\hat\beta_1=S_{xy}/S_{xx}\). ② Find \(\bar y\) from \(\sum x_iy_i=S_{xy}+n\bar x\bar y\). ③ Calculate \(\hat\beta_0=\bar y-\hat\beta_1\bar x\).

Self-check: (a) contains only \(\bar x,\bar y,S_{xy},s_x^2\), \(n\), not \(S_{xx}\). (b) is more than 1, because \(n-2<n\). In (c), \(S_{xy}+n\bar x\bar y=3378\).

Example · The answers Added

Answer to Q1(a)
1

Write the slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{S_{xy}}{S_{xx}}\]

Basis: OLS = MLE with normal errors.

2

Replace \(S_{xx}\) L2 p.5 · \(s_x^2=S_{xx}/(n-1)\)

\[\hat\beta_1=\frac{S_{xy}}{(n-1)s_x^2}\]
3

Write the intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=\bar y-\frac{S_{xy}}{(n-1)s_x^2}\,\bar x\]
Answer to Q1(b)
1

Write the two estimators L2 p.35 · \(\hat\sigma^2_{ML}\) and \(\hat\sigma^2\)

\[\hat\sigma^2=\frac{\sum_{i=1}^n e_i^2}{n-2},\qquad \hat\sigma^2_{MLE}=\frac{\sum_{i=1}^n e_i^2}{n}\]
2

Divide and take the square root (Why (ii))

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{n}{n-2}}\]
3

Put in \(n=20\)

\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{20}{18}}\]
\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{10}{9}}\]
\[\frac{\hat\sigma}{\hat\sigma_{MLE}}\approx 1.0541\]

\(\hat\sigma\) is approximately 5.4% larger than \(\hat\sigma_{MLE}\).

Answer to Q1(c)
1

Calculate the slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{353}{180}\]
\[\hat\beta_1\approx 1.961111\]
2

Put the numbers in the identity \(\sum x_iy_i=S_{xy}+n\bar x\bar y\) (Why (iii))

\[3378=353+50\times4.8\times\bar y\]
\[3378=353+240\,\bar y\]
3

Find \(\bar y\)

\[240\,\bar y=3025\]
\[\bar y=\frac{3025}{240}\]
\[\bar y\approx 12.604167\]
4

Calculate the intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=12.604167-1.961111\times4.8\]
\[\hat\beta_0=12.604167-9.413333\]
\[\hat\beta_0\approx 3.190833\]

Self-check: \(353+240\times12.604167\approx3378\).

\(\hat\beta_1\approx1.9611\), \(\hat\beta_0\approx3.1908\).

Why · The derivations Added

(i) \(S_{xx}\) and the sample variance

1
\[s_x^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar x)^2\]

Basis: Lecture 2 · p.5

2
\[s_x^2=\frac{S_{xx}}{n-1}\]

Basis: definition of \(S_{xx}\)

3
\[S_{xx}=(n-1)s_x^2\]

(ii) The ratio \(\hat\sigma/\hat\sigma_{MLE}\)

1
\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{\sum e_i^2/(n-2)}{\sum e_i^2/n}\]

Basis: Lecture 2 · p.35

2
\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{\sum e_i^2}{n-2}\cdot\frac{n}{\sum e_i^2}\]
3
\[\frac{\hat\sigma^2}{\hat\sigma^2_{MLE}}=\frac{n}{n-2}\]
4
\[\frac{\hat\sigma}{\hat\sigma_{MLE}}=\sqrt{\frac{n}{n-2}}\]

Basis: both are positive

(iii) The identity \(\sum x_iy_i=S_{xy}+n\bar x\bar y\)

1
\[S_{xy}=\sum_{i=1}^n(y_i-\bar y)(x_i-\bar x)\]
2
\[S_{xy}=\sum x_iy_i-\bar x\sum y_i-\bar y\sum x_i+n\bar x\bar y\]
3
\[S_{xy}=\sum x_iy_i-\bar x(n\bar y)-\bar y(n\bar x)+n\bar x\bar y\]

Basis: \(\sum y_i=n\bar y\), \(\sum x_i=n\bar x\)

4
\[S_{xy}=\sum x_iy_i-n\bar x\bar y\]
5
\[\sum x_iy_i=S_{xy}+n\bar x\bar y\qquad\blacksquare\]

06.2Practice Q2: two confidence intervals and one prediction interval Slides p.43

What · The question Slides p.43

Lecture 4 · p.43 · Practice Q2 Consider the simple linear regression model \[y_i=\beta_0+\beta_1x_i+\epsilon_i,\qquad \epsilon_i\overset{iid}{\sim}N(0,\sigma^2),\qquad i=1,\dots,n.\] Suppose \(n=10\) and \(\hat\beta_1=5,\ \hat\beta_0=-2,\ S_{xx}=16,\ \bar x=0,\ \hat\sigma=0.4\).
Also we know that if \(T\sim t_{(8)}\): \(P(T>-1.86)=0.95\), \(P(|T|>2.75)=0.025\), \(P(T<2.31)=0.975\).
(a) Calculate a two-sided 95% confidence interval for \(\beta_1\).
(b) Calculate a two-sided 95% confidence interval for \(\beta_0\).
(c) Calculate a 90% prediction interval for a new response at \(x_{new}=5\).
\(t_{(8)}\) has \(n-2=10-2=8\) degrees of freedom.
Lecture 4 · p.5 (review of Lecture 3) 95% CI for \(\beta_1\) is \(\displaystyle\hat\beta_1\pm t_{0.975,n-2}\times\sqrt{\frac{\hat\sigma^2}{S_{xx}}}\) or \(\hat\beta_1\pm t_{0.975,n-2}\times \mathrm{SE}(\hat\beta_1)\).
More generally a \(100\times(1-\alpha)\%\) CI is: \(\hat\beta_1\pm t_{1-\alpha/2,n-2}\times \mathrm{SE}(\hat\beta_1)\).
Earlier result · Lecture 3 · p.24 \[\hat\beta_0\sim N\!\left(\beta_0,\ \sigma^2\left(\frac1n+\frac{\bar x^2}{S_{xx}}\right)\right)\]

\(\hat\sigma\) for \(\sigma\) gives \(\mathrm{SE}(\hat\beta_0)=\hat\sigma\sqrt{\frac1n+\frac{\bar x^2}{S_{xx}}}\). The CI is \(\hat\beta_0\pm t_{1-\alpha/2,n-2}\,\mathrm{SE}(\hat\beta_0)\) (Lecture 3 · p.36).

Lecture 4 · p.37 So a 100(1−\(\alpha\))% prediction interval is: \[\hat y_{new}\pm t_{1-\frac{\alpha}{2},n-2}\ \hat\sigma\sqrt{1+\frac1n+\frac{(x_{new}-\bar x)^2}{\sum_{i=1}^n(x_i-\bar x)^2}}\]

How · The steps Added

Q2 · How to solve
  1. Step 0: Read the quantiles. ① Find \(1-\alpha/2\): 95% gives 0.975, 90% gives 0.95. ② Find \(P(T<c)=1-\alpha/2\) in the question. ③ Change \(P(T>-c)\) to \(P(T<c)\) by symmetry.
  2. (a) CI for \(\beta_1\). ① Calculate \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\). ② Multiply by \(t_{0.975,8}\). ③ Write \(\hat\beta_1\) plus or minus this number.
  3. (b) CI for \(\beta_0\). ① Calculate \(\frac1n+\frac{\bar x^2}{S_{xx}}\). ② Take the square root and multiply by \(\hat\sigma\): \(\mathrm{SE}(\hat\beta_0)\). ③ Write \(\hat\beta_0\pm t_{0.975,8}\,\mathrm{SE}(\hat\beta_0)\).
  4. (c) 90% PI. ① Calculate \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\). ② Calculate \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\). ③ Take the square root and multiply by \(\hat\sigma\). ④ Multiply by \(t_{0.95,8}\). Write \(\hat y_{new}\) plus or minus this number.

Self-check: Degrees of freedom \(n-2=8\). Each interval has the point estimate at its center. The PI has an extra "1" under the square root.

0 1.86 −1.86 2.31 −2.31 t₀.₉₇₅,₈ t₀.₉₅,₈ 0.025 0.025 middle 95% (±2.31) middle 90% (±1.86) T ~ t(8)
Figure 6-1. The \(t_{(8)}\) density. Blue tails (beyond ±2.31): 0.025 each, middle 0.95. Orange-plus-blue tails (beyond ±1.86): 0.05 each, middle 0.90.

Example · The answers Added

Step 0: read \(t_{0.975,8}\) and \(t_{0.95,8}\) from the question
1

A 95% interval needs \(t_{0.975,8}\) How Step 0 ①

\[1-\alpha=0.95\ \Rightarrow\ 1-\alpha/2=0.975\]
2

Third statement definition of a quantile

\[P(T<2.31)=0.975\ \Rightarrow\ t_{0.975,8}=2.31\]
3

A 90% interval needs \(t_{0.95,8}\) How Step 0 ①

\[1-\alpha=0.90\ \Rightarrow\ 1-\alpha/2=0.95\]
4

First statement, by symmetry (Why (i)) How Step 0 ③

\[P(T>-1.86)=0.95\ \Rightarrow\ P(T<1.86)=0.95\ \Rightarrow\ t_{0.95,8}=1.86\]

\(P(|T|>2.75)=0.025\) puts 0.0125 in each tail: a two-sided 97.5% interval. Q2 does not use it.

Answer to Q2(a): 95% CI for \(\beta_1\)
1

Standard error L4 p.5 · \(\mathrm{SE}(\hat\beta_1)=\hat\sigma/\sqrt{S_{xx}}\)

\[\mathrm{SE}(\hat\beta_1)=\frac{0.4}{\sqrt{16}}\]
\[\mathrm{SE}(\hat\beta_1)=\frac{0.4}{4}\]
\[\mathrm{SE}(\hat\beta_1)=0.1\]
2

Half-width Step 0 · \(t_{0.975,8}=2.31\)

\[t_{0.975,8}\times \mathrm{SE}(\hat\beta_1)=2.31\times0.1\]
\[t_{0.975,8}\times \mathrm{SE}(\hat\beta_1)=0.231\]
3

Interval L4 p.5 · \(\hat\beta_1\pm t_{0.975,n-2}\mathrm{SE}(\hat\beta_1)\)

\[5\pm0.231\]
\[(5-0.231,\ 5+0.231)=(4.769,\ 5.231)\]
Answer to Q2(b): 95% CI for \(\beta_0\)
1

SE formula L3 p.24 · \(\mathrm{SE}(\hat\beta_0)\)

\[\mathrm{SE}(\hat\beta_0)=\hat\sigma\sqrt{\frac1n+\frac{\bar x^2}{S_{xx}}}\]
2

Put in \(n=10,\ \bar x=0,\ S_{xx}=16\)

\[\frac1n+\frac{\bar x^2}{S_{xx}}=\frac1{10}+\frac{0^2}{16}\]
\[\frac1n+\frac{\bar x^2}{S_{xx}}=0.1+0\]
\[\frac1n+\frac{\bar x^2}{S_{xx}}=0.1\]
3

Square root, times \(\hat\sigma\)

\[\mathrm{SE}(\hat\beta_0)=0.4\sqrt{0.1}\]
\[\mathrm{SE}(\hat\beta_0)=0.4\times0.316228\]
\[\mathrm{SE}(\hat\beta_0)\approx0.126491\]
4

Half-width Step 0 · \(t_{0.975,8}=2.31\)

\[2.31\times0.126491\approx0.292194\]
5

Interval L3 p.36 · \(\hat\beta_0\pm t\cdot \mathrm{SE}(\hat\beta_0)\)

\[-2\pm0.292194\]
\[(-2-0.292194,\ -2+0.292194)\approx(-2.292,\ -1.708)\]
Answer to Q2(c): 90% PI at \(x_{new}=5\)
1

Center L4 p.37 · \(\hat y_{new}=\hat\beta_0+\hat\beta_1x_{new}\)

\[\hat y_{new}=-2+5\times5\]
\[\hat y_{new}=-2+25\]
\[\hat y_{new}=23\]
2

Terms under the square root L4 p.37 · \(1+\frac1n+\frac{(x_{new}-\bar x)^2}{S_{xx}}\)

\[1+\frac1{10}+\frac{(5-0)^2}{16}=1+0.1+\frac{25}{16}\]
\[1+0.1+1.5625=2.6625\]
3

Square root, times \(\hat\sigma\)

\[\hat\sigma\sqrt{2.6625}=0.4\times1.631717\]
\[\hat\sigma\sqrt{2.6625}\approx0.652687\]
4

Half-width Step 0 · \(t_{0.95,8}=1.86\)

\[1.86\times0.652687\approx1.213997\]
5

Interval L4 p.37 · \(\hat y_{new}\pm t_{1-\alpha/2,n-2}\,\hat\sigma\sqrt{\cdots}\)

\[23\pm1.213997\]
\[(23-1.213997,\ 23+1.213997)\approx(21.786,\ 24.214)\]
0123456 0102030 24.214 21.786 ŷnew = 23 ŷ = −2 + 5x x (x̄ = 0 is at the left end)
Figure 6-2. Fitted line \(\hat y=-2+5x\) (blue) and 90% prediction band (green). Orange bar: the PI \((21.786,\ 24.214)\) at \(x_{new}=5\). The band is narrowest at \(\bar x=0\) and widens as \((x_{new}-\bar x)^2/S_{xx}\) increases.

Why · The derivations Added

Only the two new steps of Q2.

(i) From \(P(T>-1.86)=0.95\) to \(t_{0.95,8}=1.86\)

1
\[P(T>-1.86)=0.95\]

Basis: the question

2
\[P(-T<1.86)=0.95\]
3
\[P(T<1.86)=0.95\]

Basis: \(-T\) has the same distribution as \(T\) (symmetry)

4
\[t_{0.95,8}=1.86\]

Basis: \(P(T<t_{q,8})=q\)

(ii) A 90% interval uses \(t_{0.95,8}\)

1
\[100(1-\alpha)\%=90\%\]
2
\[\alpha=0.10\]
3
\[1-\frac{\alpha}{2}=1-0.05\]
4
\[1-\frac{\alpha}{2}=0.95\]
5
\[t_{1-\frac{\alpha}{2},n-2}=t_{0.95,8}\qquad\blacksquare\]

Basis: Slides p.37, \(n-2=8\)

06.3Practice Added

Exercise 6.1

Model \(y_i=\beta_0+\beta_1x_i+\epsilon_i\), \(\epsilon_i\overset{iid}{\sim}N(0,\sigma^2)\). \(n=12\), \(S_{xx}=40\), \(\bar x=3\), \(S_{xy}=60\), \(\bar y=10\). Calculate the MLEs for \(\beta_0\) and \(\beta_1\).

Answer
1

Slope L3 p.7 · \(\hat\beta_1=S_{xy}/S_{xx}\)

\[\hat\beta_1=\frac{60}{40}\]
\[\hat\beta_1=1.5\]
2

Intercept L3 p.7 · \(\hat\beta_0=\bar y-\hat\beta_1\bar x\)

\[\hat\beta_0=10-1.5\times3\]
\[\hat\beta_0=10-4.5\]
\[\hat\beta_0=5.5\]
Exercise 6.2

Setting of Practice Q2 (\(n=10,\ \hat\beta_0=-2,\ \hat\beta_1=5,\ \hat\sigma=0.4\)). Find \(\hat\sigma_{MLE}\).

Answer
1

Ratio formula 06.1 Why (ii) · \(\hat\sigma/\hat\sigma_{MLE}=\sqrt{n/(n-2)}\)

\[\hat\sigma_{MLE}=\hat\sigma\sqrt{\frac{n-2}{n}}\]
2

Put in \(n=10\), \(\hat\sigma=0.4\)

\[\hat\sigma_{MLE}=0.4\sqrt{\frac{8}{10}}\]
\[\hat\sigma_{MLE}=0.4\times0.894427\]
\[\hat\sigma_{MLE}\approx0.3578\]
Exercise 6.3

Setting of Practice Q2 (\(n=10,\ \hat\beta_1=5,\ \hat\beta_0=-2,\ S_{xx}=16,\ \bar x=0,\ \hat\sigma=0.4\), \(P(T<2.31)=0.975\) for \(T\sim t_{(8)}\)). Calculate a 95% PI for a new response at \(x_{new}=2\).

Answer
1

Quantile How Step 0

\[95\%\Rightarrow 1-\alpha/2=0.975\Rightarrow t_{0.975,8}=2.31\]
2

Center L4 p.37 · \(\hat y_{new}\)

\[\hat y_{new}=-2+5\times2\]
\[\hat y_{new}=8\]
3

Terms under the square root L4 p.37

\[1+\frac1{10}+\frac{(2-0)^2}{16}=1+0.1+0.25\]
\[1+0.1+0.25=1.35\]
4

Times \(\hat\sigma\), then times the quantile

\[0.4\sqrt{1.35}=0.4\times1.161895\approx0.464758\]
\[2.31\times0.464758\approx1.073591\]
5

Interval

\[8\pm1.073591\approx(6.926,\ 9.074)\]