proof-mse-bias-variance.qmd

Let's start by expanding each term of the right-hand side:

$$
\ba
\sqf{\bias{\hth}}
&=\sqf{\E{\hth} - \th}\\
&=\sqf{\E{\hth}} - 2\E{\hth}\th+\th^2\\
\ea
$$

$$\var{\hth} = \E{\hth^2} - \sqf{\E{\hth}}\\$$

Now, add them together and simplify:

$$
\ba
\sqf{\bias{\hth}} + \var{\hth}
&=\sqf{\E{\hth}} - 2\E{\hth}\th+\th^2 + \E{\hth^2} - \sqf{\E{\hth}}\\
&=\E{\hth^2} - 2\E{\hth}\th+\th^2\\
\ea
$$

Now let's expand the left-hand side to reach the same expression:

$$
\ba
\mselr{\hth} 
&= \E{(\e(\hth))^2}\\
&= \E{(\hth - \th)^2}\\
&= \E{\hth^2 - 2\hth\th - \th^2}\\
&=\E{\hth^2} - \E{2\hth\th}+\E{\th^2}\\
&=\E{\hth^2} - 2\E{\hth}\th+\th^2\\
\ea
$$

$\mselr{\hth}$ and $\sqf{\bias{\hth}} + \var{\hth}$ both equal $\E{\hth^2} - 2\E{\hth}\th+\th^2$. Equality is transitive, so $\mselr{\hth}$ and $\sqf{\bias{\hth}} + \var{\hth}$ are equal to each other:

$$\mselr{\hth} = \sqf{\bias{\hth}} + \var{\hth}$$