《概率论与数理统计》by 陈希儒

$X= (X_1,X_2,...,X_n)$ is a size of n sample from Normal population. $T(X)$ is a desired statistic. The statistic test:

$$ H_0: \theta \in \Theta_0 \leftrightarrow H_1: \theta\in\Theta_1$$

where the parameters are well defined.

Power function(功效函数) is the probability of $H_0$ being rejected with the sample $X$ given the parameter$\theta \in \Theta$:

$$\beta(\theta)= Prob(H_0 \ is \ rejected | \theta \in \Theta) \ =P(T(X) \ is \ one \ realization \ of\ those\ lower\ density\ points|\theta \in \Theta )$$

So it is a function with respect to $\theta\in \Theta$ .

Power of a test is the power function limited on $\Theta_1$. We must careful distinct the two concepts power function VS power

Type I error: it is the probability that the desired statistic $T(X)$ is one realization of lower density points (i.e. $\ H_0 $ is rejected) when $\theta\in\Theta_0$, which is just the power function limited on $\Theta_0$. So it is equal to $\beta(\theta)| _{\theta\in\Theta_0}$

Type II error: it is the probability that the desired statistic $T(X)$ is not a realization of lower density points (i.e. $\ H_0 $ is accepted) when $\theta\in\Theta_1$, which is just the power function limited on $\Theta_1$. So it is equal to $1-\beta(\theta) | _{\theta\in\Theta_1}$

$\alpha$-level test: It is a hypothesis test whose power function $\beta(\theta)$ of hypothesis test related to desired statistic $T(X)$ are less than $\alpha$ when $ \theta \in \Theta_0$.

$p$-Value: It is the probability that the desired statistic $T(X)$ takes $T(x)$ or even lower density realizations than $T(x)$ given the null hypothesis. Or we can say in this way, it is the probability under the null hythothesis of obtaining evidience as extreme or more extreme than that obtianed.

The $p$-Value takes very small values when something uncommon happened given the null hypothesis established, we call the $p$-Value is eminent(显著的)， and we tend to reject the null hypothesis.

How to find a desirable statistic $T(X)$ and define the reject field?

Generally, we first find a desirable estimation of the parameter with respect to the hypothesis test, then build the statistic $T(X)$ in some certain ways, at last we define the reject field intuitively.

For more imformation about these ideas on wiki