AskSia

Plus

Q. 1. Consider the observations x[n] = A + w[n] n = 0, 1, . . . , N - 1 ...

Oct 7, 2024

Q. 1. Consider the observations x[n] = A + w[n] n = 0, 1, . . . , N - 1 where A is an unknown constant and w[n] is a sequence of independent, identi- cally distributed (IID) Gaussian random variables having a zero mean and a known variance σ 2 . (a) To estimate the unknown parameter θ 1 = A, the following estimator N-1 is proposed. Find a0 , . . . , aN -1 so that the estimator is unbiased and the vari- ance is minimized. Show all the detailed steps involved. (b) By using the optimal values of an’s obtained in part (a), ﬁnd the variance of θ(ˆ)1 , denoted by var (θ(ˆ)1 ). What will happen to var (θ(ˆ)1 ) as N → ∞? (c) To estimate the unknown parameter θ2 = A2 , the following estimator is proposed. Is the estimator θ(ˆ)2 unbiased? Is θ(ˆ)2 asymptotically unbiased?

Solution by Steps

step 1

To ensure that the estimator

\hat{\theta}_{1} = \sum_{n=0}^{N-1} a_{n} x[n]

is unbiased, we require

E[\hat{\theta}_{1}] = A

. Since

x[n] = A + w[n]

, we have:

E[\hat{\theta}_{1}] = \sum_{n=0}^{N-1} a_{n} E[x[n]] = \sum_{n=0}^{N-1} a_{n} (A + E[w[n]]) = A \sum_{n=0}^{N-1} a_{n}

Thus, we need

\sum_{n=0}^{N-1} a_{n} = 1

step 2

To minimize the variance of

\hat{\theta}_{1}

, we calculate

\operatorname{var}(\hat{\theta}_{1})

\operatorname{var}(\hat{\theta}_{1}) = \operatorname{var}\left(\sum_{n=0}^{N-1} a_{n} w[n]\right) = \sum_{n=0}^{N-1} a_{n}^2 \operatorname{var}(w[n]) = \sigma^2 \sum_{n=0}^{N-1} a_{n}^2

We need to minimize

\sum_{n=0}^{N-1} a_{n}^2

subject to

\sum_{n=0}^{N-1} a_{n} = 1

step 3

Using the method of Lagrange multipliers, we set up the function:

L(a_{0}, a_{1}, \ldots, a_{N-1}, \lambda) = \sum_{n=0}^{N-1} a_{n}^2 - \lambda \left(\sum_{n=0}^{N-1} a_{n} - 1\right)

Taking partial derivatives and setting them to zero gives:

\frac{\partial L}{\partial a_{n}} = 2a_{n} - \lambda = 0 \implies a_{n} = \frac{\lambda}{2}

Substituting into the constraint

\sum_{n=0}^{N-1} a_{n} = 1

leads to

N \cdot \frac{\lambda}{2} = 1 \implies \lambda = \frac{2}{N}

and thus

a_{n} = \frac{1}{N}

step 4

Substituting

a_{n} = \frac{1}{N}

into the variance expression:

\operatorname{var}(\hat{\theta}_{1}) = \sigma^2 \sum_{n=0}^{N-1} \left(\frac{1}{N}\right)^2 = \sigma^2 \cdot \frac{N}{N^2} = \frac{\sigma^2}{N}

N \rightarrow \infty

\operatorname{var}(\hat{\theta}_{1}) \rightarrow 0

step 5

For

\hat{\theta}_{2} = \left(\frac{1}{N} \sum_{n=0}^{N-1} x[n]\right)^{2}

, we find its expected value:

E[\hat{\theta}_{2}] = E\left[\left(\frac{1}{N} \sum_{n=0}^{N-1} (A + w[n])\right)^{2}\right] = E\left[\left(A + \frac{1}{N} \sum_{n=0}^{N-1} w[n]\right)^{2}\right]

Using the properties of expectation, we find that

E[\hat{\theta}_{2}] = A^2 + \frac{\sigma^2}{N}

. Thus,

\hat{\theta}_{2}

is biased

step 6

To check for asymptotic unbiasedness, we consider the limit as

N \rightarrow \infty

\lim_{N \to \infty} E[\hat{\theta}_{2}] = A^2 + \lim_{N \to \infty} \frac{\sigma^2}{N} = A^2

Thus,

\hat{\theta}_{2}

is asymptotically unbiased

Answer

(a)

a_{n} = \frac{1}{N}

for

n = 0, 1, \ldots, N-1

; (b)

\operatorname{var}(\hat{\theta}_{1}) = \frac{\sigma^2}{N}

and approaches 0 as

N \to \infty

; (c)

\hat{\theta}_{2}

is biased but asymptotically unbiased.

Key Concept

Unbiased estimators and their variances are crucial in statistical inference.

Explanation

The solution demonstrates how to derive unbiased estimators and analyze their variances, highlighting the importance of asymptotic properties in estimation.