《FUNDAMENTALS OF STATISTICAL SIGNAL PROCESSING ESTIMATION THEORY》

1Introduction1

1.1 Estimation in Signal Processing1

1.2 The Mathematical Estimation Problem7

1.3 Assessing Estimator Performance9

1.4 Some Notes to the Reader12

2Minimum Variance Unbiased Estimation15

2.1 Introduction15

2.2 Summary15

2.3 Unbiased Estimators16

2.4 Minimum Variance Criterion19

2.5 Existence of the Minimum Variance Unbiased Estimator20

2.6 Finding the Minimum Variance Unbiased Estimator21

2.7 Extension to a Vector Parameter22

3Cramer-Rao Lower Bound27

3.1 Introduction27

3.2 Summary27

3.3 Estimator Accuracy Considerations28

3.4 Cramer-Rao Lower Bound30

3.5 General CRLB for Signals in White Gaussian Noise35

3.6 Transformation of Parameters37

3.7 Extension to a Vector Parameter39

3.8 Vector Parameter CRLB for Transformations45

3.9 CRLB for the General Gaussian Case47

3.10 Asymptotic CRLB for WSS Gaussian Random Processes50

3.11 Signal Processing Examples53

3A Derivation of Scalar Parameter CRLB67

3B Derivation of Vector Parameter CRLB70

3C Derivation of General Gaussian CRLB73

3D Derivation of Asymptotic CRLB77

4 Linear Models83

4.1Introduction83

4.2 Summary83

4.3 Definition and Properties83

4.4 Linear Model Examples86

4.5 Extension to the Linear Model94

5General Minimum Variance Unbiased Estimation101

5.1 Introduction101

5.2 Summary101

5.3 Sufficient Statistics102

5.4 Finding Sufficient Statistics104

5.5 Using Sufficiency to Find the MVU Estimator107

5.6 Extension to a Vector Parameter116

5A Proof of Neyman-Fisher Factorization Theorem （Scalar Parameter）127

5B Proof of Rao-Blackwell-Lehmann-Scheffe Theorem （Scalar Parameter）130

6 Best Linear Unbiased Estimators133

6.1Introduction133

6.2 Summary133

6.3 Definition of the BLUE134

6.4 Finding the BLUE136

6.5 Extension to a Vector Parameter139

6.6 Signal Processing Example141

6A Derivation of Scalar BLUE151

6B Derivation of Vector BLUE153

7 Maximum Likelihood Estimation157

7.1Introduction157

7.2 Summary157

7.3 An Example158

7.4 Finding the MLE162

7.5 Properties of the MLE164

7.6 MLE for Transformed Parameters173

7.7 Numerical Determination of the MLE177

7.8 Extension to a Vector Parameter182

7.9 Asymptotic MLE190

7.10 Signal Processing Examples191

7A Monte Carlo Methods205

7B Asymptotic PDF of MLE for a Scalar Parameter211

7C Derivation of Conditional Log-Likelihood for EM Algorithm Example214

8Least Squares219

8.1 Introduction219

8.2 Summary219

8.3 The Least Squares Approach220

8.4 Linear Least Squares223

8.5 Geometrical Interpretations226

8.6 Order-Recursive Least Squares232

8.7 Sequential Least Squares242

8.8 Constrained Least Squares251

8.9 Nonlinear Least Squares254

8.10 Signal Processing Examples260

8A Derivation of Order-Recursive Least Squares282

8B Derivation of Recursive Projection Matrix285

8C Derivation of Sequential Least Squares286

9 Method of Moments289

9.1Introduction289

9.2 Summary289

9.3 Method of Moments289

9.4 Extension to a Vector Parameter292

9.5 Statistical Evaluation of Estimators294

9.6 Signal Processing Example299

10 The Bayesian Philosophy309

10.1 Introduction309

10.2 Summary309

10.3 Prior Knowledge and Estimation310

10.4 Choosing a Prior PDF316

10.5 Properties of the Gaussian PDF321

10.6 Bayesian Linear Model325

10.7 Nuisance Parameters328

10.8 Bayesian Estimation for Deterministic Parameters330

10A Derivation of Conditional Gaussian PDF337

11 General Bayesian Estimators341

11.1 Introduction341

11.2 Summary341

11.3 Risk Functions342

11.4 Minimum Mean Square Error Estimators344

11.5 Maximum A Posteriori Estimators350

11.6 Performance Description359

11.7 Signal Processing Example365

11A Conversion of Continuous-Time System to Discrete-Time System375

12 Linear Bayesian Estimators379

12.1 Introduction379

12.2 Summary379

12.3 Linear MMSE Estimation380

12.4 Geometrical Interpretations384

12.5 The Vector LMMSE Estimator389

12.6 Sequential LMMSE Estimation392

12.7 Signal Processing Examples - Wiener Filtering400

12A Derivation of Sequential LMMSE Estimator415

13 Kalman Filters419

13.1 Introduction419

13.2 Summary419

13.3 Dynamical Signal Models420

13.4 Scalar Kalman Filter431

13.5 Kalman Versus Wiener Filters442

13.6 Vector Kalman Filter446

13.7 Extended Kalman Filter449

13.8 Signal Processing Examples452

13A Vector Kalman Filter Derivation471

13B Extended Kalman Filter Derivation476

14 Summary of Estimators479

14.1 Introduction479

14.2 Estimation Approaches479

14.3 Linear Model486

14.4 Choosing an Estimator489

15 Extensions for Complex Data and Parameters493

15.1 Introduction493

15.2 Summary493

15.3 Complex Data and Parameters494

15.4 Complex Random Variables and PDFs500

15.5 Complex WSS Random Processes513

15.6 Derivatives，Gradients，and Optimization517

15.7 Classical Estimation with Complex Data524

15.8 Bayesian Estimation532

15.9 Asymptotic Complex Gaussian PDF535

15.10Signal Processing Examples539

15A Derivation of Properties of Complex Covariance Matrices555

15B Derivation of Properties of Complex Gaussian PDF558

15C Derivation of CRLB and MLE Formulas563

A1 Review of Important Concepts567

A1.1 Linear and Matrix Algebra567

A1.2 Probability，Random Processes，and Time Series Models574

A2 Glossary of Symbols and Abbreviations583

INDEX589