《数字视频处理》求取 ⇩

Ⅰ REPRESENTATION OF DIGITAL VIDEO1

1 BASICS OF VIDEO1

1.1 Analog Video1

1.1.1 Analog Video Signal2

1.1.2 Analog Video Standards4

1.1.3 Analog Video Equipment8

1.2 Digital Video9

1.2.1 Digital Video Signal9

1.2.2 Digital Video Standards11

1.2.3 Why Digital Video?14

1.3 Digital Video Processing16

Preface17

About the Author19

2 TIME-VARYING IMAGE FORMATION MODELS19

2.1 Three-Dimensional Motion Models20

2.1.1 Rigid Motion in the Cartesian Coordinates20

About the Notation21

2.1.2 Rigid Motion in the Homogeneous Coordinates26

2.1.3 Deformable Motion27

2.2 Geometric Image Formation28

2.2.1 Perspective Projection28

2.2.2 Orthographic Projection30

2.3.1 Lambertian Reflectance Model32

2.3 Photometric Image Formation32

2.3.2 Photometric Effects of 3-D Motion33

2.4 Observation Noise33

2.5 Exercises34

3 SPATIO-TEMPORAL SAMPLING36

3.1 Sampling for Analog and Digital Video37

3.1.1 Sampling Structures for Analog Video37

3.1.2 Sampling Structures for Digital Video38

3.2 Two-Dimensional Rectangular Sampling40

3.2.1 2-D Fourier Transform Relations41

3.2.2 Spectrum of the Sampled Signal42

3.3 Two-Dimensional Periodic Sampling43

3.3.1 Sampling Geometry44

3.3.2 2-D Fourier Transform Relations in Vector Form44

3.3.3 Spectrum of the Sampled Signal46

3.4 Sampling on 3-D Structures46

3.4.1 Sampling on a Lattice47

3.4.2 Fourier Transform on a Lattice47

3.4.3 Spectrum of Signals Sampled on a Lattice49

3.4.4 Other Sampling Structures51

3.5 Reconstruction from Samples53

3.5.1 Reconstruction from Rectangular Samples53

3.5.2 Reconstruction from Samples on a Lattice55

3.6 Exercises56

4 SAMPLING STRUCTURE CONVERSION57

4.1 Sampling Rate Change for 1-D Signals58

4.1.1 Interpolation of 1-D Signals58

4.1.2 Decimation of 1-D Signals62

4.1.3 Sampling Rate Change by a Rational Factor64

4.2 Sampling Lattice Conversion66

4.3 Exercises70

Ⅱ TWO-DIMENSIONAL MOTION ESTIMATION72

5 OPTICAL FLOW METHODS72

5.1 2-D Motion vs.Apparent Motion72

5.1.1 2-D Motion73

5.1.2 Correspondence and Optical Flow74

5.2 2-D Motion Estimation76

5.2.1 The Occlusion Problem78

5.2.2 The Aperture Problem78

5.2.3 Two-Dimensional Motion Field Models79

5.3 Methods Using the Optical Folw Equation81

5.3.1 The Optical Flow Equation81

5.3.2 Second-Order Differentail Methods82

5.3.3 Block Motion Model83

5.3.4 Horn and Schunck Method84

5.3.5 Estimation of the Gradients85

5.3.6 Adaptive Methods86

5.4 Examples88

5.5 Exercises93

6 BLOCK-BASED METHODS95

6.1 Block-Motion Models95

6.1.1 Translational Block Motion96

6.1.2 Generalized/Deformable Block Motion97

6.2 Phase-Correlation Method99

6.2.1 The Phase-Correlation Function99

6.2.2 Implementation Issues100

6.3 Block-Matching Method101

6.3.1 Matching Criteria102

6.3.2 Search Procedures104

6.4 Hierarchical Motion Estimation106

6.5 Generalized Block-Motion Estimation109

6.5.1 Postprocessing for Improved Motion Compensation109

6.5.2 Deformable Block Matching109

6.6 Examples112

6.7 Exercises115

7 PEL-RECURSIVE METHODS117

7.1 Displaced Frame Difference118

7.2 Gradient-Based Optimization119

7.2.1 Steepest-Descent Method120

7.2.2 Newton-Raphson Method120

7.3 Steepest-Descent-Based Algorithms121

7.2.3 Local vs.Global Minima121

7.3.1 Netravali-Robbins Algorithm122

7.3.2 Walker-Rao Algorithm123

7.3.3 Extension to the Block Motion Model124

7.4 Wiener-Estimation-Based Algorithms125

7.5 Examples127

7.6 Exercises129

8 BAYESIAN METHODS130

8.1 Optimization Methods130

8.1.1 Simulated Annealing131

8.1.2 Iterated Conditional Modes134

8.1.4 Highest Confidence First135

8.1.3 Mean Field Annealing135

8.2 Basics of MAP Motion Estimation136

8.2.1 The Likelihood Model137

8.2.2 The Prior Model137

8.3 MAP Motion Estimation Algorithms139

8.3.1 Formulation with Discontinuity Mode?139

8.3.2 Estimation with Local Outlier Rejection146

8.3.3 Estimation with Region Labeling147

8.4 Examples148

8.5 Exercises150

9 METHODS USING POINT CORRESPONDENCES152

Ⅲ THREE-DIMENSIONAL MOTION ESTIMATION AND SEGMENTATION152

9.1 Modeling the Projected Displacement Field153

9.1.1 Orthographic Displacement Field Model153

9.1.2 Perspective Displacement Field Model154

9.2 Methods Based on the Orthographic Model155

9.2.1 Two-Step Iteration Method from Two Views155

9.2.2 An Improved Iterative Method157

9.3 Methods Based on the Perspective Model158

9.3.1 The Epipolar Constraint and Essential Parameters158

9.3.2 Estimation of the Essential Parameters159

9.3.3 Decomposition of the E-Matrix161

9.3.4 Algorithm164

9.4 The Case of 3-D Planar Surfaces165

9.4.1 The Pure Parameters165

9.4.2 Estimation of the Pure Parameters166

9.4.3 Estimation of the Motion and Structure Parameters166

9.5 Examples168

9.5.1 Numerical Simulations168

9.5.2 Experiments with Two Frames of Miss America173

9.6 Exercises175

10 OPTICAL FLOW AND DIRECT METHODS177

10.1 Modeling the Projected Velocity Field177

10.1.1 Orthographic Velocity Field Model178

10.1.2 Perspective Velocity Field Model178

10.1.3 Perspective Velocity vs.Displacement Models179

10.2 Focus of Expansion180

10.3 Algebraic Methods Using Optical Flow181

10.3.1 Uniqueness of the Solution182

10.3.2 Affine Flow182

10.3.3 Quadratic Flow183

10.3.4 Arbitrary Flow184

10.4 Optimization Methods Using Optical Flow186

10.5 Direct Methods187

10.5.1 Extension of Optical Flow-Based Methods187

10.5.2 Tsai-Huang Method188

10.6 Examples190

10.6.1 Numerical Simulations191

10.6.2 Experiments with Two Frames of Miss America194

10.7 Exercises196

11 MOTION SEGMENTATION198

11.1 Direct Methods200

11.1.1 Thresholding for Change Detection200

11.1.2 An Algorithm Using Mapping Parameters201

11.1.3 Estimation of Model Parameters203

11.2 Optical Flow Segmentation204

11.2.1 Modified Hough Transform Method205

11.2.2 Segmentation for Layered Video Representation206

11.2.3 Bayesian Segmentation207

11.3 Simultaneous Estimation and Segmentation209

11.3.1 Motion Field Model210

11.3.2 Problem Formulation210

11.3.3 The Algorithm212

11.3.4 Relationship to Other Algorithms213

11.4 Examples214

11.5 Exercises217

12 STEREO AND MOTION TRACKING219

12.1 Motion and Structure from Stereo219

12.1.1 Still-Frame Stereo Imaging220

12.1.2 3-D Feature Matching for Motion Estimation222

12.1.3 Stereo-Motion Fusion224

12.1.4 Extension to Multiple Motion227

12.2 Motion Tracking229

12.2.1 Basic Principles229

12.2.2 2-D Motion Tracking232

12.2.3 3-D Rigid Motion Tracking235

12.3 Examples239

12.4 Exercises241

Ⅳ VIDEO FILTERING245

13 MOTION COMPENSATED FILTERING245

13.1 Spatio-Temporal Fourier Spectrum246

13.1.1 Global Motion with Constant Velocity247

13.1.2 Global Motion with Acceleration249

13.2.1 Sampling in the Temporal Direction Only250

13.2 Sub-Nyquist Spatio-Temporal Sampling250

13.2.2 Sampling on a Spatio-Temporal Lattice251

13.2.3 Critical Velocities252

13.3 Filtering Along Motion Trajectories254

13.3.1 Arbitrary Motion Trajectories255

13.3.2 Global Motion with Constant Velocity256

13.3.3 Accelerated Motion256

13.4.1 Motion-Compensated Noise Filtering258

13.4.2 Motion-Compensated Reconstruction Filtering258

13.4 Applications258

13.5 Exercises260

14 NOISE FILTERING262

14.1 Intraframe Filtering263

14.1.1 LMMSE Filtering264

14.1.2 Adaptive(Local)LMMSE Filtering267

14.1.3 Directional Filtering269

14.1.4 Median and Weighted Median Filtering270

14.2 Motion-Adaptive Filtering270

14.2.1 Direct Filtering271

14.2.2 Motion-Detection Based Filtering272

14.3 Motion-Compensated Filtering272

14.3.1 Spatio-Temporal Adaptive LMMSE Filtering274

14.3.2 Adaptive Weighted Averaging Filter275

14.4 Examples277

14.5 Exercises277

15 RESTORATION283

15.1 Modeling283

15.1.1 Shift-Invariant Spatial Blurring284

15.1.2 Shift-Varying Spatial Blurring285

15.2 Intraframe Shift-Invariant Restoration286

15.2.1 Pseudo Inverse Filtering286

15.2.2 Constrained Least Squares and Wiener Filtering287

15.3 Intraframe Shift-Varying Restoration289

15.3.1 Overview of the POCS Method290

15.3.2 Restoration Using POCS291

15.4 Multiframe Pestoration292

15.4.1 Cross-Correlated Multiframe Filter294

15.4.2 Motion-Compensated Multiframe Filter295

15.5 Examples295

15.6 Exercises296

16 STANDARDS CONVERSION302

16.1 Down-Conversion304

16.1.1 Down-Conversion with Anti-Alias Filtering305

16.1.2 Down-Conversion without Anti-Alias Filtering305

16.2 Practical Up-Conversion Methods308

16.2.1 Intraframe Filtering309

16.2.2 Motion-Adaptive Filtering314

16.3 Motion-Compensated Up-Conversion317

16.3.1 Basic Principles317

16.3.2 Global-Motion-Compensated De-interlacing322

16.4 Examples323

16.5 Exercises329

17 SUPERRESOLUTION331

17.1 Modeling332

17.1.1 Continuous-Discrete Model332

17.1.2 Discrete-Discrete Model335

17.2 Interpolation-Restoration Methods336

17.1.3 Problem Interrelations336

17.2.1 Intraframe Methods337

17.2.2 Multiframe Methods337

17.3 A Frequency Domain Method338

17.4 A Unifying POCS Method341

17.5 Examples343

17.6 Exercises346

Ⅴ STILL IMAGE COMPRESSION348

18 LOSSLESS COMPRESSION348

18.1 Basics of Image Compression349

18.1.1 Elements of an Image Compression System349

18.1.2 Information Theoretic Concepts350

18.2.1 Fixed-Length Coding353

18.2 Symbol Coding353

18.2.2 Huffman Coding354

18.2.3 Arithmetic Coding357

18.3 Lossless Compression Methods360

18.3.1 Lossless Predictive Coding360

18.3.2 Run-Length Coding of Bit-Planes363

18.3.3 Ziv-Lempel Coding364

18.4 Exercises366

19 DPCM AND TRANSFORM CODING368

19.1 Quantization368

19.1.1 Nonuniform Quantization369

19.1.2 Uniform Quantization370

19.2 Differential Pulse Code Modulation373

19.2.1 Optimal Prediction374

19.2.2 Quantization of the Prediction Error375

19.2.3 Adaptive Quantization376

19.2.4 Delta Modulation377

19.3 Transform Coding378

19.3.1 Discrete Cosine Transform380

19.3.2 Quantization/Bit Allocation381

19.3.3 Coding383

19.3.4 Blocking Artifacts in Transform Coding385

19.4 Exercises385

20 STILL IMAGE COMPRESSION STANDARDS388

20.1 Bilevel Image Compression Standards389

20.1.1 One-Dimensional RLC389

20.1.2 Two-Dimensional RLC391

20.1.3 The JBIG Standard393

20.2 The JPEG Standard394

20.2.1 Baseline Algorithm395

20.2.2 JPEG Progressive400

20.2.3 JPEG Lossless401

20.2.4 JPEG Hierarchical401

20.2.5 Implementations of JPEG402

20.3 Exercises403

21 VECTOR QUANTIZATION,SUBBAND CODING AND OTHER METHODS404

21.1 Vector Quantization404

21.1.1 Structure of a Vector Quantizer405

21.1.2 VQ Codebook Design408

21.1.3 Practical VQ Implementations408

21.2 Fractal Compression409

21.3 Subband Coding411

21.3.1 Subband Decomposition411

21.3.2 Coding of the Subbands414

21.3.3 Relationship to Transform Coding414

21.4 Second-Generation Coding Methods415

21.3.4 Relationship to Wavelet Transform Coding415

21.5 Exercises416

Ⅵ VIDEO COMPRESSION419

22 INTERFRAME COMPRESSION METHODS419

22.1 Three-Dimensional Waveform Coding420

22.1.1 3-D Transform Coding420

22.1.2 3-D Subbband Coding421

22.2 Motion-Compensated Waveform Coding424

22.2.1 MC Transform Coding424

22.2.2 MC Vector Quantization425

22.3 Model-Based Coding426

22.2.3 MC Subband Coding426

22.3.1 Object-Based Coding427

22.3.2 Knowledge-Based and Semantic Coding428

22.4 Exercises429

23 VIDEO COMPRESSION STANDARDS432

23.1 The H.261 Standard432

23.1.1 Input Image Formats433

23.1.2 Video Multiplex434

23.1.3 Video Compression Algorithm435

23.2 The MPEG-1 Standard440

23.2.1 Features440

23.2.2 Input Video Format441

23.2.3 Data Structure and Compression Modes441

23.2.4 Intraframe Compression Mode443

23.2.5 Interframe Compression Modes444

23.2.6 MPEG-1 Encoder and Decder447

23.3 The MPEG-2 Standard448

23.3.1 MPEG-2 Macroblocks449

23.3.2 Coding Interlaced Video450

23.3.3 Scalable Extensions452

23.3.4 Other Improvements453

23.3.5 Overview of Profiles and Levels454

23.4 Software and Hardware Implementations455

24 MODEL-BASED CODING457

24.1 General Object-Based Methods457

24.1.1 2-D/3-D Rigid Objects with 3-D Motion458

24.1.2 2-D Flexible Objects with 2-D Motion460

24.1.3 Affine Transformations with Triangular Meshes462

24.2 Knowledge-Based and Semantic Methods464

24.2.1 General Principles465

24.2.2 MBASIC Algorithm470

24.2.3 Estimation Using a Flexible Wireframe Model471

24.3 Examples478

25 DIGITAL VIDEO SYSTEMS486

25.1 Videoconferencing487

25.2 Interactive Video and Multimedia488

25.3 Digital Television489

25.3.1 Digital Studio Standards490

25.3.2 Hybrid Advanced TV Systems491

25.3.3 All-Digital TV493

25.4 Low-Bitrate Video and Videophone497

25.4.1 The ITU Recommendation H.263498

25.4.2 The ISO MPEG-4 Requirements499

APPENDICES502

A MARKOV AND GIBBS RANDOM FIELDS502

A.1 Definitions502

A.1.1 Markov Random Fields503

A.1.2 Gibbs Random Fields504

A.2 Equivalence of MRF and GRF505

A.3 Local Conditional Probabilities506

B BASICS OF SEGMENTATION508

B.1 Thresholding508

B.1.1 Finding the Optimum Threshold(s)509

B.2 Clustering510

B.3 Bayesian Methods512

B.3.1 The MAP Method513

B.3.2 The Adaptive MAP Method515

B.3.3 Vector Field Segmentation516

C KALMAN FILTERING518

C.1 Linear State-Space Model518

C.2 Extended Kalman Filtering520