Smart grid using big data analytics : a random matrix theory approach /
Robert C. Qiu and Paul Antonik.
- 1 online resource (632 pages).
ABOUT THE AUTHOR
Robert Caiming Qiu, Professor, Dept. of ECE, Tennessee Technological University, Cookeville, TN, USA. Professor Qiu was Founder-CEO and President of Wiscom Technologies, Inc., manufacturing and marketing WCDMA chipsets. Wiscom was acquired by Intel in 2003. Prior to Wiscom, he worked for GTE Labs, Inc. (now Verizon), Waltham, MA, and Bell Labs, Lucent, Whippany, NJ. He holds 5 U.S. patents (another two pending) in WCDMA. Professor Qiu has contributed to 3GPP and IEEE standards bodies, and delivered invited seminars to institutions including Princeton University and the U.S. Army Research Lab. Dr. Qiu was made an IEEE Fellow in 2014.
Dr. Paul Antonik, Chief Scientist, Information Directorate, Air Force Research Laboratory, Rome, N.Y., USA. Dr. Antonik serves as the directorate's principal scientific and technical adviser and primary authority for the technical content of the science and technology portfolio, providing principal technical oversight of a broad spectrum of information technologies.
Includes bibliographical references (pages 567-599) and index.
Preface xv
Acknowledgments xix
Some Notation xxi
1 Introduction 1
1.1 Big Data: Basic Concepts 1
1.2 Data Mining with Big Data 9
1.3 A Mathematical Introduction to Big Data 13
1.4 A Mathematical Theory of Big Data 28
1.5 Smart Grid 34
1.6 Big Data and Smart Grid 36
1.7 Reading Guide 37
Bibliographical Remarks 39
Part I Fundamentals of Big Data 41
2 The Mathematical Foundations of Big Data Systems 43
2.1 Big Data Analytics 44
2.2 Big Data: Sense, Collect, Store, and Analyze 45
2.3 Intelligent Algorithms 48
2.4 Signal Processing for Smart Grid 48
2.5 Monitoring and Optimization for Power Grids 48
2.6 Distributed Sensing and Measurement for Power Grids 49
2.7 Real-time Analysis of Streaming Data 50
2.8 Salient Features of Big Data 51
2.9 Big Data for Quantum Systems 54
2.10 Big Data for Financial Systems 55
2.11 Big Data for Atmospheric Systems 73
2.12 Big Data for Sensing Networks 74
2.13 Big Data forWireless Networks 75
2.14 Big Data for Transportation 78
Bibliographical Remarks 78
3 Large Random Matrices: An Introduction 79
3.1 Modeling of Large Dimensional Data as Random Matrices 79
3.2 A Brief of Random MatrixTheory 81
3.3 Change Point of Views: From Vectors to Measures 85
3.4 The Stieltjes Transform of Measures 86
3.5 A Fundamental Result: The Marchenko–Pastur Equation 88
3.6 Linear Eigenvalue Statistics and Limit Laws 89
3.7 Central LimitTheorem for Linear Eigenvalue Statistics 99
3.8 Central LimitTheorem for Random Matrix S−1T 101
3.9 Independence for Random Matrices 103
3.10 Matrix-Valued Gaussian Distribution 110
3.11 Matrix-ValuedWishart Distribution 112
3.12 Moment Method 112
3.13 Stieltjes Transform Method 113
3.14 Concentration of the Spectral Measure for Large Random Matrices 114
3.15 Future Directions 117
Bibliographical Remarks 117
4 Linear Spectral Statistics of the Sample Covariance Matrix 121
8.13 Hypothesis Testing for Matrix Elliptically Contoured Distributions 446
Bibliographical Remarks 452
Part II Smart Grid 455
9 Applications and Requirements of Smart Grid 457
9.1 History 457
9.2 Concepts and Vision 458
9.3 Today’s Electric Grid 459
9.4 Future Smart Electrical Energy System 464
10 Technical Challenges for Smart Grid 471
Bibliographical Remarks 483
11 Big Data for Smart Grid 485
11.1 Power in Numbers: Big Data and Grid Infrastructure 485
11.2 Energy’s Internet:The Convergence of Big Data and the Cloud 486
11.3 Edge Analytics: Consumers, Electric Vehicles, and Distributed Generation 486
11.4 CrosscuttingThemes: Big Data 486
11.5 Cloud Computing for Smart Grid 488
11.6 Data Storage, Data Access and Data Analysis 488
11.7 The State-of-the-Art Processing Techniques of Big Data 488
11.8 Big Data Meets the Smart Electrical Grid 488
11.9 4Vs of Big Data: Volume, Variety, Value and Velocity 489
11.10 Cloud Computing for Big Data 490
11.11 Big Data for Smart Grid 490
11.12 Information Platforms for Smart Grid 491
Bibliographical Remarks 491
12 Grid Monitoring and State Estimation 493
12.1 Phase Measurement Unit 493
12.2 Optimal PMU Placement 495
12.3 State Estimation 495
12.4 Basics of State Estimation 495
12.5 Evolution of State Estimation 496
12.6 Static State Estimation 497
12.7 Forecasting-Aided State Estimation 500
12.8 Phasor Measurement Units 501
12.9 Distributed System State Estimation 502
12.10 Event-Triggered Approaches to State Estimation 502
12.11 Bad Data Detection 502
12.12 Improved Bad Data Detection 504
12.13 Cyber-Attacks 504
12.14 Line Outage Detection 504
Bibliographical Remarks 504
13 False Data Injection Attacks against State Estimation 505
13.1 State Estimation 505
13.2 False Data Injection Attacks 507
13.3 MMSE State Estimation and Generalized Likelihood Ratio Test 508
13.4 Sparse Recovery from Nonlinear Measurements 512
13.5 Real-Time Intrusion Detection 515
Bibliographical Remarks 515
14 Demand Response 517
14.1 Why Engage Demand? 517
14.2 Optimal Real-time Pricing Algorithms 520
14.3 Transportation Electrification and Vehicle-to-Grid Applications 522
14.4 Grid Storage 522
Bibliographical Remarks 523
Part III Communications and Sensing 525
15 Big Data for Communications 527
15.1 5G and Big Data 527
15.2 5GWireless Communication Networks 527
15.3 Massive Multiple Input, Multiple Output 528
15.4 Free Probability for the Capacity of the Massive MIMO Channel 537
15.5 Spectral Sensing for Cognitive Radio 539
Bibliographical Remarks 539
16 Big Data for Sensing 541
16.1 Distributed Detection and Estimation 541
16.2 Euclidean Random Matrix 547
16.3 Decentralized Computing 548
Appendix A: Some Basic Results on Free Probability 551
Appendix B: Matrix-Valued Random Variables 557
References 567
Index 601
This book is aimed at students in communications and signal processing who want to extend their skills in the energy area. It describes power systems and why these backgrounds are so useful to smart grid, wireless communications being very different to traditional wireline communications.