Experimental methods in survey research : techniques that combine random sampling with random assignment / edited by Paul J. Lavrakas, [and five others].

Contributor(s): Lavrakas, Paul J [editor.]
Language: English Series: Wiley series in survey methodologyPublisher: Hoboken, NJ : Wiley, 2019Description: 1 online resourceContent type: text Media type: computer Carrier type: online resourceISBN: 9781119083757; 9781119083771; 9781119083764Subject(s): Social surveys | Social sciences -- Research -- Methodology | Surveys -- EvaluationGenre/Form: Electronic books.DDC classification: 300.72/3 LOC classification: HM538Other classification: SOC027000 Online resources: Full text available at Wiley Online Library Click here to view
Incomplete contents:
TABLE OF CONTENTS List of Contributors xix Preface by Dr. Judith Tanur xxv About the Companion Website xxix 1 Probability Survey-Based Experimentation and the Balancing of Internal and External Validity Concerns 1 Paul J. Lavrakas, Courtney Kennedy, Edith D. de Leeuw, Brady T. West, Allyson L. Holbrook, and Michael W. Traugott 1.1 Validity Concerns in Survey Research 3 1.2 Survey Validity and Survey Error 5 1.3 Internal Validity 6 1.4 Threats to Internal Validity 8 1.5 External Validity 11 1.6 Pairing Experimental Designs with Probability Sampling 12 1.7 Some Thoughts on Conducting Experiments with Online Convenience Samples 12 1.8 The Contents of this Book 15 References 15 Part I Introduction to Section on Within-Unit Coverage 19 Paul J. Lavrakas and Edith D. de Leeuw 2 Within-Household Selection Methods: A Critical Review and Experimental Examination 23 Jolene D. Smyth, Kristen Olson, and Mathew Stange 2.1 Introduction 23 2.2 Within-Household Selection and Total Survey Error 24 2.3 Types of within-Household Selection Techniques 24 2.4 Within-Household Selection in Telephone Surveys 25 2.5 Within-Household Selection in Self-Administered Surveys 26 2.6 Methodological Requirements of Experimentally Studying Within-Household Selection Methods 27 2.7 Empirical Example 30 2.8 Data and Methods 31 2.9 Analysis Plan 34 2.10 Results 35 2.11 Discussion and Conclusions 40 References 42 3 Measuring within-Household Contamination: The Challenge of Interviewing More Than One Member of a Household 47 Colm O’Muircheartaigh, Stephen Smith, and Jaclyn S.Wong 3.1 Literature Review 47 3.2 Data and Methods 50 Investigators 53 Field/Project Directors 53 3.3 The Sequence of Analyses 55 3.4 Results 55 3.5 Effect on Standard Errors of the Estimates 57 3.6 Effect on Response Rates 58 3.7 Effect on Responses 61 3.8 Substantive Results 64 References 64 Part II Survey Experiments with Techniques to Reduce Nonresponse 67 Edith D. de Leeuw and Paul J. Lavrakas 4 Survey Experiments on Interactions and Nonresponse: A Case Study of Incentives and Modes 69 A. Bianchi and S. Biffignandi 4.1 Introduction 69 4.2 Literature Overview 70 4.3 Case Study: Examining the Interaction between Incentives and Mode 73 4.4 Concluding Remarks 83 Acknowledgments 85 References 86 5 Experiments on the Effects of Advance Letters in Surveys 89 Susanne Vogl, Jennifer A. Parsons, Linda K. Owens, and Paul J. Lavrakas 5.1 Introduction 89 5.2 State of the Art on Experimentation on the Effect of Advance Letters 93 5.3 Case Studies: Experimental Research on the Effect of Advance Letters 95 5.4 Case Study I: Violence against Men in Intimate Relationships 96 5.5 Case Study II: The Neighborhood Crime and Justice Study 100 5.6 Discussion 106 5.7 Research Agenda for the Future 107 References 108 Part III Overview of the Section on the Questionnaire 111 Allyson Holbrook and Michael W. Traugott 6 Experiments on the Design and Evaluation of Complex Survey Questions 113 Paul Beatty, Carol Cosenza, and Floyd J. Fowler Jr. 6.1 Question Construction: Dangling Qualifiers 115 6.2 Overall Meanings of Question Can Be Obscured by Detailed Words 117 6.3 Are Two Questions Better than One? 119 6.4 The Use of Multiple Questions to Simplify Response Judgments 121 6.5 The Effect of Context or Framing on Answers 122 6.6 Do Questionnaire Effects Vary Across Sub-groups of Respondents? 124 6.7 Discussion 126 References 128 7 Impact of Response Scale Features on Survey Responses to Behavioral Questions 131 Florian Keusch and Ting Yan 7.1 Introduction 131 7.2 Previous Work on Scale Design Features 132 7.3 Methods 134 7.4 Results 136 7.5 Discussion 141 Acknowledgment 143 7.A Question Wording 143 7.A.1 Experimental Questions (One Question Per Screen) 143 7.A.2 Validation Questions (One Per Screen) 144 7.A.3 GfK Profile Questions (Not Part of the Questionnaire) 145 7.B Test of Interaction Effects 145 References 146 8 Mode Effects Versus Question Format Effects: An Experimental Investigation of Measurement Error Implemented in a Probability-Based Online Panel 151 Edith D. de Leeuw, Joop Hox, and Annette Scherpenzeel 8.1 Introduction 151 8.2 Experiments and Probability-Based Online Panels 153 8.3 Mixed-Mode Question Format Experiments 154 8.4 Summary and Discussion 161 Acknowledgments 162 References 162 9 Conflicting Cues: Item Nonresponse and Experimental Mortality 167 David J. Ciuk and Berwood A. Yost 9.1 Introduction 167 9.2 Survey Experiments and Item Nonresponse 167 9.3 Case Study: Conflicting Cues and Item Nonresponse 170 9.4 Methods 170 9.5 Issue Selection 171 9.6 Experimental Conditions and Measures 172 9.7 Results 173 9.8 Addressing Item Nonresponse in Survey Experiments 174 9.9 Summary 178 References 179 10 Application of a List Experiment at the Population Level: The Case of Opposition to Immigration in the Netherlands 181 Mathew J. Creighton, Philip S. Brenner, Peter Schmidt, and Diana Zavala-Rojas 10.1 Fielding the Item Count Technique (ICT) 183 10.2 Analyzing the Item Count Technique (ICT) 185 10.3 An Application of ICT: Attitudes toward Immigrants in the Netherlands 186 10.4 Limitations of ICT 190 References 192 Part IV Introduction to Section on Interviewers 195 Brady T. West and Edith D. de Leeuw 11 Race- and Ethnicity-of-Interviewer Effects 197 Allyson L. Holbrook, Timothy P. Johnson, and Maria Krysan 11.1 Introduction 197 11.2 The Current Research 205 11.3 Respondents and Procedures 207 11.4 Measures 207 11.5 Analysis 210 11.6 Results 211 11.7 Discussion and Conclusion 219 References 221 12 Investigating Interviewer Effects and Confounds in Survey-Based Experimentation 225 Paul J. Lavrakas, Jenny Kelly, and Colleen McClain 12.1 Studying Interviewer Effects Using a Post hoc Experimental Design 226 12.2 Studying Interviewer Effects Using A Priori Experimental Designs 230 12.3 An Original Experiment on the Effects of Interviewers Administering Only One Treatment vs. Interviewers Administrating Multiple Treatments 232 12.4 Discussion 239 References 242 Part V Introduction to Section on Adaptive Design 245 Courtney Kennedy and Brady T. West 13 Using Experiments to Assess Interactive Feedback That Improves Response Quality in Web Surveys 247 Tanja Kunz and Marek Fuchs 13.1 Introduction 247 13.2 Case Studies – Interactive Feedback in Web Surveys 251 13.3 Methodological Issues in Experimental Visual Design Studies 258 References 269 14 Randomized Experiments for Web-Mail Surveys Conducted Using Address-Based Samples of the General Population 275 Z. Tuba Suzer-Gurtekin, Mahmoud Elkasabi, James M. Lepkowski, Mingnan Liu, and Richard Curtin 14.1 Introduction 275 14.2 Study Design and Methods 278 14.3 Results 281 14.4 Discussion 285 References 287 Part VI Introduction to Section on Special Surveys 291 Michael W. Traugott and Edith D. de Leeuw 15 Mounting Multiple Experiments on Longitudinal Social Surveys: Design and Implementation Considerations 293 Peter Lynn and Annette Jäckle 15.1 Introduction and Overview 293 15.2 Types of Experiments that Can Be Mounted in a Longitudinal Survey 294 15.3 Longitudinal Experiments and Experiments in Longitudinal Surveys 295 15.4 Longitudinal Surveys that Serve as Platforms for Experimentation 296 15.5 The Understanding Society Innovation Panel 298 15.6 Avoiding Confounding of Experiments 299 15.7 Allocation Procedures 301 15.8 Refreshment Samples 304 15.9 Discussion 305 15.A Appendix: Stata Syntax to Produce Table 15.3 Treatment Allocations 306 References 306 16 Obstacles and Opportunities for Experiments in Establishment Surveys Supporting Official Statistics 309 Diane K. Willimack and Jaki S. McCarthy 16.1 Introduction 309 16.2 Some Key Differences between Household and Establishment Surveys 310 16.3 Existing Literature Featuring Establishment Survey Experiments 312 16.4 Key Considerations for Experimentation in Establishment Surveys 314 16.5 Examples of Experimentation in Establishment Surveys 318 16.6 Discussion and Concluding Remarks 323 Acknowledgments 324 References 324 Part VII Introduction to Section on Trend Data 327 Michael W. Traugott and Paul J. Lavrakas 17 Tracking Question-Wording Experiments across Time in the General Social Survey, 1984–2014 329 Tom W. Smith and Jaesok Son 17.1 Introduction 329 17.2 GSS Question-Wording Experiment on Spending Priorities 330 17.3 Experimental Analysis 330 17.4 Summary and Conclusion 338 17.A National Spending Priority Items 339 References 340 18 Survey Experiments and Changes in Question Wording in Repeated Cross-Sectional Surveys 343 Allyson L. Holbrook, David Sterrett, Andrew W. Crosby, Marina Stavrakantonaki, Xiaoheng Wang, Tianshu Zhao, and Timothy P. Johnson 18.1 Introduction 343 18.2 Background 344 18.3 Two Case Studies 347 18.4 Implications and Conclusions 362 Acknowledgments 364 References 364 Part VIII Vignette Experiments in Surveys 369 Allyson Holbrook and Paul J. Lavrakas 19 Are Factorial Survey Experiments Prone to Survey Mode Effects? 371 Katrin Auspurg, Thomas Hinz, and Sandra Walzenbach 19.1 Introduction 371 19.2 Idea and Scope of Factorial Survey Experiments 372 19.3 Mode Effects 373 19.4 Case Study 378 19.5 Conclusion 388 References 390 20 Validity Aspects of Vignette Experiments: Expected “What-If” Differences between Reports of Behavioral Intentions and Actual Behavior 393 Stefanie Eifler and Knut Petzold 20.1 Outline of the Problem 393 20.2 Research Findings from Our Experimental Work 399 20.3 Discussion 411 References 413 Part IX Introduction to Section on Analysis 417 Brady T. West and Courtney Kennedy 21 Identities and Intersectionality: A Case for Purposive Sampling in Survey-Experimental Research 419 Samara Klar and Thomas J. Leeper 21.1 Introduction 419 21.2 Common Techniques for Survey Experiments on Identity 420 21.3 How Limited are Representative Samples for Intersectionality Research? 426 21.4 Conclusions and Discussion 430 Author Biographies 431 References 431 22 Designing Probability Samples to Study Treatment Effect Heterogeneity 435 Elizabeth Tipton, David S. Yeager, Ronaldo Iachan, and Barbara Schneider 22.1 Introduction 435 22.2 Nesting a Randomized Treatment in a National Probability Sample: The NSLM 446 22.3 Discussion and Conclusions 451 Acknowledgments 453 References 453 23 Design-Based Analysis of Experiments Embedded in Probability Samples 457 Jan A. van den Brakel 23.1 Introduction 457 23.2 Design of Embedded Experiments 458 23.3 Design-Based Inference for Embedded Experiments with One Treatment Factor 460 23.4 Analysis of Experiments with Clusters of Sampling Units as Experimental Units 466 23.5 Factorial Designs 468 23.6 A Mixed-Mode Experiment in the Dutch Crime Victimization Survey 472 23.7 Discussion 477 Acknowledgments 478 References 478 24 Extending the Within-Persons Experimental Design: The Multitrait-Multierror (MTME) Approach 481 Alexandru Cernat and Daniel L. Oberski 24.1 Introduction 481 24.2 The Multitrait-Multierror (MTME) Framework 482 24.3 Designing the MTME Experiment 487 24.4 Statistical Estimation for the MTME Approach 489 24.5 Measurement Error in Attitudes toward Migrants in the UK 491 24.6 Results 494 24.7 Conclusions and Future Research Directions 497 Acknowledgments 498 References 498 Index 501 DOWNLOADS Chapter 5 Download Chapter 8 Download Chapter 15 Download Chapter 16 Download Chapter 19 Download Chapter 20 Download Chapter 23
Summary: "A thorough and comprehensive guide to the theoretical, practical, and methodological approaches used in survey experiments across disciplines such as political science, health sciences, sociology, economics, psychology, and marketing. This book bridges the gap in literature and explores the broad range of experimental designs in experiments that use probability samples. It approaches the usage of survey-based experiments with a Total Survey Error (TSE) perspective, which provides insight on the strengths and weaknesses of the techniques used. Experimental Methods in Survey Research: Techniques that Combine Random Assignment with Random Probability Sampling addresses experiments on within-unit coverage, reducing nonresponse, question and questionnaire design, minimizing interview measurement bias, using adaptive design, trend data, vignettes, the analysis of data from survey experiments, and other topics, across social, behavioral, and marketing science domains. Each chapter begins with a description of the experimental method or application and its importance, followed by reference to relevant literature. At least one detailed original experimental case study then follows to illustrate the experimental method's deployment, implementation, and analysis from a TSE perspective. The chapters conclude with theoretical and practical implications on the usage of the experimental method addressed. In summary, this book fills a gap in the current literature by successfully combining the subjects of survey methodology and experimental methodology in an effort to maximize both internal validity and external validity. Offers a wide range of types of experimentation in survey research with in-depth attention to their various methodologies and applications. Is edited by internationally recognized experts in the field of survey research/methodology and in the usage of survey-based experimentation-featuring contributions from across a variety of disciplines in the social and behavioral sciences. Presents advances in the field of survey experiments, as well as relevant references in each chapter for further study Includes more than 20 types of original experiments carried out within probability sample surveys. Addresses myriad practical and operational aspects for designing, implementing, and analyzing survey-based experiments by using a Total Survey Error perspective to address the strengths and weaknesses of each experimental technique and method. Experimental Methods in Survey Research: Techniques that Combine Random Assignment with Random Probability Sampling is an ideal reference for survey researchers and practitioners in areas such political science, health sciences, sociology, economics, psychology, public policy, data collection, data science, and marketing. It is also a very useful textbook for graduate-level courses on survey experiments and survey methodology"-- Provided by publisher.Summary: "With a recent increase of experiments in survey research and methodology, survey researchers have found a need to bring together the many types of experiments used by survey researchers and their applications throughout various disciplines. This book bridges the gap in literature and explores the broad range of experimental design in survey-based experiments with a combined methodological and substantive focus on the ways the experiments are conducted and used in survey research. The book is edited by a group of established experts in the field of survey research/methodology and the chapters feature expert contributors across a variety of disciplines in the social and behavioral sciences. The book approaches coverage on survey-based experiments with a Total Survey Error (TSE) perspective, which provides insight on the strengths and weaknesses of the technique. Using the same basic structure, each chapter starts with a description of the experimental method or application, followed by a reference to any relevant reviews in current literature"-- Provided by publisher.
Tags from this library: No tags from this library for this title. Log in to add tags.
    Average rating: 0.0 (0 votes)
Item type Current location Home library Call number Status Date due Barcode Item holds
EBOOK EBOOK COLLEGE LIBRARY
COLLEGE LIBRARY
300.723 Ex715 2019 (Browse shelf) Available CL-51029
Total holds: 0

ABOUT THE AUTHORS
Paul J. Lavrakas, PhD, is Senior Fellow at the NORC at the University of Chicago, Adjunct Professor at University of Illinois-Chicago, Senior Methodologist at the Social Research Centre of Australian National University and at the Office for Survey Research at Michigan State University.

Michael W. Traugott, PhD, is Research Professor in the Institute for Social Research at the University of Michigan.

Courtney Kennedy, PhD, is Director of Survey Research at Pew Research Center in Washington, DC.

Allyson L. Holbrook, PhD, is Professor of Public Administration and Psychology at the University of Illinois-Chicago.

Edith D. de Leeuw, PhD, is Professor of Survey Methodology in the Department of Methodology and Statistics at Utrecht University.

Brady T. West, PhD, is Research Associate Professor in the Survey Research Center at the University of Michigan-Ann Arbor.

Includes bibliographical references and index.

TABLE OF CONTENTS
List of Contributors xix

Preface by Dr. Judith Tanur xxv

About the Companion Website xxix

1 Probability Survey-Based Experimentation and the Balancing of Internal and External Validity Concerns 1
Paul J. Lavrakas, Courtney Kennedy, Edith D. de Leeuw, Brady T. West, Allyson L. Holbrook, and Michael W. Traugott

1.1 Validity Concerns in Survey Research 3

1.2 Survey Validity and Survey Error 5

1.3 Internal Validity 6

1.4 Threats to Internal Validity 8

1.5 External Validity 11

1.6 Pairing Experimental Designs with Probability Sampling 12

1.7 Some Thoughts on Conducting Experiments with Online Convenience Samples 12

1.8 The Contents of this Book 15

References 15

Part I Introduction to Section on Within-Unit Coverage 19
Paul J. Lavrakas and Edith D. de Leeuw

2 Within-Household Selection Methods: A Critical Review and Experimental Examination 23
Jolene D. Smyth, Kristen Olson, and Mathew Stange

2.1 Introduction 23

2.2 Within-Household Selection and Total Survey Error 24

2.3 Types of within-Household Selection Techniques 24

2.4 Within-Household Selection in Telephone Surveys 25

2.5 Within-Household Selection in Self-Administered Surveys 26

2.6 Methodological Requirements of Experimentally Studying Within-Household Selection Methods 27

2.7 Empirical Example 30

2.8 Data and Methods 31

2.9 Analysis Plan 34

2.10 Results 35

2.11 Discussion and Conclusions 40

References 42

3 Measuring within-Household Contamination: The Challenge of Interviewing More Than One Member of a Household 47
Colm O’Muircheartaigh, Stephen Smith, and Jaclyn S.Wong

3.1 Literature Review 47

3.2 Data and Methods 50

Investigators 53

Field/Project Directors 53

3.3 The Sequence of Analyses 55

3.4 Results 55

3.5 Effect on Standard Errors of the Estimates 57

3.6 Effect on Response Rates 58

3.7 Effect on Responses 61

3.8 Substantive Results 64

References 64

Part II Survey Experiments with Techniques to Reduce Nonresponse 67
Edith D. de Leeuw and Paul J. Lavrakas

4 Survey Experiments on Interactions and Nonresponse: A Case Study of Incentives and Modes 69
A. Bianchi and S. Biffignandi

4.1 Introduction 69

4.2 Literature Overview 70

4.3 Case Study: Examining the Interaction between Incentives and Mode 73

4.4 Concluding Remarks 83

Acknowledgments 85

References 86

5 Experiments on the Effects of Advance Letters in Surveys 89
Susanne Vogl, Jennifer A. Parsons, Linda K. Owens, and Paul J. Lavrakas

5.1 Introduction 89

5.2 State of the Art on Experimentation on the Effect of Advance Letters 93

5.3 Case Studies: Experimental Research on the Effect of Advance Letters 95

5.4 Case Study I: Violence against Men in Intimate Relationships 96

5.5 Case Study II: The Neighborhood Crime and Justice Study 100

5.6 Discussion 106

5.7 Research Agenda for the Future 107

References 108

Part III Overview of the Section on the Questionnaire 111
Allyson Holbrook and Michael W. Traugott

6 Experiments on the Design and Evaluation of Complex Survey Questions 113
Paul Beatty, Carol Cosenza, and Floyd J. Fowler Jr.

6.1 Question Construction: Dangling Qualifiers 115

6.2 Overall Meanings of Question Can Be Obscured by Detailed Words 117

6.3 Are Two Questions Better than One? 119

6.4 The Use of Multiple Questions to Simplify Response Judgments 121

6.5 The Effect of Context or Framing on Answers 122

6.6 Do Questionnaire Effects Vary Across Sub-groups of Respondents? 124

6.7 Discussion 126

References 128

7 Impact of Response Scale Features on Survey Responses to Behavioral Questions 131
Florian Keusch and Ting Yan

7.1 Introduction 131

7.2 Previous Work on Scale Design Features 132

7.3 Methods 134

7.4 Results 136

7.5 Discussion 141

Acknowledgment 143

7.A Question Wording 143

7.A.1 Experimental Questions (One Question Per Screen) 143

7.A.2 Validation Questions (One Per Screen) 144

7.A.3 GfK Profile Questions (Not Part of the Questionnaire) 145

7.B Test of Interaction Effects 145

References 146

8 Mode Effects Versus Question Format Effects: An Experimental Investigation of Measurement Error Implemented in a Probability-Based Online Panel 151
Edith D. de Leeuw, Joop Hox, and Annette Scherpenzeel

8.1 Introduction 151

8.2 Experiments and Probability-Based Online Panels 153

8.3 Mixed-Mode Question Format Experiments 154

8.4 Summary and Discussion 161

Acknowledgments 162

References 162

9 Conflicting Cues: Item Nonresponse and Experimental Mortality 167
David J. Ciuk and Berwood A. Yost

9.1 Introduction 167

9.2 Survey Experiments and Item Nonresponse 167

9.3 Case Study: Conflicting Cues and Item Nonresponse 170

9.4 Methods 170

9.5 Issue Selection 171

9.6 Experimental Conditions and Measures 172

9.7 Results 173

9.8 Addressing Item Nonresponse in Survey Experiments 174

9.9 Summary 178

References 179

10 Application of a List Experiment at the Population Level: The Case of Opposition to Immigration in the Netherlands 181
Mathew J. Creighton, Philip S. Brenner, Peter Schmidt, and Diana Zavala-Rojas

10.1 Fielding the Item Count Technique (ICT) 183

10.2 Analyzing the Item Count Technique (ICT) 185

10.3 An Application of ICT: Attitudes toward Immigrants in the Netherlands 186

10.4 Limitations of ICT 190

References 192

Part IV Introduction to Section on Interviewers 195
Brady T. West and Edith D. de Leeuw

11 Race- and Ethnicity-of-Interviewer Effects 197
Allyson L. Holbrook, Timothy P. Johnson, and Maria Krysan

11.1 Introduction 197

11.2 The Current Research 205

11.3 Respondents and Procedures 207

11.4 Measures 207

11.5 Analysis 210

11.6 Results 211

11.7 Discussion and Conclusion 219

References 221

12 Investigating Interviewer Effects and Confounds in Survey-Based Experimentation 225
Paul J. Lavrakas, Jenny Kelly, and Colleen McClain

12.1 Studying Interviewer Effects Using a Post hoc Experimental Design 226

12.2 Studying Interviewer Effects Using A Priori Experimental Designs 230

12.3 An Original Experiment on the Effects of Interviewers Administering Only One Treatment vs. Interviewers Administrating Multiple Treatments 232

12.4 Discussion 239

References 242

Part V Introduction to Section on Adaptive Design 245
Courtney Kennedy and Brady T. West

13 Using Experiments to Assess Interactive Feedback That Improves Response Quality in Web Surveys 247
Tanja Kunz and Marek Fuchs

13.1 Introduction 247

13.2 Case Studies – Interactive Feedback in Web Surveys 251

13.3 Methodological Issues in Experimental Visual Design Studies 258

References 269

14 Randomized Experiments for Web-Mail Surveys Conducted Using Address-Based Samples of the General Population 275
Z. Tuba Suzer-Gurtekin, Mahmoud Elkasabi, James M. Lepkowski, Mingnan Liu, and Richard Curtin

14.1 Introduction 275

14.2 Study Design and Methods 278

14.3 Results 281

14.4 Discussion 285

References 287

Part VI Introduction to Section on Special Surveys 291
Michael W. Traugott and Edith D. de Leeuw

15 Mounting Multiple Experiments on Longitudinal Social Surveys: Design and Implementation Considerations 293
Peter Lynn and Annette Jäckle

15.1 Introduction and Overview 293

15.2 Types of Experiments that Can Be Mounted in a Longitudinal Survey 294

15.3 Longitudinal Experiments and Experiments in Longitudinal Surveys 295

15.4 Longitudinal Surveys that Serve as Platforms for Experimentation 296

15.5 The Understanding Society Innovation Panel 298

15.6 Avoiding Confounding of Experiments 299

15.7 Allocation Procedures 301

15.8 Refreshment Samples 304

15.9 Discussion 305

15.A Appendix: Stata Syntax to Produce Table 15.3 Treatment Allocations 306

References 306

16 Obstacles and Opportunities for Experiments in Establishment Surveys Supporting Official Statistics 309
Diane K. Willimack and Jaki S. McCarthy

16.1 Introduction 309

16.2 Some Key Differences between Household and Establishment Surveys 310

16.3 Existing Literature Featuring Establishment Survey Experiments 312

16.4 Key Considerations for Experimentation in Establishment Surveys 314

16.5 Examples of Experimentation in Establishment Surveys 318

16.6 Discussion and Concluding Remarks 323

Acknowledgments 324

References 324

Part VII Introduction to Section on Trend Data 327
Michael W. Traugott and Paul J. Lavrakas

17 Tracking Question-Wording Experiments across Time in the General Social Survey, 1984–2014 329
Tom W. Smith and Jaesok Son

17.1 Introduction 329

17.2 GSS Question-Wording Experiment on Spending Priorities 330

17.3 Experimental Analysis 330

17.4 Summary and Conclusion 338

17.A National Spending Priority Items 339

References 340

18 Survey Experiments and Changes in Question Wording in Repeated Cross-Sectional Surveys 343
Allyson L. Holbrook, David Sterrett, Andrew W. Crosby, Marina Stavrakantonaki, Xiaoheng Wang, Tianshu Zhao, and Timothy P. Johnson


18.1 Introduction 343

18.2 Background 344

18.3 Two Case Studies 347

18.4 Implications and Conclusions 362

Acknowledgments 364

References 364

Part VIII Vignette Experiments in Surveys 369
Allyson Holbrook and Paul J. Lavrakas

19 Are Factorial Survey Experiments Prone to Survey Mode Effects? 371
Katrin Auspurg, Thomas Hinz, and Sandra Walzenbach

19.1 Introduction 371

19.2 Idea and Scope of Factorial Survey Experiments 372

19.3 Mode Effects 373

19.4 Case Study 378

19.5 Conclusion 388

References 390

20 Validity Aspects of Vignette Experiments: Expected “What-If” Differences between Reports of Behavioral Intentions and Actual Behavior 393
Stefanie Eifler and Knut Petzold

20.1 Outline of the Problem 393

20.2 Research Findings from Our Experimental Work 399

20.3 Discussion 411

References 413

Part IX Introduction to Section on Analysis 417
Brady T. West and Courtney Kennedy

21 Identities and Intersectionality: A Case for Purposive Sampling in Survey-Experimental Research 419
Samara Klar and Thomas J. Leeper

21.1 Introduction 419

21.2 Common Techniques for Survey Experiments on Identity 420

21.3 How Limited are Representative Samples for Intersectionality Research? 426

21.4 Conclusions and Discussion 430

Author Biographies 431

References 431

22 Designing Probability Samples to Study Treatment Effect Heterogeneity 435
Elizabeth Tipton, David S. Yeager, Ronaldo Iachan, and Barbara Schneider

22.1 Introduction 435

22.2 Nesting a Randomized Treatment in a National Probability Sample: The NSLM 446

22.3 Discussion and Conclusions 451

Acknowledgments 453

References 453

23 Design-Based Analysis of Experiments Embedded in Probability Samples 457
Jan A. van den Brakel

23.1 Introduction 457

23.2 Design of Embedded Experiments 458

23.3 Design-Based Inference for Embedded Experiments with One Treatment Factor 460

23.4 Analysis of Experiments with Clusters of Sampling Units as Experimental Units 466

23.5 Factorial Designs 468

23.6 A Mixed-Mode Experiment in the Dutch Crime Victimization Survey 472

23.7 Discussion 477

Acknowledgments 478

References 478

24 Extending the Within-Persons Experimental Design: The Multitrait-Multierror (MTME) Approach 481
Alexandru Cernat and Daniel L. Oberski

24.1 Introduction 481

24.2 The Multitrait-Multierror (MTME) Framework 482

24.3 Designing the MTME Experiment 487

24.4 Statistical Estimation for the MTME Approach 489

24.5 Measurement Error in Attitudes toward Migrants in the UK 491

24.6 Results 494

24.7 Conclusions and Future Research Directions 497

Acknowledgments 498

References 498

Index 501

DOWNLOADS
Chapter 5
Download
Chapter 8
Download
Chapter 15
Download
Chapter 16
Download
Chapter 19
Download
Chapter 20
Download
Chapter 23

"A thorough and comprehensive guide to the theoretical, practical, and methodological approaches used in survey experiments across disciplines such as political science, health sciences, sociology, economics, psychology, and marketing. This book bridges the gap in literature and explores the broad range of experimental designs in experiments that use probability samples. It approaches the usage of survey-based experiments with a Total Survey Error (TSE) perspective, which provides insight on the strengths and weaknesses of the techniques used. Experimental Methods in Survey Research: Techniques that Combine Random Assignment with Random Probability Sampling addresses experiments on within-unit coverage, reducing nonresponse, question and questionnaire design, minimizing interview measurement bias, using adaptive design, trend data, vignettes, the analysis of data from survey experiments, and other topics, across social, behavioral, and marketing science domains. Each chapter begins with a description of the experimental method or application and its importance, followed by reference to relevant literature. At least one detailed original experimental case study then follows to illustrate the experimental method's deployment, implementation, and analysis from a TSE perspective. The chapters conclude with theoretical and practical implications on the usage of the experimental method addressed. In summary, this book fills a gap in the current literature by successfully combining the subjects of survey methodology and experimental methodology in an effort to maximize both internal validity and external validity. Offers a wide range of types of experimentation in survey research with in-depth attention to their various methodologies and applications. Is edited by internationally recognized experts in the field of survey research/methodology and in the usage of survey-based experimentation-featuring contributions from across a variety of disciplines in the social and behavioral sciences. Presents advances in the field of survey experiments, as well as relevant references in each chapter for further study Includes more than 20 types of original experiments carried out within probability sample surveys. Addresses myriad practical and operational aspects for designing, implementing, and analyzing survey-based experiments by using a Total Survey Error perspective to address the strengths and weaknesses of each experimental technique and method. Experimental Methods in Survey Research: Techniques that Combine Random Assignment with Random Probability Sampling is an ideal reference for survey researchers and practitioners in areas such political science, health sciences, sociology, economics, psychology, public policy, data collection, data science, and marketing. It is also a very useful textbook for graduate-level courses on survey experiments and survey methodology"-- Provided by publisher.

"With a recent increase of experiments in survey research and methodology, survey researchers have found a need to bring together the many types of experiments used by survey researchers and their applications throughout various disciplines. This book bridges the gap in literature and explores the broad range of experimental design in survey-based experiments with a combined methodological and substantive focus on the ways the experiments are conducted and used in survey research. The book is edited by a group of established experts in the field of survey research/methodology and the chapters feature expert contributors across a variety of disciplines in the social and behavioral sciences. The book approaches coverage on survey-based experiments with a Total Survey Error (TSE) perspective, which provides insight on the strengths and weaknesses of the technique. Using the same basic structure, each chapter starts with a description of the experimental method or application, followed by a reference to any relevant reviews in current literature"-- Provided by publisher.

Description based on print version record and CIP data provided by publisher.

There are no comments for this item.

to post a comment.