2017 Annual Conference

Agenda & Abstracts

The Society for Benefit-Cost Analysis (SBCA) is an international group of practitioners, academics and others who are working to improve the theory and application of the tools of benefit-cost analysis.

Thank you to those who joined us at our Ninth Annual Conference! Slides from select presentations will be made available on our website here pending authors' permission.

We are excited to announce that this year's program will feature a keynote address by Katharine G. Abraham, Professor of Economics and Survey Methodology, University of Maryland and Chair, Commission on Evidence-Based Policymaking. She will speak on the topic of "Improving the Evidence Base for Government Decisions."

2017 Conference Program Committee

Lynn Karoly, Chair (RAND Corporation)
Daniel Acland (University of California, Berkeley)
Luc Baumstark (University of Lyon)
Susan Dudley (The George Washington University)
Margaret Kuklinski (University of Washington)
David Luskin (U.S. Department of Transportation)
John Mendeloff (University of Pittsburgh)
George Parsons (University of Delaware)
Emile Quinet (École des Ponts ParisTech)
Anne Smith (NERA Economic Consulting)
Fran Sussman (ICF International)
Judy Temple (University of Minnesota)
Craig Thornton (Mathematica Policy Research, Inc.)
Jack Wells (retired - U.S. Department of Transportation)

2017 Workshop Subcommittee

Gary VanLandingham, Chair (Florida State University)
Susan Dudley (The George Washington University)
Lynn Karoly (RAND Corporation)
Lisa Robinson (Harvard University)
Aylin Sertkaya (Eastern Research Group, Inc.)
Kip Viscusi (Vanderbilt University)
Darcy White (The Pew Charitable Trusts)


 

Agenda & Abstracts

Wednesday, March 15

  • 8:00 9:00       Registration for morning workshops

  • 9:00 12:30     Professional Development Workshops

  • 12:30 1:30     Lunch (provided for those registered for morning and afternoon sessions)

  • 1:00 1:30       Afternoon workshop registration

  • 1:30 5:00       Professional Development Workshops

  • 5:00 6:30       Opening Reception (registered workshop and conference participants, sponsored by ExxonMobil)

Thursday, March 16

  • 8:00 9:00        Registration and Breakfast 

  • 9:00 10:30      Session 1

    • A1:   Braathen Slides A1,   Cropper Slides A1,   Robinson Slides A1

    • B1:   Yinger Slides B1

    • E1:   Cox Slides E1

  • 10:30 10:45    Break 

  • 10:45 12:15    Session 2

    • A2:   Combined Roundtable Slides A2

    • B2:   Cox Slides B2

    • C2:   Garcia Ryder Slides C2,   Vainio Slides C2

    • E2:   Pike Slides E2

  • 12:15 2:00      Keynote Luncheon (sponsored by the Laura and John Arnold Foundation)

  • 2:00 3:30        Session 3

    • A3:   Vainio Slides A3

    • C3:   Helfand Slides C3

    • E3:   Sheehan-Connor Slides E3

    • F3:   Hamann Slides F3

  • 3:30 3:45        Break 

  • 3:45 5:15        Session 4

    • A4:   Dudley Slides A4,   Mannix Slides A4

    • B4:   Wilmoth Slides B4

    • F4:   Bos Slides F4,   Rose Slides F4,   Scheffler Slides F4

  • 5:30 7:30        Networking Reception 

Friday, March 17

  • 8:00 9:00        Registration and Breakfast 

  • 9:00 10:30      Session 5

    • A5:   Combined Slides A5

    • D5:   Srinivasan Gupta Slides D5

    • E5:   Brent Slides E5

    • F5:   Anglmayer Slides F5,   Emberger Slides F5,   Ramstedt Slides F5 and   Ramstedt Paper F5

  • 10:30 10:45    Break

  • 10:45 12:15    Session 6

    • B6:   Brennan Slides B6

    • C6:   Pasurka Slides C6,   Thornton Slides C6

    • D6:   Portuese Slides D6

    • E6:   Acks Slides E6,   Chilton Slides E6

    • F6:   Luskin Slides F6

  • 12:15 2:00      Luncheon featuring Annual Meeting of the SBCA Membership (sponsored by the                                    American Chemistry Council)

  • 2:00 3:30        Session 7

    • A7:   Robinson Slides A7

    • C7:   Raich Slides C7

    • D7:   Yaduma Slides D7

  • 3:30 3:45        Break

  • 3:45 5:15        Session 8

    • A8:   Dudley Slides A8

    • D8:   Ray Slides D8

  • 5:15 6:30        Closing Reception (sponsored by GW Regulatory Studies Center)


Workshop Offerings

Retrospective Benefit-Cost Analysis

Organizer: Jennifer Baxter, Industrial Economics, Incorporated

Description

While prospective benefit-cost analysis is a well-established component of the regulatory development process in the United States and other developed economies, retrospective analysis is less commonly undertaken. Yet such analysis provides important insights into how to best improve existing regulations, as well as into how to improve the conduct of prospective analysis. Retrospective analysis is now strongly encouraged under Executive Order 13563 for significant federal regulations, and is increasingly advocated in many other policy contexts in the United States and internationally.

Many presume that retrospective analysis will be more accurate than prospective analysis, assuming that analysts can simply sum the incurred costs and benefits. In reality, retrospective analysis is very challenging and often highly uncertain. The most difficult step is disentangling the incremental effects of the policy from the effects of other factors that influence current conditions, so as to compare outcomes in the relevant setting with the policy against counterfactual (and hypothetical) outcomes in the same setting had the policy never been adopted.

This workshop brings together four leading experts with diverse perspectives to discuss both the institutional context for these analyses and their conduct. It is targeted on both those interested in conducting these analyses and those interested in better understanding the strengths and limitations of analyses they review. Prior to the workshop, participants will receive a list of optional readings. The workshop itself will consist of a series of presentations with ample time for discussion. There are no prerequisites or requirements for participation.

Preliminary Agenda (subject to change)

9:00 9:45

Introduction & Institutional Frameworks and Processes (Cary Coglianese, University of Pennsylvania)

9:45 - 10:30

Fundamentals of Conducting Retrospective Benefit-Cost Analysis (Jennifer Baxter, Industrial Economics, Incorporated)

10:30 - 10:45

Break (with refreshments)

10:45 11:30

Insights from Program Evaluation (David Bernstein, Westat)

11:30 12:15

Example(s) of Recent Analyses (Maureen Cropper, Resources for the Future and University of Maryland)

12:15 - 12:30

Wrap-up (Jennifer Baxter)


Using Experts to Estimate Parameter Values in BCA

Organizer: Aylin Sertkaya, Eastern Research Group

Description

Decision makers must frequently rely on data or information that is incomplete or inadequate in one way or another. Judgment, often from experts and occasionally from non-experts, then plays a critical role in the interpretation and characterization of those data as well as in the completion of information gaps. Further, OMB expects agencies to examine relevant uncertainties in benefit-cost analyses of especially complex rules that exceed the $1 billion annual threshold using simulation models and/or expert judgment (OMB Circular A-4). But how experts or other stakeholders are selected and their judgments elicited matters they can also strongly influence the opinions obtained and the analysis on which they rely.

Several approaches to eliciting expert judgments have evolved. The workshop will cover topics ranging from expert recruitment, elicitation protocol design, different elicitation techniques (e.g., individual elicitations, Delphi method, nominal group technique, etc.) to aggregation methods for combining opinions of multiple experts. The role of expert elicitation and its limitations, problems, and risks in policy analysis will also be addressed. The workshop will include presentation of two case studies that will include a discussion of the selection process; elicitation protocol development, elicitation technique utilized, and the various issues that arose before, during, and after the elicitation process and the manner in which they were resolved. The class will conclude with a hands-on exercise where participants will learn about the importance of pre-elicitation training for calibrating experts.

Participation in this workshop will benefit economists, policy analysts and decision analysts. There are no prerequisites or requirements for attending this workshop.

Preliminary Agenda (subject to change)

1:30 - 1:45

Introduction to workshop-Motivation for having this workshop and its relevance for benefit-cost analyses (Aylin Sertkaya, ERG)

1:45 - 2:15

A quick guide to eliciting opinions for decision-making Introduction to the process as a whole (Aylin Sertkaya, ERG)

2:15 - 3:00

Case Study 1: Pilot Expert Elicitation Study of Uncertainty in VSL When Applied in an Air Pollution Context (Henry Roman, Industrial Economics)

3:00 - 3:15

Break (with refreshments)

3:15 - 4:00

Case Study 2: Effectiveness of Pre- & Post Harvest Practices in Reducing Salmonella Contamination Risk in Fresh and Fresh-Cut Tomatoes (Cristina McLaughlin, FDA)

4:00 - 5:00

Hands-on Exercise: Effect of Training on Expert Judgment (Frank Hearl, NIOSH)


Causal Analytics for Benefit-Cost Analysts: What Effects do Policies Cause?

Organizer: Tony Cox, Cox Associates

Description

Benefit-cost analysis (BCA) prescribes taking actions that cause total benefits larger than their total costs, but accurate prediction of benefits and costs is technically challenging, as is retrospective evaluation of effects. Effects caused by a policy may depend on what else happens, raising the challenge of attributing effects to one cause among many. Actions often have uncertain consequences, given realistically incomplete and imperfect knowledge and understanding of how the world works. Causal laws are often best modeled as probabilistic, and the relevant probabilities may not initially be known or agreed to. The psychology of causal reasoning is also notoriously misleading. The net result is that BCA must often rely on uncertain causal models of costs and benefits inferred from complex data with multiple interacting variables and imperfect, partly missing data. Despite these challenges, drawing valid causal conclusions from such data is crucial for predicting and evaluating effects of policies. 

This workshop introduces current machine-learning and computational statistics methods, implemented in free R packages, that can help to do so by discovering, testing, and validating causal models from observational data. The workshop will cover the principles underlying causal discovery and modeling algorithms, such as that (a) Causes provide unique information about their effects; (b) Changes in causes help to predict and explain subsequent changes in their effects; and (c) Information flows from causes to their effects over time. 

Participants will learn to apply information-based methods of causal inference to support BCA modeling while acknowledging and characterizing realistic uncertainties based on data. Practical illustrations will be given using a Causal Analytics Toolkit (CAT) that allows Excel users to apply advanced R packages to identify and quantify potential causal relationships in data. No previous knowledge of R or causal analysis is required.

The workshop will be taught by Tony Cox. 

Preliminary Agenda (subject to change)

1:30 - 2:00

Types of causality: Associational/attributive (Hill), counterfactual, computational, predictive, manipulative

2:00 - 2:30

Uncertain and probabilistic causality in tort law liability, insurance, biology, economics, complex systems physics, and engineering 

2:30 - 3:00

Principles and methods for causal analysis: Inference, explanation, and prediction

3:00 - 3:15

Break (with refreshments)

3:15 - 3:45

Information-based algorithms for causal discovery and inference

3:45 - 4:15

Making the algorithms useful: R packages and the Causal Analytics Toolkit (CAT)

4:15 - 5:00

Example applications: Toxicology, epidemiology, and policy analysis