The Reliability of Evidence in Evidence-Based Legislation

• A Introduction

  When Cicero went in search of Archimedes’ grave in Sicily, he found the headstone had a cylinder and a sphere and the formula showing the spatial relationship between the two.¹x InTusculan Disputations, Cicero writes, “When I was questor in Sicily [in 75 BC, 137 years after the death of Archimedes] I managed to track down his grave… Finally I noted a little column just visible above the scrub: it was surmounted by a sphere and a cylinder. I immediately said to the Syracusans, some of whose leading citizens were with me at the time, that I believed this was the very object I had been looking for.” Cicero, Tusculan Disputations, Book V, Sections 64-66 (Translation: Michael Grant, Cicero-On the Good Life, Penguin Books, 1971, p. 86-87. How wonderful for an academic to have an intellectual accomplishment so great that it should be carved into stone and represent a life’s work. Archimedes’ pride could have been simple bragging, but we prefer to think that he was challenging and inspiring the mathematicians who came after him: “here is what I did, improve on it.”
  During their long careers, Bob and Ann Seidman became pioneers in the field of Law & Development and created the Institutional Legislative Theory and Methodology (ILTAM), which has guided generations of legislative drafters worldwide. ILTAM is an evidence-based system for designing and drafting legislation that is meant to be superior to other drafting practices. ILTAM requires that the drafter study a social problem from several different angles, gather evidence of what is happening and why it happens, and design solutions according to that evidence. Each bill was accompanied by a research report where the drafter justifies the policy decisions and bill provisions with evidence sufficient to convince the ‘rational sceptic’ that the proposed bill was the best solution to a social problem.²x Each semester, our students would take on difficult social problems and would spend several months gathering evidence and drafting their bill and research report. The amount of evidence students could find was impressive; the reports were often between 75 and 100 pages with hundreds of footnotes citing the relevant evidence. Their hope was that the legislative debate would be elevated through evidence, producing better, more effective legislation.
  As evidence-based legislation develops, and as technology puts more information at our fingertips, there should be a better understanding of what exactly constitutes reliable evidence. With that aim in mind, this article will first examine evidence-based legislation and describe the Seidmans’ evidence-based methodology. The second part will discuss how evidence may be deemed ‘reliable.’ This review includes the standard for reliable expert testimony in trial courts as established by the Daubert line of cases, and a short review of the misuses of evidence such as confirmation bias and naïve beliefs. Third, we offer some examples of evidence-based legislation in practice: a World Health Organization treaty on tobacco control; The PEW Charitable Trust’s efforts to promote cost–benefit analysis by the states; and the mission for the recently created Evidence-Based Policymaking Commission. Finally, we offer a hierarchy of evidence from most to least reliable. We do not claim it to be a complete review, but a starting point for discussion to refine and improve evidence-based legislation. In this way, we hope to honour the memory of Bob Seidman by advancing and refining, in some small way, his extraordinarily valuable theories.

• B Evidence-Based Legislation and Institutional Legislative Theory and Methodology

  The desire to see policy makers use ‘evidence-based legislation’ has become widespread.³x R. Ismer & K. Meßerschmidt, ‘Evidence-Based Judicial Review of Legislation: Some Introductory Remarks’, The Theory and Practice of Legislation, Vol. 1, 2016, citing as examples: R.A.J. van Gestel, ‘Evidence-Based Lawmaking and the Quality of Legislation: Regulatory Impact Assessment in the European Union and the Netherlands’, in H. Schaffer & J. Iliopoulos-Strangas (Eds.), State Modernization in Europe, Berlin, Berliner Wissenschaftsverlag, 2007, p. 139; A. Seidman & R.B. Seidman, ‘ILTAM: Drafting Evidence-Based Legislation for Democratic Social Change’, Boston University Law Review, Vol. 89, 2009, p. 435 (hereinafter ‘ILTAM’). Evidence-based legislation is drafted in conjunction with rigorous research regarding the bill’s subject matter, followed by extensive monitoring and evaluation once the bill is in effect.⁴x World Development Report, United Nations, at 249. A solid evidentiary footing helps build political support by offering an objective method for winning the approval of those who are unconvinced or opposed to a measure.⁵x Id. at 250. See also, Ismer & Meßerschmidt, 2016, p. 2 (EBL ‘seeks to improve the quality of legislation by grounding legislative proposals on sound empirical evidence rather than mere presumptions’). Some jurisdictions, such as the European Union, use evidence-based legislation to lend legitimacy to its legislation.⁶x Ismer & Meßerschmidt, 2016, p. 2. The Seidmans taught that evidence-based legislation, especially in the form they developed, Institutional Legislative Theory and Methodology (ILTAM), was superior to the commonly used alternatives.⁷x These alternatives are: copying a law drafted elsewhere; resorting to a simple compromise without an evidentiary basis; creating a simplistic or symbolic solution; or drafting in broad terms allowing an agency great discretion to shape the law. ILTAM, 2009, pp. 439-440. Working and teaching in the newly independent African nations during the 1960s, led the Seidmans to ask why the post-colonial laws and programmes, although well-intentioned, failed to work, with governments losing a ‘fatal race.’⁸x ILTAM, 2009, pp. 439-440, n.20 (citing A. Seidman & R.B. Seidman, ‘The Fatal Race: Law-Making and the Implementation of Development Goals’, Third World Legal Studies, 1992, pp. 79, 81). Their conclusion was that the laws on some occasions did not affect the dysfunctional institutions often formed under colonial regimes.⁹x Id. An ‘institution’ consists of repetitive patterns of social behaviours, and legislation, if it is to work, must address the social behaviours that comprise the institution.¹⁰x Id. at 440. The Seidmans took a very broad view of an ‘institution.’ It could be businesses, government agencies, hospitals, schools, farms or a family. Institutions do change over time but, “for the most part[,] they change haphazardly.” Id. at 441. ILTAM provides legislative drafters and legislators with a methodology that allows statutes to be used instrumentally¹¹x See, A. Seidman & R.B. Seidman, ‘Instrumentalism 2.0’, Legisprudence, 2011, p. 95; A. Seidman, R.B. Seidman & N. Abeyesekere, Legislative Drafting for Democratic Social Change: A Manual for Drafters, London, Kluwer Law International, 2001; ILTAM, 2009, p. 436, n.1. and enact ‘transformative laws in the public interest.’¹²x ILTAM, 2009, p. 451.
  ILTAM consists of four steps to assist a drafter to conceptualize and develop a bill and structure the research report.¹³x Id. at 451-452. The first step is a description of the social problem and the behaviour of role occupants, including government agencies, that constitute or contribute to the problem.¹⁴x Id. at 452. The second step requires the drafter to find evidence and formulate hypotheses that explain each of the problematic behaviours identified in Step I.¹⁵x Id. at 453. Every actor in the law-making process is confronted with certain constraints and resources; one must ask why role occupants and government officials act the way they do? Problematic behaviour can be broken into three categories: “(1) the actor’s understanding of the relevant rule; (2) the actor’s anticipation of the implementing agency’s behavior; and (3) the non-legal constraints and resources of the actor’s own environment.”¹⁶x Id. at 453-454 (citing Seidman et al., 2001, p. 93). The Seidmans suggest seven sub-categories that help explain behaviour: the quality of the existing rules or laws; whether an actor has the opportunity to obey the rule; whether the actor has the capacity to obey the law; whether the rule has been properly communicated; an understanding of the incentives in place to follow or disobey the law; what process is in place to carry out the law; and what ideology contributes to an actor obeying or disobeying the law. Id. at 454. Evidence allows the drafter to develop ‘explanatory hypotheses,’ which direct the drafter to capture “the evidence required to test their validity.”¹⁷x ILTAM, 2009, p. 454. ILTAM’s third step is to create a legislative solution based on the evidence gathered in Step II.¹⁸x Id. at 455. Finally, the fourth step requires provisions for monitoring and evaluation “to assess whether and how [the law] works.”¹⁹x Id. (emphasis in the original). A well-written provision allows the legislature to gather more and better evidence on the issue going forward and may lead to a new round of law making.²⁰x Id.
  ILTAM is an evidence-based methodology and in their last article together Ann and Bob used some form of the term ‘evidence’ 31 times.²¹x Instrumentalism 2.0, 2011. The Seidmans’ methodology demands evidence that the proponent of a bill can persuade a hypothetical rational sceptic reader that the law ‘will likely work.’²²x Id. at 138-139 (citing J. Habermas & W. Rehg (tr.), Between Facts and Norms; Contributions to a Discourse Theory of Law and Democracy, Cambridge, MA, The MIT Press, 1996, p. 107). If the evidence wins over the ‘rational sceptic,’ the proposal likely advances the public – rather than private – interest.²³x Id. at 139-140. Jerry Mashaw’s prescription that if a law is to command obedience, those affected by it must agree that the law’s provisions are necessary and reasonable, and “[u]njustifiable law demands reform, unjustifiable legal systems demand revolution.” Id. (citing J. Mashaw, ‘Small Things Like Reasons’, Fordham Law Review, 2001 , Vol. 70, pp. 17, 19). The Seidmans acknowledge that the drafter confronts a vast amount of information while doing research but, they offer no guidance on what kind of evidence is better or more reliable and that is more likely to persuade the ‘rational sceptic.’ In this information age, where drafters and legislators alike can be flooded with information, there should be more of a guide.

• C Determining Reliable Evidence

  What is evidence? Black’s Law Dictionary defines it as:

  Testimony, writings, or material objects offered in proof of an alleged fact or proposition. That probative material, legally received, by which the tribunal may be lawfully persuaded of the truth or falsity of a fact in issue…That which tends to produce conviction in the mind as to existence of a fact.²⁴x Black’s Law Dictionary (6th ed. 1990) (defining ‘evidence’).

  Although the concept of ‘evidence’ is well established in litigation, there is little guidance to the drafter or policy maker as to what evidence is reliable. We are now in the information age – which can be both a blessing and a curse to the researcher. With the internet there is now far more data, reports, studies, anecdotes and opinions cheaply and instantly available at our fingertips than ever before. That is a blessing. The curse is that the researcher can be easily overwhelmed by this information and not know what to rely on – what is good or reliable evidence?²⁵x In this article, we make a distinction between evidence that is ‘relevant’ from evidence that is ‘reliable.’ The Federal Rules of Evidence defines ‘relevant evidence’ as that which has “any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without evidence.” Fed. R. Evid. 401.

  I Daubert and the Reliability of Expert Evidence

  In litigation, the standard for the reliability of expert and scientific evidence is extremely broad, but the US Supreme Court has offered some guidance. In Daubert v. Merrell Dow Pharmaceuticals, Inc.²⁶x Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 at 590, 113 S. Ct. 2786 (1993). the trial judge faced a battle of the experts over whether an anti-nausea drug caused a significant birth defect.²⁷x Id. at 582. The judge granted summary judgment because the plaintiff expert’s finding was ‘novel’ scientific evidence, which would only be admissible if the findings are “generally accepted in the field to which it belongs.”²⁸x Id. at 583. The Court found that the Federal Rules of Evidence Rule 702, which governs expert testimony,²⁹x Fed. R. Evid. 702. The Rule stated, “If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact at issue, a witness qualified as an expert by knowledge, skill, experience training or education, may testify thereto in the form of an opinion or otherwise.” The United States has no common law of evidence, and that judicial rulings were superseded by Congress’ passage of the Federal Rules of Evidence in 1975. See, Pub. L. No. 93–595, Jan. 2, 1975, 88 Stat. 1926. did not require the scientific evidence be ‘generally accepted.’³⁰x Daubert, 1993, p. 588. The ‘generally accepted’ test was formulated in Frye v. United States, 54 App. D.C 46, 47, 293 F. 1013, 1014 (1923). The Court points out that the Frye test had become the dominant standard for determining the admissibility of novel scientific evidence at trial but had come ‘under increasing attack of late,’ and some circuits had rejected the general acceptance standard. Daubert, 1993, p. 585 (citing DeLuca v. Merrell Dow Pharmaceuticals, Inc., 911 F.2d 941, 955 (CA3 1990). The evidence, however, cannot not be merely relevant, but must be grounded in the methods and procedures of science and be more than a ‘subjective belief’ or ‘unsupported speculation.’³¹x Daubert, 1993, p. 588. The rule establishes a standard for evidentiary reliability and the trial judge has the power to screen evidence to “ensure that any and all scientific testimony or evidence admitted is not only relevant, but reliable.”³²x Id. at 589. This power is commonly referred to as the trial court’s ‘gatekeeping’ function. Id. at 597. The judge must assess “whether the reasoning or methodology underlying the testimony is scientifically valid” and whether that reasoning can be applied to the facts under consideration.³³x Id. at 592-593. Because many factors will bear on a judge’s decision, the Court did not set out a firm test or checklist, but offered ‘general observations:’
  

  • Has the scientific knowledge been (or can it be) tested?

  • Has the theory or technique been subjected to peer review and publication?

  • What is the known or potential rate of error?

  • Are there standards controlling the technique’s operation?

  • Is there general acceptance of the reasoning or methodology?³⁴x Id. At 594-595. While the Court rejected this as the standard for accepting such evidence, it left room for general acceptance to have a bearing on the trial judge’s decision, ‘widespread acceptance can be an important factor in ruling particular evidence admissible.’ Further, a technique that has gathered only minimal support “may properly be viewed with skepticism.” Id. at 594.

  The Daubert standard has been modified through a series of cases to both give greater direction to the trial judge³⁵x General Electric Co. v. Joiner, 522 U.S. 136, 118 S.Ct. 512 (1997) (Court suggested that a trial judge could exclude expert testimony that, while methodologically sound, reaches questionable conclusions.). and apply to any type of specialized expert testimony, and not just scientific testimony.³⁶x Kumho Tire Co. v. Carmichael, 526 U.S. 137, 119 S.Ct. 1167 (1999). These cases led to an amendment to Federal Rules of Evidence Rule 702.³⁷x Fed. R. Evid. 702, 28 USCA states, “A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if: The expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; the testimony is based on sufficient facts or data; the testimony is the product of reliable principles and methods; and the expert has reliably applied the principles and methods to the facts of the case.” The amended Federal Rule, however, did not try to codify the Daubert checklist, which the court itself emphasized were neither exclusive nor dispositive.³⁸x See, Fed. R. Evid. 702 advisory committee’s note to 2000 amendment, citing Tyus v. Urban Search Management, 102 F.3d 256 (7th Cir. 1996) (noting that Daubert factors do not neatly apply to testimony by a sociologist); Kannankeril v. Terminix Int’l, Inc. 128 F.3d 802, 809 (3d Cir. 1997) (lack of peer review or publication was not dispositive where the expert’s opinion was supported by ‘widely accepted scientific knowledge’). In fact, the Federal Rules Advisory Committee’s notes to the 2000 amendment listed other factors relevant to whether expert testimony is sufficiently reliable:
  

  1. Whether experts are “proposing to testify about matters growing naturally and directly out of research they have conducted independently of the litigation, or whether they have developed their opinions expressly for the purposes of testifying.”³⁹x Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1317 (9th Cir. 1995).

  2. Whether the expert has unjustifiably extrapolated from an accepted premise to an unfounded conclusion.⁴⁰x General Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997) (trial court ‘may conclude that there is simply too great an analytical gap between the data and the opinion proffered’).

  3. Whether the expert has adequately accounted for obvious alternative explanations.⁴¹x Claar v. Burlington N.R.R., 29 F.3d 499 (9th Cir. 1994) (testimony excluded where the expert failed to consider other obvious causes for the plaintiff’s condition); Ambrosini v. Labarraque, 101 F3d. 129 (D.C. Cir. 1996) (the possibility of some uneliminated causes presents a question of weight, so long as the most obvious causes have been considered and reasonably ruled out by the expert).

  4. Whether the expert is being as careful as he would be in his regular professional work outside his paid litigation consulting.⁴²x Sheehan v. Daily Racing Form, Inc., 104 F.3d 940, 942 (7th Cir. 1997).

  5. Whether the field of expertise claimed by the expert is known to reach reliable results for the type of opinion the expert would give.⁴³x Kumho Tire, 1999, p. 1175 (Daubert’s general acceptance factor does not ‘help show that an expert’s testimony is reliable where the discipline itself lacks reliability, as for example, do theories grounded in any so-called generally accepted principles of astrology or necromancy.’).

  Given these many factors, and the various types of expert testimony, the trial judge must have ‘considerable leeway’ in this gatekeeping role and deciding whether expert testimony is reliable.⁴⁴x Id. at 1176. Even after Daubert, the rejection of expert testimony has been the exception rather than the rule, See, Fed. R. Evid. 702 (advisory committee notes to the 2000 amendment), with contrary expert evidence admitted and subject to the adversary system. Id. citing Heller v. Shaw Industries, Inc. 167 F3d. 146, 160 (3d Cir. 1999).

  1 The Difficulties of Evaluating Evidence

  Two difficulties facing EBL are confirmation bias and naïve beliefs. Often there will be evidence supporting both sides of an argument. Legislators must avoid simply searching for the best evidence supporting their position and calling this evidence-based legislation. Rather, EBL works best when legislators and drafters reserve judgment until the evidence has been fully researched and examined with a healthy amount of scepticism.
  Confirmation bias plagues all types of statistical analysis,⁴⁵x See B. Ravech, ‘On Being Certain: Believing You are Right Even When You’re Not’, Massachusetts Law Review, Vol. 93, 2011, pp. 364, 367. and is contrary to the goals of EBL, because it gives more credence to data that confirms the analyst’s beliefs.⁴⁶x Id. The nature of human memory makes it very difficult to put aside the predispositions that affect our worldview, subverted certitude in a process, even when this process is grounded in seemingly objective evidence.⁴⁷x Id. If a legislator approaches an issue with a closed mind, already certain of the outcome, she is likely to construe any evidence presented in the light most favourable to herself and her beliefs.⁴⁸x Id. In 2006, a British House of Commons committee report called into question the government’s use of scientific evidence in policy making, seemingly a problem of confirmation bias.⁴⁹x Science and Technology Committee, Scientific Advice, Risk and Evidence Based Policy Making (HC 2005-6, 900-1) at 47, 50-52. The British Parliament routinely makes a ‘call for evidence.’ to formulate policy. The evidence often includes stakeholder opinions, personal accounts, scientific research and think tank suggestions. The Committee took issue with the government selectively picking evidence to support existing policy or commissioning research to produce justifications for agreed-to policies, which the committee called ‘policy-based evidence-making.’⁵⁰x Id. at 47. Ultimately, the Committee was concerned that the misuse or the imprecise use of evidence would discredit all evidence.⁵¹x Id.
  Similarly, naïve beliefs challenge the efficacy of EBL by encouraging legislators to believe in previously heard anecdotes instead of concrete, contradictory evidence.⁵²x J.J. Rachlinski, ‘Evidence-Based Law’, Cornell Law Review, Vol. 96, 2011, pp. 901, 919-921. In many ways, “[t]he power of anecdote stands as a significant impediment to the development of evidence-based law[,]” especially when these anecdotes are inaccurate or grounded in unreasoned ideals.⁵³x Id. at 919. Anecdotes are particularly invidious because they masquerade as evidence and allow legislators to justify policy decisions despite a lack of actual evidence.⁵⁴x Id. at 919-920. For example, many members of the American public believe that the death penalty decreases crime and that gay marriage will increase the divorce rate when, in fact, there is much social science evidence to the contrary.⁵⁵x Id. at 922. These naïve beliefs are supported by the constant retelling of anecdotes.⁵⁶x Id. Legislators are not immune from these naïve beliefs solely by virtue of being legislators.⁵⁷x See Shai Wozner, ‘Evidence-Based Law by Jeffrey J. Rachlinski’, Cornell Law Review, Vol. 96, 2011, pp. 925-928. Indeed, they are arguably more susceptible to them because they must appeal to their constituencies every election season and must, therefore, rely upon such anecdotes to efficiently and convincingly demonstrate their commitment to certain core values.⁵⁸x Id.
  Confirmation bias and naïve beliefs may pose substantial challenges to EBL,⁵⁹x See Ravech, 2011, p. 367; Rachlinski, 2011; Wozner, 2011. but do not condemn EBL to failure.⁶⁰x See Rachlinski, 2011, pp. 922-923; Wozner, 2011, pp. 929-930. For example, popular opinion may actually operate as an ally of EBL; if an empirical legal study gets publicity, the study may gain widespread support, and legislators may be encouraged to fight against confirmation bias and naïve beliefs to advance legislation in support of the objective evidence offered by the study.⁶¹x Rachlinski, 2011, p. 923. Further, policy makers in the executive branch may rely upon evidence when creating rules and regulations, thereby offering a model for legislators and encouraging them to approach EBL with open minds.⁶²x Id.; see also W. Rhee, ‘Evidence-Based Federal Civil Rulemaking: A New Contemporaneous Case Coding Rule’, Pace Law Review, Vol. 33, 2013, p. 60. If legislators train themselves to fight naïve beliefs and confirmation bias, they can assess all presented evidence objectively and produce EBL beneficial to their constituents and to society at large.

• D Current Efforts to Use EBL

  In recent years, several scholars have argued for the adoption of evidence-based legislation in a variety of areas: laws for sex offenders and sex traffickers,⁶³x A. Shajnfeld & R.B. Krueger, ‘Reforming (Purportedly) Non-Punitive Responses to Sexual Offending’, Developments in Mental Health Law, Vol. 25, 2006, pp. 81, 97 (2006); see also, R. Weitzer, ‘Sex Trafficking and the Sex Industry: The Need for Evidence-Based Theory and Legislation’, Journal of Criminal Law & Criminology, Vol. 101, 2011, p. 1337. immigration,⁶⁴x D.E. Hill, ‘Guest Worker Programs Are No Fix for Our Broken Immigration System: Evidence from the Northern Mariana Islands’, New Mexico Law Review, Vol. 41, 2011, p. 131. physical education,⁶⁵x A.A. Ayler et al., ‘Examination of Trends and Evidence-Based Elements in State Physical Education Legislation: A Content Analysis’, Journal of School Health, Vol. 80, 2010, p. 326. public safety,⁶⁶x J. James et al., ‘A View from the States: Evidence-Based Public Safety Legislation’, Journal of Criminal Law & Criminology, Vol. 102, 2012, p. 821. prison reform,⁶⁷x S. Aos et al., ‘Evidence-Based Public Policy Options to Reduce Future Prison Construction, Criminal Justice Costs, and Crime Rates’, Federal Sentencing Reporter, Vol. 19, 2007, p. 275. public health,⁶⁸x S. Zaxa et al., ‘Using Science-Based Guidelines to Shape Public Health Law’, Journal of Law, Medicine & Ethics, Vol. 31, 2003, p. 65. the Federal Rules of Civil Procedure,⁶⁹x Rhee, 2013, p. 60. copyright⁷⁰x B. Patry, ‘Who’s Afraid of Evidence-Based Copyright Law?’, Suffolk University Law Review, Vol. 1, 2013, p. 55. and reproductive rights.⁷¹x R.J. Cook et al., ‘Emergency Contraception, Abortion and Evidence-Based Law’, International Journal of Gynecology & Obstetrics, Vol. 93, 2006, p. 191. In practice, some organizations are also actively using or encouraging the use of EBL. The UN’s World Health Organization has employed EBL in tobacco cessation programmes. In the United States, the Pew Charitable Trusts has an initiative to encourage state governments to use particular forms of EBL such as cost–benefit analysis on major bills. Third, the US Congress often attempts to report on the evidence gathered and used during the committee process and has recently created a commission to facilitate EBL. This section will describe these efforts in the context of evidence-based legislation.

  I World Health Organization

  Between 1998 and 2003, the World Health Organization (WHO) investigated tobacco use and developed the WHO Framework Convention on Tobacco Control, the first global health treaty.⁷²x K. Shibuya, C. Ciecierski, et al., ‘WHO Framework Convention on Tobacco Control: Development of an Evidence Based Global Public Health Treaty’, BMJ, Vol. 327, 2003, p. 154. Although negotiating the treaty was both ‘tough’ and ‘highly political,’⁷³x Id. at 155, citing H.A. Waxman, ‘The Future of the Global Tobacco Treaty Negotiations’, New England Journal of Medicine, Vol. 346, 2002, pp. 936-939. the scientific evidence on the health effects of tobacco and the availability of cost-effective interventions to reduce smoking proved essential.⁷⁴x Id.
  First, the WHO marshalled convincing studies that tobacco use was increasing in low- and middle-income countries; was increasing among women and children; was a major factor in disease burden; and caused 45 per cent more deaths in 2000 than in 1990. Further, the studies showed the problem would continue to grow, especially in developing regions.⁷⁵x At current levels of consumption, the burden from tobacco is estimated to double by 2020. Id. at 155 (citing, ‘The World Health Report 2002: Reducing Risks, Extending Healthy Life’, WHO [2002]). The WHO also produced evidence concerning the practicality and cost effectiveness of various interventions.⁷⁶x Id. at 155. Four interventions were ‘very cost effective’ in all sub-regions⁷⁷x WHO used a standard method to study 14 sub-regions of the world, while “accounting for subregional differences in demographic, epidemiological, and cost characteristics, and…specific information on exposure.” Id. at 155. of the world: taxation, clean indoor air laws, a ban on advertising, and information campaigns on the health risks of tobacco.⁷⁸x Id. at 155.
  The evidence presented to the various negotiating parties was key to overcoming the concerns of financial ministries in several countries, which were worried taxation would cause revenues to fall along with consumption. WHO evidence demonstrated that these fears were unfounded and that taxation raises revenue because consumption usually “falls at a lower rate than the percentage increase in price,” and “there is… room for many countries, particularly in low and middle income countries, to increase rates of tobacco taxes substantially.”⁷⁹x Id. at 156 (citing F.J. Chaloupka, T.W. Hu, K.E. Warner, R. Jacobs, & A. Yurekli, The Taxation of Tobacco Products in Tobacco control in Developing Countries, Oxford, Oxford University Press, 2000, p. 237-272; G.E. Guindon, S. Tobin, & D. Yach, ‘Trends and Affordability of Cigarette Prices: Ample Room for Tax Increases and Related Health Gains’, Tobacco Control, Vol. 11, 2002, pp. 35-43). The WHO Framework Convention on Tobacco Control is a landmark agreement addressing a global problem, and an example of effective use of evidence-based legislation.

  II PEW Charitable Trusts

  The PEW Charitable Trusts have been promoting evidence-based legislation in the American states. Starting in 1983, Washington State has been a leader in evidence-based policy-making, when the legislature created the Washington State Institute for Public Policy. The Institute’s mission is to “carry out practical, non-partisan research at the direction of the legislature” to “answer relevant policy questions.”⁸⁰x S.K. Urahn, ‘A Tipping Point on Evidence-Based Policymaking’, Governing, October 27, 2015, <www.governing.com/columns/smart-mgmt/col-state-local-government-tipping-point-evidence-based-policymaking.html>. Washington State Institute for Public Policy, <www.wsipp.wa.gov/About>. The Pew–MacArthur Results First Initiative (‘Initiative’) is now promoting a legislative approach similar to Washington’s and to embrace evidence-based legislation.⁸¹x The Initiative, a joint project of The Pew Charitable Trusts and the John D. and Catherine T. MacArthur Foundation, started in 2011. See, The Pew Charitable Trusts, <www.pewtrusts.org/en/projects/pew-macarthur-results-first-initiative>. The Initiative promotes a cost–benefit analysis of proposed legislation, by examining concrete evidence of plausible returns before investing in a law.⁸²x Id. This approach has, to some extent, been implemented in 20 states.⁸³x The Pew Charitable Trusts, <www.pewtrusts.org/en/projects/pew-macarthur-results-first-initiative/where-we-work>. To enter the Initiative, Initiative staff talks with state representatives to discuss evidence-based legislation and determine if the state is committed to an evidence-based system, can provide the data needed to develop a cost–benefit analysis model for the state, and is willing to dedicate the staff and resources necessary to succeed with the Initiative. The Pew Charitable Trusts, <www.pewtrusts.org/~/media/assets/2013/results-first-in-your-state-brief.pdf?la=en>. The Initiative uses a four-step process to implement evidence-based legislation in each state.⁸⁴x The Pew Charitable Trusts, <www.pewtrusts.org/en/research-and-analysis/fact-sheets/2015/07/the-pew-macarthur-results-first-initiative>. First, it creates an inventory of all currently funded programmes, which allows the Initiative and the state to identify wasteful spending.⁸⁵x Id. Second, the Initiative conducts rigorous research studies to determine which of the state’s current programmes are generating a positive return on the state’s investment.⁸⁶x Id. These studies utilize the Results First Clearinghouse Database, an online archive of research on state programmes across the country, to assess the likely effectiveness of the state’s current initiatives. Third, the Initiative collects cost information on the State’s programmes and services to craft a tailored cost–benefit model to estimate the return for the state on each proposed bill.⁸⁷x Id. The evidence should assist the state in making informed policy and budgeting decisions, such as eliminating programmes with poor performance and reallocating resources to more promising alternatives.⁸⁸x Id. For example, New Mexico now requires state executive agencies to support budget increase requests with evidence supporting programme effectiveness. Similarly, New Mexico has used the Initiative’s techniques since 2013 to develop and utilize customized cost–benefit model programmes involving criminal and juvenile justice, early childhood and child welfare. Through this process, New Mexico was able to allocate $104.4 million to its most effective programmes. Id.

  III The Evidence-Based Policymaking Commission Act of 2016

  Recently, the federal government promoted evidence-based legislation when President Obama signed the Evidence-Based Policymaking Commission Act of 2016.⁸⁹x Pub. L. No. 114-140 See, <https://www.whitehouse.gov/omb/management/commission_evidence>. House of Representatives Speaker Paul Ryan touted the commission as integral to a new approach to fighting poverty that emphasizes policies and programmes with a track record of success.⁹⁰x <www.speaker.gov/press-release/evidence-based-policy-commission-gets-to-work>. The Act establishes a 15-member bipartisan commission⁹¹x The appointees were divided between the President, the Speaker of the House, The Senate Majority Leader and the Senate and House Minority Leaders. The bill required that appointments should be made to ‘Individuals with expertise in economics, statistics, program evaluation, data security, confidentiality, or database management.’ Pub. L. No. 114-140 Sec. 3 (b). with a mandate to: integrate administrative and survey data and to facilitate research, evaluation and analysis; recommend how best to incorporate rigorous evaluation into programme design; and consider whether a Federal clearinghouse should be created for government survey and administrative data.⁹²x Id. In September 2016, the Commission requested comments from the public on several questions related to its charge, including: whether states, localities and international governments had designed successful frameworks, policies, practices and methods related to evidence-building; how can data, statistics, results of research and findings from evaluation be best used to improve policies and programmes; and to what extent should evaluations specifically with either experimental (sometimes referred to as ‘randomized control trials’) or quasi-experimental designs be institutionalized in programmes?⁹³x <https://www.regulations.gov/document?D=USBC-2016-0003-0001>. One commentator urged the Commission to find methods for incorporating randomized evaluations into the design of government programmes similar to the trials that ‘revolutionized modern medicine in the 20th century.’ The key to such randomized evaluations is the availability of administrative data from ‘hospitals, governments, school systems, and other institutions’ with little added cost to researchers and more possibility for long-term follow-up. Q. Palfrey, ‘Promoting Policies That Work: Six Steps for the Commission on Evidence-Based Policymaking’, The Hill, 1 November 2016. (<http://thehill.com/blogs/congress-blog/the-administration/303781-promoting-policies-that-work-six-steps-for-the>). Palfrey points out that use of administrative data had been successfully used to make informed policy decisions.
  The Evidence-Based Policymaking Act of 2016 demonstrates the growing popular support for evidence-based legislation by the federal government. As a commentator stated, “Analyzing administrative data to adjust government programs may seem like dry, behind-the-scenes work, but when scaled up to redirect national policies, it can have a significant impact on millions of Americans.”⁹⁴x Id.

• E A Proposed Hierarchy of EBL Evidence

  In 1908, Harvard Law Dean Roscoe Pound wrote on the future of legislation and compared legislatures to laboratories, “We are told that law-making of the future will consist in putting the sanction of society on what has been worked out in the sociological laboratory.”⁹⁵x Id. at 406 (citing, Ward, Applied Sociology, Boston, Ginn & Company, 1906, p. 338). Although legislation will never be a purely scientific endeavour, reliable evidence can and should play a larger role in legislative development and approval. To assist the legislator, or an observer to the legislative process, determine whether the legislature is using reliable evidence, we propose the following hierarchy from most reliable to least:
  

  1. Experiments within the jurisdiction / lessons from other jurisdictions.

  2. Information on a topic or issue that was formally requested by the legislature or produced to the legislature under oath or under the penalties of perjury.

  3. Studies / information provided by a government agency.

  4. Expert or scientific studies.

  5. Economic or mathematical models and statistics.

  6. Information provided by special interests.

  7. Stories, apocrypha and uncorroborated tales.

  We do not claim this list to be precise or complete, but offer it as a starting point to refine what is called ‘evidence-based legislation.’ Each part of this hierarchy is described in this section.

  I Experiments within the Jurisdiction/Lessons from Other Jurisdictions

  In evidence-based medicine data and information gained from controlled experimental work, particularly randomized controlled trials, is considered the most reliable evidence.⁹⁶x Evidence-Based Medicine advocates consider controlled experimental work, particularly randomized controlled trials, to be the most reliable evidence. See, D.L. Sackett et al., ‘Evidence Based Medicine: What It Is and What It Isn’t’, BMJ, Vol. 312, 1996, p. 71. This form of testing medical practices, and especially new drugs, utilizes a control group for comparison, which does not receive the new treatment, but receives a reference treatment or placebo. Further, the decision about whether a patient receives the new treatment or is a part of the control group is made randomly. See, M. Macgill, ‘What is a Randomized Controlled Trial in Medical Research?’, Medical News Today, <www.medicalnewstoday.com/articles/280574.php>. It would be difficult, if not impossible, to design a similar trial for a statute with randomly picked control groups. Still, legislature can experiment within its own jurisdiction, or look to other jurisdictions to see whether various legislative options had the desired effect on a social problem. How legislation works in practice is perhaps the most reliable form of evidence.

  1 Experiments within a Jurisdiction

  States can, and at times do, test a novel policy idea by initially limiting its scope in area and time, and using the newly gained evidence to decide whether to end, alter or expand the programme. An example of such limited experiment dealing with a difficult social problem is the Massachusetts Legislature’s efforts to control sex offenders through an intensive form of parole.⁹⁷x D.K. Baker, J. Skolnick, G. Doucette, G. Levitt, & R. Baker, ‘Intensive Parole Supervision of the Sex Offender—Putting the Containment Approach Into Practice’, Handbook of Sex Offender Treatment, Barbara K. Schwartz, ed. 2011, pp. 62-1-62-15, <https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=265228>. In 1996 the legislature authorized the Massachusetts Parole Board to experiment with Intensive Parole for Sex Offenders (IPSO) in a limited area to the west of the metro Boston area. Rather than having the typical one parole officer, the parolees were supervised by a team of specialists who implemented stricter standards than those for other types of parolees⁹⁸x This programme uses some or all of the following supervision techniques: mandatory counselling, electronic monitoring, unannounced home and work visits, curfews, polygraph testing, random urinalysis, travel restrictions, mandatory daily diaries, and sex offender registration. Parole Practices in Massachusetts and Their Effect on Community Reintegration’, Boston Bar Association Task Force on Parole & Community Reintegration, August 2002, p. 9 (<https://www.bostonbar.org/prs/reports/finalreport081402.pdf>). After five years of operation, there were 114 offenders under supervision with only four parolees returned to custody, and none for a sex offense.⁹⁹x ‘Intensive Parole for Sex Offenders’, Massachusetts Parole Board (2002) (on file with the author). During that time there have been 114 offenders under supervision; only four have been returned to custody, none for a sex offense; Boston Bar Association Task Force on Parole Report, 2002, p. 9. Based on this information, the Massachusetts Legislature devoted the resources needed to expand the programme state-wide.
  Another method of experimentation is to limit a statute to a particular period of time through a sunset clause. As the statute approaches its sunset date, the legislature can do nothing and the statute goes out of force or the legislature can reauthorize the statute, often with modifications. For example, the controversial USA PATRIOT Act¹⁰⁰x Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act, Pub. L. No. 107-56, 115 Stat. 272 (2001). contained a sunset clause giving its provisions a term of four years.¹⁰¹x When originally passed in 2001, portions of the PATRIOT Act were to sunset on December 31, 2005. Since 2001, parts of the Act have been reauthorized,¹⁰²x See, USA PATRIOT Improvement and Reauthorization Act of 2005, Pub. L. No. 109-177 (2006); USA PATRIOT Act Additional Reauthorizing Amendments Act of 2006, Pub. L. No: 109-178 (2006); PATRIOT Sunsets Extension Act of 2011, Pub. L. No. 112-14, Pub. L. No. 112-14 (2011). The Act was allowed to lapse due to a lack of Congressional action in June 2015; and later reenacted in the USA Freedom Act, Pub.L. 114–23 Title II, sec. 201-202 (2015), which extended certain provisions until 2019. each time with Congress assessing the effects of the law and making amendments. For example, in the most recent version, Congress ended the National Security Agency’s (NSA) mass phone data collection programme.¹⁰³x See, USA Freedom Act, Pub.L. 114–23 Title II, sec. 201-202 (2015).

  2 Lessons from Other Jurisdictions

  Justice Louis Brandeis famously declared that the various states were ‘laboratories of democracy’¹⁰⁴x New State Ice Co. v. Liebmann, 285 U.S. 262 (1932) (Brandeis, J. dissenting) (“It is one of the happy incidents of the federal system that a single courageous State may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country.”). as they grapple with the same social problems as other parts of the country. Evidence of how a statute or programme worked elsewhere and the ability to question the legislative and agency personnel as to what they have learned is extremely helpful. In 2016, Massachusetts voters approved the recreational use of marijuana through the initiative petition process.¹⁰⁵x An Act Relative to Legalized Recreational Marijuana for Individuals At Least 21 Years Old, 2016 Mass. Acts ch. 334, (Approved by the People at the State Election on November 8, 2016, effective in part December 15, 2016). In anticipation of this change in the law, eight Massachusetts senators travelled to Colorado to see what the effects of legalized marijuana had on that state and to gather facts as to how the legislature might have to respond to a new law that was drafted without legislative help.¹⁰⁶x J. Miller, ‘Marijuana Dispensary among State Senators’ Stops in Colorado’, Boston Globe, 11 January 2016, <https://www.bostonglobe.com/metro/2016/01/10/mass-senators-colorado-study-marijuana/luFCdZnQoUyw45GwOBvgdO/story.html>. The senators were to tour a cultivation facility, a marijuana dispensary and to question “state, municipal and law enforcement officials” with questions about Colorado’s voter-approved law. Id. This was especially necessary given what many saw as Massachusetts’ ‘disastrous’ implementation of a 2012 medical marijuana law.¹⁰⁷x Id. In that case, a dispensary for patients did not open until 2015. One commentator suggested looking to other states like Colorado for good public policy, just as businesses seek best practices from other companies.¹⁰⁸x Id. (commentary of Tufts University Prof. Jeffery M. Berry). This type of information is becoming easier to obtain, with organizations such as the National Conference of State Legislatures acting as a clearinghouse for information on policy development in the various states.¹⁰⁹x See, <www.ncsl.org/aboutus.aspx>. This type of evidence is very reliable since it has been tested by application. Still, the Professors Seidman would frequently warn their students that although one can learn much from what happened in other places, it was counter-productive to simply copy a law from elsewhere.

  II Information on a Topic or Issue That Was Formally Requested by the Legislature or Produced to the Legislature under Oath or under the Penalties of Perjury

  The legislature may rely on its own internal research for reliable evidence. It may also seek information from external sources and increase the reliability through oaths and perjury laws.

  1 Information from Internal Offices of the Legislature

  Congress has created research offices to provide it with reliable, unbiased information and empirical evidence. The Congressional Budget Office (CBO) provides economic information and analysis to committees and members of Congress.¹¹⁰x See, 2 U.S.C. sec. 604 (2014). Each year, the agency’s economists and budget analysts “produce dozens of reports and hundreds of cost estimates for proposed legislation.”¹¹¹x <https://www.cbo.gov/about/overview>. CBO does not make policy recommendations, and each report and cost estimate summarizes the methodology underlying the analysis. Id. The Congressional Research Service (CRS), a division of the Library of Congress, serves congressional committees and members of Congress by providing “insightful and comprehensive analysis” on current policies and present the impact of proposed policy alternatives.¹¹²x See, Pub. L. 91-510, title III, sec. 321(a), 26 October 1970, (codified at 84 Stat. 1181; 2 U.S.C. 166.) CRS employs experts in nearly every field of study and can be called upon throughout the legislative process to provide policy reports; briefings and consultations; seminars and workshops; and expert testimony.¹¹³x Congressional Research Service (<www.loc.gov/crsinfo/about/>). CRS employs more than 400 are policy analysts, attorneys and information professionals working across a variety of disciplines. See, <www.loc.gov/crsinfo/research/>. In 2015, CRS answered over 62,046 requests for custom analysis and research; hosted over 7,400 Congressional participants at seminars, briefings and trainings; and summarized over 8,000 pieces of legislation.¹¹⁴x Congressional Research Service, ‘Annual Report Fiscal Year 2015’, (<www.loc.gov/crsinfo/about/crs15_annrpt.pdf>).
  Information coming from these organizations is considered very reliable even on highly controversial and complex issues. An example is the debate leading up to the passage of the Affordable Care Act.¹¹⁵x The Affordable Care Act (‘Obamacare’) was enacted through two bills: Patient Protection and Affordable Care Act (PPACA), Pub. L. No. 111-148, 124 Stat. 119 (2010) and the Health Care and Education Reconciliation Act of 2010 (HCERA) Pub. L. No. 111-152, 124 Stat. 1029 (2010). One of the major committee reports leading up to this law cited the Congressional Budget Office’s estimates of the legislation’s fiscal impact.¹¹⁶x The Financial Services Committee issued one of only two reports on health care reform when considering a precursor to the ACA, America’s Healthy Future Act of 2009, S. Rep. No. 111-89 (2009) (‘Fin. Comm. Report’). See, 124 Stat. at 1083. The Congressional Budget Office predicted that the legislation will reduce the deficit by $81 billion in the first 10 years, will provide health insurance to 29 million Americans, increase the rate of insurance to 94 per cent, and would ultimately cost of $829 billion. Id. at 9. The Report also cites the CBO when it states that health care costs are the single most important factor influencing the Federal Government’s long-term fiscal balance. Id. at 2. Although the CBO overestimated the number of people who would buy insurance on the Obamacare exchanges, its estimate as to how many people would gain insurance under the law was extremely close.¹¹⁷x The CBO estimated that 23 million would be on the exchanges by 2016, but the number was actually 10 million. CBO projected that 30 million people (or 11 per cent of the population under 65) would not have health insurance in 2016, and the actual number turned out to be 27.9 million (10.3 per cent). The Washington Post, March 14, 2017, <https://www.washingtonpost.com/news/fact-checker/wp/2017/03/14/fact-checking-the-white-houses-rhetoric-on-the-cbo-report/?utm_term=.cdbab5fbe57c>. A 2015 study by the Commonwealth Fund concluded, “The CBO’s projections were closer to realized experience than were those of many other prominent forecasters.”¹¹⁸x The Washington Post, 2017.

  2 Required Reporting and Monitoring by Agencies

  Another very reliable form of evidence is information that agencies are required to provide the legislature. This evidence is distinguished from an agency’s voluntarily provided information discussed in category 3, which will almost always put the agency in a good light or openly advocate for the agency. In a legislatively mandated report of particular information, an agency must report both what makes it look good and bad. A well-written reporting and monitoring mandate will also likely cause the agency to gather information it would not absent a mandate.
  This form of evidence fits in well with the Seidmans’ ILTAM methodology. How a statute will work in practice is difficult to predict because the drafter cannot gather all evidence and will not always make a correct hypothesis.¹¹⁹x ILTAM, 2009, pp. 454-455. ILTAM encourages the drafter to think of legislation as an ongoing endeavour with monitoring and evaluation of laws to assess whether and how the law works.¹²⁰x ILTAM, 2009, p. 455. Agencies have the resources and expertise to gather, analyse and produce reliable information leading to new rounds of law making grounded on better and more complete evidence.¹²¹x Id. (Only after the drafter analyses the information gathered after enactment, the ‘law-in-action,’ can improvements be drafted, debated and enacted to ‘ensure the law works.’).

  3 Testimony to the Legislature under Oath

  The legislative process requires the constant gathering of information, and the ability to compel testimony is necessary to a legislature’s investigative and informational powers.¹²²x O. Hetzel, M. Libonati, & R. Williams, Legislative Law and Statutory Interpretation, 4th ed., Newark New Jersey, LexisNexis, 2008, p. 283. As the Supreme Court noted in Watkins v. United States,¹²³x Watkins v. United States, 354 U.S. 178 (1957).

  The power of the Congress to conduct investigations is inherent in the legislative process…It includes surveys of defects in our social economic or political system for the purpose of enabling the Congress to remedy them. It comprehends probes into departments of the Federal Government to expose corruption, inefficiency or waste…It is unquestioningly the duty of all citizens to cooperate with the Congress in its efforts to obtain the facts needed for intelligent legislative action. It is their unremitting obligation to respond to subpoenas, to respect the dignity of the Congress and its committees and to testify fully with respect to matters within the province of proper investigation.¹²⁴x Id. at 187.

  Congress has an inherent power to issue subpoenas, and each chamber’s rules provide this authority on the standing committees and subcommittees.¹²⁵x See, J. Hamilton, R.F. Muse, & K.R. Amer, ‘Congressional Investigations: Politics and Process’, American Criminal Law Review, Vol. 44, 2007, pp. 1115, 1124-1126; M.E. O’Neill, ‘The Fifth Amendment in Congress: Revisiting the Privilege Against Compelled Self-Incrimination’, Georgetown Law Journal, Vol. 90, 2002, p. 2445. Further, Congressional committees have the power to administer oaths to witnesses.¹²⁶x 2 U.S.C. § 191 (2000). If a witness provides false testimony under oath, they may be prosecuted for perjury.¹²⁷x See, Hamilton et al., 2007, pp. 1126-1129. Perjury prosecutions may be brought under federal law, 18 U.S.C. §1621 (2000), or under the District of Columbia law, D.C. Code Ann. § 22-2402 (2001), or under the relevant state law. Generally the standard for prosecution is that a false statement be made before a competent tribunal, which includes Congressional committees, that the statement be made wilfully, and that it concern a material matter. Id. Although witnesses may refuse to answer questions based on their 5th Amendment right against self-incrimination, a house of Congress or a committee may obtain a court order compelling the desired testimony in exchange for immunity against subsequent prosecution.¹²⁸x Id. at 1129-1131. Finally, Congress may enforce its subpoenas and orders by bringing contempt proceedings against witnesses who refuse to cooperate with its investigations.¹²⁹x Id. at 1132-1137.
  Given these sweeping powers to compel testimony, punish false statements and offer immunity for incriminating testimony, evidence gathered in such a manner should be considered very reliable.

  III Studies / Information Provided by a Government Agency

  Agencies are often one of the best sources of information because they: can study an issue over a lengthy period of time; develop true expertise in a field; and have the resources to gather and analyse data. Although legislators and staff should, and typically do, work closely with agency personnel, there should also be independent corroboration and research. An agency can be so focused on one problem, its personnel may not understand the ‘big picture’ and why resources are allocated the way they are. Agencies are also institutions with a culture and institutional history. While this offers stability and predictability, the Seidmans pointed out that, like any institution, agencies change over time, often haphazardly, and at times become dysfunctional.¹³⁰x See, ILTAM, 2009; Seidman & Seidman, 2011; Seidman, et al., 2001 and the accompanying texts. To change behaviour, a statute needs to target the agency’s behaviour and give clear instructions to agency personnel to carry out the wishes of the legislature. For this reason, while the information provided by agencies is likely to be reliable, it should still be corroborated.

  IV Academic / Scientific Studies

  Legislators often rely on academic and scientific studies as an independent source of evidence. Of course, there are researchers and studies of differing quality and reliability. Legislative debates, like trials, often feature a ‘battle of experts’ with different legislators citing studies to back up their positions. To determine the reliability of the various studies, legislators and their staff could utilize a version of the questions suggested to trial judges in the Daubert line of cases discussed here.¹³¹x Id. at 5-8. These include:
  

  • Have the observations and opinions being presented been tested or subjected to peer review and publication?¹³²x Daubert, 1993, pp. 593-594.

  • What standards are in place to produce reliable evidence?¹³³x Id.

  • Do other experts generally accept the reasoning or methodology?¹³⁴x Id. at 594-595.

  • Is the expert testifying about matters growing out of their research independent of a legislative debate, or did they produce research at the request of a party with an interest in the legislation?¹³⁵x Id.; Sheehan, 1997.

  • Has the expert made an unjustifiable leap from an accepted premise to an unfounded conclusion?¹³⁶x General Elec. Co. v. Joiner, 1997.

  • Has the expert adequately accounted for obvious alternative explanations?¹³⁷x Claar v. Burlington, 1994; Ambrosini, 1996.

  • Is the field of expertise known to reach reliable results for the type of opinion?¹³⁸x Kumho Tire, 1999, p. 1175.

  Using these standards, a reasonable sceptic should be able to separate reliable research and analysis from lower quality studies.

  V Data from Statistics and Economic or Mathematical Models

  Statistics and models are extremely common and can be very useful, but have drawbacks that can cause unreliability. To properly evaluate the offered data, one must have a thorough understanding of the assumptions and biases that are built into the model. With that understanding, however, and if the assumptions stay consistent, this data can provide accurate predictions on the effectiveness of a law.

  1 Statistics

  Statistics can be a powerful form of evidence in a legislative debate because the results have a patina of respectability as they are rooted in data and mathematical formulae. Still, statistics can be extremely misleading, and need to be carefully analysed before placing too much reliance on them. During the Megan’s Law debate statistics figured prominently in both the Congressional and New York debates to prove the extent of the sex-crime problem and the high recidivism rate for offenders.¹³⁹x Megan’s Law is officially known as the Sexual Offender Registration Act of 1994, NJ Pub. L. ch. 128 and (codified as N.J.S.A. 2C:7-1 through 2C:7-11.). After New Jersey adopted Megan’s Law, several other states followed with similar statutes. In 1996, the Congress enacted a federal version of the law. See, Megan’s Law, Pub. L. No. 104-145, 110 Stat. 1345 (1996) (amending 42 U.S.C. § 14071(d) (1994). D. Filler, ‘Making the Case for Megan’s Law: A Study in Legislative Rhetoric’, Indiana Law Journal,Vol. 76, 2001, pp. 315, 352. During the Congressional debate, representatives asserted that: over 50,000 Texas children suffered child abuse or neglect in 1995;¹⁴⁰x Id. at 335, citing, 142 Cong. Rec. 10,313 (1996) (statement of Rep. Jackson-Lee). that 114,000 children were the victims of attempted abductions, with 4,600 actual disappearances in 1988;¹⁴¹x Id. at 335, citing, 139 Cong. Rec. 31,251 (1993) (statement of Rep. Ramstad). and that 65,000 violent felons in prison report having victimized a child.¹⁴²x Id. at 335, citing, 142 Cong. Rec. 18,765-66 (1996) (statement of Sen. Hutchinson). These numbers effectively convinced the Congress, media and public that there was a widespread and growing problem. As one commentator states, “numbers have a way of dazzling the listener.”¹⁴³x Id. at 353.
  These dazzling numbers, however, had little to do with the proposed Megan’s Law: of the 50,000 abused Texas children, few would be covered by the law;¹⁴⁴x Id. Using a definition of abuse and neglect from the Children’s Trust Fund of Texas, Filler concludes, “The degree to which this broad classification exceeds the scope of Megan’s Law is virtually self-evident. Megan’s Law would do nothing to protect children from irresponsible parents, for instance.” and the 4,600 abducted children statistic was deceptive because the study defined ‘abduction’ very broadly.¹⁴⁵x Id. at 353, citing D. Finkelhor et al., Missing, Abducted, Runaway, and Throwaway Children in America: First Report: Numbers and Characteristics National Incidence Studies 4, 66 (1990). This study was commissioned by the US Office of Juvenile Justice and Delinquency Prevention. The reports’ definition of ‘abduction’ included cases of minimal coercion, very brief detentions, and non-family perpetrators such as acquaintances and babysitters. Id. The types of abductions covered by Megan’s Law would have occurred just 200 to 400 times in 1988.¹⁴⁶x Id. at 353-354. By using the inflated numbers, legislators advocating for Megan’s Law misled their colleagues and the public.¹⁴⁷x Id. at 354.

  2 Economic and Mathematical Models

  Policy makers often use data analysis from economic and mathematical models. Often these models play an important role forecasting what effects a policy decision will have, they also have significant limitations.
  A mathematical model is, “a representation in mathematical terms of the behavior of real devices and objects.”¹⁴⁸x V. Dabbaghian & V.K. Mago, ‘What is Mathematical Modeling?’, in V. Dabbaghian & V.K. Mago (Eds.), Theories and Simulations of Complex Social Systems, New York, Springer, 2014 (<www.sfu.ca/~vdabbagh/Chap1-modeling.pdf>). An economic model is “a simplified framework for describing the workings of the economy.”¹⁴⁹x D. Zenghelis, ‘What Do Economic Models Tell Us?’, London School of Economics, Grantham Research Institute on Climate Change and the Environment, <www.lse.ac.uk/GranthamInstitute/news/why-economic-models-tell-us-so-little-about-the-future-2/>. Models are useful to analyse information through observation and gathering empirical evidence, modelling, and prediction.¹⁵⁰x Dabbaghian & Mago, 2014. To be useful, the modeller must “formally articulate assumptions and tease out relationships behind this assumptions.”¹⁵¹x Zenghelis, <www.lse.ac.uk/GranthamInstitute/news/why-economic-models-tell-us-so-little-about-the-future-2/>. Models are also reliant on abstractions because not every variable can be included and not every causal process can be simulated.¹⁵²x Id. The model is useful to explain behaviour or to predict future behaviour.¹⁵³x Id., “Models are used for two main purposes: simulating (e.g. how would the world change relative to some counterfactual if we assume a change in this or that variable) and forecasting (e.g. what the world might look like in 2030)”. See also, Dabbaghian & Mago, 2014.
  For example, researchers have analysed the British Columbia criminal justice system with a system dynamics model that measured the impact of the management decisions on departments such as prosecution and court services and corrections.¹⁵⁴x V. Dabbaghian, P. Jula, P. Borwein, E. Fowler, C. Giles, N. Richardson, A.R. Rutherford, A. Van der Wall, ‘High-Level Simulation Model of a Criminal Justice System’, in V. Dabbaghian & V.K. Mago (Eds.), Theories and Simulations of Complex Social Systems, 2014. The model serves as a simulation tool for the following questions:
  

  • What is the impact of police resources on crime in general?

  • How to allocate police resources in order to balance the workload across a criminal justice system?

  • How does the distribution of police resources affect specific crime types, such as impaired driving or organized crime?

  • What is the impact of changes in upstream practices on corrections – especially with regard to community versus institutional sentences?¹⁵⁵x Id.

  There are, however, significant limitations to models. First, the models are only as good as the assumptions and inputs used.¹⁵⁶x Zenghelis (“key inputs are often chosen arbitrarily and even the best model spits rubbish out if you pump rubbish in.”) Models are also very limited in making long-term predictions because it is so hard to make valid assumptions.¹⁵⁷x Id. (“The further out the forecast, the larger the structural uncertainties making model projections at best illustrative”). Zenghelis points out that macroeconomic models of an economy used by finance ministries banks and central banks rarely look beyond a four-year horizon and are essentially used as a ‘consistency check and not a source of projections.’ Models are also difficult to ‘calibrate,’ that is, adjusting the model parameters to make it applicable to the specific conditions.¹⁵⁸x D.H. Freedman, ‘Why Economic Models are Always Wrong’, Scientific American, 26 October 2011 (<https://www.scientificamerican.com/article/finance-why-economic-models-are-always-wrong/>). Many complex models use historical data and ‘various computational techniques’ to adjust the parameters so the model would have ‘predicted’ the historical data.¹⁵⁹x Id. Once the parameters are set, the model should predict what will happen going forward, but even tiny flaws in the model or the historical data can cause bad predictions.¹⁶⁰x Id. (noting that it has become routine for modellers in finance to keep recalibrating their models over and over again and still produce bad predictions.) Further, even analysis of well-designed and regarded models can be misused by government officials to exaggerate public benefits of proposed government projects.¹⁶¹x E.S. Mills, ‘The Misuse of Regional Economic Models’, Cato Journal, Vol. 13, Spring /Summer 1993, p. 29. Mills points out that the economic model he examines is the use of the REMI model, ‘one of the very best regional impact models extant,’ which he characterizes as widely used, and easily evaluated since it is ‘available in explicit detail.’ But even analysis from this reliable model can mislead the public into thinking that there are benefits to government projects that would not come from similar private projects. Id. at 38.
  While statistics and models are essential tools to examine and understand interactive relationships, their inherent limitations must be taken into consideration when determining reliability.¹⁶²x Zenghelis. The questions offered above to help determine the reliability of expert testimony¹⁶³x Id. at 5-8. may be relevant to assessing the reliability of analysis from economic or mathematical models and statistics.

  VI Information Provided by Special Interests

  Special interests, and their lobbyists provide legislators and staff with a great deal of information. Lobbying is not just a Constitutionally protected activity, but absolutely essential to the proper functioning of a legislature.¹⁶⁴x See, V.R. Johnson, ‘Regulating Lobbyists: Law, Ethics, and Public Policy’, Cornell Journal of Law and Public Policy, Vol. 16, 2006, pp. 1, 5-10 (Lobbying is directly tied to the right to petition in the 1st Amendment of the US Constitution: “In some instances, it is necessary or appropriate for persons seeking to petition the government to channel their efforts through volunteer or paid intermediaries.” Id. at 8. ‘Special interests’ is defined broadly here as any advocacy group: a corporate or business interest, a labour union, a political party, a trade association, a charity, a church, or a municipality. Id. at 10, Congressional Quarterly’s Guide to Congress 700-715 (5th ed. 2000); A. Rosenthal, The Third House: Lobbyists and Lobbying in the States, 2nd ed., Washington DC, C.Q. Press, 2001, pp. 61-63. Legislators and their staff, however, must take the information given to them with caution and a critical eye. Like lawyers advocating in court, lobbyists are making an argument to win the day for their client. To truly assess reliability, the legislator must seek out other sources of information and opinions.
  Lobbyists provide essential information to the legislative process, ‘Though widely vilified, lobbyists representing individuals or groups can make a valuable contribution to informed and effective government.’¹⁶⁵x Johnson, 2006, p. 9. As experts in the legislative process, lobbyists can direct the right information to the right person to influence how a bill is drafted or amended.¹⁶⁶x Rosenthal, 2001, p. 195 (Rosenthal quotes a New Jersey lobbyist as saying, “Getting the right information to the right people at the right time…It’s a home run to make that combination work.”) Lobbyists are often also skilled advocates who can effectively present their client’s point of view or desired outcome.¹⁶⁷x Id. at 194 (“Today’s lobbyist will invariably have to make an argument on the merits.”). W.K. Muir, Jr., Legislature: California’s School for Politics, Vol. 19, 1982 (“What a lobbyist does, therefore, is to provide basic information—quickly and intelligibly.”) The lobbyist can also provide insight into the consequences of a particular piece of legislation and the political ramifications of voting for a bill.¹⁶⁸x Id. at 9. Lobbyists often inform legislators how intensely positions are held and how other actors and their districts will be affected by a measure.¹⁶⁹x Id. at 196. Of course, if a lobbyist wishes to be sought out by legislators, they must have a reputation of being both knowledgeable and trustworthy.¹⁷⁰x Id. Rosenthal cites a Florida lawyer-lobbyist with an expertise in land use who is very much in demand with Florida legislators and a California lobbyist whose reputation for providing reliable evidence, makes his evidence especially useful for the record. Many of the lobbyists interviewed stated that, “People trust us.” For example, a Florida lobbyist spent his career working with alcoholic beverages, including time directing the division of alcoholic law in a previous gubernatorial administration, has made himself a “trusted source” of information on alcohol-related issues. Id. at 195-196.
  Many lobbyists prepare position papers providing substantive and technical information on various issues.¹⁷¹x Id. These are readily used by legislators – either to explore the actual effects of ongoing programmes and policies, and others that challenge proposed policies.¹⁷²x Id. Rosenthal points out that “Policy and fiscal analyses are in great currency.” In addition to research and reports, lobbyists will deliver experts to the legislators and their staffs.¹⁷³x Id. at 197. At times this will be in public testimony and at other times at one-on-one meetings with the policy makers. Rosenthal gives the example of a Maryland lobbyist who set up a panel of finance and budget experts to explain the effects of a proposed tax-collection bill. This information led to the legislature establishing a study group to further study the matter. Lobbyists will, at times, represent clients with positions that are not popular with the public, but need to present their position to legislators. These opinions are unlikely to be presented at public hearings or cited in a floor speech, but influences the debate. For instance, a legislator considering a popular animal cruelty bill may quietly meet with the pharmaceutical industry to hear its argument before voting. A lobbyist in this position explained his approach as:

  You try to get people who are involved with that issue in the legislature to meet with people who are involved in your research and have them share all the information as to why we need to have animals – not because it’s something that we want to do, but because it’s something we think we have to do.¹⁷⁴x Id.

  When Congress studied health care reform, the Finance Committee utilized studies by several special interests including: the Institute of Medicine study; a 2003 RAND Corporation study; a National Health Care Anti-Fraud Association study; data from the Centers for Medicare and Medicaid Services; a 2009 Journal of Health Affairs study; a 2002 MedPAC report; and a 2009 survey of Medicaid participation rates.
  Some special interest groups have also begun to advocate for evidence-based legislation to lobby for their preferred legislation. The Ohio Justice and Policy Center “helped form the Colorado Sex Offense Information Coalition to promote evidence-based legislation and policies to reduce the risk of sexual offender recidivism.”¹⁷⁵x D.A. Singleton, ‘Kids, Cops, and Sex Offenders: Pushing the Limits of the Interest-Convergence Thesis’, Howard Law Journal,Vol. 57, 2013, pp. 353, 390.
  Over time, legislators and their staff may build a relationship with these interests and be able to easily assess the reliability of their evidence. It is always important to independently verify this information and look for other points of view to give legislators a full and accurate picture of a problem and potential solution.

  VII Stories, Apocrypha, Uncorroborated Tales

  Stories maybe the least reliable source of evidence, but is a commonly used tool in the legislative process.¹⁷⁶x See, Rachlinski, 2011 and the accompanying discussion of naïve beliefs. Several legal academics have argued that storytelling is the “strongest non-violent persuasive method we know.”¹⁷⁷x See, e.g. B. Foley & R. A. Robbins, ‘Fiction 101: A Primer For Lawyers on How to Use Fiction Writing Techniques to Write Persuasive Fact Sections’, Rutgers Law Journal, Vol. 32, 2001, pp. 459, 465 (2001) (quoting, D. Ball, Theater Tips and Strategies for Jury Trials 66 (1994); See also R.A. Robbins, ‘Harry Potter, Ruby Slippers and Merlin: Telling the Client’s Story Using the Characters and Paradigm of the Archetypal Hero’s Journey’, Seattle University Law Review,Vol. 29, 2006, p. 767; Clare Keefe Coleman, ‘Dangerous Tongues: Storytelling in Congressional Testimony and an Evidence Based Solution’, New York University Journal of Legislation and Public Policy,Vol. 19, 2016, pp. 291, 296-301. The root of this persuasion is but emotion, which corresponds better to the way the mind makes sense of experiences, as opposed to the logic of legal abstractions.¹⁷⁸x Foley & Robbins, 2001 (quoting Sol Stein, Stein on Writing: A Master Editor of Some of the Most Successful Writers of our Century Shares His Craft Techniques and Strategies 224 (1995); Robbins, 2006, p. 769, quoting S.L. Winter, ‘The Cognitive Dimension of the Agony Between Legal Power and Narrative Meaning’, Michigan Law Review, Vol. 87. 1989, pp. 2225, 2228. See also, Filler, 2001, p. 346 (citing S. Bandes, ‘Empathy, Narrative, and Victim Impact Statements’, University Chicago Law Review, Vol. 63, 1996, pp. 361, 383 (“We make sense of the world by ordering it into metaphors, and ultimately narratives with familiar structures and conventions”). See also, Coleman, 2016, p. 298 (Stories can make complex ideas easy, and they are hard to refute.) Legislators use stories because they are easily understood, powerful and persuasive. Stories can also be misunderstood, and sometimes serve to stifle debate. Stories may be unreliable, but still have a place in legislative debate, if used carefully.
  Legislators have long used powerful stories to justify and win support for a bill.¹⁷⁹x Coleman, 2016, p. 300 (“Storytelling is, in a sense, the heart of political reasoning.”) (narrative stories are the principal way policy problems are defined and contested). Id. citing D. Stone, Policy Paradox: The Art of Political Decision Making, 3rd ed., New York, WW Norton & Company, 2012, p. 158. An example is the passage of sex offender criminal registries and public notification,¹⁸⁰x The Jacob Wetterling Crimes Against Children and Sexually Violent Offenders Registration Act, Pub. L. No. 103-322, tit. XVII, §170101, 18 Stat. 1796, 2038 (1994) (codified as amended at 42 U.S.C. §14071 (1994 & Supp. IV 1998). Megan’s Law, or the Sexual Offender Registration Act of 1994, NJ Pub. L. ch. 128 and (codified as N.J.S.A. 2C:7-1 through 2C:7-11.); the federal statute is: Megan’s Law, Pub. L. No. 104-145, 110 Stat. 1345 (1996) (amending 42 U.S.C. § 14071(d) (1994). which swept the United States in the 1990s and were named after youthful victims Jacob Wetterling and Megan Kanka.¹⁸¹x On July 29, 1994, 7-year-old Megan Kanka was raped and murdered by a neighbour with two prior convictions for sexual offenses against children. Filler, 2001, p. 315. In October 1989, Danny Heinrich abducted, sexually assaulted and killed 11-year-old Jacob Wetterling. In September 2016, Heinrich confessed to the killing. P. Louwagie & J. Brooks, ‘Danny Heinrich Confesses to Abducting and Killing Jacob Wetterling’, Minnesota Star Tribune, 7 September 2016, <www.startribune.com/danny-heinrich-confesses-to-abducting-and-killing-jacob-wetterling/392438361/\. The powerful narratives behind the bills caused Congress and the state legislatures to pass the bills by overwhelming majorities, usually relying heavily on storytelling and narratives, “Every congressional story told in support of Megan’s Law featured a child victim who suffered serious abuse.”¹⁸²x Filler, 2001, p. 316. Only one representative in Congress spoke in opposition to the bill and the final vote in the House of Representatives was 418-0. Several states, such as Florida did not even have a debate about the bill before passing Megan’s Law unanimously. Id. at 316-317. New York Legislators invoked both the crime against Megan Kanka, and stories of New York child victims and abusers.¹⁸³x Id. at 332-333. Interestingly, one legislator cautioned about excessive reliance on the ‘emotionally powerful’ Megan Kanka story to the exclusion of logic and reason. Id. at 332, n. 105 (citing, N.Y. Senate Minutes of S-11-B, at 6624 [May 24, 1995] [statement of Senator Leichter]). One Assemblyman even told his own personal story of being abducted as a 12-year old.¹⁸⁴x Id. at 334 (citing, N.Y. Assembly Minutes of A1059C, at 342-46 [June 28, 1995] [statement of Mr. Spano]).
  There are, however, significant problems with relying on stories to shape or justify legislation. First, it is difficult to get a single meaning for a story because interpretation often depends more on a listener’s preconceptions than the story’s content.¹⁸⁵x Id. at 350. Second, stories may also mislead because they “are always told within particular historical, institutional, and interactional contexts that shape their telling, meanings and effects.”¹⁸⁶x Coleman, 2016, p. 298 (quoting P. Ewick & S.S. Silbey, ‘Subversive Stories and Hegemonic Tales: Toward a Sociology of Narrative’, Law & Society Review, Vol. 29, 1995, pp. 197, 205. Third, the narrative may take the place of other, more reliable evidence.¹⁸⁷x For instance, the debate surrounding an amendment to the FDA Food Safety and Modernization Act, Pub. l. No. 111-353, 124 Stat. 3885 (2011) (codified as amended in scattered sections of the US Code.), which was meant to protect small farmers, was only narrative, with “No data, no study, and no science” to back up the proponent’s assertions. Coleman, 2016, p. 318. Fourth, a narrative might stifle debate because of the difficulty inherent in disagreeing with an emotional story.¹⁸⁸x Filler, 2001, p. 350, (citing, D.A. Farber & S. Sherry, Beyond All Reason: The Radical Assault on the Truth in American Law, 89 (1997) (‘it’s hard to say anything critical about the story without implicating the storyteller.’). If legislators argued Megan’s Law would not have prevented her death, that would have directly challenged her parents. Id. at 351. Ultimately, only the supporters of Megan’s Law utilized the power of storytelling, and the few opponents to the law used logic-based arguments that ‘lacked rhetorical vibrancy.’¹⁸⁹x Id. at 352. Storytelling in these cases becomes one-sided, bolstering a worldview that the public had already accepted, and, therefore doing little to educate the public about the complicated nature of these crimes.¹⁹⁰x Id. Filler proposes that legislatures consider adopting a ‘code of debate’ that would encourage legislators to make honest claims and address all policy concerns. This code could also require legislators to ‘tell stories ethically.’ Id. at 365. This would require storytellers to: (1) rely on a broad factual basis, (2) demonstrate a regard for interpersonal complexities, (3) emphasize the psychological apparatus and states of mind of the participants and (4) acknowledge the narrator’s bias. Id. (citing, D.D. Troutt, ‘Screws, Koon, and Routine Aberrations: The Use of Fictional Narratives in Federal Police Brutality Prosecutions’, New York University Law Review, Vol. 74, 1999, pp. 18, 96. It is hard to see, however, how such an ethics code squares with the principles of free speech and how such a code would be regulated or enforced in the legislative context.
  Ultimately, stories can serve a legitimate and important purpose for the legislator: these can be effectively used to illustrate a social problem or to help other legislators and the public understand an issue on an emotional level. Legislators, however, should take great care to avoid having powerful or emotional stories overwhelm other, more reliable forms of evidence.

  1 Who Decides Reliability?

  Whose responsibility is it to determine whether evidence should be reliable when formulating policy and drafting legislation? In a trial, the judge is the ‘gatekeeper’ of reliable expert testimony.¹⁹¹x Daubert, 1993. Who shall be the gatekeeper in the legislature?
  Filler proposes the creation of a ‘public advocate’ to improve legislative debate, or at least guard against some of its potential abuses. The public advocate’s role would be to ensure that no law was passed without “a full, honest, and comprehensive debate.”¹⁹²x Filler, 2001, p. 365. Filler imagines this advocate participating in debates where a bill has little or no opposition, and argue reasons to oppose the bill, challenge claims made by the bill’s supporters, and suggest alternatives to the bill.¹⁹³x Id. Coleman also proposes a public advocate to examine all legislation under serious consideration and to employ empirical tools to analyse bills. This would cause Congress to create evidence-based legislation.¹⁹⁴x Coleman, 2016, p. 331. Further, the goals of the legislation would be “scientifically tested in controlled studies and proven effective.”¹⁹⁵x Id. at 332 (citing, ‘Evidence-Based Practice’, Cal. Cts, <www.courts.ca.gov/5285.htm>). Coleman would require bill sponsors to articulate a bill’s goals and rationale. Those bills with murky goals would be sent back to committee for redrafting, and bills containing goals not supported by data would not be permitted to advance.¹⁹⁶x Id. at 333. Coleman admits that not all theories would be testable, but points out that empirical methods can be used to test the reasons people offer to support or oppose a rule. Id. Will Rhee recommends that, “policy makers scrutinize publicly available empirical evidence” before making decisions.¹⁹⁷x W. Rhee, ‘Entitled to be Heard: Improving Evidence-Based Policy Making Through Audience and Public Reason’, Indiana Law Journal, Vol. 85, 2010, pp. 1315, 1318.
  We believe that it is up to the legislators – and the staff as the legislators’ alter-egos – to be the gatekeepers. At the committee level, the chair plays an outsized role in sorting evidence. As was true of the health care bill discussed here, there may be a dissenting report with its own evidence. One side or the other is legitimized when the chamber as a whole votes to adopt one version of the bill and the evidentiary basis upon which it was built. Another chamber may make different, even diametrically opposing, choices. The version of the bill adopted by a conference committee, based on the facts and evidence the committee finds most persuasive, which may then be ratified by the two chambers. Finally, the evidence and facts underlying the bill gain further legitimacy when the executive signs the bill into law.
  A cornerstone of Seidmans’ philosophy was that the evidence-based methodology forced legislators who sought to change a bill to produce better evidence to overcome what had convinced the original producer of the bill.¹⁹⁸x See, ‘Instrumentalism 2.0’ and accompanying text. A system where evidence is required to justify any bill or amendment, would produce more rational discourse and better legislation. It may take time and effort to establish a culture of evidence-based legislation within a legislature, but the society it represents will be stronger for it.

• F Conclusion

  During a 50-year-plus career, Bob and Ann Seidman studied legislation and legislative drafting. They taught that legislation rooted in evidence could be used to solve complex social problems by targeting the institutions that perpetuated dysfunctional situations. Evidence revealed what was wrong with society and evidence pointed the way to a solution. The Seidmans’ Institutional Legislative Theory and Methodology is their lasting monument – and challenge to other practitioners of evidence-based legislation. In the information age, it is very difficult to determine what evidence is reliable and should be the basis of legislative debate and policy making. Hopefully the hierarchy of evidence offered in this article will be a starting point for conversation and make evidence-based legislation more useful for legislators and legislative drafters.

Noten

• 1 InTusculan Disputations, Cicero writes, “When I was questor in Sicily [in 75 BC, 137 years after the death of Archimedes] I managed to track down his grave… Finally I noted a little column just visible above the scrub: it was surmounted by a sphere and a cylinder. I immediately said to the Syracusans, some of whose leading citizens were with me at the time, that I believed this was the very object I had been looking for.” Cicero, Tusculan Disputations, Book V, Sections 64-66 (Translation: Michael Grant, Cicero-On the Good Life, Penguin Books, 1971, p. 86-87.

• 2 Each semester, our students would take on difficult social problems and would spend several months gathering evidence and drafting their bill and research report. The amount of evidence students could find was impressive; the reports were often between 75 and 100 pages with hundreds of footnotes citing the relevant evidence.

• 3 R. Ismer & K. Meßerschmidt, ‘Evidence-Based Judicial Review of Legislation: Some Introductory Remarks’, The Theory and Practice of Legislation, Vol. 1, 2016, citing as examples: R.A.J. van Gestel, ‘Evidence-Based Lawmaking and the Quality of Legislation: Regulatory Impact Assessment in the European Union and the Netherlands’, in H. Schaffer & J. Iliopoulos-Strangas (Eds.), State Modernization in Europe, Berlin, Berliner Wissenschaftsverlag, 2007, p. 139; A. Seidman & R.B. Seidman, ‘ILTAM: Drafting Evidence-Based Legislation for Democratic Social Change’, Boston University Law Review, Vol. 89, 2009, p. 435 (hereinafter ‘ILTAM’).

• 4 World Development Report, United Nations, at 249.

• 5 Id. at 250. See also, Ismer & Meßerschmidt, 2016, p. 2 (EBL ‘seeks to improve the quality of legislation by grounding legislative proposals on sound empirical evidence rather than mere presumptions’).

• 6 Ismer & Meßerschmidt, 2016, p. 2.

• 7 These alternatives are: copying a law drafted elsewhere; resorting to a simple compromise without an evidentiary basis; creating a simplistic or symbolic solution; or drafting in broad terms allowing an agency great discretion to shape the law. ILTAM, 2009, pp. 439-440.

• 8 ILTAM, 2009, pp. 439-440, n.20 (citing A. Seidman & R.B. Seidman, ‘The Fatal Race: Law-Making and the Implementation of Development Goals’, Third World Legal Studies, 1992, pp. 79, 81).

• 9 Id.

• 10 Id. at 440. The Seidmans took a very broad view of an ‘institution.’ It could be businesses, government agencies, hospitals, schools, farms or a family. Institutions do change over time but, “for the most part[,] they change haphazardly.” Id. at 441.

• 11 See, A. Seidman & R.B. Seidman, ‘Instrumentalism 2.0’, Legisprudence, 2011, p. 95; A. Seidman, R.B. Seidman & N. Abeyesekere, Legislative Drafting for Democratic Social Change: A Manual for Drafters, London, Kluwer Law International, 2001; ILTAM, 2009, p. 436, n.1.

• 12 ILTAM, 2009, p. 451.

• 13 Id. at 451-452.

• 14 Id. at 452.

• 15 Id. at 453.

• 16 Id. at 453-454 (citing Seidman et al., 2001, p. 93). The Seidmans suggest seven sub-categories that help explain behaviour: the quality of the existing rules or laws; whether an actor has the opportunity to obey the rule; whether the actor has the capacity to obey the law; whether the rule has been properly communicated; an understanding of the incentives in place to follow or disobey the law; what process is in place to carry out the law; and what ideology contributes to an actor obeying or disobeying the law. Id. at 454.

• 17 ILTAM, 2009, p. 454.

• 18 Id. at 455.

• 19 Id. (emphasis in the original).

• 20 Id.

• 21 Instrumentalism 2.0, 2011.

• 22 Id. at 138-139 (citing J. Habermas & W. Rehg (tr.), Between Facts and Norms; Contributions to a Discourse Theory of Law and Democracy, Cambridge, MA, The MIT Press, 1996, p. 107).

• 23 Id. at 139-140. Jerry Mashaw’s prescription that if a law is to command obedience, those affected by it must agree that the law’s provisions are necessary and reasonable, and “[u]njustifiable law demands reform, unjustifiable legal systems demand revolution.” Id. (citing J. Mashaw, ‘Small Things Like Reasons’, Fordham Law Review, 2001 , Vol. 70, pp. 17, 19).

• 24 Black’s Law Dictionary (6th ed. 1990) (defining ‘evidence’).

• 25 In this article, we make a distinction between evidence that is ‘relevant’ from evidence that is ‘reliable.’ The Federal Rules of Evidence defines ‘relevant evidence’ as that which has “any tendency to make the existence of any fact that is of consequence to the determination of the action more probable or less probable than it would be without evidence.” Fed. R. Evid. 401.

• 26 Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 at 590, 113 S. Ct. 2786 (1993).

• 27 Id. at 582.

• 28 Id. at 583.

• 29 Fed. R. Evid. 702. The Rule stated, “If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact at issue, a witness qualified as an expert by knowledge, skill, experience training or education, may testify thereto in the form of an opinion or otherwise.” The United States has no common law of evidence, and that judicial rulings were superseded by Congress’ passage of the Federal Rules of Evidence in 1975. See, Pub. L. No. 93–595, Jan. 2, 1975, 88 Stat. 1926.

• 30 Daubert, 1993, p. 588. The ‘generally accepted’ test was formulated in Frye v. United States, 54 App. D.C 46, 47, 293 F. 1013, 1014 (1923). The Court points out that the Frye test had become the dominant standard for determining the admissibility of novel scientific evidence at trial but had come ‘under increasing attack of late,’ and some circuits had rejected the general acceptance standard. Daubert, 1993, p. 585 (citing DeLuca v. Merrell Dow Pharmaceuticals, Inc., 911 F.2d 941, 955 (CA3 1990).

• 31 Daubert, 1993, p. 588.

• 32 Id. at 589. This power is commonly referred to as the trial court’s ‘gatekeeping’ function. Id. at 597.

• 33 Id. at 592-593.

• 34 Id. At 594-595. While the Court rejected this as the standard for accepting such evidence, it left room for general acceptance to have a bearing on the trial judge’s decision, ‘widespread acceptance can be an important factor in ruling particular evidence admissible.’ Further, a technique that has gathered only minimal support “may properly be viewed with skepticism.” Id. at 594.

• 35 General Electric Co. v. Joiner, 522 U.S. 136, 118 S.Ct. 512 (1997) (Court suggested that a trial judge could exclude expert testimony that, while methodologically sound, reaches questionable conclusions.).

• 36 Kumho Tire Co. v. Carmichael, 526 U.S. 137, 119 S.Ct. 1167 (1999).

• 37 Fed. R. Evid. 702, 28 USCA states, “A witness who is qualified as an expert by knowledge, skill, experience, training, or education may testify in the form of an opinion or otherwise if: The expert’s scientific, technical, or other specialized knowledge will help the trier of fact to understand the evidence or to determine a fact in issue; the testimony is based on sufficient facts or data; the testimony is the product of reliable principles and methods; and the expert has reliably applied the principles and methods to the facts of the case.”

• 38 See, Fed. R. Evid. 702 advisory committee’s note to 2000 amendment, citing Tyus v. Urban Search Management, 102 F.3d 256 (7th Cir. 1996) (noting that Daubert factors do not neatly apply to testimony by a sociologist); Kannankeril v. Terminix Int’l, Inc. 128 F.3d 802, 809 (3d Cir. 1997) (lack of peer review or publication was not dispositive where the expert’s opinion was supported by ‘widely accepted scientific knowledge’).

• 39 Daubert v. Merrell Dow Pharmaceuticals, Inc., 43 F.3d 1311, 1317 (9th Cir. 1995).

• 40 General Elec. Co. v. Joiner, 522 U.S. 136, 146 (1997) (trial court ‘may conclude that there is simply too great an analytical gap between the data and the opinion proffered’).

• 41 Claar v. Burlington N.R.R., 29 F.3d 499 (9th Cir. 1994) (testimony excluded where the expert failed to consider other obvious causes for the plaintiff’s condition); Ambrosini v. Labarraque, 101 F3d. 129 (D.C. Cir. 1996) (the possibility of some uneliminated causes presents a question of weight, so long as the most obvious causes have been considered and reasonably ruled out by the expert).

• 42 Sheehan v. Daily Racing Form, Inc., 104 F.3d 940, 942 (7th Cir. 1997).

• 43 Kumho Tire, 1999, p. 1175 (Daubert’s general acceptance factor does not ‘help show that an expert’s testimony is reliable where the discipline itself lacks reliability, as for example, do theories grounded in any so-called generally accepted principles of astrology or necromancy.’).

• 44 Id. at 1176. Even after Daubert, the rejection of expert testimony has been the exception rather than the rule, See, Fed. R. Evid. 702 (advisory committee notes to the 2000 amendment), with contrary expert evidence admitted and subject to the adversary system. Id. citing Heller v. Shaw Industries, Inc. 167 F3d. 146, 160 (3d Cir. 1999).

• 45 See B. Ravech, ‘On Being Certain: Believing You are Right Even When You’re Not’, Massachusetts Law Review, Vol. 93, 2011, pp. 364, 367.

• 46 Id.

• 47 Id.

• 48 Id.

• 49 Science and Technology Committee, Scientific Advice, Risk and Evidence Based Policy Making (HC 2005-6, 900-1) at 47, 50-52. The British Parliament routinely makes a ‘call for evidence.’ to formulate policy. The evidence often includes stakeholder opinions, personal accounts, scientific research and think tank suggestions.

• 50 Id. at 47.

• 51 Id.

• 52 J.J. Rachlinski, ‘Evidence-Based Law’, Cornell Law Review, Vol. 96, 2011, pp. 901, 919-921.

• 53 Id. at 919.

• 54 Id. at 919-920.

• 55 Id. at 922.

• 56 Id.

• 57 See Shai Wozner, ‘Evidence-Based Law by Jeffrey J. Rachlinski’, Cornell Law Review, Vol. 96, 2011, pp. 925-928.

• 58 Id.

• 59 See Ravech, 2011, p. 367; Rachlinski, 2011; Wozner, 2011.

• 60 See Rachlinski, 2011, pp. 922-923; Wozner, 2011, pp. 929-930.

• 61 Rachlinski, 2011, p. 923.

• 62 Id.; see also W. Rhee, ‘Evidence-Based Federal Civil Rulemaking: A New Contemporaneous Case Coding Rule’, Pace Law Review, Vol. 33, 2013, p. 60.

• 63 A. Shajnfeld & R.B. Krueger, ‘Reforming (Purportedly) Non-Punitive Responses to Sexual Offending’, Developments in Mental Health Law, Vol. 25, 2006, pp. 81, 97 (2006); see also, R. Weitzer, ‘Sex Trafficking and the Sex Industry: The Need for Evidence-Based Theory and Legislation’, Journal of Criminal Law & Criminology, Vol. 101, 2011, p. 1337.

• 64 D.E. Hill, ‘Guest Worker Programs Are No Fix for Our Broken Immigration System: Evidence from the Northern Mariana Islands’, New Mexico Law Review, Vol. 41, 2011, p. 131.

• 65 A.A. Ayler et al., ‘Examination of Trends and Evidence-Based Elements in State Physical Education Legislation: A Content Analysis’, Journal of School Health, Vol. 80, 2010, p. 326.

• 66 J. James et al., ‘A View from the States: Evidence-Based Public Safety Legislation’, Journal of Criminal Law & Criminology, Vol. 102, 2012, p. 821.

• 67 S. Aos et al., ‘Evidence-Based Public Policy Options to Reduce Future Prison Construction, Criminal Justice Costs, and Crime Rates’, Federal Sentencing Reporter, Vol. 19, 2007, p. 275.

• 68 S. Zaxa et al., ‘Using Science-Based Guidelines to Shape Public Health Law’, Journal of Law, Medicine & Ethics, Vol. 31, 2003, p. 65.

• 69 Rhee, 2013, p. 60.

• 70 B. Patry, ‘Who’s Afraid of Evidence-Based Copyright Law?’, Suffolk University Law Review, Vol. 1, 2013, p. 55.

• 71 R.J. Cook et al., ‘Emergency Contraception, Abortion and Evidence-Based Law’, International Journal of Gynecology & Obstetrics, Vol. 93, 2006, p. 191.

• 72 K. Shibuya, C. Ciecierski, et al., ‘WHO Framework Convention on Tobacco Control: Development of an Evidence Based Global Public Health Treaty’, BMJ, Vol. 327, 2003, p. 154.

• 73 Id. at 155, citing H.A. Waxman, ‘The Future of the Global Tobacco Treaty Negotiations’, New England Journal of Medicine, Vol. 346, 2002, pp. 936-939.

• 74 Id.

• 75 At current levels of consumption, the burden from tobacco is estimated to double by 2020. Id. at 155 (citing, ‘The World Health Report 2002: Reducing Risks, Extending Healthy Life’, WHO [2002]).

• 76 Id. at 155.

• 77 WHO used a standard method to study 14 sub-regions of the world, while “accounting for subregional differences in demographic, epidemiological, and cost characteristics, and…specific information on exposure.” Id. at 155.

• 78 Id. at 155.

• 79 Id. at 156 (citing F.J. Chaloupka, T.W. Hu, K.E. Warner, R. Jacobs, & A. Yurekli, The Taxation of Tobacco Products in Tobacco control in Developing Countries, Oxford, Oxford University Press, 2000, p. 237-272; G.E. Guindon, S. Tobin, & D. Yach, ‘Trends and Affordability of Cigarette Prices: Ample Room for Tax Increases and Related Health Gains’, Tobacco Control, Vol. 11, 2002, pp. 35-43).

• 80 S.K. Urahn, ‘A Tipping Point on Evidence-Based Policymaking’, Governing, October 27, 2015, <www.governing.com/columns/smart-mgmt/col-state-local-government-tipping-point-evidence-based-policymaking.html>. Washington State Institute for Public Policy, <www.wsipp.wa.gov/About>.

• 81 The Initiative, a joint project of The Pew Charitable Trusts and the John D. and Catherine T. MacArthur Foundation, started in 2011. See, The Pew Charitable Trusts, <www.pewtrusts.org/en/projects/pew-macarthur-results-first-initiative>.

• 82 Id.

• 83 The Pew Charitable Trusts, <www.pewtrusts.org/en/projects/pew-macarthur-results-first-initiative/where-we-work>. To enter the Initiative, Initiative staff talks with state representatives to discuss evidence-based legislation and determine if the state is committed to an evidence-based system, can provide the data needed to develop a cost–benefit analysis model for the state, and is willing to dedicate the staff and resources necessary to succeed with the Initiative. The Pew Charitable Trusts, <www.pewtrusts.org/~/media/assets/2013/results-first-in-your-state-brief.pdf?la=en>.

• 84 The Pew Charitable Trusts, <www.pewtrusts.org/en/research-and-analysis/fact-sheets/2015/07/the-pew-macarthur-results-first-initiative>.

• 85 Id.

• 86 Id. These studies utilize the Results First Clearinghouse Database, an online archive of research on state programmes across the country, to assess the likely effectiveness of the state’s current initiatives.

• 87 Id.

• 88 Id. For example, New Mexico now requires state executive agencies to support budget increase requests with evidence supporting programme effectiveness. Similarly, New Mexico has used the Initiative’s techniques since 2013 to develop and utilize customized cost–benefit model programmes involving criminal and juvenile justice, early childhood and child welfare. Through this process, New Mexico was able to allocate $104.4 million to its most effective programmes. Id.

• 89 Pub. L. No. 114-140 See, <https://www.whitehouse.gov/omb/management/commission_evidence>.

• 90 <www.speaker.gov/press-release/evidence-based-policy-commission-gets-to-work>.

• 91 The appointees were divided between the President, the Speaker of the House, The Senate Majority Leader and the Senate and House Minority Leaders. The bill required that appointments should be made to ‘Individuals with expertise in economics, statistics, program evaluation, data security, confidentiality, or database management.’ Pub. L. No. 114-140 Sec. 3 (b).

• 92 Id.

• 93 <https://www.regulations.gov/document?D=USBC-2016-0003-0001>. One commentator urged the Commission to find methods for incorporating randomized evaluations into the design of government programmes similar to the trials that ‘revolutionized modern medicine in the 20th century.’ The key to such randomized evaluations is the availability of administrative data from ‘hospitals, governments, school systems, and other institutions’ with little added cost to researchers and more possibility for long-term follow-up. Q. Palfrey, ‘Promoting Policies That Work: Six Steps for the Commission on Evidence-Based Policymaking’, The Hill, 1 November 2016. (<http://thehill.com/blogs/congress-blog/the-administration/303781-promoting-policies-that-work-six-steps-for-the>). Palfrey points out that use of administrative data had been successfully used to make informed policy decisions.

• 94 Id.

• 95 Id. at 406 (citing, Ward, Applied Sociology, Boston, Ginn & Company, 1906, p. 338).

• 96 Evidence-Based Medicine advocates consider controlled experimental work, particularly randomized controlled trials, to be the most reliable evidence. See, D.L. Sackett et al., ‘Evidence Based Medicine: What It Is and What It Isn’t’, BMJ, Vol. 312, 1996, p. 71. This form of testing medical practices, and especially new drugs, utilizes a control group for comparison, which does not receive the new treatment, but receives a reference treatment or placebo. Further, the decision about whether a patient receives the new treatment or is a part of the control group is made randomly. See, M. Macgill, ‘What is a Randomized Controlled Trial in Medical Research?’, Medical News Today, <www.medicalnewstoday.com/articles/280574.php>.

• 97 D.K. Baker, J. Skolnick, G. Doucette, G. Levitt, & R. Baker, ‘Intensive Parole Supervision of the Sex Offender—Putting the Containment Approach Into Practice’, Handbook of Sex Offender Treatment, Barbara K. Schwartz, ed. 2011, pp. 62-1-62-15, <https://www.ncjrs.gov/App/Publications/abstract.aspx?ID=265228>.

• 98 This programme uses some or all of the following supervision techniques: mandatory counselling, electronic monitoring, unannounced home and work visits, curfews, polygraph testing, random urinalysis, travel restrictions, mandatory daily diaries, and sex offender registration. Parole Practices in Massachusetts and Their Effect on Community Reintegration’, Boston Bar Association Task Force on Parole & Community Reintegration, August 2002, p. 9 (<https://www.bostonbar.org/prs/reports/finalreport081402.pdf>).

• 99 ‘Intensive Parole for Sex Offenders’, Massachusetts Parole Board (2002) (on file with the author). During that time there have been 114 offenders under supervision; only four have been returned to custody, none for a sex offense; Boston Bar Association Task Force on Parole Report, 2002, p. 9.

• 100 Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism Act, Pub. L. No. 107-56, 115 Stat. 272 (2001).

• 101 When originally passed in 2001, portions of the PATRIOT Act were to sunset on December 31, 2005.

• 102 See, USA PATRIOT Improvement and Reauthorization Act of 2005, Pub. L. No. 109-177 (2006); USA PATRIOT Act Additional Reauthorizing Amendments Act of 2006, Pub. L. No: 109-178 (2006); PATRIOT Sunsets Extension Act of 2011, Pub. L. No. 112-14, Pub. L. No. 112-14 (2011). The Act was allowed to lapse due to a lack of Congressional action in June 2015; and later reenacted in the USA Freedom Act, Pub.L. 114–23 Title II, sec. 201-202 (2015), which extended certain provisions until 2019.

• 103 See, USA Freedom Act, Pub.L. 114–23 Title II, sec. 201-202 (2015).

• 104 New State Ice Co. v. Liebmann, 285 U.S. 262 (1932) (Brandeis, J. dissenting) (“It is one of the happy incidents of the federal system that a single courageous State may, if its citizens choose, serve as a laboratory; and try novel social and economic experiments without risk to the rest of the country.”).

• 105 An Act Relative to Legalized Recreational Marijuana for Individuals At Least 21 Years Old, 2016 Mass. Acts ch. 334, (Approved by the People at the State Election on November 8, 2016, effective in part December 15, 2016).

• 106 J. Miller, ‘Marijuana Dispensary among State Senators’ Stops in Colorado’, Boston Globe, 11 January 2016, <https://www.bostonglobe.com/metro/2016/01/10/mass-senators-colorado-study-marijuana/luFCdZnQoUyw45GwOBvgdO/story.html>. The senators were to tour a cultivation facility, a marijuana dispensary and to question “state, municipal and law enforcement officials” with questions about Colorado’s voter-approved law. Id.

• 107 Id. In that case, a dispensary for patients did not open until 2015.

• 108 Id. (commentary of Tufts University Prof. Jeffery M. Berry).

• 109 See, <www.ncsl.org/aboutus.aspx>.

• 110 See, 2 U.S.C. sec. 604 (2014).

• 111 <https://www.cbo.gov/about/overview>. CBO does not make policy recommendations, and each report and cost estimate summarizes the methodology underlying the analysis. Id.

• 112 See, Pub. L. 91-510, title III, sec. 321(a), 26 October 1970, (codified at 84 Stat. 1181; 2 U.S.C. 166.)

• 113 Congressional Research Service (<www.loc.gov/crsinfo/about/>). CRS employs more than 400 are policy analysts, attorneys and information professionals working across a variety of disciplines. See, <www.loc.gov/crsinfo/research/>.

• 114 Congressional Research Service, ‘Annual Report Fiscal Year 2015’, (<www.loc.gov/crsinfo/about/crs15_annrpt.pdf>).

• 115 The Affordable Care Act (‘Obamacare’) was enacted through two bills: Patient Protection and Affordable Care Act (PPACA), Pub. L. No. 111-148, 124 Stat. 119 (2010) and the Health Care and Education Reconciliation Act of 2010 (HCERA) Pub. L. No. 111-152, 124 Stat. 1029 (2010).

• 116 The Financial Services Committee issued one of only two reports on health care reform when considering a precursor to the ACA, America’s Healthy Future Act of 2009, S. Rep. No. 111-89 (2009) (‘Fin. Comm. Report’). See, 124 Stat. at 1083. The Congressional Budget Office predicted that the legislation will reduce the deficit by $81 billion in the first 10 years, will provide health insurance to 29 million Americans, increase the rate of insurance to 94 per cent, and would ultimately cost of $829 billion. Id. at 9. The Report also cites the CBO when it states that health care costs are the single most important factor influencing the Federal Government’s long-term fiscal balance. Id. at 2.

• 117 The CBO estimated that 23 million would be on the exchanges by 2016, but the number was actually 10 million. CBO projected that 30 million people (or 11 per cent of the population under 65) would not have health insurance in 2016, and the actual number turned out to be 27.9 million (10.3 per cent). The Washington Post, March 14, 2017, <https://www.washingtonpost.com/news/fact-checker/wp/2017/03/14/fact-checking-the-white-houses-rhetoric-on-the-cbo-report/?utm_term=.cdbab5fbe57c>.

• 118 The Washington Post, 2017.

• 119 ILTAM, 2009, pp. 454-455.

• 120 ILTAM, 2009, p. 455.

• 121 Id. (Only after the drafter analyses the information gathered after enactment, the ‘law-in-action,’ can improvements be drafted, debated and enacted to ‘ensure the law works.’).

• 122 O. Hetzel, M. Libonati, & R. Williams, Legislative Law and Statutory Interpretation, 4th ed., Newark New Jersey, LexisNexis, 2008, p. 283.

• 123 Watkins v. United States, 354 U.S. 178 (1957).

• 124 Id. at 187.

• 125 See, J. Hamilton, R.F. Muse, & K.R. Amer, ‘Congressional Investigations: Politics and Process’, American Criminal Law Review, Vol. 44, 2007, pp. 1115, 1124-1126; M.E. O’Neill, ‘The Fifth Amendment in Congress: Revisiting the Privilege Against Compelled Self-Incrimination’, Georgetown Law Journal, Vol. 90, 2002, p. 2445.

• 126 2 U.S.C. § 191 (2000).

• 127 See, Hamilton et al., 2007, pp. 1126-1129. Perjury prosecutions may be brought under federal law, 18 U.S.C. §1621 (2000), or under the District of Columbia law, D.C. Code Ann. § 22-2402 (2001), or under the relevant state law. Generally the standard for prosecution is that a false statement be made before a competent tribunal, which includes Congressional committees, that the statement be made wilfully, and that it concern a material matter. Id.

• 128 Id. at 1129-1131.

• 129 Id. at 1132-1137.

• 130 See, ILTAM, 2009; Seidman & Seidman, 2011; Seidman, et al., 2001 and the accompanying texts.

• 131 Id. at 5-8.

• 132 Daubert, 1993, pp. 593-594.

• 133 Id.

• 134 Id. at 594-595.

• 135 Id.; Sheehan, 1997.

• 136 General Elec. Co. v. Joiner, 1997.

• 137 Claar v. Burlington, 1994; Ambrosini, 1996.

• 138 Kumho Tire, 1999, p. 1175.

• 139 Megan’s Law is officially known as the Sexual Offender Registration Act of 1994, NJ Pub. L. ch. 128 and (codified as N.J.S.A. 2C:7-1 through 2C:7-11.). After New Jersey adopted Megan’s Law, several other states followed with similar statutes. In 1996, the Congress enacted a federal version of the law. See, Megan’s Law, Pub. L. No. 104-145, 110 Stat. 1345 (1996) (amending 42 U.S.C. § 14071(d) (1994). D. Filler, ‘Making the Case for Megan’s Law: A Study in Legislative Rhetoric’, Indiana Law Journal,Vol. 76, 2001, pp. 315, 352.

• 140 Id. at 335, citing, 142 Cong. Rec. 10,313 (1996) (statement of Rep. Jackson-Lee).

• 141 Id. at 335, citing, 139 Cong. Rec. 31,251 (1993) (statement of Rep. Ramstad).

• 142 Id. at 335, citing, 142 Cong. Rec. 18,765-66 (1996) (statement of Sen. Hutchinson).

• 143 Id. at 353.

• 144 Id. Using a definition of abuse and neglect from the Children’s Trust Fund of Texas, Filler concludes, “The degree to which this broad classification exceeds the scope of Megan’s Law is virtually self-evident. Megan’s Law would do nothing to protect children from irresponsible parents, for instance.”

• 145 Id. at 353, citing D. Finkelhor et al., Missing, Abducted, Runaway, and Throwaway Children in America: First Report: Numbers and Characteristics National Incidence Studies 4, 66 (1990). This study was commissioned by the US Office of Juvenile Justice and Delinquency Prevention. The reports’ definition of ‘abduction’ included cases of minimal coercion, very brief detentions, and non-family perpetrators such as acquaintances and babysitters. Id.

• 146 Id. at 353-354.

• 147 Id. at 354.

• 148 V. Dabbaghian & V.K. Mago, ‘What is Mathematical Modeling?’, in V. Dabbaghian & V.K. Mago (Eds.), Theories and Simulations of Complex Social Systems, New York, Springer, 2014 (<www.sfu.ca/~vdabbagh/Chap1-modeling.pdf>).

• 149 D. Zenghelis, ‘What Do Economic Models Tell Us?’, London School of Economics, Grantham Research Institute on Climate Change and the Environment, <www.lse.ac.uk/GranthamInstitute/news/why-economic-models-tell-us-so-little-about-the-future-2/>.

• 150 Dabbaghian & Mago, 2014.

• 151 Zenghelis, <www.lse.ac.uk/GranthamInstitute/news/why-economic-models-tell-us-so-little-about-the-future-2/>.

• 152 Id.

• 153 Id., “Models are used for two main purposes: simulating (e.g. how would the world change relative to some counterfactual if we assume a change in this or that variable) and forecasting (e.g. what the world might look like in 2030)”. See also, Dabbaghian & Mago, 2014.

• 154 V. Dabbaghian, P. Jula, P. Borwein, E. Fowler, C. Giles, N. Richardson, A.R. Rutherford, A. Van der Wall, ‘High-Level Simulation Model of a Criminal Justice System’, in V. Dabbaghian & V.K. Mago (Eds.), Theories and Simulations of Complex Social Systems, 2014.

• 155 Id.

• 156 Zenghelis (“key inputs are often chosen arbitrarily and even the best model spits rubbish out if you pump rubbish in.”)

• 157 Id. (“The further out the forecast, the larger the structural uncertainties making model projections at best illustrative”). Zenghelis points out that macroeconomic models of an economy used by finance ministries banks and central banks rarely look beyond a four-year horizon and are essentially used as a ‘consistency check and not a source of projections.’

• 158 D.H. Freedman, ‘Why Economic Models are Always Wrong’, Scientific American, 26 October 2011 (<https://www.scientificamerican.com/article/finance-why-economic-models-are-always-wrong/>).

• 159 Id.

• 160 Id. (noting that it has become routine for modellers in finance to keep recalibrating their models over and over again and still produce bad predictions.)

• 161 E.S. Mills, ‘The Misuse of Regional Economic Models’, Cato Journal, Vol. 13, Spring /Summer 1993, p. 29. Mills points out that the economic model he examines is the use of the REMI model, ‘one of the very best regional impact models extant,’ which he characterizes as widely used, and easily evaluated since it is ‘available in explicit detail.’ But even analysis from this reliable model can mislead the public into thinking that there are benefits to government projects that would not come from similar private projects. Id. at 38.

• 162 Zenghelis.

• 163 Id. at 5-8.

• 164 See, V.R. Johnson, ‘Regulating Lobbyists: Law, Ethics, and Public Policy’, Cornell Journal of Law and Public Policy, Vol. 16, 2006, pp. 1, 5-10 (Lobbying is directly tied to the right to petition in the 1st Amendment of the US Constitution: “In some instances, it is necessary or appropriate for persons seeking to petition the government to channel their efforts through volunteer or paid intermediaries.” Id. at 8. ‘Special interests’ is defined broadly here as any advocacy group: a corporate or business interest, a labour union, a political party, a trade association, a charity, a church, or a municipality. Id. at 10, Congressional Quarterly’s Guide to Congress 700-715 (5th ed. 2000); A. Rosenthal, The Third House: Lobbyists and Lobbying in the States, 2nd ed., Washington DC, C.Q. Press, 2001, pp. 61-63.

• 165 Johnson, 2006, p. 9.

• 166 Rosenthal, 2001, p. 195 (Rosenthal quotes a New Jersey lobbyist as saying, “Getting the right information to the right people at the right time…It’s a home run to make that combination work.”)

• 167 Id. at 194 (“Today’s lobbyist will invariably have to make an argument on the merits.”). W.K. Muir, Jr., Legislature: California’s School for Politics, Vol. 19, 1982 (“What a lobbyist does, therefore, is to provide basic information—quickly and intelligibly.”)

• 168 Id. at 9.

• 169 Id. at 196.

• 170 Id. Rosenthal cites a Florida lawyer-lobbyist with an expertise in land use who is very much in demand with Florida legislators and a California lobbyist whose reputation for providing reliable evidence, makes his evidence especially useful for the record. Many of the lobbyists interviewed stated that, “People trust us.” For example, a Florida lobbyist spent his career working with alcoholic beverages, including time directing the division of alcoholic law in a previous gubernatorial administration, has made himself a “trusted source” of information on alcohol-related issues. Id. at 195-196.

• 171 Id.

• 172 Id. Rosenthal points out that “Policy and fiscal analyses are in great currency.”

• 173 Id. at 197. At times this will be in public testimony and at other times at one-on-one meetings with the policy makers. Rosenthal gives the example of a Maryland lobbyist who set up a panel of finance and budget experts to explain the effects of a proposed tax-collection bill. This information led to the legislature establishing a study group to further study the matter.

• 174 Id.

• 175 D.A. Singleton, ‘Kids, Cops, and Sex Offenders: Pushing the Limits of the Interest-Convergence Thesis’, Howard Law Journal,Vol. 57, 2013, pp. 353, 390.

• 176 See, Rachlinski, 2011 and the accompanying discussion of naïve beliefs.

• 177 See, e.g. B. Foley & R. A. Robbins, ‘Fiction 101: A Primer For Lawyers on How to Use Fiction Writing Techniques to Write Persuasive Fact Sections’, Rutgers Law Journal, Vol. 32, 2001, pp. 459, 465 (2001) (quoting, D. Ball, Theater Tips and Strategies for Jury Trials 66 (1994); See also R.A. Robbins, ‘Harry Potter, Ruby Slippers and Merlin: Telling the Client’s Story Using the Characters and Paradigm of the Archetypal Hero’s Journey’, Seattle University Law Review,Vol. 29, 2006, p. 767; Clare Keefe Coleman, ‘Dangerous Tongues: Storytelling in Congressional Testimony and an Evidence Based Solution’, New York University Journal of Legislation and Public Policy,Vol. 19, 2016, pp. 291, 296-301.

• 178 Foley & Robbins, 2001 (quoting Sol Stein, Stein on Writing: A Master Editor of Some of the Most Successful Writers of our Century Shares His Craft Techniques and Strategies 224 (1995); Robbins, 2006, p. 769, quoting S.L. Winter, ‘The Cognitive Dimension of the Agony Between Legal Power and Narrative Meaning’, Michigan Law Review, Vol. 87. 1989, pp. 2225, 2228. See also, Filler, 2001, p. 346 (citing S. Bandes, ‘Empathy, Narrative, and Victim Impact Statements’, University Chicago Law Review, Vol. 63, 1996, pp. 361, 383 (“We make sense of the world by ordering it into metaphors, and ultimately narratives with familiar structures and conventions”). See also, Coleman, 2016, p. 298 (Stories can make complex ideas easy, and they are hard to refute.)

• 179 Coleman, 2016, p. 300 (“Storytelling is, in a sense, the heart of political reasoning.”) (narrative stories are the principal way policy problems are defined and contested). Id. citing D. Stone, Policy Paradox: The Art of Political Decision Making, 3rd ed., New York, WW Norton & Company, 2012, p. 158.

• 180 The Jacob Wetterling Crimes Against Children and Sexually Violent Offenders Registration Act, Pub. L. No. 103-322, tit. XVII, §170101, 18 Stat. 1796, 2038 (1994) (codified as amended at 42 U.S.C. §14071 (1994 & Supp. IV 1998). Megan’s Law, or the Sexual Offender Registration Act of 1994, NJ Pub. L. ch. 128 and (codified as N.J.S.A. 2C:7-1 through 2C:7-11.); the federal statute is: Megan’s Law, Pub. L. No. 104-145, 110 Stat. 1345 (1996) (amending 42 U.S.C. § 14071(d) (1994).

• 181 On July 29, 1994, 7-year-old Megan Kanka was raped and murdered by a neighbour with two prior convictions for sexual offenses against children. Filler, 2001, p. 315. In October 1989, Danny Heinrich abducted, sexually assaulted and killed 11-year-old Jacob Wetterling. In September 2016, Heinrich confessed to the killing. P. Louwagie & J. Brooks, ‘Danny Heinrich Confesses to Abducting and Killing Jacob Wetterling’, Minnesota Star Tribune, 7 September 2016, <www.startribune.com/danny-heinrich-confesses-to-abducting-and-killing-jacob-wetterling/392438361/\.

• 182 Filler, 2001, p. 316. Only one representative in Congress spoke in opposition to the bill and the final vote in the House of Representatives was 418-0. Several states, such as Florida did not even have a debate about the bill before passing Megan’s Law unanimously. Id. at 316-317.

• 183 Id. at 332-333. Interestingly, one legislator cautioned about excessive reliance on the ‘emotionally powerful’ Megan Kanka story to the exclusion of logic and reason. Id. at 332, n. 105 (citing, N.Y. Senate Minutes of S-11-B, at 6624 [May 24, 1995] [statement of Senator Leichter]).

• 184 Id. at 334 (citing, N.Y. Assembly Minutes of A1059C, at 342-46 [June 28, 1995] [statement of Mr. Spano]).

• 185 Id. at 350.

• 186 Coleman, 2016, p. 298 (quoting P. Ewick & S.S. Silbey, ‘Subversive Stories and Hegemonic Tales: Toward a Sociology of Narrative’, Law & Society Review, Vol. 29, 1995, pp. 197, 205.

• 187 For instance, the debate surrounding an amendment to the FDA Food Safety and Modernization Act, Pub. l. No. 111-353, 124 Stat. 3885 (2011) (codified as amended in scattered sections of the US Code.), which was meant to protect small farmers, was only narrative, with “No data, no study, and no science” to back up the proponent’s assertions. Coleman, 2016, p. 318.

• 188 Filler, 2001, p. 350, (citing, D.A. Farber & S. Sherry, Beyond All Reason: The Radical Assault on the Truth in American Law, 89 (1997) (‘it’s hard to say anything critical about the story without implicating the storyteller.’). If legislators argued Megan’s Law would not have prevented her death, that would have directly challenged her parents. Id. at 351.

• 189 Id. at 352.

• 190 Id. Filler proposes that legislatures consider adopting a ‘code of debate’ that would encourage legislators to make honest claims and address all policy concerns. This code could also require legislators to ‘tell stories ethically.’ Id. at 365. This would require storytellers to: (1) rely on a broad factual basis, (2) demonstrate a regard for interpersonal complexities, (3) emphasize the psychological apparatus and states of mind of the participants and (4) acknowledge the narrator’s bias. Id. (citing, D.D. Troutt, ‘Screws, Koon, and Routine Aberrations: The Use of Fictional Narratives in Federal Police Brutality Prosecutions’, New York University Law Review, Vol. 74, 1999, pp. 18, 96. It is hard to see, however, how such an ethics code squares with the principles of free speech and how such a code would be regulated or enforced in the legislative context.

• 191 Daubert, 1993.

• 192 Filler, 2001, p. 365.

• 193 Id.

• 194 Coleman, 2016, p. 331.

• 195 Id. at 332 (citing, ‘Evidence-Based Practice’, Cal. Cts, <www.courts.ca.gov/5285.htm>).

• 196 Id. at 333. Coleman admits that not all theories would be testable, but points out that empirical methods can be used to test the reasons people offer to support or oppose a rule. Id.

• 197 W. Rhee, ‘Entitled to be Heard: Improving Evidence-Based Policy Making Through Audience and Public Reason’, Indiana Law Journal, Vol. 85, 2010, pp. 1315, 1318.

• 198 See, ‘Instrumentalism 2.0’ and accompanying text.