On Identifying Assumptions Underlying Legal Arrangements

Legal arrangements rest on behavioural, cognitive, social, and other assumptions regarding their role and function in society and the legal system. The identification and subsequent evaluation of these assumptions is an important task for legal scholarship. In this article, we focus on the identification and categorisation of these assumptions, providing conceptual distinctions and methodological guidance. We distinguish between assumptions about the value(s), norm(s), or interest(s) underlying a legal arrangement, which can be legal or non-legal, and assumptions about the relationship between the legal arrangement and its underlying value(s), norm(s), or interest(s), which can be logical, causal, or contributory. Regarding the identification, we consider explicit references and inference to the best explanation and theory-driven evaluations as possible methods. Inference to the best explanation, we posit, functions as a manner of reconstructing the theory that the person(s) creating a legal arrangement had in mind regarding the place and function of that legal arrangement in society. Given this, we offer a step-by-step approach to reconstructing this theory in use, drawing from theory-driven evaluations and its sources in the social sciences. These distinctions and guidelines can contribute to understanding the context and untangling the complexities involved in identifying the assumptions that underlie legal arrangements.

• 1. Introduction

  In 1996, the World Bank revised its guidelines to state explicitly that corruption would be grounds for cancelling a (borrowing) contract (Elliott, 1997). Around the same time, the OECD signed an international convention to criminalize transnational bribery (Rose-Ackerman, 1999). To curb governmental corruption, the World Bank proposed fundamental governmental reforms, which include institutions such as regulatory authorities, taxation agencies, the judiciary and other public institutions (Rose-Ackerman, 1999). To support the implementation of these loan-dependent reforms, the World Bank Group’s anti-corruption programme also focused on raising awareness and involving civil society (Pope, 1996), with a view to ‘provid[ing] participants from developing countries with the tools to develop a participatory and integrated national action plan of institutional reforms to combat corruption’ (World Bank, 1999). Underlying this was the assumption that ‘strengthening institutional capacity and enhancing national integrity (systems) are important’ to combat corruption, as well as ‘assumptions about causal mechanisms underlying the functioning of national integrity and good governance’ (Leeuw, van Gils & Kreft, 1999: 201). Given the Bank’s non-political mandate, there were debates not only about the effectiveness of anti-corruption activities, but also about whether addressing the issue of corruption would violate the Bank’s mandate (Marquette, 2007: 29).
  This article is not about the World Bank’s anti-corruption programme, but we start with this programme because it demonstrates that in creating and implementing legal arrangements, policies and programmes,¹x For the sake of brevity, we will speak mainly of legal arrangements from now. This should be read as encompassing policies and programmes as well. inevitably one makes behavioural, cognitive, social and other assumptions: the person(s), organization(s), or institution(s) creating and implementing a particular legal arrangement necessarily make assumptions about the place and function of this legal arrangement, policy or programme in society and within the legal system. Our aim in this article is to provide legal(-doctrinal) scholars with conceptual distinctions and methodological rules of thumb to identify the assumptions underlying legal arrangements.
  We will proceed in three steps: Section 2 of this article addresses the question of why it is important to identify the assumptions underlying legal arrangements and who should do so. Section 3 is dedicated to the conceptual toolkit for identifying these assumptions; Section 4 considers different approaches (and rules of thumb) to identifying the assumptions and discusses some (methodological) pitfalls that play a role in doing so. Section 5 briefly touches on additional complicating factors, before we conclude in Section 6.
  Some terminological notes are in order: we generally use the term ‘assumptions’ in this article. However, Section 4, in particular the part on theory-driven evaluations, draws from the social sciences where (‘intervention’) theories (‘of change’) are more commonly used to denote the same idea. For our present purposes, these terms are considered interchangeable. Moreover, we will use the World Bank’s anti-corruption programme as a recurring example, but the conceptual distinctions and methodological rules of thumb we aim to introduce are of a more general nature, applicable also to examples from legislation or public policy (with the legislator or public officials as the ones making these assumptions) to contracts and private parties making assumptions about, for example, the causal effects that concluding the contract will have.

• 2. The Why and the Who (Rationale)

  Our aim in this article is to provide scholars engaged in legal(-doctrinal) research with conceptual distinctions and methodological rules of thumb to identify the assumptions underlying legal arrangements. A first question one might ask is why anyone should want to identify the assumptions underlying legal arrangements; a second question is why scholars engaged in legal(-doctrinal) research should.
  We focus on the assumptions underlying legal arrangements in this article because of the following: when policies and legal arrangements are developed and implemented, assumptions play a role. Some of these assumptions can be encapsulated in the form of ‘proverbial folk wisdoms’ (Elster, 2007), in ‘pet theories’ (of administrators or policy makers²x ‘Each [family planning] administrator has his own style which means that each administrator has his own pet theories …which he uses over and over in carrying out his work’ (Bogue, 1974: 1-2).) (Bogue, 1974) or in ‘implicit theories’ (of judges) (Quintanilla, 2013). The degree to which these assumptions are accurate or justified has an impact on whether the legal arrangement functions as intended, but also with its place in the larger legal system. This means that it is relevant to anyone creating or implementing a legal arrangement for a particular purpose to critically reflect on the assumptions they are making, but it also means that it is an important task for social scientists and legal scholars to evaluate these assumptions and the degree to which they are accurate and justified. Before one can evaluate these assumptions, however, they first need to be identified. It is not the case that the assumptions behind legal arrangements are always made explicit and transparent. Even when attempts are made to do so, for example in explanatory memoranda, assumptions relating to behavioural and cognitive mechanisms or logical relations³x More on these in Section 3. often remain ‘hidden’. This can also be true for studies that aim to investigate assumptions: Giesen et al. (2019), for example, address the question of whether eleven assumptions found in (Dutch) case law are correct, in the sense of having sufficient empirical basis, but do not discuss how these assumptions were identified. While it may at times seem obvious what the legislator’s espoused assumptions are, their implicit assumptions (which are actually in use) may not be clear. This poses the risk that certain assumptions are not studied and/or that researchers make their own assumptions and substitute their own gut feelings for those of the legislator. In our view, this would be troubling, as it negatively impacts the repeatability and reliability of the study, particularly considering that there is an increasing consensus that gut feelings often do not provide us with accurate information (cf. Carruthers, 2009; Caruso, 2015; Kahneman, 2011). If, as we do, one considers it to be important that the assumptions underlying legal arrangements are evaluated, it is also important that they are first identified in a conceptually and methodologically clear manner.
  This explains why we focus on identifying the assumptions underlying legal arrangements; it does not yet explain why we think it is important that scholars engaged in legal(-doctrinal) research have the means to do so.
  The nature and scope of legal research are at times unclear and contested (Taekema & van der Burg, 2015; van Hoecke, 2011). Legal research often considers the fit of a particular legal arrangement with one or several norm(s) or value(s) of the legal system (Westerman, 2011). This includes considering the logical relationship between the legal arrangement and these norm(s) and value(s); this relationship is one legislators and others might make implicit assumptions about, as we will see in Section 3. Evaluating legal arrangements against moral standards and considering the conceptual and logical links between legal arrangements and moral or legal values and norms has a long-standing tradition in legal philosophy and theory as well. Increasingly, empirical legal studies, law and economics, or legal psychology studies are bringing the evaluation of causal or contributory mechanisms underlying a legal arrangement into the purview of legal scholars in addition to social scientists as well. These activities of legal scholars involve the assumptions underlying legal arrangements; as such, there is a need for legal scholars to have the means to identify these assumptions.
  In the next section, we will turn first to the conceptual distinctions relevant to categorizing different kinds of assumptions.

• 3. Different Types of Assumptions

  We distinguish between different types of assumptions: the first type of assumption concerns the abstract norm,⁴x As a matter of terminology, we consider norms to be those rules with a deontic conclusion, for example, rules that impose duties or rules that create obligations (Hage, 2018a: 165; 2018b). An abstract norm in this connection could be a higher constitutional norm or norm of international public law, but it could also be a norm of morality or religion. value or interest that the legal arrangement in question aims to make more concrete, promote or pursue. In implementing and maintaining its anti-corruption programme, for example, the World Bank (2022) assumes that combatting corruption is valuable:

  The World Bank Group considers corruption a major challenge to its twin goals of ending extreme poverty by 2030 and boosting shared prosperity for the poorest 40 percent of people in developing countries.

  Corruption has a disproportionate impact on the poor and most vulnerable, increasing costs and reducing access to services, including health, education and justice. Corruption in the procurement of drugs and medical equipment drives up costs and can lead to sub-standard or harmful products. The human costs of counterfeit drugs and vaccinations on health outcomes and the life-long impacts on children far exceed the financial costs. Unofficial payments for services can have a particularly pernicious effect on poor people.

  The first paragraph indicates that it is assumed that this is not an intrinsic value (fighting corruption is valuable in and of itself) but an instrumental one (fighting corruption is valuable because corruption is a challenge to ending poverty and boosting shared prosperity), although the degree to which this assumption has been held has been subject to controversy and debate (Marquette, 2007: 31).
  The second type of assumption concerns the relationship between the legal arrangement and the realization of the (more abstract) norm, value or interest in society. This relationship can be causal or contributory in nature, or it can be logical/conceptual. Simply put, a causal relationship is one in which A causes B. A contributory relationship, meanwhile, is one where A contributes to B; as such, contribution may be termed a weaker variant of causation.⁵x We will leave aside the question of where to draw the line between and how to distinguish causation, contribution and correlation. The World Bank’s anti-corruption programme (as referred to previously) features numerous examples of assumptions regarding the causal relationship between specific legal arrangements or interventions and the underlying norm, value or interest: one element of the anti-corruption programme, for example, is the strengthening of civil society involvement in the fight against corruption by, inter alia, offering interactive workshops to civil society actors. Here, one can identify assumptions such as that interactive workshops would be effective in strengthening civil society actors and that stronger civil society actors would help curb or prevent corruption (Leeuw et al., 1999). Generally, when it comes to assumptions about the causal and contributory relationship between the legal arrangement and the underlying value, norm or interest, we are referring to assumptions (or ‘intervention theories/theories of change’) regarding behavioural, cognitive and/or social mechanisms.
  Not all assumptions concerning the relationship between the legal arrangement or concrete intervention and the more abstract norm, value or interest are related to causal or contributory mechanisms: conceptual or logical assumptions can also play an important role. A logical or conceptual relationship means that, for example, B falls under A: considering accepting bribes a form of corruption assumes that there is a logical relationship between corruption and accepting bribes and that instances of accepting bribes are instances of corruption. This is not a causal or contributory relationship: it is not assumed that accepting bribes causes corruption, but that it is a form of corruption (Figure 1).

  Logical/conceptual relationship between accepting bribes and corruption
  

  In a similar vein, one can identify an assumption relating to the logical/conceptual relationship between combatting corruption and the World Bank’s mandate as laid down at the time in its Articles of Agreement: the Articles of Agreement (1945) stipulate that the World Bank’s mandate is non-political and that only economic considerations shall be relevant to its decisions. In developing an anti-corruption programme, the World Bank apparently makes the assumption that fighting corruption is an economic matter. This, however, is not an assumption shared by all (Marquette, 2007: 29).
  Lastly, although not focusing specifically on assumptions themselves, it is useful to distinguish between the creation of a legal arrangement and its implementation:⁶x Although we think it is useful to draw this distinction between creating and implementing a legal arrangement, the two cannot always be neatly separated, and those implementing legal arrangements are often creating rules by doing so themselves, cf. Kelsen et al. (1992), Pottie and Sossin (2005). a legislature may make certain assumptions when drafting and passing a piece of legislation, while those tasked with implementing or interpreting it may make other assumptions – or ascribe different assumptions to the legislature.

  Table 1 summarizes these distinctions:

  Table 1 Types of Assumptions

  Assumptions about…
  The value(s), norm(s), or interest(s) underlying a legal arrangement	The relationship between the legal arrangement and the underlying V/N/I
  For example: Constitutional or international norms, moral, political, religious values	Logical: X makes Y more concrete; X is a part of Y
  	Causal: X leads to Y
  	As written	As implemented
  	Contributory: X contributes to Y
  	As written	As implemented

  The distinctions between these different kinds of assumptions may help in untangling and classifying complex sets of assumptions, but they do not yet say anything about how to identify assumptions in the first place. We turn to this question in the next section.

• 4. Identifying Assumptions

  When it comes to identifying assumptions, we distinguish three different possibilities to do so, although these three options cannot, as will become clear, be as neatly separated in reality as we present them here. The three possibilities we will discuss are

  1. Explicit references

  2. Inference to the best explanation

  3. Theory-driven evaluations (TDE)

  1. Explicit references

  Some legal arrangements – especially those legal arrangements that are made by public bodies, rather than private entities – contain explicit references on the rationale for their adoption. Moreover, there may be explanatory documents. Here, one may think of preambles, parliamentary proceedings, memoranda of explanation, preparatory works of treaties or comparable (textual) sources. In some cases, one or several of these sources will explicitly refer to assumptions made.
  This is most likely the most obvious and seemingly easiest manner of identifying the assumptions underlying a legal arrangement, but it is not without its pitfalls: first, where is the line between what is explicit and what is not, and to what extent do indications or vague(r) references count? Secondly, do explicit references offer any insight into the real assumptions that underlie the legal arrangement in question? If we distinguish between the assumptions put forward explicitly in a preamble and the real assumptions underlying the arrangement, we are distinguishing between what is more commonly called espoused theories and theories in use: espoused theories are those that are put forward or that an individual, organization or institution claims to follow; theories in use are those that can be inferred from action (Argyris, Putnam & Smith, 1985). In other words, when we look to explicit mentions of assumptions in, for example, a memorandum of explanation or the preamble of a piece of legislation, we are investigating espoused theories. It is not necessarily the case that there is congruence between espoused theories and theories in use.
  In connection with this, it is also relevant to distinguish between causes and justifications: the mere fact that something is explicitly referred to in a preamble (or comparable text) as the reason for the existence of, for example, a piece of legislation does not prove that the legislators had this reason in mind when drafting the text, nor does it (necessarily) point to the causes and underlying assumptions that the drafters actually made. Determining the actual causes (as opposed to the justification) becomes even more difficult when it is not an individual but an institution composed of many individuals with potentially divergent or even conflicting assumptions/theories in use. (Another question that goes beyond the scope of this article is whether and to what degree identifying the actual causes, as opposed to the justifications, matters.⁷x Moreover, sometimes the use of justifications – and at times even the existence of the entire legal arrangement – is symbolic more than anything (cf. Van Klink, 2014).)

  2. Inference to the best explanation

  If one does not, for whatever reason, take lawmakers or those who have created the legal arrangement at their word and is thus disinclined to believe the veracity of explicit references, a different way of identifying assumptions may be inference to the best explanation, also called abduction or abductive reasoning. Inference to the best explanation entails assuming the (approximate) truth of a hypothesis or theory that cannot be logically deduced from the available data on the grounds that the hypothesis best explains the available data (Douven, 2021; Haig, 2009; Vogel, 1998).
  For present purposes, that means investigating which assumptions seem to best explain the legal arrangement in question. To return to the example of the World Bank’s anti-corruption programme: what assumptions about the logical/conceptual relationship or the causal/contributory mechanisms best explain the introduction of interventions such as interactive workshops as a means to strengthen civil society?
  Inference to the best explanation is controversial, given that it is not clear what constitutes the best explanation (although criteria such as depth, comprehensiveness, simplicity or unifying power have been put forward (Vogel, 1998)) and given that when it comes to this kind of reasoning, the conclusion does not follow logically from the premises. This is an issue of a philosophy of science nature that cannot be solved here. However, we posit that inference to the best explanation, in this context, functions as a manner of reconstructing the theory that the person(s) creating and/or implementing a legal arrangement had in mind regarding the place and function of that legal arrangement in society, that is, a manner of identifying the theory in use (as opposed to espoused theory). If this is the case, insights from theory-driven evaluations (also: TDE) can offer further methodological guidance in making the reconstructive/inferential work explicit and thereby (more) traceable and replicable. Therefore, we turn to TDE in the next section. While the methodological approaches in this connection were developed primarily within the social sciences and with a view to identifying assumptions (in this context called theories) regarding causal or contributory mechanisms, the approach can also be suitable, mutatis mutandis, to reconstructing assumptions about the logical relationship between the legal arrangement and its underlying value(s), norm(s), or interest(s), as well as about the underlying value(s), norm(s) or interest(s).

  3. Theory-driven evaluations

  In the 1960s, policy analysts and evaluators started to develop (methodological) approaches to find the assumptions underlying the (non-legal) interventions and arrangements they were evaluating. The (methodological) approaches to identify assumptions (sometimes also known as small-t-theories; Leeuw & Donaldson, 2015)⁸x Policy evaluators are not the only ones making these distinctions; some legal scholars such as Lempert (2010) similarly distinguish between Grand Theory, Middle Range theory, Micro-level theory and between small t and capital T theory. fall within the tradition of theory-driven evaluations (TDE). TDE is not a one-size-fits-all approach, and that caveat applies also to the substance of the theories and the methods used in the finding and articulation of them. Some scholars reconstruct the theory underlying a legal arrangement before it is implemented or executed (ex ante); others focus on the implementation process and investigate which behavioural and other assumptions play a role during that process. Often, however, the focus is on the ex post assessment of the programme or policy theory.⁹x Usually referred to as ‘realist evaluations’, in which realism is related to work by Bashkar and Popper (Emmel et al., 2018; Pawson, 2013). Sometimes people refer to ‘critical realism’.
  TDEs investigate how Contexts (Cs), Mechanisms (Ms) and Outcomes (Os) are related when a policy, programme or legal arrangement is developed and introduced in society (Lemire et al., 2020).¹⁰x The jury is still out when it comes down to the precise definition and operationalization of mechanisms. Lemire et al. (2020) inventoried and analysed hundreds of (realist) evaluations in which up to 200 CMO-configurations were used and distinguished between a number of definitions of mechanism. Context can be defined according to ‘four i’s’: (1) the individual capacities of the people involved, (2) the interpersonal relationships between them, (3) the institutional setting and (4) the wider infrastructural and welfare system within which the policy, programme or arrangement is embedded. Outcomes are the results of programmes, such as compliance with rules and regulations – or the opposite, in the form of resistance, for example (Figure 2).

  The role of mechanisms
  

  Mechanisms can be seen as the cognitive-behavioural processes within and between people (‘drivers’ or ‘engines’ behind choices) that have a certain degree of stability in occurring (going hand in hand with certain decisions and behaviours) (Leeuw, 2003; 2021). If a policy- or lawmaker assumes that a legal arrangement will turn on one or more mechanisms, whereas instead it activates another mechanism or none at all, this will greatly impact the success (or lack thereof) of the legal arrangement. As such, mechanisms can make (or break) the impact of legal arrangements. Therefore, we will focus on mechanisms in the following, particularly on methods to trace assumptions about mechanisms and ways to reconstruct them into testable theories. This type of work is also referred to as ‘unpacking the blackbox’ (Astbury & Leeuw, 2010).
  While TDEs were not, initially, developed with legal arrangements in mind, the methods of TDEs can be useful when it comes to identifying the assumptions underlying legal arrangements. Quintanilla (2013), for example, studied assumptions underlying legal arrangements and referred to them as ‘implicit theories’. Farnsworth (2007), too, outlines how legal rules and their functioning relates to various mechanisms, including to psychological ones such as hindsight bias and framing effect. Lempert (2010: 893) holds that for lawyers ‘theory is not just theoretical; it is useful’. Examples of TDE-like research can be found regarding different legal instruments, such as the naming and shaming mechanisms assumed by Megan’s Law (Pawson, 2002), the investigation of regulatory disclosure of offending companies in the Dutch financial market as a form of consumer protection (van Erp, 2010), the fields of civilology and criminology more generally (Giesen et al., 2019) and recently concerning public health legislation, including Covid-19 regulations and the (behavioural) assumptions underlying them (Pawson, 2021).
  With all this in mind, how does one find assumptions about mechanisms and articulate them?
  Assumptions in general, and certainly those about mechanisms behind legal arrangements, are not readily available. Using explicit references and inference to the best explanation are ways to identify assumptions, but then one can wonder about their accuracy. TDE has therefore invested in developing and testing ways to find the assumptions on mechanisms and the mechanisms themselves. Pawson and Sridharan (2010: 44), for example, present ‘a five-stage overview of the method how to elicit and surface the underlying program theories’:

  * Step 1

  Eliciting and surfacing the underlying programme theories can be done by reading and closely analysing program documentation, guidance, regulations, etc., on how the programme will achieve its ends.

  * Step 2

  Do interviews with programme architects, managers or practitioners on how their intervention will generate the desired change.

  * Step 3

  Map and select the theories to put to research by using an array of techniques like concept mapping, logic modelling, system mapping, problem and solution trees, scenario building, configuration mapping and so on.

  * Step 4

  This step regards formalizing the theories to put to test.

  * Step 5

  Data collection and analysis is the next step in order to test the validity (and sometimes relevance) of the programme theory.

  Other suggestions are to work with argumentational analysis, content analysis (including natural language processing), process tracing and comparing theories (Carvalho & White, 2004).
  In the following, we present a synthesized step-by-step approach,¹¹x See also Leeuw (2003); EU, DG Regional and Urban Policies (2013), Pawson (2013); Emmel et al. (2018). including rules of thumb to use and questions to ask in order to identify assumptions:

  Step 1: Describe and categorize the (types) of interventions/arrangements.

  The first step is to inventory and categorize the legal arrangement, policy or programme in question. This is not yet about finding assumptions underlying it but a necessary precursor. Here, the following are useful:

  • Describe the goals that the legal arrangement, policy or programme wants to achieve. In doing so, distinguish between proximal and distal: proximal goals are concrete output goals, such as the amount of money for subsidies made available; distal goals are more abstract, such as increasing well-being and welfare or justice.

  • Describe the scope of the arrangement in terms of the (type[s] of) population(s) addressed (persons, organizations).

  • Describe the category of intervention types (see later) under which the intervention/arrangement can be brought. This provides a first indication of the direction in which one can search to gain an insight into the underlying mechanisms/assumptions.

  Step 1A: Categories of intervention types

  In the literature, a number of categorizations of programmes, etc. are available, all of which disclose some information on the foundations of the interventions. Knowing how to categorize a legal arrangement can help in finding and articulating more precise mechanisms that operate as triggers to make the legal arrangement work.
  One category (also called lens in evaluation literature) is Hood’s (1986) NATO typology, where N stands for nodality or information resources, A for authority, T for treasure or money and O for organization or personnel. Another is the sticks, carrots and sermons typology, where sticks represent legal sanctions, carrots financial incentives and sermons government communication (Vedung, 1998). Meyer and Homburg (2009) added the pillory to this, namely to promote compliance and realize impact by disclosure: data about companies are disclosed to expose their level of compliance. Often, this is referred to as the naming and shaming intervention type. A third typology by Schneider and Ingram (1990) proposed five broad types of interventions (or tools), based on assumptions about the compliance mechanism of policy subjects. These are as follows:

  • Authority tools rely solely on legitimacy.¹²x We will not, here, dive into conceptions of legitimacy and authority, but both have been conceptualized in various ways. Especially when it comes to multi- or interdisciplinary research, it is important to keep in mind that different fields and different authors may mean (very) different things with the same term. Policy targets are expected to do what they are told.

  • Incentive tools rely on tangible payoffs that could be positive or negative. Behaviour is incentivized.

  • Capacity tools provide information, education and resources to enable policy subjects to make decisions and carry out activities.

  • Symbolic and hortatory tools guide towards desired behaviour by manipulating symbols and influencing values.

  • Learning tools assume that policy addressees do not know what needs to be done, or what is possible to do, and policy interventions are therefore being used to promote learning.

  Steps 1 and 1A serve to take inventory and categorize the instrument(s) in question.

  Step 2: Scan and map the environment in which the programme, legislation will operate/is operating.

  Legal arrangements and policies do not operate in a social or administrative vacuum. They are part of what can be called the legal and policymaking ‘fabric’ (or: industry), which is one of the characteristics of complexity.¹³x When it comes to complexity, one can think of the institutional context in which regulation and legislation operate, from the layers of government (municipality, region, state, international) to individual organizations (agencies, quangos, ministries and government companies that play a role) and – without being exhaustive – to the diversity of the stakeholders, beneficiaries and/or end-users of policies, regulations, etc. These contexts are usually of a multi-actor and multi-level character. See Westhorp on how to relate this complexity to the question of what the impact of policies are and what the role is of theories in understanding that process (Westhorp, 2013: 364-382 and in particular, Figure 2 (‘Theory map’)). Another aspect is potential ‘rivalry’ between interventions/arrangements, which, seen through the eye of the beholder, complicates the functioning of mechanisms (see Section 5). Arguments why new interventions or intervention types are tried may become clear when scanning this ‘Umfeld’.
  Checklists like Pawson’s VICTORE are useful in this connection to grasp the environment of the legal arrangement:

  Victore Checklist to Focus on Issues Relevant in Finding Assumptions in Complex Societies (Pawson, 2013).

  • Address Volitions: people make (behavioural) choices. Their agency is volatile and often unpredictable.

  • Do not forget Implementation: what you see is not always what you get; implementation often implies a long-term chain of activities.

  • Contexts are important: they are partly given and partly created while the intervention/arrangement is implemented; context addresses macro and micro issues.

  • Time is important to take on board: interventions always come from somewhere and have a ‘history’ (in the minds of people, addressees). This adds to the complexity. Path dependency is also a feature of time.

  • Outcomes: they can be desired, undesired (including adverse); they can be proximal or distal.

  • Rivalry: during implementation of arrangements/interventions, dozens of other interventions and arrangements already exist and are relevant for the Umfelt of the one that is studied. There can be, and often is, rivalry between multiple existing interventions.

  • Emergence: society, individuals and events change during the implementation process.

  Step 3: Identify statements/assumptions underlying these interventions (types) addressing the why of these interventions/why they are assumed to be able to realize the goals set (proximal/ distal)?

  Step 1 made an inventory of the legal arrangement and its apparent characteristics and Step 2 of its environment. It is now time to dive deeper and begin opening up the ‘black box’. In order to look beyond explicit references made in, for example, preambles and the justification(s), if any, for the existence of the legal arrangement contained in its official documentation, it is useful to also consider interviews with politicians, parliamentary proceedings and debates, as well as, for example, interoffice communication and social media utterances, if available and traceable.
  Statements that have the following form are especially relevant for detecting assumptions about mechanisms:

  • ‘It is evident that x will work.’

  • ‘In our opinion, the best way to go about this problem is to …’

  • ‘The only way to solve this problem is to …’

  • ‘Our institution’s x years of experience tells us that …’

  • ‘Legal doctrine suggests or wants us to do …’

  • ‘We need x in order to achieve y.’

  Such documents and (interview) transcripts can contain statements about the goals of the policy or programme under review and why it is believed to be important. These statements point to mechanisms considered to be the ‘engines’ driving the arrangements, policies or programmes and believed to make them effective.
  It is useful to note that these kinds of statements give information about the mechanisms, that is, the causal and contributory assumptions – but they can also give information about the values, norms or interests that underlie the legal arrangement as well as their contexts.

  Step 4: Reformulate the statements from Step 3 in conditional ‘if-then’ or similar proposition structures (e.g. ‘The more x, the less y’).

  This reformulation is important, first, because it helps to ‘visualize’ the relationships between the different assumptions/propositions. When a reconstruction of underlying assumptions of a policy or legal arrangements consists of over 10 to 15 statements, such a structure is important. The second point is that by formulating the statement in terms of if-then/the more the format makes it particularly clear that these are statements of a hypothetical nature. If the statements are not formulated as such, the chances increase that they are taken by policymakers, beneficiaries and (even) evaluators as ‘for granted’, despite the idea that an intervention theory or the theory behind legal arrangements is always provisional and needs to be tested.

  Step 5: Search for ‘warrants’ identifying missing links in or between different propositions through argumentation analysis.

  A central concept of argumentation analysis is the warrant – the because part of an argument (Toulmin, 1958).¹⁴x We focus here on Toulmin (1958), but the literature on argumentational analysis vis-a-vis legal issues has grown substantially since then. In their introduction to the special issue on ‘Methodologies for Research on Legal Argumentation’, Araszkiewicz and Zurek (2016: 265) observe that ­[a]rgumentation plays a key role in the process of legislative deliberation, negotiations of various kinds, and other forms of dispute resolution. Thus it is hardly surprising that research in legal argumentation has become a prominent tendency in the field of contemporary legal theory for several decades.’ Ashley (2017) discusses different ‘Computational Models of Legal Reasoning’, and ‘argumentation support tools’ have also been developed (Van den Braak, 2010); Prakken (2018) analysed a number of them. It says that B follows from A because of a (generally) accepted principle. For example, ‘the organization’s performance will not improve next year’ follows from ‘the performance of this organization has not improved over the last 5 years’ because of the principle ‘past performance is the best predictor of future performance’. The because part of such an argument is often not made explicit and must be inferred by the analysis. The focus when searching for warrants is on mechanisms: social, behavioural, cognitive and institutional.
  This demonstrates the link between TDEs and inference to the best explanation – but while TDEs do not, and indeed cannot, negate the need for inferences, the steps and questions of each step serve to make the inferences made more explicit and thereby also more traceable for others and more repeatable.

  Step 6: Reformulate these warrants in terms of conditional ‘if-then’ (or similar) propositions and draw a chart of the (mostly causal) links.

  Here, it is important to note that it may be necessary to ‘layer’ different assumptions about mechanisms. This means that the theory underlying a legal arrangement may consist of several ‘sub-theories’.

  Step 7: Evaluate the validity of the propositions by looking into the logical consistency of the set of propositions and their empirical content.

  The previous steps are useful rules of thumb to reconstruct the theory/assumptions underlying a legal arrangement. How can one know whether this reconstruction is reliable? A final step in TDE is to evaluate this reconstructed theory.
  When it comes to evaluating the theory one has reconstructed,¹⁵x Here, we are not talking about an evaluation of whether the reconstructed theory is correct but about an evaluation of whether the theory was reconstructed well. While it is important to evaluate whether the theories held by, e.g., lawmakers are correct, this first requires that the theories they hold are reconstructed well by evaluators. If the reconstruction is badly done, this can have policy implications, since the ‘wrong’ (badly construed) theory is then evaluated and those evaluations used for future policy choices. this can be done in different ways: one is to confront (or juxtapose) different theories. For example, Carvalho and White (2004) reconstruct the theory underlying the (World Bank) social funds programme, but at the same time an anti-theory, which assumed the opposite of the first theory. Another approach is to test the theory by gathering or making use of primary or secondary data, both qualitative and quantitative. A third possibility is to organize an iterative process of continuous refinement using stakeholder feedback and multiple data collection techniques and sources (in the realist tradition). This involves constructing the theory (T), receiving reactions on it from experts or users, refining it on the basis of the said feedback (T becomes T’) and repeating the process as needed. A fourth approach to evaluation is to use already published reviews and synthesis studies.
  Examples of applications of the preceding approach can be found in Klein Haarhuis (2005), Ehren, Leeuw & Scheerens (2005), van Noije and Wittebrood (2010), Siebert and Myles (2019), Nagtegaal (2020), Van der Laan et al. (2021) and other evaluations.
  In this connection, it is important to also consider the goals of the evaluation itself. Social scientists tend to aim at testing the validity of the underlying theory, while legal scholars may be more interested in evaluative judgments and normative recommendations.¹⁶x For these purposes, consider Van der Burg’s (2018) argumentative framework. Much like an individual legal arrangement, however, law does not operate in a vacuum. If one takes law to be a tool to guide behaviour (Fuller, 1969), for example, analysing assumptions about the mechanisms underlying a legal arrangement and their validity (or lack thereof) forms an important part of analysing and evaluating the law and making recommendations on the basis of said analysis.

• 5. Some Complicating Factors

  In this article, we have sought to provide conceptual distinctions and methodological rules of thumb for the categorization and identification of the assumptions underlying legal arrangements. Before we conclude, we consider some additional complications: it must be acknowledged that even if one follows the preceding step-by-step approach, there is no absolute guarantee that no assumptions will be missed or wrongly construed. Tilley (1999), for instance, provides a number of examples of ‘faulty theories’, that is, wrongly construed assumptions.¹⁷x See also Leeuw (2010) for an analysis of, inter alia, the costs of errors. Despite this, however, we believe that this and other step-by-step approaches minimize the risk of missing or misconstruing assumptions.
  Another potential complicating factor is that a particular legal arrangement may promote or relate to more than one norm, value or interest and that most (if not all) individual legal arrangements are embedded in a larger system of legal arrangements that point towards other or additional norms, values and interests as well. In addition, we have already seen that the context in which a concrete legal arrangement is embedded matters. In social reality, in law and in ethics, it is often the case that norms, values or interests compete with one another. Conversely, a legal arrangement may not in itself be good, but, embedded in a larger context that promotes important values, it may well be, just as the legal arrangement considered on its own may not have the desired causal effect but, embedded in a bigger context and in conjunction with other legal arrangements or policies, it might. Alternatively, a particular legal arrangement might be well-suited for solving problem A, but might at the same time negatively impact another norm, value or interest; that is, it might cause or worsen problem B. If only the impact of the legal arrangement on problem A is considered, the legal arrangement would be evaluated positively; if a more inclusive view is taken, that may be different. This highlights the relationship between scope of research and research outcomes,¹⁸x Van der Burg (2018) makes a similar point in distinguishing between pro tanto and all things considered evaluations. which is an important methodological issue. To quote a commonplace: context matters, and things are complicated. Nonetheless, we believe that the categorizations and suggestions in the step-by-step plan can contribute to understanding the context and untangling the complexities involved in order to make any description and/or evaluation of the assumptions underlying a legal arrangement more explicit. This, we believe, will contribute to legal science.

• 6. Conclusion

  Given that legal arrangements of all kinds rest on behavioural, cognitive, social and other assumptions regarding their role and function in society and within the legal system, the identification and subsequent evaluation of these assumptions is, in our view, an important task for legal scholarship. In this article we have focused on the identification of these assumptions. In this connection we have done two things: first, we have developed conceptual distinctions for categorizing and understanding the different kinds of assumptions that may underlie a legal arrangement; second, we have provided methodological guidance for identifying these assumptions.
  With regard to the conceptual distinctions, we have identified assumptions about the value(s), norm(s), or interest(s) underlying a legal arrangement, which can be legal or non-legal (moral, political, prudential, etc.), and assumptions about the relationship between the legal arrangement and its underlying value(s), norm(s) or interest(s), which can be logical, causal or contributory.
  With regard to the methodological guidance, we have considered different approaches: a focus on explicit references, on the one hand, and inference to the best explanation on the other. Inference to the best explanation, we posit, functions as a manner of reconstructing the theory that the person(s) creating and/or implementing a legal arrangement had in mind (that is, the assumptions they made) regarding the place and function of that legal arrangement in society. Given this, we offer a step-by-step approach to reconstructing this theory in use, drawing from TDEs in the social sciences.
  As stated in the previous section of this article, these conceptual distinctions and methodological rules of thumb can contribute to understanding the context and untangling the complexities involved in identifying the assumptions that underlie legal arrangements.

Bibliography

• Araszkiewicz, T. & Zurek, M. (2016). Methodologies for research on# legal argumentation: Preface. Informal Logic, 36, 265-270.

• Argyris, C., Putnam, R. & Smith, D. (1985). Action science. San Francisco: Jossey-Bass.#

• Ashenfelter, O. & Greenstone, M. (2004). Using mandated speed limits to measure the value of a statistical life. Journal of Political Economy, 112(S1), S226-S267. doi:10#.1086/379932.

• Ashley, K. (2017). Artificial Intelligence and legal analytics. New tools for law practice in the digital age. Cambridge: Cambridge University Press.

• Astbury, B. & Leeuw, F. (2010). Unpacking black boxes: Mechanisms and theory-building in evaluation. American Journal of Evaluation, 31, 363-381. doi:10#.1177/1098214010371972.

• Bogue, D. (1974). Policy implications of theory and research on# motivation and induced behaviour change for fertility and family planning. Paper, AID Asia Population Officers Conference. Honolulu.

• Carruthers, P. (2009). How we know our own minds: The relationship between mindreading and metacognition. Behavioral and Brain Sciences, 32(2), 121-138#. doi:10.1017/S0140525X09000545.

• Caruso, G. (2015). If consciousness is necessary for moral responsibility, then people are less responsible than we think. Journal of Consciousness Studies, 22, 49-60#.

• Carvalho, S. & White, H. (2004). Theory-based evaluation: The case of social funds. American Journal of Evaluation, 25(2), 141-160#. doi:10.1177/109821400402500202.

• Douven, I. (2021). Abduction. In E. N. Zalta (Ed.). The Stanford Encyclopedia of Philosophy (Summer 2021 edition). Retrieved from https://plato.stanford.edu/archives/sum2021/entries/abduction/#.

• Ehren, M., Leeuw, F. & Scheerens, J. (2005). On the impact of the Dutch Educational Supervision Act: analyzing assumptions concerning the inspection of primary education. American Journal of Evaluation, 26(1), 60-76.

• Elliott, K. A. (1997). Corruption as an international policy problem: overview and recommendations. In K. A. Elliott (ed.) Corruption and the global economy. Washington, DC: Institute for International Economics#.

• Elster, J. (2007). Explaining social behavior: More nuts and bolts for the social sciences. Cambridge: Cambridge University Press.#

• Emmel, N., Greenhalgh, J., Manzano, A., Monaghan, M. & Dalkin, S. (Eds.) (2018). Doing realist research (pp#. 41-58). London: SAGE Publications Ltd. doi:10.4135/9781526451729.

• EU, DG Regional and Urban Policies (2013). Evalsed sourcebook methods and techniques. Retrieved from https://ec.europa.eu/regional_policy/en/information/publications/evaluations-guidance-documents/2013/evalsed-the-resource-for-the-evaluation-of-socio-economic-development-evaluation-guide (last accessed 20 August 2021).

• Farnsworth, W. (2008). The legal analyst: A toolkit for thinking about the law. Chicago: University of Chicago Press.#

• Fuller, L. (1969). The morality of law: Revised edition#. New Haven and London: Yale University Press. Retrieved from www.jstor.org/stable/j.ctt1cc2mds (last accessed 20 August 2021).

• Giesen, I., Wiznitzer, S., Keirse, A. & De Zanger, W. (Eds.) (2019). Assumpties annoteren. Rechterlijke veronderstellingen empirisch en juridisch getoetst. Den Haag: Boom# juridisch.

• Hage, J. (2018a). Foundations and building blocks of law. The Netherlands: Eleven International Publishing.#

• Hage, J. (2018b). Of norms. In G. Bongiovanni, G. J. Postema, A. Rotolo, G. Sartor, C. Valentini# & D. Walton (Eds.). Handbook of legal reasoning and argumentation. Switzerland: Springer.

• Haig, B. (2009). Inference to the best explanation: A neglected approach to theory appraisal in psychology. The American Journal of Psychology, 122(2), 219-234#. Retrieved from www.jstor.org/stable/27784393 (last accessed 17 August 2021).

• Hood, C. (1986). Tools of government. Chatham: Chatham House Publishers.#

• IBRD Articles of Agreement, December 27, 1945, https://thedocs.worldbank.org/en/doc/722361541184234501-0330022018/original/IBRDArticlesOfAgreementEnglish.pdf.

• Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.#

• Kelsen, H., Paulson, B. L. & Paulson, S. L. (1992). Introduction to the problems of legal theory: A translation of the first edition of the ‘Reine Rechtslehre’ or pure theory of law. Oxford: Clarendon Press.

• Klein Haarhuis, C. (2005). Promoting anti-corruption reforms. Evaluating the implementation of a World Bank anti-corruption program in seven African countries (1999-2001) (Doctoral dissertation, University Utrecht)#.

• Leeuw, F., Van Gils, G. H. C. & Kreft, C. (1999). Evaluating anti-corruption initiatives: Underlying logic and mid-term impact of a World Bank program. Evaluation, 5, 194-219. doi: 10#.1177/13563899922208922.

• Leeuw, F. (2003). Reconstructing program theories: Methods available and problems to be solved. American Journal of Evaluation, 24, 5-20. doi:10#.1177/109821400302400102.

• Leeuw, F. (2010). Benefits and costs of evaluation – An essay. Zeitschrift für Evaluation, 9(2), 211-227#.

• Leeuw, F. (2021). Improving the search for (hidden) mechanisms: More precise methods and using SNS (Social Neurosciences), Presentation, International Realist Conference 2021: Fostering innovation in practice, Dublin#.

• Leeuw, F. L. & Donaldson, S. I. (2015). Theory in evaluation: Reducing confusion and encouraging debate. Evaluation, 21(4)#, 467-480.

• Lemire, S., Kwako, A., Nielsen, S. B., Christie, C. A., Donaldson, S. I. & Leeuw, F. L. (2020). What is this thing called a mechanism? Findings from a review of realist evaluations. In J. Schmitt (Ed.). Causal mechanisms in program evaluation#. New Directions for Evaluation, 167, 73-86.

• Lempert, R. (2010). The inevitability of theory. California Law Review, 98(3), 877-906#. Retrieved from www.jstor.org/stable/27896696 (last accessed 20 August 2021).

• Marchal, B., Kegels, G. & Van Belle, S. (2018). Theory and realist methods#. In N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan & S. Dalkin (Eds.). Doing realist research (pp. 79-90). SAGE Publications Ltd. doi:10.4135/9781526451729.

• Marquette, H. (2007). The World Bank’s fight against corruption. Brown Journal of World Affairs, 13(2), 27-40.

• Meyer, A. & Homburg, V. (2009). Disclosure and compliance: The ‘pillory’ as an innovative regulatory instrument. Information Polity, 14(4), 279-294#. doi:10.3233/IP-2009-0191.

• Nagtegaal, M. (2020). Wet Langdurig toezicht. Reconstructie van de beleidstheorie, eerste evidentie en nadere onderzoekthema’s. Den Haag: WODC.#

• Pawson, R. (2002). Does Megan’s law work?: A theory-driven systematic review. Swindon: ESRC UK Centre for Evidence Based Policy and Practice.#

• Pawson, R. & Sridharan, S. (2010). Theory-driven evaluation of public health programmes, In A. Killoran and M. Kelly (Eds.). Evidence-based public health: Effectiveness and efficiency (pp. 43-62). Oxford: Oxford University Press#.

• Pawson, R. (2013). The science of evaluation: A realist manifesto. London: SAGE Publications Ltd.#

• Pawson, R. (2021). The coronavirus response: Boxed in by models. Evaluation, 27(2)#, 149-167. doi.org/10.1177/1356389020968579.

• Pope, J. (Ed.) (1996). The TI source book. Berlin: Transparency International.

• Pottie, L. & Sossin, L. (2005). Demystifying the boundaries of public law: Policy, discretion, and social welfare. UBCL Rev., 38(1)#, 147-188.

• Prakken, H. (2018). A new# use case for argumentation support tools: Supporting discussions of Bayesian analyses of complex criminal cases. Artificial Intelligence and Law, 28, 27-49. doi:10.1007/s10506-018-9235-z.

• Quintanilla, V. (2013). Judicial mindsets: The social psychology of implicit theories and the law. Nebraska Law Review, 90(3)#, 611-646.

• Rose-Ackerman, S. (1997). The role of the World Bank# in controlling corruption. Law & Policy in International Business 29(1), 113-114.

• Schneider, A. & Ingram, H. (1990). Behavioral assumptions of policy tools. The Journal of Politics, 52(2), 510-529#. doi:10.2307/2131904.

• Schroeder, M. (2021). Value theory. In E. N. Zalta (Ed.). The Stanford encyclopedia of philosophy (Fall 2021 edition). Retrieved from https://plato.stanford.edu/archives/fall2021/entries/value-theory/#.

• Siebert, P. & Myles, P. (2019). Eliciting and reconstructing programme theory: An exercise in translating theory into practice. Evaluation, 25(4)#, 469-476.

• Taekema, S. & Van der Burg, W. (2015). Introduction: The incorporation problem in interdisciplinary legal research. Erasmus Law Review, 8(2), 39-42#. doi:10.5553/ELR.000050.

• Tilley, N. (1999). Evaluation and evidence-(mis)led policy. Evaluation Journal of Australasia, 11, 48-63#.

• Toulmin, S. E. (1958). The uses of argument. Cambridge: Cambridge University Press.#

• Van den Braak, S. (2010). Sensemaking software for crime analysis (PhD thesis). Utrecht: Universiteit Utrecht.#

• Van der Burg, W. (2018). The merits of law: An argumentative framework for evaluative judgements and normative recommendations in legal research#. Erasmus Working Paper Series on Jurisprudence and Socio-Legal Studies, No. 17-01. doi:10.2139/ssrn.3020624.

• Van der Laan, A. et al. (2021). Evaluatie van het adolescentenstrafrecht. Een multi-criteria evaluatie. Den Haag: WODC

• Van Erp, J. (2010). Regulatory disclosure of offending companies in the dutch financial market: Consumer protection or# enforcement publicity? Law & Policy, 32, 407-433. doi:10.1111/j.1467-9930.2010.00326.x.

• Van Hoecke, M. (2011). Methodologies of legal research which kind of method for what kind of discipline? London: Hart# Pub. doi:10.5040/9781472560896.

• Van Klink, B. (2016). Symbolic legislation: an essentially political concept. In B. van Klink, B. van Beer# & L Poort (Eds.). Symbolic Legislation Theory and Developments in Biolaw (pp. 19-35). Switzerland: Springer.

• Van Noije, L. & Wittebrood, K. (2010). Fighting crime by fighting misconceptions and blind spots in policy theories: An evidence-based evaluation of interventions and assumed causal mechanisms. American Journal of Evaluation, 31(4), 499-516#.

• Vedung, E. (1998). Policy instruments: Typologies and theories. In M. L. Bemelmans-Videc, R. Rist# & E. Vedung (Eds.). Carrots, sticks and sermons. Policy instruments and their evaluation (pp. 21-59). New Brunswick: Transaction Publishers.

• Vogel, J. (1998). Inference to the best explanation. In The Routledge encyclopedia of philosophy. Taylor and Francis. Retrieved from www.rep.routledge.com/articles/thematic/inference-to-the-best-explanation/v-1 (last accessed 17 August 2021). doi:10.4324/9780415249126-P025-1.

• Westerman, P. C. (2011). Open or# autonomous? The debate on legal methodology as a reflection of the debate on law. In M. van Hoecke (Ed.). Methodologies of legal research: What kind of method for what kind of discipline? (pp. 87-110). London: Hart Publishing. doi:10.5040/9781472560896.ch-005.

• Westhorp, G. (2013). Developing complexity-consistent theory in a realist investigation. Evaluation, 19(4)#, 364-382. doi:10.1177/1356389013505042.

• Westhorp, G. (2018). Understanding mechanisms in realist evaluation and research. In N. Emmel, J. Greenhalgh, A. Manzano, M. Monaghan# & S. Dalkin (Eds.). Doing realist research (pp. 41-58). London: SAGE Publications Ltd.

• World Bank. (1999). Anti-Corruption Core Programme. Washington, DC: World Bank Institute.

• World Bank. (2022). Combating corruption. Retrieved from www.worldbank.org/en/topic/governance/brief/anti-corruption (last accessed 11 March 2022).

Noten

• * The order of authors is alphabetical; both authors have contributed equally to this article.

• 1 For the sake of brevity, we will speak mainly of legal arrangements from now. This should be read as encompassing policies and programmes as well.

• 2 ‘Each [family planning] administrator has his own style which means that each administrator has his own pet theories …which he uses over and over in carrying out his work’ (Bogue, 1974: 1-2).

• 3 More on these in Section 3.

• 4 As a matter of terminology, we consider norms to be those rules with a deontic conclusion, for example, rules that impose duties or rules that create obligations (Hage, 2018a: 165; 2018b). An abstract norm in this connection could be a higher constitutional norm or norm of international public law, but it could also be a norm of morality or religion.

• 5 We will leave aside the question of where to draw the line between and how to distinguish causation, contribution and correlation.

• 6 Although we think it is useful to draw this distinction between creating and implementing a legal arrangement, the two cannot always be neatly separated, and those implementing legal arrangements are often creating rules by doing so themselves, cf. Kelsen et al. (1992), Pottie and Sossin (2005).

• 7 Moreover, sometimes the use of justifications – and at times even the existence of the entire legal arrangement – is symbolic more than anything (cf. Van Klink, 2014).

• 8 Policy evaluators are not the only ones making these distinctions; some legal scholars such as Lempert (2010) similarly distinguish between Grand Theory, Middle Range theory, Micro-level theory and between small t and capital T theory.

• 9 Usually referred to as ‘realist evaluations’, in which realism is related to work by Bashkar and Popper (Emmel et al., 2018; Pawson, 2013). Sometimes people refer to ‘critical realism’.

• 10 The jury is still out when it comes down to the precise definition and operationalization of mechanisms. Lemire et al. (2020) inventoried and analysed hundreds of (realist) evaluations in which up to 200 CMO-configurations were used and distinguished between a number of definitions of mechanism.

• 11 See also Leeuw (2003); EU, DG Regional and Urban Policies (2013), Pawson (2013); Emmel et al. (2018).

• 12 We will not, here, dive into conceptions of legitimacy and authority, but both have been conceptualized in various ways. Especially when it comes to multi- or interdisciplinary research, it is important to keep in mind that different fields and different authors may mean (very) different things with the same term.

• 13 When it comes to complexity, one can think of the institutional context in which regulation and legislation operate, from the layers of government (municipality, region, state, international) to individual organizations (agencies, quangos, ministries and government companies that play a role) and – without being exhaustive – to the diversity of the stakeholders, beneficiaries and/or end-users of policies, regulations, etc. These contexts are usually of a multi-actor and multi-level character. See Westhorp on how to relate this complexity to the question of what the impact of policies are and what the role is of theories in understanding that process (Westhorp, 2013: 364-382 and in particular, Figure 2 (‘Theory map’)).

• 14 We focus here on Toulmin (1958), but the literature on argumentational analysis vis-a-vis legal issues has grown substantially since then. In their introduction to the special issue on ‘Methodologies for Research on Legal Argumentation’, Araszkiewicz and Zurek (2016: 265) observe that ­[a]rgumentation plays a key role in the process of legislative deliberation, negotiations of various kinds, and other forms of dispute resolution. Thus it is hardly surprising that research in legal argumentation has become a prominent tendency in the field of contemporary legal theory for several decades.’ Ashley (2017) discusses different ‘Computational Models of Legal Reasoning’, and ‘argumentation support tools’ have also been developed (Van den Braak, 2010); Prakken (2018) analysed a number of them.

• 15 Here, we are not talking about an evaluation of whether the reconstructed theory is correct but about an evaluation of whether the theory was reconstructed well. While it is important to evaluate whether the theories held by, e.g., lawmakers are correct, this first requires that the theories they hold are reconstructed well by evaluators. If the reconstruction is badly done, this can have policy implications, since the ‘wrong’ (badly construed) theory is then evaluated and those evaluations used for future policy choices.

• 16 For these purposes, consider Van der Burg’s (2018) argumentative framework.

• 17 See also Leeuw (2010) for an analysis of, inter alia, the costs of errors.

• 18 Van der Burg (2018) makes a similar point in distinguishing between pro tanto and all things considered evaluations.