Assessing the restorativeness of American school discipline programmes

• 1 Introduction

  One need not look long to find places where restorative justice has been implemented. A search for ‘restorative justice programmes’ finds literature related to efforts in many of the disciplines that focus on the human condition, such as forensic mental health (Cook, 2019), education (Katic, Alba & Johnson, 2020), criminal justice (Clarke, Brown & Vollm, 2017), victimisation (Curtis-Fawley & Daly, 2005), human rights (Lokugamage & Pathberiya, 2017), family violence (Cripps & McGlade, 2008), war (Cooke, 2019) and reconciliation courts (Ishiyama & Laoye, 2016), from nearly all parts of the world. Reading the articles in this diverse literature leads to a somewhat confusing variety of descriptions, definitions and efforts within restorative justice.
  Often these descriptions, definitions and efforts appear to contradict each other, leaving the reader asking,

  how are these all restorative? Seemingly, one result of restorative justice’s global reach has been an abundance of ambiguities about whether programmes and strategies are restorative in action as well as description. (Umbreit & Armour, 2011; Wood & Suzuki, 2016)

  Of particular difficulty is reconciling proactive and reactive definitions of restorative justice, especially in spaces where both whole system change is desired but punitive systems are still necessary. For instance, Mustian, Cervantes and Lee (2022) describe a whole school implementation effort where teachers and administration were focused on reducing school discipline issues, students were concerned with relationships, and the cultural origins of restorative justice were lacking from both initial training and project implementation. The contradictions in what restorative justice is and what it means can come from within the schools (Martinez, Villegas, Ayoub, Jensen & Miller, 2022) or from confusion from what schools do and what is reported to the public in the media (Vaandering & Reimer, 2019). Ultimately, much of the bureaucratic infrastructure seems ill-prepared for these dual goals, which can lead to inconsistency and watering down of objectives.
  Resolving such ambiguities has been complicated by the inherent flexibility of restorative justice itself, often demonstrated by the maxim that restorative justice is whatever people need it to be. The resolution of ambiguities has also been complicated by the lack of an acceptable measure of restorativeness.
  The latter is important because implementing and evaluating effective restorative justice strategies to improve the human condition requires several fundamental steps. Generally, these steps include problem identification, testing of theoretical propositions believed to explain or neutralise the causes of the problem, creating goals to resolve the problem, ensuring that strategy development and implementation are closely related to the theoretical propositions, and determining whether strategy goals are met (Mallicoat & Gardiner, 2014; Mears, 2010). Without a measure of restorativeness, evaluators of restorative justice-based strategies are too often left unable to confirm restorative propositions, to ensure that development and implementation align with those propositions, to evaluate short- and long-term outcomes. This state of uncertainty also leaves the field to question whether any one restorative justice strategy is similar to other strategies in its own discipline and to those in other disciplines. As such, uncertainties around comparisons of effective practices or lessons that can be learned along the way have become commonplace. As restorative justice has increased in its global reach, so too has it grown in these uncertainties. Because motivation and funding for initiation, continuation and expansion of restorative justice strategies are often dependent on demonstrating their effectiveness, these uncertainties place further growth of effective restorative justice strategies at risk.
  To help clarify these uncertainties and to reduce these risks to restorative justice, Olson and Sarver (2021) created the Restorative Index. Based on their search of the restorative justice literature, the Restorative Index authors identified a total of nineteen elements common to the restorative justice literature throughout the world. They categorised these nineteen elements into two broad areas of a restorative justice strategy, Mission (five elements) and Implementation (twelve elements). In doing so, the authors hoped to guide restorative justice researchers, developers and administrators into using a consistent and complete set of restorative descriptions and terms. While the inherent flexibility of restorative justice allows for incorporation of a variety of strategies and languages, the ability to compare and/or pseudo-replicate programmes across cultures, systems and agencies can be enhanced by a consistent set of restorative descriptors and terms such as those in the Restorative Index. Olson and Sarver (2021) also presented their recommendations for using the Restorative Index to assess the restorativeness of restorative justice programmes and strategies, including a sample scoring of a restorative justice programme implemented within a U.S. school. The Restorative Index authors further recommended that those interested in restorative justice programme development, implementation and assessment begin their own assessments of the restorativeness of restorative justice programmes.
  Taking up the recommendation for additional restorative assessments, the present article utilises the Restorative Index to assess the restorativeness of twelve American school projects purported to have foundations based firmly in restorative justice. The primary research question guiding our assessment was ‘how restorative are restorative justice programmes in American schools?’ First, we discuss why we chose American schools as a starting point for an assessment of restorativeness.

• 2 The restorative movement in American schools

  Broadly, restorative justice practices in education are about nurturing relationships and ensuring that when there is harm caused by conflict, there is support for all involved as well as expectations of harm repair (Evans & Vanndering, 2016). However, Mustian et al. (2022) argue that restorative justice practices are retrofitted to a deeply ingrained school culture that is fundamentally antithetical to the core tenets of restorative justice. Woods and Stewart (2018) state that ‘most of the principles of restorative justice are part of human nature’ and as a formal practice developed as a ‘cultural approach to conflict resolution’ (85). Mustian and colleagues (2022) support this by noting that restorative justice principles that are deeply ingrained in many Indigenous cultural approaches are often minimised or left out of the narrative altogether. Moreover, when there is a failure to understand the prevailing culture (within both the school and the broader social-political climate) students can be disempowered. In their research examining the utility of restorative justice practices and inclusivity within schools, Llewellyn and Parker (2018) specify the need to understand the unique identities of students and staff and how the processes of inclusion and exclusion along inequitable power relations impact the ways in which restorative justice practices are approached and understood. More broadly, the authors see restorative justice within the classroom as a lens through which students are able to engage in critical thinking, navigate difficult conversations and reflect on a broad range of diverse social issues (409).
  With the expanded use of restorative justice practices across the United States (Martinez et al., 2022), it is only becoming increasingly more important to have consistency in the conceptualisation of what these strategies look like, how they are understood and how they are implemented. For instance, are restorative justice practices conceptualised similarly across whole school approaches? Are they similar across disciplinary approaches? Finding these answers requires studying restorative practices both within and across approaches. One disciplinary approach where restorative justice has been directly and heavily implemented within American schools is in addressing the school-to-prison pipeline.

  2.1 The school-to-prison pipeline

  In the 1980s and 1990s, American schools began to implement Zero Tolerance policies for disruptive and deviant student behaviours (Marsh, 2017). Under Zero Tolerance, school administrators were authorised to exclude students from school grounds for violations of established school codes of conduct. Often, these policies also required administrators to contact the police and file criminal or juvenile charges when excluding students from schools. As a result of Zero Tolerance policies, the number of students excluded from school in 2009-2010 reached one in nine, a 40 per cent increase in exclusion from 1972 to 1973, which translates into approximately 2,000,000 annual suspensions from American secondary schools (Kang-Brown, Trone, Fratello & Daftary-Kapur, 2013). These suspensions were not always for serious or violent behaviours. Students increasingly incurred exclusion for behaviours such as talking out in class, disrespect towards teachers, and tardiness (Fronius et al., 2019; Gonzalez, 2012).
  As called for in Zero Tolerance policies, school referrals to the juvenile and criminal justice systems also increased. For instance, after implementation of Zero Tolerance policies, Gonzalez (2012) noted increases in juvenile referrals of 71 per cent in Denver, Colorado; 300 per cent in Philadelphia, Pennsylvania; and 600 per cent in Clayton, Georgia. These increases in delinquency referrals were not accompanied by any noticeable increase in youthful offending in the communities outside of school (Kang-Brown et al., 2013). The pathway from school exclusion to justice system referral is not always direct. There is also an indirect pathway to justice system referral where school exclusion leads to circumstances such as disengagement from school, then to involvement in street crimes and then to justice system referral. Whether direct or indirect, this pathway became known as the school-to-prison pipeline. Involvement in the school-to-prison pipeline is disproportionate for students depending on their races. Black/African American and Hispanic youths face the highest rates of both exclusion and referral to the justice system in the United States (Fronius et al., 2019; Gregory, Clawson, Davis & Gerewitz, 2016).

  2.2 Restorative justice as a response to the school-to-prison pipeline

  Growing awareness of the school-to-prison pipeline led school personnel as well as both federal and state governments to begin looking for alternatives to Zero Tolerance policies. Based on perceptions of its early successes in the American juvenile justice system and its use in a particular case of school violence in Australia, restorative justice arose as a contender to address behaviours and to reduce reliance on Zero Tolerance policies within American schools (Gonzalez, 2012; Marsh, 2017). Schools might provide unique communities within which to implement restorative justice because the community can be clearly defined and because schools can take a long-term approach to restorative justice strategies with their students, staff and cultures (Gonzalez, 2012). Restorative efforts in American schools often focus on providing an alternative to exclusion for student behaviours, reducing racial disparities in punishments and building or repairing relationships between essential people¹x Potential stakeholders, or essential people, in a school-based restorative justice programme include the person harmed, the person who harmed, family and/or support for these individuals, and members of the greater community, whether affiliated with the greater school-wide community or community at large. in schools (Kohli, Montano & Fisher, 2019). As implemented in American schools, restorative justice strategies are often founded on two similar philosophical ideas. The first is the belief that students will respond better when teachers and administrators do things with them rather than for or to them (Wachtel, 2016). The second is the belief that students are valuable people who are to be honoured by school personnel instead of being controlled by administrators and police (Morrison & Vaandering, 2012). Thus, restorative justice practices within American schools are built on fundamental ideals of student engagement and inclusion, as opposed to the exclusion and control on which Zero Tolerance is built.
  Recognising the potential for restorative justice to address these school issues, federal and state governments, as well as private organisations, began to fund initiatives to implement restorative justice in schools. For instance, in September 2014, the Children’s Defence Fund identified ten sources of potential funding to address school discipline or school safety (www.childrensdefense.org/wp-content/uploads/2018/06/funding-school-discipline.pdf). These funding sources represented two private foundations and the U.S. Departments of Education, Justice, and Health and Human Services. Although in restorative justice specific amounts are not available, just one of these sources, the U.S. Department of Education’s School Climate Transformation Grant, awarded $36,192,603 in continuation grants the following year (www2.ed.gov/programs/schoolclimatelea/awards.html). It is not uncommon to see grant offers and awards exceeding $500,000USD per site at the state (e.g. NJDOE, 2021) and federal (e.g. Vincent & Girvan, 2017) level. With the help of such funding, restorative practices have found their way into American schools.

  2.3 Evaluating success of school-based restorative justice programmes

  Despite the growth of restorative practices in American schools, many questions remain about their efficacy. Few systematic reviews of restorative practices have been published to date. Even when published, these reviews only include a few programmes and offer scant detailed information about the actual restorative practices in place for those programmes. For instance, the most recently published systematic review of restorative practices in schools identified 34 peer-reviewed articles discussing restorative practices in schools, with only 23 of them occurring in the United States (Lodi, Perrella, Lepri, Scarpa & Patrizi, 2022). For inclusion in this systematic review, the restorative intervention needed to include a restorative practice facilitated by an outside expert, have essential people in the school trained in restorative practices, and had to include outside restorative justice experts as researchers. Likewise, Katic et al. (2020) were only able to find ten studies that fit their inclusion criteria when searching for programmes that met Zehr’s (2015) restorative justice elements, were implemented in a K-12 school, and assessed outcomes of the restorative practices. Eight of these studies were on U.S. programmes.
  In the order of most to least common practices implemented, Lodi and colleagues (2022) found that schools used circles, conferences, peer-mediation, restorative conversations, mediation and community-building circles. In the same order, Katic and colleagues (2020) found that schools used circles, conferences and peer-mediation. Both systematic reviews reported that restorative practices were used variably in schools to address student discipline, bullying, aggression and attendance. Sometimes restorative practices were used to build or repair relationships. Implementations included school-wide efforts involving the entire school community and incident-based efforts involving the person harmed, the person who harmed or support people of those parties. Often, circles were used for minor incidents and relationship building, while conferences were more common for serious offences. Generally, outcomes of these restorative programmes were assessed for changes in school discipline, school climate and safety, conflict management, attendance, skills learned, academic achievement and/or racial disparities in discipline. While the studies found some overall positive outcomes, both systematic reviews noted concerns with the original studies for their descriptions of the restorative programmes, training of personnel, programme dosage and outcome measurement instruments.
  In an examination of the different programmes included in these reviews, it becomes difficult to ascertain whether the included programmes were similar to each other in actual restorative philosophy and strategies. For instance, while each of the systematic reviews noted the use of circles, there is no clarification on whether the circles implemented in each study addressed similar concepts and elements of restorative justice. The lack of clarity has been echoed in what is perhaps the largest current systematic review of school programmes in the world (Zakszeski & Rutherford, 2021) and in at least one qualitative review focusing on guiding educational leaders on restorative justice implementations (Gregory, Ward-Seidel & Cater, 2021). Importantly, it is also not limited to restorative justice research in education. Wong and colleagues (2016) noted encountering similar concerns when preparing for and conducting their own meta-analysis of restorative diversion programmes within juvenile justice.
  This cloudy state of restorative justice research arises largely from three concerns. First, the flexibility of restorative justice inherently allows for the freedom of programme developers to envision and implement restorative elements in ways best suited to their environments and cultures. Because restorative justice seeks to shift responsibility for addressing community harms from the government to the people directly affected by those harms (baliga, 2021; Christie, 1977; Zehr, 2015) and because its philosophy is implemented across many disciplines where humans rely on right relationships with each other (Gavrielides, 2008; Umbreit & Armour, 2011), it manifests as a wide array of ideas, practices and methodologies (Popa, 2012; Toews, 2006). Such individual or disciplinary perspectives on restorative justice philosophy are wholly appropriate but leave practitioners wondering if they are creating or implementing truly restorative programmes and leave research wondering if they are evaluating the same types of interventions or approaches (Bazemore, Ellis & Green, 2007; Bazemore & Green, 2007; Olson & Sarver, 2021; Roland, Rideout, Salinitri & Frey, 2012).
  Second, systematic reviews often do not progress with the desire to discuss the restorative elements of programmes. Rather, they seek to collate information about the effectiveness of restorative strategies that are under way in each field, such as reducing crime in criminal justice (Latimer, Dowden & Muise, 2005), reducing racial or gender disparities in education (Fronius et al., 2019), or improving maternal input into birthing decisions (Lokugamage & Pathberiya, 2017). Finally, until recently, there has not been an instrument available to systematically assess which restorative elements support restorative programmes. The end result of these concerns is that we cannot be certain that restorative programmes under review are consistent with each other in restorative approach, nor can we be sure that we are comparing or contrasting the restorative elements underlying each programme.

• 3 Parameters of the current study

  The current study seeks to begin to clarify the ambiguity of restorative justice programming in American schools by asking ‘how restorative are American school-based restorative justice discipline programmes in their design?’ Answering this research question kept us focused on where these programmes start – that is, the mission and design – rather than on analysing the implementation or outcomes of the programmes.
  Our efforts began with a conceptualisation of restorative justice as a philosophy focused on building right relationships with others and where attention is given to preventing or responding to potential or actual harms to those relationships. As such, we view the practices arising from restorative justice philosophy as appropriate for both proactive and reactive uses. A proactive use would entail building or strengthening relationships among people, where a harm has not occurred to those relationships. Proactive restorative justice is also appropriate where a potential harm has been identified but has not yet been perpetrated. A reactive use of restorative justice would entail efforts to repair or transform relationships where one or more essential people have engaged in acts that harm others. Thus, our overall conception of restorative justice closely aligns with the most comprehensive view described by Olson and Sarver (2021).
  However, for the current study, we chose to focus on reactive restorative justice programmes within American schools. We chose this focus for two important reasons. First, much of the current funding for school-based restorative justice in the United States has been applied to programmes that seek to disrupt the school-to-prison pipeline, resulting in a primary focus on addressing one or more disciplinary behaviours. Second, we wanted to be sure our comparisons of restorativeness were based on programmes with a similar mission. As described by the authors of the Restorative Index (Olson & Sarver, 2021), different missions can result in different possible overall scores. For instance, a whole school programme that seeks primarily to build relationships May not have an opportunity in its mission to directly address harms and needs. Thus, its potential Mission score would be reduced by 2. However, a reactive programme would be much more likely to focus on harms and needs in its Mission, exactly because it is reacting to a harm that has occurred. Further, where a reactive programme also addresses Relationships, it would logically be considered more restorative than a reactive programme that does not address Relationships, even though it seeks to address harms and needs.
  We believe that assessing restorativeness in school environments using the Restorative Index is appropriate. When developing the Restorative Index, Olson and Sarver (2021) intentionally sought and analysed restorative justice literature from a wide array of international fields, including within education and schools. As evident in the original article, the ideas and findings of many of these authors are included in the final version of the Restorative Index. Further, in their own example of scoring using the Restorative Index, the authors chose a whole school implementation reported by Acosta and colleagues (2019). Because it is evident to us that the authors explicitly included school environments when creating and demonstrating their Restorative Index, including both proactive and reactive perspectives, we are confident about using it for assessing the restorativeness of school programmes herein.
  Because starting at the beginning is important to programme evaluation, we were interested in first assessing what restorative justice elements school programmes intended to implement. Knowing the intended implementation elements is important because the intended implementation of an evidence-based practice should directly address the identified needs of each local school (USDOE, 2016). Likewise, the intended elements should logically impact the anticipated outcomes of the chosen intervention (USDOE, 2016). If schools are following this guidance from the U.S. Department of Education, knowing which restorative elements they choose for which needs will help shed light on their beliefs about how restorative justice elements can address and change factors such as differences in minority suspensions, overall exclusions regardless of race/ethnicity, violence, relationship repair or other concerns arising in the education of America’s youth. We were not concerned at this stage of research with whether or how schools implemented their chosen restorative justice elements or whether those elements effected any change in the school’s originally identified needs. We leave that question for future studies.
  This is also the first attempt to qualitatively assess the philosophical and elemental foundations of restorative justice programmes in American schools using the Restorative Index (Olson & Sarver, 2021). The elemental foundations of the Restorative Index are captured in its scoring criteria, shown in Table 1. To this end, we were interested in assessing how each implementation would score on the Restorative Index and in gaining practical insights into the use and validity of the Restorative Index. We now turn to the methodology of the current study.

  Table 1 The Restorative Index

  Programme Mission	Score
  Mission (up to 8 points) Harms: Within its mission, does the initiative identify the specific harms of an interpersonal offence (1 point), identify the needs of the stakeholders resulting from the harms (1 point), assign obligations for repairing those harms (1 point), and/or engage the essential people in any of the above mechanisms (1 point) Scoring Notes: Relationships: Within its mission, does the initiative: work to build, equalise or rebuild interpersonal relationships by understanding and validating essential people’s values, norms and cultures (4 points if ‘yes’) Scoring Notes: If no mission score, programme is not restoratively focused. Consider desire to move forward.	/8
  Implementation
  Inclusion (up to 3 points) Information: essential people provided with details of programme goals, strategies and expectations (1 point) Opportunity: programme allows involvement by all interested essential people (1 point) Voluntariness: participants free to choose/withdraw participation throughout programme (1 point)	/3
  Scoring Notes: Encounter (up to 7 points) Respect & Safety: value essential people as people and provide for physical/emotional safety (1 point) Restorative Dialogue: dialogue encourages free and open communication about harms, needs, relationships (1 point) Equality of parties: Encounter process and facilitators ensure that all voices and participants have equality in decision-making. No person makes a decision for another. (1 point) Number of Essential People: assign 1 point for each essential person or group engaged in encounters; person harmed, person who harmed, family/support group, community (up to 4 points) Scoring Notes:	/7
  Amends (up to 2 points) Apology: mechanism(s) for person who harmed to acknowledge responsibility for harm, offered to person harmed (1 point) Restitution: mechanism for person who harmed to repair harms caused to person harmed (1 point) Scoring Notes:	/2
  Reintegration (up to 4 points) Acceptance of Person: mechanism to reduce/remove stigmatisation of harm from Encounter participants (1 point) Safety: mechanism to rebalance power to person harmed and/or provide for physical & emotional safety of Encounter participants as they re-enter society (1 point) Competency Development: initiative offers essential people skill building and/or practice of skills (1 point) Follow Through: mechanisms to monitor change, verify outcomes, assure accountability & restitution (1 point) Scoring Notes:	/4
  Transformation (up to 3 points) Programme Approach: incidental (1 point); corollary (2 points); or systemic (3 points) implementation Scoring Notes:	/3
  Total Score: 1-13 Low restorativeness 14-19 Moderate restorativeness 20+ High restorativeness	/27

• 4 Methodology

  To determine the restorativeness of American school discipline programmes and to qualitatively assess the utility of the Restorative Index, we employed a form of subject specialist review similar to that used by Knoch (2014) in their aviation scale validation. As described there, subject specialist review assembles a group of people with experience in a subject of interest. These subject specialists independently consider the material chosen for review and then rate the materials against the measurement instrument or scale. Because we saw our attempt to assess the restorativeness of programmes in American schools as an important first step in programme evaluation and because these programmes are often implemented to address a wide range of student behaviours, inclusion as a reviewer on our team required more than a passing knowledge in at least two of three areas of interest. These were restorative justice, evidence-based programme evaluation and school-based prevention/intervention programming. Ultimately, four reviewers comprised our team.
  Included on this team were one of the developers of the Restorative Index (Olson). This reviewer has an extensive background in restorative justice both professionally and academically as well as recognised expertise in evidence-based programme evaluation. The second reviewer (Connell) has an extensive background in academic research related to the aetiology of youth violence and victimisation, as well as experience in conducting several evidence-based programme evaluations. The third reviewer (Barbieri) has a strong academic and professional background examining youth violence, school climate and adolescent behaviours broadly having worked in the K-12 school environment for several years. The fourth reviewer (Rodriguez) has professional experience working with high need youths, including youth who are justice involved. She is currently doing her doctoral studies, focusing on restorative justice practices and juvenile delinquency reform.
  The team worked together to set inclusion criteria for programmes in this review. To be considered for inclusion, programmes would need to meet all selected criteria. The criteria were: programmes implemented within a U.S. school; programmes that explicitly implemented a restorative practice in an attempt to address specific student misconduct of various forms (acting out in class, tardiness, fighting, etc.); and programmes that were subject to at least one outcomes study available in an English language journal, thesis/dissertation or report. Where the article made it clear that different schools within the same district implemented the same restorative programmes, we considered the district’s efforts to be a single implementation and would score all reports as one.
  Using our university library resources, Google Scholar, and the Internet, we conducted searches for different arrangements of the keywords restorative justice or restorative practices, school(s) and evaluation. Our initial search identified 24 articles that discussed 16 different restorative justice implementations in schools. As we read and scored these programmes, we eliminated four owing to their failure to identify specific restorative justice elements within their programmes or because they also evaluated programmes outside of the United States. Our final review consists of twelve unique restorative justice-based programmes within U.S. schools, presented in Table 2. These criteria were consistent with the approach of that of Katic et al. (2020) in their systematic review, although the current project did eliminate one study from review because there was not enough information to determine what programme was implemented. This overlap not only demonstrates the small number of meaningful restorative justice programmes available in the literature for review, despite the influx of funding, but also lends context to Katic et al.’s (2020) findings of support (or lack thereof) for restorative justice programming.
  Prior to individual review of programmes, all team members read the original Restorative Index (Olson & Sarver, 2021). The team member who was a co-author of the Restorative Index then trained the other three reviewers on the Restorative Index elements and the current team practised scoring against the programme (Acosta et al., 2019) scored in the original Restorative Index publication. The team then piloted individual scoring of two separate programmes and held a consensus meeting immediately after scoring each one. During the consensus meetings, we discussed concerns arising in the reviews, clarified expectations for scoring, discussed our independent scores and then worked to reach a consensus score for each programme. After these first two meetings, we then began independent reviewing and scoring of four programmes at a time and held consensus meetings for each set of four programmes approximately two months apart. At least three reviewers read and scored each programme. All published information about each programme was included in the scoring materials; in some cases, there were two or more manuscripts pertaining to a specific programme, so they were combined for more accurate results. Reviewers used the Restorative Index scoring instrument to note whether they believed the programmes intended to implement each element of the Restorative Index.

  Table 2 Overview of programmes reviewed

  Author(s)	Year(s)	Location	Target Population	Mission/Goal
  Augustine et al. (2018)	2013-2014	Pittsburgh Public Schools (PPS)	Public K-12 students	Evaluation of PPS’s ‘Pursuing Equitable and Restorative Communities’ (PERC) to determine: Implementation Impacts Likelihood of PPS will continue PERC
  Dewitt and Dewitt (2012)	2006	Midwestern high school, UT	11th grade students	Case study of one high school to determine whether their restorative justice plan affected students’ knowledge of hazing, their understanding of the consequences, awareness of other local high school-related incidents, current rates of hazing and other lasting effects on behaviour
  Gregory and Clawson (2016)	2011-2013	Two high schools, Northeast	All enrolled students	Did the restorative practices implemented reduce gender and racial disparities in disciplinary decisions, and did an increase in these practices within the classroom yield fewer disciplinary decisions by teachers
  Long (2015)	2011-2014	Midwestern School District	PreK-8 students	Evaluation of the relationship between RJ and suspension, as well as exploring discipline gaps
  Mansfield, Fowler and Rainbolt (2018)	2010-2015	Algonquin High School, VA	All enrolled students	Preliminary results of how restorative practices were implemented and outcomes found
  Sumner Silverman & Frampton (2010)	2006-2008	West Oakland, CA	All enrolled students	Case study of one middle school
  Schotland, MacLean, Junker and Phinney (2016)	2013-2014	Davidson Middle School, CA	Students receiving a disciplinary referral for: persistent misconduct, insubordination, swearing, verbal fighting and/or disorderly conduct	Determine whether disciplinary approaches transformed and reduction in suspensions, particularly by racial/ethnic group
  Katic (2017)	2011-2016	One middle school, San Bernardino City Unified School District	All enrolled students	Has the implementation of RJ practices been effective in reducing office referrals and suspensions
  Brown-Kersey	2009-2010	Midwestern urban school district	All enrolled students	Impact on office referrals, suspensions and grade point averages
  Stinchcomb, Bazemore and Riestenberg (2006)	1998-2001	South St. Paul district, MN	All enrolled students	Pre/post measures for changes in rates of suspensions, expulsions, behavioural referrals and attendance
  Carrol (2017); Gattuso (2016)	2012-2015	Alternative high schools, CA	Students enrolled in grades 6-12	Changes in suspension rates
  Anyon et al. (2016)	2012-2013	Denver Public Schools	All students who received an office discipline referral (ODS) or out-of-school suspension (OSS)	What is the association between a student’s participation in one or more restorative intervention (RI) and odds of ODRs and/or OSSs; participation in RIs and later discipline incidents moderated by student racial background or school-level use of RI; and whether disciplined students from disadvantaged backgrounds have equitable participation in RIs

• 5 Findings

  5.1 Levels of restorativeness

  A total of twelve programmes were chosen for scoring and validation. First, we discuss the absolute scores on the Restorative Index for each programme and describe the general findings around restorativeness of school-based restorative justice programmes. Next, we describe the process of Index validation and the challenges faced by the team throughout the validation process. Finally, we discuss ways in which the Restorative Index can be adapted in future iterations to address these concerns.
  Throughout coding, the absence of the description of a component of one of the domains was coded as a 0. It should be noted, however, that this is not necessarily indicative that the component was not present in the actual implementation of the restorative justice programme under review. It can mean either that the programme did not implement this component or that the programme did not describe the component (at least not in sufficient detail to be appropriately considered).
  There was a great deal of variation between restorativeness scores among the programmes. Total programme scores ranged from 7 to 26, out of a possible score of 27 on the Restorative Index. The average score was 16 (sd = 5.87; median = 15.5). Table 3 presents reconciled scores for each element of the Restorative Index and the reconciled total restorativeness score for each programme.

  Table 3 Reconciled element and overall restorativeness scores of programmes

  Study	Schotland	Sumner	Augustine	DeWitt	Gregory & Clawson	Katic	Mansfield	Brown-Kersey	Long	Stinthcomb	Carroll & Gattuso	Anyon
  Harms	1	1	1	1	1	0	1	1	1	1	1	1
  Needs	0	1	1	0	1	0	1	0	1	1	1	1
  Obligations	1	1	1	1	1	0	1	1	1	1	0	1
  Engagements	1	1	1	0	1	0	1	1	1	1	0	1
  Relationships	4	4	4	0	4	4	4	0	4	4	4	0
  Mission Total	7	8	8	2	8	4	8	3	8	8	6	4
  Information	1	1	1	1	0	0	0	1	0	1	0	1
  Opportunity	1	1	1	0	0	0	1	1	1	1	1	1
  Voluntariness	0	1	0	1	0	0	0	0	0	1	0	1
  Inclusion Total	2	3	2	2	0	0	1	2	1	3	1	3
  Respect & Safety	1	1	1	0	1	0	1	0	0	0	0	0
  Restorative Dialogue	1	1	1	0	1	0	1	1	1	0	1	1
  Equality of Parties	0	1	1	0	1	0	1	0	0	0	0	0
  Number of Essential People	4	4	4	0	3	1	4	2	3	4	2	3
  Encounter Total	6	7	7	0	6	1	7	3	4	4	3	4
  Apology	1	1	1	0	0	0	1	0	0	0	0	0
  Restitution	1	1	1	1	0	0	1	1	0	1	0	1
  Amends Total	2	2	2	1	0	0	2	1	0	1	0	1
  Acceptance of Person	0	1	1	0	0	0	1	0	0	0	0	0
  Safety	0	0	1	0	0	0	1	0	0	0	0	0
  Competency Development	1	1	0	1	0	0	0	0	0	0	0	0
  Follow Through	1	1	0	1	0	0	0	1	0	0	0	0
  Reintegration Total	2	3	2	2	0	0	2	1	0	0	0	0
  Transformation	2	3	2	1	2	2	2	1	2	2	1	2
  Restorativeness Total	21	26	23	8	16	7	22	11	15	18	11	14

  Mission: On average, programmes received a score of 6 (of 8) on the components of the Mission domain. Programmes were most likely to mention the importance of identifying specific harms (11/12 programmes), followed by assigning obligations to those who were responsible for harms (10/12 programmes) [e.g. having offending students answer questions such as ‘what do you think needs to happen to make things right?’ (Gregory et al., 2016)]. In addition, most programmes (9/12) made mention of the engagement of specific stakeholders in the process. The fewest programmes (8/12) identified the needs resulting from the harms [e.g. asking students questions such as ‘who has been affected by what you did?’ or ‘how has this affected you and others’ (Gregory et al., 2016) and what could be done to repair the harm (Schotland et al., 2016)]. Similarly, only 9 (out of 12) programmes explicitly described the need to build (or rebuild) relationships damaged by the harm. There is discussion within the original Restorative Index (Olson & Sarver, 2021) calling for pause and further consideration as to whether programmes are restorative when they do not explicitly address harms, needs, obligations, engagements or the (re)building of relationships within their missions. All twelve of our identified programmes explicitly addressed at least one of the mission elements and were retained for further coding.
  Inclusion: On average, programmes received a score of 1.67 (of 3) on the components of the Inclusion domain. Programmes were most likely (9/12) to identify the opportunities for all stakeholders to engage in the restorative process. Half of the programmes (6/12) provided explicit information to essential people or the general community members about goals, strategies and expectations either prior to their involvement in the programme or on an ongoing basis once they were involved. Very few (4/12) programmes were voluntary in nature, where the individuals involved (whether they be offenders or victims) could decide their level of participation in the restorative justice process.
  Encounter: On average, programmes received a score of 4.33 (out of 7) on the components of the Encounter domain. Programmes were most likely to identify the use of Restorative Dialogue (9/12) to encourage free and open communication. While only 5 (of 12) programmes explicitly identified how respect and safety were implemented to support physical and emotional safety, only 4 described if/how equality of the parties was supported throughout the process. Each programme also had the potential to involve up to 4 essential people or groups (person harmed, person who harmed, family/support persons, community members) in the process. On average, programmes involved approximately 3 people (2.8), with a range of no stakeholder involvement identified to all 4 stakeholder groups involved.
  Amends: On average, programmes received a score of 1 (out of 2) on the components of the Amends domain. The majority of the programmes (8/12) described a mechanism of repair of harm through restitution. What was less common, however, was whether these programmes (4/12) offered a path to a formal acknowledgement of and/or apology for the harms caused. For example, DeWitt and DeWitt (2012) discussed a personal action plan required of the student participants that included a presentation to their peers on the consequences associated with hazing, as well as a community service requirement of 20 hours to compensate for the associated tax dollars expended as part of the hazing incidents.
  Reintegration: On average, programmes received a score of 1 (out of 3) on the components of the Reintegration domain. Least common were indicators of a mechanism to rebalance harm (2/12) for the injured individual’s physical and emotional safety. Programmes were equally likely to describe mechanisms to reduce/remove stigma of harm from the encounter and include components of competency development to support skill building and increase practice, either directly as a result of an individual’s participation in the encounter or as part of the overall programme (4/12). One-third of the programmes (4/12) implemented follow-up procedures to ensure compliance with agreements (or where necessary, increased engagement with sanctions) after the encounter.
  Transformation: Transformation scores were based on the level of systemic change attempted by the restorative justice programme. While the overarching goal of restorative justice is to create high levels of social change, we acknowledge that not all restorative justice programmes are in the position to strive for this goal. The majority of programmes (8/12) were scored as a 2 (attempting corollary change). Only one programme scored the highest score of 3 on transformation (attempting system, school-wide change).

  5.2 Validation issues

  Beyond individually scoring each construct within the six domains for the twelve programmes, this validation included close contact between reviewers to ensure both reliability and validity between scores. Of particular interest was whether difficulties in scoring were related to the need for further clarification on the part of the Restorative Index or whether difficulties in scoring were related to the ways in which programme evaluators and developers described school-based restorative justice programmes. It will surely come as no surprise to readers of systematic evaluations and meta-analyses that published works do not always include the same sets of information, adding challenges to endeavours such as these.
  While in general, reviewers were in high agreement for all domains, three constructs stood out for their lack of straightforward agreement: Information (Inclusion); Number of Essential People (Encounter); and Transformation. These three constructs, while in separate domains, do exhibit a similar thread in that they are directly related to the ways that programmes describe outward and community-focused engagement. We describe each construct next, including specific difficulties around coding agreement issues.
  Information: As part of the Inclusion domain, coders rated how well the programme engaged stakeholders by providing details about programme goals, strategies and expectations. The intention is to understand the level of true community engagement throughout the process of restorative justice implementation. Just over half (60 per cent) of the programmes included this information (see Table 2). Of interest here is the amount of difficulty reviewers had when finding and interpreting details about distribution of programme information to stakeholders in the published studies. In only two programmes (Anyon et al., 2014, 2016; Carroll, 2017; Gattuso, 2016) was there unanimous reviewer agreement about the presence (or absence) of information dissemination in programme descriptions. At least three out of four reviewers agreed on the scoring of information in only 75 per cent of the cases, in stark contrast to the majority of the other domains.
  Confusion generally occurred because programmes did not explicitly discuss the ways in which information was given to the community, nor did they speak to the community information at various times throughout the process. Examples of the ways that the community was informed about the implementation of the new restorative justice initiatives among the programmes included community meetings, group/school assemblies and information campaigns to members of the school community (e.g. parents). Some programmes discussed district and/or school management-level information campaigns, often targeted towards teachers and other academic staff, but did not clearly articulate how students were informed and engaged in the process. After considerable consultation among reviewers, information was coded as available if the key stakeholders (i.e. students and teachers) were directly informed about the implementation of a new restorative justice disciplinary practice.
  Number of Essential People: Each programme received a score for the number of stakeholders engaged in the restorative justice process. Potential stakeholders include the person harmed, the person who harmed, family and/or support for these individuals, and members of the greater community, whether affiliated with the greater school-wide community or community at large. In only three programmes were reviewers unanimous on the number of stakeholders involved in the restorative justice process. In just over half (58%) of the programmes were three out of four reviewers able to come to an agreement on the number of stakeholders engaged in the process. Complications in determining stakeholders generally centred on the level of engagement that individuals outside of the immediate transgression (anyone who was not the person harmed or the person who has harmed) had with the process. Teachers, for example, could be the person harmed, the person who has harmed or the support for one of the parties. Other members of the school community could be considered persons harmed or support for parties. Additional confusion was evident when members of the community were given information about the restorative justice process but there was no discussion about whether they were engaged in actual encounters.
  Transformation: In line with difficulties interpreting programme implementation around areas of stakeholder engagement were difficulties in determining the level of desired transformation expected of the new restorative justice disciplinary practices. In only two of the programmes was initial unanimity reached; initial coding scores only mustered agreement of three out of four reviewers in half (50 per cent) of the programmes. In both cases of perfect agreement, the programme was given a score of 1 on transformation (incidental). Confusion over corollary versus systemic transformation generally stemmed around the lack of clarity about programme goals versus generally described restorative justice principles within programme descriptions. Many programmes recognised the goals of systemic change inherent in restorative justice approaches but did not describe their own processes in enough detail to determine whether that particular programme was attempting whole school and/or whole system restorative justice change or was merely integrating restorative justice options into traditional disciplinary practices.

• 6 Discussions

  Best practices in the design and implementation of evidence-based interventions call for programmes to clearly identify the theory on which they are built and to connect that theory to each activity, service or operation leading to an expected outcome (Mallicoat & Gardiner, 2014; Mears, 2010). This is an important part of the logic model as it allows evaluators to determine whether the propositions of a strong theory were properly designed into the activities of an intervention, whether those activities were engaged by the participants, whether the activities then led to changes, and whether those changes were expected based on the stated theory. Evaluators’ efforts to assess the fidelity of interventions are hindered when they cannot determine who the intended participants of restorative interventions were and whether those participants were informed of the programme expectations. Likewise, their ability to assess outcomes are reduced when they are unclear whether the interventions hope to support, divert from or replace traditional exclusionary discipline. We recommend that future writings, whether they discuss existing and yet-to-be-developed programmes, include pertinent details on all the developers’ intended restorative elements.
  The Restorative Index can aid that end by giving programme developers a strong foundation on which to build and report their restorative efforts. Once programmes are framed within the elements of the Restorative Index, evaluators and funders can use the same instrument to assess the fidelity of implementations, the strength of outcomes and the utility of the interventions. In doing so, it will be important for evaluators to ensure that they are weighing the programmes against the restorative elements outlined by developers. Assessing programmes within their described set of restorative elements will also help to avoid penalising them for not being ‘restorative enough’ as May be the case when they are weighed against other efforts that aim to include more, or more diverse, restorative elements.
  On the other hand, the Restorative Index can help uncover instances of restorative co-option by helping to identify programmes that integrate restorative terminology without restorative practices. For example, we reviewed a programme implemented within alternative schools in California (Carroll, 2017; Gattuso, 2016). In this instance, the programme claimed to be a restorative justice–like implementation of other established programmes (STRIVE, Urban Essentials 101, and Discipline That Restores). As we progressed through scoring, we uncovered a programme that was being called restorative but that included very limited actual restorative practices. In fact, it appeared to us that the totality of restorative practices was a disciplinary form given to students as they entered in-school suspension after committing an act against school policy. This form included language of restorative dialogue; however no student answers were processed with the students themselves, nor were they offered to, or read, by any other essential people. When we weighed this implementation against the Restorative Index, it scored 11 of 27 (Low Restorativeness), despite claims otherwise within the programme itself.

• 7 Conclusion

  The aim of this work was to assess the mission and design of restorative justice programmes in American schools while also qualitatively assessing the elements of the Restorative Index (Olson & Sarver, 2021). Our search of the literature found twelve programmes that sought to address a specific student behaviour or set of student behaviours through a restorative practice and that were subject to at least one published peer-review outcomes study in an English language journal or where a programme evaluation was part of a thesis or dissertation. We then piloted the scoring process before undertaking individual scoring of programmes.
  On reconciling our individual scores, we agreed that designers of these U.S.-based school programmes most often envisioned a corollary and simultaneous existence with western punitive discipline where students could be diverted from exclusionary discipline if they participated in at least one restorative practice. On overall restorativeness, these programmes ranged from 7 (low restorativeness) to 26 (high restorativeness). Within individual domains of the Restorative Index, we found that these programmes were largely designed with a restorative Mission and that they appeared to consider more than 50 per cent of the Inclusion and Encounter elements. However, programmes faltered in designing in elements of Amends and Reintegration, with 50 per cent or less of these elements explicitly discussed.
  We enjoyed high agreement between reviewers within five of the six main domains of the Restorative Index, the exception being Transformation. We also noted a lack of consistent reviewer agreement within two of the individual elements in two domains. These were Information (Inclusion domain) and Number of Essential People (Encounter domain). In all three of these areas, reviewer confusion arose in interpreting how the evaluation authors described their programmes to internal and external stakeholders. Where confusion was high, we most often saw that programmes appropriately described the general restorative justice principles around these elements but then failed to discuss whether and how these principles were incorporated into their specific approach and strategy. In other words, authors consistently acknowledged awareness of important restorative principles but did not detail their planned implementation into programmes.
  Taking up the recommendation for additional restorative assessments, we trained reviewers in reading and understanding the Restorative Index (Olson & Sarver, 2021). We then used the Restorative Index to assess the restorativeness of twelve American school projects purported to have foundations based firmly in restorative justice. The primary research question guiding our assessment was ‘how restorative are restorative justice programmes in American schools?’ On reconciling individual scores to group scores, we found that the twelve programmes scored equally (four each) in categories of ‘High Restorativeness’, ‘Moderate Restorativeness’, and ‘Low Restorativeness’. In addition to natural fluctuations between programme efforts at restoration, we found that programmes varied in scores because of consistent gaps in their restorative descriptions and, to a lesser degree, because of possible co-option of restorative justice.
  As existing restorative justice-based programmes move forward, we suggest they review their literature and design to ensure that they have both integrated and described the restorative elements they hoped to build within their programmes. As new programmes come to development, we suggest that developers carefully consider the restorative elements that would best fit their needs and then build those elements into their intervention designs in a way that they can clearly explicate where the elements live within the programme. With attention to those elements, evaluators and funders can make better determinations of fidelity and outcomes for restorative efforts. We believe that using the Restorative Index as a basis for programme development, revision and assessment can address these important focal points.

References

• Acosta, J., Chinman, M., Ebener, P., Malone, P.S., Phillips, A. & Wilks, A. (2019). Evaluation of a whole-school change intervention: findings from a two-year cluster-randomized trial of the restorative practices intervention. Journal of Youth and Adolescence, 48(5), 876-890 doi: 10.1007/s10964-019-01013-2.

• Anyon, Y., Gregory, A., Stone, S., Farrar, J., Jensen, J.M., McQueen, J., Downing, B., Greer, E. & Simmons, J. (2016). Restorative interventions and school discipline sanctions in a large urban school district. American Educational Research Journal, 53(6), 1663-1697. doi: 10.3102/0002831216675719.

• Anyon, Y., Jenson, J.M., Alstschul, I., Farrar, J., McQueen, J., Greer, E., Downing, B. & Simmons, J. (2014). The persistent effect of race and the promise of alternatives to suspension in school discipline outcomes. Children and Youth Services Review, 44, 379-386. doi: 10.1016/j.childyouth.2014.06.025.

• Augustine, C. H., Engberg, J., Grimm, G. E., Lee, E., Wang, E. L., Christianson, K. & Joseph, A. A. (2018). Can Restorative Practices Improve School Climate and Curb Suspensions? An Evaluation of the Impact of Restorative Practices in a Mid-Sized Urban School District. Retrieved from Santa Monica, CA: www.rand.org/t/RR2840.

• baliga, s. (2021). Whose harm? The role of the state in restorative justice. New Political Science, 43(1), 35-45. doi: 10.1080/07393148.2021.1880700.

• Bazemore, G., Ellis, L. & Green, D.L. (2007). The “independent variable” in restorative justice: theory-based standards for evaluating the impact and integrity of victim sensitive process (Part II). Victims and Offenders, 2(4), 351-373. doi: 10.1080/15564880701404056.

• Bazemore, G. & Green, D.L. (2007). “Yardsticks” for victim sensitive process: principle-based standards for gauging the integrity of restorative justice process. Victims and Offenders, 2(3), 289-301. doi: 10.1080/15564880701404031.

• Carroll, P. (2017). Evaluating attempts at the implementation of restorative justice in three alternative education high schools (Doctor of Philosophy). University of California, Merced. Retrieved from escholarship.org/uc/item/2t95r24f#.

• Christie, N. (1977). Conflicts as property. The British Journal of Criminology, 17(1), 1-15.

• Clarke, M., Brown, S. & Vollm, B. (2017). Circles of support and accountability for sex offenders: a systematic review of outcomes. Sexual Abuse, 29(5), 446-478. doi: 10.1177/1079063215603691.

• Cook, A. (2019). Restorative practice in a forensic mental health service: three case studies. Journal of Forensic Psychiatry & Psychology, 30(5), 876-893. doi: 10.1080/14789949.2019.1637919.

• Cooke, M. (2019). Murad vs. ISIS: rape as a weapon of war. Journal of Middle Eastern Women’s Studies, 15(3), 261-285. doi: 10.1215/15525864-7720627.

• Cripps, K. & McGlade, H. (2008). Indigenous family violence and sexual abuse: considering pathways forward. Journal of Family Studies, 14(2/3), 240-253. doi: 10.5172/jfs.327.14.2-3.240.

• Curtis-Fawley, S. & Daly, K. (2005). Gendered violence and restorative justice: the views of victim advocates. Violence Against Women, 11(5), 603-638. doi: 10.1177/1077801205274488.

• DeWitt, D.M. & DeWitt, L.J. (2012). A case of high school hazing: applying restorative justice to promote organizational learning. NASSP Bulletin, 96(3), 228-242. doi: 10.1177/0192636512452338.

• Evans, K. & Vanndering, D. (2016). The little book of restorative justice in education: fostering responsibility, healing, and hope in schools. New York: Good Books.

• Fronius, T., Darling-Hammond, S., Persson, H., Guckenburg, S., Hurley, N. & Petrosino, A. (2019). Restorative justice in US schools: an updated research review. San Francisco. Retrieved from http://jprc.wested.org/.

• Gattuso, L. (2016). The Effects of using Restorative Justice in Alternative Education High Schools. (Master of Arts), California State University, Stanislaus, Stanislaus, CA, USA.

• Gavrielides, T. (2008). Restorative justice: the perplexing concept. Criminology & Criminal Justice, 8(2), 165-183. doi: 10.1177/1748895808088993.

• Gonzalez, T. (2012). Keeping kids in schools: restorative justice, punitive discipline, and the school to prison pipeline. Journal of Law Education, 41(2), 281-336.

• Gregory, A., & Clawson, K. (2016). The potential of restorative approaches to discipline for narrowing racial and gender disparities. In R. J. Skiba, K. Mediratta, & M. K. Rausch (eds.), Inequality in School Discipline: Research and Practice to Reduce Disparities (pp. 153-170). New York: Plagrave Macmillan.Gregory, A., Clawson, K., Davis, A. & Gerewitz, J. (2016). The promise of restorative practices to transform teacher-student relationships and achieve equity in school discipline. Journal of Educational and Psychological Consultation, 26(4), 325-383.

• Gregory, A., Ward-Seidel, A.R. & Cater, K.V. (2021). Twelve indicators of restorative practices implementation: a framework for educational leaders. Journal of Educational and Psychological Consultation, 31(2), 147-179. doi: 10.1080/10474412.2020.1824788.

• Ishiyama, J. & Laoye, O. (2016). Do truth commissions promote trust in the judiciary in African states? Journal of Asian and African Studies, 51(5), 528-544. doi: 10.1177/0021909614552915.

• Kang-Brown, J., Trone, J., Fratello, J. & Daftary-Kapur, T. (2013). A generation later: what we’ve learned about Zero Tolerance in schools. Issue Brief. Retrieved from www.vera.org/downloads/Publications/a-generation-later-what-weve-learned-about-zero-tolerance-in-schools/legacy_downloads/zero-tolerance-in-schools-policy-brief.pdf (last accessed 2 March 2022).

• Katic, B. (2017). Restorative Justice Practices in Education: A Quantitative Analysis of Suspension Rates at the Middle School Level. (MS in Counseling and Guidance), California State University, San Bernardino, San Bernardino, CA, USA. Retrieved from scholarworks.lib.csusb.edu/etd/495 (495). Last accessed 17 August 2021.

• Katic, B., Alba, L.A. & Johnson, A.H. (2020). A systematic evaluation of restorative justice practices: School violence prevention and response. Journal of School Violence, 19(4), 1-15. doi: 10.1080/15388220.2020.1783670.

• Knoch, U. (2014). Using subject specialists to validate an ESP rating scale: the case of the International Civil Aviation Organization (ICAO) rating scale. English for Specific Purposes, 33, 77-86. doi: 10.1016/j.esp.2013.08.002.

• Kohli, R., Montano, E. & Fisher, D. (2019). History matters: challenging an a-historical approach to restorative justice in teacher education. Theory into Practice, 58(4), 377-384. doi: 10.1080/00405841.2019.1626613.

• Latimer, J., Dowden, C. & Muise, D. (2005). The effectiveness of restorative justice practices: a meta-analysis. The Prison Journal, 85(2), 127-144. doi: 10.1177/0032885505276969.

• Llewellyn, K.R. & Parker, C. (2018). Asking the ‘who’: a restorative purpose for education based on relational pedagogy and conflict dialogue. The International Journal of Restorative Justice, 1(3), 399-412. doi: 10.5553/IJRJ/258908912018001003005.

• Lodi, E., Perrella, L., Lepri, G.L., Scarpa, M.L. & Patrizi, P. (2022). Use of restorative justice and restorative practices at school: a systematic literature review. International Journal of Environmental Research and Public Health, 19(1), 96-130. doi: 10.3390/ijerph19010096.

• Lokugamage, A.U. & Pathberiya, S.D.C. (2017). Human rights in childbirth, narratives and restorative justice: a review. Reproductive Health, 14(1), 1-8. doi: 10.1186/s12978-016-0264-3.

• Long, P. K. (2015). Diminishing the Discipline Gap: Restorative Justice as a promising Alternative in One Urban School. (Education Specialist in Psychology), Univerity of Dayton, Dayton, OH, USA.

• Mallicoat, S.L. & Gardiner, C.L. (eds.). (2014). Criminal justice policy. Los Angeles: Sage.#

• Mansfield, K. C., Fowler, B., & Rainbolt, S. (2018). The potential of restorative practices to ameliorate discipline gaps: The story of one high school’s leadership team. Educational Administration Quarterly, 54(2), 303-323#.

• Marsh, V.L. (2017). Restorative practice: history, successes, challenges# & recommendations. Retrieved from www.rochester.edu/warner/cues/wp-content/uploads/2020/12/Restorative-Practices-Brief-1_marsh_final.pdf (last accessed 2 March 2022).

• Martinez, A., Villegas, L., Ayoub, L.H., Jensen, E. & Miller, M. (2022). Restorative justice and school-wide transformation: identifying drivers of implementation and system change. Journal of School Violence, 21(2)#, 190-205. doi: 10.1080/15388220.2022.2039682.

• Mears, D.P. (2010). American criminal justice policy: an evaluation approach to increasing accountability and effectiveness. New York: Cambridge University Press.#

• Morrison, B.E. & Vaandering, D. (2012). Restorative justice: pedogogy, praxis and discipline. Journal of School Violence, 11(2)#, 135-155. doi: 10.1080/15388220.2011.653322.

• Mustian, A.L., Cervantes, H. & Lee, R. (2022). Reframing restorative justice in education: shifting power to heal and transform school communities. The Educational Forum, 86#(1), 51-66. doi: 10.1080/00131725.2022.1997510.

• NJDOE. (2021). Restorative justice in education pilot program. Trenton, New Jersey: New Jersey Department of Education.

• Olson, J. & Sarver, R.S. (2021). How restorative are you? Introducing the restorative index. Victims & Offenders, 17(6), 941-943. doi: 10.1080/15564886.2021.1942358.

• Popa, C.N. (2012). Restorative justice: a critical analysis. Union of Jurists of Romania Law Review, 2(3)#, 1-13.

• Roland, K., Rideout, G., Salinitri, G. & Frey, M.P. (2012). Development and use of a restorative justice ideology instrument: assessing beliefs. Contemporary Justice Review, 15(4)#, 435-447. doi: https://doi.org/10.1080/10282580.2012.734574

• Schotland, M., MacLean, H., Junker, K. & Phinney, J. (2016). From punitive to restorative: one school’s journey to transform its culture and discipline practices to reduce disparities. In R.J. Skiba, K. Mediratta & M.K. Rausch (eds.), Inequality in school discipline: research and practice to reduce disparities (pp. 225-242). New York: Palgrave Macmillan.

• Stinchcomb, J. B., Bazemore, G., & Riestenberg, N. (2006). Beyond zero tolerance: Restoring justice in secondary schools. Youth Violence and Juvenile Justice, 4(2), 123-147#. doi:10.177/1541204006286287.

• Sumner, M. D., Silverman, C. J., & Framptom, M. L. (2010). School-based Restorative Justice as an Alternative to Zero-Tolerance Policies: Lessons from West Oakland. Retrieved from https://www.law.berkeley.edu/files/thcsj/10-2010_School-based_Restorative_Justice_As_an_Alternative_to_Zero-Tolerance_Policies.pdf (last accessed 18 March 2024).

• Toews, B. (2006). The little book of restorative justice for people in prison: rebuilding the web of relationships. New York: Good Books.#

• Umbreit, M.S. & Armour, M.P. (2011). Restorative justice and dialogue: impact, opportunities, and challenges in the global community. Washington University Journal of Law & Policy, 36, 65-89#.

• USDOE. (2016). Non-regulatory guidance: using evidence to strengthen education investments. Washington, DC: US Department of Education. Retrieved from www2.ed.gov/policy/elsec/leg/essa/guidanceuseseinvestment.pdf (last accessed 28 March 2022).

• Vaandering, D. & Reimer, K. (2019). Listening deeply to public perceptions of restorative justice: what can researchers and practitioners learn? The International Journal of Restorative Justice, 2(2)#, 186-208. doi: 10.5553/IJRJ/258908912019002002002.

• Vincent, C. & Girvan, E. (2017). Positive and restorative investment in discipline reform in education (PRIDE): Integrating school-wide positive behavior interventions and supports (SWPBIS) and restorative discipline (RD). University of Oregon, National Center for Education Research. Funded Research Grants and Contracts. Washington DC: Institute of Education Sciences. https://ies.ed.gov/funding/grantresearch/details.asp?ID=2038 (last accessed 17 August 2021).

• Wachtel, T. (2016). Defining restorative. Retrieved from https://www.iirp.edu/images/pdf/Defining-Restorative_Nov-2016.pdf (last accessed 8 March 2023)#.

• Wood, W.R. & Suzuki, M. (2016). Four challenges in the future of restorative justice. Victims and Offenders, 11(1)#, 149-172. doi: 10.1080/15564886.2016.1145610.

• Woods, C.A. & Stewart, M.L. (2018). Restorative justice: a framework for examining issues of discipline in schools serving diverse populations. The International Journal of Restorative Justice, 1(1)#, 81-95. doi: 10.5553/IJRJ/258908912018001001005.

• Wong, J. S., Bouchard, J., Gravel, J., Bouchard, M., & Morselli, C. (2016). Can at-risk youth be diverted from crime?: A meta-analysis of restorative diversion programs. Criminal Justice and Behavior, 43(10), 1310–1329. https://doi.org/10.1177/0093854816640835

• Zakszeski, B. & Rutherford, L. (2021). Mind the gap: a systematic review of research on# restorative practices in schools. School Psychology Review, 50(2-3), 371-387. doi: 10.1080/2372966X.2020.1852056.

• Zehr, H. (2015). Changing lenses: restorative justice in our times (25th Anniversary ed.). Harrisonburg: Herald Press.#

Noten

• 1 Potential stakeholders, or essential people, in a school-based restorative justice programme include the person harmed, the person who harmed, family and/or support for these individuals, and members of the greater community, whether affiliated with the greater school-wide community or community at large.