As ODR rolls out in courts across the United States, we need to learn more about how litigants experience it. This is particularly true about their experience of access to justice, which is a primary motivator for courts to adopt ODR. Assessing how litigants view their experience with the court helps courts learn what works and what does not and can better ensure that they provide access to justice for all their constituents.
This article discusses plans for an evaluation Resolution Systems Institute and the University of California, Davis, will be conducting of ODR programmes in Hawaii and Texas for small claims cases.¹xThe principal investigator for this evaluation is Donna Shestowsky, Director of Lawyering Skills Education and Martin Luther King Jr. Professor of Law, UC Davis School of Law; Faculty Member of the Psychology Graduate Group, UC Davis. She and Jennifer Shack are partnering on the project. The project is funded by the Pew Charitable Trusts. Both programmes will be mandatory and will be using a platform involving asynchronous, text-based negotiations, followed by asynchronous, text-based mediation if agreement is not reached through negotiation.
The two programmes will also be mandatory. For that reason, we cannot do a comparison between cases going through ODR and those going through the regular court process during the same period. Additionally, the Texas programme has already been launched. We will therefore collect pre-launch baseline survey and court data in Hawaii for those going through the regular court process. In Texas, we will examine baseline court data, but instead of gathering baseline survey data, we will ask litigants going through ODR to complete a survey before they start negotiations on the ODR platform and another one at the end of their case.
Just a quick note: For the evaluation, we will be looking at a wide range of variables, including effects on filing rates, outcomes and court costs. For this presentation, I will focus specifically on the part of the evaluation that will measure litigant experience.

• 1 Access to Justice

  Courts are often motivated to implement ODR as a way of helping litigants, particularly self-represented litigants, to have greater access to court services and to be better able to navigate court procedures. Ideally, ODR does so while preserving or enhancing litigants’ perception that they are being treated justly. There are many ways to measure access to justice. For our evaluations, we will be working with the concepts of access and justices separately, although they are symbiotic.

  1.1 Access

  We will examine the ‘access’ part of access to justice in terms of the barriers litigants face in navigating the court process and obtaining an outcome they perceive as just. These barriers include cost, time, knowledge and negative emotions.

  1.1.1 Cost and Time Barriers

  Cost is considered to be a significant barrier to access to justice, particularly for those with low incomes. Costs such as lost wages and the need to pay for child care may figure into an individual’s decision as to whether to pursue a case or defend one. Associated with cost is time, both how much time a litigant must spend on a case and how long it takes for a case to reach resolution.
  For this reason, we will be asking litigants a number of questions about their costs and the time spent on their case. We will not be asking for them to provide dollar figures, but instead will ask what types of expenses they incurred. Dollar figures would be difficult for them to assess and would not necessarily provide us with better information than if we asked them what types of expenses they incurred.
  We will be collecting time data from the court and the ODR platform as well as from the litigants themselves. From the litigants, we will ask how much time they spent on the case, including time preparing for trial or negotiations. For those who go to court, we will ask about the time it took to get there and the amount of time they spent in the courthouse. For those going through ODR, we will ask about the amount of time they spent on the platform.
  Because we do not believe that litigants will be able to be precise or possibly accurate in their responses about the time they spend on the platform (because they would be online multiple times over the course of the negotiations and mediation), we will be using analytics from the ODR platform to determine the amount of time litigants spend on the platform in general. This will allow us to place in context the litigants’ responses to questions such as ‘How do you feel about the amount of time you spent to resolve your case?’ and ‘How did the amount of time the negotiation/mediation took compare with what you expected?’
  In addition to examining the amount of time the litigants spent on their case, we will also assess the amount of time it took for them to proceed through each step of the process. From the court, we will get information on time from filing to hearing and disposition. From the ODR platform, the data will include days in negotiation and days in mediation, as well as the number of days from the first contact with the platform to the last contact.

  1.1.2 Knowledge Barriers

  A common theme in access to justice is self-represented litigants’ (SRLs’) lack of knowledge of their legal rights, such as whether they have a defence against a debt collection case, and of court processes. Without knowing their legal rights, SRLs cannot properly vindicate their rights or defend themselves. Without understanding what they need to do with their case, SRLs may lose valuable time or, worse, have their case dismissed or be subject to a default judgment. ODR provides the possibility of addressing this by providing access to information and guiding the litigants through the process.
  We will examine whether ODR is addressing SRL knowledge through two sources. First, we will be asking SRLs directly about their understanding of the process and their rights, as well as what information they accessed both from the platform and from more traditional sources, such as the court website and self-help centres. Second, we will use the ODR platforms’ analytics to determine how often litigants clicked on links to information or accessed information pages on the platforms.

  1.1.3 Emotional Barriers

  Negative emotions can wear out litigants involved in lawsuits. Our evaluation will include survey questions exploring what the litigants felt as they proceeded through ODR. These will include questions about how they felt before the process started – were they excited, nervous or overwhelmed? – and whether they felt frustrated or angry with the process or the other party.
  We will also explore litigants’ level of comfort with the process, including, for ODR participants, their comfort with the technology and their comfort level with negotiating with the other party. For those going through the traditional court process, we will ask them about their comfort in speaking before the judge.

  1.1.4 Process Completion

  A final check of the impact that barriers have on the litigants’ engagement with the process will be to examine the reasons that litigants cease their participation before resolving their case – that is, they quit the process. We will ask those who did not complete the ODR process to say why. Was it due to time or cost issues? Did they not understand the process? Were they frustrated with the process or the other party? Did they just give up?

  1.2 Justice

  For the evaluation, we will explore two areas of justice: the litigants’ perceptions of procedural justice and outcome justice. When assessing these, we will include the effect of reaching agreement and trial result on the litigants’ responses.

  1.2.1 Procedural Justice

  Procedural justice examines issues that affect litigants’ perception of the process. When litigants feel that they are respected, that they have voice, that they were treated impartially and that the process was transparent, they perceive the process to be just. To explore these factors, we will ask litigants questions such as ‘Were you able to talk about the issues and concerns that were most important to you?’ ‘Did the mediator/judge understand what was important to you?’ ‘Did the mediator/judge treat you with respect?’ ‘Were the rules clear to you?’ ‘Were they equally applied to you and the other party?’

  1.2.2 Outcome Justice

  We will be looking at whether the litigants perceive the outcome to be just, rather than attempting to determine whether it is objectively just. The latter requires more information than we will have and would in any case not necessarily be a relevant measure of justice, as those who negotiate an outcome have many reasons to agree to particular terms.
  Although we will look at types of outcomes, such as the number of default judgments and agreed judgments (which are particularly important for cases in which there is an inexperienced litigant on one side and a repeat player on the other), in terms of litigant experience, we will be examining other aspects of outcome justice. This includes whether the outcome meets the litigants’ needs. To find this out, we will ask such questions as whether the outcome fits with their needs, whether they felt they had control over the outcome and whether they were satisfied with it. We will also ask whether they understand the outcome of the case, and, finally, we will look at whether the outcome worked. That is, did the case return for an enforcement of the judgment or agreement?

• 2 Demographics

  Our evaluation will include an examination of how these factors vary based on litigant demographics. These include age, race/ethnicity, education level, income and gender. In addition, we will explore possible differences based on whether they were represented by an attorney and whether they have been previously involved in court cases.

• 3 Conclusion

  ODR has the promise to increase access to justice for litigants, particularly those who lack legal counsel. This evaluation will, hopefully, help courts better understand how litigants experience access to justice on ODR platforms and what might increase litigants’ sense of both access and justice.

Noten

• 1 The principal investigator for this evaluation is Donna Shestowsky, Director of Lawyering Skills Education and Martin Luther King Jr. Professor of Law, UC Davis School of Law; Faculty Member of the Psychology Graduate Group, UC Davis. She and Jennifer Shack are partnering on the project. The project is funded by the Pew Charitable Trusts.