Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Archive for the ‘Online Dispute Resolution’ Category

Significant Participation, Agreement Levels Highlight Potential of ODR for Family Cases

Jennifer Shack, August 31st, 2022

University of California, Davis, Professor Donna Shestowsky and I recently had the pleasure of conducting the first neutral evaluation of any family law court online dispute resolution (ODR) program in the United States. The program was launched by the 20th Circuit of Michigan’s Friend of the Court (FOC) in August 2020 with the goal of providing parties with post-judgment family law disputes a simpler, more convenient and cost-effective way to reach agreements related to child custody, parenting time and child support. It also aimed to increase efficiency in the disposition of these matters. By and large, we found that the program was providing the benefits the FOC hoped it would.

What We Studied

The FOC used Matterhorn’s text-based ODR platform, which allows participants to communicate with each other and their caseworker via asynchronous text messages and document exchanges. We used case data, ODR data, pre- and post-process party surveys, and staff interviews to gain insight into:

  • Parties’ expectations for the ODR process at the time it was offered to them, and their views on a video mediation alternative
  • ODR access, including the percentage of parties who participated and opted out, information about ODR available to parties, and parties’ capacity to use ODR
  • Participants’ evaluation of their experience of ODR in terms of procedural justice, satisfaction, fairness of the process, and ability to control the outcome of their matter
  • Parties’ impressions of the FOC and the other party
  • The agreement rates, hearing rates, and efficiency (time to disposition, caseworker time spent on matters) associated with ODR use
  • Direct costs and the FOC’s staff members’ perceptions of the effect of ODR on their work

Excitement and Anxiety

We found that before ODR, parties tended to be confident they could reach agreement, but the majority did not believe the other party would be truthful. They expressed high levels of excitement about using ODR, but also high levels of anxiety. Those who were planning on using it were twice as likely to report a high level of fear of the other party as those who weren’t sure they would, or who were not going to, use ODR.

Because of the platform’s technological limitations, caseworkers did not offer ODR to parties who had attorneys, those who had limited English proficiency, or to those who were blind or visually impaired. In addition, the FOC had decided that those who had a history of a high level of conflict would not benefit from ODR, and they were therefore not offered the opportunity.

Uptake Higher than for Other ODR

Almost half of the parties who were offered the chance to use ODR did so, which was high compared with other early ODR programs, which have had participation rates of 21% to 36%. In half the cases in which ODR was offered but not used, at least one of the parties simply did not register to use it despite the FOC’s intent to require its use. We noted that the emails sent to the parties informing them to register for ODR used language that could have confused the parties both about what the program was and their requirement to participate.  We also found that parties lacked an understanding of the main features of the program. This, too could have reduced program use.

A surprising finding was that almost all parties who used ODR accessed the platform with their mobile phone at least part of the time. Only 8% exclusively accessed the platform with a computer, while 71% only used their mobile phone. This indicates that the platform and any auxiliary activities, such as communications to the parties and agreement forms, must be optimized for phones.

When parties used ODR, they were four times as likely to give high ratings for fairness of the process as those who did not use ODR (50% vs. 12.5%), and twice as likely to give high ratings for satisfaction (50% vs. 25%). They were also much more likely to reach agreement than those who were offered ODR but didn’t use it (59% vs. 11%). In cases involving child support, parties using the online platform reached resolution nearly twice as fast as those who did not.

Positive, with Room for Improvement

Our evaluation indicates that the FOC’s ODR program has provided parties with a positive experience, improves agreement rates and reduces time to resolution for some cases. Program use is high in comparison with other programs.

However, our analysis suggests that the FOC could do more to educate parties about the program, direct parties to use it, and increase access to parties with disabilities as well as those who need an interpreter’s assistance to use the platform. The FOC should also explore ways to reduce access barriers for those identified by caseworkers as less likely to use or benefit from the program because they have lawyer representation or have high-conflict relationships. Attempts to reduce access barriers should also be directed at those who lack digital literacy and those who would use their mobile phone to access the platform. Our evaluation was limited by a small survey and case data sample size, which may have obscured statistical significance of some findings and did not allow us to conduct more detailed analyses of participant experience. We are looking forward to seeing more evaluations of ODR programs conducted, to build on our findings.  

Limited Participation Reduces Success of Otherwise Promising Texas Pilot ODR Program, Evaluation Finds

Just Court ADR, May 31st, 2022

A newly published study conducted by RSI Director of Research Jennifer Shack and University of California, Davis, Professor Donna Shestowsky highlights both the potential of online dispute resolution (ODR) and the importance of appropriate outreach and education on ODR to maximize participation and, thus, program impact.

Jen and Donna evaluated a pilot program in Collin County, Texas, that used a text-based ODR platform to resolve debt and small claims cases in a single court. By adopting ODR, the court sought to reduce the burden of a growing caseload while providing access to justice through a process that did not require the parties to travel or miss work to resolve their case.

The evaluation produced evidence that ODR can be an effective method of dispute resolution, especially for debt cases. In 73% of cases where both parties used the ODR platform, participants resolved their dispute and avoided trial. The rate is similar to that of cases that had the opportunity to use in-person mediation. Unfortunately, however, the program’s goals were not met, because both sides used ODR in only 24% of cases uploaded to the platform. Findings and recommendations related to these outcomes are discussed later in this summary.

The Pilot Program

The pilot ran from September 2019 through August 2020; thus it was concurrent with the COVID-19 pandemic from March to August 2020. Additionally, because ODR is a rather new practice in this context, some details of the program and the platform’s application were being worked out even as the evaluation took place.

The Modria ODR platform allowed parties to communicate by text one-on-one or with the help of a mediator. Parties were ostensibly required to use the platform before their first hearing. If parties reached agreement, their case was dismissed without a trial. During the 12-month pilot period, 1,874 debt and 274 small claims cases were filed, for a total of 2,148 cases.

When a defendant filed an answer, the civil clerk determined whether the case was eligible for ODR. It was ineligible if one side had multiple parties, if a party was not equipped to use ODR, or (until the second quarter of 2020) if the court did not have email addresses for both parties or their attorneys. If the case was eligible, the clerk uploaded it to the ODR platform, which in turn rejected any cases that contained errors, such as missing information, and sent an error report to the IT department so the errors could be fixed. When email addresses and phone numbers were available, the platform sent an automated email (and after April 2020, also a text) to the parties, instructing them to use ODR. The clerk also set the case for trial and mailed the parties, or their attorneys, a notice of their trial date and informed them they were required to use ODR prior to that date. The notice included a link to the platform.

Once a case was uploaded to ODR, participants had 45 days to negotiate one-on-one via the platform’s chat function. At any time during this window, either side could ask for a mediator. Mediation cost each party $40 and had to be completed within 30 days.

If participants reached agreement on the platform, they were given the opportunity to sign an online agreed judgment form, which was automatically sent to the case management system, and the trial was cancelled. If the participants did not reach agreement, the parties continued to trial unless they otherwise reached agreement before the trial date.

Key Findings

Below are some of Jen and Donna’s main findings and top recommendations from their evaluation. For more details and complete recommendations, read the full report here.

Litigant Use of ODR

  • 49% of cases with answers filed were uploaded to ODR. During the pilot period, answers were filed in 698 cases. These 698 cases form the subset that could potentially have been uploaded to ODR. Of these, 341 cases (49%) were eligible and did not contain errors that barred their upload. These were ultimately offered ODR. According to court staff, the most common reason that cases with answers filed were not uploaded to ODR appears to be that the court lacked email addresses for at least one side of the case.
  • One party completed at least one activity online in 50% of cases uploaded to ODR. In 170 of 341 cases (50%), at least one case participant performed at least one activity on the ODR platform, such as asserting a claim, uploading a file, or using the chat function to communicate with the other side.
  • Both sides completed at least one activity on the ODR platform in about one-fourth of eligible cases. In 81 cases (24%) uploaded to ODR, both sides used the platform. Parties in small claims cases were more likely to use ODR (76%) than parties in debt claim cases (45%).
  • Litigants appeared to be unaware of the ODR program. Litigant survey responses suggested that parties were generally not aware of the ODR program, despite participation being required. Only one survey respondent out of ten indicated having received information about the program. When asked what would make them more likely to use ODR for a similar case in the future, half said more information.
  • Litigants had limited access to information about the ODR program. According to court staff, the only ways litigants received information from the court about the ODR program was through the notice the court mailed to them (or their lawyers) about their court date and through an email or text from the platform when the court uploaded their case, if their side had an email address or cellphone number on file with the court. Both of these events occurred only after the defendant filed an answer.
  • Litigants appear open to online options. Among survey respondents, none of whom had participated in ODR, two out of three indicated that the option to use it in future similar cases was attractive. Similarly, when asked to consider using video mediation to resolve future similar cases, 60% responded favorably. 

Outcomes and Time to Disposition

  • 73% of cases in which both parties used ODR resolved before trial. The percentage of ODR cases that resolved before trial was similar to that of cases that did not use ODR, both before and during the ODR program.
  • Debt claim cases were significantly more likely than small claims cases to resolve before trial. Additionally, debt claim cases in which defendants were represented were significantly more likely to resolve before trial than debt claim cases in which defendants were unrepresented.
  • Time to resolution was, on average, 4.6 months for cases that used ODR. This figure includes cases delayed either because of the court’s closure amid the COVID-19 pandemic or because of an upload error on a court server.

Program Costs

It is important to note that workload and cost conclusions are derived from self-reports made during interviews and are inherently subjective.

  • Direct costs to the court to implement ODR were covered by a filing fee. Litigants covered the costs through an extra $5 filing fee the court instituted for all civil cases filed in Collin County except eviction and mental health cases.
  • There were significant indirect costs to the court. Court personnel indicated that they devoted a significant amount of time to ODR prior to its launch. The project manager estimated that the cost in staff time approached six figures and was largely due, in his opinion, to the numerous meetings that involved many court personnel as well as the high percentage of time that he and the responsible IT staff member spent on the project in this phase. Some of this effort laid the groundwork for an anticipated county-wide rollout of ODR.
  • Costs to administer ODR were minimal. After the program’s launch, the time that personnel spent on ODR appeared to drop considerably. No one interviewed reported spending more than a couple of hours per week on the project.
  • ODR did not appreciably change administrative workload. The court administrator and the civil clerk did not perceive an appreciable increase or decrease in their workload. However, it is hard to determine what their workload may have been in the absence of the COVID-19 pandemic, or how much it would have been had greater effort been expended on promoting litigants’ awareness of the program and otherwise attempting to increase ODR use.

Recommendations

Based on the findings of this evaluation, the following recommendations may be relevant for any court considering implementing ODR:

  • Expect to spend significant time and resources to get the program up and running.
  • Notify parties and lawyers about the ODR program early in the process.
  • Educate litigants and lawyers more fully about the program.
  • Conduct outreach to raise awareness of, and promote interest in, the ODR program.
  • Explore video mediation as a dispute resolution option.

Q&A Highlights Importance of RSI/UC Davis Research on Procedural Justice in ODR

Susan M. Yates, June 28th, 2021

The Pew Charitable Trusts has published a Q&A with Professor Donna Shestowsky of the UC Davis School of Law about the empirical study of online dispute resolution (ODR) that the law school and Resolution Systems Institute are conducting, which looks at court ODR programs in Michigan and Texas. The Q&A explores procedural justice, an integral element of the study.

RSI’s Director of Research, Jennifer Shack, and Professor Shestowsky are working with the Pew Charitable Trusts to evaluate online platforms that courts are using to assist parties in engaging in mediation and negotiation. RSI and UC Davis are one of three research teams involved in a larger project initiated by Pew to empirically study issues surrounding the provision of ODR by courts and its effects.

While these platforms could offer benefits, such as eliminating the need for some parties to appear in person for a trial, the study looks at possible drawbacks as well. In addition to procedural justice, the study also looks at other questions, such as whether ODR reduces the time and stress on parties and courts; affects case outcomes; and improves litigants’ use of court procedures and navigation of court rules.

For more on how courts are using ODR, visit RSI’s Special Topic on Online Dispute Resolution.

New York Court Launches ODR Pilot Program for Small Claims Cases

Nicole Wilmet, February 26th, 2021

In January, Manhattan’s Civil Court launched an online dispute resolution (ODR) pilot program for small claims cases. The program is designed to assist self-represented parties and eligible cases include disputes regarding the purchase or sale of goods or services up to $10,000. Cases meeting these criteria will automatically be referred to the program and then be screened for eligibility. If eligible, parties will have the opportunity to negotiate through the platform to resolve their dispute.

New York’s new program uses the Matterhorn platform and is free for parties. In a press release for the program, the court indicates that the new ODR platform will lead parties through an “automated double-blind bidding process, in which each party makes an offer that may only be disclosed after both offers match [and] depending on the outcome.” In this helpful video, the court explains what this process looks like in practice. As the video explains, once on the platform, both parties will be invited to submit a proposed settlement amount. In the example from the video, if one party submits a settlement amount for $800 and the other party presents a settlement amount for $200, then the program will use the overlap between these two amounts and split the difference in half (ex: $500). From there, the platform will present this amount to the parties as a proposed solution. If the parties are able to agree to the proposed amount, the platform will then guide the parties through negotiating the other terms of the settlement.

In the event parties are unable to reach an agreement, the ODR platform will then present several other resolution options to the parties. First, the platform will give parties the opportunity to communicate with each other through a chat interface to see if they are able to reach a settlement. Second, if the parties are still unable to reach agreement, they will be given the option to work with a mediator. Finally, if the parties do not want to proceed with chat and mediation, or were unable to reach an agreement in mediation, parties will then have the option to proceed to court. For additional information about the program, visit the program’s website or contact the Civil Court of the City of New York by phone at 646-386-5484.

Research Year in Review

Jennifer Shack, December 18th, 2020

The past year we focused on research that related to the times we’ve been experiencing. With courts going online and an expected surge in evictions on the horizon, I turned my attention to those topics, summarizing research on online dispute resolution (ODR) and presenting outcomes from housing mediation programs. 

Online Dispute Resolution

In March, I rounded up the research to date on ODR. A study in the Netherlands found that participants in ODR for divorcing couples perceived the process to be fair, with procedural fairness, interpersonal justice and informational justice all given high marks. On a scale of 1 to 5, they had averages of 4.27, 4.5 and 4.19, respectively. The participants’ perception of the outcome was also positive, though to a lesser extent than for the procedure. They gave an average of 3.91 for distributive justice, 3.37 for restorative justice, 3.18 for functionality and 3.0 for transparency.

A small study of a pilot small claims ODR program had less positive results. It found that 47% of cases reached agreement. The 18 parties who responded to a survey had some issues with the technology, with only 47% saying the technology was easy to use. In addition, only 53% were satisfied with their experience and only 23% felt the outcome was fair.

In the round up, I also summarized research about the potential advantages and disadvantages of using video-based and text-based ODR in cases with a history of intimate partner violence or abuse (IPV/A). The researchers suggest that mediators on IPV/A cases must carefully consider a variety of potential issues including the parties’ suspicion of mediator bias, confidentiality concerns, and victim-perpetrator power dynamics. 

While others in 2020 wrote about the possibilities for ODR, Jean Sternlight examined some of its weaknesses. Her article explored online dispute resolution (ODR) through the lens of the psychology of dispute resolution, focusing on four different areas: the psychology of perception and memory, the psychology of human wants, the psychology of communication, and judgment and decision making. She concluded that ODR may not be the best tool to assist individuals in creatively working things out with a fellow disputant and may be better employed for small and predictable disputes, like small online purchases.

An RSI survey found that the COVID pandemic has led most states to adopt video mediation for family cases. Others are moving forward with formal ODR platforms. Despite the increased availability of online services, almost half of the states that responded to the survey said there was an unmet need for family ODR and that funding was the main requirement for meeting that need. 

During the past year, we also learned about how to design ODR platforms from a study of Utah’s small claims platform, and were given tips on researching the impact of ODR on access to justice. 

Eviction Mediation

Two articles published in this year discussed programs in Minnesota and Missouri to help landlords and tenants avoid eviction. The results of these programs indicate that they help keep evictions off tenant credit histories and reduce forcible evictions. 

In St. Paul, Minnesota, the court instituted multiple changes to its housing court, including expanding access to mediation and making it, along with financial and legal resources, available at the court during eviction hearings. After a year and a half, the court’s numbers appear to show an improvement in outcomes. The court has a goal of reducing evictions by 50% in five years. In the first 18 months, evictions declined by 8%, to the lowest eviction rate in 10 years. Settlements increased by 5%, to the highest rate in five years. The impact was highest on expungements, which doubled. On the other end, fears of increased trial numbers and longer court calls didn’t come true. The number of trials as a proportion of cases declined and court call length increased by 10 minutes on average.

In St. Louis, in a voluntary program for cases in which neither landlord nor tenant is represented, 71% of mediated cases resulted in a settlement in 2018. The terms of more than half of these agreements were completed, resulting in a dismissal. One-third of agreements resulted in a consent judgment for eviction against the tenant and 25% resulted in the sheriff executing the judgment through forcible removal of the tenant. Cases that went to trial, on the other hand, were significantly more likely to end in eviction. Consent judgments were entered against tenants in 92% of these cases and resulted in forcible removal in 40%. The authors extrapolate from that data that 279 families avoided eviction in 2018 by settling in mediation and completing the terms of their agreement rather than going to trial.

Family ADR

Studies of family ADR programs continue to demonstrate the benefits of helping parents to resolve their issues outside of court. 

In Anchorage, Alaska, an Early Resolution Program (ERP) for family cases reduced time to resolution, reduced staff time spent on cases and had no impact on the number of post-disposition motions to modify, according to a recently completed evaluation. The study found that 80% of the parties who participated in ERP reached agreement in a three-hour hearing. Unsurprisingly, ERP cases reached disposition more quickly, with a median of 42 days as compared to a median of 104 for cases in the control group. The program also led to significant time savings for staff. For cases undergoing ERP, there were 28 to 30 processing steps, taking a total of 240 minutes (4 hours). The number of steps for the average non-ERP case was 49, taking a total of 1,047 minutes (17.45 hours).

study of parenting time mediation in Massachusetts found multiple benefits for parents and families. In surveys, parents said that conflict between them and the other parent was diminished in about 2/3 of the mediations. This benefit appeared to last for weeks after mediation for many parents, as 53% of those who were interviewed said that conflict continued to be reduced. Similarly, more than 2/3 of surveyed parents reported greater civility between them and the other parent. Again, this benefit remained over time, with 50% saying that they and the other parent treated each other with greater civility. Most parents also said that their communication had improved, with 72% of those surveyed saying so and 54% of those interviewed weeks later agreeing.  

Litigant Perception Research

Litigant attendance at a dispute resolution process impacts their assessment of the fairness of that process, according to research conducted by Donna Shestowsky. Shestowsky found that when litigants attended a settlement procedure used to resolve their case, they rated that procedure as fairer than those litigants who attended an adjudicative procedure. However, when litigants did not attend the procedure used to resolve their case, they saw settlement and adjudicative procedures as similarly fair. When comparing attendance within procedures, she found that attendance did not affect fairness ratings for settlement procedures, but that those who attended an adjudicative procedure rated the procedure as less fair than those who did not attend the procedure.

I wish you all a happy, safe and healthy holiday season!