Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Archive for the ‘Research’ Category

Resources for Courts Considering and Developing ODR Programs

Jennifer Shack, March 19th, 2020

With the COVID-19 epidemic leading courts to cease in-person activities, courts may be feeling a greater urgency to start down the path of online dispute resolution programs. I thought I’d share some resources that may help courts to make decisions regarding whether and how to implement such programs.

Considerations and Concerns in ODR Program Design

Online Dispute Resolution Special Topic

Resolution Systems Institute

RSI has written a guide for courts who are considering, have started developing or already have ODR programs. It discusses important considerations for ODR implementation, from goal setting to costs to ethical concerns. 

Read RSI’s advice about ODR on our website.

Considerations in Implementing Court ODR Systems

Doug Van Epps and Michelle Hilliker. Michigan Supreme Court State Court Administrators Office of Dispute Resolution. Jan. 6, 2020

Van Epps and Hilliker share their insights and the knowledge gained from their development and implementation of ODR in the Michigan courts in this guide. Based on both the issues they encountered and their discussions with others involved in implementing ODR systems, their considerations are meant to assist courts to determine how to design, implement and evaluate an online dispute resolution (ODR) system. 

The considerations span a variety of topics including leadership and court staff; prospective users and stakeholders; goals; implementing authority and legal implications administration; platform attributes and functions; mediators; non-court dispute resolution service staff; costs, fees and funding sources; confidentiality; protections; vendor selection; data collection and evaluation; and marketing plans. The guide also includes a list of recent ODR publications and resources.

Access the Considerations document on Michigan’s Supreme Court State Court Administrator’s Office website.

Case Studies in ODR for Courts

Joint Technology Committee, 2020

This paper presents seven case studies of ODR implementation in the courts. The case studies are short, but include key takeaways about what worked and what didn’t. The ODR programs include two outside the US, and deal with small claims, family, tax and traffic cases.

Read the case studies.

Online Dispute Resolution: A Digital Door to Justice or Pandora’s Box? Parts I and II

Doug McQuiston and Sharon Sturges, Colorado Lawyer, February and March, 2020.

McQuiston and Sturges are in the midst of publishing a three-part series on ODR in the courts that examines the use of videoconferenced mediation. They note that videoconferencing may be appropriate for family cases and those involving intimate partner violence. The main obstacle to providing this service is limited or poor internet connectivity. 

Part II focuses on the use of artificial intelligence in ODR. McQuiston and Sturges cite the many benefits of AI-assisted ODR for small claims and family cases, such as the ability to negotiate asynchronously, which eliminates the need to coordinate schedules. Self-represented litigants who may be reluctant to attend mediation without an attorney may be more inclined to use this technology. Further, they can save money by using this technology. McQuiston and Sturges note some drawbacks, however. These include AI’s inability to understand and address human emotions and its tendency to deviate to the mean, without reference to shades of gray in disputes or situational fairness. To help readers understand how AI in ODR would work, they end by describing systems already in place around the world. 

Read Part I and Part II of the series. Part III is forthcoming in April.

Designing and Implementing a State Court ODR System: From Disappointment to Celebration

David Allen Larson, Journal of Dispute Resolution, Vol. 2019, No. 2, Jun. 5, 2019

This article chronicles the author’s work to develop an online dispute resolution (ODR) system to handle credit card debt collection in New York State courts. The author worked with the New York State Unified Court system for a little over two years to design and implement their ODR platform. The article discusses the issues related to dispute system design in this setting, explains how the project was derailed and ends with lessons learned. The four lessons discussed are (1) anticipate conflicts and resistance (2) obtain support from judges and court staff at the beginning (3) figure out the technology while also ensuring a fair vendor bidding process and (4) pick your case type carefully.

The article also touches on some issues specific to ODR such as how ODR relates to a court system that is not fully digitized and how long to retain records of online communications. It also offers general advice, such as recommending that ODR processes should “balance efficiencies, neutrality and self-determination.”

Read the full article on SSRN.

Pouring a Little Psychological Cold Water on ODR

Jean Sternlight, Journal of Dispute Resolution, 2020

This article explores online dispute resolution (ODR) from a psychological lens to examine the strengths and weakness of ODR. The article examines the psychology of dispute resolution by focusing on four different areas: the psychology of perception and memory, the psychology of human wants, the psychology of communication, and judgment and decision making. Sternlight’s article suggests that ODR may not be the best tool to assist individuals in creatively working things out with a fellow disputant and may be better employed for small and predictable disputes, like small online purchases. The article also posits that computers may not be the best forum for communication and argues that human mediators, lawyers or friends are more effective than computers in helping humans deal with their emotions and other judgement and decision-making issues. Sternlight ends by calling for empirical research for both online and in-person dispute resolution.

Read the full article on SSRN.

Studies of Online Dispute Resolution Programs

So far, there have been few published studies of online dispute resolution programs in the courts. Below are two conducted a while back. 

Getting Divorced Online: Procedural and Outcome Justice in Online Divorce Mediation

Martin Gramatikov and Laura Klaming, Journal of Law & Family Studies, Jan. 1, 2012

This study of a Dutch experiment with ODR for divorcing couples found that the participants perceived the process to be fair, with procedural fairness, interpersonal justice and informational justice all given high marks. On a scale of 1 to 5, they had averages of 4.27, 4.5 and 4.19, respectively. The participants’ perception of the outcome was also positive, though to a lesser extent than for the procedure. They gave an average of 3.91 for distributive justice, 3.37 for restorative justice, 3.18 for functionality and 3.0 for transparency. The ratings were similar for both men and women. Other findings included men reporting higher out of pocket costs and time spent in mediation than women, and women reporting higher levels of frustration and anger than men.

The participants were referred to ODR, which was provided free of charge, if both parties had an email account and the issues were not complex. Once referred, the parties completed an intake questionnaire to provide the mediator with some details about the dispute. The parties could communicate with the mediator and each other via text message or email. The mediator moderated all communications. Each party was required to respond to the other within 48 hours as a condition of the agreement to mediate. Once all issues in dispute were finalized, the parties completed an evaluation of the procedure before the agreement could be finalized.

Read the complete abstract and access the full study in RSI’s Research Library. 

Evaluation of the Small Claims Online Dispute Resolution Pilot

Marc Mason, Avrom Sherr. Sep. 1, 2008

Two courts in England tested online mediation to resolve 25 small claims cases. Those parties who were willing to try mediation were given the option of mediating face-to-face, by telephone, or online. Two mediators were responsible for all online mediations, which were conducted using TheMediationRoom.com.

The online mediations resulted in settlement in 48% of the cases, which was similar to the settlement rate for the face-to-face and telephone mediations, but lower than other small claims mediation programs have reported. Mediators and parties were surveyed post-mediation about their experiences with the process. Mediators reported using more than one method of communication outside TheMediationRoom.com platform – generally email or telephone – to complete the mediation in most cases, and as many settlements were completed outside the platform as within it. The mediators attributed this to difficulties in getting responses from the defendants, as well as to technical difficulties. Because of this and because they lacked the ability to judge non-verbal cues, the mediators said they would have preferred using telephone or email in all but four cases.

The 18 parties who responded to the questionnaire were less frustrated with their experience than the mediators. They expressed fewer issues with the technology, with 47% saying the technology was easy to use. However, they were not overwhelmingly satisfied with the process or the fairness of the outcome. Only 53% were satisfied with their experience and only 23% felt the outcome was fair. Responses to both satisfaction and fairness of outcome were more positive for those who settled their case. The small number of responses limits the reliability of these findings.

Read the full study on SSRN.

Studies Regarding Particular Issues Related to ODR

Shuttle and Online Mediation: A Review of Available Research and Implications for Separating Couples Reporting Intimate Partner Violence or Abuse

Fernanda S. Rossi, Amy Holtzworth-Munroe, Amy G. Applegate, Connie J. Beck, Jeannie M. Adams, Darrell F. Hale. Family Court Review (Association of Family and Conciliation Courts), Aug. 17, 2017

This article examines the published research on shuttle mediation, online audio-visual mediation, and online text-based mediation to discuss the applicability of these mediation methods to family law cases with a history of intimate partner violence and/or abuse (IPV/A). It first presents potential advantages and disadvantages of each mediation method in cases with IPV/A history. The authors suggest that mediators on IPV/A cases must carefully consider a variety of potential issues including the parties’ suspicion of mediator bias, confidentiality concerns and victim-perpetrator power dynamics. The authors also note the need for more empirical research comparing different effects of various mediation methods.

This article is behind a paywall on the Wiley Online Library.

Building Trust Online: The Realities of Telepresence for Mediators Engaged in Online Dispute Resolution, 

Susan Nauss Exon and Soomi Lee. Stetson Law Review, Vol 49, No. 1, 2019

Exon and Lee found that trust in an experienced mediator is the same whether a mediation participant interacts with that mediator via video or face-to-face. In their experiment, a single experienced mediator conducted 31 simulated mediations with one party in the room with him and the other interacting via telepresence. Telepresence is sophisticated video conferencing, in which sensitive microphones and special cameras that pan and zoom are used to help participants follow the flow of the conversation. 

During the experiment, the participants were asked to complete a questionnaire before the mediation began that measured their level of interpersonal trust. They then completed a second questionnaire after mediation that asked them about their interactions with the mediator and their perceptions of him in order to determine how much they trusted him and found him to be trustworthy. In all, 59 participants provided usable data. 

Nauss Exon and Lee found that although the participants’ questionnaire responses before mediation indicated they were on average more likely to distrust others than trust them, all participants agreed mildly or strongly that they could trust the mediator and that the mediator was trustworthy. Further, they found no difference in the level of response (mild or strong) between those who were in the same room as the mediator and those who participated via telepresence, with one exception. They found that those who had a lower predisposition to trust were more likely to see the mediator as trustworthy.

Read the full study on SSRN.

Studies Regarding Topics Related to ODR

ADR Empirical Research Studies

James Coben and Donna Steinstra. Mitchell Hamline Dispute Resolution Institute, Jun. 1, 2018

This compilation of abstracted studies includes a number on topics that are related to the use of ODR. These include: 

  • A study of compliance with emailed requests
  • Team decision-making in a virtual environment
  • A qualitative analysis of email negotiation
  • Honesty in face-to-face communication as compared to through an intermediary

Find the abstracted studies on the Mitchell Hamline website.

Parents See Conflict Reduction and Relationship Benefits from Mediation in Massachusetts

Jennifer Shack, March 2nd, 2020

Custody and parenting time mediation in Massachusetts is providing parents with multiple benefits while facilitating agreements. The most recent evaluation of the Parent Mediation Program in four counties, published by the Massachusetts Office of Public Collaboration in 2019, found that 74% of mediations ended in an agreement. Additionally, parents reported multiple benefits beyond agreement, including a reduction in conflict, better conflict resolution skills, greater civility and better communication.

Services for the program are provided by community mediation centers, who conduct intake with the parents and are contracted to provide one session at no charge to the parents. If additional sessions are needed, the parents agree to pay the center on a sliding fee schedule. For the evaluation, mediators were asked to complete a report after each mediation session. Additionally, mediators asked parents to complete a survey after the last mediation session (150 parents across 80 cases did so) and center staff conducted phone interviews with 94 parents in 70 cases four to ten weeks after mediation ended.

During fiscal year 2019 (July 2018 – June 2019), 141 cases were referred to the centers. Almost 2/3 of these referrals were from the courts and the rest were from the community. During this same time period, 129 mediations were completed. In 74% of these, some form of agreement was reached: 30% full agreement, 34% partial agreement, 16% temporary agreement). In surveys, 93% of parents said they needed to devise a parenting plan, and 77% said that mediation either fully (43%) or partially (34%) helped them with that. In their reports, mediators indicated that mediation led to progress on the parenting plan in a similar percentage of cases, at 80%.

Parents and mediators were asked about other benefits experienced through mediation. In surveys, parents said that conflict between them and the other parent was diminished in about 2/3 of the mediations, an assessment with which mediators agreed – stating conflict was diminished in 69% of mediations. This benefit appeared to last for weeks after mediation for many parents, as 53% of those who were interviewed said that conflict continued to be reduced.

Similarly, more than 2/3 of surveyed parents reported greater civility between them and the other parent. Again, this benefit remained over time, with 50% saying that they and the other parent treated each other with greater civility. Most parents also said that their communication had improved, with 72% of those surveyed saying so and 54% of those interviewed weeks later agreeing.  It’s not surprising, then, that 70% of surveyed parents, and 54% of those who were interviewed, believed their skills for resolving conflict had improved.

While research has shown these benefits to be important for the emotional well-being of the children, this study points to another effect. Nearly half of surveyed parents said that less conflict with the other parent and 33% said better communication with the other parent would help them to financially support their children.

Reduced conflict and better communication did not necessarily lead to greater involvement with their children, however. Roughly half of those who were surveyed said that the other parent’s time with the children decreased and 20% said there was no difference. In interviews, parents continued to see little to no difference in the other parent’s involvement in their children’s lives. Nonetheless, 36% of custodial parents reported that the other parent’s involvement was greater than before.

The many benefits identified by parents were likely one reason they had a positive experience in mediation. Fully 97% said they would use mediation again and 99% would recommend it to others. Large majorities also thought the mediator was fair and unbiased (84%), listened well to their concerns (82%), identified relevant issues (80%) and helped generate ideas (78%).

The full study includes more background information on the level of conflict between the parents (29% had a high level), complications between the parents, demographics and the parents’ custodial status. 

Early Resolution Triage Program for Family Cases Increases Efficiency without Reducing Satisfaction

Jennifer Shack, February 3rd, 2020

In Anchorage, Alaska, an Early Resolution Program for family cases has reduced time to resolution, reduced staff time spent on cases and had no impact on the number of post-disposition motions to modify, according to a recently completed evaluation. The program includes triage, dispute resolution and attorney assistance with documents. Stacey Marz reports on her study in, “Faster and as Satisfying: An Evaluation of Alaska’s Early Resolution Triage Program” (Family Court Review, 2019).

In the Early Resolution Program (ERP), cases involving two self-represented litigants are reviewed by a staff attorney to determine if they are appropriate for the program. The staff attorney generally screens out cases involving: current or serious domestic violence incidents; issues that require evidentiary findings; a pending child abuse or neglect case; or a non-parent who is asserting a custodial claim.

If a case is appropriate, the parties are notified of the opportunity to participate. If they agree to do so, they are either provided two unbundled lawyers who work collaboratively (and only on dispute resolution), a court mediator or a settlement judge to help them resolve the case. If the parties reach agreement, they go to court to finalize their divorce approximately three weeks after entering the program. There, a staff attorney completes the final documents, including the findings of fact and conclusions of law, parenting plan, divorce decree, and child support order. The judge reviews and signs all the documents, which are then copied and distributed in the courtroom. The judge then grants the divorce and the parties leave with all the documents.

For the evaluation, Marz looked at 299 ERP cases from 2011 to 2013 that resolved through settlement and compared them to a control group of 392 cases closed before the ERP was implemented. These cases followed a traditional court track involving an initial status conference, a pre-trial conference and a trial or settlement conference. For the study, the pre-ERP cases were screened the same way as cases filed after the implementation of the ERP in order to make them comparable.   

Marz found a high rate of resolution for ERP participants, with 80% reaching agreement in a three-hour dispute resolution session. In addition, ERP cases were able to close more quickly. The time from filing to closure for ERP cases was a median of 42 days as compared to a median of 104 for cases in the control group.

As a proxy for party satisfaction with the process, Marz also looked at the number of motions filed per case to modify the judgment. The assumption is that parties file motions to modify soon after the final judgment if they are unhappy with the outcome. She found no significant difference between the two groups: for the ERP group,.18 motions were filed per case within two years of disposition, compared to .22 motions per case for the control group. According to Marz, this indicates that concerns that the ERP process was too quick and parties didn’t have enough time to think about the issues were not supported.  

The last item Marz examined was the number of processing steps staff undertook for each case. For cases undergoing ERP, there are 28 to 30 processing steps, taking a total of 240 minutes (4 hours). The number of steps for the average non-ERP case is 49, taking a total of 1,047 minutes (17.45 hours). Marz notes that the reason for this increased efficiency is two-fold: First, once the staff attorney screens and accepts a case into ERP, the file stays with the attorney, eliminating many case-processing steps that occur in typical cases. Second, there are great efficiencies in scheduling multiple cases during the same ERP hearing block, especially when most cases resolve in one court event.

The evaluation indicates that the Early Resolution Program in Anchorage has increased efficiency in cases involving self-represented litigants without inducing parties to enter into agreements that aren’t sustainable.

Research Year in Review

Jennifer Shack, December 19th, 2019

This year we learned about the efficacy of a variety of dispute resolution programs, explored party trust in mediators, and considered ways to assess and improve the party experience in dispute resolution processes.

Program evaluations

The theme of evaluations of ADR programs is that participants like them. This includes the multiple studies I summarized for our Mediation Effectiveness Studies database. Each of the studies that looked at the participant experience found high levels of satisfaction and, in comparative studies, the level of satisfaction was greater for those going through mediation than those who did not.

Nevada and Michigan provided us with insights into their child protection mediation (CPM) programs. The evaluation of Nevada’s statewide program found that family members were highly positive about all aspects of the mediation. All of them thought the process was fair. Almost all said they were able to voice their opinions, were treated with respect and were able to be a part of finding answers to the problems discussed. Almost 90% said the others really listened to them. For all practical purposes, these responses did not vary based on whether they were foster parents or natural parents. The study also found that there was no difference in satisfaction rate based on the stage at which mediation occurred, but that satisfaction was higher when mediation resulted in agreement as compared to when it did not.

Child protection mediation participants in Michigan were also highly satisfied with their experience in mediation. In the two counties with experience data, participants said that they had the opportunity to express themselves, gained a better understanding of the issues, felt respected and felt the process was fair to them. In a statewide comparison between those family members who participated in mediation and those who did not, through who went through CPM gave slightly higher ratings on case resolution, staff courtesy and courtesy of the judge.

I also discussed an evaluation I conducted of the Civil Stalking Pilot Mediation Program in Ohio.  While there wasn’t enough survey data to determine how satisfied participants were with their experience, we did find that almost ¾ of these cases reached agreement, with most of the agreements being for the parties to have no further contact.

Switching to restorative justice, we learned that participants in Nebraska’s Youth Conferencing Program also have high rates of satisfaction. In addition to being satisfied with the process, most participating youth and victims believed that the victim youth conferencing VYC made the justice system more responsive to their needs. Further, almost all of the victims agreed that it was helpful to talk directly with the person who was responsible for the harm, and most said that meeting that person reduced any fear that he/she would commit another crime against them. The youth held similar opinions: most said it was helpful to talk directly with the victim and almost all of them said that after the meeting they had a better understanding of the full impact of the crime on others. In addition to providing a positive experience to participants, youth conferencing led to a high rate of agreement completion, with almost 95% of the youth ending with full compliance.

Research

From a study of one restorative justice program in Nebraska, we move to a compilation of forty years of research on victim offender mediation (VOM). The research found that victims receive multiple benefits from VOM. Victims who participate in VOM as compared to those who went through the traditional process received more restitution, had less fear of re-victimization and were less likely to remain upset after the process. Additionally, participation in VOM leads victims to feel empowered and to have a more humanized view of the offender.

Offenders were also found to receive benefits from the VOM process. These included being more empathetic toward victims of their crimes, being held directly accountable to victims (which is a core need for many offenders), being able to deal with their feelings and seeing victims change their feelings toward them, feeling empowered, and avoiding jail or court.

Other benefits accrue to the courts and community. Some studies found that VOM led to a higher rate of agreement completion than the traditional process. Many studies, but not all, found that VOM led to a lower recidivism rate than the traditional process. A few studies also found VOM to be cost-effective. Short-term, VOM lowers costs by being less expensive than court. Long-term, lowered recidivism and lower incarceration rates lead to cost savings. One study found as well that mature programs were more cost-effective than those that were just started up, and another found that those involving community mediation centers were more cost-effective.

Two of the studies I summarized were about trust in the mediator. We learned that participant trust in the mediator is the same whether mediation is in person or via a sophisticated video conferencing tool, in which sensitive microphones and special cameras that pan and zoom are used to help participants follow the flow of the conversation. We also learned that mediators and parties have similar perspectives on what mediator behaviors and characteristics engender trust in parties, with the exception of eight factors for which the parties were significantly more likely than the mediators to say built trust with mediator. These were:

  • The mediator’s familiarity with legal aspects relating to the dispute
  • The mediator suggests an alternative or a way out of the dispute
  • The mediator provides candid and frank input about the dispute
  • The mediator does not linger too long on the dispute but advances toward its settlement
  • The mediator is appointed by an authority, such as a reputed judge
  • The mediator shows an interest in the parties’ mutual concerns and focuses on their common goals
  • The mediator highlights the rules of mediation
  • The mediator talks to the parties about informal matters as opposed to just talking about the dispute

We also looked at research focusing on access to justice. A study of judicial settlement conferences in Canada examined what factors were related to parties’ sense of access to justice, as indicated by their sense that the process was fair and useful, as well as their sense that the settlement judge provided professional support. These included: the judge being active in helping the parties to reach a resolution that is fair and suited to their needs; communication that creates trust; the judge facilitates a just process and a process that complies with ethical norms.

Two of my summaries were of articles that used research to argue for changes to how we approach disputes resolution. Nancy Welsh argued that transparency is essential to help to equalize the knowledge of one-shot users and repeat players, allow for public oversight, and make it less likely that mediators would engage in unethical behaviors. She then noted that mediation is now often imposed on parties through contracts and mandated by the courts. For these mediations, there is a greater responsibility to ensure that the processes are fair and effective. Thus, Welsh argues for a new set of standards for these “compelled” mediations that call for more information to be imparted about mediations.

Jean Sternlight focused her attention on online dispute resolution (ODR), arguing that ODR should be designed to take into account human psychology. That requires an imaginative approach to determining whether and how to incorporate technology into dispute resolution. She suggests that “computers are not likely the best tool for helping humans think through how they want to respond to a dispute, and how they might creatively work things out with a fellow disputant.” She concludes that ODR may best be used for disputes that involve wants that are simple and predictable, such as small online purchases.

I’m looking forward to a research-filled 2020!

Perceptions of Trust-Building in Family Mediation

Jennifer Shack, November 26th, 2019

A small study of family mediation in Spain paints a picture of what parties and mediators believe promotes the parties’ trust in the mediator and points out differences in perspectives between the two groups. Identification of these differences can lead to improved training and professional practice, according to the researchers (Joan Albert Riera Adrover, Maria Elena Cuatrero Castaner, Juan Jose, Montano Moreno). They discuss this in their article, “Mediators’ and Disputing Parties’ Perception of Trust-Building in Family Mediation” (Negotiation and Conflict Management Research, 2019).

For the study, the researchers recruited mediators from a single mediation provider organization. The mediators notified the parties of the research project at the outset of the mediation. If the parties consented to participate in the research, a researcher would meet with the parties at the end of the third joint session and had them complete a survey. Mediators also completed a survey at this time. In all, 31 mediators and 54 parties participated in the research over the course of a year. The mediators were primarily lawyers with an average of 1-6 years’ experience .

The researchers found eight factors which the parties were significantly more likely to say built trust with their mediator than compared to the mediator group. These were:

  • The mediator is familiar with legal aspects relating to the dispute
  • The mediator suggests an alternative or a way out of the dispute
  • The mediator provides candid and frank input about the dispute
  • The mediator does not linger too long on the dispute but advances toward its settlement
  • The mediator is appointed by an authority, such as a reputed judge
  • The mediator shows an interest in the parties’ mutual concerns and focuses on their common goals
  • The mediator highlights the rules of mediation
  • The mediator talks to the parties about informal matters as opposed to just talking about the dispute

Most of these should not be surprising. Indeed, for all of the above factors except suggesting an alternative and talking about informal matters, a majority of the mediators agreed with the parties about their importance for building trust. However, it does reinforce the notion that trust is about more than just rapport. Parties in this study trusted mediators who focused on resolution and presented alternative options to the parties.

It should be noted that there were numerous other factors for which a large majority of both mediators and parties said built trust. These included the mediator’s listening skills, fairness, impartiality, demeanor and capacity to understand the dispute. These all should be familiar to anyone involved in mediation, as they are generally considered necessary for trust-building.

This research is unique in that it matched up mediator and party perspectives on trust. It can help mediators to know what they should be emphasizing as they attempt to build trust. However, it was limited by a small sample size and lack of qualitative follow up to help elucidate the mediators’ and parties’ thinking.