Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Archive for the ‘Program Evaluation’ Category

RSI Evaluating Online Dispute Resolution

Just Court ADR, January 30th, 2020

Resolution Systems Institute is pleased to announce that we are partnering with the University of California, Davis, (UC Davis) and The Pew Charitable Trusts (Pew) to evaluate certain small claims court online dispute resolution (ODR) programs in Hawaii and Texas. These programs enable parties involved in small claims matters to negotiate online and, if needed, receive online mediation services to attempt to resolve their case. These programs could eliminate the need for some parties to appear in physical courthouses.

This evaluation examines the impact of ODR on litigants and courts. For litigants, the focus will be on access to justice, with an eye toward determining whether certain groups in society tend to experience the benefits or costs associated with ODR more than others. To that end, the study will explore, among other questions, whether ODR reduces the time stress on parties; affects case outcomes; improves litigants’ use of court procedures and navigation of court rules; and increases users’ sense of procedural fairness. The study will also look at the effect ODR has on court efficiency, such as time from filing to resolution, the number of hearings held and staff time spent per case.

ODR is increasingly being adopted by courts in the United States, but there have not been neutral, empirical studies before this evaluation and other concurrent evaluations in partnership with Pew. An initial report is expected in mid-2021.

“We are excited for this opportunity to bring rigorous empirical research to the nascent field of court ODR,” said Susan M. Yates, RSI Executive Director. “While we are hopeful that ODR can improve access to justice, we will await the results to see what we learn.”

Resolution Systems Institute’s principal investigator for the project is Jennifer Shack, Director of Research, the go-to evaluator for court ADR in the US. The principal investigator for the overall project is Dr. Donna Shestowsky of the UC Davis School of Law; she is a nationally recognized scholar in alternative dispute resolution.

Two New Chapters of RSI’s Guide to Program Success now Available!

Nicole Wilmet, November 27th, 2019

We are thrilled to announce that two new chapters of our Guide to Program Success are now available! RSI’s Guide to Program Success combines the expertise of Executive Director Susan Yates and Director of Research Jennifer Shack and discusses how to effectively design, manage and evaluate successful court ADR programs.

Newly released Chapter 12: Create Your Program Forms addresses how to create forms for court programs and includes a set of exemplary sample forms from courts around the United States. Chapter 13: Launch Your Program, focuses on the successful launch of a court program takes a look at things that court sometimes don’t think about when starting a program.

We hope you find these resources valuable in your work!

Characteristics of Quality Court ADR Programs

Susan M. Yates, September 18th, 2019

What characteristics do you think are shared by quality court ADR programs? I took a swing at a list here. What would you add? Change?

1. Goals

There must be a shared understanding of the goals of the ADR program so that there can also be a shared understanding about whether it is succeeding.

  1. The court must be clear about its goals for the ADR program
  2. Those goals must be shared with stakeholders

 

2. Principles

The ADR program must operate on a common set of foundational principles, including ethics as appropriate for the ADR process being provided.

  1. Confidentiality (in mediation)
  2. Fairness of process
  3. Fairness of outcomes
  4. Procedural justice
  5. Accessibility
  6. Neutral quality
  7. Self-determination (in mediation)
  8. Timeliness

 

3. Accessibility

Everyone – including litigants and lawyers – must be able to readily access the ADR program.

  1. Parties who are unable to pay for ADR are afforded the opportunity to use ADR
  2. Parties who are representing themselves are able to participate fully in ADR
  3. Parties with disabilities are able to participate fully in ADR

 

4. Process Quality

Notwithstanding the importance of other characteristics, the true quality of a court ADR program boils down to what happens during each ADR session.

  1. Whichever ADR process is being used, it adheres to the foundational principles of that process
  2. Participants have an experience of procedural justice when engaging in the ADR process:
    1. They feel they had a voice in the process (e.g., had a chance to talk, felt they were heard)
    2. They feel they were respected in the process
    3. They feel the process was fair to them

 

5. Program Support

The ADR program will only succeed if it receives steady support from a number of sources.

1. Stakeholders understand and support the ADR program

  • Judges
  • Litigants
  • Court staff
  • Neutrals

2. The program has sufficient, stable financial support

 

6. Neutrals

Neutrals are the face of the program to litigants and lawyers, so they must provide quality services.

  1. Neutrals share a common understanding of the service they are to provide
  2. Neutrals provide services in the manner expected by the program and its stakeholders
  3. Neutrals operate in an ethical manner
  4. Neutrals have the necessary skills and knowledge
  5. Neutrals are selected for the roster fairly
  6. Neutrals are appointed to cases in a fair manner
  7. Neutrals receive appropriate initial and ongoing training
  8. Neutrals provide a sufficiently uniform version of the ADR process
  9. Parties have a way to lodge complaints about neutrals
  10. Neutrals are treated fairly when a complaint against them is made
  11. Neutrals are compensated fairly (which does not preclude volunteer mediators)
  12. Neutrals are assessed fairly
    1. Participant surveys
    2. Peer review

 

7. Lawyers

Lawyers must support, or at the very least accept, the ADR program.

  1. Lawyers are often repeat users of the ADR program, therefore their support is especially important
  2. This applies equally to those who typically represent one side or the other, e.g., landlords’ lawyers and tenants’ lawyers.
  3. Lawyers should:
    1. Find the program useful
    2. Be educated about the program
    3. Fulfill their responsibilities in the program

 

8. Program Safety

ADR processes must be safe, both literally and figuratively.

  1. Participants, neutrals and staff are all safe when participating in ADR and otherwise interacting with the ADR program
  2. Participants are screened prior to ADR, when appropriate, to identify intimate partner violence and other potential barriers to participation

 

9. Data Collection and Dissemination

Reliable data must be collected regularly and shared appropriately, or it won’t be available when it is time to sustain or improve the ADR program.

  1. Data is collected regularly and includes both quantitative and qualitative information (e.g., statistics and success stories)
  2. Changes in the program are tracked and acted on, e.g., changes in:
    1. The number of cases being sent to ADR
    2. The number of agreements being reached
    3. The number of mediators signing up to mediate
    4. The number of parties showing up for ADR sessions
  3. Data is turned into reports that can be readily digested
    1. Easy to read
    2. No use of insider language or acronyms
    3. Fitting use of charts and other visuals
  4. Reports are adapted to their particular audience and disseminated appropriately, typically as follows:
    1. Judges and court administrators with direct responsibility get the most detailed reports
    2. Those higher up in the court system get summaries with explanations
    3. Funders’ reports depend on what the funder requires. They may also get some anecdotes about how parties are experiencing the program.
  5. Reports include recommendations for action when appropriate
  6. The program is evaluated near the end of the first year, after a substantial period of time, or when there has been a major change in the program or the context in which the program operates

 

10. Program Promotion

Reminding stakeholders of the value of the ADR program will help maintain its support.

  1. Brochures are made available in courtrooms and other locations as needed
    1. Brochures are only produced if they will serve a particular purpose, such as giving parties the number they need to call to schedule a mediation
    2. If brochures are targeted to self-represented litigants, they must be in easy-to-understand language
  2. News about the program is publicized via:
    1. Local radio, TV, newspaper, news websites
    2. Court newsletter
    3. Bar association newsletter
  3. Program is featured on appropriate websites
    1. Information about the ADR program can be readily found on the court’s site
    2. If the program maintains its own site, instead of appearing as part of the court’s site, it must be easy to find and navigate and kept up to date
  4. As members of stakeholder groups change, the new individuals are familiarized with the program
    1. Judges are educated on how it works, ethical limitations, what to expect
    2. Lawyers, e.g., child protection attorneys, assigned to courtrooms are trained in how to use the program
    3. Staff at funders are educated about the program’s benefits and history
    4. Court staff are educated about the program’s benefits and how it fits into court processes

 

11. Program Administration

Someone wakes up every morning with the feeling that it is their job to do everything on this list to ensure the quality and continuity of the ADR program.

  1. Individuals with power over the program, e.g., funding or case referrals, are kept informed about the program
  2. The ADR program operates in accordance with applicable laws, court procedures and rules
  3. The program functions efficiently and effectively, e.g.:
    1. Cases are referred to ADR in a consistent manner
    2. Cases are scheduled promptly
    3. Reports are provided to court on time
    4. The benefits provided by the ADR program are reasonable in relationship to the costs of the program
    5. Changes in the program are tracked and potential responses suggested when needed

 

Participants Are Highly Satisfied with Nevada’s Child Protection Mediation Program

Jennifer Shack, July 1st, 2019

Last month, I talked about a new evaluation of child protection mediation in Michigan. I’m following this up with a 2017 evaluation of child protection mediation in Nevada. Both evaluations were of several programs taking place throughout the respective states, but their focuses are quite different. Where the Michigan study primarily examined time to permanency, the Nevada study focused much more on participant experience in the mediation and process issues.

The Nevada study, “Process Evaluation of Nevada’s Statewide Dependency Mediation Program,” by Shamini Ganasarajah, et al, of the National Council of Juvenile and Family Court Judges, found high levels of satisfaction with mediation and agreement, as well as a possible impact on whether scheduled hearings after mediation were cancelled. The study also found that there was no difference in satisfaction rate based on the stage at which mediation occurred, but that satisfaction was higher when mediation resulted in agreement as compared to when it did not.

The study looked at mediation in seven counties. In these counties, mediation can be used at any point of the case. However, most cases used the mediation program at the termination of parental rights (TPR) stage, which is at the end of the case. (This finding regarding the timing of mediation is skewed somewhat by one county using mediation almost exclusively at the TPR stage.) Time in mediation averaged two hours.

Those who participated in mediation were asked to complete post-mediation surveys. For the purposes of this study, these people were divided into program participants (these are natural parents and foster parents) and system stakeholders (the attorneys and case workers involved in the case). During the study period (July 2016 through April 2017), participants completed 113 post-mediation surveys and stakeholders completed 267. In their responses, 84% of the participants and 98% of the stakeholders expressed satisfaction with the mediation program. Their satisfaction was statistically related to whether they reached agreement in the mediation.

The participants (family members) were highly positive about all aspects of the mediation. All of them thought the process was fair. Almost all said they were able to voice their opinions, were treated with respect and were able to be a part of finding answers to the problems discussed. Almost 90% said the others really listened to them. For all practical purposes, these responses did not vary based on whether they were foster parents, natural mothers or natural fathers.

The evaluators analyzed whether there was a relationship among the participants’ responses. One that stood out was that when participants believed others in the mediation had “really listened” to what they had to say, they were more likely to express satisfaction with the mediation regardless of whether an agreement was reached.

The stakeholders (attorneys and caseworkers) were also highly positive about the mediation, with all or almost all believing the process was fair, that they had an opportunity to express their opinion, were treated with respect, were listened to and were able to be a part of finding answers to the problems discussed.

Interestingly, both participants and stakeholders were most likely to mention communication as what was most helpful about the mediation. Both groups were also most likely to say that parties being unable or unwilling to compromise was the reason no agreement was reached.

Also interesting was that the mediators reported agreements in 84% of cases, while the stakeholders reported that agreement resulted from only 71% of their mediations. There is no explanation as to why. The study also found that hearings were cancelled after 51% of the mediations were held. The evaluators recommended further examination of the relationship between mediation and vacated hearings.

Other recommendations included expanding the use of mediation to all stages of the case, as most mediations occurred at the TPR stage; assessing implementation of domestic violence screening protocols; and enhancing mediator training to include additional strategies for effectively listening to participants and stakeholders and making them feel heard.

Verified by ExactMetrics