Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Posts Tagged ‘evaluation’

Most Give High Ratings for Mediator Fairness, Trust in Mediator in Recent Surveys of RSI’s Kane County Eviction Mediation Program

Jasmine Henry, January 10th, 2024

RSI administers an eviction mediation program in Kane County, Illinois. Every quarter, we provide a report to the court on the participants’ experience in mediation based on their responses to a post-mediation survey.

Between July 1, 2023, and September 30, 2023, 174 eviction mediations were held in the 16th Judicial Circuit of Illinois (Kane County). After every mediation, participants were invited via email or text to complete an online survey about their experience; not all of the participants completed surveys. In our latest survey report, we examined participant responses from those three months. Specifically, we focused on participant opinions regarding fairness, trust and satisfaction. In all, 21 tenants, one landlord and 11 attorneys responded. The participants responded to the questions according to a seven-point scale, which we consolidated into three categories: low (1–2), medium (3–5), and high (6–7). Participants were invited to add comments to some of their responses. Their responses are summarized below.

Trust in Mediator, Perceived Fairness

We asked respondents about their perception of the mediator. Specifically, we asked: “How fairly did the mediator treat you?” And, “How much did you trust the mediator?” Almost two-thirds of participants gave high ratings for mediator fairness and trust. However, respondents tended to rate mediator fairness higher than mediator trust. For example, fewer than 3% of respondents thought the mediator did not treat them fairly, while 15% of respondents had low trust in the mediator. There was a parallel, albeit smaller, difference observed in the positive ratings, with 63% of respondents rating the mediator as very fair, compared with 58% who had high trust in the mediator.

Turning more broadly to respondents’ perception of the mediation process as a whole, we asked: “Overall, how fair was the mediation process?” Most of the participants who responded felt that the mediation was fair overall, with 62% saying it was highly fair. Notably, this is very similar to the percentage of respondents who said the mediator was highly fair. Not all of the respondents were impressed with the process, and 10% of respondents rated the mediation a little fair or not at all fair.

Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.”

Comments of Tenants, Attorneys

We asked respondents to explain their overall fairness ratings. The landlord did not comment, but many tenants and some attorneys did. Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.” A quarter of the tenant comments mentioned court-based rental assistance, which tenants were often referred to by the program. Several tenants also saw the mediators as helping, saying, “They stood up for me … They didn’t let [the landlord] push me,” and “[We asked] for what we wanted and [the mediator] basically fought for us to get it.”

In contrast, tenants who gave medium and low ratings on overall fairness tended to focus their frustrated comments on the mediator’s relationship with the landlord. One tenant said the mediator “may have been more partial to the landlord” because they “were familiar with one another”; another tenant said plainly that “they are there to mostly help the landlord.” One tenant felt frustrated that the mediator did not seem to believe what the tenant said at mediation, saying, “The mediator seemed to take what I had to say about the situation with a grain of salt.” Attorneys who rated the overall fairness at a medium or low level focused on efficiency, with one saying, “I was disappointed that the mediator allowed the opposing side to spend valuable time on issues irrelevant to the case.”

Likelihood to Recommend Eviction Mediation

To further explore participant satisfaction, we asked participants: “If a friend or colleague had a dispute like yours, how likely are you to recommend eviction mediation?” Most of the participants who responded were likely to recommend mediation to a friend or colleague, with 67% saying they were highly likely to recommend it. One tenant commented, “I would recommend all mediation options; sometimes tenants are unaware of the resources available due to lack of communication or shame.” However, another tenant who was less satisfied with the process commented, “It doesn’t help the tenant. At all. It helps landlords.”

As was the case with the first question on participant satisfaction, the landlord did not comment on their responses to this question, but we did receive two attorney comments. One attorney who was highly satisfied with the mediation process commented, “We made the exact same settlement offer that was accepted at mediation to the landlord’s attorney months ago, and they never responded in any way despite multiple phone calls. I assume this was on their client’s instructions. Because of the mediation process, I believe they would have continued stonewalling us.” The attorney who was unlikely to recommend mediation to a colleague said: “The lengthy mediation process is not helpful in my view. Before this system was implemented, and still now (in other counties), I am often able to reach agreements with the tenants within 5–10 minutes in the hallway outside the Courtroom. There is no need for the mediator, in my opinion.”

Conclusion

In conclusion, the survey responses indicate that the program continues to provide a positive experience to most participants. Those who completed the survey generally had positive perceptions of the mediators and the program, with the majority giving high ratings on fairness, trust and satisfaction. However, some participants’ comments point to a perception among tenants that mediators are biased toward the other side and a perception among attorneys that the mediation process is not efficient.

Spring 2022: How RSI’s Work Has Expanded and Evolved During the COVID-19 Pandemic

Just Court ADR, May 18th, 2022

As the COVID-19 pandemic has strained finances, families and the social fabric in general, the need for and potential of well-designed alternative dispute resolution (ADR) to stitch together solutions has become increasingly apparent. Throughout this challenging time, Resolution Systems Institute (RSI) has continued to monitor and evaluate new dispute resolution initiatives and developed new court mediation programs of our own. Below is a description of some of our work during the pandemic.

In 2020, as a potential pandemic-driven eviction wave loomed, RSI began to design our first eviction mediation program. In the course of about nine months, we developed the court rules, procedures and forms the program would need to operate. We also recruited a cadre of mediators and provided them with specialized eviction mediation training through our friends at the Center for Conflict Resolution. The Kane County (Illinois) Eviction Mediation Program launched in spring 2021. Later that fall, RSI launched similar programs in Illinois’ Kankakee and Winnebago counties. Overall, our programs operate in judicial districts that serve over 920,000 Illinois residents.

To learn more about RSI’s work over the last two years in the eviction arena, we recommend you check out our blog entries on the topic, as well as our Eviction Mediation Special Topics resource. RSI is now working with Kane and Winnebago counties to bolster their longstanding foreclosure mediation programs as homeowners now begin to feel the squeeze that renters started experiencing last year.

While RSI is spearheading these new program development and administration initiatives, evaluation remains the central pillar of upholding RSI’s mission of improving access to justice. RSI Director of Research Jennifer Shack recently published two reports evaluating the program development experience in Kane County: Eviction Mediation Design and Implementation in Illinois’ 16th Judicial Circuit: Challenges and Keys to Success and Participant Experience in Eviction Mediation: Summary Of Early Survey Responses in the 16th Judicial Circuit of Illinois’ Video Mediation Program.

In partnership with the University of California, Davis, RSI has also evaluated online dispute resolution (ODR) pilot programs in Texas and Michigan. The pandemic generated greater interest among courts for ADR processes that parties could access remotely. The evaluations, which will be released soon, will provide courts with a better understanding of what ODR adoption requires and what possible benefits it can provide.

The last two years have provided many of us, RSI included, with a complex mix of setbacks and new opportunities. RSI is committed to innovating and adapting to meet the challenges that courts, and the litigants they serve, encounter in this ever-changing world. We are grateful to our program partners, our funders and each of you who come to RSI in search of expertise and guidance. We hope you will continue to take this journey with us as we work towards our mission of expanding access to justice through court alternative dispute resolution.

Characteristics of Quality Court ADR Programs

Susan M. Yates, September 18th, 2019

What characteristics do you think are shared by quality court ADR programs? I took a swing at a list here. What would you add? Change?

1. Goals

There must be a shared understanding of the goals of the ADR program so that there can also be a shared understanding about whether it is succeeding.

  1. The court must be clear about its goals for the ADR program
  2. Those goals must be shared with stakeholders

 

2. Principles

The ADR program must operate on a common set of foundational principles, including ethics as appropriate for the ADR process being provided.

  1. Confidentiality (in mediation)
  2. Fairness of process
  3. Fairness of outcomes
  4. Procedural justice
  5. Accessibility
  6. Neutral quality
  7. Self-determination (in mediation)
  8. Timeliness

 

3. Accessibility

Everyone – including litigants and lawyers – must be able to readily access the ADR program.

  1. Parties who are unable to pay for ADR are afforded the opportunity to use ADR
  2. Parties who are representing themselves are able to participate fully in ADR
  3. Parties with disabilities are able to participate fully in ADR

 

4. Process Quality

Notwithstanding the importance of other characteristics, the true quality of a court ADR program boils down to what happens during each ADR session.

  1. Whichever ADR process is being used, it adheres to the foundational principles of that process
  2. Participants have an experience of procedural justice when engaging in the ADR process:
    1. They feel they had a voice in the process (e.g., had a chance to talk, felt they were heard)
    2. They feel they were respected in the process
    3. They feel the process was fair to them

 

5. Program Support

The ADR program will only succeed if it receives steady support from a number of sources.

1. Stakeholders understand and support the ADR program

  • Judges
  • Litigants
  • Court staff
  • Neutrals

2. The program has sufficient, stable financial support

 

6. Neutrals

Neutrals are the face of the program to litigants and lawyers, so they must provide quality services.

  1. Neutrals share a common understanding of the service they are to provide
  2. Neutrals provide services in the manner expected by the program and its stakeholders
  3. Neutrals operate in an ethical manner
  4. Neutrals have the necessary skills and knowledge
  5. Neutrals are selected for the roster fairly
  6. Neutrals are appointed to cases in a fair manner
  7. Neutrals receive appropriate initial and ongoing training
  8. Neutrals provide a sufficiently uniform version of the ADR process
  9. Parties have a way to lodge complaints about neutrals
  10. Neutrals are treated fairly when a complaint against them is made
  11. Neutrals are compensated fairly (which does not preclude volunteer mediators)
  12. Neutrals are assessed fairly
    1. Participant surveys
    2. Peer review

 

7. Lawyers

Lawyers must support, or at the very least accept, the ADR program.

  1. Lawyers are often repeat users of the ADR program, therefore their support is especially important
  2. This applies equally to those who typically represent one side or the other, e.g., landlords’ lawyers and tenants’ lawyers.
  3. Lawyers should:
    1. Find the program useful
    2. Be educated about the program
    3. Fulfill their responsibilities in the program

 

8. Program Safety

ADR processes must be safe, both literally and figuratively.

  1. Participants, neutrals and staff are all safe when participating in ADR and otherwise interacting with the ADR program
  2. Participants are screened prior to ADR, when appropriate, to identify intimate partner violence and other potential barriers to participation

 

9. Data Collection and Dissemination

Reliable data must be collected regularly and shared appropriately, or it won’t be available when it is time to sustain or improve the ADR program.

  1. Data is collected regularly and includes both quantitative and qualitative information (e.g., statistics and success stories)
  2. Changes in the program are tracked and acted on, e.g., changes in:
    1. The number of cases being sent to ADR
    2. The number of agreements being reached
    3. The number of mediators signing up to mediate
    4. The number of parties showing up for ADR sessions
  3. Data is turned into reports that can be readily digested
    1. Easy to read
    2. No use of insider language or acronyms
    3. Fitting use of charts and other visuals
  4. Reports are adapted to their particular audience and disseminated appropriately, typically as follows:
    1. Judges and court administrators with direct responsibility get the most detailed reports
    2. Those higher up in the court system get summaries with explanations
    3. Funders’ reports depend on what the funder requires. They may also get some anecdotes about how parties are experiencing the program.
  5. Reports include recommendations for action when appropriate
  6. The program is evaluated near the end of the first year, after a substantial period of time, or when there has been a major change in the program or the context in which the program operates

 

10. Program Promotion

Reminding stakeholders of the value of the ADR program will help maintain its support.

  1. Brochures are made available in courtrooms and other locations as needed
    1. Brochures are only produced if they will serve a particular purpose, such as giving parties the number they need to call to schedule a mediation
    2. If brochures are targeted to self-represented litigants, they must be in easy-to-understand language
  2. News about the program is publicized via:
    1. Local radio, TV, newspaper, news websites
    2. Court newsletter
    3. Bar association newsletter
  3. Program is featured on appropriate websites
    1. Information about the ADR program can be readily found on the court’s site
    2. If the program maintains its own site, instead of appearing as part of the court’s site, it must be easy to find and navigate and kept up to date
  4. As members of stakeholder groups change, the new individuals are familiarized with the program
    1. Judges are educated on how it works, ethical limitations, what to expect
    2. Lawyers, e.g., child protection attorneys, assigned to courtrooms are trained in how to use the program
    3. Staff at funders are educated about the program’s benefits and history
    4. Court staff are educated about the program’s benefits and how it fits into court processes

 

11. Program Administration

Someone wakes up every morning with the feeling that it is their job to do everything on this list to ensure the quality and continuity of the ADR program.

  1. Individuals with power over the program, e.g., funding or case referrals, are kept informed about the program
  2. The ADR program operates in accordance with applicable laws, court procedures and rules
  3. The program functions efficiently and effectively, e.g.:
    1. Cases are referred to ADR in a consistent manner
    2. Cases are scheduled promptly
    3. Reports are provided to court on time
    4. The benefits provided by the ADR program are reasonable in relationship to the costs of the program
    5. Changes in the program are tracked and potential responses suggested when needed

 

New Mediator Self-Reflection Tool

Susan M. Yates, January 9th, 2019

The Supreme Court of Virginia has developed a wonderful new self-reflection form for mediators. While the Court developed this tool for their certified mediators as part of their re-certification process, it is a valuable tool for any mediator (just ignore the instructions about continuing mediator education credits). There is a lot of content, so if you are using this on your own you will probably want to pick and choose among the questions. This new tool coordinates with Virginia’s excellent Mediator Self-Reflection Treasury.

Even though mediators work very closely with people when we mediate, typically no one else in the room shares our mediator perspective. There are exceptions, such as co-mediation or when we are observed by new mediators, but mediation can be an isolated activity (made especially so by the limits of confidentiality). This isolation makes self-reflection particularly important.

I can imagine many uses for these tools beyond self-reflection. A group of mediators could pick a few of the questions to discuss over lunch. For co-mediators, the tools could aid their debriefing. The forms might help a new court or community mediation program get clear about what they expect from mediators. The tools will probably spark other ideas when you read them.

Many thanks to the good people of the Supreme Court of Virginia for taking the time to produce and share these tools. They are a real gift to the mediation community.

Findings from an Evaluation of Eight Foreclosure Mediation Programs

Jennifer Shack, December 5th, 2018

As I mentioned last month, I recently completed a comprehensive evaluation of eight foreclosure mediation programs in Illinois. One great benefit of evaluating eight programs with different approaches to resolving the same cases is that it allowed me to uncover program design factors and other variables that promote program success. The three big takeaways from the evaluation are that proper program design is essential, provision of services has an impact on homeowner outcomes, and data is crucial to program improvement. The evaluation was a final look at the eight programs that were funded by the Illinois Attorney General, encompassing up to four years for each program. Seven of the eight programs used relatively uniform data that was collected on the same online case management system. Further, I worked with each program to define the variables used so that we had a clear understanding of the meaning of each variable. This allowed me to develop uniform measures for the programs that enabled comparisons of program performance across them.

First, some basic findings. The eight programs helped 4,766 homeowners, representing 23% of all foreclosure filings in their jurisdictions. They saved 1,100 homes. Once homeowners entered the programs, 21% to 40% saved their homes, depending on the program. More than 90% of homeowners who completed surveys said that they gained a better understanding of their options and how to work with their lenders. Almost all homeowners felt that they were treated fairly and with respect. Most felt that they were able to talk about the issues and concerns that were most important to them and almost all felt the mediator understood what was important to them. Most were satisfied with their experience.

Now to the takeaways. Program design played a significant role in how many homeowners a program was able to help and how many homeowners participated in the program. The two variables are different because most programs helped homeowners to understand their options and the foreclosure process, even if they could not or decided not to participate. Those programs that told homeowners that they must appear for their initial session and provided a date and time for that had significantly higher proportions of homeowners appear and participate than programs that had them contact the program in other ways. And those programs that told homeowners they had to call the program coordinator, provided a deadline to do so and sent additional reminders had significantly higher proportions of homeowners contact the program and participate than those programs that informed the homeowners of the program and told them how to start the process to participate.

Participation rate is very important, not just because higher participation means that more homeowners are helped. The greater the proportion of homeowners facing foreclosure who participate in the program, the greater the proportion of homeowners who save their homes.

Other aspects matter as well. Having the homeowners meet with a representative for their lender from the outset appears to improve program completion rates and possibly improves the probability that participating homeowners save their homes. Within individual programs, those homeowners who worked with a housing counselor are more likely to complete the program. Those who worked with attorneys were much more likely to complete the program. Interestingly, they weren’t more likely to save their homes.

It was very gratifying to see that those programs that made changes based on the data they were collecting and the recommendations from my first evaluation were improved by those changes. For example, the 19th Circuit and 20th Circuit programs made changes to the manner in which homeowners contacted and entered the program, significantly improving participation. The 16thand 19th Circuits worked with mediators to improve their skills, leading to fewer mediator issues and more participants leaving mediation with a good experience.

For a quick take on the evaluation, see the Executive Summary.
To access a digital summary of the evaluation, click here.
For the Full Evaluation, download PDF .

Verified by ExactMetrics