Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Posts Tagged ‘evaluation’

Survey Comments Highlight What Tenants, Landlords Think About Mediation Program

Jasmine Henry, September 16th, 2024

Since 2020, Resolution Systems Institute has administered the Kane County (Illinois) Eviction Mediation Program. Our team works with the 16th Judicial Circuit Court and community services to mitigate the negative effects of eviction filings. We conduct intake, schedule mediations, provide guidance for parties going through eviction mediation, manage the mediator roster, and provide assistance to the mediators as needed. After parties participate in mediation, we ask them to complete a survey about their experience, including answering multiple open-response questions.

Landlord and tenant comments on the program and their mediations illuminate what they value and what falls short for them. We have seen how the process makes parties feel respected and listened to, and what that means for them. We have also learned of issues that arise with power imbalances and with mediators who repeatedly facilitate cases involving the same attorney.

Mediation as a ‘Life Preserver’

Comments from the past year make clear that many parties have been profoundly affected by their experience with our program and their mediation. This is perhaps best exemplified by this comment from a tenant: “Everyone has their own reasons for difficulties. I got a chance to say how this happened and also how I can prevent it from happening in the future. I’m thankful for everyone who has helped me through this and being treated fairly and with respect. This program is a life preserver to a drowning person. Thank you.”

Another tenant shared a similar sentiment, saying, “When people go through a difficult time, this program gives them a chance.”

Mediator Role is Key

As with so many mediation programs, mediators are central to parties’ positive experience in the Kane County program, survey responses show. In fact, the presence of a knowledgeable, impartial mediator is one of the most appreciated aspects of the program. One participant noted that the mediator “listened to both sides of the story.” Another commented: “I really appreciated . . . the way the mediator helped to point out the facts of the situation and allow both parties to express themselves. As a tenant without a lawyer who’s been through a major financial crisis, it gave me the opportunity to be listened to and to get my head above water.”

Many survey respondents praised their mediators’ ability to help parties find compromise. As one tenant put it, “They really try to meet in the middle and make everyone happy in the end. It gives so much clarity and security.” Another tenant succinctly stated, “It gave a compromise to a no win situation.”

Participants noted that both the skills of the mediators and the structured nature of the mediation process ensure that all parties have a chance to contribute to the conversation. A tenant shared: “Both sides were given the opportunity to speak uninterrupted. Everything was explained clearly and the mediator was very pleasant to work with. I came in feeling anxious not knowing what to expect but was pleasantly surprised by the whole experience and outcome.” A landlord shared that it was “good to talk to the other party in the presence of some responsible people.”

“The meeting was not at all stressful once the call was started. Each party received ample time to discuss any offer, explanations of situation, possible resolutions to ensure both parties benefited.”

Tenants in particular mentioned the importance of feeling listened to. When asked what they liked about the mediation, tenants described mediators who “listened to my doubts and questions,” were “helpful and caring and want to hear what you have to say,” or gave all parties “the opportunity to discuss their thoughts and concerns.” A landlord added that their mediator “listened and asked good questions.”

Additionally, mediators were often commended for their patience and thoroughness, as exemplified by this comment: “The [mediator] was very respectful and kind and listened to all my doubts and questions. The [mediator] helped a lot and explained everything in very good detail.” Another participant described their experience this way: “The meeting was not at all stressful once the call was started. Each party received ample time to discuss any offer, explanations of situation, possible resolutions to ensure both parties benefited.”

These comments underscore the program’s success in creating an environment where both tenants and landlords feel they can speak openly, are listened to and are treated equitably.

Resources for Vulnerable Individuals

Several survey responses highlighted the program’s value for vulnerable individuals. For example, a number of comments mentioned the importance of the resources beyond mediation that the program can connect parties with. One tenant wrote that the program “was there to help me [with court paperwork] when they really didn’t have to.” Multiple commenters emphasized that they had applied to the court-based rental assistance program — which the mediation program directed many parties to, and which parties were able to access until funds ran out in June 2024. When asked if they would recommend eviction mediation to a friend, a tenant shared that they would (even though they didn’t settle their case in mediation) because “sometimes tenants are unaware of the resources available due to lack of communication or shame.”

Others expressed more general appreciation for being able to participate in the program as someone in a challenging situation: “Just every single mother that actually works and tries 100% for her and children deserves this opportunity.” Another shared: “I think it is an excellent program for families who are going through difficulties, personally I felt supported and confident in the ability of this incredible program to help.”

Perceptions of Mediator as Biased

Some survey respondents indicated that their mediators worked to minimize the usual power imbalance between tenants and landlords. For example, one commented: “My mediator was a rockstar ! The attorney tried to bully us and was interrupting them but they kept their calm and brought up my rights.”

However, some tenants expressed concerns about potential bias in the mediation process, with one tenant stating, “They are there to mostly help the landlord … It doesn’t help the tenant. At all … The mediator is on the landlord side to help them evicted you.”

“I felt because the lawyer and the mediator were familiar with one another they may have been more partial to the landlord.”

Similarly, a tenant shared, “Not only was I not informed what mediation was going to be like, but also the mediator didn’t take everything I said and just went with what the landlord [said].” And one tenant went so far as to say, “I feel like I was gaslit by the Lawyer, and [the lawyer], as well as the mediator didn’t want to hear my side.”

Some comments about pro-landlord bias reflected a tenant’s understanding that the mediator already knew or had worked with the landlord’s attorney in the past, with one tenant writing: “I felt because the lawyer and the mediator were familiar with one another they may have been more partial to the landlord.”

Another explained: “[I]t’s like the mediator and the landlord and the landlord Lawyer have some kind of friendship already[,] so that kind of singles you out when they’re cracking jokes and laughing with each other[. A]nd then [the mediator is] only hearing [the landlord’s] side and telling you [that] you have to take your side to court.”

Other criticisms related to feeling unduly pressured to reach an agreement. One tenant explained how they felt both unheard and pressured: “I feel as if, regardless what was disclosed, my landlord and his attorney pushed too hard at a hard no to mediation, not giving me a chance, and the mediator seemed to take what I had to say about the situation with a grain of salt.” Another tenant stated, “My concerns were not fully addressed, felt bullied into settling.” One expressed their frustration, saying: “The mediator didn’t listen to my needs and disregarded any of my opinions[. The mediator] listened to every word the lawyer said and also wrote anything they wanted word for word.”

How RSI Uses Feedback

RSI uses party responses to our post-mediation surveys both to evaluate and improve our program, as well as to tell the full story of the eviction mediation process. These firsthand accounts allow us to go beyond statistics to better understand how eviction mediation programs affect real people in our communities. Furthermore, these comments provide invaluable evidence of the program’s tangible benefits. By thoughtfully analyzing and presenting these comments, we strengthen our partnerships, justify program funding and, ultimately, improve our ability to effectively serve the Kane County community.

RSI takes party criticisms of our mediators and program very seriously. Every quarter, we evaluate the participant post-mediation survey responses to determine what is or is not working. Every month, we hold meetings with our program staff to review our findings. We also use our research to train mediators on how to cultivate better party experiences. When necessary, we provide further training and support to mediators with recurring issues. Occasionally, if feedback was provided but no improvement made, we remove underperforming mediators from our roster.

Negative party perceptions, even if not reflective of actual bias, indicate a need to ensure and communicate impartiality more effectively. While it can be demoralizing to read party comments complaining that a mediator was biased, these comments help us to understand the impact that such perceptions have on party experience. Mediation is meant to be a level playing field for all parties — landlords and tenants, unrepresented parties and those with attorneys.

Most Give High Ratings for Mediator Fairness, Trust in Mediator in Recent Surveys of RSI’s Kane County Eviction Mediation Program

Jasmine Henry, January 10th, 2024

RSI administers an eviction mediation program in Kane County, Illinois. Every quarter, we provide a report to the court on the participants’ experience in mediation based on their responses to a post-mediation survey.

Between July 1, 2023, and September 30, 2023, 174 eviction mediations were held in the 16th Judicial Circuit of Illinois (Kane County). After every mediation, participants were invited via email or text to complete an online survey about their experience; not all of the participants completed surveys. In our latest survey report, we examined participant responses from those three months. Specifically, we focused on participant opinions regarding fairness, trust and satisfaction. In all, 21 tenants, one landlord and 11 attorneys responded. The participants responded to the questions according to a seven-point scale, which we consolidated into three categories: low (1–2), medium (3–5), and high (6–7). Participants were invited to add comments to some of their responses. Their responses are summarized below.

Trust in Mediator, Perceived Fairness

We asked respondents about their perception of the mediator. Specifically, we asked: “How fairly did the mediator treat you?” And, “How much did you trust the mediator?” Almost two-thirds of participants gave high ratings for mediator fairness and trust. However, respondents tended to rate mediator fairness higher than mediator trust. For example, fewer than 3% of respondents thought the mediator did not treat them fairly, while 15% of respondents had low trust in the mediator. There was a parallel, albeit smaller, difference observed in the positive ratings, with 63% of respondents rating the mediator as very fair, compared with 58% who had high trust in the mediator.

Turning more broadly to respondents’ perception of the mediation process as a whole, we asked: “Overall, how fair was the mediation process?” Most of the participants who responded felt that the mediation was fair overall, with 62% saying it was highly fair. Notably, this is very similar to the percentage of respondents who said the mediator was highly fair. Not all of the respondents were impressed with the process, and 10% of respondents rated the mediation a little fair or not at all fair.

Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.”

Comments of Tenants, Attorneys

We asked respondents to explain their overall fairness ratings. The landlord did not comment, but many tenants and some attorneys did. Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.” A quarter of the tenant comments mentioned court-based rental assistance, which tenants were often referred to by the program. Several tenants also saw the mediators as helping, saying, “They stood up for me … They didn’t let [the landlord] push me,” and “[We asked] for what we wanted and [the mediator] basically fought for us to get it.”

In contrast, tenants who gave medium and low ratings on overall fairness tended to focus their frustrated comments on the mediator’s relationship with the landlord. One tenant said the mediator “may have been more partial to the landlord” because they “were familiar with one another”; another tenant said plainly that “they are there to mostly help the landlord.” One tenant felt frustrated that the mediator did not seem to believe what the tenant said at mediation, saying, “The mediator seemed to take what I had to say about the situation with a grain of salt.” Attorneys who rated the overall fairness at a medium or low level focused on efficiency, with one saying, “I was disappointed that the mediator allowed the opposing side to spend valuable time on issues irrelevant to the case.”

Likelihood to Recommend Eviction Mediation

To further explore participant satisfaction, we asked participants: “If a friend or colleague had a dispute like yours, how likely are you to recommend eviction mediation?” Most of the participants who responded were likely to recommend mediation to a friend or colleague, with 67% saying they were highly likely to recommend it. One tenant commented, “I would recommend all mediation options; sometimes tenants are unaware of the resources available due to lack of communication or shame.” However, another tenant who was less satisfied with the process commented, “It doesn’t help the tenant. At all. It helps landlords.”

As was the case with the first question on participant satisfaction, the landlord did not comment on their responses to this question, but we did receive two attorney comments. One attorney who was highly satisfied with the mediation process commented, “We made the exact same settlement offer that was accepted at mediation to the landlord’s attorney months ago, and they never responded in any way despite multiple phone calls. I assume this was on their client’s instructions. Because of the mediation process, I believe they would have continued stonewalling us.” The attorney who was unlikely to recommend mediation to a colleague said: “The lengthy mediation process is not helpful in my view. Before this system was implemented, and still now (in other counties), I am often able to reach agreements with the tenants within 5–10 minutes in the hallway outside the Courtroom. There is no need for the mediator, in my opinion.”

Conclusion

In conclusion, the survey responses indicate that the program continues to provide a positive experience to most participants. Those who completed the survey generally had positive perceptions of the mediators and the program, with the majority giving high ratings on fairness, trust and satisfaction. However, some participants’ comments point to a perception among tenants that mediators are biased toward the other side and a perception among attorneys that the mediation process is not efficient.

Spring 2022: How RSI’s Work Has Expanded and Evolved During the COVID-19 Pandemic

Just Court ADR, May 18th, 2022

As the COVID-19 pandemic has strained finances, families and the social fabric in general, the need for and potential of well-designed alternative dispute resolution (ADR) to stitch together solutions has become increasingly apparent. Throughout this challenging time, Resolution Systems Institute (RSI) has continued to monitor and evaluate new dispute resolution initiatives and developed new court mediation programs of our own. Below is a description of some of our work during the pandemic.

In 2020, as a potential pandemic-driven eviction wave loomed, RSI began to design our first eviction mediation program. In the course of about nine months, we developed the court rules, procedures and forms the program would need to operate. We also recruited a cadre of mediators and provided them with specialized eviction mediation training through our friends at the Center for Conflict Resolution. The Kane County (Illinois) Eviction Mediation Program launched in spring 2021. Later that fall, RSI launched similar programs in Illinois’ Kankakee and Winnebago counties. Overall, our programs operate in judicial districts that serve over 920,000 Illinois residents.

To learn more about RSI’s work over the last two years in the eviction arena, we recommend you check out our blog entries on the topic, as well as our Eviction Mediation Special Topics resource. RSI is now working with Kane and Winnebago counties to bolster their longstanding foreclosure mediation programs as homeowners now begin to feel the squeeze that renters started experiencing last year.

While RSI is spearheading these new program development and administration initiatives, evaluation remains the central pillar of upholding RSI’s mission of improving access to justice. RSI Director of Research Jennifer Shack recently published two reports evaluating the program development experience in Kane County: Eviction Mediation Design and Implementation in Illinois’ 16th Judicial Circuit: Challenges and Keys to Success and Participant Experience in Eviction Mediation: Summary Of Early Survey Responses in the 16th Judicial Circuit of Illinois’ Video Mediation Program.

In partnership with the University of California, Davis, RSI has also evaluated online dispute resolution (ODR) pilot programs in Texas and Michigan. The pandemic generated greater interest among courts for ADR processes that parties could access remotely. The evaluations, which will be released soon, will provide courts with a better understanding of what ODR adoption requires and what possible benefits it can provide.

The last two years have provided many of us, RSI included, with a complex mix of setbacks and new opportunities. RSI is committed to innovating and adapting to meet the challenges that courts, and the litigants they serve, encounter in this ever-changing world. We are grateful to our program partners, our funders and each of you who come to RSI in search of expertise and guidance. We hope you will continue to take this journey with us as we work towards our mission of expanding access to justice through court alternative dispute resolution.

Characteristics of Quality Court ADR Programs

Susan M. Yates, September 18th, 2019

What characteristics do you think are shared by quality court ADR programs? I took a swing at a list here. What would you add? Change?

1. Goals

There must be a shared understanding of the goals of the ADR program so that there can also be a shared understanding about whether it is succeeding.

  1. The court must be clear about its goals for the ADR program
  2. Those goals must be shared with stakeholders

 

2. Principles

The ADR program must operate on a common set of foundational principles, including ethics as appropriate for the ADR process being provided.

  1. Confidentiality (in mediation)
  2. Fairness of process
  3. Fairness of outcomes
  4. Procedural justice
  5. Accessibility
  6. Neutral quality
  7. Self-determination (in mediation)
  8. Timeliness

 

3. Accessibility

Everyone – including litigants and lawyers – must be able to readily access the ADR program.

  1. Parties who are unable to pay for ADR are afforded the opportunity to use ADR
  2. Parties who are representing themselves are able to participate fully in ADR
  3. Parties with disabilities are able to participate fully in ADR

 

4. Process Quality

Notwithstanding the importance of other characteristics, the true quality of a court ADR program boils down to what happens during each ADR session.

  1. Whichever ADR process is being used, it adheres to the foundational principles of that process
  2. Participants have an experience of procedural justice when engaging in the ADR process:
    1. They feel they had a voice in the process (e.g., had a chance to talk, felt they were heard)
    2. They feel they were respected in the process
    3. They feel the process was fair to them

 

5. Program Support

The ADR program will only succeed if it receives steady support from a number of sources.

1. Stakeholders understand and support the ADR program

  • Judges
  • Litigants
  • Court staff
  • Neutrals

2. The program has sufficient, stable financial support

 

6. Neutrals

Neutrals are the face of the program to litigants and lawyers, so they must provide quality services.

  1. Neutrals share a common understanding of the service they are to provide
  2. Neutrals provide services in the manner expected by the program and its stakeholders
  3. Neutrals operate in an ethical manner
  4. Neutrals have the necessary skills and knowledge
  5. Neutrals are selected for the roster fairly
  6. Neutrals are appointed to cases in a fair manner
  7. Neutrals receive appropriate initial and ongoing training
  8. Neutrals provide a sufficiently uniform version of the ADR process
  9. Parties have a way to lodge complaints about neutrals
  10. Neutrals are treated fairly when a complaint against them is made
  11. Neutrals are compensated fairly (which does not preclude volunteer mediators)
  12. Neutrals are assessed fairly
    1. Participant surveys
    2. Peer review

 

7. Lawyers

Lawyers must support, or at the very least accept, the ADR program.

  1. Lawyers are often repeat users of the ADR program, therefore their support is especially important
  2. This applies equally to those who typically represent one side or the other, e.g., landlords’ lawyers and tenants’ lawyers.
  3. Lawyers should:
    1. Find the program useful
    2. Be educated about the program
    3. Fulfill their responsibilities in the program

 

8. Program Safety

ADR processes must be safe, both literally and figuratively.

  1. Participants, neutrals and staff are all safe when participating in ADR and otherwise interacting with the ADR program
  2. Participants are screened prior to ADR, when appropriate, to identify intimate partner violence and other potential barriers to participation

 

9. Data Collection and Dissemination

Reliable data must be collected regularly and shared appropriately, or it won’t be available when it is time to sustain or improve the ADR program.

  1. Data is collected regularly and includes both quantitative and qualitative information (e.g., statistics and success stories)
  2. Changes in the program are tracked and acted on, e.g., changes in:
    1. The number of cases being sent to ADR
    2. The number of agreements being reached
    3. The number of mediators signing up to mediate
    4. The number of parties showing up for ADR sessions
  3. Data is turned into reports that can be readily digested
    1. Easy to read
    2. No use of insider language or acronyms
    3. Fitting use of charts and other visuals
  4. Reports are adapted to their particular audience and disseminated appropriately, typically as follows:
    1. Judges and court administrators with direct responsibility get the most detailed reports
    2. Those higher up in the court system get summaries with explanations
    3. Funders’ reports depend on what the funder requires. They may also get some anecdotes about how parties are experiencing the program.
  5. Reports include recommendations for action when appropriate
  6. The program is evaluated near the end of the first year, after a substantial period of time, or when there has been a major change in the program or the context in which the program operates

 

10. Program Promotion

Reminding stakeholders of the value of the ADR program will help maintain its support.

  1. Brochures are made available in courtrooms and other locations as needed
    1. Brochures are only produced if they will serve a particular purpose, such as giving parties the number they need to call to schedule a mediation
    2. If brochures are targeted to self-represented litigants, they must be in easy-to-understand language
  2. News about the program is publicized via:
    1. Local radio, TV, newspaper, news websites
    2. Court newsletter
    3. Bar association newsletter
  3. Program is featured on appropriate websites
    1. Information about the ADR program can be readily found on the court’s site
    2. If the program maintains its own site, instead of appearing as part of the court’s site, it must be easy to find and navigate and kept up to date
  4. As members of stakeholder groups change, the new individuals are familiarized with the program
    1. Judges are educated on how it works, ethical limitations, what to expect
    2. Lawyers, e.g., child protection attorneys, assigned to courtrooms are trained in how to use the program
    3. Staff at funders are educated about the program’s benefits and history
    4. Court staff are educated about the program’s benefits and how it fits into court processes

 

11. Program Administration

Someone wakes up every morning with the feeling that it is their job to do everything on this list to ensure the quality and continuity of the ADR program.

  1. Individuals with power over the program, e.g., funding or case referrals, are kept informed about the program
  2. The ADR program operates in accordance with applicable laws, court procedures and rules
  3. The program functions efficiently and effectively, e.g.:
    1. Cases are referred to ADR in a consistent manner
    2. Cases are scheduled promptly
    3. Reports are provided to court on time
    4. The benefits provided by the ADR program are reasonable in relationship to the costs of the program
    5. Changes in the program are tracked and potential responses suggested when needed

 

Verified by ExactMetrics